Data Pre- and Post-Processing
Preparing Geospatial Datasets for Further Analyses and Information Extraction
Both data and database management need to be established early on in a project. Data management is the term used to describe all aspects of how data will be “managed” in a project. Management of data is an important consideration since it facilitates the storage, organization, sharing, and retrieval of data using a database management system or DBMS (Heywood et al., 2002).Data often require corrections to be applied in advance of extracting information from the data. These corrections are particularly important when using remote-sensing data, but are also relevant when using geographic information system (GIS) and GPS datasets. If these corrections are not applied prior to processing the data, the outputs of the data analyses may be erroneous. This result could potentially have dire consequences if these outputs are used to inform and influence decision-makers.
Common Remote-Sensing Data Pre-Processing Operations
Before useful information can be extracted from remote-sensing data and analyses can be performed, it is important for the analyst to pre-process the data through image restoration and rectification. In this context, the saying “garbage in equals garbage out” is relevant. If an analyst fails to adequately pre-process an image, then any analysis and information derived may be erroneous. This could have profound implications if the data and information is to be used to inform the decision-making process (see Decision-Making Processes). Pre-processing operations may be divided into geometric corrections or radiometric corrections.1. Geometric Corrections
Geometric corrections involve ensuring that the pixels in a remote-sensing image are in their proper geographic (x, y) location. Geometric errors may be caused by the altitude, velocity, and motion of a sensor on a remote-sensing platform due to several factors such as the Earth’s curvature. The goal of geometric correction is to compensate for and mitigate any geometric distortions so that the imagery resembles the real world as closely as possible. Geometric errors may be divided into two categories: systematic or random.Systematic
Some geometric errors are systematic and thus predictable. Accordingly, these types of errors can be corrected by modelling the sensor and the motion of the platform in relation to the curvature and rotation of the Earth.Random
Conversely, some geometric errors are random and thus unpredictable. To correct for these types of errors, ground control points (GCPs) are used. A GCP may be defined as “a location on the surface of the Earth (e.g., a road intersection) that can be identified on the imagery and located accurately on a map” (Jensen, 2016, pg. 242). For example, bridges or highway intersections make good GCPs because they are features that are not likely to change location over time. GCPs are typically collected by the analyst through field work using a GPS. The GCPs are then inputted into remote-sensing software which can calculate the mathematical relationship between the correct real-world map coordinates and the distorted image coordinates. Based on this relationship, the remote-sensing software can then determine the actual real-world location of each pixel in the image. In most instances, the more GCPs collected and used in the geometric correction process, the more the geometric distortion is mitigated, which ultimately produces a more accurate image that best resembles the “real world”.2. Radiometric Corrections
Sometimes the reflectance/emitted values recorded by the sensor can be impacted by sensor malfunctions or the environment (e.g., clouds, haze, atmospheric scattering), and thus result in radiometric errors. Radiometric corrections aim to correct image data so that the image accurately represents the amount of radiation reflected/emitted by an Earth surface feature.Radiometric errors result in image “noise”.
Some common radiometric errors caused by sensor malfunctions include:
- Random bad pixels (also known as shot noise)
- Random noise problems in an image scene. The distribution of the noise throughout the image is sporadic and appears as if the image has been shot with a shot-gun, hence the term “shot noise”.
- Line or column drop outs
- A portion of a line or column within an image scene that has no data.
- Line or column striping
- Image noise appears as a strip or band in the image scene.
Correcting Atmospheric Errors
- Do nothing. This depends on the application of the image data. In some instances, modelling the atmosphere is not necessary.
- Take in-situ (i.e., in the field) measurements
- Model the atmosphere. The remote-sensing software can model the atmosphere based on the day and time the image was acquired and the latitude and longitude of the image location.
Correcting Sensor Malfunctions
- For random bad pixels
- Involves taking the average value of the pixels surrounding the bad pixel.
- For line or column drop outs
- Take the average pixel values of the lines above and below the drop out. For column drop outs, take the average pixel values of the column to the left and right of the drop out.
Common DGPS Data Pre-Processing and Unmanned Aerial Vehicle (UAV)-Acquired Data Post-Processing
Images acquired from a UAV can be post-processed using software such as Pix4D and Agisoft to generate corrected image data. Images typically overlap, usually by about 70%, to maximize the number of individual features (e.g., trees, buildings) that are captured in multiple images. The location of the UAV is tracked using an internal GPS. The position of the UAV in space and the angles of rays extending from the UAV to common features are used to triangulate the position of the features that are common among images. An individual 100 m2 plot can have hundreds of thousands of detected common features. These detected positions (x, y, z coordinates) become the point cloud and can be used to produce a digital model of the Earth’s surface.