Accuracy assessment of different photogrammetric software for processing data from low-cost UAV platforms in forest conditions

Abstract

To obtain precise cartometric measurements of forests is always a challenge and high-resolution data from Unmanned Aerial Vehicle (UAV) is currently the quickest method. Generation of a fine quality orthomosaic of the acquired image series is a pre-requisite for full exploitation of such data. This study examines six of the most frequently used photogrammetric software for popular and inexpensive UAV systems. It is assumed that ground control points (GCPs) are not required. The average Root Mean Square Error (RMSE) for raw orthophoto was 1.24 m and around 0.2 m precision for both X and Y axes. Additionally, the accuracy of UAV internal GNSS receiver was checked on reference points which slightly exceeds 2 m RMSE. The range of accuracy and precision of orthomosaic are provided as a valuable reference for the use of low-cost UAV in forest inventory.

Introduction

Environmental changes are very dynamic and actions and measures for its protection and reconstruction are urgently needed ([46]) as expressed in the 2015 Paris Agreement. Monitoring of changes connected to the shrinking of forested areas is best implemented by remote sensing approaches ([15]). At present, spatial data obtained not only from satellites but also with mobile technologies such as a smartphone or UAV are mostly open-source and freely accessible on the internet ([36]). Some of the most popular applications of such spatial data are precision agriculture and forestry ([20]). Employment of non-metric cameras and inexpensive unmanned platforms has become the key to quickly acquiring high-resolution data from small areas covered by vegetation ([14]). The possibility of UAV in characterizing individual tree variables ([19]), forest structure ([27]), biophysical attributes ([28]), tree stump detection ([30]) and real-time monitoring has dramatically increased its application in forest inventory. In addition, UAV 3D topographic representation is an excellent alternative to costly airborne laser scanning (ALS - [10]) or even terrestrial laser scanning (TLS - [18]). Compared to ALS technology, the weakness of photogrammetric data is the lacking of precise digital terrain models (DTMs - [3]). The data cannot capture ground structure under tree crowns but it has been successfully exploited for canopy cover documentation. The imaging data enables a canopy height model (CHM) which can be used for the estimation of biomass, changes in canopy heights or tree counts, and crown detection ([17], [25]). The relatively low costs of these ready-made solutions, together with high data quality ([2]), operability in restricted places for take-off and landing, and practical independence from clouds’ impacts ([35]) due to the low level of flight, are typical advantages among many unmanned platforms. On the down sides, there are limiting factors like limited battery life, the relatively small area coverage for processing at one time, the loss of the line-of-sight leading to a weakening of data transmission ([38]), and radiometric quality of orthomosaic easily influenced by changing weather conditions ([47]). Finally, there is a problem with storing large data collections, as often very high image overlapping ratio is needed for accurate mosaicking ([40]). Advanced photogrammetric software allows quick transformation of data from the use of drones. The cloud storage accessible in the field allows the generation of various products: an orthomosaic DTM or the state of health assessment of vegetation using moisture content. The accuracy of end products depends on the quality of the Global Navigation Satellite Systems (GNSS) receiver which is the most important positioning tool with the inertial measurements unit (IMU). Depending on the platform, inexpensive UAV solutions can achieve accuracy of positioning at about 2 meters with a single frequency GNSS receiver ([11]). One of the methods to improve the accuracy of image location and orientation is direct georeferencing technique ([41]). Higher flying height allows to increase orthomosaic horizontal accuracy from 5 to 0.6 meter ([9], [29], [42]) and about 2 meter for digital surface model (DSM - [32]). However, in some cases an access to raw navigational data is needed for attaining such accuracies. Further improvement is possible through the use of GCPs or application of real time kinematic (RTK) technology. In both cases, the costs of processing grow significantly. The application of a GNSS surveying-class receiver in a forest environment for the sake of GCP is difficult because of the limited access to navigation signals ([6]). The installation of a GNSS receiver with the RTK technology is several times more expensive than the platform itself, which makes this solution a less preferred one. Most related studies involve costly platforms built for specific applications ([4], [22], [34], [1]). It is therefore important to investigate the most cost-effective approach to obtain a readymade orthomosaics based on an inexpensive navigation system with robust computational algorithm. However, such research is lacking and few reports provide the necessary knowledge across different approaches for drone-based orthomosaicking.

In this work, we investigate the possibility of an inexpensive model of an unmanned platform which is widely accessible and runs without the need for extensions or intervention. The consumer grade UAV segment directly leads to the necessity of implementing cheaper navigation units ([39]). However, the accuracy of GNSS receivers is low in this case, because they do not incorporate the expensive RTK technology. Improvements can be made through orthomosaics and 3D models, with the application of complex alignment algorithms ([44]). Precise UAV mapping using high quality orthomosaics are a basis for forest applications such as basic map updating ([31]). We report the accuracy of a GNSS receiver of an inexpensive and commonly accessible unmanned aerial platform. We also discuss the impact of such accuracy on the orthomosaic based on photographic material generated by six software models including cloud-based processing and standalone PC based processing.

The study area included a forest environment, which constituted an additional challenge for constructing a finished orthomosaic. To our knowledge, there are still very few reports which compare different photogrammetric software for UAV based mapping in a forested environment. As UAV deployments for forest inventory is getting more popular every day, this report will be valuable for researchers who are going to use the technology as a standard tool for data capture.

Materials and methods

Study area and equipment

An unmanned aerial platform Phantom^® 4 (SZ DJI Technology Co., Shenzhen, Guangdong, China) was used throughout the entire study. It is an inexpensive platform (1100 US$ in the production year 2016) that weighs less than 1.5 kg. The drone can easily fit in a medium-size backpack and deployed in small openings of a wooded area. The timespan of its flight in favorable weather conditions is about 20-25 minutes. The platform is equipped with GNSS/IMU navigational systems which are responsible for flight stabilization and assigning the coordinates to the images centers. Because it is typical consumer grade UAV model, there is no standard option to extract raw navigation data for further processing. All data are stored in Exchangeable Image File Format (EXIF) assigned to the header of each image separately. The camera has a sensor of 1/2.3″ with 12.4 effective megapixel and 20 mm f/2.8 lens. It is mounted on a gimbal which reduces any pitch, yaw and roll of the platform.

The experiments were carried out in three phases. In the first phase, the accuracy of internal UAV navigation receiver was checked based on 26 reference points. The second phase was realized by one flight above the city of Rogów (central Poland) and six orthomosaics were generated using six software. After accuracy assessment, the best software was used for the third phase where image acquisition of a real forested area was made at the Pokrzywnica forest (east-central Poland). Accuracy assessment was then performed on the orthomosaic of the forested area.

Accuracy analysis of the UAV navigation receiver was carried out on an open area at 52° 00′ 29.03″ N, 20° 00′ 37.65″ E (POLREF national grid reference system). The positioning error tolerance was set to below 0.01 m ([23]). To enhance precision positioning, the real-time kinematic RTK method was used with a HiPer SR^® GNSS surveying-class receiver (Topcon Positioning Systems, Inc., Tokyo, Japan) using additional 25 points.

The horizontal measurement precision was ±0.03 m according to ASG-EUPOS reference stations network ([8]). In total 26 points were checked for quality by comparing coordinates values of the UAV platform at ground level and 10 m above the ground. This is to avoid reflected signals from the ground.

Analysis of orthomosaics accuracy was carried out at the Forestry Experimental Station in Rogow (51° 49′ 20.85″ N, 19° 54′ 2.38″ E) located 100 km west of Warsaw, central Poland. The study area stretches across about 17 ha. This area was selected because reference materials can easily be acquired. There is a large diversity of tree species in the site. Because there are only low buildings (less than 20 meters), and practically no difference in terrain elevation, there are no obvious obstacles for the process of mosaicking.

The photogrammetric mission by UAV has permission from the Civil Aviation Authority and Polish Air Navigation Services Agency. The flyover was in late spring, May 2016. Images were acquired at a height of 115 meters and maximum velocity 5 m s^-1. In the course of about 15 minutes around 113 photos were taken with 80% overlap and sidelap. The ground sample distance (GSD) along the 3.8 kilometer route was 5 cm. The sky was 100% overcast and the wind was 3 m s^-1. The images obtained had good photographic sharpness, favourable colour tonality of the orthomosaic without shadow. For accuracy assessment, reference points with clearly distinguishable objects/ details have been collected. The reference points were distributed evenly across the whole area. A Topcon HiPer SR surveying-class navigational receiver was used. Altogether, 53 reference points were recorded in RTK mode with horizontal measurement precision at ±0.03 m. They were measured at 10 epochs averaging and real time differential correction was delivered by ASG-EUPOS network of reference stations. Processing of the acquired images was conducted using six different photogrammetric applications: AgiSoft PhotoScan™ (3499 USnode−lockedlicense),APS™(US" role="presentation"> node−lockedlicense),APS™(US node−lockedlicense),APS™(4500US one year license), Pix4Dmapper™ (3000 USoneyearlicense),andthecloudservicesDroneDeploy™(US" role="presentation"> oneyearlicense),andthecloudservicesDroneDeploy™(US oneyearlicense),andthecloudservicesDroneDeploy™(1000US one year license), Maps Made Easy™ (around 150 USperhectares),andPrecisionMapper™(US" role="presentation"> perhectares),andPrecisionMapper™(US per2000hectares),andPrecisionMapper™(3500US one year license).

Fig. 1 shows part of the orthomosaic results generated from all the tested methods, which do not visually show any significant differences. The first three methods were applied on a desktop computer. The speed of mosaicking can vary depending on the capacity of a given computer. For each tested type of photogrammetric software, it took about 7 h for creating a final orthomosaic (the PC was equipped with a fifth-generation Intel Core i5 2.7 GHz processor and 8GB RAM DDR3 1867 MHz). All software programs offer the functions of creating an orthomosaic, DSM/DTM model, 3D point cloud, texture models, capacity measurements, 2D measurements, maps of NDVI indicators, video animations from the 3D model, exporting of vector and raster data, and sharing of products through cloud storage. The solutions offered by the DroneDeploy, Maps Made Easy, and PrecisionMapper providers are based on a cloud service, which means that the user must send all acquired images and metafiles to the servers. After final accuracy test and precision analysis of the orthomosaics, one software was selected for further image processing of pure forest environment.

Fig. 1 - A subimage of the orthomosaics generated by different models: (A) DroneDeploy; (B) Pix4Dmapper; (C) AgiSoft PhotoScan; (D) APS; (E) PrecisionMapper; (F) Maps Made Easy.

Field verification in typical forest conditions was carried out in the Pokrzywnica forest in east-central Poland (52° 36′ 37″ N, 20° 58′ 23″ E), in which a total of 137 reference points were selected for accuracy assessment. The mean location error of each point was 0.11 m after traverse adjustment. Reference points were selected at the edge of the five openings across an area of about 15 hectares. Every point was marked on the ground and precisely recognized on orthomosaics. The UAV acquisition was performed at the height of 115 meters, with a velocity of 8.7 m s^-1, and a route of 6.5 km in length. In the course of about 15 minutes, 240 photographs were taken (with 85% overlap and sidelap). The original GSD was at 5 cm (Fig. 2). GCPs were not used for the generation of the mosaic. All measurements and finished orthomosaics were transformed from the WGS 84 ellipsoid system to the 2000 national reference (EPSG code 2178) grid system. Flight missions were realized by MapPilot^® for iOS (Drones Made Easy, San Diego, CA, USA) which guarantees optimal trajectory.

Fig. 2 - An orthomosaic of a fragment of forest area with gaps. A total of 137 reference points were marked for accuracy assessment.

Results

Accuracy of the unmanned platform’s GNSS receiver (first experiment phase) directly above the measurement point was ±2.07 m, ±1.87 m for the X axis and ±0.89 m for the Y axis (Tab. 1). Hovering at 10 meters above ground level (AGL), the RMSE for the unmanned platform was ±2.01 m, with 1.99 m for the X axis and 0.25 m for the Y axis.

Tab. 1 - Internal GNSS RMSE (XY) values (in meters) for 26 control points at 0 and 10 m above ground level (AGL).

Stats	UAV
	AGL 0 m	AGL 10 m
RMSE	2.071	2.007
RMSE (X)	1.871	1.991
RMSE (Y)	0.888	0.253
Min (X)	1.44	1.727
Max (X)	2.391	2.289
Min (Y)	0.456	0.034
Max (Y)	1.322	0.424

Tab. 1 shows that the maximum and minimum values are smaller in the case of 10 m above ground level, despite at 0 m AGL the horizon was unobscured. For the component of X axis, the minimum value was 2.98 m, and for the Y axis it was 0.00 m. The maximum values for the X axis and Y axis were 5.24 m and 0.18 m, respectively.

For the acquisition phase, a total of 113 photographs were taken, which occupied 592 MB of disk space (second experiment phase). For each tested software, it took about 7 hours for obtaining the final orthomosaic. The averaged RMSE error for all types of software was about 1.24 m. It should be noted that it is nearly 4 times higher for the X axis than the Y axis (Fig. 3).

Fig. 3 - RMSE of orthomosaic from six different software. The horizontal bars represented the minimum and maximum values (in meters) and the green solid dots represent the average values.

As reported by Willmott & Matsuura ([48]), the average error had slightly smaller values and was on average 1.18 m for the X axis and 0.43 m for the Y axis. The minimum values of RMSE error for all the tested software models were about 1 m and the maximum values about 1.55 m. Only the PrecisionMapper computational cloud diverges from the rest, with a minimum error of 0.17 m and a maximum exceeding 3 m. Elimination of the systematic error (Bias) allowed higher precision of the finished orthomosaics, significantly lowering the error of positioning of the GNSS receiver. Congruency of the ground elements represented on the orthomosaic was about 0.21 m for the X axis and 0.25 m for the Y axis (Tab. 2).

Tab. 2 - Accuracy (eqn. 1-3), mean average error (eqn. 4-5), and precision (eqn. 6-9) of orthomosaics obtained from six different photogrammetric software. Values for every from 53 points are in meters.

Software	Accuracy			Bias		Precision
	RMSE	RMSE(X)	RMSE(Y)	MAE(X)	MAE(Y)	SE(X)	SE(Y)
DroneDeploy	1.224	1.169	0.364	1.164	0.348	0.106	0.107
Pix4D	1.271	1.227	0.329	1.223	0.313	0.117	0.109
Agisoft	1.265	1.218	0.342	1.215	0.324	0.090	0.111
APS	1.212	1.173	0.304	1.145	0.254	0.259	0.212
PrecisionMapper	1.413	1.108	0.877	0.950	1.389	0.602	0.885
Maps Made Easy	1.247	1.187	0.385	1.185	0.373	0.066	0.094
All	1.272	1.180	0.434	1.147	0.500	0.207	0.253

Based on the results in Tab. 2, the DroneDeploy software was selected for orthomosaic creation using the 240 drone images (1.24 GB) acquired at the Pokrzywnica forest (third experiment phase). The RMSE error was 1.66 m and overall precision did not exceed 1 m (Tab. 3). The minimum RMSE was 0.44 m and the maximum 3.41 m (Fig. 4). Almost 80% of RMSE vector bearings was located in the northern part (Fig. 5). For the area error, the gaps located on the southern part of the orthomosaic had smaller errors. The average error of area estimation of the analyzed gaps slightly exceeded 1.3% (Tab. 3).

Tab. 3 - Point accuracy (eqn. 1-3), mean average error (eqn. 4-5), precision (eqn. 6-9) and area percentage error (eqn. 10) on the forest orthomosaic. The area errors are expressed in percentage and all other values are expressed in meters for every 137 reference points.

Gap number	Accuracy			Bias		Precision		Area error
	RMSE	RMSE(X)	RMSE(Y)	MAE(X)	MAE(Y)	SE(X)	SE(Y)
1	1.569	1.069	1.149	0.909	0.969	1.012	0.912	1.01
2	1.725	1.162	1.275	0.995	1.003	0.954	1.203	2.01
3	1.510	0.575	1.396	0.468	1.301	0.472	0.944	2.02
4	1.775	0.729	1.618	0.627	1.534	0.626	1.065	0.31
5	1.749	1.273	1.199	1.175	1.080	0.701	0.554	0.78
All	1.664	1.041	1.298	0.875	1.118	0.943	0.989	1.34

Fig. 4 - Histogram of RMSE errors for pure forest orthomosaic generated in DroneDeploy.

Fig. 5 - Bearings and RMSE values for pure forest orthomosaic generated in DroneDeploy.

Discussion

Accuracy assessment of the unmanned platform’s GNSS receiver clearly demonstrates that the navigation module (GNSS and IMU) is not of the best quality. Average RMSE errors are commensurate with a GIS-class receiver, or smartphones ([37]). In this work, only a typical code receiver was used, with an assumption that the positioning errors should be at the level of a few meters ([21]). It is an inexpensive solution that allows a relatively quick receiver initialization to achieve good readiness for flight. In order to achieve a high positioning accuracy, however, RTK technology should be used and in that case external reference stations providing data directly to the UAV will be needed. Otherwise, an additional postprocessing is required to achieve higher accuracy. Since the purpose of this study is to investigate low-cost drone-based acquisition for forest applications, high-accuracy positioning solutions have not been implemented as it increases its price several times, and the financial grounds for its application would no longer hold.

During the first experiment we observed that the navigational receiver installed on the UAV platform is sensitive to reflected signals. Positioning results will improve during flight when GNSS multipath phenomenon is minimum. However, it does not significantly improve the accuracy of the navigation module itself, which still stays at the level of ±2.0 m and the maximum values could exceed 5 m. The negative impact of the low GNSS accuracy can be ameliorated by using stereo-matching techniques.

The Structure for Motion algorithm is used to search for the common features in adjacent images ([13], [5]) in all analyzed software. As a result, errors in positioning caused by GNSS receiver have been almost eliminated in the final orthomosaic. As shown in Tab. 2, the RMSE for situational details drops down to about 1.2 m using all of the applied photogrammetric software. In this way, the errors of the GNSS receiver mounted in UAV (Tab. 1) are reduced by nearly 60%.

Tab. 1 shows that the errors are nearly three times higher in the north-south axis than the east-west one. This phenomenon has been partially explained in the studies of Dawidowicz & Krzan ([16]), and is related to the limited number of satellites available for geo-positioning in northern Poland, whereby the calculation of the final coordinates stored in EXIF are less accurate in the northern direction ([21]). Better results of accuracy (Tab. 2) compared to the accuracy of UAV located on the ground (Tab. 1) can be also explained by the influence of IMU which start operating after take-off. Elimination of the systematic error makes it possible to evaluate the relative congruity of a finished orthomosaic. Most of the tested photogrammetric software have similar error calculated from eqn. 6-9, at the level of 0.13 m for both X axis and Y axis. The results from PrecisionMapper computational cloud, however, deviate significantly from the average performance. This is related to large errors in the process of stereo-matching in areas where the trees are close to reference points.

For the most popular processing tools tested in this study, no significant differences were found. The time of processing of an orthomosaic was similar in all the studied cases and amounts to a few hours. For the software programs installed directly (AgiSoft, Pix4D, APS) on the personal computer, they allow more configuration options and thus better adjustment of the end product is realized. DroneDeploy, Maps Made Easy, and PrecisionMapper are cloud-based and have presented practical and efficient solutions. Cloud-based computation gives the possibility to process several separate areas simultaneously. It does not overload the computer and provides significantly faster processing of an orthomosaic than a program that process images on PC. The good results of DroneDeploy in terms of accuracy and precision, correct trees shape representation (Fig. 1A) and very easy user-interface, led to use it for further analysis in forested areas. The advantage of such a solution is the complete independence from computational capacity of the user’s computer. In comparison to desktop solutions, the user has only a limited range of configuration options and control over the final product.

For the Pokrzywnica forest, it is clear that image texture has a significant influence on orthomosaic precision. The Structure for Motion algorithm does not work so well without clear edges and similar pixel colors because with rather homogenous forest texture, fewer tie points are available to aid image matching ([24]). Lower quality of the orthomosaic can be shown by shape distortion of tree crowns and the reference points are mis-located. There is no significant pixel movement on the final forest orthomosaic but it can be observed that the largest distortions are oriented to north direction (Fig. 5) consistent with GNSS residuals (Tab. 2). The final precision of the orthomosaic is estimated at below 1 m (Tab. 3) and in terms of area percentage error, it is translated into around 1.3%. This result is much worse comparing to areas with clear edges in Rogów and is strictly connected with feature extraction procedure which is challenging for homogenous forest texture.

The main objective of the study is to investigate latest drone technology for obtaining image data in a cost-efficient and quick manner. It is understood that the use of GCPs based on RTK technology would practically eliminate the errors of the GNSS receiver. However, localization and measurement of GCPs in a forest environment is a very difficult task and potential errors can occur as a result of mistaken indication and low reference points accuracy. Visual analysis of the final orthomosaic shows that the impact of positioning error is small. Measurement results obtained from orthomosaic are significantly better than those obtained from GNSS receivers commonly used in the forest environment, whose errors are still difficult to estimate despite continuous development of navigation systems.

Conclusions

The speed of obtaining finished and precise orthomosaic with the use of inexpensive unmanned platforms constitutes an advantage over other technologies. The error in estimation of the forest boundary obtained in the study at a level of 1.66 m is acceptable from the perspective of forestry management. The study by Sálek et al. ([33]) indicate that estimation error of the actual forest boundary can be several tens of meters. Taking into account the high precision of the obtained photogrammetric material, it can be assumed that estimation of areas of clearings and gaps in managed forests that are larger than 200 m² ([26]) is entirely sufficient. The obtained results confirm the assertion that application of inexpensive UAV solutions is entirely sufficient as a source of data for forest inventory. Taking into account the archived results the following conclusions may be made:

1. although the accuracy of an autonomous GNSS receiver applied in inexpensive unmanned platforms is low, it can be partially reduced by photogrammetric software;

2. an orthomosaic obtained from a photogrammetric survey carried out by an inexpensive unmanned platform is of high precision on the clear texture objects; however, accuracy decreases to one meter in pure forest conditions.

3. the application of GCP lacks financial justification when obtaining spatial data for environmental analyses based on a single UAV flight mission;

4. the selection of the type of photogrammetric software should be based on the combination of its price, the computer’s loading during processing and the precision achieved by the completed orthomosaic;

5. in order to obtain correct height coordinates there is a need to use GCPs or existing precise DTM from ALS.

Acknowledgements

This research was funded by REMBIOFOR project titled “Remote sensing estimation of tree biomass and coal resource in forests” and co-financed by resources provided by the National Centre for Research and Development, as part of the program titled “Natural environment, agriculture and forestry” BIOSTRATEG, as stated in agreement no. BIOSTRATEG1/267755/4/NC BR/2015.

The authors would like to thank Prof. Janusz Walo for his advice in the preparation of this study. Also, special thanks to Mr. Lech Stanczak for his help in field work and conducting calculations.

Conflicts of Interest

The authors of this paper declare that they were not supported, paid or hired by any photogrammetric software company and there is no conflict of interest.

References

(1) Abdollahnejad A, Panagiotidis D, Surovy P (2018). Estimation and extrapolation of tree parameters using spectral correlation between UAV and Pléiades data. Forests 9: 85.

(2) Agüera-Vega F, Carvajal-Ramírez F, Martínez-Carricondo P (2017). Assessment of photogrammetric mapping accuracy based on variation ground control points number using unmanned aerial vehicle. Measurement 98: 221-227.

(3) Agüera-Vega F, Carvajal-Ramírez F, Martínez-Carricondo P, Sánchez-Hermosilla López J, Mesas-Carrascosa FJ, García-Ferrer A, Pérez-Porras FJ (2018). Reconstruction of extreme topography from UAV structure from motion photogrammetry. Measurement 121: 127-138.

(4) Aicardi I, Dabove P, Lingua A, Piras M (2017). Integration between TLS and UAV photogrammetry techniques for forestry applications. iForest - Biogeosciences and Forestry 10: 41-47.

(5) Alonzo M, Andersen H-E, Morton D, Cook B (2018). Quantifying boreal forest structure and composition using UAV structure from motion. Forests 9: 119.

(6) Becker R, Keefe R, Anderson N (2017). Use of real-time GNSS-RF data to characterize the swing movements of forestry equipment. Forests 8: 44.

(7) Bemis SP, Micklethwaite S, Turner D, James MR, Akciz S, Thiele ST, Bangash HA (2014). Ground-based and UAV-Based photogrammetry: a multi-scale, high-resolution mapping tool for structural geology and paleoseismology. Journal of Structural Geology 69: 163-178.

(8) Bosy J, Graszka W, Leonczyk M (2007). ASG-EUPOS - a multifunctional precise satellite positioning system in Poland. European Journal of Navigation 5: 2-6. Online | Gscholar

(9) Chiang K-W, Tsai M-L, Chu C-H (2012). The development of an UAV borne direct georeferenced photogrammetric platform for ground control point free applications. Sensors 12: 9161-9180.

(10) Clapuyt F, Vanacker V, Van Oost K (2016). Reproducibility of UAV-based earth topography reconstructions based on Structure-from-Motion algorithms. Geomorphology 260: 4-15.

(11) Clark A (2017). Small unmanned aerial systems comparative analysis for the application to coastal erosion monitoring. GeoResJ 13: 175-185.

(12) Colomina I, Molina P (2014). Unmanned aerial systems for photogrammetry and remote sensing: a review. ISPRS Journal of Photogrammetry and Remote Sensing 92: 79-97.

(13) Cook KL (2017). An evaluation of the effectiveness of low-cost UAVs and structure from motion for geomorphic change detection. Geomorphology 278: 195-208.

(14) Cruzan MB, Weinstein BG, Grasty MR, Kohrn BF, Hendrickson EC, Arredondo TM, Thompson PG (2016). Small unmanned aerial vehicles (micro-UAVs, drones) in plant ecology. Applications in Plant Sciences 4: 1600041.

(15) Dash JP, Watt MS, Pearse GD, Heaphy M, Dungey HS (2017). Assessing very high resolution UAV imagery for monitoring forest health during a simulated disease outbreak. ISPRS Journal of Photogrammetry and Remote Sensing 131: 1-14.

(16) Dawidowicz K, Krzan G (2014). Accuracy of single receiver static GNSS measurements under conditions of limited satellite availability. Survey Review 46: 278-287.

(17) Dempewolf J, Nagol J, Hein S, Thiel C, Zimmermann R (2017). Measurement of within-season tree height growth in a mixed forest stand using UAV imagery. Forests 8: 231.

(18) Gruszczynski W, Matwij W, Cwiakala P (2017). Comparison of low-altitude UAV photogrammetry with terrestrial laser scanning as data-source methods for terrain covered in low vegetation. ISPRS Journal of Photogrammetry and Remote Sensing 126: 168-179.

(19) Guerra-Hernández J, González-Ferreiro E, Monleón V, Faias S, Tomé M, Díaz-Varela R (2017). Use of multi-temporal UAV-derived imagery for estimating individual tree growth in Pinus pinea stands. Forests 8: 300.

(20) Hird J, Montaghi A, McDermid G, Kariyeva J, Moorman B, Nielsen S, McIntosh A (2017). Use of unmanned aerial vehicles for monitoring recovery of forest vegetation on petroleum well sites. Remote Sensing 9: 413.

(21) Hofmann-Wellenhof B, Lichtenegger H, Wasle E (2008). GNSS - global navigation satellite systems: GPS, GLONASS, Galileo, and more. Springer, Wien, New York, pp. 516. Gscholar

(22) James MR, Robson S, D’Oleire-Oltmanns S, Niethammer U (2017). Optimising UAV topographic surveys processed with structure-from-motion: ground control quality, quantity and bundle adjustment. Geomorphology 280: 51-66.

(23) Jaworski L, Swiatek A (2012). Comparison of coordinates of Polref - the first order Polish national network - obtained from campaigns in 1994/95 and 2008/2010. Artificial Satellites 47.

(24) Kedzierski M, Wierzbicki D (2015). Radiometric quality assessment of images acquired by UAVs in various lighting and weather conditions. Measurement 76: 156-169.

(25) Matese A, Di Gennaro SF, Berton A (2017). Assessment of a canopy height model (CHM) in a vineyard using UAV-based multispectral imaging. International Journal of Remote Sensing 38: 2150-2160.

(26) Muscolo A, Settineri G, Bagnato S, Mercurio R, Sidari M (2017). Use of canopy gap openings to restore coniferous stands in Mediterranean environment. iForest - Biogeosciences and Forestry 10: 322-327.

(27) Ota T, Ogawa M, Mizoue N, Fukumoto K, Yoshida S (2017). Forest structure estimation from a UAV-based photogrammetric point cloud in managed temperate coniferous forests. Forests 8: 343.

(28) Otero V, Van De Kerchove R, Satyanarayana B, Martínez-Espinosa C, Fisol MAB, Ibrahim MRB, Sulong I, Mohd-Lokman H, Lucas R, Dahdouh-Guebas F (2018). Managing mangrove forests from the sky: forest inventory using field data and Unmanned Aerial Vehicle (UAV) imagery in the Matang Mangrove Forest Reserve, peninsular Malaysia. Forest Ecology and Management 411: 35-45.

(29) Pfeifer N, Glira P, Briese C (2012). Direct georeferencing with on board navigation components of light weight UAV platforms. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XXXIX-B7: 487-492.

(30) Puliti S, Talbot B, Astrup R (2018). Tree-stump detection, segmentation, classification, and measurement using unmanned aerial vehicle (UAV) imagery. Forests 9: 102.

(31) Rusnák M, Sládek J, Kidová A, Lehotsky M (2018). Template for high-resolution river landscape mapping using UAV technology. Measurement 115: 139-151.

(32) Ruzgiene B, Berteška T, Gečyte S, Jakubauskiene E, Aksamitauskas VÄŒ (2015). The surface modelling based on UAV Photogrammetry and qualitative estimation. Measurement 73: 619-627.

(33) Sálek L, Zahradník D, Marušák R, Jerábková L, Merganič J (2013). Forest edges in managed riparian forests in the eastern part of the Czech Republic. Forest Ecology and Management 305: 1-10.

(34) Sankey T, Donager J, McVay J, Sankey JB (2017). UAV lidar and hyperspectral fusion for forest monitoring in the southwestern USA. Remote Sensing of Environment 195: 30-43.

(35) Schirrmann M, Hamdorf A, Giebel A, Gleiniger F, Pflanz M, Dammer K-H (2017). Regression kriging for improving crop height models fusing ultra-sonic sensing with UAV imagery. Remote Sensing 9: 665.

(36) Tian J, Wang L, Li X, Gong H, Shi C, Zhong R, Liu X (2017). Comparison of UAV and WorldView-2 imagery for mapping leaf area index of mangrove forest. International Journal of Applied Earth Observation and Geoinformation 61: 22-31.

(37) Tomaštík J, Tomaštík J, Salon S, Piroh R (2016). Horizontal accuracy and applicability of smartphone GNSS positioning in forests. Forestry 90: 187-198.

(38) Tomaštík J, Mokroš M, Salon S, Chudy F, Tunák D (2017). Accuracy of photogrammetric UAV-based point clouds under conditions of partially-open forest canopy. Forests 8: 151.

(39) Torresan C, Berton A, Carotenuto F, Di Gennaro SF, Gioli B, Matese A, Miglietta F, Vagnoli C, Zaldei A, Wallace L (2017). Forestry applications of UAVs in Europe: a review. International Journal of Remote Sensing 38: 2427-2447.

(40) Torres-Sánchez J, López-Granados F, Borra-Serrano I, Peña JM (2018). Assessing UAV-collected image overlap influence on computation time and digital surface model accuracy in olive orchards. Precision Agriculture 19: 115-133.

(41) Turner D, Lucieer A, Wallace L (2014). Direct georeferencing of ultrahigh-resolution UAV imagery. IEEE Transactions on Geoscience and Remote Sensing 52: 2738-2745.

(42) Turner D, Lucieer A, Watson C (2012). An automated technique for generating georectified mosaics from ultra-high resolution unmanned aerial vehicle (UAV) imagery, based on Structure from Motion (SfM) point clouds. Remote Sensing 4: 1392-1410.

(43) Uysal M, Toprak AS, Polat N (2015). DEM generation with UAV Photogrammetry and accuracy analysis in Sahitler hill. Measurement 73: 539-543.

(44) Wallace L, Lucieer A, Malenovsky Z, Turner D, Vopenka P (2016). Assessment of forest structure using two UAV techniques: a comparison of airborne laser scanning and Structure from Motion (SfM) point clouds. Forests 7: 62.

(45) Walther BA, Moore JL (2005). The concepts of bias, precision and accuracy, and their use in testing the performance of species richness estimators, with a literature review of estimator performance. Ecography 28: 815-829.

(46) Watson JEM, Evans T, Venter O, Williams B, Tulloch A, Stewart C, Thompson I, Ray JC, Murray K, Salazar A, McAlpine C, Potapov P, Walston J, Robinson JG, Painter M, Wilkie D, Filardi C, Laurance WF, Houghton RA, Maxwell S, Grantham H, Samper C, Wang S, Laestadius L, Runting RK, Silva-Chávez GA, Ervin J, Lindenmayer D (2018). The exceptional value of intact forest ecosystems. Nature Ecology and Evolution 2: 599-610.

(47) Wierzbicki D, Kedzierski M, Fryskowska A (2015). Assesment of the influence of uav image quality on the orthophoto production. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XL-1/W 4: 1-8.

(48) Willmott CJ, Matsuura K (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research 30: 79-82.