Comparison of Free and Commercial Software in the Processing of Data Obtained from Non-Metric Cameras

The main goal of the work was an attempt to compare the free and commercial photogrammetry software for processing the pictures taken with a camera drone, a traditional digital camera and a smartphone. Due to a wide range of programs on the market, four were selected for comparison (Agisoft Metashape, DroneDeploy, VisualSfM, COLMAP). Their brief description was presented, and then the photos were processed in each of them. Three sets of photographs were used for the processing (part of a residential area, photos of a building, and photos of a tree trunk). As a result, the capabilities of the selected applications were presented on the basis of various input data. Not every program was able to deliver all the desired products. Moreover, they differ depending on the software. The commercial applications have more functionalities. On the other hand, the open-source solutions allow for the development of algorithms. Working in any environment had its own characteristics. The selected applications were compared on the basis of the processing and the results obtained. Due to many aspects of their evaluation, it turned out that the research topic was very extensive. Moreover, it was found that it is very difficult to make an objective statement of the tested programs, because the same program can be scored differently, depending on the user’s needs, capabilities and knowledge.


INTRODUCTION
In recent years, drones have found many different, unusual applications and have become a space for scientific research [Prisacariu 2017]. They are used, among others, in geology, to make observations, on the basis of which maps and 3D models are created. They are the tools that quickly provide a lot of data in sufficient resolution for analysis and research [Bemis et al. 2014]. Almost every branch of geodesy uses technological innovations. Drones are considered as an alternative way to update the existing land and building registry resource sets (EGiB) [Plichta et al. 2017]. Environmental protection is also applied to unmanned aerial vehicles as mobile stations for monitoring the concentrations of particulate matter and other pollutants in the air [Adamski et al. 2018;Villa et al. 2016]. Ecologists look for solutions using photogrammetry and the 3D model obtained in the process of image processing as research material [Ourloglou et al. 2020]. In mining, unmanned aerial vehicle (UAV) monitors the condition and operation of the mine [Ren et al. 2019]. In forestry, the potential of drones has also been used to replace the traditional, unprofitable and cyclically repeated forest inventories [Mohan et al. 2017, Mikita et al. 2016]. Due to the many design advantages and possibilities to enrich them with different sensors, the development of their use has long been predicted. Most often, the UAV are equipped with cameras, mainly registering visible light in the form of high-resolution digital photography. They give the products which are the easiest to process for professional and advanced users using the photogrammetric methods [Villa et al. 2016, Niethammer et al. 2011]. In addition to the photos obtained from drones, terrestrial photographs in a dynamic projection, taken using smartphones and classic digital cameras, are used increasingly often. They are increasingly becoming the basis for the 3D reconstruction using algorithms instead of the classic photogrammetric conversions [Abed Al Aziz et al. 2015]. Already in 2011 it was confirmed that the results obtained with a low-quality phone camera are intriguing and prospective for the future [Sarhan 2011]. Owing to the size of the phones and their ease of use, they can be taken anywhere. Additionally, the technology of taking pictures with them is cheap and common. The examples of photograph applications appear in the inventory of architectural monuments [Aicardi et al. 2014] and natural objects -e.g. Bossea Cave [Dabove et al. 2019].
Due to the growing number of acquired photographs, there is a need to search for the tools that will allow transforming a tens or even hundreds of overlapping photographs, perform 3D reconstructions in order to enable conducting further analyses on them. On Internet forums and websites [Uebel 2020, Corrigan 2020], scientific and professional literature [Królikowski 2017[Królikowski , 2018 has repeatedly tried to classify the existing free and commercial photogrammetric or computer vision (CV) applications for UAV images, cameras and phones. Their disadvantages, advantages and possibilities were evaluated. Each program works on different processing algorithm. There are no objective lists, because the evaluation of a program depends on many factors.
Expensive laser scanners can be used to obtain a point cloud for modeling. An alternative for reconstruction, documentation and inventory purposes is to process the photos in an appropriate program based on 3D modeling and reconstruction [Rahaman, Champion 2019].
The photogrammetric processing of photos is always based on similar steps of the transformation algorithm performed from photos to 3D geospatial products: • taking digital photographs and adding them in to the program environment, • detection of objects in photos, matching, reconstruction of internal orientation [Wilfried 2009], • reconstruction of a sparse point cloud, reconstruction of the beam of rays and determination of external orientation, triangulation On this basis, individual products that can be obtained in the processing process are visible: parameters of internal and external orientation, point clouds, 3D models, numerical terrain models and orthophotos.
This paper involves a comparison of the possibilities of using free and commercial applications available on the market to process the images of various objects from UAVs, traditional digital camera and smartphone, with a view to use the products for inventory purposes. The most common environments were selected based on the literature. The presented commercial programs, i.e. Agisoft Metashape, DroneDeploy and COLMAP, and VisualSfM, were used to present open-source solutions. Then, they were used to process three sets of data and evaluate the received products, which in the long run can be used in processing for the inventory and research goal.

MATERIAL AND METHOD Evaluation criteria for software
During the search for suitable application examples for the processing of the research material, an attempt was made to specify various criteria for evaluation and comparison. The basic, most important of them are the available application functions and the type of the final product of the study. In practice, the selection of the application usually starts with identifying the products that users want. Among the testers of free and commercial programs there is a large variety of needs and skills, so in addition to the effects of work for potential users, other components are also important. Among other things, the possible workflows and their degree of automation must also be taken into account. Professionals and scientists usually look for the programs with the ability to edit parameters.
Hobbyists and laymen pay more attention to the simplicity of program operation [Rahaman, Champion 2019]. The tools for checking and validating the results to evaluate the generated material are also welcome in applications [Sona et al. 2014]. Another aspect of evaluation will be the performance of the program, sometimes independent of hardware capabilities and the algorithm used. Of course, the fastest possible processing is most beneficial. In addition to photo processing, the data analysis tools would be an ideal addition to the application. Distance, area and more advanced measurements i.e. maps of slopes, exposure [Zou et al. 2016]. Additionally, we have to check the types of sensors the products of which are supported in the application (e.g. multispectral photos). The last important element are the hardware requirements in relation to the type of program and duration of its use. Sometimes, it may be more profitable to work in the cloud than to buy expensive equipment [Królikowski 2017].

Description of the study area
The computer used to develop the test block of photos was equipped with an Intel (R) Core (TM) i7-9700K CPU 3.00 GHz, 32 GB RAM and an NVIDIA GeForce RTX 2070 graphics card with CUDA Compute Unified Device Architecture (CUDA) support, a technology that allows using the computing power of the graphics card to solve general numerical problems [Błaszczak-Bąk et al. 2016]. The computer is running the Windows 10 Professional operating system.
Three types of data were used to perform the processing in selected applications. The photos of a fragment of the Jaroty estate, a data sample provided by Agisoft -the building, and smartphone images of the remains of a tree trunk after its cutting.
The first data sample used for processing were the photos from the flight taken over the Jaroty housing estate in Olsztyn. They presented various terrain forms, from buildings to green areas and forests. The flight was carried out with the DJI Inspire 1 PRO multi-rotor UAV. The platform was moving at a constant speed and the photos were taken at regular intervals. About 80% transverse and longitudinal coverage was obtained. The flight was carried out in 9 rows at a height of 100 m above the ground level. In total, 63 parallel images were obtained (Fig. 1).
The drone transported a DJI Zenmuse X5 camera with a focal length of 15 mm with a field of view of 720 and a 4/3 CMOS matrix with a resolution of 16 Mpix. The stabilization of the sensor in flight was provided by a three-axis gimbal. It allowed the orientation of the camera axis with an accuracy of 0.02° [DJI 2017]. The photos were taken in the resolution of 4000 x 3000 pixels with an aperture of f/2.8, ISO 100 and a different exposure time. Before the flight, seven ground control points (GCP) were marked in the field with yellow neon paint (Fig. 2).
Additionally, thirty-five check points were selected including permanent and unambiguously identifiable points in the field in the form of curbs, wells, and sewage grids (Fig. 3).
GCP and check points were measured with the Leica CS15 receiver with the GS08plus antenna. The flat coordinates were determined in the PL2000 coordinate system for zone 7, while the heights were determined in the Kronsztad PL-KRON86-NH system. In the course of data processing, the coordinates were transformed, as required, using TRANSPOL 2.06 software.
The photos of the building provided by Agisoft [https://www.agisoft.com/downloads/ sample-data/] were used as another research material. The data sample consists of 50 convergent terrestrial photographs taken with a Canon EOS 5D digital camera with a focal length of 24 mm. The resolution of the images is 4368 x 2912 pixels. Natural light was used. The aperture unit was f/9, ISO 200 and the exposure time was 1/200 s. The subject was photographed only from the side. It has many characteristic elements, regular construction. There are no photos showing the roof covering. Its location is unknown, and it is missing in the EXIF file of the photos. The examples of photographs are shown in Figure 4.
The last processed set of data were photos taken for the purpose of research, showing a fragment of a tree trunk (Fig. 5). The object is asymmetrical, with many colors and an irregular surface. The images were taken with a Xiaomi Redmi Note 7 smartphone equipped with a 48 Mpix camera, which takes photos at a resolution of 6000 x 8000 pixels. 5 mm focal length. In total, 92 convergent photos were acquired, also from the top of the object, at different distances around, under a cloudy sky. The aperture value was f/1.8, ISO 800, the exposure time was 1/60 s. The phone created EXIF files which also contain the GPS location.
All the photos used show a range of different objects that can be modeled -from engineering, land development to natural objects.

Description of tested programs
Many lists of the programs used were found in literature and online sources. After reviewing them, four programs (two free and two paid) were selected for processing, which were most often and best rated. In order to make the presentation of the commercial programs more attractive, one desktop program (Agisoft Metashape) and one working in the cloud (DroneDeploy) were selected. The examples of free applications are COLMAP and VisualSfM.

Agisoft Metashape 1.5.4
Produced by Agisoft LCC. Agisoft Metashape is a program for processing digital images (aerial and terrestrial) from RGB, thermal and multispectral cameras into dense point clouds, textured 3D and DEM models, orthophotos. The program is based on the Python programming language, but the exact operation algorithms are unknown. It has an extensive interface and Python console. Two processing paths are possible. The first is Fig. 2. View of the marked GCP automatic step-by-step guidance through the subsequent stages of development. The second way is advanced, enabling to change many parameters at each stage of development. Detailed settings concern products and program operation. It comes in two extensions: Professional and Standard Editions. The mentioned software belongs to the group of stationary, paid programs. Package prices start from approx. PLN 900 to even PLN 17000 for a professional user [Agisoft].

VisualSfM
VisualSFM is another completely free system. The name itself indicates the method used to build point clouds from Structure-from-Motion (SfM). There are two modes of operation in the system: command line or GUI. A Google Designer -developer Changchang Wu adopted VisualS-fM as part of another research. The environment includes basic and extended functions, so that each user can find something for themselves -a quick process or the ability to edit each parameter separately [Uebel 2020]. The developer declares that the program will run very smoothly by using parallel processing and multi-core code to build  COLMAP 3.6 is free software available for download from the GitHub website hosting service. Processing can be started both from the command line and from the graphical user interface. COLMAP is created by the scientific community. The program can generate a 3D mesh, but also work with a dense point cloud. The application uses the SfM algorithm to calculate the initial orientation of photos and the MVS Multi-View Stereo (MVS) as the basis of its operation [Schoenberger 2020].

Processing
In order to check the functionality of the programs in more detail, three types of pictures were used: parallel from a drone (Jaroty estate), convergent from a digital camera (building) and convergent from a smartphone (a fragment of a tree trunk). Three sets of data, in different environments, may have given different processing effects (not every set of photos allows for all processing steps). In each of the programs, if it was possible to choose, an automatic processing path with medium calculation accuracy was used. The instructions and recommendations from the websites were followed. In order to make a comparison, it was necessary to carry out the processing in the selected applications and to check the possibilities of manufacturing products in different environments, while evaluating their additional aspects [Remondino et al. 2012].

Application features and processing products
After completing the entire processing paths, the information about the possibility of producing given products in the analyzed environments was obtained. The obtained elements of photogrammetric photo processing in four environments were summarized. The results are presented in Table 1.
Undoubtedly, Agisoft Metashape is at the forefront of the paid programs when considering the issue of available functions and products. The applied algorithms allowed obtaining correct reconstructions of objects, regardless of the type of photos. The advantage is the division of work into steps and the option of exporting the results. At each stage, it was possible to change the parameters, which resulted in the selection of the optimal settings, and thus influenced the final result. Unfortunately, without sufficient knowledge, the end result can easily be worsened. Additionally, the license price is high. The program allowed for the development of typical photogrammetric products, such as DTM (Fig. 6b) or orthomosaic (Fig. 6a).
In four programs, processing was carried out for two sets of photos: the Jaroty estate and photos of a trunk fragment. The third set of photos of the building, not having the location data in the EXIF file, made it impossible to apply them to Drone-Deploy. This program is not suitable for modeling reality with photos without georeference. In addition, Drone Deploy has a path of work that is simplified to a minimum, and also limits the access to intermediate products, and does not describe their properties. The cloud app only showed one point cloud, it did not classify it as sparse or dense. On the other hand, it has the ability to plan, perform a UAV raid, send data in real time and give the opportunity to perform chamber work on a regular basis to another person in the office. An orthophotomap (Fig. 7a) and DTM (Fig. 7b) were also generated using a commercial DroneDeploy.
The most abundant dense point clouds come from Metashape. Approximately four times lower point cloud density was generated in VisualS-fM, and the rarest cloud is created by COLMAP. On the example of the photos of the building being developed in the tested programs, it can be stated that Agisoft unnecessarily builds a cloud of points from the photos on which fragments of the sky are visible, which is its serious disadvantage (Fig. 8).
None of the free programs create DTM and orthophotos. VisualSfM does not generate a dense cloud of points from parallel photos (Jaroty estate). Among the free programs, a) b)  COLMAP produced more products. In contrast, VisualSfM generates more complex point clouds, both sparse and dense from convergent images (Fig. 9). The free programs have lesser processing capacity than the paid ones, which can be circumvented by using more free applications like Meshlab [Khalil, Grussenmeyer 2019]. Both free and commercial programs allow obtaining 3D models that have a wide range of applications and are an alternative to expensive laser scanning. For modeling, it is possible to use the photos of engineering structures (Fig. 10) as well as nature objects (Fig. 11). The more diverse the object is, the better the program does with matching.

Ways and comfort of work
During the work, attention was paid to the possibility of adapting the program to the user and the availability of analysis and accuracy control tools. Particular attention was paid to the construction of the interface and the existence of a command line that could be used by advanced users. The observations are presented in Table 2.
Each of the tested programs allows editing the graphic user interface to a greater or lesser extent. Agisoft has the most extensive menu, and DroneDeploy the most simplified, and does not allow image processing using a programming language code. Only VisualSfM does not allow processing photos through an automatic workflow.
Nevertheless, the interface is quite intuitive and easy to learn [Vacca et al. 2018]. COLMAP and VisualSfM do not have data accuracy and analysis tools, only commercial environments have them [Niederheiser et al. 2016].

Working time
Due to the fact that the program's work path was analyzed on one computer, the speed of the performed processing can be compared. Processing of three sets of photos was performed in the average quality determined by the given application. The program work times on some of the modeling stages performed were compared. The study boundaries, which were automatically set by the programs, have not been edited ( Table 3).
The free programs take longer to load photos. Of course, cloud computing is the quickest. The process itself takes place regardless of the user's computing power, and an automatically sent email informs about its completion. However, a stable internet connection is required. There is a significant difference between Agisoft Metashape and COLMAP in the time of working with photos of the estate, which unfortunately does not translate into the number of points generated [Jiang et al. 2020]. Colmap builds clouds faster for parallel photos, whereas Agisoft Metashape -for concurrent photos. In the case of 92 converging images of the trunk, despite the average result number of reproduced points, the process in VisualSfM finishes work much faster than the other applications. Smaller collections of photos are processed the fastest in Colmap.
Depending on the program and processing algorithm, different numbers of points were created in the models (Table 4).
Agisoft generates the most points, regardless of the data set. A similar assessment was made during other studies, during which point clouds were also generated on various sets of photos [Jiang et al. 2020]. DroneDeploy does not provide the information about the size of the generated cloud or does not create it (photos without EXIF). Desktop programs are primarily based on the work of the graphics processor, the efficiency of which is crucial for the processing time. The free programs will also achieve similar results, but the user must be more patient.

The use of GCP and checkpoints
When preparing the UAV photos of a fragment of Olsztyn, attempts were also made to use the ground control points and checkpoints at hand. They were included in the analysis process in three programs -without COLMAP. Additionally, the value of the aerotriangulation adjustment error and information about the coordinate system of the environment were searched ( Table 5).
The paid programs allow the use of GCP and checkpoints in the process of aerotriangulation and modeling, which makes it possible to geo-reference the products and extend the analysis. Additionally, the coordinate system can be adjusted to the data held. VisualSfM is a free environment that will suit the needs of non-commercial users  for photogrammetric dense cloud generation; adding GCPs is, unfortunately, more complicated and time-consuming. For other characteristics, specific hardware requirements were checked -only COLMAP requires the CUDA graphics card support. It is also worth noting that Agisoft accepts most of the input photo formats. Other programs use JPEG as the basic format. Point cloud export formats for commercial programs are, among others, LAS, XYZ, TXT. The free applications mainly save them in NVM and PLY extensions.

CONCLUSIONS
The presented analysis of the workflow in different types of programs shows a wide range of available functions in the programs available on the market. Many studies are trying to evaluate the best software, but for each user a different program may become the most suitable. Each program has its strengths and weaknesses (a given functionality will be beneficial for one group, and not for another). The only way to choose the best working environment is to test and learn about its performance characteristics. It is not possible to objectively compare applications. It all depends on the user's knowledge, desired products, financial and computational capabilities, and the purpose of processing. The free programs have many more limitations, but they will find supporters among the amateurs who are looking for interesting solutions in 3D modeling, and the accuracy of development is not the most important factor for them. Scientists, on the other hand, have a chance to develop the proposed algorithms and improve them. The commercial systems have greater functionality and are more convenient, but the operation itself is performed in the so-called black box, the algorithms are not widely known and cannot be modified. At the same time, it is possible to achieve relatively accurate results, and this, in turn, is important for the people who perform processing for profit.
Among the programs tested, when looking at the aspect of data analysis, one can see a significant difference between the paid and free applications. The commercial software allows generating a full report of the processed data. Both free and paid programs create products necessary for 3D modeling, but they differ in detail. Users must decide for themselves whether they should work in the cloud or invest in hardware. It is necessary to determine whether the product received in the tested application will be final or further processed, and then pay attention to the available export formats. By comparing the compatibility of the result sets and the ability to export them to common formats, both commercial and free programs offer a wide range of possibilities. The work performed on a free program can be continued in a paid one. The free programs require greater knowledge because they are usually the result of scientific work, so they do not have extensive forms of support and technical assistance. Not every program has an automatic workflow, which suggests that it can be used mainly by advanced users. In case the calculation time is not crucial, some funds can be saved by choosing free software. To summarize, the topic of the work is very extensive, and this work indicates its complexity and the need to continue research.