1. Introduction
With the constant progression of urbanization and industrialization, the mobility of rural young and middle-aged laborers has intensified [1]. The sustainable transfer of non-agricultural labor has led to a decline in agricultural labor, and the problem of aging of the agricultural labor force has become more serious [2]. For example, the proportion of agricultural production and management personnel aged 55 and over is as high as 33.6% according to the main data of the Third Agricultural Census Bulletin of China in 2017. Furthermore, as aging continues, the physical health of the elderly labor force continues to decline, which results in a significant reduction in the supply of effective rural labor and an adverse effect on agricultural output [3,4]. Most agricultural production tasks are labor-intensive and seasonally oriented projects that exacerbate the constraint of seasonal labor shortages [3,4,5] and increase the cost of agricultural labor. For example, according to a survey conducted by Zhen et al. [6], during the rural busy season, the labor cost of agricultural planting increased from 80 CNY per person per day in 2015 to 90 CNY in 2016 to 100 CNY per person per day in 2017, and the labor cost of technical agricultural labor is even higher [7]. Some statistics show that the agricultural unit labor cost in developed countries such as Germany, Japan, and the United States decreased by 30.17%, 44.22%, and 23.44%, respectively, while agricultural labor productivity increased by 64.78%, 81.22%, and 34.83%, respectively, from 2005 to 2014. On the contrary, China’s agricultural unit labor cost increased by 45.17%, and agricultural labor productivity, which is much smaller than that of developed countries, increased by 50.28% [8]. Therefore, the increasing cost of labor will lead to an increase in the cost of agriculture, which will result in a relative decrease in agricultural productivity and international competitiveness [9]. It is urgent to enhance the innovation of agricultural science and technology and replace extensive and expensive repetitive manual operations with intelligent agricultural machines or robots [10].
Due to the growing maturity of computer technology, sensor technology, and control theory, different types of agricultural robots have been developed based on characteristics of agronomy, such as fruit- or vegetable-picking robots, spraying robots, and harvesting robots. The agricultural robot can replace traditional human efforts to engage in all kinds of labor-intensive and complicated agricultural production activities and reduce the decline of output caused by improper human operation, negligence, inaccurate operation, and other reasons, as well as major physical injuries and even casualties of operators [11]. However, the operation efficiency of a single agricultural robot is too low and cannot meet the operation demand in busy seasons without coordination and cooperation by artificial auxiliary resources or other robots [12]. As early as 2009 and 2012, Johnson et al. [13], Moorehead et al. [14] and others in the United States replaced a single robot with a group of agricultural mobile robots to complete mud moss harvesting and orchard spraying successively with appropriate cooperative operation mechanism, which can reduce production costs and improve operational efficiency [15]. Therefore, to adapt to the increasing scale of production, meet the needs of social development, and narrow the gap with other international world powers with advanced scientific capabilities, it is necessary to research the relevant technology of agricultural multiple robot systems. This work focuses on the research progress of the cooperative operation, one of the key technologies of agricultural multiple robot systems.
There are many types of agricultural multi-robots, and this article mainly focuses on unmanned aerial vehicles (UAVs), unmanned ground vehicles (UGVs), and unmanned underwater vehicles (UUVs). This paper summarizes the research progress on synergistic technologies of cooperative operation of the abovementioned multiple robots and characterizes the expected research of related technologies. 2. Problems with Multi-Robot Applied to Agricultural Environments
Crops are fixed in cultivation season and time, which determines the labor demand pattern of agricultural production in a year, and agricultural operators need to make flexible responses and treatments according to the growth pattern of crops, such as plowing, planting, management, and harvesting [16]. At the same time, to adapt to the development of agricultural intensification, scale, and industrialization, and to reduce the economic losses caused by untimely processing, a collaboration of multiple farm machinery operated by people has widely appeared in agricultural production (as shown in Figure 1).
In Figure 1a multiple rotary tillers are being used to plow rice fields on sloping land to safeguard food production and mitigate the impact of the phenomena of lack of labor resources, which is brought about by the New Crown epidemic [17]. In Figure 1b, multiple corn planters are employed to sow seeds in a large field, which saves labor and ensures the quality of seeding, and directly improves the yield and quality of corn planting [18]. In Figure 1c, multiple drones are used to spray pesticides in cotton fields, which could be targeted according to the types of pests and diseases, and also prevented their rapid spread in the early stages of infestation [19]. In Figure 1d, it has become a trend to manually operate multiple combines simultaneously during the wheat harvesting season to avoid the effects of rainfall on wheat quality and yield [20].
By manually operating the machines, the quality of operation of the implements relies heavily on human experience, while the use of robots instead of manual operation can free up manpower and ensure the quality of operation. However, the topography, soil, light, and climate conditions of the crop growing environment are different from those of indoor and urban transportation environments, and these conditions pose a challenge to the application of multiple robots in agriculture.
Take the example of multiple harvesters harvesting grain in farmland. First, farmland is an unstructured environment, which means that the road conditions are undulating, there are various types of obstacles, and there are missing or blurred lane lines on the ground, and agricultural machines both share the same resource and interact with each other to become dynamic obstacles to each other. Secondly, agricultural operation tasks have strong requirements for operation time, such as harvesting grains in a fixed short time frame. Furthermore, the amount of grain output varies from plot to plot, and the number of agricultural machines needed and the number of operational tasks assigned to them is dynamically changing (such as harvesters with large loading capacity should match the plots with large grain output). Finally, even if the same type of farm machines work together, the characteristics of the machines are not the same (for example, the harvester with the same loading capacity, the fuel consumption is different, the harvester with high fuel consumption should be assigned the operation task more than its work cost, and its operation path should be as short as possible but the harvesting volume should be as large as possible). To ensure that multiple machines can cooperate, a multi-robot system is required to be able to organize multiple robots flexibly, quickly, and efficiently according to the changes in the environment and tasks, and to fully utilize the capabilities of each robot to finally complete the given task with high quality [21]. At the technical level, in addition to being accurately informed of the positioning information of the swarm and the environmental information of the operation, and solving collision and obstacle avoidance, it is also necessary to assign operational tasks to multiple machines, plan operational paths (such as in areas where multiple robots work together), coordinate the formation control of multiple robots, and maintain the information interaction between multiple robots.
3. Research Progress on Synergistic Technologies of Agricultural Multi-Robots To solve the above problems, it is necessary to study technologies of collaborative operation, such as environment perception, task allocation, path planning, formation control, and communication-based on multi-robot architecture. Since each technology does not solve the same agricultural problems, this section first classifies the types of development of these technologies, then describes and reviews each of these cooperative technologies in terms of research methods or problem solving, and finally summarizes their research development status and characteristics. 3.1. Architecture of Agricultural Multiple Robot Systems
A reasonable architecture can guarantee information flow and control flow in the agricultural multi-robot system and make effective cooperation among multiple robots possible [22]. At present, the architecture of agricultural multi-robots can be divided into centralized architecture and distributed architecture. It is found that the earliest recorded structure of agricultural multiple robot systems comes from hay harvesting and transportation robots in farmland [23,24], for which these multi-robot systems were operated under the principle of centralized architecture. As shown in Figure 2a, in leader-follower mode, a relatively powerful robot is selected as the “leader” of the swarm robots, performing specific motion planning for the remaining robots after analyzing and processing the sensory information, but these remaining robots are just executors, without the ability to choose their actions or coordinate with each other. Alternatively, as shown in Figure 2b, in the central controller mode, each robot can perform tasks independently and is commanded by a central controller [25]. The advantage of this centralized architecture is that the theoretical background is clear, and the implementation is intuitive, but the flexibility, fault tolerance, and adaptability are poor [26].
Compared with the operating environment of UGVs, the operating area of UAVs has the advantage of no obstacles, so these systems generally adopt the distributed structure [28]. As shown in Figure 3, the three UAVs in the multi-robot system carry out agricultural situation monitoring in the individual workspaces independently; each robot of this system had a high degree of autonomous operation ability and can complete a given task according to its aims; robots can communicate with each other, exchange information, and coordinate their behaviors equally and independently to complete a given task [28]. This structure has strong scalability and certain advantages in real-time operation, fault tolerance, and reliability, etc. [29] and is suitable for handling tasks related to spatial states [30]. However, “it costs a lot in terms of the coordination mechanism, such as task allocation and motion planning” [29].
The centralized architecture that can be divided into a leader and follower robots is suitable for highly coordinated tasks and is advantageous in a fully known environment. The distributed architecture, in which there is no affiliation among robots, is suitable for weakly coordinated tasks and is advantageous in large-scale, complex, and varying environments. 3.2. Environment Perception
Environmental perception is the premise of the cooperative operation of multiple robots in agriculture. That is, the mobile carrier can use the sensors carried by itself (these sensors include internal sensors and external sensors [31], where the internal sensors include odometers, magnetic compasses, inertial navigation, and global positioning systems to determine the speed, position, and direction of the robot in the environment; external sensors include ultrasound, infrared, laser and vision, used to sense surrounding information [26]) to obtain the information of the surrounding environment, extract the effective feature information within the environment for processing and analysis, and finally establish the environment model [32]. This technology mainly involves collaborative positioning, data fusion, and environmental construction.
3.2.1. Co-Location Technology
The concept of co-location was originally proposed in 1994 by Kurazume and Nagata in Japan [33] The concept refers to a robot “sharing” its positioning results with other mobile robots and to other robots using this shared information to integrate their calculation results to improve the accuracy of positioning themselves and, in turn, sharing their positioning results with other mobile robots, repeatedly achieving the precise positioning of mobile robots [34]. According to the collaborative positioning method, this approach can be divided into active positioning, passive positioning, and interactive positioning [35]. However, no research on interactive localization has been reported in the literature of agricultural multi-robots.
- Active positioning
In the absence of information exchange, the robot relies on its sensors to obtain relative distances and angles by observing the neighboring robots for self-positioning [35,36]. As shown in Figure 4, the leading robot guides the follower robot to steer along the leading robot reference route [37].
For example, the black-and-white checkerboard feature board was fixed to the leader robot as the following feature, and the 3D information of each corner point on the black-and-white checkerboard feature board was obtained by the binocular vision camera fixed to the following robot, and the information was analyzed to finally obtain the longitudinal spacing, lateral offset and heading declination of the following robot relative to the leader robot. Using this navigation information to realize the automatic following of the following vehicle, the following system of master-slave orchard operation vehicle is established [38].
- Passive positioning
In an environment where information exchange exists, the robot indirectly obtains the relative distance and angle through the “observed” data provided by the friendly neighboring robot to perform its positioning [35]. As shown in Figure 5, the follower robot dynamically creates a reference heading for itself from the position point of the leading robot [39].
Passive positioning mode is usually used in combination with kinematic control models and is one of the most used in master-slave robots. For example, GPS positioning was used for multiple robots, and the travel trajectory of the pilot robot was the main one. Under the premise of communicable, the pilot robot estimated the motion trajectory of other robots using the kinematic model, and the difference between the sensor positioning and the estimated positioning value was calculated by the other robots or the pilot robot, and the lateral and longitudinal displacement of each robot was adjusted according to the difference [40].
However, the above positioning methods are highly dependent on the positioning accuracy of the leader robot, which requires good stability and robustness. When the leader robot fails, it is easy to cause the localization accuracy of the whole team to drop, or even the localization fails. In contrast, the interactive localization approach has an environment of information exchange, where robots achieve joint localization through these steps of mutual observation, data exchange, and information fusion [41]. In this case, the robots are in the same position in the team and there is no master-slave distinction, namely, they do not depend on the positioning accuracy of a fixed robot, and this type of phenomenon is reduced. The above co-location methods are summarized as shown in Table 1.
It can be seen from Table 1 that the cooperative localization technology for agricultural multi-robots mainly adopts active localization and passive localization, which are computationally small and easy to implement. The real concept of “co-location” should be interactive positioning; through communication among robots, information sharing can be realized, and then the robot’s positioning error can be corrected to achieve accurate positioning. However, this method incurs a large amount of calculation and a large complexity of the algorithm, making it difficult to implement. No examples have been found in the literature for agricultural multi-robots.
More practical applications indicate that the cooperative positioning of multiple robots in agriculture is replaced by a central controller or task manager coordination mechanism [49,50]. That is, the working area and travel path of each robot are planned by a task manager or central controller based on the established environment map. The work areas and work paths of these multiple robots usually do not intersect and each robot only needs to localize and navigate based on its sensors. For example [27], the vineyard area was divided into three UAV-monitored vineyard areas using the task manager according to set rules, and path planning was performed for the sub-areas of individual UAV operations, with each UAV flying at a different altitude to avoid collisions between the UAVs.
3.2.2. Data Fusion Technology
A single sensor has certain limitations. For positioning accuracy and reliability, it is necessary to utilize the advantageous features of each sensor, that is, data fusion of multiple sensors. Data fusion technology, also known as multi-sensor information fusion technology (MSIF), essentially involves the comprehensive processing of target information originating from different sensors at different times or multi-target information simultaneously, to obtain more accurate positioning, identification information of the measured environment or object, and comprehensive and timely assessment of the current situation [51,52], thus facilitating the subsequent planning and decision making of the robot. The strength of the data fusion capability directly affects whether the robot can effectively achieve mutual coordination and collaborative work.
The internal sensors in the agricultural robot mainly include a global positioning system (GPS), inertial measurement unit (IMU), steering angle potentiometer, and encoder, which are used to provide the robot with the position, heading, and steering angle information; the external sensors mainly include various LIDAR and cameras, which are used to avoid obstacles and collect environmental information, as shown in Figure 6. Among them, GPS can provide a unified coordinate system and accurate position information for field robots and is used most frequently [53]. With the promotion of satellite positioning technology, agricultural robots equipped with GPS positioning and navigation systems will become increasingly popular. Take the automatic navigation system System150 researched by TOPCON company of Tokyo in Japan as an example [54], the system adopts GPS-based advanced inertial guidance and terrain compensation technology, which can realize navigation in complex terrain environments with ±2.5 cm accuracy for straight line and turn. However, data fusion techniques are still very important for robot localization when GPS cannot obtain accurate position information in greenhouses or forests [55], or when the robot is too small to install high-precision sensors.
For example, to obtain information on the ambient temperature, humidity, light, and CO2 concentration in a greenhouse [56], a human remotely operated UAV was first operated to obtain a map of the greenhouse environment, and then the ground robot fused IMU, GPS and odometer information to output the actual location information of the robot through the extended Kalman filter (EKF) algorithm. In practical applications, because of the poor GPS signal in the greenhouse, the EKF was used to fuse the odometer and IMU. EKF is used to linearize the nonlinear system at the reference point using Taylor expansions, and then Kalman filtering theory is used to achieve the prediction and correction of the system. But the EKF still cannot solve the global localization problem [57].
Another example is an agricultural spraying multi-robot [58] that used particle filtering to fuse information from multiple low-cost sensors of the odometer, IMU, wheel encoder sensors, and GPS, which incorporates the open-source library RTKLIB and correction signals. The particle filter could also determine robot attitude based on a series of particles in noisy environments and when the GPS was offline. Particle filtering [59] is a basic method based on Bayesian filtering theory (the robot can determine the poses with a certain degree of confidence based on all available information [60]) and differs from Kalman in using particle sets to describe the probability distribution. However, a large number of particles need to be maintained for higher localization accuracy, which will consume a large number of computational resources, especially as the walking distance gets farther, which will put greater pressure on the computing platform with limited memory resources.
According to the sensor information fusion processing hierarchy, the technique is divided into three levels, namely the data layer, the feature layer, and the decision layer [61]. Their specific scopes, characteristics, and fusion algorithms are shown in Table 2.
From Table 2, it can be seen that information fusion in agricultural multi-robots is focused on multi-sensor information fusion of single robots, and there is almost no research on sensor information fusion between homogeneous or heterogeneous agricultural multi-robots. To obtain more information, the information collected by different types of sensors on a single robot is mostly different (e.g., GPS collects position information, IMU collects robot heading, odometer, etc.), but some information has a mutual transformation relationship (e.g., the integration of velocity from odometer can give distance), which can be regarded as the same kind of sensors at this time. The fusion algorithms are mainly based on classical EKF and particle filtering.
3.2.3. Mapping
Once the multi-robot has determined its position, it also needs to determine information about the multiple robots’ surrounding environment, for instance, the presence of obstacles. Mapping is the task of accurately describing the spatial position of the robot working environment and various objects (such as obstacles and road signs) in the environment, that is, to establish a spatial model (two-dimensional or three-dimensional) or map [26]. The purpose of creating this map is to provide path planning for the robot, so the map must be easy for the robot to understand and computationally manipulate and accommodate revision when new environmental information is detected [64].
At present, many methods have been developed for constructing environmental models for multiple robots, which are mainly summarized in the three types of grid-based model, geometric mode, and topological mode [22]. Probably due to the low environmental information of the topological model, no literature was found on the use of multiple robots in agriculture. On the contrary, the grid model and the geometric model provide abundant information about the agricultural environment, the purpose of multi-robot operations is clear, and more literature is applied to agricultural operations. And the detailed environment model facilitates the task allocation to multiple robots, real-time observation of multiple robots’ motion, effective coordination mechanisms, and detection of robot motion faults.
- Grid model
The uniformly distributed grid represents the working environment of the robot, and each grid corresponds to the position of whether an obstacle exists or not [22]. For example, if a grid has an obstacle, it is marked as 1; otherwise, it is 0.
As shown in Figure 7, Bouzouita et al. [28] obtained georeferenced pictures through UAVs and then created a full map of a vineyard. The processing entails first dividing the UAV workspace into multiple regular grids and defined borders (line segments of workspace in the grid-based environment) using Bressenham’s linear algorithm (BLA). Then, the procedure to distinguish the subareas in the grid-based environment is a recursive flood-fill algorithm that picks an empty cell (not marked as occupied) and floods in four directions while there are empty cells, and each flooded cell is marked as occupied. This mapping procedure is repeated until all nodes of the grid are occupied by georeferenced pictures. However, the drawback of this mapping is a tradeoff between acquiring an equally sized set of images in cells and inefficient aerial sampling with UAVs.
- Geometric model
Geometric features such as line segments or curves are extracted from the environmental perception information and used by robots to construct an environmental map [22]. As shown in Figure 8, Ball et al. [65] defined the boundary of a 55-hectare sorghum stubble field by manually selecting appropriate latitude and longitude coordinates but omitted the headland where the UGVs turned around at the end of each row. Besides, the positions of multiple robots and obstacle information (human, pole, vehicle, etc.) were periodically sent to the central controller through sensors of a real robot and 12 simulated robots. Finally, the central controller constructed a 2D map of the environment surrounding the robot.
A more scientific approach than manually labeling is the Voronoi diagram [66]. For example, a fruit tree on the robot’s driving path was taken as a point in the orchard map, and the two-dimensional coordinates of this point were found in the image acquired by the UAV. At the same time, the data of planting interval and density of fruit trees were learned from this image, and the two-dimensional coordinates of other fruit trees were obtained, and then the K-means clustering algorithm was used to find the center of clusters of multiple fruit trees, and the orchard was divided into multiple sub-regions for robot operation according to the principle that the fruit tree point is most adjacent to this center. But the experimental results after mapping are to be further validated on a real agricultural spraying multi-robot.
- Topological model
The topological relationship among objects in the environment is represented by the environment model [21]. The topological model consists of multiple unrelated points and lines with a simple model. Some researchers proposed to use this model in agricultural multi-robot path planning, but it is only an idea and has not been applied [67].
The specific application scope and characteristics of the multi-robot environmental model are shown in Table 3.
Compared with 2D maps [14,73,74,75], 3D maps can provide more information. By fusing the original measurement or small local maps generated from multiple robots to construct global maps at the same time, or matching the 3D maps constructed by heterogeneous robots, more and more abundant data are obtained, which is a new research direction of mapping.
For example [71], the point clouds of the two images acquired by the UAVs and UGVs respectively were first represented by a grid model, and each cell stored excess vegetation index information and ground height information. After that, the relative displacement and rotation parameters between the two grid maps were extracted using the data provided by GPS and Attitude and Heading Reference Systems (AHRS), and the maps were matched by these parameters. Then the correspondence of point clouds between the matched maps was calculated using the Large Displacement Dense Optical Flow (LDOF) system and a voting scheme was used to select a coherent correspondence dataset with high data similarity to infer the initial conversion relationship between the maps, and a non-rigid alignment algorithm was used to eliminate orientation errors. Finally, the input point clouds were brought in for a point-to-point alignment, and non-vegetation points were removed to obtain the 3D maps. The experimental results showed that the constructed maps matched well, but the time taken to construct each such image was quite long.
As the farming environment is often dynamic, learning specific farming information while shortening the time to build maps will help improve the efficiency of multi-robot operations, which should be a direction for further research. For example, simultaneous localization and composition (SLAM) allows the robot to get information about the image while localizing it, and this technique is currently used more often for indoor multi-robots, especially for catastrophic search and rescue where high responsiveness and mobility are required [76]. In the literature [56] it was mentioned that this technique was used on a ground robot for heterogeneous multi-robot in a greenhouse and that the robot was motion controlled using an Augmented Monte Carlo Localization (AMCL) methods. The experimental results showed that the UGV using SLAM takes less time to construct the map than the remotely operated UAV. And based on the map, the ground robot can complete collision-free autonomous movement between the initial and target points. It is obvious that SLAM technology has great potential for multi-robot mapping [77], and the application of multi-SLAM on agricultural multi-robots will be one of the future research directions.
3.3. Multi-Robot Task Allocation
Multi-robot task allocation (MRTA) provides evaluation indicators of a multi-robot system, a task set, and system performance and finds a suitable robot for each subtask to perform, bringing the benefit of a robot system to perform mostly task collection. Thus, the quality of the MRTA results directly affects the efficiency of the entire system and whether each robot of the system can maximize its capabilities [78].
Solve the MRTA problem [79] involves many aspects, such as the capability attributes of the system members, the structural attributes of the tasks, the robot coordination mechanism, and the strategy of task allocation [80]. This approach divides the assignment of agricultural multi-robots into centralized components based on the decision of the central controller in this part.
- Centralized Allocation
The centralized allocation means that the leading robot or control center of the system decomposes the global task and then sends the decomposed subtask to each robot according to the corresponding allocation method.
In 2012, reference [14] reported the work area division and task assignment of two UGVs by a person through remote monitoring based on a citrus orchard map. However, as the number of robots increased, manual task assignment stops when a robot hits a tree or fails to make a reliable turn. Some scholars [66] solved the manual assignment of tasks to multiple robots from the perspective of dividing maps, namely, as many sub-regions as there are robots needed. The edges of these sub-regions are generated from discrete fruit tree points and K-mean clustering points on the map boundary using an integer programming approach. However, this view is too ideal and does not consider the case where the number of robots is less or more than the total number of tasks.
A similar study on the number of robots over the number of tasks is multiple robots and a small number of refueling stations [81]. In this reference, an approximate arbitrary time algorithm based on the branch delimitation method was used to obtain the task sequence of multi-robot collaborative refueling. The limit value of the path length distance was first calculated and used as the upper and lower bounds of the algorithm nodes based on the rules for robots walking infield and the total time cost function of refueling and waiting in the queue. Then the optimal solution was obtained by deleting the sub-nodes whose lower limits were greater than or equal to the optimal upper limits during iteration. Simulation experiments showed that the optimal approximate solution on resource utilization can be found by this method, but it is difficult to apply the method to other aspects of agriculture, such as spraying.
A reinforcement learning-based method (Dyna-Q +) was used to find the optimal search path from the current point to the endpoint, that is, multiple robots randomly selected actions (front, back, left, right) and each action was recorded, and the optimal path was obtained by rewarding and punishing the actions selected by the robots according to the presence or absence of obstacles [62]. Then a weighted graph was used to represent the Gird model, including parameters such as the current position of the robot, the set of grids, and whether there is a path between the grids. The minimum cost time for the UAVs to fly at different speeds under the optimal path was calculated using Dijkstra’s algorithm, and finally, the search space was obtained based on the optimal path and time and allocated to multiple robots in proportion to the size of the space. A reinforcement learning [82] is a Markovian decision process where the basic idea is all about modeling or fitting a strategy using a function for more complex decision problems. However, the method requires a large number of samples and a long time when used.
Also, multiple robots working in an agricultural environment are often subject to resource-sharing conflicts. For predictable conflicts, relying only on a central task allocation approach to avoid conflicts, the adaptability of multiple robots is very limited, while adding a decision support system (DDS) to provide options for multi-robot collaboration, i.e., identifying problems and building or modifying decision models to avoid resource conflicts based on explicit goals. For example, as shown in Figure 9, in reference [58], the authors adopted a central entity (OptiVisor) to build a multi-robot seeding map based on inputs such as the location of static obstacles in a large field, the seeding method, the seed density, the location of the central controller, and the number of robots. Based on this map, the location and density of each seed are precisely located and the sowing task is assigned to multiple robots. When the robot finished the job, the path from the robot location to the Central Logistic Unit (CLU) was recorded and the robot was allowed to return to the CLU. Especially when one or more robots failed, the task of the failed robot was reassigned to the other robots of the failed robot, and the sowing path was updated for these replacement robots. OptiVisor could also stop a robot’s motion when a multi-robot collision is imminent, and define restricted motion areas for the faulty robot. However, this task assignment method was implemented in a simulation environment and needs to be further tested in real applications.
- Distributed allocation
In general, the distributed task allocation method has the feature that each robot does not have global knowledge of the task but calculates and plans individually according to the local information obtained by the sensor. The performance of the whole system not only is closely related to the individual but also depends on the combined effects of individuals [83].
It was reported in the literature [84] that when the number of tasks is more than the number of farm machines, the ant colony algorithm can be used to find a suitable task sequence for multiple robots. First, the distance between plots was calculated based on the specified coordinate information of each plot, and the relevant parameters of the ant colony algorithm were set. Then randomly generated the starting points of multiple parcels, according to the state transfer probability formula for path selection, and put the generated path parcels into the forbidden table. Finally, the path distance of the plots was calculated, and the pheromones on the path were continuously updated according to the set rules, and iterations were repeated until the optimal task planning for multiple robots was found. The method is mainly used in simulated farmland environments and has not yet seen the practical application.
In reference [85], researchers adopted Semantic MozartSpaces to describe a task allocation data model based on a resource description framework (RDF) and SPARQL (a query language and data acquisition protocol developed by the RDF) in task storage where the RDF was used to construct nested blank nodes and SPARQL was used for querying, updating and interactions of the entry. As shown in Figure 10a, a task was mapped to a nested blank node to generate a semantic tuple (entry) in the task-allocation model. The entry was stored in the task storage with an internal ID that concluded the URL (uniform resource locator). Then, the entry could be selected with a URL according to the relationship between the robot’s function and the task requirement. The results of the simulation experiment suggested that the execution time increases correspondingly with an increasing number of tasks, followed by a gradual decrease in production efficiency. It would be necessary to add new robots temporarily to ensure productivity, but the production cost would also increase, so the tasks need to be set in advance.
As shown in Figure 10b, researchers developed market and auction-based approaches for task subdivision and allocation based on the Rubinstein negotiation protocol [28]. The advantage of such a protocol was that the auctioneer robot had to split the task into subtasks during the negotiation. In each round of negotiation, each robot initially started proposing the largest possible task allocation for itself and decreased its offers based on a negotiation of cost functions (discount factors) at each round until the other party indicated acceptance. With distributed task assignment under this protocol, each robot comes with a DDS, and each robot can dynamically adjust to the actual situation to get a suitable task. However, the task allocation result was generally an approximate optimal solution because of the discount factors, such as the area of the mission area, the distance of the robot from the goal, the area beyond the target, and the overlapping work area.
Combined with the above task allocation technology, the research progress of agricultural multi-robots in task allocation technology in the past 10 years is summarized in Table 4.
It can be seen from Table 5 that the current task allocation methods for agricultural multi-robots are mainly centralized, and most of them are implemented in the simulation experiment.
The centralized task allocation mainly adopts the integer programming method, which describes the nature of the task allocation problem by establishing the objective function and constraints. Integral programming (IP) and mixed-integer programming (MIP) problems are an important branch in the field of operations research, which includes branch and bound method [89], cutting plane algorithm [90], graph theory method, etc [91]. The idea is to determine the transfer method and transfer relationship from one search point to another, and the result is a unique optimal solution, which is suitable for small scale; when the scale is extended, the computation is considerable and the computation time will be greatly increased [92]. Thus, the computation of the algorithm grows exponentially with increases in the number of tasks and robots. In general, it is often difficult to meet real-time requirements in task allocation issues.
Distributed task allocation mainly uses a method based on behavioral motivation and a market-based approach. The former applies to multi-robot systems with strong autonomy, but the method has low system allocation efficiency; the latter has a wide range of applications based on the resource optimization configuration idea of economics, but the difficulty of this research method is how to design the negotiation mechanism and reasonably determine the cost-income models of the task [44].
3.4. Path Planning
Path planning is the fundamental guarantee for multiple robots to accomplish tasks together. This technology refers to using the known static environment information, or the dynamic environment information obtained by the sensor, to autonomously plan a collision-free optimal path for each robot from a known starting point to a target point, which requires not only a single robot to avoid obstacle but also a plan to satisfy collision avoidance among multiple robots [93]. The path planning methods for single robots to avoid obstacles mainly include traditional methods, intelligent methods, and other methods. The traditional methods include the construction space method, V-Graphic (visibility graph), Voronoi diagram, grid method, A* algorithm, and artificial potential field [94]. Intelligent methods include the ant colony algorithm, particle swarm algorithm (PSO), reinforcement learning algorithm, immune algorithm, genetic algorithm (GA), neural network, and fuzzy logic algorithm [95]. Other methods include dynamic programming (DP) and optimal control algorithms. Collision avoidance strategies among multiple robots include the priority method, rate adjustment method, traffic management rule, and consultation method.
The path planning method can be divided into centralized path planning and distributed path planning according to the ability classification of path planning [96].
- Centralized path planning
The centralized path planning method uses a centralized control unit to plan the optimal path for swarm robots. This method can improve the ability of “close coordination and optimal coordination” among mobile robots [97]. However, it encounters other problems such as “dimensions”, “computational complexity”, and “non-deterministic polynomial problem (NP) difficulties” with the increase in the number of robots, task difficulty, and space complexity. In particular, the “NP difficulties” problem [98,99], in theory, has not yet received a simple or fast solution.
Connecting known path points into a line, which is simulated to a topological model, and letting multiple robots walk along their respective paths is one of the simplest and easiest path planning methods to implement. But, this method requires obtaining accurate known point information in advance, which is a large preliminary job and is not suitable for situations with a large number of robots or a large operating area. A method similar to the point-to-point method is the visible map method, which aims to reduce collisions. That is, the information of the edge projection points of each obstacle is obtained in advance, and the robot free walking path, path points that can be combined or disconnected are represented by edges and nodes, respectively. Then the starting node is connected to the target node, or the starting node is connected to the raised point of each obstacle edge until it reaches the target node, forming a multi-robot walking path. Finally, depending on the size of the robot, the path width is increased or decreased appropriately. Similarly, the workload of measuring points is larger and does not apply when there are more obstacles. Planning multi-robot paths on mapping is another method of global path planning. For example, in the grid method, the map was divided into multiple grid cells, and the paths were extended in eight directions with each grid cell as the center, and the path segments were formed by connecting the center of the grid vertically and diagonally with the centers of other grid cells. To get the globally optimal path in the grid, the A* algorithm was used to search for the path segment with the lowest travel cost, in which the cost of the free space cell was set to 0, the cost of the cell with obstacles was set to the maximum, and the travel cost was the sum of all grid cells on the travel path segment, and the globally optimal path was set when the sum was the smallest.
Since information about the farmland changes dynamically, multiple robots operating with precision need to re-plan to create multiple paths each time based on different information. This multi-robot path planning problem with time windows has also been solved as a multi-objective optimization problem. As shown in Figure 11, where a multi-robot system including two aerial robots and three ground robots was jointly developed in Spain and other countries [49]. This system adopted a centralized control unit to provide global path planning for the multiple robots on a grid map with weed information, which divides the sequence of operations for the multiple robots in advance. The No dominated Sorting Genetic Algorithm II (NSGA-II) algorithm [100,101] was then used to coordinate the relationship between the distance of the robot travel path, the number of turns, the number of robots, the amount of weed killer used, and the capacity function to obtain an approximate optimal solution between the time and money spent by the multiple robots and the cost of weed treatment. It is a type of genetic algorithm, which mainly focuses on the simulation of crossover, variation, and hereditary phenomena occurring during natural selection and genetic inheritance, incorporating the natural law of superiority and inferiority, and deriving the candidate solutions for each generation based on the results, and finally deriving the optimal solution from the derived candidate solutions. However, this method is more computationally intensive and the experimental results are not suitable for fields that are unstructured or d fields without a fixed column or row lengths.
- Distributed path planning
The distributed path planning method requires little calculation and is robust but exhibits low efficiency and can provide only a suboptimal solution [102,103]. In a fully known environment, it is necessary to consider each robot obstacle avoidance method and collision avoidance strategy among robots [104], that is, selecting a robot for path planning first, then broadcasting its path to other robots, and finally planning paths of other robots by themselves. However, this method is difficult to achieve [97] for large numbers of robots. In an unknown environment, the preferred method is to plan a path for each robot to avoid static obstacles based on neglecting the movement of other mobile robots in the environment and then using the multi-robot collision avoidance strategy to solve the conflict problem among mobile robots.
At present, there is little research on the distributed path planning of agricultural multi-robots. As shown in Figure 12, Bouzouita et al. [28] developed UAVs to monitor agricultural information in the vineyard. The path planning of the UAVs is based on the grid map and A*. On the map, the UAV path planning function is constructed according to constraint conditions, such as the number of UAV turns, the number of covered grid visits, and the time to complete the single partition. And using a heuristic algorithm like A*, the next best node to be expanded is obtained by partying the generation value of each node. Then, the breadth-first search (BFS) algorithm is used to find the local maximum of the function by the distance between the cells. The result is the path from any starting point in the environment to the target unit. The practice shows that the method can find the approximate optimal solution, reduce the possibility of repeated access to the same cell, and facilitate the avoidance of known obstacles. However, it needs to consider the local environmental conditions to find wide applicability.
Combining the above path planning techniques, we summarize the research progress of agricultural multi-robots in path planning methods over the past 10 years, as shown in Table 5.
It can be seen from Table 5 that agricultural mobile robots mainly use ground robots in farmland, and the number of robots is no more than four. The path planning method of multi-robots is mainly conducted in fully known conditions, and the grid method and V-Graphic under centralized planning are the methods most commonly used.
The collision avoidance strategy between agriculture robots usually does not incorporate changes in the path [110], and the obstacle avoidance strategy of a single robot comprises mainly speed adjustment and the priority principle. The path planning of multi-robot generally does not consider the presence of obstacles in agricultural production.
When the UGVs turn around at the headland, they need to consider the relationship between the minimum turning radius and the headspace. The general head-turning method has a bulb shape (as shown in Figure 11b), zigzag or U-shape, etc. The U-shape predominates in practice. As shown in Figure 13, the zigzag (forward-reverse-forward) is used in smaller spaces; in contrast, the U-shape turn (turn-straight- turn) is used in larger spaces. However, the difficulty of robot control is increased with the zigzag turning shape, and the task allocation and path planning of the multi-robot are prepared for U-shape turning in advance.
3.5. Formation Control
Multi-robot formation control technology means that multiple robots maintain a certain formation to the target while still adapting to environmental constraints such as obstacle blocking or spatial physical limitations [111]. This technology can improve the robustness of the multi-robot system, and the robots can complete the task with higher efficiency and shorter time [112]. At present, the formation control technology of agricultural multi-robots is divided into forming formation and formation control.
- Formation forming
As shown in Figure 14, the formation of agricultural multi-robots generally has five types: column, I-shape, linear, V-shape, and W-shape, and the circular nodes in the formation structure represent robots. Each robot is represented by RID, such as R1 and R2, and the black arrow indicates the direction of robot movement.
It is necessary to achieve the desired formation by determining the formation position reference point after determining the root formation. There are usually three reference points: center, neighbors, and leading robot; as shown in Figure 15, the position of each root node is represented by PID, such as P1 and P2, and the arrows indicate the relationship between robot dependence and information transfer.
- Formation control
From the perspective of a multi-robot system control framework, formation control is divided into two types: centralized control and distributed control. The former uses a centralized control unit to make decisions, optimize robot coordination, accommodate individual robot failures, and supervise the entire group of robots. The latter does not have a unified control unit, and a single robot makes decisions based on its local information [112,113].
At present, the method of centralized formation control of agricultural multi-robots includes the virtual structure, graph-theoretic approach [114], and model predictive control [115,116]. The method of distributing the formation control of agricultural multi-robots includes leader-follower [34,41,75,117] and the artificial potential field [113].
Guillet et al. in France [44] adopted the bidirectional control strategy based on the virtual structure method. As shown in Figure 16a, each robot of the whole queue is a fixed point on the virtual structure. In this structure, the queue also increases two virtual leaders’ interaction with the extreme robots and carries the desired global velocity for the whole fleet. The advantage of this method is simpler communication protocols and lower communication costs; however, the reaction of the robots is slower because of different acceleration performances.
Berman et al. in the USA adopted the graph theory approach in bee pollination [108]. When a beehive was opened, the swarm robot flew radially from a moving beacon at a constant speed. And once it encountered the edge of the graph, it flew eastward at a fixed speed. As the robot flies over the plant, it acquires at least one flower within its range through sensors and hovers over the flower with unit time probability to pollinate it and record the location of the pollinated flower, returning to the hive after pollination and starting the next flight until complete coverage of the whole field is achieved. However, this method takes a long time and the model used in the simulation is too idealized. Whether they can be used for practical production needs to be further explored.
Smith et al. in Korea adopted model predictive control (MPC) and nonlinear feedback control respectively in fish tracking (simulation) [117]. MPC is a finite-domain rolling optimal control strategy with three parts: model, prediction, and decision, sacrificing optimality to some extent [118]. The fish population location was first divided into discrete points, and the discrete points were clustered to get the vertices of fish population density, and the transition model was constructed by transforming the movement of the fish population into the movement between the vertices. The transition model and nonlinear feedback were used to obtain the transition matrix, and the underwater robot was guided to the vertices with high fish population density according to the transition matrix. The simulation results showed that the model-based control of the underwater robot could approach the nearest point, while the feedback control made the underwater robot approach the target point. However, in practice, the underwater robot movement speed is smaller than the fish population movement speed, and the method needs further improvement when applied in practice. The leader-follower method [48,105,117,119,120] is also another classic model and widely used in the formation control of agricultural multi-robots. Japan’s Zhang et al. [106] used the leader-follower method to control UGV formation. As shown in Figure 16b, the relative positional relationship between the leader and the follower is determined according to the lateral and longitudinal safety distances (l−l) between the robots first, and then the distances are dynamically adjusted with feedback linearization technology to assemble different formations. Based on the leader-follower model, Bai et al. in China also combined slide mode control with the harvester swarm [48]. The kinematic model of the farmland leader-follower harvester swarm was established first, and based on this model; the asymptotically stable path-following control law and the formation-keeping control law were designed by combining feedback linearization and sliding-mode control theory. The advantage of this leader-follower model is that the behavior of the fleets can be controlled through the determined trajectory of the leading robot. The method decouples the cooperative navigation control problem into lateral distance keeping control and longitudinal distance keeping control. The formation control is mainly accomplished by establishing the location and gesture of the following robot relative to the leading robot, such as (l−φ), (l−l) first, then obtaining the formation information through feedback linearization, and finally adjusting the formation according to the threshold value. The leader-follower test results show that the real paths of robots can achieve centimeter-level average error with the planned path based on the safe distance of the vehicle. But this method is only applicable for environments involving a single-tasking of agricultural production and a fixed site. The adaptability to the headland turns is not strong. The question of how to maintain robot formation in encountering static or dynamic obstacles is not considered. If the leading robot malfunctions, the formation of the fleets cannot be maintained. Once the leading robot fails, the multi-robot system is susceptible to deadlock, and the formation cannot be maintained. The “leader” replacement method was proposed [121] to overcome this shortcoming, but the method has not been applied to agricultural multi-robots.
Ju and Son in Korea adopted Ramadge-Wonham theory in supervisory control to solve the above deadlock problem [122]. Supervisory control is a feedback control theory for discrete-event systems, where the control goal is achieved by observing the occurrence of events or states and using allowable or prohibited controllable events. Finally, a time-driven system is combined with a low-level controller and an event-driven system with a high-level controller with the criterion of satisfying the behavior specification and maximizing the allowable events. Time-driven is used when there is no fault, and once the queue encounters a fault, the control outcome is selected based on event-driven. Simulation results demonstrate that the method can be used to control complex dynamic systems, but it has not been tested in practical applications.
The characteristics and formation process of the formation control methods are shown in Table 6.
From Table 6, it can be found that more complex or hybrid control methods are mostly used in simple or simulation environments, and the application in actual agricultural production is still dominated by the leader-follower method, and the research is also mainly focused on multiple machines traveling in a straight line in a fixed column. Further research should be conducted on how to continue driving, maintain the formation, or adjust the formation after multiple robots encounter obstacles.
3.6. Communication
Communication is the basis of information interaction and collaboration among multiple robots. In agricultural production, many factors affect the fine operation of agricultural robots, and to maintain coordination and cooperation among multiple robots and to gain a more comprehensive understanding of the environment in which multiple robots perform tasks, robots need to interact with each other through information to better perform a given task [29]. Balch and Arkin concluded that even a small amount of communication can improve the performance of multi-robot systems tremendously through experiments [125].
At present, the communication technology of agricultural multi-robots mainly involves three parts: multi-robot communication mode, communication network, and communication protocol.
- Communication mode
The multi-robot communication mode is divided into three categories from a macro perspective: explicit communication, implicit communication, and explicit and implicit communication, as shown in Figure 17. Explicit communication is an interactive mode through communication as a medium, requiring a clear communication protocol between interacting parties. This method is often used for concordant communication among robots, but it incurs fairly large costs. Implicit communication is the acquisition of the required information through the external environment and internal sensors without an explicit exchange of data, so some advanced coordination strategies cannot be used, which affects the capacity to perform certain complex tasks.
Since explicit communication and implicit communication have their advantages [120], explicit communication is used for the integrated control of robots in the upper layer, and implicit communication is used for integrated control of robots in the bottom level. Explicit communication means that the robot communicates directly or indirectly with other robots via wireless networks. For example, robot 1 sends a message to all robots in the communication range in broadcast communication, that is, without specifying a particular robot, robot 2, which does not need the message, receives the message. In implicit communication, the intermediary for inter-robot communication is often the surrounding environment. For example, the UAV can be informed about the farmland in advance and build a model of the farmland environment, and the ground robots operate on the ground based on this farmland model [122]. The combination of both communication modes can be used to develop their advantages, improve the flexibility to confront the various dynamic and unknown environments, and complete many complex tasks in agricultural production.
For the implicit communication of multiple robots, you can refer to Section 3.2 environment perceptions, here we focus on robot explicit communication techniques.
In literature [25], two aerial drones were equipped with GPS, visible and near-infrared spectral cameras, which took pictures of the farmland at a set series of ordered waypoints and uploaded them to the backend, which sends the processed information of weeds in the farmland to the ground robot. The ground robots were equipped with RTK-GPS, RGB camera, and LIDAR. RTK-GPS provided accurate heading for the ground robot, RGB camera detects weeds and crop rows, and LIDAR detects obstacles on the vehicle trajectory. While the ground robots were safely walking along their respective set paths, weeding operations start if the weeds detected by the cameras were the same as the weed information in the farmland. In this multi-robot system, the aerial drones and ground robots did not communicate directly but completed the cooperative operation through the interaction of environmental information.
- Wireless communication net of multiple robots
As shown in Figure 18, the agricultural multi-robots need to adjust their pose in real-time. Therefore, the data exchange of communication among multiple robots is mainly based on wireless communication technology in agricultural production. This technology mainly involves a wireless local area network (WLAN) and a wireless personal area network (WPAN), such as WI-FI, Bluetooth, ZigBee, and IRDA (infrared data association). Among them, WI-FI technology has been developed most rapidly in agricultural multi-robots.
- The wireless communication protocol of multiple robots
The wireless communication protocols are primarily used based on wireless communication standards and the unlicensed band. Taking the WLAN as an example, the IEEE 802.11 series standards and the 2.4 GHz or 5 GHz bands are used in this communication. The IEEE 802.15 series of transmission technology protocols are selected in WPAN.
Combined with the above communication technology, the research progress of agricultural multi-robots in communication in the past 10 years is summarized, as shown in Table 7.
In addition to the above wireless communication technologies, Albani et al. adopted a mobile ad hoc (peer-to-peer) network [128,129], which regarded the UAV as a communication node in the network and used three communication strategies (simple, flooding, geo-aware) to solve the communication problem of UAVs flying in the field. The simplest communication strategy is a single broadcast mode, that is, the source node sends information to the nearest node. Flooding constitutes a multi broadcast mode, that is, the source node sends information to multiple agents. Geo-aware employs a source node with the highest utilization rate, and this node sends the messages. All three communication strategies ignore communication errors and focus on the impact of the communication range and protocol on work efficiency. The simulation results show that the effective information of weed monitoring can be transmitted with a minimum number of UAVs under the geo-aware approach. However, the communication strategy discards new information obtained by UAVs of the distributed architecture, and messages cannot be effectively transmitted with a wide range of communication (such as over wide areas of farmland). Agricultural multi-robots working in the farmland often encounter signal occlusion, atypical weather, etc.
Large agricultural multi-robots working in agricultural fields rarely encounter problems such as signal occlusion and atypical weather. However, in other agricultural products, such as greenhouse and orchard, when the size of the multi-robot is smaller than the height of the crop, its communication signal strength is extremely attenuated by factors such as crop planting, growth characteristics, planting scale, and weather (natural wind and rain). Previous references [126,127,130] showed that the test results of the WI-FI communication system of agricultural multi-robots suffered from WI-FI signal intensity attenuation largely because of the reflection and scattering effects of crops, and the effective communication distance was less than 50 m (far less than the theoretical communication distance of 300 m) in mature wheat fields, cornfields, and peach gardens. Therefore, it is a future research direction to select suitable multi-robot communication technology according to the characteristics of crops and to carry out research on multi-robot communication patterns based on crop shape characteristics.
4. Discussion In general, in the past 10 years, the synergistic technologies of agricultural multi-robots have achieved some research results, and multi-robot collaborative operations have been realized in specific agricultural scenarios. But with the increasing demand for agricultural operations, the following challenges in the application of multi-robots in agriculture still exist to be solved:
- Flexible agricultural multi-robot system architecture
Multi-robot architecture is the basis for collaborative operations of multi-robot systems. In the last decade, agricultural multi-robot systems have mainly focused on centralized or distributed architectures to accomplish collaborative operations under pre-defined conditions. Both architectures have their advantages and disadvantages, but as the number of robots increases (such as multiple aerial robots cooperating with multiple ground robots) and new agricultural operational needs increase (such as sampling in marine environments [131], cargo handling in hilly mountainous areas, pest control in orchards, etc.), it is clear that the scalability and flexibility of multi-robot systems relying on only one architecture are limited. The advantages of centralized and distributed architectures are combined to form a hybrid architecture, or the application architecture is dynamically selected according to the task attributes, which can overcome the low performance caused by the self-centeredness in the distributed architecture and reduce the lack of control flexibility in the centralized architecture.
- Fast and precise environmental perception
- Reasonable task assignment in real-time
The task assignment is related to the multi-robot coordination and collaboration mechanism, and the simple zoning assignment of robots cannot adapt to the dynamically changing operational tasks. Also, the number of robots, operating time, and cost of robot operations need to be dynamically adjusted to the operating task. Even for the same type of robots, items such as fuel or electricity, fertilizers, herbicides, and pharmaceuticals can change differently depending on the target of the operation. It is impossible to obtain fast and reasonable response results based on the changes of these uncertainties even depending on a priori knowledge. How to enable multi-robots to timely self-adjust to dynamic task changes and obtain reasonable operational tasks or operational task sequences through real-time interaction with dynamic environments is an urgent problem for multi-robot task assignments in agriculture.
- Dynamic and reliable path planning
The path of agricultural multi-robot operations is designed to accomplish dynamic operational tasks, and the robot’s travel rules are usually fixed. The global path of multi-robot offline planning only considers fixed travel rules, such as the point-to-point method and image method, which can avoid static obstacles smoothly, but cannot be extended to be applied to similar agricultural scenarios. In particular, if the dynamics of the agricultural environment change rapidly (e.g., weeds are growing in the field after the rainy season) and the agricultural information is not fully known (e.g., the constructed mapping usually does not contain dynamic obstacles), fixed path planning cannot meet the needs of complex tasks (e.g., weeds are not on the planned path). Therefore, how to perform reliable path planning for multiple robots based on operational tasks with distinct temporal characteristics is a problem that needs to be solved for multi-robot path planning in agriculture.
- Flexible and robust formation control
Multi-robot formation control currently focuses mostly on robot swarms walking steadily along a straight line in a fixed formation. However, when a multi-robot system encounters unexpected events, such as robot failure, communication failure, or stopping travel due to dynamic obstacles, how to mitigate the impact on other robots, respond quickly, and adjust the robot formation shape to continue the task is a concern for agricultural multi-robot formations. Although some studies have shown that multiple robots can be selectively controlled based on time or event drivers, or by replacing the “leader” in the queue, none have been applied in real production.
- Communication system based on plant characteristics
Communication is the basis of multi-robot collaboration in agriculture, whether it is multi-robot positioning, collaborative control, or remote supervision, communication is indispensable. The agricultural environment lacks communication infrastructure construction, and most of them directly used industrial communication systems do not consider the relationship between outdoor plant growth and communication signals, and their communication range and signals will be attenuated to different degrees in the agricultural environment. Therefore, the construction of a communication system adapted to the agricultural multi-robot operating environment is a problem that needs to be solved for multi-robot communication. 5. Conclusions Given the current challenges in agricultural multi-robot research, this paper points out future research directions in six areas to enhance the application of agricultural multi-robots in practice. Firstly, to build a flexible and changeable agricultural multi-robot system architecture based on hybrid architecture so that the multi-robot system has good environmental adaptability and robustness. Secondly, to develop sensor information fusion technology among agricultural multi-robots based on mutual positioning methods to improve the positioning accuracy of multi-robots in agricultural environments without GPS. Meanwhile, SLAM technology for agricultural multi-robots is studied to rapidly build environment models to adapt to the dynamically changing agricultural information. Third, to introduce deep learning mechanisms in agricultural multi-robot task assignment enables multi-robots to self-identify, evaluate, compare, remember and adjust during their interaction with the environment, and adjust the way they interact with other individuals according to specific tasks so that the group as a whole is equipped with the ability to complete multiple types of tasks. Fourth, dynamic planning of multi-robot paths based on a combination of centralized and distributed path planning methods enables multi-robot systems to adapt to real-time changing operational tasks and avoid obstacles and other robots promptly. Fifth, to modify the reference points of multi-robot formations flexibly according to changing events, adjust the distance and direction between formation members, reduce the impact on other mobile robots, and complete operational tasks. Sixth, to study the relationship between plant growth characteristics and communication system, establish a communication signal attenuation model, and design an agricultural multi-robot communication protocol based on this model to build a communication system. In summary, the multiple robot system represents the future of robot development. The synergistic technologies for the research of agricultural multi-robots have a great value and bright prospects but are also extremely challenging. Therefore, it requires participation by researchers to combine the former research results, recognize the developing trends, and use practicality as the ultimate goal to drive forward the coordination technology of agricultural multi-robots.
| Co-Location Methods | Active Positioning | Passive Positioning | Interactive Positioning [41] |
|---|---|---|---|
| Data measurement object | Robot itself | Neighbors | The robot itself; Neighbors |
| Observation reference object | Neighbors | Robot itself | The robot itself; Neighbors |
| Measuring object | Single | Single | Mutual observation |
| Measurement data | The distance and angle of the robot itself relative to the reference object | ||
| Data exchange | None | Exists | Exists |
| Data Fusion | None | None | Exists |
| Advantages | Little computation | Information interaction | Algorithm of high precision and robustness |
| Disadvantages | Affected by the environment | Affected by the environment | A huge amount of information |
| Lower algorithm accuracy and robustness | Lower algorithm accuracy and robustness | Large complexity of the algorithm | |
| No information interaction | |||
| Applications | Drove in orchards [38] | Drove on farmland [42] | None |
| Drove on sloping land (11°) [43] | |||
| Plowing [44] | |||
| Lawn [45] | |||
| Drove in fields [46] | |||
| Plowed [47] | |||
| Harvested farmland [48] | |||
| Fusion Level | Data Level | Feature Level | Decision Level |
|---|---|---|---|
| Processing level | Bottom layer | Middle layer | High level |
| Sensor types | Homogeneous | Heterogeneous | Homogeneous/Heterogeneous |
| Degree of dependence on the sensor | Maximum | Medium | Minimum |
| Advantages | Less information loss; | Takes into account the advantages of data and decision; | Less calculation; |
| Good fault tolerance; | |||
| Good anti-interference; | |||
| High precision; | Low algorithm difficulty; | ||
| Fusion is more convenient; | A small amount of communication data; | ||
| Wide range of applications; | Good real-time performance; | ||
| Disadvantages | A large amount of calculation; | Global decision; | A large amount of information loss; |
| Poor fault tolerance; | |||
| Poor anti-interference; | |||
| The algorithm is complex; | Strict requirements for sensor pretreatment; | Low precision; | |
| A large amount of communication data; | Need to balance constraints (speed, reliability, cost, etc.); | ||
| Poor real-time performance; | |||
| Fusion algorithm | Wavelet transform; | Kalman filter; | Bayesian theory (Particle filter); |
| Artificial neural networks; | |||
| Weighted average; | |||
| Production rules, etc. | Extended Kalman filter, etc; | The statistical decision, etc.; | |
| Applications | None | Drove in a greenhouse [56] | Seeded fields [58] |
| Mapping Model | Grid Model | Geometric Feature Model | Topological Model |
|---|---|---|---|
| Algorithm | Bressenham’s linear algorithm, Bayesian; D-S theory, etc. | Voronoi Diagram; | Baum-Welch, etc. |
| Extended Kalman filter; | |||
| Particle filter, etc; | |||
| Scope of application | Suitable for processing systems such as laser radar and sonar priority ranging; | Local area modeling; | Large-scale, simple environment; |
| Specific environment; | |||
| Advantages | Simple and intuitive, easy to create and maintain; | More compactness; | Model is simple and easy to change, and motion planning is fast; |
| Good spatial consistency; | Modeling in local areas can achieve higher accuracy; | Better robustness against position errors; | |
| Better robustness; | Conducive to the estimation of the position of the robot and the identification of the target; | A high degree of abstraction, easy to store; | |
| Disadvantages | A large amount of calculation; | Difficult to address and extract irregular geometric features; | Less information; |
| Takes up substantial storage space; | Image data are processed cumulatively; | The topology network establishment process is complicated; | |
| Long storage time; | |||
| Application | Harvested muddy moss in farmland [13] | Plowed [47] | None |
| Monitored vineyards [28] | Drove in a greenhouse [56] | ||
| Environmental monitored [49,50] | Harvested farmland [48] | ||
| Collected farmland information [71] | Precision irrigation in the vineyards [72] | ||
| Sprayed in the Orchard [66] |
| Classification | Centralized Distribution | Distributed Distribution |
|---|---|---|
| Advantages | Algorithm design is simpler; | Communication is more dispersed, which can avoid the situation that affects the efficiency of the algorithm due to communication congestion; |
| It has the potential to generate globally optimal solution; | Good real-time performance; | |
| Disadvantages | The communication is concentrated and susceptible to congestion, affecting the algorithm efficiency; | The solution result can easily fall into a local optimum; |
| Poor real-time performance; | Consultation increases the communication burden of the system; | |
| Scope of the application | The task is known and determined in a static environment; | The task is unknown and uncertainly determined in the dynamic environment; |
| The environment model is fully known; | The environmental model is partially known or unknown; | |
| The scale of this system is relatively small; | The scale of this system is medium or large | |
| Allocation method | Mixed integer programming (branch-and-bound); | A method based on behavioral motivation; |
| Heuristic search algorithm (reinforcement learning); | A market-based approach; | |
| Group intelligence algorithm (greedy algorithm, ant colony algorithm [84]); | ||
| Application | Sprayed and weeding in an orchard (manned) [14]; | Weeded farmland [85]; |
| Seeded in a field [58]; | Agricultural monitored in vineyards [28]; | |
| Sprayed in the Orchard [66] | Harvested fields (simulation) [88]; | |
| Refill scheduling [78] |
| Planning Method | Environmental Circumstances | Head-Turning Mode | Coordination Strategy | Obstacle | Application |
|---|---|---|---|---|---|
| Point-to-point planner | Fully known | Right-angle turn; | None | None | Drove in farmland [42] |
| Plowed [105,106] | |||||
| Seeded in farmland (simulation) [107] | |||||
| Visibility graph (V-Graphic) | Fully known | Zigzag; | Priority; | Vehicle | Harvested muddy moss in the farmland [13]; |
| U-shape; | Rate adjustment; | ||||
| Right-angle turn | Bee pollination (simulation) [108]; | ||||
| Grid method | Partially known | Zigzag; | Others | Pole | Seeded in a field [58] |
| Fully known | U-shape; | Rate adjustment | Human | Harvested muddy moss in the farmland [13]; | |
| Vehicle | |||||
| A * algorithm 1 | Partially known | None | Others | None | Monitored in a vineyard [28] |
| Ballistic method | Unknown | None | Others | None | Weeded in a rice field [109] |
| Breadth-first search algorithm (BFS) | Partially known | None | Others | None | Monitored in a vineyard [28] |
| Genetic algorithm (GA) | Partially known | U-shape | Rate adjustment | None | Weeded in farmland [101] |
| The light bulb shape | |||||
| Others | Fully known | Right-angle turn; | Rate adjustment | None | Drove in a greenhouse [56] |
1 The A * algorithm is a direct search method for solving the shortest path in a road network.
| Formation Control Method | Advantages | Disadvantages | Steps | Application |
|---|---|---|---|---|
| Bidirectional Weighted Constraints Approach | Simplified description and assignment of tasks; | Difficult to handle fault tolerance; | Defining the dynamic model of the virtual structure; | Plowed [44]; |
| High formation control accuracy; | Poor reliability; | Transforming the overall movement of the structure into the motion of the robot; | ||
| No complicated communication protocols, low communication costs; | Low mobility limits range of motion; | Obtain the tracking control law of each robot; | ||
| Graph theory approach | Suitable for large-scale robot formation; | The implementation is more complicated, mainly limited to simulation research; | Define the formation right map G = (set of points V, edge E, weight W); | Bee pollination (simulation) [108]; |
| Easy addition and deletion of robot nodes; | Specify the ideal distance of each edge in the formation; | |||
| Easy to change between different formations; | Seeded in a field (12 simulation robots) [107] | |||
| Artificial potential field | Little calculation; | The design of the potential field function is difficult; | Design artificial potential field (environment and constraints between robots in the formation); | Precision irrigation in the vineyards [72] |
| Easy to implement real-time control; | Problem with local extreme points; | Establishing a potential field function; | ||
| Easy to handle collision avoidance problems in obstacle spaces; | ||||
| Model predictive control (MPC) | Has a strong theoretical foundation; | A large amount of calculation, mostly used for simulation; | Real-time planning formation retains the reference center and target control amount of each robot; | Tracked fish (simulation) [117] |
| Adds multiple constraints in the control process, and optimizes the control sequence by online scrolling optimization combined with feedback correction of real-time information; | Building a linear programming model with multiple constraints; | Harvested farmland [48]; | ||
| Leader-follower | Simplified system control; | The ability to adapt to a dynamic environment is not strong. If the leader fails, the entire system crashes; | Identify leading robot and formations; | Collaborative air-ground surveillance [122] |
| A follower follows the leader | Drove in a field [46]; | |||
| Control the spacing between the leading robot and the following robot–angle (l−φ) or lateral spacing-longitudinal spacing (l−l ); | Lawn [119]; | |||
| Plowed [106]; |
| Communication Technology | WI-FI | Bluetooth | ZigBee |
|---|---|---|---|
| Transmission distance [m] | 10~300 | 10~100 | 10~75 |
| Theoretical transmission speed [bps] | 54~300 M | 100 k | 10 k |
| Wireless communication network | Wireless local area network (WLAN) | Wireless personal area network (WPAN) | WPAN |
| Working frequency [Hz] | 2.4~5.4 G | 2.4 G | 2.4 G (global); 868 M (Europe); 915 M (United States); |
| Communication protocol | IEEE 802.11 a; 1 | IEEE 802.15 | IEEE 802.15.4 |
| IEEE 802.11 b 2; | |||
| IEEE 802.11 g 3; | |||
| IEEE 802.11 p 4; | |||
| Advantage | Fast transfer speed; | High transmission rate; | Low power consumption; |
| Low cost; | |||
| Strong anti-interference ability; | Large transmission range; | ||
| Long effective distance; | Flexible networking; | Good scalability | |
| Reliable connection | Low power consumption; | Strong anti-interference ability; | |
| Wide coverage; | Small volume; | Good security; | |
| Disadvantages | High power consumption; | Slow; | Low transmission rate; |
| Expensive; | Short distance; | Short distance; | |
| Weak networking ability | The mobile phone cannot communicate directly; | ||
| Protocol coding complexity; | Poor security and confidentiality; | ||
| Application | Plowed in wheat fields or orchards [126,127]; | Plowed [47,105,106]; | Monitored vineyards [28]; |
| Plowed [44,48]; | Monitored and weeded farmland [49]; | ||
| Plowed [46]; |
1 IEEE802.11a standard, operating in the 5 GHz band, has a data transmission rate of 54 Mb/s. 2 IEEE802.11b standard, operating in the 2.4 GHz band, has a data transmission rate of 11 Mb/s, and is not compatible with IEEE 802.11 a. 3 IEEE802.11g is a standard that increases the transmission speed of 802.11b from 11 Mb/s to 54 Mb/s. 4 IEEE 802.11p standard is a communication protocol expanded by IEEE 802.11 standard, which is mainly used in wireless communication of automotive electronics.
Author Contributions
Writing-review and editing, W.M.; supervision, H.L., Z.L., F.Y. and M.W.; project administration, F.Y. All authors have read and agreed to the published version of the manuscript.
Funding
This research received funding from the Major Science and Technology Project of Shaanxi Province of China (Program No. 2020zdzx03-04-01).
Institutional Review Board Statement
The study in the paper did not involve humans or animals. The project supporting the research of this paper is to study agricultural machinery and equipment for apple orchards, and the research does not involve humans or animals.
Informed Consent Statement
The study in the paper did not involve humans.
Data Availability Statement
This paper is a scientific review paper that provides a detailed analysis and summary of agricultural multi-robots, independent of the data.
Acknowledgments
Authors thank the funding received from the Major Science and Technology Project of Shaanxi Province of China (Program No. 2020zdzx03-04-01). We also thank the critical comments and suggestions from the anonymous reviewers for improving the manuscript.
Conflicts of Interest
The authors declare no conflict of interest.
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer
© 2021. This work is licensed under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Abstract
Multi-robots have shown good application prospects in agricultural production. Studying the synergistic technologies of agricultural multi-robots can not only improve the efficiency of the overall robot system and meet the needs of precision farming but also solve the problems of decreasing effective labor supply and increasing labor costs in agriculture. Therefore, starting from the point of view of an agricultural multiple robot system architectures, this paper reviews the representative research results of five synergistic technologies of agricultural multi-robots in recent years, namely, environment perception, task allocation, path planning, formation control, and communication, and summarizes the technological progress and development characteristics of these five technologies. Finally, because of these development characteristics, it is shown that the trends and research focus for agricultural multi-robots are to optimize the existing technologies and apply them to a variety of agricultural multi-robots, such as building a hybrid architecture of multi-robot systems, SLAM (simultaneous localization and mapping), cooperation learning of robots, hybrid path planning and formation reconstruction. While synergistic technologies of agricultural multi-robots are extremely challenging in production, in combination with previous research results for real agricultural multi-robots and social development demand, we conclude that it is realistic to expect automated multi-robot systems in the future.
You have requested "on-the-fly" machine translation of selected content from our databases. This functionality is provided solely for your convenience and is in no way intended to replace human translation. Show full disclaimer
Neither ProQuest nor its licensors make any representations or warranties with respect to the translations. The translations are automatically generated "AS IS" and "AS AVAILABLE" and are not retained in our systems. PROQUEST AND ITS LICENSORS SPECIFICALLY DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING WITHOUT LIMITATION, ANY WARRANTIES FOR AVAILABILITY, ACCURACY, TIMELINESS, COMPLETENESS, NON-INFRINGMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Your use of the translations is subject to all use restrictions contained in your Electronic Products License Agreement and by using the translation functionality you agree to forgo any and all claims against ProQuest or its licensors for your use of the translation functionality and any output derived there from. Hide full disclaimer