A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

Abstract

Wireless communication networks have been witnessing unprecedented demand due to the increasing number of connected devices and emerging bandwidth-hungry applications. Although there are many competent technologies for capacity enhancement purposes, such as millimeter wave communications and network densification, there is still room and need for further capacity enhancement in wireless communication networks, especially for the cases of unusual people gatherings, such as sport competitions, musical concerts, etc. Unmanned aerial vehicles (UAVs) have been identified as one of the promising options to enhance capacity due to their easy implementation, pop-up fashion operation, and cost-effective nature. The main idea is to deploy base stations on UAVs and operate them as flying base stations, thereby bringing additional capacity where it is needed. However, UAVs mostly have limited energy storage, hence, their energy consumption must be optimized to increase flight time. In this survey, we investigate different energy optimization techniques with a top-level classification in terms of the optimization algorithm employed—conventional and machine learning (ML). Such classification helps understand the state-of-the-art and the current trend in terms of methodology. In this regard, various optimization techniques are identified from the related literature, and they are presented under the above-mentioned classes of employed optimization methods. In addition, for the purpose of completeness, we include a brief tutorial on the optimization methods and power supply and charging mechanisms of UAVs. Moreover, novel concepts, such as reflective intelligent surfaces and landing spot optimization, are also covered to capture the latest trends in the literature.

1. Introduction

The fifth generation of mobile communication networks (5G) is no longer a future technology; it has become a reality as we have been witnessing its initial deployments around the globe. It has come with some rigorous requirements as well as promising scenarios and great expectations. In fact, the expectations mainly originate from those requirements; for example, 5G New Radio (5G NR) includes ultra-reliable low-latency communications (URLLC) scenario (enhanced mobile broadband (eMBB) and massive machine-type communications (mMTC) are the other two scenarios mentioned in 5G NR) with stringent reliability and delay requirements, and it is expected to enable various use-cases, such as remote surgery, industrial automation, etc. [1]. Therefore, the expectations of the previously mentioned spectacular technologies are based on the requirements, which determine the level of advancement in mobile communications technology. Moreover, the expectations also make the requirements stiffer, thus, there is a loop around the expectations (demands) and the toughness in the requirements (advancement in the technology).

In addition to the requirements, scenarios, and potential use-cases, there is another challenge that 5G networks are projected to face: Internet of things (IoT) networks. Billions of IoT devices will be connected to the Internet over the next few years, and a considerable amount of them will be cellular IoT with a significant portion of broadband IoT [2,3]. There is an important takeaway from this: IoT not only dramatically increases the number of connected devices but also significantly surges the bandwidth demand. Given that many bandwidth-hungry applications, including metaverse technology, 4K video streaming, tactile Internet, etc., are already tied together with 5G networks, the extra bandwidth demand coming from broadband IoT will put the existing challenges to a higher level. However, 5G already has some tools to alleviate such pressure in terms of capacity enhancement, such as millimeter wave (mmWave) communications, network densification, and massive multiple-input multiple-output (MIMO) technology, to name a few.

Since 5G is a reality, studies on the sixth generation of mobile communication networks (6G) have already started and moved quite forward [4,5,6,7]. Similar to the sharp advancement from the legacy networks to 5G, 6G is expected to be a paradigm shift in terms of network approaches, as mentioned in a recent report in [8]. More specifically, 6G is envisioned to shift the focus from conventional requirements including delay, data rate, reliability to global coverage, CO₂ emission, spectral efficiency, etc. [8]. However, this does not mean that 6G will not improve the data rate, latency, and reliability of the existing networks, because it is expected that 6G will enhance the aforementioned conventional requirements significantly [4], which indeed makes it more challenging for 6G networks. That is why, for instance, the terahertz (THz) spectrum is considered for 6G in order to open up more bandwidth for capacity enhancement and increased data rate purposes [4,5,6,7].

The picture that we have been trying to frame so far is that there is and will be a need for capacity enhancement in the current and upcoming generations of mobile communication networks. As mentioned earlier in this section, although each generation involves various concepts to make them immune to the rise in capacity demand, there is always a need for higher capacity provision. Furthermore, during scenarios involving unusual gathering of people, such as music concerts, sports competitions, fairs, etc., which only take place for short durations over long periods of time, a crowd cluster around a hotspot (i.e., the location of the event), increasing the level of challenge for the existing cellular network infrastructures, which are typically not designed for such rare events. If not for special occasions, in a usual scenario, there are spatio-temporal changes in the traffic loads of base stations (BSs), so that, for a particular BS, the traffic load changes dynamically over the time of a day, and days of a week [9,10,11,12]. This poses an interesting phenomenon that BSs do not always operate at full load; rather, the peak times of a day, for example, are limited to a certain period.

In this regard, a pop-up solution would be required for such kinds of scenarios, and unmanned aerial vehicle (UAV)-assisted wireless networking has been introduced as a viable option for pop-up networking [13]. The idea here is that small cell (SC) BSs are deployed on UAVs (called UAV-BSs), and they are sent to the hotspots for capacity enhancement purposes as long as there is a need, providing pop-up connectivity. With that, not only the users are satisfied as they secure connectivity, but also the mobile network operators are relaxed as no fixed infrastructure deployment is necessary.

In addition to the capacity enhancement purposes, UAV-BSs can also be used in emergency scenarios, including earthquakes, floods, etc., where the fixed infrastructure is either fully or partially damaged and down [14]. Since the end-to-end connectivity cannot be provided in such scenarios, UAV-BSs come as helping hands, such that they are sent to affected regions and operate either as standalone BSs in place of the damaged ones or as a relay BS that relays the data to the nearest healthy node of the communication system. This is an important vertical of UAV-assisted wireless networking, as in such scenarios, the transmitted message would be critical and can save the lives of people.

To this end, some key benefits of UAV-assisted wireless networking can be summarized as follows:

• Ubiquitous connectivity can be provided to the users as UAV-BSs are mobile and are capable of tracking the users;

• Pop-up scenarios are well managed in terms of connectivity;

• The business of mobile network operators becomes more sustainable as their capital expenditure (CAPEX) and operational expenditure (OPEX) are reduced because they do not need to deploy new fixed BSs since the same UAV-BSs can be reused in various occasions;

• Emergency scenarios can be managed more efficiently, as the UAV-BSs provide a good amount of flexibility.

However, despite its myriad of benefits, there are certain design challenges for UAV-assisted networking, including energy efficiency (EE), trajectory planning, positioning, resource management, privacy, regulations, etc. [13,15]. Of these, EE has a particular place as UAV-BSs are mostly battery-operated, therefore, their energy storage is limited. As a result, energy consumption becomes critical in order to: (1) prolong the flight time of UAV-BSs; (2) minimize CO₂ emission; (3) reduce CAPEX and OPEX of mobile network operators by making them require less UAV-BSs (since each can serve more) and reducing the energy costs. Each of these items is quite important during the design process of the network. Prolonging the flight time, for example, is not only instrumental in reducing the costs of mobile network operators but it is also effective in keeping the system fully operational. In an extreme case where there is very high demand, if a mobile network operator does not possesses sufficient number of UAV-BSs to replace one another once their batteries die, then there will be coverage holes and some users will be out of coverage, making the system partially down. On the other hand, if the EE is in place, then the flight time of each UAV-BS increases and that minimizes the time without connectivity for some users.

This suggests that energy optimization should always be in the equation when it comes to UAV-assisted networking, meaning that even the earlier-mentioned design challenges should also be coupled with EE through multi-objective optimization models. Positioning, for instance, can be done not only to increase the number of connected users or to maximize the quality-of-service (QoS) but also to minimize the energy consumption. In other words, the optimum position of a UAV-BS can be determined in such a way that it can cover more users while consuming less power. The main takeaway here is that the energy consumption of UAV-BSs requires special attention in order to make the concept more sustainable and feasible [16,17]. A summary of energy optimization perspectives and approaches along with their brief descriptions are given in Figure 1.

There is an avalanche of studies in the literature about the energy optimization for UAV-assisted wireless networking [18,19]. As the nature of each problem is quite diverse, the studies in the literature employ various types of optimization techniques, including conventional (e.g., heuristics, game theory, etc.) and recent (e.g., machine learning (ML)) ones. Even though the conventional techniques are relatively stable and robust, we have witnessed a shift towards ML-based solutions, not only in energy optimization but also in UAV-assisted networking in general [16]. This mainly originated from the fact that ML techniques offer additional benefits, such as being adaptable and dynamic, which are vital features for wireless communication networks whose environments rapidly change. On the other hand, error is an inevitable outcome of ML techniques [20], and error rate and how much error can be tolerated are important design challenges. Moreover, the availability, quality, and sufficiency of the data to be employed in training is a big issue for ML-based solutions [21,22], overshadowing the above-mentioned advantages of ML. Therefore, it is obvious that each method—either conventional optimization theory or ML counterpart—has certain pros and cons, and it is not straightforward to point to one method as superior to another; instead, they should be employed according to the conditions and requirements.

Apart from the optimization method being employed, there are also various and distinct perspectives to be focused on in terms of energy optimization for UAV-assisted wireless networking. We can observe some well-studied concepts, including (1) positioning where the optimum three-dimensional positions of UAV-BSs are determined [23,24]; (2) trajectory planning where the paths of UAV-BSs are designed [25,26]; (3) resource management [27,28]; (4) flight and transmission scheduling [29,30]. In addition to these techniques, there is also a novel technique that has recently attracted considerable attention, namely, landing spot optimization, where UAV land on designated platforms to minimize propulsion energy due to flying or hovering, which is the most significant source of energy consumption [31,32].

In this survey, we first provide an overview of UAVs with top-level taxonomy in terms of their wing types (i.e., fixed and rotary) followed by an in-depth analysis of the power supply and charging mechanisms of UAVs. Considering the discussion on EE so far, the power supply and charging system are at the heart of the UAV-assisted wireless networking. Then, we thoroughly investigate the role of UAVs in wireless communication networks by presenting various use-cases, including backhauling, load balancing, capacity enhancement, etc. With this discussion, we aim to reveal how broad the application range of UAVs in wireless communications is and that they will be at the core of 5G and 6G networks. The types of UAV-BS deployments are also reviewed in this survey to distinguish between standalone UAV deployment and UAV-BS deployment with fixed BSs. In the former, there is no existing cellular network infrastructure available, while in the latter, UAV-BSs assist the existing cellular network. We then investigate the energy optimization phenomena in UAV-assisted wireless networking to showcase different types of energies to optimize, such as propulsion energy and communication energy. This discussion is important to understand the main energy consumers in a UAV-BS, which specifies the energy optimization objectives accordingly.

As we mentioned earlier, there is a mountain of diverse optimization techniques available in the literature, and thus, these techniques are surveyed here under the top-level taxonomy of conventional techniques (e.g., heuristics) and ML techniques. The idea here is to capture general frameworks of those algorithms to better understand the state of the art in the literature. Then, energy optimization techniques—as the primary focus of this survey—are extensively surveyed under three-layer categorization: (1) deployment type; (2) energy optimization techniques (e.g., positioning, trajectory planning, etc.); and (3) the optimization method (e.g., conventional and ML). Under the second layer (e.g., the optimization techniques), we included the above-mentioned landing spot optimization concept. It is better to highlight this because it is quite novel and one of the most important contributions of this survey since, to the best of our knowledge, the landing spot concepts has never been covered in any survey work so far. The three-layer categorization is performed in order to provide a better understanding with such a detailed mapping.

After that, we also discuss some enabling technologies for energy-efficient UAV-assisted wireless networking. In that, we cover reconfigurable intelligent surfaces (RIS), mobile edge and cloud computing, network slicing, and cooperative communications, which are quite timely and promising technologies. This discussions aims to reveal such promising and emerging technologies and how they can be used in favor of EE. Moreover, within this discussion, we also present the energy harvesting approach, which also has the potential of boosting the EE in UAV-assisted networking. Even though it has been quite an extensive discussion so far with the inclusion of many perspectives, there is still room for improvement in order to make the whole concept much more reliable, sustainable, and feasible. We captured such improvement opportunities and various design challenges, such as security, data availability, regulations, etc., as well within the scope of this survey.

1.1. Related Works

Since UAV-assisted wireless networking has attracted considerable attention due to the advantages mentioned above, in addition to the research works—which will be detailed later in this survey—there are also numerous survey papers available in the literature. In this section, we will present them and make a comprehensive comparison in terms of their content and focus. It is worth noting that, among many, we carefully select the ones that are most relevant and recent in order to avoid diverting the scope of the discussion and keep this survey paper up to date with the latest literature.

The authors in [33] investigated how UAVs can be utilized in public safety communications, and they took the subject from the EE perspective. After making an extensive classification of UAVs in terms of application, altitude, and network, they proposed a UAV network architecture for public safety. Then, they highlighted the critical need for EE in UAVs, followed by various energy optimization techniques and optimization methodologies. The authors did not go into much detail about the optimization algorithms and how they work, but they briefly discussed various methods. A seminal paper in [13] presents a tutorial in UAV-assisted wireless networking. In particular, UAVs were first classified in terms of their wing types (e.g., fixed and rotary) and altitude (e.g., low and high). Then, UAV assistance in wireless networks was thoroughly discussed by considering their roles in the next generations of wireless networks under different scenarios, including UAVs as flying BSs and UAVs as flying user equipment.

Artificial intelligence (AI) integration to UAV-assisted wireless networking was analyzed in [34] with brief tutorials for different ML algorithms from different categories, such as supervised, unsupervised, and reinforcement learning (RL). Intelligent reflective surfaces (IRS) combined with UAVs were discussed, as well as how RL can be utilized for optimization purposes. Federated learning (FL), its advantages, and its applications to UAV-assisted wireless communications were also included with a detailed discussion. A similar survey paper in [16] also revealed how ML would help make UAV-assisted communications more effective. Starting with the introduction of the characteristics of UAVs, AI, and ML, the authors grouped the role of ML for UAV-assisted networks into four different categories: namely, physical layer aspects, resource management, positioning, and security. Various subjects, such as interference management, data caching, jamming, and mobility, were discussed separately under each category with an ML focus.

Albeit not discussed from a wireless communication perspective, the survey paper in [35] focuses on deep learning (DL)—a sophisticated version of ML—applications for UAV systems. The authors extensively reviewed DL algorithms and their uses in UAV systems for different applications, including motion control, situational awareness, and path planning for search and rescue missions. The authors concluded their work with the challenges in both DL and UAV autonomy. The short review paper in [36] tried to reiterate the importance of self-organization (for a detailed discussion on self-organization in cellular networks, the readers are referred to [37]) in UAV-assisted connectivity. The scalability issue in UAV-assisted networking was raised, and the role of distributed algorithms in mitigating such issues was also discussed.

The survey in [18]—being the closest one to our survey—mainly investigated the concept of green networking for UAV-assisted 6G networks. Different approaches in green UAV communications, including energy saving, energy harvesting, and RIS-assistance, were presented, followed by the enablers of green UAV communications. The functionality of UAV-assisted green 6G networks were also analyzed, and new research directions aimed at making the UAV-assisted 6G networks more feasible were identified. The authors in [19] provided a detailed survey in terms of UAV cellular communications. Unlike the survey papers mentioned above, the standardization issue involving the study item phase by the third-generation partnership project (3GPP) was also covered. The authors also included the 3GPP work item phase along with some non-3GPP standardization initiatives, such as The International Telecommunication Union Telecommunication Standardization Sector (ITU-T), The European Telecommunications Standards Institute (ETSI), and The Institute of Electrical and Electronics Engineers (IEEE). Such in-depth standardization discussion makes the survey in [19] unique and quite informative in that sense.

1.2. Motivation, Contributions, and Organization of the Survey

There is a myriad of survey papers on UAVs and their roles in different domains, including agriculture, surveillance, search and rescue, etc. As discussed so far in this survey paper, their roles in wireless communications are also phenomenal due the additional degrees of freedom that they provide. In this regard, the number of studies investigating the assistance of UAVs in wireless communications is huge, and there is also a considerable amount of survey papers that bring the studies in particular perspectives together.

Since energy optimization lies at the heart of UAV-assisted wireless communication networking, we intend to produce a dedicated survey paper on this topic in order to compile the research efforts in energy optimization, with three main objectives:

• To highlight the cruciality of energy optimization in UAV-assisted wireless networking;

• To reveal the state of the art in order to understand where we currently stand;

• To identify the gaps in the literature, on which further research should focus.

Furthermore, given that ML has proven to be an efficient and effective tool in wireless communications networking [38], it has been widely used in UAV-assisted networking as well [16,35,39]. Therefore, the application of ML in energy-efficient UAV-assisted networking is also included in this survey in order to capture the current research trend in the literature. In addition, conventional optimization algorithms, such as genetic algorithm, game theory, and particle swarm optimization, are covered in this survey since they also have a large application area in the case of energy efficient UAV-assisted networking. The primary intention of including both the conventional and ML methods is to provide a holistic approach that does not prioritize one over another, given that the corresponding works themselves have studied all types of optimization methods. Therefore, with this approach, the literature is better represented.

To this end, although there are many survey papers on UAVs in the literature—some are detailed in Section 1.1, to the best of our knowledge, there is no survey paper that precisely investigates the energy optimization methods for UAV-assisted wireless communication networking by combining the conventional and ML methods. To be more specific, even though almost all the surveys include a small or large amount of content on energy consumption in UAV-assisted wireless networking [13,18,33,34], none of them has the energy optimization methodology as the sole focus. The work in [18] is the closest to our survey as its primary focus is also green UAV-assisted networking; however, our survey has a number of differences:

• The focus of our survey is not only on energy optimization in general but also on the optimization methods employed in energy efficient UAV-assisted wireless communication networking;

• We categorized the methods according to their type (i.e., conventional and ML) and investigated each energy optimization method accordingly;

• For the sake of completeness, a brief tutorial about the optimization methods is included in our survey;

• We also included the novel landing spot approach, which has gained momentum in the research community.

The summary of contributions and the paper organization is as follows:

• Section 2 discusses different types of UAVs in order to reveal their characteristics and capabilities, which is quite important in selecting the UAV for a particular application;

• The power supply and charging mechanisms of UAVs are extensively covered in Section 3. This is particularly important because the optimization can be performed according to the power supply (e.g., battery, grid, fuel, renewable, hybrid), and various charging/recharging mechanisms (e.g., battery swapping, refuelling, wireless power transfer, etc.) can be placed into the optimization model.

• The role of UAV-BSs in wireless communication networks is investigated in Section 4. Since this survey is oriented towards wireless communication networks, we include a thorough discussion on how UAVs can help and what their primary use-cases are. Such discussion also reiterates the reasoning behind using UAVs in today’s and future wireless communication networks and somehow uncovers the importance of the efforts trying to make the whole concept feasible;

• Section 5 presents different types of UAV deployments in order to explain the difference between standalone UAV deployments and UAV-assisted cellular networking because, according to this, the energy optimization model changes significantly;

• The energy optimization in UAV-assisted wireless networking is covered in Section 6, wherein the energy optimization is categorized according to the optimization objective (i.e., propulsion energy, communication energy, and joint optimization of propulsion and communication energies);

• For the sake of completeness of this survey paper, the overview of both conventional and ML algorithms employed in energy optimization of UAV-assisted wireless networks is given in Section 7;

• The energy optimization techniques, as the core part of this survey, are thoroughly discussed in Section 8. Various techniques are introduced by presenting the related literature, which is categorized in terms of the type of UAV deployment and optimization method employed (e.g., conventional and ML). With this section, the state-of-the-art is demonstrated, and a recently proliferating concept—called landing spot optimization—is also included to capture the energy optimization in UAV-assisted wireless networks holistically;

• To understand what and how other technologies can boost EE in UAV-assisted wireless networks, Section 9 mainly introduces the enabling technologies. In this section, a novel technology called RIS is included as one of the enablers, since RIS has recently gained a significant amount of interest in the research community;

• Section 10 identifies the primary challenges and possible future research directions in order to fill the gaps in the literature that would enable the overall UAV-assisted wireless communications concept to be more feasible. Lastly, Section 11 concludes the survey with final remarks.

The summary of the paper’s organization is illustrated in Figure 2.

2. Types of UAVs

UAVs, also known as drones, are of two main categories: fixed-wing and rotary-wing. However, with the advancement in UAV technology and the wide range of applications of UAVs, rotary-, and fixed-wing UAVs can be combined to form a hybrid design [19].

Rotary-wing UAVs are designed to perform vertical take-offs and landings. One of the main design features of rotary-wing UAVs is that they can hover on a fixed and specified location, making them perfect candidates to perform tasks such as continuous cellular coverage and sensing [19]. However, rotary-wing UAVs consume more energy since they operate at a low altitude with little mobility, and their constant flight against gravity results in greater power consumption [19].

Fixed-wing UAVs are another type of UAV that can glide through the air and operate at higher altitudes, making them more energy efficient and capable of carrying heavier payloads. Moreover, fixed-wing UAVs, such as tiny planes, have heavier weights, faster speeds, and must move forward to stay airborne [13]. However, fixed-wing UAVs require a runway for landing and take-off and are more expensive than rotary-wing UAVs [40].

The limitations of both rotary-, and fixed-wing UAVs led to the emergence of a new type of UAV in terms of shape and aerodynamics, called hybrid UAVs [41]. The fundamental design strategy behind the hybrid ones is to combine the design features of both rotary-, and fixed-wing UAVs. Hybrid UAVs employ different features of both rotary and fixed-wings for various maneuvers and flights dynamics. These UAVs can perform vertical takeoff and landing (VTOL) in copter mode and shift to high-speed forward flight in aeroplane mode [42]. For example, a parrot swing UAV [19], which is an improved version of the traditional four-arm quadcopter (a rotary-wing UAV), has been equipped with some fixed-wing UAV features. As such, it can take off vertically quickly, hover, and fly horizontally at supersonic speed.

3. UAV Power Supply and Charging Mechanisms

In recent years, there has been an increase in the application of UAVs in both commercial and military domains, due to their easy adaptability, the flexibility of deployment, and cost-effectiveness [13]. However, to fully exploit the capabilities of UAVs in different application domains, there is a need to consider the challenges and limitations of UAVs. One of the most important limitations in UAVs is power and energy consumption [19]. Commercial UAVs are usually powered with rechargeable batteries for operations, while large UAVs, such as military UAVs, use non-renewable resources, such as fuel and gas, to provide more energy to the UAV for longer flight time. In recent years, new and alternative methods for UAV power supply, including solar energy, wireless charging, laser beam charging systems, etc., have been developed and tested [43]. In the following paragraphs, we discuss different methods of power supply and charging mechanisms for UAVs while Figure 3 illustrates various UAV power supply and charging mechanisms.

3.1. Battery Powered UAVs

The battery power supply is one of the main power supply techniques used to meet the energy requirements of UAVs. Batteries are mostly preferred in relatively smaller UAVs because classical batteries, such as lithium polymer (LiPo), can power a UAV for a maximum of 90 min [44,45]. This severely limits the commercial and industrial applications of battery-operated UAVs. Despite significant advancements in battery technology, the limitations associated with the use of batteries to power UAVs are still far from over. Since batteries are the most predominant source of energy for miniature UAVs, and the energy stored in the battery is limited and can be easily depleted during the UAV operation, to increase the flight time and operation of UAVs, batteries must be frequently charged in order to replenish the depleted energy stored in them. To address such limitations, several battery charging techniques have been developed. Hence, in the following, we consider different battery charging mechanisms devised to recharge the UAV battery and ensure the longevity of UAV operation.

3.1.1. Battery Swapping

The swapping process consists of recharging or replacing the UAV batteries and can be done conventionally or via hot swapping. In the conventional swapping method, the UAV whose battery is depleted has to leave its service location to the charging station and be replaced by an already charged UAV. The challenge with this method is that it requires several backup UAVs to be able to replace the UAVs whose batteries are depleted; as pointed out in [46], up to two standby UAVs may be needed per UAV to ensure continuous coverage for a commercial UAV that has a charging power of 180 V. However, the exact number of backup UAVs would depend on the UAV downtime, which comprises both the time taken for the UAV to fly to and from the charging station as well as that taken to recharge the battery to full capacity [43].

In the hot swapping approach, the UAVs do not have to be powered off or remain in the charging station until the battery is recharged; instead, as soon as they reach the charging station, new batteries are quickly inserted into them so that they can return to their operating station immediately. The limitation of hot swapping is that it requires human involvement to replace depleted batteries with new ones. To address this challenge, automated battery swapping mechanisms have been developed whereby a robotic actuator can be used to remove and replace UAV batteries [47]. For swapping to be implemented effectively, a battery recharging station is required to recharge or replace depleted batteries. This recharging station can be located on cellular towers, rooftops of buildings, and specialized standalone pylons. There is also the need for multiple UAVs to ensure continuous service provision. In addition, a management system must be put in place to coordinate the battery recharging and replacement cycle of the swarm of UAVs [48].

3.1.2. Laser Beam Charging

Another technique for battery recharge is laser beam in-flight recharging [48]. This technique involves recharging the UAV battery during the flight, without making it land for the battery recharging or replacement, as is the case in the battery swapping technique. To recharge a UAV’s battery using the laser beam technique, an external energy source feeds the laser beam at a particular frequency and wavelength. The laser is directed towards a photo-voltaic (PV) cell on the UAV that converts the laser beam into electrical energy required to recharge the UAV’s batteries. To ensure maximum output from laser beam charging technique, a power tracking device is used to identify the maximum energy point in the laser beam. In order to maintain power transfer from the laser transmitter, the UAV must operate at low heights and occasionally in a restricted area while using the laser-beam in-flight recharging strategy [48]. Furthermore, as each UAV requires its own dedicated laser source, the number of UAVs must be reduced in order to maintain a fair operational cost [46] The laser beam is an effective method for charging UAVs, and it can be applied to both rotary-, and fixed-wing UAVs. However, the use of laser beam recharging methods restricts the operation of the UAV to a limited area in order to ensure that the recharging link is active.

3.1.3. Wireless Power Transfer/Wireless Charging

Wireless power transfer (WPT) is another technique for countering and overcoming the limitations of conventional battery power supply mechanisms to UAVs. The concept of wireless charging was introduced by Nikola Tesla [49]. To perform the wireless charging of electronic devices, a transmission pad is used to charge the device using the resonant inductive coupling [49]. A typical WPT consists of two main components: namely, a transmitting device and a receiving device. A transmission device is used for the conversion of the source energy into “time-varying electromagnetic fields” which is then transmitted using a transfer media, which converts the time-varying electromagnetic arrays into electrical power that is, in turn, used to power the UAV.

Currently, the WPT technique is in its early stages and there are still several issues and limitations that need to be addressed. For instance, due to the significant propagation loss of RF signals over long distances, the performance of WPT systems with a wide coverage range is essentially hampered by their low end-to-end power transmission efficiency. As a result, fixed-location energy transmitters (ETs) must be distributed in an ultra-dense way in order to provide pervasive wireless energy access for enormous low power energy receivers (ERs) deployed over a wide area.

Using WPT, some UAVs can serve as recharging UAVs to ensure that batteries of other UAVs—that are serving user requests—are not depleted by flying to their respective locations to recharge their batteries while they are in operation. This will prevent service interruptions that might occur if each UAV has to fly to a charging station located at a distance from the UAV operating region to replenish its depleted battery. In such an arrangement, there is normally a transmitting UAV (tUAV) whose responsibility is to transmit RF signals to the serving UAVs (sUAVs), which have been equipped with sensors to convert the received RF signals to electrical energy needed to power the UAVs [50].

3.2. Grid Powered UAVs: Tethering

Another technique for powering UAVs is called tethering, which allows the connection to be established between the UAV and power supply on the ground using a fiber optic cables [48]. Fiber optic connections allow several kilowatts of power to be transferred using high-intensity light. This technique provides more autonomy to UAVs and allows secure and quick data transfer in real time. However, the tethered UAVs (TUAV) are only allowed to hover within a certain hovering region since the tether has a maximum length and the launching point of the TUAV is typically located on a rooftop. A study in [51] investigated the best location for tethered UAVs (TUAVs) to reduce the average path-loss between the TUAV and a terrestrial receiver. They calculated the upper and lower bounds for appropriate tether length and inclination angles in order to reduce average path-loss. Aside from the maximum tether length, the heights of the buildings surrounding the TUAV rooftop determine the hovering region, which necessitates that the tether’s inclination angle does not fall below a certain minimum value to avoid tangling and to ensure safety. It is worth noting that if the tethered cable delivers constant power, the UAV flight is expected to last a few days or possibly a few months [52].

Tethered UAVs have been considered for various wireless network applications in the literature. The work in [53] presents a comparative performance analysis of untethered-UAV (U-UAV) and T-UAV-assisted cellular traffic offloading in a traffic-heavy location. They employed stochastic geometry methods to calculate joint distance distributions between users, the terrestrial BS (TBS) and UAV. A user association strategy is presented, and the relevant association areas are analytically discovered to optimize the end-to-end signal-to-noise ratio (SNR). The total coverage probability of U-UAV/T-UAV-assisted systems is then calculated for specified TBS and U-UAV/T-UAV locations. The authors in [54] introduced a novel UAV-based post-disaster communication system where U-UAVs are employed to offer cellular service in disaster-affected areas, while T-UAVs provide backhaul for the U-UAVs.

3.3. Fuel Cell-Powered UAVs

Fuel cell is also used to power UAVs; the use of fuel cell is more effective and efficient as compared to the battery power UAVs, such that a fuel cell power increases the flight time of UAVs six times as compared to battery [55]. However, the fuel cells have their own limitations; for example, the fuel cells have lower energy density and require special consideration for fuel tanks [56]. Hydrogen cannot be stored at high pressure and lower temperatures. To overcome fuel cell density issues, compressed hydrogen gas, liquid hydrogen, or chemical hydrogen can be used in fuel-powered UAVs [57]. Fuel cells are much lower than lithium batteries in terms of power storage and efficiency, according to [56], fuel cells can achieve a maximum level of efficiency up to 60% compared with the lithium batteries that achieve an efficiency level of 90% in terms of power storage [43]. The limited power storage capacity in fuel cells is due to the required auxiliary subsystem needed for fuel cell stacking operations. To tackle this issue, the authors in [58] used compressed hydrogen to power the UAV, resulting in a total flight time of about 24 h.

The generation of chemical hydrogen requires a specialized infrastructure and equipment, making the power supply system very complex and heavy. Moreover, the process of hydrogen extraction usually takes more time, leading to increase in response time of the UAV to load changes, which creates load balancing problems.

3.4. Renewable Energy Powered UAVs

Various types of renewable energy (e.g., wind, solar, etc.) can be used to power UAVs in order to increase their travel time and power efficiency. To power UAVs with wind energy, gust soaring can be used by adjusting the trajectory of the UAVs, which enables the UAV to extract the energy from the wind by converting the potential energy of the wind into kinetic energy [59]. One of the main limitations is the dependence on the environmental condition and airflow [60]. Another prominent technique to power UAVs is to use solar PV cells that can be mounted on the wings of the UAVs in order to recharge them via the irradiation from the Sun. One of the main drawbacks of this type of power supply is that it limits the flight of UAVs in rain and during nighttime, when there is less or no irradiation from the Sun. Hence, the unavailability of the sun in the nighttime and rain entails that the UAVs needs to be powered with another form of power supply [61].

3.5. Hybrid Powered UAVs

Hybrid power supply methods can be used to power UAVs, and they combine battery, fuel cells, and renewable energy sources to provide a blend of power supply [62].

3.5.1. Fuel Cell-Battery

There are some limitations associated with the use of fuel cell and battery-powered UAVs. On one hand, the process of electricity generation from fuel cells takes a long time as several components such as compressors for air supply, pumps, and valves are involved, which leads to an increase in response time of the UAV to load changes. On the other hand, battery-powered UAVs have limited flight time and would require frequent recharging, which can negatively affect the effectiveness of their service. Therefore, hybridization of fuel cells and batteries can be used to minimize the delays associated with electricity generation as well as prolonging the service time and effectiveness of UAVs [63].

Hybrid power supply resources can be used interchangeably to power UAVs. For instance, battery power can be utilized by UAVs during the take-off and ascending process because batteries have more density and power storage as compared to fuel cells, and UAVs require more power when taking off and ascending to higher altitudes [64]. Then, fuel cell can be used in the flight time and descending process. Furthermore, the fuel cell can be used to charge the batteries. In [65], the authors used a technique called hardware in the loop (HIL) in the hybrid UAVs to investigate the power consumption of fuel cells and battery power sources. Several simulations were conducted to determine the level of flight endurance that can be achieved. Another study was conducted by [66], using fuel cells and battery power supply to assess the battery power contribution under different conditions and scenarios.

3.5.2. Solar Cells plus Battery

Another method for extending the mission length of UAVs is the installation of PV panels to operate alongside the existing batteries. When solar irradiation is available, PV cells are often employed to power a UAV or refill its battery. The battery, in turn, is utilized for functioning at night or during hours when solar radiation is limited [67]. The use of solar power as an energy source allows small size UAVs to carry larger payloads and can increase flight periods to more than 24 h, allowing for multi-day flying [68]. The authors in [69] studied the design and validation of a solar power management system (SPMS) for their solar-cell and battery-powered experimental UAV. Their results reveal that when the angle of incidence of sunlight varies from 0 to 45 degrees, the power consumed from the solar cells fluctuates depending on the load situations and may be reduced by up to 30%. This means that changes in aircraft attitude will have a direct impact on the power generated by the solar system.

4. The Role of UAV-Base Station in Wireless Communications

UAVs are employed for different operations in both military and civilian domains, including object detection, location tracking, goods delivery, disaster monitoring, information dissemination, etc. [13]. Recently, they have also found several applications in wireless communications because of their flexibility, adaptability, and easy deployment [19]. Figure 4 shows various use cases of UAV-BSs in wireless communications. In the following paragraphs, we briefly discuss some of these applications.

4.1. Emergency Services (Pop-Up Networks)

UAVs are becoming one of the most active areas of research and industrial development due to their promising applications in different domains. One of the application areas for UAVs is emergency response [70]. During large-scale natural disasters such as floods, earthquakes, or major fire outbreaks, hurricanes, etc., several properties and infrastructure are normally destroyed, including cellular infrastructure, thereby leading to loss of communication services in the affected area [71]. In such situations, UAVs can be deployed to such areas to replace malfunctioning or destroyed BSs and provide emergency communication services. This helps to provide coverage and improve the QoS of users in the affected area. UAV parameters such as trajectory, altitude, etc., can be optimized based on the user traffic demand and distribution to maximize their throughput and coverage. In this regard, the work in [14] considered the deployment of UAVs as emergency pop-up networks to restore communication services to users in an area where cellular network infrastructure may have been destroyed due to natural disasters. They applied RL to optimize the trajectories of the UAVs in order to maximize the coverage and throughput of ground users.

In addition, UAV-BSs can serve as relays to help connect isolated groups of users that are separated from each other. This is vital in disaster situations such as earthquakes or floods where a single community can be split into isolated groups. Under this scenario, the UAV-BSs allow the mobile devices of separated community members to communicate locally with one another, even though they do not have access to the internet. Similarly, UAVs can provide backhaul services to other existing wireless networks, in essence connecting such isolated communities to the global internet. UAV-BS can also serve as a viable backhaul solution for device-to-device (D2D) communications, as shown in [72]. They can also be equipped with efficient processors to enable real-time data aggregation from different devices and information processing from remote sensors, mobile devices, and other data collection and communication technologies (e.g., IoT) within an affected area. This real-time processing of information enables faster decision making by emergency response teams and other stakeholders because the quick response time and accurate decision making are vital to executing effective rescue mission during emergencies [73].

4.2. Data Harvesting from IoT Devices

Due to the promising solutions that UAVs provide in different application areas, UAVs has attracted research attention in IoT applications in order to facilitate data harvesting and relaying [74]. This is very important for cases where the IoT devices are deployed in areas where there are no wireless network facilities, such as in rural farmlands or offshore locations, to help transmit the data to the decision-making centers. In such situations, UAVs can be deployed to that location to assist in data collection [75,76]. Research works on UAV-enabled data harvesting mainly focus on UAV path planning and throughput maximization in order to ensure that the UAV collect sufficient data and returns to data center before its battery depletes [77,78,79].

Following this research direction, a deep RL framework for path planning in a multi-UAV-based data harvesting system that can adapt to dynamically changing network parameters, such as varying number of UAVs and IoT devices, different flight schedules, and amount of data to be harvested, was proposed in [78]. The path planning problem was modeled as a cooperative team of UAVs saddled with the responsibility of maximizing data collection from several IoT devices distributed in an area. Then, a multi-agent Q-learning algorithm was developed to determine the optimal trajectories that would maximize data collection from the distributed IoT devices. In [77], the authors considered the joint optimization of both the UAV trajectory and power allocation to the ground nodes in order to maximize the throughput and coverage probability in a UAV-enabled data harvesting system with distributed beamforming. Heuristic algorithms based on convex optimization and approximations were developed to find the optimal trajectory and power allocation strategy that would maximize both performance metrics.

4.3. Content Caching and Computation Offloading

Content caching is an important process in modern-day wireless communication networks as users move from one location to another. It involves storing important information such as username, location, and popularly requested content at multiple BSs in order to provide seamless communication and minimize the latency involved in information retrieval and transmission to the user [80]. However, most of the caches are usually installed in a fixed location and would not be suitable for highly mobile users in vehicles and high-speed trains as the requested content would have to be stored in all BSs along the user path. In such situations, dynamic caching can be achieved by mounting the caches on UAVs, in what is known as UAV-enabled caching [81], and making the UAVs follow such highly mobile devices to provide the requested services. In this regard, the authors in [82] proposed a learning-based joint caching and trajectory optimization scheme in vehicular networks in order to enhance the throughput of the network.

Mobile edge computing (MEC) was developed to assist user devices in offloading and processing computation-intensive tasks that are beyond their battery capacity [83]. However, these MEC servers are usually deployed in a fixed location within the network and may not always be accessible. To solve this problem, UAV-enabled MEC was proposed in [84] where UAVs equipped with edge servers can assist in computation offloading from ground user devices in order to minimize their energy consumption and maximize their battery lifetime as well as respond to real-time computationally intensive data processing demands. Regarding this, the authors in [85] proposed a UAV-enabled computation offloading framework based on deep RL to minimize the network latency, energy, and bandwidth cost.

4.4. Load Balancing

Due to the movement of mobile users from one place to another as well as variation in user traffic demands, the traffic loads of BSs vary both temporally and spatially. This makes some BSs to be lightly loaded while others are heavily or overloaded, thereby leading to service denial or poor QoS from the overloaded BSs [86]. Although small cells help to reduce the level of traffic imbalance in cellular networks, due to the static nature of their deployment, they cannot respond to sudden changes in traffic demands that may be occasioned by large events taking place in an area for a short time or other impromptu surges in network traffic that require quick intervention to reduce the pressure on the available BSs. In such cases, UAV-BSs can be of great help because they can be quickly deployed to respond to such traffic demands and ensure that load is properly distributed among the various BS in order to avoid network congestion [87].

In this regard, the authors in [88] proposed a learning-based framework for UAV deployment to crowded regions of the network during periods of peak traffic to assist in load balancing, thereby preventing degradation in the QoS of users. In the proposed framework, both auto-regressive integrated moving average (ARIMA) and extreme gradient boosting (XGBOOST) ML algorithms were employed to forecast future high-traffic intensity regions based on historical data for proactive UAV deployment. The work in [89] considered UAV deployment for load balancing to reduce communication latency between IoT devices and macro BSs (MBSs) during periods of high traffic load on the MBS. Heuristic algorithms were developed to determine the optimal location of the UAVs, and association strategy for the IoT devices. The work in [90] considered the application of UAV-BSs for load balancing and capacity enhancement in high-traffic areas via efficient trajectory design. A deep reinforcement learning algorithm was proposed to determine the optimal trajectories of the UAVs that would efficiently balance the load among the UAV-BSs in order to achieve high spectral efficiency.

4.5. Coverage Extension/Relaying

The fast growth of mobile devices, such as smartphones, tablets, and wearables, has increased the demand for high-speed wireless access. As a result, the capacity and coverage of existing wireless cellular networks have been overstretched, thereby leading to the development of new wireless technologies to address this issue [13]. Because of their vast range of use-cases, UAVs have sparked many attention [91]. The air-to-ground (AtG) dominant line-of-sight (LoS) link enhances the performance of UAV-enabled wireless communications [92], thereby resulting in reduced propagation loss and better link QoS. As a result, UAVs functioning as aerial BSs or relays are commonly employed to increase network capacity or achieve more flexible coverage [93,94,95].

The authors in [96] investigated mobile relaying in wireless powered communication networks (WPCN), where a UAV is used to aid in the transportation of information from numerous sources to a destination with severely obstructed communication channels. The sources are low-power and do not have any energy source. The UAV serves as a hybrid access point (AP) that is used both as WPT to power the user devices and as a means of information transmission and reception. In [97], the authors consider the use of UAV as a relay to serve users at the cell edge by extending coverage of an existing network using non-orthogonal multiple access (NOMA) technology, thereby enhancing their QoS.

4.6. Capacity/Throughput Enhancement

The exponential growth in the telecommunication and information technology landscape over the last decade has involved a tremendous increase in the amount of user demands and requests for more resources in terms of data traffic. To cope with the situation, macro BSs and small BSs are used to provide coverage to users. However, due to the continuous increase and fluctuation in data traffic at various locations in the network, the dense deployment of only terrestrial small BS networks is no longer sufficient to address this capacity demands. As a result, UAVs have been identified as a potential solution to provide more reliable and effective coverage to users due to their flexibility, adaptability, and quick configuration [98,99].

The authors in [100] considered the optimal deployment of UAVs in heterogeneous networks in order to enhance the capacity of the network. To determine the optimal geographic location to deploy the UAVs, a utility function was developed to model the traffic intensity in various parts of the network, after which a heuristic algorithm was proposed to assign the UAVs to their optimal positions. Compared with existing ground-based wireless networks, the suggested model was proven to offer superior capacity, consistency, and extended connectivity. The work in [101] considered the deployment of multiple-UAVs to offload traffic from a single terrestrial BS that is heavily loaded in order to maximize the throughput of ground users at the cell edge.

4.7. Backhauling

In cellular networks, the backhaul serves as a connection between the BSs and the core network and this connection is usually established using fibre cable, microwave links, etc. However, in emergency scenarios, where existing backhaul infrastructures have been damaged, or when there is a need to enhance the capacity of existing backhaul links, ad-hoc backhaul connections can be established using UAVs [102,103]. With respect to UAV-enabled backhaul connections, the authors of [102] proposed a UAV-enabled wireless backhaul mechanism for ultra-dense networks. Game theory was employed to model the formation of the multi-hop backhaul network comprising multiple UAVs. Then, a heuristic algorithm was proposed to determine the optimal network formation strategy that maximizes data rate and minimizes network delays. UAVs can also be used for backhauling in high mobility network scenarios such as in high-speed trains where, due to the very high speed of the train, the channel condition is subject to continuous turbulence and instability. In this regard, a backhauling mechanism using the combination of UAVs and free-space optical (FSO) communications was developed in [104] to enhance the coverage probability of the high-speed train network.

4.8. Energy Efficiency

The UAVs can also help enhance the EE of existing cellular networks. This can be achieved by deploying UAVs to assist heterogeneous networks to enhance their capacity faster than their power consumption, which in turn helps to improve their EE. The authors in [105] demonstrated this by deploying millimeter wave UAVs alongside macro and small BSs in a three-tier heterogeneous network to enhance cell edge users throughput, which also led to the overall improvement in the EE of the network. UAVs can also be deployed in existing terrestrial cellular networks to serve delay and rate sensitive users while offloading the traffic of lightly loaded small BSs in order to put them into sleep mode. This approach is known as UAV-assisted BS sleeping strategy and it results in a significant reduction in the energy consumption of the network as was introduced in [106].

5. Types of UAV Deployments

As highlighted in Section 4, there is ever-increasing application of UAVs in different aspects of wireless communications. In each of these applications, wireless networks can comprise standalone UAV-BSs, such as in emergency scenarios where, due to a natural disaster, the fixed cellular network infrastructures are destroyed, and a pop-up network must be implemented to restore network service in the affected areas [14,72]. Wireless networks can also comprise both UAV-BSs and terrestrial BSs. This is the deployment type where the UAV-BSs are deployed to assist existing cellular networks in ensuring service continuity during failure or breakdown of a BS site or to provide coverage and capacity enhancement [107]. The two major types of UAV-BS deployments are illustrated in Figure 5, while in the remaining part of this section, we briefly highlight the main features of each type of UAV deployment.

5.1. Standalone UAV Deployments

Standalone UAV deployment involves the deployment of single or multiple UAV-BS to provide network service in an area without fixed cellular network coverage. Two major deployment scenarios exist in this approach. The first scenario involves the deployment of a single UAV-BS to provide wireless network service in an area, including harvesting data from IoT networks deployed in a field, acting as a relay to provide wireless service to users that have been separated by large obstacles such as mountains and hills, etc. [108]. In the second scenario, multiple UAVs are deployed in the form of a swarm network for the following purposes: (i) to provide a complete wireless network service to a particular area, which could be a dedicated network for an organization; (ii) to provide wireless service in rural areas without prior cellular network infrastructure; (iii) to restore network coverage for a large area that has been affected by a natural disaster such as an earthquake or a volcano [109]. The major challenge with single UAV deployment is that when a fault occurs in the UAV-BS, it could result in complete network failure. On the other hand, in a multi-UAV-BS system, when a single UAV fails, we can reconfigure the system and still have a sub-optimal solution. However, this does not mean that the use of multiple UAV-BSs does not have disadvantages, as the problem of proper coordination among the multiple UAVs deployed exists [110]. In summary, choosing between a single UAV or multiple UAV deployment depends on the nature of the communication system that is being developed and the problem that the network would address.

5.2. UAV Deployment with Fixed BSs (UAV-Assisted Cellular Networks)

This involves the deployment of either single or multiple UAV-BSs on top of existing fixed cellular network infrastructure (terrestrial BSs) to provide enhanced capacity and coverage in different scenarios [105], as shown in Figure 5. This deployment scenario can be used to enable many of the application scenarios highlighted in Section 4, such as load balancing, pop-up networks, backhauling, capacity enhancement, etc. For example, UAVs can be deployed as pop-up networks at the edge of existing fixed networks to extend coverage to disaster zones, or at entertainment venues such as football stadia and concert halls to improve data streaming speeds for fans, as an on-demand MEC server to improve data processing when required, etc. In such scenarios, UAV-BSs can be deployed to provide additional capacity in order to reduce network congestion and to enhance the data rate of the users in hotspot regions and at the cell edges [107]. In addition, UAV-BSs can help to provide communications with highly mobile user equipment (UE), thereby preventing frequent handovers, which can negatively affect their QoS [111]. One of the main advantages of deploying UAVs in existing cellular networks is that they can change their locations depending on the network conditions, which helps in increasing the QoS, restoring the network at failed BS sites, achieving load balancing, offloading traffic from macro BSs, and extending the coverage. Hence, UAV-BS deployment in cellular networks is a key design consideration in future heterogeneous wireless networks that can enable applications such as smart cities, mobile computing, autonomous vehicle networks, etc. [112].

6. Types of Energy Optimization in UAV-Based Cellular Networks

The energy consumption of UAVs is one of the major challenges that limit their applications in many areas including in wireless communications networks [18,111]. As 5G and beyond networks target drastic improvement in network EE, the use of UAV-BSs, though very promising, could be a hindrance to the actualization of this objective if the issue of increased energy consumption is not carefully considered [19]. Since UAVs are designed to fly from one location to another, their application in cellular networks brings a lot of flexibility and adaptability to the network as they can be deployed on demand to various parts of the network to handle different challenges ranging from network restoration, coverage and capacity enhancement, traffic offloading, load balancing, backhauling, etc., as mentioned in Section 2.

However, the large amount of energy consumption involved in flying or hovering the UAV to or over the service area makes their use very challenging. Although various UAV power supply and charging mechanisms were discussed in Section 3, it is still very important to optimize the energy consumption of the UAV-BSs because of the limitations of these methods including low energy storage capacity, need for frequent recharging, low energy conversion efficiency, unpredictability of renewable energy sources, etc., which can negatively impact the performance of UAVs in wireless networks [46]. In addition, most energy optimization techniques are easier and less expensive to implement compared with the various power supply and recharging techniques discussed in Section 3. Therefore, in this section, we highlight the major areas of energy optimization, while in subsequent sections, we review the various algorithms that have been proposed for tackling the energy optimization problems identified in this section.

There are four major aspects of energy optimization in UAV-based cellular networks:

6.1. Optimization of the Propulsion Energy

The propulsion energy is the energy consumption associated with flying or hovering the UAV-BSs over the service area. This is the most significant energy consumption of the UAV-BS [19]. Hence, energy optimization strategies proposed in this direction aim at reducing the energy consumption due to the movement of the UAV-BS. In this regard, a few models to quantify the energy consumption of the UAV due to propulsion were developed in [25,113,114]. In addition, various approaches to minimize the propulsion energy consumption have been introduced, including regulating UAV altitude and planning the trajectory, etc. [70,113,115].

6.2. Optimization of the Communication Energy

Communication energy is the energy consumption associated with signal processing and data transmission during the UAV operation in wireless communication networks. This is usually less significant compared with the energy consumption due to propulsion [19], therefore, energy optimization approaches proposed in this area are targeted at reducing the energy consumed while processing and transmitting user information. The techniques proposed in this area include transmission power allocation and control, scheduling the transmission of UAV-BSs—particularly for the cases where they need to fly over a predefined trajectory, and optimally positioning UAVs in a service area [116,117,118,119].

6.3. Joint Optimization of the Communication and Propulsion Energy

Unlike the first and second cases that consider optimizing either the communication or propulsion energy consumption, the works conducted in this direction consider the simultaneous optimization of both the communication and propulsion energy consumption of UAV-BSs. These approaches result in the most energy conservation as both components of the UAV-BS energy consumption are considered together. The strategies considered in this area involve a combination of the approaches proposed in propulsion energy optimization as well as that of communication energy optimization [25,120].

6.4. Optimization of the Energy Consumption in UAV-Assisted Cellular Networks

The previous cases (i.e., propulsion and communication energy) considered mainly deal with the energy consumption of only the UAV-BSs when they are deployed alone or as a swarm network comprising multiple UAV-BSs. The UAV-assisted cellular network is the case where a single/multiple UAV-BSs is/are deployed in existing terrestrial BSs in order to enhance certain network performance indicators such as throughput, coverage, etc. [121,122]. However, their deployment could result in an increase in the overall energy consumption of the network if not properly managed. Hence, the strategies developed in this direction are meant to reduce the energy consumption of both the UAV-BSs and the fixed BSs, or a scenario where the UAV-BSs can help in reducing the energy consumption in fixed BSs through a UAV-assisted BS sleeping strategy [105,106,107].

7. Overview of Algorithms for Energy Optimization UAV-Based Cellular Networks

To ensure energy efficient deployment, operation and management of UAVs, whether as a standalone UAV network or UAV-assisted cellular network, there is a need for energy optimization strategies to be devised that would ensure that the UAVs minimize their energy consumption while serving user demands due to the limited energy capacity of battery-operated UAVs. However, the development of energy-efficient solutions requires the application of conventional or traditional algorithms and ML algorithms. Hence, in this section, we review some common algorithms that are used in the development of energy-efficient strategies to minimize the energy consumption of mobile cellular networks. Then, in Section 8, we consider specific strategies that have been devised to reduce the energy consumption in UAV-based cellular networks alongside the specific algorithms that were applied to achieve the proposed energy-efficient solutions.

7.1. Conventional Algorithms (CA)

There are several conventional optimization methods that have been applied for energy optimization in UAV-based cellular networks in the literature. Conventional methods are divided into three categories: exact methods, heuristic and meta-heuristic methods. Exact methods are not sufficient, especially in operational decision processes, due to the unacceptable solution times and their inability to reach solutions in large-size problems in a reasonable time [123]. Therefore, in this survey, we focus on the remaining two categories: heuristics and meta-heuristics. Heuristic algorithms are a set of procedures that are developed to specifically address an optimization problem [115,124]. Meta-heuristic methods can be classified in terms of various features such as neighborhood structure and searching strategies. In this study, meta-heuristic methods are discussed in terms of three basic classes, namely: evolutionary-, swarm intelligence-, and trajectory-based. However, since there are many developed meta-heuristic methods in the literature in each class, the most-known methods are examined in a general framework. The general classification of conventional algorithms can be seen in Figure 6. In the following paragraphs, we briefly discuss the common conventional algorithms.

7.1.1. Heuristic Algorithms

Exact algorithms are developed in such a way that the optimal solution can be achieved in a limited time. However, for some complicated optimization problems (e.g., NP-hard or global optimization), this limited amount of time may expand exponentially in relation to the problem sizes. Heuristics lack this guarantee and, as a result, often provide solutions that are less than optimal or approximate solutions. Heuristic algorithms frequently find acceptable solutions in a reasonable amount of time. In addition, they are often problem-dependent, that is, they are designed for a specific problem such as energy optimization in cellular networks, UAV routing problems, etc. There are two heuristic approaches to solving hard optimization problems. One of such approaches is the constructive heuristics which develops solutions via iterations. It is called a constructive heuristic because it begins with an empty solution and continues to expand on it until a complete solution is discovered [125]. The other heuristic method is called improvement or local search heuristics. Improvement heuristics start with a complete solution and then strive to improve on the existing solution further by local searches. Examples of heuristic algorithms include block coordinate descent (BCD), Dinkelbacks method, successive convex approximation (SCA), circle parking theory, etc.

7.1.2. Meta-Heuristic Algorithms

Many heuristic algorithms are quite customized and problem specific. A meta-heuristic, on the other hand, is a high-level problem-independent algorithmic framework that offers a set of principles or techniques for the development of heuristic optimization algorithms. However, a specific definition is quite tricky, and many scholars and practitioners use the terms heuristics and meta-heuristics interchangeably. As a result, the term meta-heuristic can also refer to a general-purpose algorithm that is capable of working independently of the specific problem it is applied to, as opposed to a problem-specific implementation of a heuristic optimization algorithm based on the guidelines provided within such a framework. Unlike heuristic methods, a meta-heuristic knows nothing about the problem it will be applied. It can treat functions as black boxes. For more information on meta-heuristic algorithms, the studies in [126] can be examined. In the following paragraphs, we examine meta-heuristic algorithms in three categories.

Evolutionary-Based Algorithms

These are algorithms that are derived from the concept of biological evolution. In this regard, the genetic algorithm (GA) is presented as a representative of the evolutionary-based algorithms, as it is one of the most commonly used for energy-efficient UAV-based communications.

• Genetic Algorithm: GA is a meta-heuristic optimization method based on the principles of the biological evolution process that finds the best solution to problems that are difficult to solve with exact methods. The first studies on this algorithm were conducted by John Holland. Holland developed new methods for computer systems based on the principles of natural selection and adaptation existing in nature. Holland argued that processes such as crossover, mutation, and selection that take place in the evolutionary process are very important for solving optimization problems and that better individuals can be obtained in each generation. Holland modeled all these processes for solving problems by considering the perfect adaptation of living things in nature to the ecosystem [127].

  GA is one of the population-based algorithms used for solving complex problems because it provides a convenient and fast solution. The population consists of individuals that make up the solution set. By eliminating the bad solutions in the solution sets created in each generation, the next generations consist of good solutions that will lead to better results. Since there is more than one solution set in a generation, finding many best solutions in one step is one of the features that distinguish GA from other algorithms. Additionally, by focusing on the part of the solution set, it can perform an effective search and provide the best solution in a short time [128].

  In the GA application process, the first step is to define how to encode the solutions represented by chromosomes according to different problems. After the necessary parameters are received from the user, the initial population is created so that the GA steps can begin. Each chromosome is an individual and consists of genes. The initial population consists of randomly selected chromosomes. Then, the fitness function that defines the problem solution is determined. Afterwards, chromosomes that will form the next generation are selected from the population, and genetic operators based on genetic processes in nature are applied respectively to obtain better chromosomes. The crossover process is applied to generate new offspring from the individuals obtained from the selection process. The mutation tool is used after the crossover step to provide diversification in the population. At the end of all these processes, new generations are created and compared with the fitness values of other generations. Individuals with good fitness are preserved and passed on to other generations (elitism). This process continues until a specified termination criterion is met [129].

  There are several features that make the GA different from other conventional heuristic methods. The most important of these are that GA offers more than one solution and needs less information for the obtained solutions. Additionally, GA uses probabilistic transitions rules and can be parallelized very easily for application in both continuous and discrete problems. However, the drawback of using GA is that it is difficult to model the problem using the algorithm, and its implementation involves a high computational cost compared to that of other conventional heuristic approaches [128].

Swarm Intelligence-Based Algorithms

These algorithms are inspired by nature and designed based on the relationship between living organisms, including ants, birds, bees, flocks of fish, bacterial communities, etc. [130]. In this regard, Particle Swarm Optimization (PSO) and Ant Colony Optimization (ACO) and considered in the following paragraphs as representative of swarm intelligence-based algorithms.

• Particle Swarm Optimization (PSO) Algorithms:The PSO algorithm was introduced by Eberhart and Kennedy in 1995 [131]. It was developed as a population-based optimization method inspired by the two-dimensional behavioral movement of bird and fish flocks in nature. It has a more straightforward computation method than other traditional optimization methods and does not involve time-consuming complex operations. Therefore, it works faster, has shorter computation times, and is more preferred [132].

  The solution approach of the PSO algorithm is as follows: There is a flock of birds in a region where there is only one source of food. Birds are randomly placed in this food area and no bird knows where the food is. However, the end of each iteration, they know how close they are to the food. In this case, it is a good decision to follow the bird closest to the food. PSO works according to this scenario and is used to solve optimization problems. Birds trying to find food in solution space are called “particles” in PSO. Each particle has a fitness value and velocity that enables it to fly. These are calculated using the fitness function. Particles fly out of the problem space, following the optimum particle at each iteration [133].

  If there is no specific initial solution generation mechanism for a problem, the PSO is started with a group of random solutions (particle swarm) and tries to reach the global best solution with updates. The first obtained feasible solution value is kept as the best solution and the coordinates of the associated solution are determined. In each local search, this value is kept in memory for later use and is called the “local best solution”. The other best value is the coordinates that provide the best solution ever obtained by all particles in the population. This value is kept as “global best solution”. In each iteration, the local best solution is compared with the global best solution based on the objective function to develop the global best solution.

• Ant Colony Optimization Algorithms (ACO): ACO is a meta-heuristic technique used for solving optimization problems and works based on swarm intelligence as PSO. It was developed by Dorigo et al. in 1991 and tested on different sizes of Traveling Salesman Problems (TSP). Dorigo named this algorithm the ’Ant System’ [134].

  The basis of this technique is the pheromone hormone that ants use in communication. Ants start the foraging process randomly, and when food is found, they secrete the pheromone hormone to show the other ants in the colony the pathway to the discovered food. This hormone is updated by other ants and helps the colony find the shortest path to food. An intense pheromone amount indicates the quality of the path and increases the probability of preference for the use of that path. If the ants encounter any obstacle on the way between the food and the nest, the ant in front of the obstacle cannot continue and they must make a decision for the new direction of the trip. Each of the new direction options is equally likely to be selected. If the ant chooses the shortest path, this path becomes the preferred route according to the pheromone hormone density. However, if the chosen path is not the shortest, the colony route is reconstructed very quickly and the amount of pheromone on the newly chosen path is increased to create a preference for the ants that come later. Considering that each ant releases the same amount of hormone at the same rate on average, the expected situation is that it takes a long time for the colony to recognize the obstacle and choose the shortest path. However, the path selection made by the ants coming from behind, depending on the amount of pheromone, shortens the total time to trip for food [135].

Trajectory-Based Algorithms

These algorithms employ a single agent that traces a trajectory while moving through the search space to determine the global optimal solution. In the process, a better solution is accepted while a solution that is not so good may be accepted with a specific probability. Hence, in the following paragraphs, we briefly discuss two common trajectory-based methods: simulated annealing and variable neighborhood search algorithm.

• Simulated Annealing Algorithm (SA): SA is a meta-heuristic algorithm developed by Kirkpatrick et al. in 1983 to solve optimization problems. The SA method is based on the analogy between the annealing process in physical systems that minimizes the energy state of the solids and the solution process in combinatorial optimization problems [136].

  The SA algorithm starts with an initial solution and a relatively high-temperature value to avoid being trapped by the local minimum. At each iteration, the algorithm produces the next solution within the local neighborhood and the temperature decreases according to specific rules. A new solution that represents the energy level of the system and improves the objective function is always accepted. On the other hand, a workaround proposal that allows for an increase in the temperature of the system or allows a certain degree of divergence/deterioration from the objective function in the system is also accepted. The algorithm is conducted with a new solution if the new solution is accepted and with an existing solution if the new solution is rejected. These processes continue until the termination criteria (number of iterations, the smallest temperature value, etc.) are met.

• Variable Neighborhood Search Algorithm (VNS): The VNS meta-heuristic was developed by Pierre Hansen and Nenad Mladenovic in 1997 [137]. The VNS method, which has been continuously developed since its inception and has applications in numerous fields, is a single solution-based, static/dynamic objective function, based on various neighborhood structures (meta-heuristics other than VNS use a single neighborhood structure). Based on the systematic modification of neighborhood structures used in the search, VNS is a simple and effective meta-heuristic aimed at solving combinatorial optimization problems. Since the local minimum in any neighborhood may not be valid for other neighborhoods, the use of the multiple neighborhood structures is advantageous because it enables the best solutions in different regions of the search space to be obtained. In addition, these neighborhood structures are systematically changed during the search process. Thus, by providing diversification in the search space, the disadvantage of being stuck in the local optimum can be overcome. VNS offers significant advantages over other algorithms due to its simple structure, integration with different solution techniques and it requires few parameters.

7.1.3. Exact Methods

In the previous sections, the focus was on heuristics and meta-heuristic algorithms, which are generally referred to as black box algorithms in the literature. In addition to these, there are also exact solution methodologies, which are usually referred to as white box optimization algorithms. The difference between white box and black box optimization algorithms lies in their approach to problem-solving. Black box optimization algorithms are used when the internal structure of the system to be optimized is not fully known. These algorithms treat the system as a black box and attempt to optimize it using only input and output as meta-heuristic methodologies. White box optimization algorithms, on the other hand, are used when the internal structure of the system is fully known [138]. These algorithms model the system mathematically and then optimize it based on that model. Examples of such algorithms include mathematical programming, derivative-based methods, and dynamic programming. One of the key advantages of white box exact optimization algorithms is their ability to provide a globally optimal solution to a problem [139]. This means that the solution found is guaranteed to be the best possible solution, given the constraints of the problem. However, these algorithms can be computationally expensive and may require significant computing resources to run. Overall, white box exact optimization algorithms are a powerful tool for solving complex optimization problems and can provide valuable insights into the behavior of complex systems [140].

There are various exact solution methodologies in optimization problems. Linear Programming (LP), Integer Programming (IP), and Mixed-Integer Programming (MIP) [141,142] are methods for optimizing linear and discrete problems with constraints. LP seeks to optimize a linear objective function subject to linear constraints, IP requires integer variables in addition to the linear constraints, and MIP allows for both continuous and discrete variables and is useful for solving complex optimization problems with both linear and non-linear relationships. Quadratic Programming (QP) [143] optimizes a quadratic objective function subject to linear constraints, and Convex Optimization is a type of optimization where the objective function and constraints are convex [144]. Nonlinear Programming (NLP) optimizes a non-linear objective function subject to non-linear constraints, while Mixed-Integer Nonlinear Programming (MINLP) [145] combines NLP and IP. Global Optimization finds the global optimum of a non-linear objective function, which may have multiple local optima [146]. Finally, dynamic programming [147] is a method for solving optimization problems that can be decomposed into smaller subproblems. It involves breaking down a complex problem into simpler subproblems and solving each subproblem only once, saving the solutions in a table or matrix. By reusing the solutions to the smaller subproblems, dynamic programming can greatly reduce the computational complexity of solving the larger problem. Dynamic programming is commonly used in fields such as computer science, operations research, and economics, where problems can be divided into smaller parts and optimal solutions can be computed by working backwards from the end of the problem. All of these methods have applications in various fields such as finance, engineering, and machine learning.

Exact white box optimization methods are typically programmed using mathematical modeling languages, such as AMPL [148], Pyomo [149], or JuMP [150]. These optimization solvers use a range of algorithms and techniques to find the optimal solution to an optimization problem, including simplex method [151], interior-point methods [152], branch-and-bound [153], and branch-and-cut [154]. The choice of solver and algorithm depends on the characteristics of the problem, such as the size and complexity of the model, the presence of integer or nonlinear variables, and the desired level of solution quality.

7.2. Machine Learning Algorithms

ML are a class of algorithms that can learn from data without being explicitly programmed [155]. Generally, they can be classified according to the amount and type of supervision they get during their training period. There are six major categories of ML algorithms: supervised learning, unsupervised learning, semi-supervised learning, deep learning (DL), reinforcement learning (RL), federated learning (FL). These categories can be seen in Figure 7.

7.2.1. Supervised Learning (SL)

In SL, the training set comprises the input data set and the desired output, referred to as labels. SL learns a function (a match between the input data and the result data) by extracting information from the input data and the labels fed into the machine [156]. SL problems are divided into two main categories, which are regression and classification [157]. In regression problems, a continuous output is predicted; that is, the input variables are mapped to a continuous output. Examples of regression SL methods are Linear regression, ridge regression, step-wise regression, etc. [158]. In classification problems, a categorical output is predicted. That is, the input variables are mapped to different output classes, which could be a binary class (comprising two classes) or multi-class (comprising many classes). Logistic regression, naïve Bayes classifier, k-nearest neighbor, and support vector machine are examples of classification SL methods [157].

7.2.2. Unsupervised Learning (UL)

UL algorithms work with a set of inputs. The input data set for training does not have a labeled output. For this reason, in UL, clustering, association, and pattern discovery are performed over existing data. It has a working mechanism that is different from SL. The purpose of UL is to enable the identification of patterns within the training datasets and categorize input objects according to the patterns defined by the system [159]. These algorithms are expected to develop specific outputs from unstructured inputs by looking for unexplored relationships between each instance or input object. UL algorithms can be classified into three main groups: clustering, association analysis, and dimensionality reduction. K-means, K-median, hierarchical clustering, and expectation-maximization are the most common examples of clustering category [160,161]. APRIORI, Eclat and FP-Growth are examples of association analysis models [162]. Principal component analysis (PCA) and linear discriminant analysis (LDA) are examples of the dimensionality reduction category [163].

7.2.3. Semi-supervised Learning (SSL)

SSL is an approach that combines a small amount of labeled data with a large amount of unlabeled data in the training stage. It is a hybrid form of supervised and unsupervised learning with weak supervision features. A small amount of labeled data can play an important role in improving learning accuracy. While labeling the whole training data set is not a feasible solution, labeling some of the data set incurs an acceptable cost. Thus, SSL algorithms can be developed by hybridizing clustering and classification algorithms. It enables us to obtain the most suitable samples in the data set by collecting the data according to their similarity with the clustering step. Then, the data obtained from each cluster are labeled and used in the training of the supervised ML algorithm for classification [164].

7.2.4. Deep Learning (DL)

DL is a branch of ML that mimics the functioning of the human brain in the data processing. It allows machines to learn without human supervision. It gives the ability to perceive spoken, translate, identify objects, and make decisions. Despite being a branch of ML, DL systems do not have limited learning capabilities like traditional ML algorithms. Instead, DL systems can continually improve their performance as they are fed larger and more consistent data [165].

DL was realized with the help of artificial neural networks. Numerous sensors, the artificial counterparts of neurons, have come together to form artificial neural networks. The term “deep” is used to denote the number of hidden layers that neural networks have. Each layer of the neural network processes the input data and transmits them to the next layer until the data reach the output layer, where the desired output is obtained. DL algorithms have a sophisticated architecture comprising in-built feature extraction and representation capabilities absent in traditional ML algorithms. This enables them to process very complex data such as images, videos, natural language, play games, etc. [166].

Another way to understand how DL works is to look at convolutional neural networks (CNNs). Convolutional neural networks detect features directly, eliminating the need for manual labeling of data. None of the features are previously taught to the machine; instead, it trains itself on the given set of images. This automatic feature detection capability makes DL models useful for object classification and other computer vision applications. The reason deep neural networks are so sensitive in identifying features and classifying images is because of the hundreds of layers they contain. Each layer learns to describe certain features, and as the number of layers increases, the complexity of the learned image features changes in direct proportion [166,167,168].

7.2.5. Reinforcement Learning

Although it is not completely different from supervised and unsupervised learning methods, RL imitates humans’ learning process. It shows how a system can perceive its environment and learn to make the right decisions in order to reach its goal. It differs from both supervised and unsupervised learning in that the agent is not given any prior knowledge of the environment, such as input data and output data, but gathers information about the environment by interacting with the environment and learning to take the right action in any given situation (for example, by repeated trial and error over a period). This method is frequently used in fields such as robotics, game programming, disease diagnosis, and factory automation [169,170].

RL approach consists of some basic elements. One of them is the state, which is a set, S, containing the state and environment of the agent. Another element is the set of actions, A, which constitutes the whole of actions that the agent can perform. Apart from the S and A sets, there are the principles that determine the agent’s actions in the action process and the rules that describe the agent’s observations. In this context, an agent interacts with the environment by performing an action. Evaluates the new state and the rewards obtained by observing the environment. Actions and observations follow each other in a cycle. In this cycle, each action gives feedback to the agent. The learning process is normally guided by a reward function which is a measure of how much the action that the agent takes in particular state contributes to archiving the desired goal. There are three main categories of RL, including policy-based, e.g., reinforce algorithm, value based, e.g., multi-armed bandit, Q-learning, and SARSA, and actor-critic RL [171].

7.2.6. Federated Learning

FL is an emerging model that is used for enhancing data privacy in DL through distributed learning and centralized aggregation. This means that in FL, the learning algorithm is trained across multiple decentralized edge devices and servers containing local data samples without exchanging information among the various entities [172]. However, FL enables advanced devices to collaboratively train a shared model but without sharing their local data. Confidentiality and security of both central and local parameters are important when developing applications in FL. Different privacy algorithms are used for this. The application of FL enables the development of secure ML models, especially in sectors such as health, education, and banking, where data privacy is important.

FL can be divided into three different phases. These phases usually start with a basic model that is trained on a central server. In the first step, this generic model is sent to the client devices of the application. These local copies are then trained on the local data generated by the client systems, and the performance of the ML model is improved in the desired direction. In the second step, the parameters learned by the ML models are sent to a public server. These processes occur periodically on a specific schedule. In the third step, the central server collects these parameters. After the parameters are collected, the central model is updated and shared with the clients again [173,174].

Privacy is the most important advantage of FL. In addition, the reduction or elimination of network delays while working on various devices shows the importance of the FL approach in terms of efficiency. In addition, the costs of sharing the data with the server are eliminated. However, the FL approach has the challenges of synchronizing multiple clients as well as maintaining data homogeneity. Therefore, to implement FL, ML practitioners need to adopt new tools and a new way of thinking regarding model development, training, and evaluation without direct access to or classification of raw data while considering communication cost as a limiting factor [175].

8. Energy Optimization Techniques in UAV-Based Cellular Networks

Energy optimization is very important in UAV-based cellular networks because it prolongs the battery lifetime of the UAV-BSs, thereby reducing the OPEX; it minimizes greenhouse gas emissions; and it also reduces the CAPEX as it enables the deployment of fewer UAVs [13,19]. Hence, as the next generation of cellular networks envisions the adoption of both airborne and ground-based BSs in a three-dimensional deployment scenario, there is a need to more carefully consider the energy consumption of UAV-BSs in order to prevent their energy consumption from escalating. This would not only result in reduced electricity bills for mobile network operators but, more importantly, it would lead to a reduction in greenhouse gas emissions to enable both government and corporate organizations to achieve the net-zero emission drive [18].

In Section 6, we considered the four major categories of energy optimization in UAV-based cellular networks, while in Section 7, we presented an overview of some of the popular algorithms, both conventional and ML, that can be applied for energy optimization in UAV-based cellular networks. This section reviews the specific approaches proposed in the literature for energy optimization in UAV-based cellular networks and the specific algorithms used in implementing such approaches. Hence, for each deployment type, standalone UAV networks and UAV-assisted cellular networks, we first identify various approaches for energy optimization. Then, under each approach, we review the specific conventional and ML algorithms used to minimize the network’s energy consumption.

8.1. Energy Optimization of Standalone UAV Deployments

In this subsection, we consider the approaches that deal with optimization of the energy consumption of only the UAV-BSs, without considering the effect of UAV deployment on the overall energy consumption of the network. In this regard, only the communication or propulsion or both energy consumption components of the UAV-BS are considered. In addition, both single and multiple UAV-BS(s) deployments are considered as well as the various algorithms that have been proposed.

8.1.1. Positioning and Placement

Here, we review the conventional and ML algorithms used for optimal positioning or placement of the UAV-BS in order to minimize their energy consumption. The positioning includes optimizing the location, altitude, and radius of coverage of the UAV-BS to maximize EE while ensuring QoS of the ground users. We consider the positioning of single/multiple UAV-BS/BSs.

Conventional Approaches

In [116], the authors considered the problem of energy efficient 3D placement of a UAV-BS for coverage maximization. The problem was first modeled as a circle placement problem and a heuristic algorithm was proposed to determine the optimal 3D location that maximizes the coverage area while minimizing the transmit power. The proposed technique outperforms the random deployment of UAV-BS throughout the whole coefficient of variation range (CoV). The suggested technique can provide coverage for up to 70 users at a transmit power of 25.5 dBm when the CoV is 6, whereas the random deployment approach only covers 22 users at a higher transmit power of 30 dBm. The work in [23] investigated the cost and energy optimization of a UAV-based communication network while considering both the communication and propulsion energy consumption. In this regard, a multi-level circle parking (MCP) algorithm was developed to determine the optimal 3D-hovering positions of the UAVs that maximizes both the uplink and downlink global EE of the network. In addition, the result of the optimal hovering positions obtained was used to determine the number of UAV-BSs and flight parameters required to minimize the total system cost. The authors in [24] proposed a deployment decision mechanism for optimizing the number and locations of UAV-BSs in a UAV-assisted vehicular network to maximize the communication coverage and minimize the energy consumption of the UAV-BSs. The proposed mechanism employs circle packing theory to determine the optimal positions of the UAV-BSs, while an energy optimization model was developed to minimize the power consumption of the UAV-BSs.

The authors in [176] developed an analytic solution to determine the optimal altitude for a UAV-BS whereby the transmit power needed to provide coverage to a specific area is minimized. In [177], an EE maximization approach was proposed for a UAV-BS relay system to extend the battery life while maintaining network throughput. In the proposed approach, the hovering position, where the UAV-BS expends the least energy, is considered to be the optimal UAV-BS location, is determined via mathematical analysis, after which the power allocation was also optimized. The work in [178] considered the optimal positioning of a UAV-BS in order to maximize its EE with the altitude and minimum user data rate being constraints. The EE problem was formulated as a monotonic fractional optimization problem and solved using polyblock outer approximation algorithm. Two UAV location optimization algorithms were proposed in [179] to minimize the transmit power of the UAV-BS. The first algorithm assumes equal power allocation, while the second algorithm is based on successive convex approximation (SCA) and does not assume equal power allocation.

The authors of [119] proposed a UAV-BS positioning algorithm based on Coulomb’s law to maximize the EE of the UAV-BSs while considering interference between UAV-BSs and user requirements. The algorithm presented in this article outperforms existing solutions by at least 2.5 times while improving customer satisfaction in terms of data rate by about 6% to 10%. An energy-aware 3D deployment algorithm based on Lagrangian and sub-gradient projection for optimal placement of the UAV-BSs was proposed in [180]. In [181], the authors developed a framework for optimizing the energy consumption of individual UAV-BSs in a multiple UAV-BSs network while carrying out location-specific tasks. The proposed framework uses order-K Markov predictor to estimate the task locations to enable proactive deployment of UAV-BSs and minimize their energy consumption. In addition, a heuristic algorithm was developed to place the UAV-BSs in their right locations as well as assign their respective tasks to them. The authors in [118] investigated the optimal 3D placement of a UAV-BS with a tilting antenna to provide sufficient coverage for ground users while utilizing minimum energy consumption. A gradient descent algorithm was then proposed to find the optimal altitude of the UAV-BSs.

The authors of [182] considered the importance of the on-board circuit power consumption of the UAVs while addressing the problem of their optimal 3D placement in order to maximize the network lifetime. Then, using an analytical approach, the optimal hovering altitude of the UAV-BSs with respect to their coverage radius was derived to determine the coverage and on-board circuit power parameters that result in minimum power consumption. The work in [183] considered the energy efficient placement of UAV-BSs for data collection from ground users based on NOMA. A heuristic algorithm was proposed to determine the optimal hovering height of the UAV-BS that maximizes the EE of the network. The authors in [184] proposed a joint 3D location and transmit power optimization scheme for UAV-based relay networks to maximize the sum rate of users. A heuristic algorithm based on alternating descent and SCA was developed to solve the optimization problem.

The authors in [185] proposed a joint optimization scheme for both the 3D placement and pathloss factor with the aim of achieving maximum energy-efficient coverage. A heuristic algorithm was developed to find the optimal UAV placement and compensation factor that maximizes the energy efficient coverage. An optimal UAV placement framework that aims to find the optimal UAV locations that is required to minimize the total energy consumption of the network while providing a target coverage was introduced in [186]. Both centralized and localized heuristic algorithms were developed to determine the optimal UAV locations for both static and mobile users. The authors of [187] considered the joint optimization of the transmission power and location of UAV-BS in a relay NOMA network to minimize the power consumption of the network. A double loop iterative algorithm was developed to solve the joint optimization problem. In [188], the optimal 3D placement for UAVs serving as relays in IoT communications was considered in order to minimize the transmission power of the UAVs while considering the outage probability of the IoT devices. A 3D placement algorithm based on PSO was developed to minimize the transmitted power in both air-to-ground and ground-to-air links.

The case of energy efficient UAV placements in indoor environments for emergency wireless coverage was considered in [189]. Both iterative and exhaustive search algorithms were developed to determine the optimal position of the UAV in order to minimize the transmission power. Similarly, the authors in [190] investigated the optimal positioning of a UAV-BS for seamless IoT connectivity in an indoor environment comprising multiple users at random locations in order to minimize the transmit power of the UAV-BS. An energy-efficient low-complexity heuristic algorithm was developed to solve the optimal UAV placement problem. The authors in [191] proposed a UAV-BS deployment and scheduling mechanism to ensure optimal placement and effective management of UAV-BS operations while minimizing energy consumption and ensuring maximum coverage. To achieve these objectives, heuristic algorithms were proposed to ensure the UAVs are placed in the right locations as well as manage their battery recharging cycle.

The authors of [192] investigated EE maximization in UAV-assisted NOMA based network via a joint optimization of UAV placement and power allocation while considering QoS constraints. The joint optimization problem was modeled as a non-linear fractional problem, then an alternating algorithm based on a nested Dinkelbach structure was proposed to find the optimal solution. The work in [193] studied the joint optimization of the UAV location and transmit power in a NOMA-based UAV network while considering the decoding order. The joint optimization problem was first divided into two sub-problems, after which an iterative algorithm was proposed to solve the optimization problem alternately. The authors in [194] proposed an energy efficient transmission mechanism for UAV-enabled mmWave communication system with NOMA by jointly optimizing the UAV position, power allocation, and precoding in order to maximize user coverage and minimize the energy consumption of the UAVs. Due to the complexity of the optimization problem, it was first divided into three sub-problems, and three heuristic algorithms were designed to solve each problem iteratively.

The authors of [186] proposed algorithmic optimization techniques for determining the optimal locations to place drones in order to cover designated targets in a cost-efficient manner. The proposed methodologies consider factors such as target locations, drone capabilities, and communication constraints to minimize the number of drones needed while maximizing target coverage. To address the problem of optimal drone placement and cost-efficient target coverage, the article proposes two mathematical models: an integer linear optimization model and a mixed integer non-linear optimization model. These models were used to formulate the problem mathematically and find the optimal solution that minimizes costs while maximizing target coverage. In another study, [195] proposed a two-phase process where the first phase involves optimal UAV placement and the second phase involves resource allocation to the UAVs. The optimization objectives include minimizing the total energy consumption and maximizing the network coverage and data rate. The approach utilizes a combination of linear programming techniques and heuristic algorithms to achieve the desired objectives.

The authors of [196] proposed an energy-efficient framework for data gathering using multiple UAVs in WSN applications with deadlines, based on clustering techniques. The proposed framework aims to minimize energy consumption while meeting the application’s deadline. The article uses the MILP approach and greedy heuristic solution methodology as part of the proposed solution for optimizing the data gathering problem in WSN applications using multiple UAVs.

Machine Learning Approaches

The authors of [197] proposed a proactive power control and positioning framework for UAV-BSs to minimize interference and enhance EE in multi-UAV systems. The proposed framework comprises both offline and online phases. In the former, a supervised learning algorithm (random forest) leverages historical data to build a mobility prediction model, while in the latter, the predicted user positions are exploited to determine the sleep/wake status of the DSCs proactively, while an unsupervised ML algorithm (k-means) is employed to update DSCs positions and regulate the power consumption. The proposed approach resulted in a 97.3% increase in energy efficiency compared with GA and utilizes 0.4% of the execution time of GA-based approach An energy-efficient multi-UAV deployment framework was proposed in [198] in order to maximize user coverage probability. An ellipse clustering algorithm was developed to determine the optimal hovering altitude of the UAV that would result in minimal transmit power while maintaining QoS constraints. The simulation result reveals that the proposed method produced about a 50% reduction in the transmit power of the UAVs.

A predictive on-demand ML-based UAV deployment for minimizing both the communication and propulsion energy consumption was introduced in [199]. In this regard, a ML framework was developed, which uses Gaussian mixture model (GMM) and weighted expectation maximization (WEM) algorithm to forecast the network traffic congestion areas. Then, k-means algorithm was used to partition the service area of each UAV, after which a gradient-based algorithm was developed to determine the optimal location of the UAVs that results in minimum energy consumption. The performance evaluation reveals that the proposed ML framework is able to reduce the power consumption of the network by over 20% compared to non-ML methods. The authors in [200] considered the problem of reducing the energy consumption required to provide coverage in a multiple UAV network. In pursuit of this objective, a coverage model based on the actor–critic RL algorithm was developed to enhance the cooperation of the UAVs in order to provide energy efficient coverage. Compared to the Max-Sum method which served as the benchmark, the proposed method was able to minimize the energy loss by 46.7%.

Table 1 summarizes the conventional (CA) and ML approaches that are applied for optimal UAV-BS placement or positioning in order to minimize its energy consumption.

8.1.2. Trajectory Design and Path Planning

Here, the UAV’s flight path and speed are carefully planned and designed to ensure that it flies through the optimal trajectory, with the optimal speed and altitude that is required to serve ground users’ demands while expending the minimum amount of energy. In the reviewed studies, energy optimization might be the main objective, such that it can be one of the variables that were optimized or one of the constraints that is considered while optimizing other variables.

Conventional Approaches

In [201], the authors proposed a flight and communication protocol where the UAV-BS’s speed, height, and beamwidth are jointly designed to minimize the flight time and energy consumption while serving ground users. The authors in [113] investigated energy-efficient UAV-BS’s communication with ground users based on trajectory optimization by considering both the throughput and energy consumption of the UAV-BS. A propulsion power consumption model for fixed-wing UAVs was first introduced, then both unconstrained and constrained trajectory optimization was considered for energy efficient UAV-assisted communications. The former trajectory optimization was proven to be energy inefficient while, for the latter, an energy efficient algorithm based on both linear state space and sequential convex optimization was developed to optimize the communication of the UAV-BS.

The work in [25] considered energy-efficient UAV communication of rotary-wing UAV-BSs. A theoretical model of the mobility power consumption was first derived, followed by formulating the UAV energy minimization problem, which jointly optimizes its trajectory, time spent communicating with ground users, and mission completion time. To solve this optimization problem, two solution approaches were considered. In the first approach, a flight-hover communication design was considered where the UAV-BS only visits the optimal hovering location and communicates with the user in that location. As such, the energy minimization problem becomes that of finding the optimal hovering location, duration, visiting order, and speed; additionally, a convex optimization with traveling salesman algorithm was developed to tackle this problem. For the second solution approach, the energy minimization problem was modeled as a path selection problem, then an iterative algorithm based on SCA was developed to update both the trajectory and time allocated to the UAV-BS for communication. EE of UAV communication based on trajectory design was studied in [145]. The optimization problem comprises both the throughput and propulsion energy consumption of the UAV-BS. A heuristic algorithm based on SCA and the classical Dinkelbach approach was developed to find the optimal energy efficient strategy for the UAV-BS.

The authors of [26] considered a joint optimization of the 3D trajectory and transmission power of a multi-UAV based relaying system in order to maximize user throughput. A heuristic algorithm based on the block coordinate ascent (BCA) technique was proposed to solve the developed problem. In [202], the authors considered energy minimization problem in UAV-based networks by jointly optimizing the trajectory and velocity of the UAV while respecting latency constraints. First, both heuristic and dynamic programming algorithms were proposed to determine the optimal set of trajectories, after which the UAV velocity of each trajectory was also optimized to ensure that the total energy consumption of the network is minimized. The authors in [203] investigated energy-aware trajectory optimization that minimizes the on-board energy consumption in the UAV-BS while satisfying the data requirement of ground users. A double loop iterative algorithm was proposed to determine the optimal trajectory of the UAV that would maximize user coverage.

A combined optimization of both the trajectory and transmit power of a UAV relay network was carried out in [204]. They first derived an analytic expression for the outage probability, after which a heuristic algorithm was developed for power control and trajectory optimization. The work in [70] investigated the application of UAV-BSs for backhaul connection to terrestrial BSs during post-disaster scenarios. An energy-efficient trajectory design based on GA was proposed to select the optimal trajectory that results in the least energy consumption in the UAV. The trade-off in energy consumption between ground terminals and UAV-BSs was studied using the trajectory design in [205]. Analytical models were developed to determine the optimal UAV trajectory and ground terminal’s transmit power, resulting in the optimal energy consumption trade-off between them. The authors in [206] studied the problem of EE and security in a cooperative multi-UAV system. A joint optimization of the transmit power, user schedule, and UAV trajectory was proposed to maximize the secure EE of the network. A three-layer heuristic algorithm was proposed based on BCD and the Dinkelbach method to solve the joint optimization problem.

A UAV-BS path planning framework was proposed in [129] in order to maximize the EE of the UAV-BS. In the proposed framework, the path planning problem was first modeled using the traveling salesman problem, after which a GA was developed to determine the optimal path with minimal turns that would result in minimum energy consumption in the UAV-BS. The problem of energy-efficient UAV-based communication in the presence of several jammers was considered in [115]. To solve this problem, the UAV trajectory was optimized while considering its mobility constraints. In this regard, a heuristic algorithm based on SCA and Dinkelbach’s model was proposed to determine the optimal trajectory. A flight planning mechanism for solving point of interest visiting problems involving data collection, edge computing, and surveillance operations was proposed [207] to minimize the flight energy consumption of the UAV. The flight turning and switching cost were modeled as a graph and then transferred to a travelling salesman problem after which a heuristic algorithm was proposed to determine the optimal flight path.

The authors of [208] investigated the rate fairness and EE of a UAV based relay system. Two optimization objectives were considered: the first is the rate maximization while maintaining a specific power budget, and the second is optimizing the total power consumption while maintaining minimum rate requirements. Two heuristic algorithms were proposed to solve the optimization problem. In [209], a joint trajectory and transmit power optimization scheme for UAV-aided communications was investigated to maximize user average throughput. A heuristic algorithm was proposed to solve the joint optimization problem while ensuring that users’ throughput is maximized. Similarly, the work in [210] considered a joint trajectory and transmit power optimization in UAV-based secure communications to maximize the throughput of the system. An iterative algorithm-based BCD, S-procedure, and SCA were proposed to solve the optimization problem. The work in [211] investigated the problem of trajectory optimization to minimize the power consumption of the UAV while ensuring that user service demands are met. The optimization problem was solved using an iterative algorithm that is based on SCA.

The work in [212] investigated flight radius adjustment and routing in UAV swarm networks to minimize the total power consumption of multiple UAV systems. Two flight radius manipulation approaches were first developed to maximize the propulsion power consumption and battery lifetime of the UAVs. Then, a temporal path algorithm was proposed to minimize the power consumption of any pair of UAVs. In addition, an iterative algorithm was also developed to further adjust the flight radius to enhance the EE of the system. The authors in [213] studied the dynamic cooperation of UAVs in a multi-UAV system to enable power efficient communications. To achieve energy-efficient communications, the UAV trajectory and the cooperative beamforming were jointly optimized and then a low-complexity heuristic algorithm that is based on difference of convex (DC) was proposed to solve the optimization problem. Regarding the issue of energy consumption fairness in multi-UAV networks, trajectory optimization was considered in [214]. The problem of designing trajectories with fair energy consumption allocation was modeled using mixed-integer linear programming. To achieve this objective, two heuristic algorithms were proposed to find the optimal energy allocation strategy among the trajectories that would result in minimum energy consumption in the network.

In [215], the authors considered energy minimization in a UAV-based network with multiple eavesdroppers. The energy minimization problem was first modeled as a mixed integer optimization problem involving the UAV trajectory and user scheduling while maintaining QoS constraints. Then, an iterative algorithm based on BCD and SCA was proposed to find the optimal solution that results in minimum energy consumption in the network. An EE framework via trajectory optimization in a fixed-wing UAV—used for data collection and forwarding between ground nodes—was proposed in [216] to minimize the energy consumption while respecting throughput constraints. A heuristic algorithm based on SCA was formulated to determine the optimal strategy that would result in minimum propulsion energy consumption. The work in [217] studied EE and throughput maximization in UAV-enabled vehicular networks via joint optimization of the UAV trajectory design and power allocation. The joint optimization problem was first divided into two, and then a heuristic algorithm based on SCA was developed to simultaneously determine the optimal trajectory and power allocation strategy in each iteration. In addition, an EE optimization algorithm was developed based on fractional programming and sequential optimization.

In [218], the authors considered the joint optimization of UAV trajectory, velocity, acceleration, and transmit power to maximize the EE while maintaining the minimum user rate. An iterative algorithm based on SCA was proposed to determine the optimal solution of the joint optimization problem. A multi-UAV cooperative mechanism was proposed in [219] to maximize the EE by the joint optimization of the UAV trajectory and transmission power. An iterative algorithm based on BCD, SCA, and Dinkelbach method was employed to determine the joint EE maximization strategy. The work in [220] considered the problem of minimizing of the energy consumption in UAV-assisted relays. To achieve minimal energy consumption in the UAV, the transmit power, UAV trajectory, and time slots were jointly optimized using an iterative algorithm that is based on BCD and SCA techniques.

In another study [142], the authors proposed three algorithms for assigning energy-efficient trajectories to a fleet of UAVs, with a focus on multi-UAV path planning and assignment. These algorithms include an optimal path planning solution using MILP, as well as two greedy heuristic solutions. The study also incorporates the concept of space discretization for collision avoidance and examines the trade-offs between energy consumption and complexity. The work in [221] developed a new scheme for quality-aware and energy-efficient coverage and path planning for networks of UAVs in geometrically complex environments. The scheme includes an occlusion-aware waypoint generation algorithm, a vehicle routing problem formulation to assign waypoints to multiple UAVs within a global deadline, and a Min-Max energy path planning algorithm. Evaluation results show that the proposed scheme outperforms traditional techniques in terms of coverage and energy consumption. The methods used in the scheme include MILP techniques and genetic algorithms.

In [222], the authors conducted a UAV-enabled communication system to collect information from distributed ground terminals (GTs) while maximizing the energy efficiency of the UAV. A mixed-integer nonconvex optimization problem is formulated and divided into two subproblems for easier solving. An efficient algorithm (based on linear programming and convex optimization) is also proposed to optimize GT scheduling and the UAV’s trajectory. The proposed design outperforms two benchmark designs according to numerical results.

Machine Learning Approaches

The work in [223] considered the design of an energy-efficient distributed navigation system for a group of UAV-BSs for the provision of long-term coverage to ground users. The problem was first modeled as a distributed multi-agent control problem, then a deep RL (DRL) framework was developed to control each UAV while minimizing the total energy consumption of the UAVs and ensuring minimum coverage requirements and geographical fairness are achieved. An energy management strategy for solar-powered UAV-BSs based on deep Q-networks was proposed in [224], wherein the total energy consumption of the UAV-BSs as well as the 3D flight trajectory were jointly optimized to enhance their communication capacity. The authors in [225] studied the problem of 3D trajectory design and frequency allocation while considering the energy consumption of the UAV-BSs and the fairness of user coverage. In this regard, the energy consumption of the UAV-BS was first modeled as a function of the 3D movement of the UAV. After that, a DRL framework known as energy-efficient fair communication through trajectory design and band allocation (EEFC-TDBA), which is based on deep deterministic policy gradient (DDPG), was proposed to regulate the speed and direction of the UAV-BS as well as the frequency band allocation to maximize the EE, enabling the UAV-BS to arrive at its destination before the battery energy depletes and achieve fair user coverage.

The problem of trajectory planning for multi-UAV system was considered in [226] to achieve energy-efficient user coverage. The multi-UAV trajectory planning problem was first modeled as two multi-agent cooperative games, followed by developing a decentralized multi-UAV cooperative RL (DMUCRL) framework based on Q-learning to find the equilibrium of the games. Using the proposed framework, the UAV-BSs can choose their optimal trajectory and the recharging schedule that would result in minimum energy consumption in the network. The performance evaluation showed that the proposed DMUCRL framework is able to enhance the energy efficiency of the network by 27% compared to straight flight trajectory planning. An ML framework for joint trajectory design and power control to maximize user sum rate in a multi-UAV wireless network was proposed in [227]. The solution procedure involves three stages. First, a multi-agent Q-learning algorithm was developed to find the optimal UAV positions using the initial positions of users. Second, exploiting real user mobility data set obtained from Twitter, an algorithm was developed to estimate future user locations based on echo state network. Finally, another multi-agent Q-learning algorithm was developed to forecast UAV positions in each time slot based on user mobility. The proposed method resulted in about 17% increase in the throughput of the network.

The work in [228] investigated the problem of UAV trajectory optimization for post-disaster management while considering the availability of energy as constraint. To solve this problem, an energy-aware multi-armed bandit algorithm was proposed to determine the optimal trajectory that would maximize the number of ground users visited while respecting the energy constraints. Similarly, in [229], the authors studied the application of UAV-BS for the provision of wireless coverage following a natural disaster when the user distribution is not known. They aimed at finding the optimal path that would maximize user service request while considering the battery capacity of the UAV. The path selection problem was modeled as an extended multi-armed bandit problem and two algorithms: distance aware–upper confidence bound and $ϵ$-exploration algorithm were proposed to determine most energy efficient path. Simulation results reveal that the proposed method outperforms baseline methods by 85% and 100%, respectively, with respect to the number of users served. The authors in [230] considered the problem of resource-aware data offloading in a MEC-based UAV swarm network. Their goal is to select the optimal energy and processing-optimized path between the UAVs that would complete a data offloading task. A multi-armed bandit-based data offload path selection mechanism was proposed to ensure the survival of the selected path even when the size of the data to be transmitted is very large.

The authors of [231] proposed an intelligent and energy-efficient framework to learn the optimal multi-UAV trajectory planning and data offloading strategy to ensure information freshness in delay-sensitive applications. A multi-level DRL framework known as decomposed multi-agent DDPG (DE-MADDPG), which Combines DDPG and MADDPG, was developed to learn the optimal collaborative UAV control policy that would significantly minimize energy consumption and ensure global information freshness in a dynamic changing environment. Simulation results show that the proposed DE-MADDPG framework outperforms the DDPG and MADDPG approaches by 1.7% and 4.6% in terms of total reward obtained as a result of UAV cooperation. The authors in [232] considered the problem of optimal path planning for a single UAV-BS as well as a fleet of UAV-BSs to enhance user coverage while respecting energy constraints and minimizing collisions between UAV-BSs. An RL-based approach was proposed for the single UAV-BS case while a DRL-based approach was proposed for the multiple UAV-BSs scenarios. The proposed methods exceeded the performance of benchmark methods (greedy and random method) with respect to total network coverage, the number of users covered, and system fairness by an average of 15.04%, 42.54%, and 38.94%, respectively, while respecting energy constraints. The authors in [233] investigated the problem of 3D UAV-BSs trajectory design to enhance the energy efficiency of the network while satisfying the QoS requirements of ground users. To achieve their goal, the energy-efficient trajectory design problem was modeled as a multi-armed bandit and solved using an algorithm that is based on the upper confidence bound. The performance evaluation shows that the proposed approach resulted in a 95% and 33.85% increase in the energy efficiency and throughput of the network, respectively, compared to the Q-learning-based approach.

The authors of [234] investigated the problem of energy optimization of UAV-BSs for data collection application via a trajectory optimization technique. A Q-learning-based algorithm was proposed to determine the optimal trajectory selection strategy that will minimize the energy consumption of the UAV while achieving coverage targets. An ML-based framework was introduced in [235] to jointly optimize the multi-cast and trajectory for EE enhancement in UAV-based multi-cast networks. A combination of support vector regression (SVR) and k-means algorithm was first proposed to determine the optimal multicast grouping, after which a centroid adjustable traveling salesman-based algorithm was proposed to determine the optimal trajectory to maximize the energy-saving performance of the UAV. A joint trajectory and power optimization scheme based on DRL was proposed in [236] to maximize both the EE and throughput in a UAV-based communication system. The proposed scheme employs DDPG to solve the optimization problem in order to achieve the desired objective. The proposed technique is able to maximize the flight speed of the UAVs by 14% and enhance the throughput by 10% compared to the random method.

The CA and ML approaches that are applied for UAV-BS path planning and trajectory design in order to minimize its energy consumption are summarized in Table 2.

8.1.3. Resource Management

In this part, we review various resource management techniques involving bandwidth and power allocation and control methods that have been developed using conventional and ML algorithms to minimize the energy consumption of UAV-BSs.

Conventional Approaches

The authors of [237] proposed an energy-efficient multi-UAV deployment framework to maximize the coverage of a UAV network based on the game theory. The optimization problem was first divided into two parts: maximizing the coverage and power control with both being potential games. Then, a multi-UAV deployment algorithm based on spatial adaptive play was developed to tackle both power control and coverage maximization. An energy-efficient power allocation scheme for UAV-enabled spatial NOMA was proposed in [238]. The EE maximization problem comprising the power consumption, signal capacity, and spatial gain was formulated, after which a heuristic algorithm was developed to determine the optimal power allocation strategy. In [239], a resource allocation scheme, comprising user scheduling and power allocation in order to maximize the EE of a NOMA UAV networks under imperfect channel state information, was introduced. The proposed scheme was implemented using a heuristic algorithm for user scheduling to match the user to their sub-channels and a power allocation algorithm based on SCA was proposed to maximize the EE of the network.

The work in [240] considered the joint resource allocation and trajectory optimization in order to enhance the EE in multi-UAV-based networks. An iterative heuristic algorithm based on SCA was proposed to solve the optimization problem. A joint power allocation and trajectory design mechanism for EE optimization in UAV-based relay systems was proposed in [241]. The optimal power allocation strategy was determined using a heuristic algorithm based on Lagrange multiplier method while the optimal trajectory design was determined using SCA. Then, an iterative algorithm which combined both algorithms was also proposed. The authors in [242] exploited UAV for confidential information communications in the presence of eavesdroppers. To achieve this, a power splitting mechanism was proposed to enable the UAV to transmit both confidential information and artificial noise at the same time. A joint optimization problem comprising the transmit power levels, power splitting ratios, and UAV trajectory was formulated and solved using an iterative algorithm based on BCD, concave convex procedure (CCP), and alternating direction method of multipliers (ADMM).

The work in [243] considered maximizing the EE of a UAV-based secure communication system via the joint optimization of resource allocation and trajectory design. An iterative algorithm was developed to determine the efficient sub-optimal strategy. In [244], the problem of power minimization in an RIS-assisted UAV network was considered by jointly optimizing the UAV resource allocation mechanism and flight path with the help of an alternating heuristic algorithm. The authors of [245] examined the problem of power allocation in a multi-UAV based secure network while considering both small- and large-scale channel fading. Their objective was to maximize the throughput with transmission power, total energy consumption due to transmission of each UAV, and the duration of transmission as constraints. An analytical model was first developed for the throughput after which a heuristic algorithm was developed to solve the optimization problem.

A joint optimization of the UAV deployment and power allocation was performed in [246] to maximize the area of service coverage and enhance EE. The optimization problem was decomposed into two separate problems involving the determination of the optimal transmit power control and 3D location. Using game theory, a utility function was developed to model the cooperation between both problems, which was proven to have Nash equilibrium points. Then, a learning algorithm based on binary log-linear was proposed to find the equilibrium point in terms of coverage and EE maximization. A resource allocation framework involving the joint optimization of time and power resources was proposed in [247] to maximize the EE and throughput in NOMA based UAV network. To solve the optimization problem, a two-phase heuristic algorithm was designed: in the first phase, the optimal power allocation strategy required to maximize the EE is determined while the optimal time resources to enhance the throughput of all users are determined in the second phase.

In [27], the authors examined UAV-assisted traffic offloading for EE enhancement in UAV-aided cellular networks. Their aim was to maximize the UAV-BS’s EE through optimal resource allocation, trajectory design, and user scheduling. An iterative algorithm based on BCD was developed to solve the joint optimization problem. The authors in [242] proposed a joint optimization of the power allocation and trajectory design to maximize the secrecy average rate in UAV-based secure communications. An iterative algorithm based on BCD, CCP, and ADMM was developed to solve the joint optimization problem. An energy-efficient resource allocation framework was proposed in [248] to optimize computation offloading while ensuring the minimum energy consumption from the UAV. To achieve their target, a joint optimization of the UAV trajectory, computation task, and user transmission power was carried out, which is then solved by a heuristic algorithm based on SCA and Dinkelback.

In [249], the authors addressed the energy-efficient resource allocation problem in a two-hop uplink communication for Space-Air-Ground Internet of remote things (SAG-IoRT) networks assisted with unmanned aerial vehicle (UAV) relays. The goal is to maximize system energy efficiency by jointly optimizing sub-channel selection, uplink transmission power control, and UAV relays deployment. An iterative algorithm that combines two sub-problems is proposed for decomposition and convex approximation methodologies. Numerical results show that the proposed algorithm outperforms the benchmark scheme by at least 21.9% in terms of system energy efficiency. In another study [250], the authors worked on an energy-efficient multi-UAV-assisted multi-access edge computing (MEC) system where mobile devices offload computation-intensive and delay-sensitive tasks to UAVs equipped with MEC servers. However, limited energy budgets at UAVs and task completion deadlines make it challenging to determine device association and task offloading. To solve this, the paper proposed a joint optimization problem for device association, task assignment, and computing resource allocation, which is a mixed-integer nonlinear programming (MINLP) problem. The problem is decomposed into three subproblems and solved using an iterative block coordinate descent (BCD) algorithm, which outperforms the benchmark schemes in simulations.

In [251], the authors proposed a multicriterion framework for energy-efficient radio resource management (RRM) in a cooperative Internet-of-Things (IoT) network using UAV-based relays with radiofrequency (RF) energy harvesting. The proposed framework considers three conflicting objectives, namely, maximizing the network sum rate, maximizing the number of IoT devices served, and minimizing carbon dioxide emissions. To solve the formulated mixed-integer nonlinear programming problem, the authors propose an outer approximation algorithm (OAA) and compare its performance with the two most known algorithms.

Machine Learning Approaches

The authors in [252] proposed an intelligent scheme based on Q-learning to determine the optimal power allocation and 3D position of multiple UAV-BSs during emergency situations to maximize the coverage of ground users. The proposed method led to a reduction in the number of UAV deployed by 56%, minimized the percentage of users in outage, and significantly reduced the transmit power of the UAVs. In [253], the authors addressed the problem of trajectory design and power allocation for multi-UAV networks by proposing a multi-agent DDPG algorithm to find the optimal strategy to jointly optimize the trajectory and transmission power. Numerical results reveal that the proposed method results in improved network capacity with reduced computational overhead. A DRL scheme for jointly optimizing the 3D-deployment and power allocation of a UAV-BS was proposed in [28]. The proposed scheme leverages DDPG and water filling to determine the optimal 3D position as well as the power allocation strategy while maximizing system throughput. The downlink power control problem in ultra-dense UAV networks was investigated in [254] for EE enhancement purposes. In this regard, mean-field game (MFG) was used to model the power control problem after which a DRL algorithm based on deep Q-networks was developed to determine the optimal power control policy that would maximize the EE of the UAV network. Simulation results proved that the proposed DRL framework outperformed the Q-learning and random approaches in terms of learning the optimal power control even at different UAV heights. An energy efficient resource allocation scheme for UAV-based edge computing systems that considers the power control, user association, computation resource allocation, and location planning for UAV was proposed in [255]. An iterative approach based on a fuzzy c-means clustering algorithm was proposed to solve the joint EE minimization problem. The performance evaluation showed that the proposed approach results in lesser power consumption compared with conventional methods during periods of low latency, low computation capacity, and high CPU cycles. Table 3 summarizes the CA and ML approaches that are applied for resource management in UAV-BSs in order to minimize their energy consumption.

8.1.4. Flight and Transmission Scheduling

Here, we consider studies that involve the scheduling UAV-BSs flights as well as their sequence of transmission in order to ensure that minimum energy is consumed in the process. We also examine research works that investigate how the fight of UAVs are coordinated and scheduled or how UAVs cooperate to achieve energy-efficient communication.

Conventional Approaches

A flight control framework for multi-UAV systems in order to minimize the energy consumption of the UAV was proposed in [256]. The velocity control of each UAV was first modeled using a MFG theory, after which a Cucker–Smale (CS) flocking algorithm was developed to control the velocity in order to achieve minimum energy consumption in the network. An MDP-based framework for jointly scheduling data transmission and task computations in order to maximize the EE of a UAV-based network was proposed in [257] while considering a stochastic trajectory. A probabilistic scheduling algorithm based on linear programming was proposed to solve the joint transmission and computation scheduling problem in order to minimize the total energy consumption of the network.

The authors in [29] studied the formation of swarms in multiple UAV networks to minimize the energy consumption of the UAVs. A joint optimization problem comprising transmit power, beamforming vectors, and swarm formation was formulated, after which a heuristic algorithm was developed to determine the optimal swarm formation that would result in minimum energy consumption in the UAVs. The authors in [258] considered the problem of EE maximization in a UAV-based relay system used for secure communication. The EE is maximized by jointly optimizing the transmission scheduling, power allocation, and UAV trajectory. The optimization problem was solved using an SCA-based iterative algorithm. The work in [259] studied a joint flight scheduling and resource allocation in a multi-UAV network to maximize the EE of the network. Two approaches, coordinated and uncoordinated, were proposed to solve the joint optimization problem. In the former, UAVs cooperate with each other to maximize network EE via information exchange, while, in the latter, each UAV maximizes its EE separately with no information exchange. Then, heuristic algorithms were proposed to obtain the optimal solution in each approach.

The authors in [260] investigated scheduling of beaconing periods to minimize the energy consumption for UAV-based networks during emergency scenarios. The beaconing periods of the UAVs was modeled as a non-cooperative game and then a distributed learning algorithm was proposed to determine the Nash equilibrium of the game that would result in minimum energy consumption in the network. Similarly, the authors in [261] considered the problem of activity scheduling for UAV-BSs that are used to serve users in areas, which do not have cellular infrastructure, to maximize the EE of the UAVs. Then, a heuristic algorithm based on Nash bargaining theory (NBG) was proposed to determine the optimal beaconing period that maximizes the EE of each UAV-BS. In [262], the problem of multi-task cooperation in a multi-UAV system was investigated to minimize the energy consumption of the UAV. The task assignment problem was first modeled as a coalition formation game (CFG) and a heuristic algorithm was developed to solve the task allocation problem resulting in reduced energy consumption in the network.

Machine Learning Approaches

One of the ways of achieving energy efficient UAV operation, particularly for multiple-UAV networks, is the proper scheduling of their mobility as well as battery recharging cycle. In this regard, the authors in [30] proposed an energy-efficient and fair 3D UAV-BS framework for scheduling the mobility of UAV-BSs as well as timely recharging of the UAV battery. A DRL algorithm based on DDPG was employed to solve the joint optimization of both the mobility and charging cycle of the UAV-BSs, leading to maximization of EE and achievement of fair user coverage. The work in [263,264] considered the problem of user scheduling in UAV-based communication networks to minimize the energy consumption of the UAVs. Two offline optimal heuristic algorithms were proposed to solve the problem. However, since these solutions might not be suitable for online operations, an online actor–critic-based deep stochastic online scheduling (AC-DSOS) algorithm with less computational complexity was developed to solve the user scheduling problem. The proposed method resulted in a 25-30% reduction in energy consumption of the network compared to two DRL-based benchmarks. The CA and ML approaches that are applied for UAV-BSs flight and transmission scheduling in order to minimize their energy consumption are summarized in Table 4.

8.1.5. Landing Spot Concept

Most of the energy optimization techniques considered in the preceding paragraphs require the UAV to be in constant flight or hovering position in order to serve user requests. This greatly limits the amount of energy savings that can be obtained as the UAV consumes a significant amount of energy during flight or hovering [13,19]. Hence, alternative deployment approaches need to be developed which can reduce the flying or hovering time of the UAV-BSs in order to further enhance the energy saving obtained from these energy optimization techniques. To address this issue, the concept of a landing spot was introduced in [265] such that the UAVs can land on some designated locations such as roof top of tall buildings, lamp post, or some specially designed platforms which can also be equipped with charging pods, rather than having to hover continuously to serve user request and expend so much energy, which is a major challenge for battery limited UAVs. The few studies that consider the landing spot concept are discussed in the following paragraph. However, due to the very limited studies on this concept, the established taxonomy of categorizing the studies based on conventional and ML approaches is not followed here.

The authors in [31] performed a capacity comparison between landed and hovering UAV with the aim of determining which approach will be suitable for adoption. Their finding reveals that the choice of a suitable deployment option depends on certain factors including the number of UAVs deployed, the distance between the charging stations and service area, and the capacity of the UAV battery. The work in [32] proposed a deep Q-learning approach for optimizing UAV trajectories using the land spot where the UAVs do not have to continuously fly along the trajectory but can land at some locations along its path in order to minimize energy consumption while meeting user demands. More research works need to be conducted in this direction to determine the optimal locations where UAVs can land along their trajectory as well as the optimal separation distances between the UAV optimal hovering positions and the suitable landing spots in order to improve the amount of energy savings while respecting the QoS constraints of the users.

8.2. Energy Optimization of UAV-Assisted Cellular Networks

In the previous subsection, we reviewed the cases where only the UAV power consumption was considered irrespective of whether the UAV was deployed as a standalone network or UAV-assisted cellular network. However, in this subsection, we consider various techniques, based on conventional and ML algorithms, that have been employed for the optimization of the total energy consumption of UAV-assisted cellular networks where the energy consumption of both terrestrial BSs and UAV-BSs are jointly optimized. Specifically, we focus on the literature regarding energy optimization in UAV-assisted heterogeneous networks (HetNets).

8.2.1. Conventional Approaches

The authors in [107] considered EE in a UAV-assisted mmWave HetNet comprising macro BSs (MBSs), mmWave small BS (SBSs), and UAVs. A joint optimization of both the sub-carrier and power allocation was performed in order to maximize the EE of the network while considering minimum QoS and maximum transmit power as constraints. Two heuristic algorithms were developed to maximize the EE of the MBS and minimize the power consumption of the UAVs while satisfying minimum QoS requirements. In [106], the authors investigated the use of UAVs for maximizing the EE in a UAV-assisted small cell (SC) network by enabling the switch off of low EE BSs. A low-complexity algorithm was developed to determine the optimal positioning and sleeping strategy that would maximize the EE of the network. Similarly, the work in [105] considered the problem of energy optimization in UAV-assisted HetNets comprising MBSs, micro cells that can be switched off/on, and UAV SCs that are powered by solar energy. The optimization objective was to determine the optimal position of the UAVs as well as the off/on status of the micro cells that result in the minimum energy consumption in the network while respecting QoS constraints.

Energy and hover time optimization in a three-tier UAV-based HetNet comprising UAVs, MBS, and SBSs was considered in [266] to enhance the EE of the network. A two-layered optimization framework was developed where the power consumption of each tier was first optimized in order to maximize the EE of the network while respecting QoS constraints, followed by the hover time optimization of the UAVs. A heuristic algorithm based on Lagrange multiplier and subgradient approach was proposed to maximize the EE of the network. The authors in [267,268] proposed a weighted power allocation scheme for minimizing the total energy consumption of both UAVs and terrestrial BSs of a UAV-assisted HetNet while satisfying QoS constraints of mobile users. A heuristic algorithm was developed to determine the optimal power allocation strategy that would result in minimal energy consumption in the network.

The authors in [269] considered the optimal placement of UAV mounted remote radio heads (D-RRH) in a CRAN to minimize the transmission power while ensuring that a maximum number of users are covered. The D-RRH placement problem was first decomposed into two: vertical and horizontal placement problem. The former was solved using Weiszfeld algorithm, while the latter was solved analytically. Energy efficient UAV deployment in UAV-assisted C-RAN with flexible functional split selection was investigated in [266]. To achieve the minimal energy consumption, both the horizontal and vertical locations as well as the coverage radius of each UAV were optimized using mathematical analysis. In addition, a closed-form expression to determine the upper and lower bound of the number of UAVs required to achieve optimal energy consumption was derived. Table 5 summarizes the CA and ML approaches that are applied for energy optimization in UAV-assisted cellular networks.

8.2.2. Machine Learning Approaches

The preceding sections considered the few conventional techniques that have been applied for energy efficiency optimization in UAV-assisted cellular networks comprising both UAV-BSs and fixed BSs. Since the research on UAVs for wireless communications is still in its early stages, most of the work that has been carried out in the area of joint optimization of the energy consumption of both the UAV-BS and fixed BSs has mainly applied conventional approaches. The areas where machine learning techniques have been applied for energy optimization are in the areas of energy-efficient UAV trajectory design [226], energy-efficient UAV positioning [200], energy-efficient UAV resource allocation [255], and energy-efficient UAV transmission scheduling [30]. This means that there is a need to conduct research work where machine learning techniques are employed to both quantify and optimize the combined energy consumption of both UAV-BS and fixed BS. This would ensure that the advantages achieved by the deployment of the UAVs with fixed BSs are not overwhelmed by the increase in the overall energy consumption of the network. This will also help overcome the limitations that conventional techniques face in terms of scalability and adaptability to changing environmental conditions.

9. Enabling Technologies for Energy Efficiency in UAV-Based Cellular Networks

In this section, we present some of the technologies that empower UAVs to achieve high EE when used in cellular networks. Any technique that enables the UAV to process less data, focus its energy more, or extract energy from the ambient environment will prolong its operational lifetime and is therefore considered energy-efficient. Below, we discuss such technologies and show how they enable UAVs to achieve energy savings.

9.1. RIS

RISs are meta-surfaces that use electronic circuits (usually, varactor diodes or other micro-electromechanical systems) to manipulate impinging electromagnetic signals in a desired way. By intelligently manipulating each reflecting surface, RISs can change the amplitude, phase, or frequency of each incoming signal and redirect them to a desired location without using power amplifiers. In an RIS, each meta-surface is capable of independently steering an impinging electromagnetic ray to a desired angle of reflection, and manipulating the phase of the reflected ray so that it aligns with the angle and phase of other rays from other meta-surfaces in the RIS [271], thereby ensuring that all the reflected rays are received with the same angle and phase (these are usually designed to align with the angle and phase of the line-of-sight signal (LOS)) at the receiver. By acting in such a manner as a low complexity (re-) transmitter, RISs can forward signals to a receiver and have been shown to largely outperform other amplify-and-forward-based relaying techniques in terms of received power (mostly due to their large geometry of the RIS antennas) [271] as well as EE in wireless communication systems [272], since they do not require active elements. They are mostly passive and do not require external power supply. However, they do not amplify the RF signals impinging on their surface [271], which is also an advantage since they do not amplify the noise.

RISs can be combined with UAVs to improve system throughput [273], network coverage and reliability [274,275], or EE [276,277,278,279]. By acting as relays for signals without the need for additional external power supply, RISs achieve high EE when combined with UAVs in cellular networks. RISs are deployed between the UAV and ground-based user nodes. As a result, the UAV can transmit at a much lower power to reach the RISs [271], leading to significant power savings. In addition, since received power is proportional to the effective area of the receiving antenna, RISs can capture more of the transmitted RF signal compared to user devices that have much smaller antennas. Again, this enables the UAV to lower its transmit power due to the efficiency of the RISs, which helps the UAV to conserve its power and prolong its operational lifetime.

UAVs often need to move to maintain LOS connection with mobile ground terminals. With RIS, the UAV simply beams its signal on the reflecting surface, whose reflecting angle and phase are then manipulated to focus the signal in the direction of the mobile users. This has been shown to significantly lower UAV energy consumption since up to 95% of the energy consumed by rotary-wing UAVs is spent on flying [276]. It was also shown in [280] that even in the presence of a direct LOS signal between the UAV and internet-connected vehicles, RIS significantly improves the bit error rate, coverage probability, and throughput. These metrics imply not only less transmission power from the UAV but also fewer retransmissions.

9.2. Mobile Edge Computing (MEC)/Cloud

MEC can prolong the operational time of UAVs by offloading some computing tasks from the UAV. The concept of allowing UAVS to offload computational tasks to MEC servers was studied in [281], where it was shown to improve both EE and processing time for assigned tasks. Energy saving for UAVs through computational tasks offloading to MECs was extended to the case of multiple UAVs in [255], where the authors proposed a system to minimize both the computation, communication, and mechanical (due to flight or hovering) energy consumption of UAVs subject to latency and coverage constraints. The technical challenges that must be overcome to actualize UAV-based MEC include UAV trajectory/position planning [248,282,283], allocation of computational loads [283] in the case of multiple UAV/MEC servers, user transmit power/interference management [284], security against eavesdropping and jamming [285,286], etc., while minimizing the energy consumption of the UAV (and in some cases that of the users [283]) so as to prolong its operational life. The authors in [287] proposed a time division multiple access (TDMA)-based workflow scheduling model to improve UAV EE while offloading computations to MEC servers by optimizing user association, computing resource allocation, hovering duration, and service sequences of users. Offloading computing tasks to UAV MECs and switching off computational elements was shown to lower energy consumption and job loss in [288], wherein the authors implemented an RL model to manage a framework responsible for switching computing elements in order to lower power consumption, delay, and job loss.

The common architecture in UAV-based MEC systems involves tethering the MEC server to the UAV. In this way, the UAV provides coverage and connectivity to the users on ground, while all computational tasks are offloaded to the MEC server. This scheme was implemented in [289], where the UAV also provided wireless energy harvesting services to the ground users in addition to communication and computation coverage. Similarly, it was shown in [290] that, instead of serving just as a flying BS or mobile relay for ground-based users, the UAV can function as a mobile cloudlet to run compute-intensive applications for the users. This is achieved by offloading such tasks to a cloudlet attached to the UAV, thus minimizing the overall mobile energy consumption without reducing the QoS requirements of the target applications.

Some of the articles highlighted above have adopted state-of-the-art ML algorithms to improve EE for UAV-enabled MEC networks. A few examples include the work in [282] where the long short-term memory (LSTM) algorithm was employed to predict computational tasks from the users and thereby direct MEC computational resources accordingly, and in [286] where RL was employed to plan an allocation strategy for the computational tasks, whereas a transfer learning algorithm builds on the allocation strategy to achieve faster computation. Similarly, a cooperative UAV MEC system was implemented in [288], where a UAV can offload computation to nearby UAVs to conserve energy and reduce latency. A RL algorithm was employed to assist a system controller in allocating the tasks to be completed to the available UAVs to improve target metrics of EE, latency, and job loss.

9.3. Network Slicing/Network Function Virtualization

Network function virtualization (NFV) is a networking paradigm whereby network functions (such as address translation, firewall services, storage, routing, deep packet inspection, etc.) are decoupled from the underlying hardware and instead realized as software (hence the term network softwarization) that can be run on standard commercial off-the-shelf servers/equipment [291] and accessed via standardized interfaces. NFV helps to improve the flexibility, resilience, EE, and management of UAV-based networks. The softwarization of network functions leads to less processing at UAV-BSs, which significantly improves the energy usage of such BSs.

In such software-defined network scenarios, the automation of network management functions will free the UAV from performing those functions, so that its on-board energy can be dedicated to performing other tasks [292]. For instance, the authors in [293] introduced an NFV scheme where spectrum processing, allocation, and management are dynamically implemented as virtualized functions to serve UAV-BSs. Cognitive radio services are also virtualized to enable the UAV to provide both cellular, Wi-Fi, and WIMAX services according to the application requirements for the coverage area. Offloading storage and processing functions to the ground control stations gives the UAVs higher endurance. NFV not only allows UAVs to save energy but also to be rapidly adapted for different purposes based on application demands. This tack was taken in [294] where small UAVs were programmed to be deployed by a network operator to offer any chosen service such as flight control, voice over IP telephony, routing, etc. The authors demonstrated a prototype of using such a system to provide IP telephony services.

In implementation, a NFV infrastructure is used to host all the network shared hardware and virtual resources. In fact, by allowing a diverse set of network services to be run remotely on such shared infrastructure, NFV inherently enhances resource utilization [295], including on-board energy resources in UAVs.

9.4. Cooperative Communications

Cooperative communication helps to lower the energy consumption of UAVs by employing other devices within the network or other UAVs to assist in data transmission, instead of using the UAV to route all network traffic. For instance, UAVs serving as relays (for one another or between an eNodeB and ground-based terminals) can help to improve network coverage [296], reliability [297], or bypass unfavourable channels. In this way way, data are always relayed through favorable channels between communicating pairs [298], leading to both proximity gain (higher throughput) and significant energy savings. Proximity gains result from the fact that UAVs are deployed closer to user terminals than MBSs, especially when they are deployed to extend the coverage of the MBS. The energy savings in this case are due to the proximity/nearness of the UAVs to the terminals, allowing them to transmit with lower powers and reducing re-transmissions (the probability of error is low when the communicating terminals are close), thereby achieving significant energy savings [299]. In addition, cooperative communication leads to spatial diversity gains since the relays employed in such systems receive and process the signal from different locations than the original transmitter. Thus, cooperative communication improves the system reliability by ensuring that the signal is delivered to the destination even if a channel is in deep fade. It has been shown that in some implementations, cooperation achieves full diversity [300], which can significantly suppress the interference arising from multiple terminals [301], leading to fewer re-transmissions which indirectly leads to higher energy savings. An example is shown in [302] where the authors demonstrated that using cluster heads as cooperative relays lowers the energy consumption of a UAV.

RISs can also be viewed as a cooperative communication mechanism when combined with UAV networks to improve cellular coverage and throughput. They save energy by relaying transmitted data from the source to the destination. RISs not only reduce the transmit powers needed by UAVs to reach ground terminals but also improve coverage and suppress channel fading. This has been shown to improve the system EE [303] in general and to conserve the energy of the UAV in particular.

9.5. Energy Harvesting Technologies

Energy harvesting has become a hot topic among the scientific research community. As more and more power-hungry internet-connected devices are deployed in the IoT era, finding enough power sources to feed them is becoming a technical as well as an environmental problem. In addition, since most wireless devices are mobile, using stationary cables to recharge their batteries limits their mobility and therefore, their utility [304]. However, there are numerous energy sources in the ambient environment that can be harnessed to provide power to these devices to keep them operational for very long periods of time (if carefully designed, some devices can harvest enough ambient power to potentially run perpetually.)

Popular energy harvesting schemes include harvesting energy from the electromagnetic fields, solar energy from the sun using photovoltaic cells, mechanical energy due to vibration or motion of the UAV, thermal energy, optical energy using laser beams [304], etc. In electromagnetic waves-based energy harvesting (e.g., RF energy harvesting), the transmitter coverts some of the direct current (DC) from its battery into the RF energy to be transmitted. At the receiver, the received RF energy is converted back to DC and amplified using matching and rectifier circuits [305], then used to charge the on-board battery or directly used to power decoding or other communications operation. The efficiency of these RF-DC conversions significantly affects the amount of energy harvested.

Regarding solar-powered UAVS, the authors in the study [306] solved the problem of short battery life in multi-solar-based UAVs that are used for coverage route planning (CPP). They measure the energy flow efficiency to assess how well the UAV consumes energy while in operation. Moreover, they also proposed a model for coverage path optimization for both regions with and without obstacles based on a graph search approach. Similarly, the authors in [307] proposed a multi-objective genetic algorithm-based framework for a solar-powered UAV. A multiple objective optimization model comprising remaining battery charge, the electricity used from the battery, and the energy harvested by the UAVs was formulated. The performance evaluation revealed that the proposed method was able to increase the maximum energy output of the photovoltaic cells by 5.79%, output electricity from the battery by 7.00%, and the state of charge of the battery by 10.94% compared to baseline methods for different flight trajectories. In [308], the energy efficiency of a solar-powered UAV-based network was designed to serve multiple ground users (GUs) via uplink data transfer was investigated. The study explored the effect of the UAV’s transmission power, velocity acceleration, and heading angle on the uplink data rate and the harvested solar energy. A heuristic algorithm was proposed to improve the UAV’s velocity acceleration and heading angle in order to maximize the system’s energy efficiency.

In [298], a UAV scavenges RF energy from a transmitting source (such as an eNodeB) to partially power the energy used in communication to enable it to serve as a relay to forward data between the source and user terminals on the ground. The energy harvesting is based on the time-splitting technique in which the UAV harvests energy in one time slot and performs communication in another slot. A similar energy harvesting technique for UAV-supported communication was explored in [309], wherein a power-splitting technique was used instead of the time-splitting scheme adopted in [298]. In power-splitting, the energy harvesting unit (UAV in this case) has separate hardware for handling communication and energy harvesting, so that both can be performed simultaneously.

In addition, the authors in [310] derived an optimal strategy for allocating charging time and power to UAVs to wirelessly charge them during operation. The charging power is supplied by the BS, i.e., it is scavenged from the RF energy typically used for communication. A different tack was followed in [311] where the authors proposed an energy sharing scheme whereby high-capacity UAVs harvest solar energy and share it with smaller-capacity UAVs. In some implementations of energy harvesting to support UAV operations, the authors assume that the harvested energy is sufficient to support the communication functions of the UAV, whereas in others, harvested energy merely serves to support power provided by the UAV on-bard battery. In [312], Yang et al. studied the outage problem of an UAV that harvests energy from a ground-based BS and uses the harvested energy to relay data for user terminals on the ground.

10. Challenges and Open Research Problems

In this section, we present the most common challenges and open research issues that are limiting the widespread adoption of UAVs in cellular networks. The list presented is not exhaustive, but it is meant to guide the reader on some of the most pressing concerns that need to be addressed to fully exploit the advantages of UAVs as an aid to cellular networks. The reader is referred to the work in [313], which provides a more detailed look at UAV network design challenges.

10.1. Security Challenges

Security is a big challenge in UAV-aided communication. Due to the small form factor and capability of most UAVs, they are prone to both cyber and physical security threats, such as hijacking. Moreover, due to the broadcast nature of UAV communications, they are subject to eavesdropping, packet snooping, jamming, spoofing, denial of service attacks, and cyber attacks [19,314]. The security of UAV-to-UAV communication against multiple eavesdroppers was investigated in [315] where the authors derived expressions for the statistical SNR as well as for the secrecy outage probability. In extreme cases, interceptors could potentially spoof the control signals and use them to gain control of the UAVs [19,316], or jam such signals to prevent them from reaching the ground control station. Thus, authentication of users and operators is an issue in UAV-based networks [317]. UAV security issues are categorized under cyber detection, which focuses on identifying intruders within the UAV network, and cyber protection, which focuses on eliminating or reducing external threats to the UAV network [318].

The authors in [319] also distinguished different forms of security, such as information and software security, sensors security, and communications security. Information and software security is related to the security of the UAV operating systems (including its configuration files, mission-related data, and flight control files) as well as its collected data. Sensor security deals with the security of the various sensors used for the real-time maneuvering of the UAV, such as accelerometers, gyroscopes, GPS, barometers, etc. If attackers gain control of such subsystems, it could lead to the malfunctioning of the UAV or cause it to send erroneous data. Communications security is related to the security of the communications components of the UAV, including control commands, telemetry data, and transmission of the collected data. Since these communications happen over the air and some of the packets are unencrypted (e.g., data packets from simple sensors or even GPS data), they pose cybersecurity threats to the successful operation of UAV networks.

Packet routing is another potential source of attacks on UAV communications. Here, malicious entities can disguise themselves as legitimate elements in the UAV network to steal, modify, or drop packets. There are three common types of routing-related attacks: wormhole, selective forwarding, and sink hole attacks [320]. Wormhole attacks involve a malicious entity that captures packets in one location within the network, then tunnels the packet to a third party in another location where the packets are modified and retransmitted into the network [321]. This attack is common in ad hoc networks that use reactive routing protocol such as shortest-path-first routing, where the routing metric is based on the hop count. Malicious intermediaries pretend to be neighbor nodes of genuine network nodes, and route packets through private tunnels. Tunneling ensures that the packets arrive at the destination node with shorter distances or smaller number of hops than genuinely multi-hop-routed packets, thereby misleading the receiver. Selective forwarding attacks occur when malicious nodes behave like genuine network nodes and correctly forward packets most times, but occasionally selectively drop sensitive packets that may be critical to the functioning of the network. In UAV networks, other UAVs or edge devices can be used to perpetrate selective forwarding attacks. Since wireless networks randomly drop packets (due to congestion or unreliable links), this type of attack is difficult to detect because it is hard to distinguish if packet drops are due to network faults or malicious attacks [322]. In sink hole attacks, adversary nodes advertise a false hop count to the sink, tricking its neighboring nodes into believing that they have found a shorter route to the sink and then forward their data packets via the adversary node [323]. Thus, the adversary node not only gains access to packets of nodes within its radio range but also blocks these nodes from transmitting to the final receiver or genuine network sink. Again, other UAVs, relay nodes, or edge devices can be used to perpetrate this attack in UAV-assisted networks.

A potential solution for the communication security challenges in UAV-based networks is to use blockchain technology [314]. Proper authentication will also ensure that only vetted elements are admitted into the network. Designing a robust intrusion and detection system will also protect UAV-aided networks from malicious attacks [319]. In addition, innovative communication protocol design such as use of spread spectrum technologies, MIMO techniques, or cooperative UAV features can also help to protect UAV communication from eavesdroppers. Due to their directionality, mm-wave communication can also reduce the threat of jamming and eavesdropping since the electromagnetic signal beams are focused on the intended receiver(s) [324]. Data encryption is very helpful in dealing with information and data security, as is the proper design of the configuration files through a process known as system hardening [319]. Wormhole attacks can be detected and defended using packet leashes [321], where some information is added to a packet to restrict the maximum transmission distance per link. In addition, ANN have been proposed to create threat maps in the operating environment of UAVs, with recurrent neural networks used to predict the normal trajectory motion of the UAV and hence detect any deviations in real-time as suspected cyber attack [317].

10.2. Complexity

Increasingly, UAVs are being deployed as a swarm of vehicles to achieve communication, reconnaissance, search/rescue, or monitoring tasks. Due to the complexity of these tasks, multiple UAVs (also called a swarm of UAVs) are usually required to perform them more efficiently. The advantages of using multiple UAVs in such scenarios, such as cost and time efficiency, reliability/fault tolerance, flexibility/adaptability to changing requirements, and the ability to perform multiple tasks simultaneously, have been well-established [325]. However, due to the dynamism and uncertainty of the operating environments for such complex tasks, coordinating multiple UAVs to work together is highly challenging [326]. In addition, UAVs nowadays are increasingly subjected to more and more complicated cyber and physical threats. Thus, they have to be designed with additional software and hardware components to thwart such threats, which further increases their complexity. The most common issues that arise in multi-UAV coordination have been documented in [19,327,328,329,330]. They include:

• Algorithmic planning to manage communication and task allocation;

• Coverage issues and equitable distribution of workload;

• Aerial manipulation of the vehicles;

• Power management;

• Management of the communication infrastructure;

• Path planning to avoid collisions while ensuring adequate coverage without overlaps;

• Interference arising from other UAVs;

• Conflict resolution;

• Safety issues related to preventing the vehicles from flying into one another’s buffer zones;

• Safety issues related to take-off and landing (in some current implementations, a swarm of fixed-wing UAVs spent less than 20% of the time staying simultaneously in the air to execute assigned tasks) while the bulk of the time is spent trying to coordinate the flight of the UAVs;

• Network congestion and channel interference due to multiple UAVs exchanging data to coordinate the execution of assigned tasks.

Multi-UAV systems may also require more than a single pilot to manage them, which introduces another layer of complexity to the system [325].

One of the main challenges facing the implementation of a swarm of UAVs is localization. To be effective as part of a fleet, a UAV needs to be aware of its position in a given map of the environment. The UAV position is either relative to a reference point or relative to other UAVs in the fleet. As one can imagine, this is a non-trivial task that requires numerous exchanges of communication and control commands. In addition, since the positions of the UAVs are constantly changing, the map of the fleet is also constantly changing 3D. This gives rise to a dynamic 3D map of the environment, rather than a constrained static map with a reference landmark [331]. Thus, achieving partial or full localization is both energy- and bandwidth-intensive. GPS sensors are insufficient to address this problem because they provide position accuracy to only three meters [331], which is not granular enough to prevent collisions. Alternatives found in the literature include equipping the UAVs with wireless communication modules and inertial navigation systems, coupled with on-board sensor fusion to enable accurate estimation of position. As highlighted already, this incurs a heavy computational, communication (bandwidth), and energy cost, hence, finding innovative ways to do this more efficiently is still an open problem.

Path planning is another challenge that arises when multiple UAVs work together to achieve a common objective (this is related to the localization problem). As the number of vehicles in the swarm/fleet grows, it becomes more difficult to plan the trajectory of each UAV from the starting points to the goal points in order to traverse the minimum path (so as to save energy) and avoid collisions with obstacles or other vehicles in the swarm. In addition, path planning must be executed such that the UAVs maintain connectivity with one another and with the ground control station while performing assigned tasks. Path planning involves motion planning (to control the path length and turning angles), trajectory planning (involving the speed and kinematics of the vehicle), and navigation (involving localization and obstacle/collision avoidance) [332]. For UAV swarms used as aerial BSs, path planning in such cases requires high-rate exchange of positional sensor data, which involves multi-dimensional channel characterization, tracking and communication, interference management, transmit power allocation, resource block assignment, etc. [333]. Path planning for other UAV applications such as monitoring or target tracking are complicated by issues such as target location and identification, which arise due to the high mobility of the UAVs. Some of the techniques used in path planning can be categorized under representative, cooperative, and non-cooperative techniques [332]. Increasingly, ML approaches are employed to address path planning problems [334]. To save on energy, time, computational and communications costs, path planning is a complex task whose complexity grows rapidly as the number of UAVs in the swarm increases.

Other key challenges that arise due to the complexity of multi-UAV systems include security, equitable allocation of workload and coverage area. The greater the number of UAVs in a swarm, the larger the attack surface [335]. Therefore, modeling potential threats and designing a robust security framework to thwart cyber attacks is a highly complex and challenging task.

10.3. Data Availability

ML algorithms live on data; the more data we feed the algorithms, the smarter they tend to become [336]. To make strategic and intelligent decisions, data are typically required from different sources; these data are integrated and transformed before they are used to make decisions or predictions [337]. This process is the so-called data-driven or evidence-based decision-making [338]. In many cases, large data sets are required both to train and test the ML algorithms as well as to evaluate their efficacy in making predictions. In addition, both labeled and unlabeled data are required to compare the efficacy of different ML algorithms [339], and guide in selecting the best technology tool for a given situation. However, data are not always available or is available in insufficient quantity or quality. Due to the criticality of data to the success of businesses, companies are usually reluctant to share data obtained from UAV deployment trials. There are also cases where it is very difficult to obtain data, especially when a process or technique is still in infancy. Some of the most useful data sets are not publicly available. To make predictions which are generalizable, there needs to be sufficient supply of heterogeneous data repositories that are accessible to many researchers. Failing this, each institution might end up developing individual analytical pipelines that may fail under new circumstances, such as a different operating environment.

Open-access data repositories are a key enabler to unlocking insights into addressing the challenges of UAV deployment in cellular networks. With sufficient data shared collaboratively amongst stakeholders in the UAV research and development industry, robust and reproducible ML solutions can be developed for UAV applications in cellular networks. The ability to obtain and reuse data will enable easy collaboration among researchers and the industry, save costs, and minimize the time to market for new products. It is desirable to have an open database for UAV-aided cellular communication, similar to the modified National Institute of Standards and Technology (MNIST) database (c.f. [340]) widely used in computer vision studies.

10.4. Limited Energy Storage Capacity

One of the key challenges of using UAVs to support communication networks operations is that they have limited lifetime due to the low capacity of their batteries, which limits how long they can be deployed [341]. In fact, most UAVs have an endurance of just a few hours [342]. To reduce the weight of UAVs and the attendant energy drain problem, it is often necessary to use smaller batteries so as to avoid expending too much energy on flying as the energy required to fly the UAV varies with the payload size [343]. However, small batteries have low storage capacities, further complicating the energy situation of the UAV. In addition, there exist uncertainties in estimating the remaining battery charge in UAVs [344], leading to conservative estimates of left-over flight time to avoid dead stick conditions, whereby the UAV runs out of battery power in-flight, which could have disastrous consequences. Supercapacitors, on the other hand, are not ideal for UAVs due to their low energy density [49]. Improvements in battery technology are required to enhance the storage capacity of UAVs.

One of the solutions that have been proposed to address the battery capacity limitations of UAVs include wireless charging [49], whereby the UAV is recharged during operation via RF energy harvesting [343], laser power beaming [345,346], use of electrical power lines [347] by directly perching on current-carrying cables or harvesting electromagnetic energy generated by the cables. Other alternatives for recharging the UAV when its battery becomes depleted have been explored in [46], including UAV swapping (replacing those with depleted batteries with others that are fully charged) [341] and battery swapping. Alternatively, the UAV could be tethered to a mains power supply [348] or powered with fuel cells [43].

10.5. Energy Harvesting Challenges

It is well known that one of the biggest limitations to the widespread adoption of UAVs in cellular networking is their limited operating lifetime [310]. One of the most popular solutions for dealing with this problem is using energy harvesting technologies to enable UAVs to harvest energy from the environment to support the on-board battery. However, this technology is still in its infancy and fraught with many natural and technical challenges. For instance, solar-based energy harvesting still depends on climatic conditions and becomes unavailable when the sun is not shining [67,304], especially in the winter (which can last several months in the Northern Hemisphere). Solar energy harvesting solutions are based on use of photovoltaic arrays, which are only suitable for fixed-wing UAVs [43]. Similarly, the availability of RF energy depends on the density of RF devices within the area. Most of the energy harvesting technologies in the literature have low energy transfer efficiency due to environmental (such as free space pathloss for RF energy harvesting) or device limitations (such as poor energy conversion ratio). Moreover, since UAVs are usually in motion, they keep losing LOS connection with the charging BS. As a matter of fact, the amount of energy harvested is still very small compared to the amount of power that can be stored in on-board batteries, especially when such energy comes from RF environment [349].

The output of most energy harvesting techniques is still quite low due to the poor efficiency of the energy conversion and matching circuits [350]. The amount of energy captured from RF energy, for instance, depends on the area of the antenna elements [305]. Since most UAVs come in small form factors to limit their weight, their antennas are also small. Even where solar or other forms of energy are targeted for energy harvesting, the small form factor also implies that a limited area of the UAV is exposed to ambient energy, thereby limiting the amount of energy that can be harvested from such environments. Moreover, the distance between the UAVs and the charging (base) station as well as whether the UAV has an LOS connection to the base also affect the amount of energy received; the farther the UAV, the lower the energy received [311]. However, since UAVs are deployed to supplement the services provided by MBSs, it is desirable that they operate far from the BS in areas with poor coverage, which limits the power received from the macro cell.

Another limitation associated with energy harvesting is the time it takes to recharge in-flight UAVs from harvested energy due to shortcomings in the charging rates. Finally, since energy harvesting is still in its infancy, there is a lack of a unified standard on it, which limits the widespread involvement of original equipment manufacturers (OEM) to develop wireless recharging devices.

One way to improve the amount of energy harvested from the radio waves is to use directional antennas via beamforming [311,351], which ensures that more of the transmitted RF energy is received at the UAV. There are other proposed solutions aimed at optimizing the charging time of UAVs by improving the energy transfer and conversion rates. Solutions related to the above two include optimal or near optimal trajectory planning to guarantee maximum exposure of the UAV to the RF field of the charging BS in cases where energy is harvested from the radio environment.

10.6. Regulations

Due to public safety, privacy, and data protection concerns, there are strict restrictions on use of UAVs by governments around the globe. Both international and national regulations are put in place to ensure that UAVs pose minimal risks to other users of the airspace and to protect people and property on the ground [352]. For UAVs to gain wider acceptability, the public needs to trust that their deployment is in their best interests and will be used safely. For instance, there are valid concerns arising from the knowledge that UAVs have been used for illegal activities such as surveilling or tracking people. In addition, there are concerns over data protection since UAVs can be equipped with cameras and other sensors that can collect data from areas where they do not have authorization. There are also serious concerns regarding public safety, as UAVs can lose control during flight and collide with other aircraft, buildings, etc., or crash into people on the ground [19]. Moreover, UAVs can be equipped with weapons and used to perform remote attacks.

There are many regulatory barriers that need to be addressed before the full potential of UAVs as part of wireless networks can be realized. One of the commonest issues is restrictions on areas where UAVs can operate. Due to some of the concerns highlighted above, UAVs serving as cellular BSs are not yet approved to operate in many public areas, especially in areas with large crowds. In addition, there are strict restrictions on how far UAVs can fly. Even when a UAV is capable of operating autonomously, many municipalities and cities still require that there be a licensed UAV pilot present before a UAV can be deployed, which increases the operating costs of deploying such UAVs and limits their use. The red tape in the approval of the use of UAVs has also been highlighted as a challenge mitigating against their widespread use. Regulatory and policy issues are usually slow and far behind the advances in UAV development, which in turn curtail the research, development and deployment efforts by both the telecommunication industry and the research community [352].

Despite some of the current limitations, significant progress has been made in UAV regulations. In Europe, for example, roadmaps have been created on how to integrate UAV operations into the civilian aviation industry [352]. There are similar efforts around the globe to enact regulations that will both promote the wide adoption of UAVs in civilian applications as well as ensure privacy and public safety [352].

10.7. Integration of Multi-Tier Heterogeneous UAV Networks

Emerging applications and the heterogeneity of communication devices in contemporary networks with diverse QoS requirements demand the adoption of different communication technologies to satisfy these requirements. As a result, there is a need to address the integration of UAVs into the existing satellite and ground-based BSs, especially in the emerging unified space-air-ground integrated networks paradigm [353], so that the requirement of each device on the network can be satisfied by the most efficient resource available. However, despite the evidence supporting the integration of UAVs into future unified communications paradigms including space-air-ground networks (as evidenced by the large body of published work available), safety concerns and the slow progress of standardization (as highlighted in the preceding Section 10.6) continue to limit practical implementation. There are still issues related to standardization and safety that need to be addressed to ensure seamless integration while guaranteeing safety and privacy preservation. Despite these limitations, continued progress is being made in addressing communication-related issues, such as modeling of UAV-satellite channels [354], as well as addressing backhaul and access resource allocation, trajectory planning [355], etc.

11. Conclusions

This survey paper covers the energy optimization techniques for UAV-assisted wireless communication networking by categorizing them in terms of the optimization algorithm employed. On one hand, there are some well-known optimization methods, such as heuristics, game theory, etc., that have been widely used for energy optimization. The implementation of ML for optimization, on the other hand, have been gaining momentum due to its proven capabilities. Thus, we combined both conventional and ML algorithms in this survey paper in order to cover the literature in a comprehensive and inclusive manner. The studies on energy optimization in UAV-assisted wireless networking were investigated thoroughly to reveal the state of the art. Some background information on both the optimization algorithms and power supply/charging mechanisms of UAVs were given in order to cover the topic in a more complete manner. Moreover, different types of UAV deployments were also discussed to highlight how the UAV-assisted communication networks can be divergent, increasing the level of challenge in optimization. As one of the most novel parts of this survey, emerging technologies, such as RIS and landing spot optimization, were presented to capture the latest advancements in the literature. The survey was concluded by the identification of challenges and possible research directions. This will help focus the research efforts on these areas, thus making UAV-assisted wireless networking a complete and mature concept. This, in turn, would result in a feasible and applicable concept for wireless communication networks, which has the potential to mitigate the capacity scarcity issue.

Author Contributions

Conceptualization and Outline, A.I.A., M.O. and K.G.O.; writing—original draft preparation, A.I.A., I.A., K.G.O., M.O., C.O. and A.M.A.-S.; writing—review and editing, A.I.A., I.A., K.G.O., M.O., C.O., A.M.A.-S. and S.H.; visualization, M.S.M.; supervision, Q.H.A., S.H. and M.A.I. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:

Acronym	Full Meaning
3D	Three Dimensional
5G	Fifth Generation
6G	Sixth Generation
ADMM	Alternating Direction Method of Multipliers
AI	Artificial Intelligence
ANN	Artificial Neural Networks
AP	Access Point
ARIMA	Auto-Regressive Integrated Moving Average
AtG	Air-to-Ground
BCA	Block Coordinate Ascent
BCA	Block Coordinate Descent
BS	Base Station
CA	Conventional Approaches
CAPEX	Capital Expenditure
CCP	Concave Convex Procedure
CNN	Convolution Neural Networks
C-RAN	Centralized Radio Access Network
CS	Cucker-Smale
D2D	Device-to-Device
DBS	Data Base Station
DC	Direct Current
DDPG	Deep Deterimistic Policy Gradient
DRL	Deep Reinforcement Learning
D-RRH	Drone-Mounted Remote Radio Head
DSC	Drone Small Cell
EE	Energy Efficiency
GA	Genetic Algorithm
GMM	Gaussian Mixture Model
GPS	Global Positioning System
HIL	Hardware in Loop
FL	Federated Learning
FSO	Free Space Optics
kNN	k-Nearest Neighbour
IoT	Internet of Things
IRS	Intelligent Reflective Surface
LiPo	Lithium Polymer
LoS	Line of Sight
LSTM	Long and Short Term Memory
MBS	Macro Base Station
MDP	Markov Decision Process
MEC	Mobile Edge Computing
MIMO	Multiple Input Multiple Output
ML	Machine Learning
mmWave	Millimeter Wave
MNIST	Modified National Institute of Standards and Technology
NFV	Network Function Virtualization
NLP	Natural Language Processing
NR	New Radio
NOMA	Non-Orthogonal Multiple Access
OEM	Original Equipment Manufacturer
OPEX	Operating Expense
PSO	Particle Swarm Optimization
PV	Photo voltaic
QoS	Quality of Service
RF	Radio Frequency
RIS	Re-configurable Intelligent Surfaces
RL	Reinforcement Learning
RNN	Recurrent Neural Networks
SARSA	State Action State Action Reward
SCA	Successive Convex Optimization
SEE	Secrecy Energy Efficiency
SMPS	Solar Power Management System
SVM	Support Vector Machine
SVR	Support Vector Regression
TBS	Terrestrial Base Station
TDMA	Time Division Multiple Access
THz	TeraHertz
T-UAV	Tethered Unmanned Aerial Vehicle
UAV	Unmanned Aerial Vehicle
UE	User Equipment
URLLC	Ultra-Reliable Low Latency Communication
WEM	Weighted Expectation Maximization
WiFi	Wireless Fidelity
WPCN	Wireless Powered Communication Networks
WPT	Wireless Power Transfer
XGBOOST	Extreme Gradient Boosting

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. A top-level summary of energy optimization perspectives in UAV-based communication systems and corresponding approaches.

Figure 2. Outline of the survey.

Figure 3. Various UAV power supply and charging mechanisms including wireless power transfer via a recharging UAV, a tethered UAV, a solar-powered UAV, and battery swapping at a charging station.

Figure 4. Different UAV use cases involving the UAV serving as a wireless backhaul to connect users located in remote areas to the core network, as a emergency network for restoring service to an area covered by a malfunctioning BS, as a aerial relay for providing coverage extension to users separated by obstacles such hills and mountains, data offloader for offloading user traffic from overloaded terrestrial BSs to ensure load balancing and throughput enhancement, as aerial cache for serving popular content to ground users to reduce backhaul congestion and achieve EE and as a data harvester for collecting data from ground sensor nodes.

Figure 5. Illustration of the two major types of UAV deployments: UAV stand-alone deployment comprising single and multiple UAV networks, and UAV-assisted cellular networks comprising both UAV-BSs and terrestrial BSs.

Figure 6. Classification of the conventional algorithms discussed within the scope of this work.

Figure 7. Classification of machine learning algorithms within the scope of this work.

Table 1

Summary of energy optimization techniques based on optimal UAV placement and positioning.

Paper	Year	Category		Specific Algorithm
		CA	ML
[116]	2017	✓		Heuristic
[23]	2021	✓		Heuristic
[24]	2019	✓		Heuristic
[178]	2020	✓		Heuristic
[179]	2020	✓		Heuristic
[119]	2020	✓		Heuristic
[180]	2020	✓		Heuristic
[181]	2020	✓		Heuristic
[118]	2020			Heuristic
[182]	2017	✓		Heuristic
[183]	2021	✓		Heuristic
[184]	2018	✓		Heuristic
[185]	2021	✓		Heuristic
[186]	2016	✓		Heuristic, IP, MINP
[187]	2020	✓		Heuristic
[188]	2020	✓		PSO
[189]	2018	✓		Heuristic
[190]	2019	✓		Heuristic
[191]	2020	✓		Heuristic
[192]	2019	✓		Heuristic
[193]	2021	✓		Heuristic
[195]	2021	✓		Heuristic, LP
[196]	2018	✓		Heuristic, MILP
[197]	2021		✓	Random forest, k-means
[198]	2020		✓	Clustering algorithm
[199]	2018		✓	GMM, WEM, k-means
[200]	2021		✓	Actor-critic RL

Table 2

Summary of energy optimization techniques based on path planning and trajectory design.

Paper	Year	Category		Specific Algorithm
		CA	ML
[201]	2020	✓		Heuristic
[113]	2017	✓		Heuristic, MILP
[25]	2019	✓		Heuristic
[145]	2020	✓		Heuristic
[26]	2020	✓		Heuristic
[202]	2020	✓		Heuristic
[203]	2021	✓		Heuristic
[204]	2018	✓		Heuristic
[70]	2019	✓		GA
[205]	2018	✓		Heuristic
[206]	2019	✓		Heuristic
[129]	2020	✓		GA
[115]	2021	✓		Heuristic
[207]	2021	✓		Heuristic
[208]	2019	✓		Heuristic
[209]	2018	✓		Heuristic
[210]	2018	✓		Heuristic
[211]	2019	✓		Heuristic
[212]	2020	✓		Heuristic
[213]	2020	✓		Heuristic
[214]	2020	✓		Heuristic, MILP
[216]	2019	✓		Heuristic
[215]	2021	✓		Heuristic
[217]	2020	✓		Heuristic
[218]	2020	✓		Heuristic
[219]	2021	✓		Heuristic
[220]	2021	✓		Heuristic
[142]	2016	✓		Heuristic, MILP
[221]	2018	✓		GA, MILP
[222]	2018	✓		LP, Convex Optimization
[223]	2020		✓	RL
[224]	2021		✓	DQN
[225]	2020		✓	DDPG
[226]	2021		✓	Q-learning
[227]	2019		✓	Q-learning
[231]	2020		✓	DRL
[232]	2020		✓	RL, DRL
[234]	2021		✓	Q-learning
[235]	2019		✓	SVR, k-means
[236]	2021		✓	DDPG

Table 3

Summary of energy optimization techniques based on resource management.

Paper	Year	Category		Specific Algorithm
		CA	ML
[237]	2018	✓		Heuristic
[238]	2021	✓		Heuristic
[239]	2021	✓		Heuristic
[240]	2021	✓		Heuristic
[241]	2020	✓		Heuristic
[242]	2021	✓		Heuristic
[243]	2019	✓		Heuristic
[244]	2020	✓		Heuristic
[245]	2019	✓		Heuristic
[246]	2019	✓		Heuristic
[247]	2020	✓		Heuristic
[27]	2019	✓		Heuristic
[242]	2021	✓		Heuristic
[248]	2020	✓		Heuristic
[249]	2019	✓		Convex Optimization
[250]	2021	✓		Heuristic, MINLP
[251]	2021	✓		Heuristic, MINLP
[252]	2019		✓	Q-learning
[253]	2020		✓	DDPG
[28]	2021		✓	DRL
[254]	2020		✓	DQN
[255]	2019		✓	Fuzzy C-means

Table 4

Summary of energy optimization techniques based on flight and transmission scheduling.

Paper	Year	Category		Specific Algorithm
		CA	ML
[256]	2018	✓		Heuristic
[257]	2021	✓		Heuristic
[29]	2020	✓		Heuristic
[258]	2020	✓		Heuristic
[259]	2020	✓		Heuristic
[260]	2016	✓		Heuristic
[261]	2018	✓		Heuristic
[262]	2020	✓		Heuristic
[30]	2020		✓	DDPG
[263,264]	2020, 2021		✓	Actor-critic

Table 5

Summary of energy optimization in UAV-assisted cellular networks.

Paper	Year	Category		Specific Algorithm
		CA	ML
[107]	2019	✓		Heuristic
[106]	2021	✓		Heuristic
[105]	2017	✓		Heuristic
[266]	2021	✓		Heuristic
[267,268]	2019,2021	✓		Heuristic
[269]	2020	✓		Heuristic
[270]	2018	✓		Heuristic

References

1. Ali, R.; Zikria, Y.B.; Bashir, A.K.; Garg, S.; Kim, H.S. URLLC for 5G and Beyond: Requirements, Enabling Incumbent Technologies and Network Intelligence. IEEE Access; 2021; 9, pp. 67064-67095. [DOI: https://dx.doi.org/10.1109/ACCESS.2021.3073806]

2. Chettri, L.; Bera, R. A Comprehensive Survey on Internet of Things (IoT) Toward 5G Wireless Systems. IEEE Internet Things J.; 2020; 7, pp. 16-32. [DOI: https://dx.doi.org/10.1109/JIOT.2019.2948888]

3. Ericsson. Ericsson Mobility Report; Technical report Ericsson: Stockholm, Sweden, 2021.

4. Jiang, W.; Han, B.; Habibi, M.A.; Schotten, H.D. The Road Towards 6G: A Comprehensive Survey. IEEE Open J. Commun. Soc.; 2021; 2, pp. 334-366. [DOI: https://dx.doi.org/10.1109/OJCOMS.2021.3057679]

5. Alwis, C.D.; Kalla, A.; Pham, Q.V.; Kumar, P.; Dev, K.; Hwang, W.J.; Liyanage, M. Survey on 6G Frontiers: Trends, Applications, Requirements, Technologies and Future Research. IEEE Open J. Commun. Soc.; 2021; 2, pp. 836-886. [DOI: https://dx.doi.org/10.1109/OJCOMS.2021.3071496]

6. Sun, Y.; Liu, J.; Wang, J.; Cao, Y.; Kato, N. When Machine Learning Meets Privacy in 6G: A Survey. IEEE Commun. Surv. Tutor.; 2020; 22, pp. 2694-2724. [DOI: https://dx.doi.org/10.1109/COMST.2020.3011561]

7. Mao, B.; Tang, F.; Kawamoto, Y.; Kato, N. AI Models for Green Communications Towards 6G. IEEE Commun. Surv. Tutor.; 2021; [DOI: https://dx.doi.org/10.1109/COMST.2021.3130901]

8. Saadi, M. 6G: The Network of Technology Convergence; Technical report ABI Research: Oyster Bay, NY, USA, 2022.

9. Wang, X.; Zhou, Z.; Yang, Z.; Liu, Y.; Peng, C. Spatio-temporal analysis and prediction of cellular traffic in metropolis. Proceedings of the 2017 IEEE 25th International Conference on Network Protocols (ICNP); Toronto, ON, Canada, 10–13 October 2017; pp. 1-10. [DOI: https://dx.doi.org/10.1109/ICNP.2017.8117559]

10. Ozturk, M.; Abubakar, A.I.; Nadas, J.P.B.; Rais, R.N.B.; Hussain, S.; Imran, M.A. Energy Optimization in Ultra-Dense Radio Access Networks via Traffic-Aware Cell Switching. IEEE Trans. Green Commun. Netw.; 2021; 5, pp. 832-845. [DOI: https://dx.doi.org/10.1109/TGCN.2021.3056235]

11. Barlacchi, G.; De Nadai, M.; Larcher, R.; Casella, A.; Chitic, C.; Torrisi, G.; Antonelli, F.; Vespignani, A.; Pentland, A.; Lepri, B. A multi-source dataset of urban life in the city of Milan and the Province of Trentino. Sci. Data; 2015; 2, 150055. [DOI: https://dx.doi.org/10.1038/sdata.2015.55]

12. Abubakar, A.I.; Mollel, M.S.; Ozturk, M.; Hussain, S.; Imran, M.A. A lightweight cell switching and traffic offloading scheme for energy optimization in ultra-dense heterogeneous networks. Phys. Commun.; 2022; 52, 101643. [DOI: https://dx.doi.org/10.1016/j.phycom.2022.101643]

13. Mozaffari, M.; Saad, W.; Bennis, M.; Nam, Y.H.; Debbah, M. A tutorial on UAVs for wireless networks: Applications, challenges, and open problems. IEEE Commun. Surv. Tutor.; 2019; 21, pp. 2334-2360. [DOI: https://dx.doi.org/10.1109/COMST.2019.2902862]

14. Klaine, P.V.; Nadas, J.P.B.; Souza, R.D.; Imran, M.A. Distributed Drone Base Station Positioning for Emergency Cellular Networks Using Reinforcement Learning. Cogn. Comput.; 2018; 10, pp. 790-804. [DOI: https://dx.doi.org/10.1007/s12559-018-9559-8]

15. Zeng, Y.; Wu, Q.; Zhang, R. Accessing From the Sky: A Tutorial on UAV Communications for 5G and Beyond. Proc. IEEE; 2019; 107, pp. 2327-2375. [DOI: https://dx.doi.org/10.1109/JPROC.2019.2952892]

16. Bithas, P.S.; Michailidis, E.T.; Nomikos, N.; Vouyioukas, D.; Kanatas, A.G. A Survey on Machine-Learning Techniques for UAV-Based Communications. Sensors; 2019; 19, 5170. [DOI: https://dx.doi.org/10.3390/s19235170] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31779133]

17. Gupta, L.; Jain, R.; Vaszkun, G. Survey of Important Issues in UAV Communication Networks. IEEE Commun. Surv. Tutor.; 2016; 18, pp. 1123-1152. [DOI: https://dx.doi.org/10.1109/COMST.2015.2495297]

18. Jiang, X.; Sheng, M.; Zhao, N.; Xing, C.; Lu, W.; Wang, X. Green UAV communications for 6G: A survey. Chin. J. Aeronaut.; 2021; [DOI: https://dx.doi.org/10.1016/j.cja.2021.04.025]

19. Fotouhi, A.; Qiang, H.; Ding, M.; Hassan, M.; Giordano, L.G.; Garcia-Rodriguez, A.; Yuan, J. Survey on UAV Cellular Communications: Practical Aspects, Standardization Advancements, Regulation, and Security Challenges. IEEE Commun. Surv. Tutor.; 2019; 21, pp. 3417-3442. [DOI: https://dx.doi.org/10.1109/COMST.2019.2906228]

20. Dargan, S.; Kumar, M.; Ayyagari, M.R.; Kumar, G. A Survey of Deep Learning and Its Applications: A New Paradigm to Machine Learning. Arch. Comput. Methods Eng.; 2019; 27, pp. 1071-1092. [DOI: https://dx.doi.org/10.1007/s11831-019-09344-w]

21. L’Heureux, A.; Grolinger, K.; Elyamany, H.F.; Capretz, M.A.M. Machine Learning With Big Data: Challenges and Approaches. IEEE Access; 2017; 5, pp. 7776-7797. [DOI: https://dx.doi.org/10.1109/ACCESS.2017.2696365]

22. Zhou, L.; Pan, S.; Wang, J.; Vasilakos, A.V. Machine learning on big data: Opportunities and challenges. Neurocomputing; 2017; 237, pp. 350-361. [DOI: https://dx.doi.org/10.1016/j.neucom.2017.01.026]

23. Babu, N.; Virgili, M.; Papadias, C.B.; Popovski, P.; Forsyth, A.J. Cost- and Energy-Efficient Aerial Communication Networks With Interleaved Hovering and Flying. IEEE Trans. Veh. Technol.; 2021; 70, pp. 9077-9087. [DOI: https://dx.doi.org/10.1109/TVT.2021.3100255]

24. Gao, F.; Zhou, Y.; Ma, X.; Yang, T.; Cheng, N.; Lu, N. Coverage-maximization and Energy-efficient Drone Small Cell Deployment in Aerial-Ground Collaborative Vehicular Networks. Proceedings of the 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS); Singapore, 23–25 February 2019; pp. 559-564. [DOI: https://dx.doi.org/10.1109/CCOMS.2019.8821718]

25. Zeng, Y.; Xu, J.; Zhang, R. Energy Minimization for Wireless Communication With Rotary-Wing UAV. IEEE Trans. Wirel. Commun.; 2019; 18, pp. 2329-2345. [DOI: https://dx.doi.org/10.1109/TWC.2019.2902559]

26. Liu, T.; Cui, M.; Zhang, G.; Wu, Q.; Chu, X.; Zhang, J. 3D Trajectory and Transmit Power Optimization for UAV-Enabled Multi-Link Relaying Systems. IEEE Trans. Green Commun. Netw.; 2021; 5, pp. 392-405. [DOI: https://dx.doi.org/10.1109/TGCN.2020.3048135]

27. Hua, M.; Wang, Y.; Li, C.; Huang, Y.; Yang, L. Energy-Efficient Optimization for UAV-Aided Cellular Offloading. IEEE Wirel. Commun. Lett.; 2019; 8, pp. 769-772. [DOI: https://dx.doi.org/10.1109/LWC.2019.2891727]

28. Zhang, M.; Fu, S.; Fan, Q. Joint 3D Deployment and Power Allocation for UAV-BS: A Deep Reinforcement Learning Approach. IEEE Wirel. Commun. Lett.; 2021; 10, pp. 2309-2312. [DOI: https://dx.doi.org/10.1109/LWC.2021.3100388]

29. Kang, H.; Joung, J.; Kang, J. Power-Efficient Formation of UAV Swarm: Just Like Flying Birds?. Proceedings of the GLOBECOM 2020—2020 IEEE Global Communications Conference; Taipei, Taiwan, 7–11 December 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOBECOM42002.2020.9322373]

30. Qi, H.; Hu, Z.; Huang, H.; Wen, X.; Lu, Z. Energy Efficient 3-D UAV Control for Persistent Communication Service and Fairness: A Deep Reinforcement Learning Approach. IEEE Access; 2020; 8, pp. 53172-53184. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2981403]

31. Petrov, V.; Gapeyenko, M.; Moltchanov, D.; Andreev, S.; Heath, R.W. Hover or Perch: Comparing Capacity of Airborne and Landed Millimeter-Wave UAV Cells. IEEE Wirel. Commun. Lett.; 2020; 9, pp. 2059-2063. [DOI: https://dx.doi.org/10.1109/LWC.2020.3012552]

32. Bayerlein, H.; Gangula, R.; Gesbert, D. Learning to Rest: A Q-Learning Approach to Flying Base Station Trajectory Design with Landing Spots. Proceedings of the 2018 52nd Asilomar Conference on Signals, Systems, and Computers; Pacific Grove, CA, USA, 28–31 October 2018; pp. 724-728. [DOI: https://dx.doi.org/10.1109/ACSSC.2018.8645103]

33. Shakoor, S.; Kaleem, Z.; Baig, M.I.; Chughtai, O.; Duong, T.Q.; Nguyen, L.D. Role of UAVs in Public Safety Communications: Energy Efficiency Perspective. IEEE Access; 2019; 7, pp. 140665-140679. [DOI: https://dx.doi.org/10.1109/ACCESS.2019.2942206]

34. Lahmeri, M.A.; Kishk, M.A.; Alouini, M.S. Artificial Intelligence for UAV-Enabled Wireless Networks: A Survey. IEEE Open J. Commun. Soc.; 2021; 2, pp. 1015-1040. [DOI: https://dx.doi.org/10.1109/OJCOMS.2021.3075201]

35. Carrio, A.; Sampedro, C.; Rodriguez-Ramos, A.; Campoy, P. A review of deep learning methods and applications for unmanned aerial vehicles. J. Sens.; 2017; 2017, 3296874. [DOI: https://dx.doi.org/10.1155/2017/3296874]

36. Busnel, Y.; Caillouet, C.; Coudert, D. Self-organized UAV-based Supervision and Connectivity: Challenges and Opportunities. Proceedings of the 2019 IEEE 18th International Symposium on Network Computing and Applications (NCA); Cambridge, MA, USA, 26–28 September 2019; pp. 1-5. [DOI: https://dx.doi.org/10.1109/NCA.2019.8935060]

37. Aliu, O.G.; Imran, A.; Imran, M.A.; Evans, B. A Survey of Self Organisation in Future Cellular Networks. IEEE Commun. Surv. Tutor.; 2013; 15, pp. 336-361. [DOI: https://dx.doi.org/10.1109/SURV.2012.021312.00116]

38. Klaine, P.V.; Imran, M.A.; Onireti, O.; Souza, R.D. A survey of machine learning techniques applied to self-organizing cellular networks. IEEE Commun. Surv. Tutor.; 2017; 19, pp. 2392-2431. [DOI: https://dx.doi.org/10.1109/COMST.2017.2727878]

39. Abubakar, A.I.; Omeke, K.G.; Ozturk, M.; Hussain, S.; Imran, M.A. The role of artificial intelligence driven 5G networks in COVID-19 outbreak: Opportunities, challenges, and future outlook. Front. Commun. Netw.; 2020; 1, 575065. [DOI: https://dx.doi.org/10.3389/frcmn.2020.575065]

40. Nonami, K.; Kendoul, F.; Suzuki, S.; Wang, W.; Nakazawa, D. Autonomous Flying Robots: Unmanned Aerial Vehicles and Micro Aerial Vehicles; Springer: Berlin/Heidelberg, Germany, 2010.

41. Saeed, A.S.; Younes, A.B.; Islam, S.; Dias, J.; Seneviratne, L.; Cai, G. A review on the platform design, dynamic modeling and control of hybrid UAVs. Proceedings of the 2015 International Conference on Unmanned Aircraft Systems (ICUAS); Colorado, CO, USA, 9–12 June 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 806-815.

42. Ducard, G.J.; Allenspach, M. Review of designs and flight control techniques of hybrid and convertible VTOL UAVs. Aerosp. Sci. Technol.; 2021; 118, 107035. [DOI: https://dx.doi.org/10.1016/j.ast.2021.107035]

43. Boukoberine, M.N.; Zhou, Z.; Benbouzid, M. A critical review on unmanned aerial vehicles power supply and energy management: Solutions, strategies, and prospects. Appl. Energy; 2019; 255, 113823. [DOI: https://dx.doi.org/10.1016/j.apenergy.2019.113823]

44. Khofiyah, N.A.; Maret, S.; Sutopo, W.; Nugroho, B.D.A. Goldsmith’s Commercialization Model for Feasibility Study of Technology Lithium Battery Pack Drone. Proceedings of the 2018 5th International Conference on Electric Vehicular Technology (ICEVT); Surakarta, Indonesia, 30–31 October 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 147-151.

45. Verstraete, D.; Lehmkuehler, K.; Wong, K. Design of a fuel cell powered blended wing body UAV. ASME International Mechanical Engineering Congress and Exposition; American Society of Mechanical Engineers: New York, NY, USA, 2012; Volume 45172, pp. 621-629.

46. Galkin, B.; Kibilda, J.; DaSilva, L.A. UAVs as mobile infrastructure: Addressing battery lifetime. IEEE Commun. Mag.; 2019; 57, pp. 132-137. [DOI: https://dx.doi.org/10.1109/MCOM.2019.1800545]

47. Lee, D.; Zhou, J.; Lin, W.T. Autonomous battery swapping system for quadcopter. Proceedings of the 2015 International Conference on Unmanned Aircraft Systems (ICUAS); Denver, CO, USA, 9–12 June 2015; pp. 118-124. [DOI: https://dx.doi.org/10.1109/ICUAS.2015.7152282]

48. Boukoberine, M.N.; Zhou, Z.; Benbouzid, M. Power supply architectures for drones-a review. Proceedings of the IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society; Lisbon, Portugal, 14–17 October 2019; IEEE: Piscataway, NJ, USA, 2019; Volume 1, pp. 5826-5831.

49. Simic, M.; Bil, C.; Vojisavljevic, V. Investigation in wireless power transmission for UAV charging. Procedia Comput. Sci.; 2015; 60, pp. 1846-1855. [DOI: https://dx.doi.org/10.1016/j.procs.2015.08.295]

50. Hassan, J.; Bokani, A.; Kanhere, S.S. Recharging of Flying Base Stations using Airborne RF Energy Sources. Proceedings of the 2019 IEEE Wireless Communications and Networking Conference Workshop (WCNCW); Marrakech, Morocco, 15–18 April 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/WCNCW.2019.8902900]

51. Kishk, M.A.; Bader, A.; Alouini, M.S. On the 3-D placement of airborne base stations using tethered UAVs. IEEE Trans. Commun.; 2020; 68, pp. 5202-5215. [DOI: https://dx.doi.org/10.1109/TCOMM.2020.2993885]

52. Gu, B.W.; Choi, S.Y.; Choi, Y.S.; Cai, G.; Seneviratne, L.; Rim, C.T. Novel roaming and stationary tethered aerial robots for continuous mobile missions in nuclear power plants. Nucl. Eng. Technol.; 2016; 48, pp. 982-996. [DOI: https://dx.doi.org/10.1016/j.net.2016.02.014]

53. Bushnaq, O.M.; Kishk, M.A.; Celik, A.; Alouini, M.S.; Al-Naffouri, T.Y. Optimal deployment of tethered drones for maximum cellular coverage in user clusters. IEEE Trans. Wirel. Commun.; 2020; 20, pp. 2092-2108. [DOI: https://dx.doi.org/10.1109/TWC.2020.3039013]

54. Matracia, M.; Kishk, M.A.; Alouini, M.S. On the Topological Aspects of UAV-Assisted Post-Disaster Wireless Communication Networks. arXiv; 2021; arXiv: 2109.13671[DOI: https://dx.doi.org/10.1109/MCOM.121.2100166]

55. Hwang, J.J.; Kuo, J.K.; Wu, W.; Chang, W.R.; Lin, C.H.; Wang, S.E. Lifecycle performance assessment of fuel cell/battery electric vehicles. Int. J. Hydrogen Energy; 2013; 38, pp. 3433-3446. [DOI: https://dx.doi.org/10.1016/j.ijhydene.2012.12.148]

56. Pan, Z.; An, L.; Wen, C. Recent advances in fuel cells based propulsion systems for unmanned aerial vehicles. Appl. Energy; 2019; 240, pp. 473-485. [DOI: https://dx.doi.org/10.1016/j.apenergy.2019.02.079]

57. Kendall, K.; Pollet, B. 4.12-Hydrogen and Fuel Cells in Transport. Compr. Renew. Energy; 2012; 4, 301.

58. Rhoads, G.; Bradley, T.; Wagner, N.; Taylor, B.; Keen, D. Design and flight test results for a 24 h fuel cell unmanned aerial vehicle. Proceedings of the 8th Annual International Energy Conversion Engineering Conference; Nashville, TN, USA, 25–28 July 2010; 6690.

59. Richardson, P.L. Upwind dynamic soaring of albatrosses and UAVs. Prog. Oceanogr.; 2015; 130, pp. 146-156. [DOI: https://dx.doi.org/10.1016/j.pocean.2014.11.002]

60. Bonnin, V.; Bénard, E.; Moschetta, J.M.; Toomer, C. Energy-harvesting mechanisms for UAV flight by dynamic soaring. Int. J. Micro Air Veh.; 2015; 7, pp. 213-229. [DOI: https://dx.doi.org/10.1260/1756-8293.7.3.213]

61. Oettershagen, P.; Melzer, A.; Mantel, T.; Rudin, K.; Lotz, R.; Siebenmann, D.; Leutenegger, S.; Alexis, K.; Siegwart, R. A solar-powered hand-launchable UAV for low-altitude multi-day continuous flight. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA); Seattle, WA, USA, 26–30 May 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 3986-3993.

62. Donateo, T.; Ficarella, A.; Spedicato, L.; Arista, A.; Ferraro, M. A new approach to calculating endurance in electric flight and comparing fuel cells and batteries. Appl. Energy; 2017; 187, pp. 807-819. [DOI: https://dx.doi.org/10.1016/j.apenergy.2016.11.100]

63. Belmonte, N.; Staulo, S.; Fiorot, S.; Luetto, C.; Rizzi, P.; Baricco, M. Fuel cell powered octocopter for inspection of mobile cranes: Design, cost analysis and environmental impacts. Appl. Energy; 2018; 215, pp. 556-565. [DOI: https://dx.doi.org/10.1016/j.apenergy.2018.02.072]

64. Sun, Y.; Xu, D.; Ng, D.W.K.; Dai, L.; Schober, R. Optimal 3D-trajectory design and resource allocation for solar-powered UAV communication systems. IEEE Trans. Commun.; 2019; 67, pp. 4281-4298. [DOI: https://dx.doi.org/10.1109/TCOMM.2019.2900630]

65. Verstraete, D.; Harvey, J.R.; Palmer, J.L. Hardware-in-the-loop simulation of fuel-cell-based hybrid-electrical UAV propulsion. Proceedings of the 28th Congress of the International Council of the Aeronautical Sciences; Brisbane, Australia, 23–28 September 2012; Volume 4, pp. 2662-2674.

66. Gong, A.; Verstraete, D. Role of battery in a hybrid electrical fuel cell UAV propulsion system. Proceedings of the 52nd AIAA Aerospace Sciences Meeting; National Harbor, MD, USA, 13–17 January 2014.

67. Lu, M.; Bagheri, M.; James, A.P.; Phung, T. Wireless charging techniques for UAVs: A review, reconceptualization, and extension. IEEE Access; 2018; 6, pp. 29865-29884. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2841376]

68. Morton, S.; D’Sa, R.; Papanikolopoulos, N. Solar powered UAV: Design and experiments. Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); Hamburg, Germany, 28 September–3 October 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 2460-2466.

69. Shiau, J.K.; Ma, D.M.; Yang, P.Y.; Wang, G.F.; Gong, J.H. Design of a solar power management system for an experimental UAV. IEEE Trans. Aerosp. Electron. Syst.; 2009; 45, pp. 1350-1360. [DOI: https://dx.doi.org/10.1109/TAES.2009.5310303]

70. Sambo, Y.A.; Klaine, P.V.; Nadas, J.P.B.; Imran, M.A. Energy Minimization UAV Trajectory Design for Delay-Tolerant Emergency Communication. Proceedings of the 2019 IEEE International Conference on Communications Workshops (ICC Workshops); Shanghai, China, 20–24 May 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICCW.2019.8757127]

71. Jin, W.; Yang, J.; Fang, Y.; Feng, W. Research on Application and Deployment of UAV in Emergency Response. Proceedings of the 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC); Beijing, China, 17–19 July 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 277-280.

72. Deepak, G.C.; Ladas, A.; Sambo, Y.A.; Pervaiz, H.; Politis, C.; Imran, M.A. An Overview of Post-Disaster Emergency Communication Systems in the Future Networks. IEEE Wirel. Commun.; 2019; 26, pp. 132-139. [DOI: https://dx.doi.org/10.1109/MWC.2019.1800467]

73. Aasen, H.; Honkavaara, E.; Lucieer, A.; Zarco-Tejada, P.J. Quantitative remote sensing at ultra-high resolution with UAV spectroscopy: A review of sensor technology, measurement procedures, and data correction workflows. Remote Sens.; 2018; 10, 1091. [DOI: https://dx.doi.org/10.3390/rs10071091]

74. Singh, S.; Malik, A.; Kumar, R.; Singh, P.K. A proficient data gathering technique for unmanned aerial vehicle-enabled heterogeneous wireless sensor networks. Int. J. Commun. Syst.; 2021; 34, e4956. [DOI: https://dx.doi.org/10.1002/dac.4956]

75. Tsouros, D.C.; Bibi, S.; Sarigiannidis, P.G. A review on UAV-based applications for precision agriculture. Information; 2019; 10, 349. [DOI: https://dx.doi.org/10.3390/info10110349]

76. Maddikunta, P.K.R.; Hakak, S.; Alazab, M.; Bhattacharya, S.; Gadekallu, T.R.; Khan, W.Z.; Pham, Q.V. Unmanned aerial vehicles in smart agriculture: Applications, requirements, and challenges. IEEE Sens. J.; 2021; 21, pp. 17608-17619. [DOI: https://dx.doi.org/10.1109/JSEN.2021.3049471]

77. Feng, T.; Xie, L.; Yao, J.; Xu, J. UAV-Enabled Data Collection for Wireless Sensor Networks with Distributed Beamforming. IEEE Trans. Wirel. Commun.; 2021; [DOI: https://dx.doi.org/10.1109/TWC.2021.3103739]

78. Bayerlein, H.; Theile, M.; Caccamo, M.; Gesbert, D. UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach. Proceedings of the GLOBECOM 2020—2020 IEEE Global Communications Conference; Virtual, 7–11 December 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOBECOM42002.2020.9322234]

79. Liu, Y.; Dai, H.N.; Wang, H.; Imran, M.; Wang, X.; Shoaib, M. UAV-enabled data acquisition scheme with directional wireless energy transfer for Internet of Things. Comput. Commun.; 2020; 155, pp. 184-196. [DOI: https://dx.doi.org/10.1016/j.comcom.2020.03.020]

80. Liu, D.; Chen, B.; Yang, C.; Molisch, A.F. Caching at the wireless edge: Design aspects, challenges, and future directions. IEEE Commun. Mag.; 2016; 54, pp. 22-28. [DOI: https://dx.doi.org/10.1109/MCOM.2016.7565183]

81. Dai, H.; Zhang, L.; Bian, H.; Wang, B. UAV relaying assisted transmission optimization with caching in vehicular networks. Phys. Commun.; 2020; 43, 101214. [DOI: https://dx.doi.org/10.1016/j.phycom.2020.101214]

82. Wu, H.; Lyu, F.; Zhou, C.; Chen, J.; Wang, L.; Shen, X. Optimal UAV Caching and Trajectory in Aerial-Assisted Vehicular Networks: A Learning-Based Approach. IEEE J. Sel. Areas Commun.; 2020; 38, pp. 2783-2797. [DOI: https://dx.doi.org/10.1109/JSAC.2020.3005469]

83. Mach, P.; Becvar, Z. Mobile Edge Computing: A Survey on Architecture and Computation Offloading. IEEE Commun. Surv. Tutor.; 2017; 19, pp. 1628-1656. [DOI: https://dx.doi.org/10.1109/COMST.2017.2682318]

84. Cui, Y.; Zheng, J.; Wang, X.; Diao, X.; Cai, Y.; Liu, S. UAV-enabled MEC system: Locations and coverage optimization. Proceedings of the IET 8th International Conference on Wireless, Mobile Multimedia Networks; Beijing, China, 12–15 October 2019; pp. 12-18. [DOI: https://dx.doi.org/10.1049/cp.2019.1142]

85. Wang, H.; Ke, H.; Sun, W. Unmanned-Aerial-Vehicle-Assisted Computation Offloading for Mobile Edge Computing Based on Deep Reinforcement Learning. IEEE Access; 2020; 8, pp. 180784-180798. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.3028553]

86. Sanike, A.; Subramanyam, A.; Reddy, S.S.S.; RaghuRam, G. Load balancing technique to handle the congestion in the communication networks. Proceedings of the 2015 Conference on Power, Control, Communication and Computational Technologies for Sustainable Growth (PCCCTSG); Kurnool, India, 11–12 December 2015; pp. 289-293. [DOI: https://dx.doi.org/10.1109/PCCCTSG.2015.7503919]

87. Bor-Yaliniz, I.; Yanikomeroglu, H. The New Frontier in RAN Heterogeneity: Multi-Tier Drone-Cells. IEEE Commun. Mag.; 2016; 54, pp. 48-55. [DOI: https://dx.doi.org/10.1109/MCOM.2016.1600178CM]

88. Hu, J.; Zhang, H.; Liu, Y.; Li, X.; Ji, H. An Intelligent UAV Deployment Scheme for Load Balance in Small Cell Networks Using Machine Learning. Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC); Marrakesh, Morocco, 15–18 April 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/WCNC.2019.8885648]

89. Fan, Q.; Ansari, N. Towards Traffic Load Balancing in Drone-Assisted Communications for IoT. IEEE Internet Things J.; 2019; 6, pp. 3633-3640. [DOI: https://dx.doi.org/10.1109/JIOT.2018.2889503]

90. Saxena, V.; Jaldén, J.; Klessig, H. Optimal UAV Base Station Trajectories Using Flow-Level Models for Reinforcement Learning. IEEE Trans. Cogn. Commun. Netw.; 2019; 5, pp. 1101-1112. [DOI: https://dx.doi.org/10.1109/TCCN.2019.2948324]

91. Hayat, S.; Yanmaz, E.; Muzaffar, R. Survey on unmanned aerial vehicle networks for civil applications: A communications viewpoint. IEEE Commun. Surv. Tutor.; 2016; 18, pp. 2624-2661. [DOI: https://dx.doi.org/10.1109/COMST.2016.2560343]

92. Al-Hourani, A.; Kandeepan, S.; Jamalipour, A. Modeling air-to-ground path loss for low altitude platforms in urban environments. Proceedings of the 2014 IEEE Global Communications Conference; Sydney, Australia, 8–12 December 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 2898-2904.

93. Jaziri, A.; Nasri, R.; Chahed, T. Congestion mitigation in 5G networks using drone relays. Proceedings of the 2016 International Wireless Communications and Mobile Computing Conference (IWCMC); Paphos, Cyprus, 5–9 September 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 233-238.

94. Zhan, P.; Yu, K.; Swindlehurst, A.L. Wireless relay communications with unmanned aerial vehicles: Performance and optimization. IEEE Trans. Aerosp. Electron. Syst.; 2011; 47, pp. 2068-2085. [DOI: https://dx.doi.org/10.1109/TAES.2011.5937283]

95. Zeng, Y.; Zhang, R.; Lim, T.J. Throughput maximization for mobile relaying systems. Proceedings of the 2016 IEEE Globecom Workshops (GC Wkshps); Washington, DC, USA, 4–8 December 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 1-6.

96. Li, Y.; Yang, D.; Xu, Y.; Xiao, L.; Chen, H. Throughput maximization for UAV-enabled relaying in wireless powered communication networks. Sensors; 2019; 19, 2989. [DOI: https://dx.doi.org/10.3390/s19132989] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31284622]

97. Zaidi, S.K.; Hasan, S.F.; Gui, X.; Siddique, N.; Ahmad, S. Exploiting UAV as NOMA based Relay for Coverage Extension. Proceedings of the 2019 2nd International Conference on Computer Applications Information Security (ICCAIS); Riyadh, Saudi Arabia, 1–3 May 2019; pp. 1-5. [DOI: https://dx.doi.org/10.1109/CAIS.2019.8769542]

98. You, X.; Wang, C.X.; Huang, J.; Gao, X.; Zhang, Z.; Wang, M.; Huang, Y.; Zhang, C.; Jiang, Y.; Wang, J. et al. Towards 6G wireless communication networks: Vision, enabling technologies, and new paradigm shifts. Sci. China Inf. Sci.; 2021; 64, pp. 1-74. [DOI: https://dx.doi.org/10.1007/s11432-020-2955-6]

99. Saad, W.; Bennis, M.; Chen, M. A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems. IEEE Netw.; 2020; 34, pp. 134-142. [DOI: https://dx.doi.org/10.1109/MNET.001.1900287]

100. Sharma, V.; Bennis, M.; Kumar, R. UAV-assisted heterogeneous networks for capacity enhancement. IEEE Commun. Lett.; 2016; 20, pp. 1207-1210. [DOI: https://dx.doi.org/10.1109/LCOMM.2016.2553103]

101. Song, Q.; Zheng, F.C.; Jin, S. Multiple UAVs Enabled Data Offloading for Cellular Hotspots. Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC); Marrakesh, Morocco, 15–18 April 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/WCNC.2019.8885574]

102. Challita, U.; Saad, W. Network Formation in the Sky: Unmanned Aerial Vehicles for Multi-Hop Wireless Backhauling. Proceedings of the GLOBECOM 2017—2017 IEEE Global Communications Conference; Singapore, 4–8 December 2017; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOCOM.2017.8254715]

103. Gapeyenko, M.; Petrov, V.; Moltchanov, D.; Andreev, S.; Himayat, N.; Koucheryavy, Y. Flexible and Reliable UAV-Assisted Backhaul Operation in 5G mmWave Cellular Networks. IEEE J. Sel. Areas Commun.; 2018; 36, pp. 2486-2496. [DOI: https://dx.doi.org/10.1109/JSAC.2018.2874145]

104. Khallaf, H.S.; Uysal, M. Comprehensive study on UAV-based FSO links for high-speed train backhauling. Appl. Opt.; 2021; 60, pp. 8239-8247. [DOI: https://dx.doi.org/10.1364/AO.435970]

105. Alsharoa, A.; Ghazzai, H.; Kadri, A.; Kamal, A.E. Energy management in cellular hetnets assisted by solar powered drone small cells. Proceedings of the 2017 IEEE Wireless Communications and Networking Conference (WCNC); San Francisco, CA, USA, 19–22 March 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1-6.

106. Chang, W.; Meng, Z.T.; Liu, K.C.; Wang, L.C. Energy-Efficient Sleep Strategy for the UBS-Assisted Small-Cell Network. IEEE Trans. Veh. Technol.; 2021; 70, pp. 5178-5183. [DOI: https://dx.doi.org/10.1109/TVT.2021.3075603]

107. Chakareski, J.; Naqvi, S.; Mastronarde, N.; Xu, J.; Afghah, F.; Razi, A. An Energy Efficient Framework for UAV-Assisted Millimeter Wave 5G Heterogeneous Cellular Networks. IEEE Trans. Green Commun. Netw.; 2019; 3, pp. 37-44. [DOI: https://dx.doi.org/10.1109/TGCN.2019.2892141]

108. Ouyang, J.; Zhuang, Y.; Lin, M.; Liu, J. Optimization of beamforming and path planning for UAV-assisted wireless relay networks. Chin. J. Aeronaut.; 2014; 27, pp. 313-320. [DOI: https://dx.doi.org/10.1016/j.cja.2014.02.011]

109. Chen, X.; Tang, J.; Lao, S. Review of unmanned aerial vehicle swarm communication architectures and routing protocols. Appl. Sci.; 2020; 10, 3661. [DOI: https://dx.doi.org/10.3390/app10103661]

110. Hentati, A.I.; Fourati, L.C. Comprehensive survey of UAVs communication networks. Comput. Stand. Interfaces; 2020; 72, 103451. [DOI: https://dx.doi.org/10.1016/j.csi.2020.103451]

111. Zeng, Y.; Zhang, R.; Lim, T.J. Wireless communications with unmanned aerial vehicles: Opportunities and challenges. IEEE Commun. Mag.; 2016; 54, pp. 36-42. [DOI: https://dx.doi.org/10.1109/MCOM.2016.7470933]

112. Zhong, Y.; Quek, T.Q.; Ge, X. Heterogeneous cellular networks with spatio-temporal traffic: Delay analysis and scheduling. IEEE J. Sel. Areas Commun.; 2017; 35, pp. 1373-1386. [DOI: https://dx.doi.org/10.1109/JSAC.2017.2687379]

113. Zeng, Y.; Zhang, R. Energy-Efficient UAV Communication With Trajectory Optimization. IEEE Trans. Wirel. Commun.; 2017; 16, pp. 3747-3760. [DOI: https://dx.doi.org/10.1109/TWC.2017.2688328]

114. Abeywickrama, H.V.; Jayawickrama, B.A.; He, Y.; Dutkiewicz, E. Comprehensive Energy Consumption Model for Unmanned Aerial Vehicles, Based on Empirical Studies of Battery Performance. IEEE Access; 2018; 6, pp. 58383-58394. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2875040]

115. Wu, Y.; Yang, W.; Guan, X.; Wu, Q. Energy-Efficient Trajectory Design for UAV-Enabled Communication Under Malicious Jamming. IEEE Wirel. Commun. Lett.; 2021; 10, pp. 206-210. [DOI: https://dx.doi.org/10.1109/LWC.2020.3024548]

116. Alzenad, M.; El-Keyi, A.; Lagum, F.; Yanikomeroglu, H. 3-D Placement of an Unmanned Aerial Vehicle Base Station (UAV-BS) for Energy-Efficient Maximal Coverage. IEEE Wirel. Commun. Lett.; 2017; 6, pp. 434-437. [DOI: https://dx.doi.org/10.1109/LWC.2017.2700840]

117. Wang, L.; Hu, B.; Chen, S.; Cui, J. UAV-enabled reliable mobile relaying based on downlink NOMA. IEEE Access; 2020; 8, pp. 25237-25248. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2970206]

118. You, J.; Jung, S.; Seo, J.; Kang, J. Energy-Efficient 3-D Placement of an Unmanned Aerial Vehicle Base Station With Antenna Tilting. IEEE Commun. Lett.; 2020; 24, pp. 1323-1327. [DOI: https://dx.doi.org/10.1109/LCOMM.2020.2979437]

119. Plachy, J.; Becvar, Z. Energy Efficient Positioning of Flying Base Stations via Coulomb’s law. Proceedings of the 2020 IEEE Globecom Workshops (GC Wkshps); Taipei, Taiwan, 7–11 December 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GCWkshps50303.2020.9367495]

120. Faqir, O.J.; Kerrigan, E.C.; Gündüz, D. Joint optimization of transmission and propulsion in aerial communication networks. Proceedings of the 2017 IEEE 56th Annual Conference on Decision and Control (CDC); Melbourne, Australia, 12–15 December 2017; pp. 3955-3960. [DOI: https://dx.doi.org/10.1109/CDC.2017.8264241]

121. Li, T.; Sheng, M.; Lyu, R.; Liu, J.; Li, J. UAV assisted heterogeneous wireless networks: Potentials and challenges. ZTE Commun.; 2018; 16, [DOI: https://dx.doi.org/10.3969/j.issn.1673-5188.2018.02.002]

122. Shehzad, M.K.; Akhtar, M.W.; Hassan, S.A. Performance of mmWave UAV-Assisted 5G Hybrid Heterogeneous Networks. Auton. Airborne Wirel. Netw.; 2021; pp. 97-118. [DOI: https://dx.doi.org/10.1002/9781119751717.ch6]

123. Rothlauf, F. Optimization methods. Design of Modern Heuristics; Springer: Berlin/Heidelberg, Germany, 2011; pp. 45-102.

124. Chen, S.; Shi, L.; Ding, X.; Lv, Z.; Li, Z. Energy Efficient Resource Allocation and Trajectory Optimization in UAV-Assisted Mobile Edge Computing System. Proceedings of the 2021 7th International Conference on Big Data Computing and Communications (BigCom); Deqing, China, 13–15 August 2021; pp. 7-13. [DOI: https://dx.doi.org/10.1109/BigCom53800.2021.00004]

125. Jeong, H.Y.; Song, B.D.; Lee, S. Truck-drone hybrid delivery routing: Payload-energy dependency and No-Fly zones. Int. J. Prod. Econ.; 2019; 214, pp. 220-233. [DOI: https://dx.doi.org/10.1016/j.ijpe.2019.01.010]

126. Gogna, A.; Tayal, A. Metaheuristics: Review and application. J. Exp. Theor. Artif. Intell.; 2013; 25, pp. 503-526. [DOI: https://dx.doi.org/10.1080/0952813X.2013.782347]

127. Holland, J.H. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence; MIT Press: Cambridge, MA, USA, 1992.

128. Katoch, S.; Chauhan, S.S.; Kumar, V. A review on genetic algorithm: Past, present, and future. Multimed. Tools Appl.; 2020; 80, pp. 8091-8126. [DOI: https://dx.doi.org/10.1007/s11042-020-10139-6] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33162782]

129. Shivgan, R.; Dong, Z. Energy-Efficient Drone Coverage Path Planning using Genetic Algorithm. Proceedings of the 2020 IEEE 21st International Conference on High Performance Switching and Routing (HPSR); Newark, NJ, USA, 11–14 May 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/HPSR48589.2020.9098989]

130. Angadi, B.M.; Kakkasageri, M.S.; Manvi, S.S. Computational intelligence techniques for localization and clustering in wireless sensor networks. Recent Trends in Computational Intelligence Enabled Research; Elsevier: Amsterdam, The Netherlands, 2021; pp. 23-40.

131. Kennedy, J.; Eberhart, R. Particle swarm optimization. Proceedings of the ICNN’95—International Conference on Neural Networks; Perth, WA, Australia, 27 November–1 December 1995; Volume 4, pp. 1942-1948. [DOI: https://dx.doi.org/10.1109/ICNN.1995.488968]

132. Wang, D.; Tan, D.; Liu, L. Particle swarm optimization algorithm: An overview. Soft Comput.; 2017; 22, pp. 387-408. [DOI: https://dx.doi.org/10.1007/s00500-016-2474-6]

133. Shi, Y. Particle swarm optimization: Developments, applications and resources. Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546); Seoul, Republic of Korea, 27–30 May 2001; Volume 1, pp. 81-86. [DOI: https://dx.doi.org/10.1109/CEC.2001.934374]

134. Dorigo, M.; Maniezzo, V.; Colorni, A. Ant system: Optimization by a colony of cooperating agents. IEEE Trans. Syst. Man Cybern. Part B; 1996; 26, pp. 29-41. [DOI: https://dx.doi.org/10.1109/3477.484436] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18263004]

135. Dorigo, M.; Stutzle, T. Ant Colony Optimization: Overview and Recent Advances. Handbook of Metaheuristics; Springer: Berlin/Heidelberg, Germany, 2010; pp. 227-263. [DOI: https://dx.doi.org/10.1007/978-1-4419-1665-5-8]

136. Kirkpatrick, S.; Gelatt, C.D.; Vecchi, M.P. Optimization by Simulated Annealing. Science; 1983; 220, pp. 671-680. [DOI: https://dx.doi.org/10.1126/science.220.4598.671]

137. Mladenović, N.; Hansen, P. Variable neighborhood search. Comput. Oper. Res.; 1997; 24, pp. 1097-1100. [DOI: https://dx.doi.org/10.1016/S0305-0548(97)00031-2]

138. Loyola-Gonzalez, O. Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view. IEEE Access; 2019; 7, pp. 154096-154113. [DOI: https://dx.doi.org/10.1109/ACCESS.2019.2949286]

139. Horst, R.; Tuy, H. Global Optimization: Deterministic Approaches; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013.

140. Papadimitriou, C.H.; Steiglitz, K. Combinatorial Optimization: Algorithms and Complexity; Courier Corporation: North Chelmsford, MA, USA, 1998.

141. Radmanesh, M.; Kumar, M. Flight formation of UAVs in presence of moving obstacles using fast-dynamic mixed integer linear programming. Aerosp. Sci. Technol.; 2016; 50, pp. 149-160. [DOI: https://dx.doi.org/10.1016/j.ast.2015.12.021]

142. Ahmed, S.; Mohamed, A.; Harras, K.; Kholief, M.; Mesbah, S. Energy efficient path planning techniques for UAV-based systems with space discretization. Proceedings of the 2016 IEEE Wireless Communications and Networking Conference; Doha, Qatar, 3–6 April 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 1-6.

143. Lin, X.; Wang, Z.; Zeng, S.; Huang, W.; Li, X. Real-time optimization strategy by using sequence quadratic programming with multivariate nonlinear regression for a fuel cell electric vehicle. Int. J. Hydrogen Energy; 2021; 46, pp. 13240-13251. [DOI: https://dx.doi.org/10.1016/j.ijhydene.2021.01.125]

144. Jung, S.; Kim, J.; Kim, J.H. Joint message-passing and convex optimization framework for energy-efficient surveillance UAV scheduling. Electronics; 2020; 9, 1475. [DOI: https://dx.doi.org/10.3390/electronics9091475]

145. Ahmed, S.; Chowdhury, M.Z.; Jang, Y.M. Energy-Efficient UAV-to-User Scheduling to Maximize Throughput in Wireless Networks. IEEE Access; 2020; 8, pp. 21215-21225. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2969357]

146. Lu, G.; Zhang, J.; Xiang, L.; Ge, X. A Global Optimization Method for Energy-Minimal UAV-Aided Data Collection over Fixed Flight Path. Proceedings of the ICC 2022-IEEE International Conference on Communications; Seoul, Republic of Korea, 16–20 May 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1220-1226.

147. Mohiuddin, A.; Taha, T.; Zweiri, Y.; Gan, D. Dual-UAV Payload Transportation Using Optimized Velocity Profiles via Real-Time Dynamic Programming. Drones; 2023; 7, 171. [DOI: https://dx.doi.org/10.3390/drones7030171]

148. Mahdavi, M.; Alhelou, H.H.; Hatziargyriou, N.D.; Al-Hinai, A. An efficient mathematical model for distribution system reconfiguration using AMPL. IEEE Access; 2021; 9, pp. 79961-79993. [DOI: https://dx.doi.org/10.1109/ACCESS.2021.3083688]

149. Bynum, M.L.; Hackebeil, G.A.; Hart, W.E.; Laird, C.D.; Nicholson, B.L.; Siirola, J.D.; Watson, J.P.; Woodruff, D.L. Pyomo-Optimization Modeling in Python; Springer: Berlin/Heidelberg, Germany, 2021; Volume 67.

150. Dunning, I.; Huchette, J.; Lubin, M. JuMP: A modeling language for mathematical optimization. SIAM Rev.; 2017; 59, pp. 295-320. [DOI: https://dx.doi.org/10.1137/15M1020575]

151. Nabli, H. An overview on the simplex algorithm. Appl. Math. Comput.; 2009; 210, pp. 479-489. [DOI: https://dx.doi.org/10.1016/j.amc.2009.01.013]

152. Den Hertog, D. Interior Point Approach to Linear, Quadratic and Convex Programming: Algorithms and Complexity; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2012; Volume 277.

153. Boyd, S.; Mattingley, J. Branch and bound methods. Notes EE364b, Stanf. Univ.; 2007; 2006, 7.

154. Mitchell, J.E. Branch-and-cut algorithms for combinatorial optimization problems. Handb. Appl. Optim.; 2002; 1, pp. 65-77.

155. Zhang, C.; Patras, P.; Haddadi, H. Deep Learning in Mobile and Wireless Networking: A Survey. IEEE Commun. Surv. Tutor.; 2019; 21, pp. 2224-2287. [DOI: https://dx.doi.org/10.1109/COMST.2019.2904897]

156. Singh, A.; Thakur, N.; Sharma, A. A review of supervised machine learning algorithms. Proceedings of the 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom); New Delhi, India, 16–18 March 2016; pp. 1310-1315.

157. Kotsiantis, S.B.; Zaharakis, I.; Pintelas, P. Supervised machine learning: A review of classification techniques. Emerg. Artif. Intell. Appl. Comput. Eng.; 2007; 160, pp. 3-24.

158. Chen, S.; Goo, Y.J.J.; Shen, Z.D. A Hybrid Approach of Stepwise Regression, Logistic Regression, Support Vector Machine, and Decision Tree for Forecasting Fraudulent Financial Statements. Sci. World J.; 2014; 2014, 968712. [DOI: https://dx.doi.org/10.1155/2014/968712]

159. Li, N.; Shepperd, M.; Guo, Y. A systematic review of unsupervised learning techniques for software defect prediction. Inf. Softw. Technol.; 2020; 122, 106287. [DOI: https://dx.doi.org/10.1016/j.infsof.2020.106287]

160. Sinaga, K.P.; Yang, M.S. Unsupervised K-Means Clustering Algorithm. IEEE Access; 2020; 8, pp. 80716-80727. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2988796]

161. Ben-David, S. A framework for statistical clustering with constant time approximation algorithms for K-median and K-means clustering. Mach. Learn.; 2006; 66, pp. 243-257. [DOI: https://dx.doi.org/10.1007/s10994-006-0587-3]

162. Yürüşen, N.Y.; Uzunoğlu, B.; Talayero, A.P.; Estopiñán, A.L. Apriori and K-Means algorithms of machine learning for spatio-temporal solar generation balancing. Renew. Energy; 2021; 175, pp. 702-717. [DOI: https://dx.doi.org/10.1016/j.renene.2021.04.098]

163. Zebari, R.; Abdulazeez, A.; Zeebaree, D.; Zebari, D.; Saeed, J. A Comprehensive Review of Dimensionality Reduction Techniques for Feature Selection and Feature Extraction. J. Appl. Sci. Technol. Trends; 2020; 1, pp. 56-70. [DOI: https://dx.doi.org/10.38094/jastt1224]

164. van Engelen, J.E.; Hoos, H.H. A survey on semi-supervised learning. Mach. Learn.; 2019; 109, pp. 373-440. [DOI: https://dx.doi.org/10.1007/s10994-019-05855-6]

165. Wason, R. Deep learning: Evolution and expansion. Cogn. Syst. Res.; 2018; 52, pp. 701-708. [DOI: https://dx.doi.org/10.1016/j.cogsys.2018.08.023]

166. Alom, M.Z.; Taha, T.M.; Yakopcic, C.; Westberg, S.; Sidike, P.; Nasrin, M.S.; Hasan, M.; Essen, B.C.V.; Awwal, A.A.S.; Asari, V.K. A State-of-the-Art Survey on Deep Learning Theory and Architectures. Electronics; 2019; 8, 292. [DOI: https://dx.doi.org/10.3390/electronics8030292]

167. Pouyanfar, S.; Sadiq, S.; Yan, Y.; Tian, H.; Tao, Y.; Reyes, M.P.; Shyu, M.L.; Chen, S.C.; Iyengar, S.S. A Survey on Deep Learning: Algorithms, Techniques, and Applications. ACM Comput. Surv.; 2018; 51, [DOI: https://dx.doi.org/10.1145/3234150]

168. Dong, S.; Wang, P.; Abbas, K. A survey on deep learning and its applications. Comput. Sci. Rev.; 2021; 40, 100379. [DOI: https://dx.doi.org/10.1016/j.cosrev.2021.100379]

169. Nguyen, T.T.; Nguyen, N.D.; Nahavandi, S. Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications. IEEE Trans. Cybern.; 2020; 50, pp. 3826-3839. [DOI: https://dx.doi.org/10.1109/TCYB.2020.2977374]

170. Kober, J.; Bagnell, J.A.; Peters, J. Reinforcement learning in robotics: A survey. Int. J. Robot. Res.; 2013; 32, pp. 1238-1274. [DOI: https://dx.doi.org/10.1177/0278364913495721]

171. Vázquez-Canteli, J.R.; Nagy, Z. Reinforcement learning for demand response: A review of algorithms and modeling techniques. Appl. Energy; 2019; 235, pp. 1072-1089. [DOI: https://dx.doi.org/10.1016/j.apenergy.2018.11.002]

172. McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; y Arcas, B.A. Communication-efficient learning of deep networks from decentralized data. Proceedings of the Artificial Intelligence and Statistics; Lauderdale, FL, USA, 20 April 2017; pp. 1273-1282.

173. Li, L.; Fan, Y.; Tse, M.; Lin, K.Y. A review of applications in federated learning. Comput. Ind. Eng.; 2020; 149, 106854. [DOI: https://dx.doi.org/10.1016/j.cie.2020.106854]

174. Li, T.; Sahu, A.K.; Talwalkar, A.; Smith, V. Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Process. Mag.; 2020; 37, pp. 50-60. [DOI: https://dx.doi.org/10.1109/MSP.2020.2975749]

175. Nguyen, D.C.; Ding, M.; Pathirana, P.N.; Seneviratne, A.; Li, J.; Poor, H.V. Federated Learning for Internet of Things: A Comprehensive Survey. IEEE Commun. Surv. Tutor.; 2021; 23, pp. 1622-1658. [DOI: https://dx.doi.org/10.1109/COMST.2021.3075439]

176. Mozaffari, M.; Saad, W.; Bennis, M.; Debbah, M. Drone Small Cells in the Clouds: Design, Deployment and Performance Analysis. Proceedings of the 2015 IEEE Global Communications Conference (GLOBECOM); San Diego, CA, USA, 6–10 December 2015; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOCOM.2015.7417609]

177. Khalil, M.I. Energy Efficiency Maximization of Relay Aerial Robotic Networks. IEEE Trans. Green Commun. Netw.; 2020; 4, pp. 1081-1090. [DOI: https://dx.doi.org/10.1109/TGCN.2020.3007814]

178. Babu, N.; Ntougias, K.; Papadias, C.B.; Popovski, P. Energy Efficient Altitude Optimization of an Aerial Access Point. Proceedings of the 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications; London, UK, 31 August–3 September 2020; pp. 1-7. [DOI: https://dx.doi.org/10.1109/PIMRC48278.2020.9217265]

179. Wang, L.; Hu, B.; Chen, S. Energy Efficient Placement of a Drone Base Station for Minimum Required Transmit Power. IEEE Wirel. Commun. Lett.; 2020; 9, pp. 2010-2014. [DOI: https://dx.doi.org/10.1109/LWC.2018.2808957]

180. Chou, S.F.; Pang, A.C.; Yu, Y.J. Energy-Aware 3D Unmanned Aerial Vehicle Deployment for Network Throughput Optimization. IEEE Trans. Wirel. Commun.; 2020; 19, pp. 563-578. [DOI: https://dx.doi.org/10.1109/TWC.2019.2946822]

181. Bera, A.; Misra, S.; Chatterjee, C. Energy-Aware Multi-UAV Networks for On-Demand Task Execution. Proceedings of the 2020 IEEE International Conference on Communications Workshops (ICC Workshops); Dublin, Ireland, 7–11 June 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICCWorkshops49005.2020.9145291]

182. Lu, J.; Wan, S.; Chen, X.; Fan, P. Energy-Efficient 3D UAV-BS Placement versus Mobile Users’ Density and Circuit Power. Proceedings of the 2017 IEEE Globecom Workshops (GC Wkshps); Singapore, 4–8 December 2017; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOCOMW.2017.8269064]

183. Babu, N.; Papadias, C.B.; Popovski, P. Energy-Efficient Deployment of a Non-Orthogonal Multiple Access Unmanned Aerial System. Proceedings of the 2021 IEEE International Conference on Communications Workshops (ICC Workshops); Montreal, QC, Canada, 14–23 June 2021; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICCWorkshops50388.2021.9473727]

184. Xue, Z.; Wang, J.; Ding, G.; Wu, Q. Joint 3D Location and Power Optimization for UAV-Enabled Relaying Systems. IEEE Access; 2018; 6, pp. 43113-43124. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2862385]

185. Shakoor, S.; Kaleem, Z.; Do, D.T.; Dobre, O.A.; Jamalipour, A. Joint Optimization of UAV 3-D Placement and Path-Loss Factor for Energy-Efficient Maximal Coverage. IEEE Internet Things J.; 2021; 8, pp. 9776-9786. [DOI: https://dx.doi.org/10.1109/JIOT.2020.3019065]

186. Zorbas, D.; Pugliese, L.D.P.; Razafindralambo, T.; Guerriero, F. Optimal drone placement and cost-efficient target coverage. J. Netw. Comput. Appl.; 2016; 75, pp. 16-31. [DOI: https://dx.doi.org/10.1016/j.jnca.2016.08.009]

187. Jiang, X.; Wu, Z.; Yin, Z.; Yang, Z.; Zhao, N. Power Consumption Minimization of UAV Relay in NOMA Networks. IEEE Wirel. Commun. Lett.; 2020; 9, pp. 666-670. [DOI: https://dx.doi.org/10.1109/LWC.2020.2965096]

188. Bahr, A.; Mehaseb, M.A.; Doliel, S.A.; El-Rabaie, S.; Abd El-Samie, F.E. Power-Aware 3D UAV Placement for IoT Emergency Communications. Proceedings of the 2020 8th International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC); Virtual, 14–15 December 2020; pp. 18-23. [DOI: https://dx.doi.org/10.1109/JAC-ECC51597.2020.9355853]

189. Cui, J.; Shakhatreh, H.; Hu, B.; Chen, S.; Wang, C. Power-Efficient Deployment of a UAV for Emergency Indoor Wireless Coverage. IEEE Access; 2018; 6, pp. 73200-73209. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2882896]

190. Pandey, A.; Kushwaha, D.; Kumar, S. Energy Efficient UAV Placement for Multiple Users in IoT Networks. Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM); Waikoloa, HI, USA, 9–13 December 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOBECOM38437.2019.9014078]

191. Bozkaya, E.; Foerster, K.T.; Schmid, S.; Canberk, B. AirNet: Energy-Aware Deployment and Scheduling of Aerial Networks. IEEE Trans. Veh. Technol.; 2020; 69, pp. 12252-12263. [DOI: https://dx.doi.org/10.1109/TVT.2020.3019918]

192. Sohail, M.F.; Leow, C.Y.; Won, S. Energy-Efficient Non-Orthogonal Multiple Access for UAV Communication System. IEEE Trans. Veh. Technol.; 2019; 68, pp. 10834-10845. [DOI: https://dx.doi.org/10.1109/TVT.2019.2939186]

193. Zhang, R.; Pang, X.; Tang, J.; Chen, Y.; Zhao, N.; Wang, X. Joint Location and Transmit Power Optimization for NOMA-UAV Networks via Updating Decoding Order. IEEE Wirel. Commun. Lett.; 2021; 10, pp. 136-140. [DOI: https://dx.doi.org/10.1109/LWC.2020.3023253]

194. Pang, X.; Tang, J.; Zhao, N.; Zhang, X.; Qian, Y. Energy-efficient design for mmWave-enabled NOMA-UAV networks. Sci. China Inf. Sci.; 2021; 64, pp. 1-14. [DOI: https://dx.doi.org/10.1007/s11432-020-2985-8]

195. Nouri, N.; Abouei, J.; Sepasian, A.R.; Jaseemuddin, M.; Anpalagan, A.; Plataniotis, K.N. Three-dimensional multi-UAV placement and resource allocation for energy-efficient IoT communication. IEEE Internet Things J.; 2021; 9, pp. 2134-2152. [DOI: https://dx.doi.org/10.1109/JIOT.2021.3091166]

196. Albu-Salih, A.T.; Seno, S.A.H. Energy-efficient data gathering framework-based clustering via multiple UAVs in deadline-based WSN applications. IEEE Access; 2018; 6, pp. 72275-72286. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2882161]

197. Cheng, S.H.; Shih, Y.T.; Chang, K.C. Proactive Power Control and Position Deployment for Drone Small Cells: Joint Supervised and Unsupervised Learning. IEEE Access; 2021; 9, pp. 126735-126747. [DOI: https://dx.doi.org/10.1109/ACCESS.2021.3111964]

198. Noh, S.C.; Jeon, H.B.; Chae, C.B. Energy-Efficient Deployment of Multiple UAVs Using Ellipse Clustering to Establish Base Stations. IEEE Wirel. Commun. Lett.; 2020; 9, pp. 1155-1159. [DOI: https://dx.doi.org/10.1109/LWC.2020.2982889]

199. Zhang, Q.; Mozaffari, M.; Saad, W.; Bennis, M.; Debbah, M. Machine Learning for Predictive On-Demand Deployment of UAVs for Wireless Communications. Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM); Abu Dhabi, United Arab Emirates, 9–13 December 2018; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOCOM.2018.8647209]

200. Liu, B.; Zhang, Y.; Fu, S.; Liu, X. Reduce UAV Coverage Energy Consumption through Actor-Critic Algorithm. Proceedings of the 2019 15th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN); Shenzhen, China, 11–13 December 2019; pp. 332-337. [DOI: https://dx.doi.org/10.1109/MSN48538.2019.00069]

201. Song, Q.; Jin, S.; Zheng, F.C. Completion Time and Energy Consumption Minimization for UAV-Enabled Multicasting. IEEE Wirel. Commun. Lett.; 2019; 8, pp. 821-824. [DOI: https://dx.doi.org/10.1109/LWC.2019.2894684]

202. Tran, D.H.; Vu, T.X.; Chatzinotas, S.; ShahbazPanahi, S.; Ottersten, B. Coarse Trajectory Design for Energy Minimization in UAV-Enabled. IEEE Trans. Veh. Technol.; 2020; 69, pp. 9483-9496. [DOI: https://dx.doi.org/10.1109/TVT.2020.3001403]

203. Jing, X.; Sun, J.; Masouros, C. Energy Aware Trajectory Optimization for Aerial Base Stations. IEEE Trans. Commun.; 2021; 69, pp. 3352-3366. [DOI: https://dx.doi.org/10.1109/TCOMM.2021.3055525]

204. Zhang, S.; Zhang, H.; He, Q.; Bian, K.; Song, L. Joint Trajectory and Power Optimization for UAV Relay Networks. IEEE Commun. Lett.; 2018; 22, pp. 161-164. [DOI: https://dx.doi.org/10.1109/LCOMM.2017.2763135]

205. Yang, D.; Wu, Q.; Zeng, Y.; Zhang, R. Energy Tradeoff in Ground-to-UAV Communication via Trajectory Design. IEEE Trans. Veh. Technol.; 2018; 67, pp. 6721-6726. [DOI: https://dx.doi.org/10.1109/TVT.2018.2816244]

206. Hua, M.; Wang, Y.; Wu, Q.; Dai, H.; Huang, Y.; Yang, L. Energy-Efficient Cooperative Secure Transmission in Multi-UAV-Enabled Wireless Networks. IEEE Trans. Veh. Technol.; 2019; 68, pp. 7761-7775. [DOI: https://dx.doi.org/10.1109/TVT.2019.2924180]

207. Huang, J.; Shan, F.; Xiong, R.; Shao, Y.; Luo, J. Energy-Efficient UAV Flight Planning for a General PoI-Visiting Problem with a Practical Energy Model. Proceedings of the 2021 International Conference on Computer Communications and Networks (ICCCN); Athens, Greece, 19–22 July 2021; pp. 1-10. [DOI: https://dx.doi.org/10.1109/ICCCN52240.2021.9522183]

208. Jiang, X.; Wu, Z.; Yin, Z.; Yang, Z. Joint Power and Trajectory Design for UAV-Relayed Wireless Systems. IEEE Wirel. Commun. Lett.; 2019; 8, pp. 697-700. [DOI: https://dx.doi.org/10.1109/LWC.2018.2885056]

209. Wang, H.; Ren, G.; Chen, J.; Ding, G.; Yang, Y. Unmanned Aerial Vehicle-Aided Communications: Joint Transmit Power and Trajectory Optimization. IEEE Wirel. Commun. Lett.; 2018; 7, pp. 522-525. [DOI: https://dx.doi.org/10.1109/LWC.2018.2792435]

210. Cui, M.; Zhang, G.; Wu, Q.; Ng, D.W.K. Robust Trajectory and Transmit Power Design for Secure UAV Communications. IEEE Trans. Veh. Technol.; 2018; 67, pp. 9042-9046. [DOI: https://dx.doi.org/10.1109/TVT.2018.2849644]

211. Khamidehi, B.; Sousa, E.S. Power Efficient Trajectory Optimization for the Cellular-Connected Aerial Vehicles. Proceedings of the 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC); Istanbul, Turkey, 8–11 September 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/PIMRC.2019.8904357]

212. Hong, Y.W.P.; Cheng, R.H.; Hsiao, Y.C.; Sheu, J.P. Power-Efficient Trajectory Adjustment and Temporal Routing for Multi-UAV Networks. IEEE Trans. Green Commun. Netw.; 2020; 4, pp. 1106-1119. [DOI: https://dx.doi.org/10.1109/TGCN.2020.3014101]

213. Xiang, L.; Lei, L.; Chatzinotas, S.; Ottersten, B.; Schober, R. Towards Power-Efficient Aerial Communications via Dynamic Multi-UAV Cooperation. Proceedings of the 2020 IEEE Wireless Communications and Networking Conference (WCNC); Seoul, Republic of Korea, 25–28 May 2020; pp. 1-7. [DOI: https://dx.doi.org/10.1109/WCNC45663.2020.9120587]

214. Lee, J.; Friderikos, V. Energy Consumption Fairness for Multiple Flying Base Stations. Proceedings of the 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications; London, UK, 31 August–3 September 2020; pp. 1-5. [DOI: https://dx.doi.org/10.1109/PIMRC48278.2020.9217252]

215. Gao, Y.; Tang, H.; Li, B.; Yuan, X. Energy Minimization for Robust Secure Transmission in UAV Networks With Multiple Colluding Eavesdroppers. IEEE Commun. Lett.; 2021; 25, pp. 2353-2357. [DOI: https://dx.doi.org/10.1109/LCOMM.2021.3067477]

216. Dong, F.; Li, L.; Lu, Z.; Pan, Q.; Zheng, W. Energy-Efficiency for Fixed-Wing UAV-Enabled Data Collection and Forwarding. Proceedings of the 2019 IEEE International Conference on Communications Workshops (ICC Workshops); Montreal, QC, Canada, 14–23 June 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICCW.2019.8757098]

217. Bian, H.; Dai, H.; Yang, L. Throughput and energy efficiency maximization for UAV-assisted vehicular networks. Phys. Commun.; 2020; 42, 101136. [DOI: https://dx.doi.org/10.1016/j.phycom.2020.101136]

218. Eom, S.; Lee, H.; Park, J.; Lee, I. UAV-Aided Wireless Communication Designs With Propulsion Energy Limitations. IEEE Trans. Veh. Technol.; 2020; 69, pp. 651-662. [DOI: https://dx.doi.org/10.1109/TVT.2019.2952883]

219. Miao, J.; Li, H.; Zheng, Z.; Wang, C. Secrecy Energy Efficiency Maximization for UAV Swarm Assisted Multi-Hop Relay System: Joint Trajectory Design and Power Control. IEEE Access; 2021; 9, pp. 37784-37799. [DOI: https://dx.doi.org/10.1109/ACCESS.2021.3062895]

220. Sun, Z.; Yang, D.; Xiao, L.; Cuthbert, L.; Wu, F.; Zhu, Y. Joint Energy and Trajectory Optimization for UAV-Enabled Relaying Network With Multi-Pair Users. IEEE Trans. Cogn. Commun. Netw.; 2021; 7, pp. 939-954. [DOI: https://dx.doi.org/10.1109/TCCN.2020.3048392]

221. Dai, R.; Fotedar, S.; Radmanesh, M.; Kumar, M. Quality-aware UAV coverage and path planning in geometrically complex environments. Ad Hoc Netw.; 2018; 73, pp. 95-105. [DOI: https://dx.doi.org/10.1016/j.adhoc.2018.02.008]

222. Xu, Y.; Xiao, L.; Yang, D.; Cuthbert, L.; Wang, Y. Energy-efficient UAV communication with multiple GTs based on trajectory optimization. Mob. Inf. Syst.; 2018; 2018, [DOI: https://dx.doi.org/10.1155/2018/5629573]

223. Liu, C.H.; Ma, X.; Gao, X.; Tang, J. Distributed Energy-Efficient Multi-UAV Navigation for Long-Term Communication Coverage by Deep Reinforcement Learning. IEEE Trans. Mob. Comput.; 2020; 19, pp. 1274-1285. [DOI: https://dx.doi.org/10.1109/TMC.2019.2908171]

224. Cong, J.; Li, B.; Guo, X.; Zhang, R. Energy Management Strategy based on Deep Q-network in the Solar-powered UAV Communications System. Proceedings of the 2021 IEEE International Conference on Communications Workshops (ICC Workshops); Montreal, QC, Canada, 14–23 June 2021; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICCWorkshops50388.2021.9473509]

225. Ding, R.; Gao, F.; Shen, X.S. 3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach. IEEE Trans. Wirel. Commun.; 2020; 19, pp. 7796-7809. [DOI: https://dx.doi.org/10.1109/TWC.2020.3016024]

226. Zhao, C.; Liu, J.; Sheng, M.; Teng, W.; Zheng, Y.; Li, J. Multi-UAV Trajectory Planning for Energy-Efficient Content Coverage: A Decentralized Learning-Based Approach. IEEE J. Sel. Areas Commun.; 2021; 39, pp. 3193-3207. [DOI: https://dx.doi.org/10.1109/JSAC.2021.3088669]

227. Liu, X.; Liu, Y.; Chen, Y.; Hanzo, L. Trajectory Design and Power Control for Multi-UAV Assisted Wireless Networks: A Machine Learning Approach. IEEE Trans. Veh. Technol.; 2019; 68, pp. 7957-7969. [DOI: https://dx.doi.org/10.1109/TVT.2019.2920284]

228. Amrallah, A.; Mohamed, E.M.; Tran, G.K.; Sakaguchi, K. UAV Trajectory Optimization in a Post-Disaster Area Using Dual Energy-Aware Bandits. Sensors; 2023; 23, 1402. [DOI: https://dx.doi.org/10.3390/s23031402] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36772443]

229. Lin, Y.; Wang, T.; Wang, S. UAV-Assisted Emergency Communications: An Extended Multi-Armed Bandit Perspective. IEEE Commun. Lett.; 2019; 23, pp. 938-941. [DOI: https://dx.doi.org/10.1109/LCOMM.2019.2906194]

230. Mukherjee, A.; Misra, S.; Chandra, V.S.P.; Obaidat, M.S. Resource-Optimized Multiarmed Bandit-Based Offload Path Selection in Edge UAV Swarms. IEEE Internet Things J.; 2019; 6, pp. 4889-4896. [DOI: https://dx.doi.org/10.1109/JIOT.2018.2879459]

231. Fan, W.; Luo, K.; Yu, S.; Zhou, Z.; Chen, X. AoI-driven Fresh Situation Awareness by UAV Swarm: Collaborative DRL-based Energy-Efficient Trajectory Control and Data Processing. Proceedings of the 2020 IEEE/CIC International Conference on Communications in China (ICCC); Chongqing, China, 9-11 August 2020; pp. 841-846. [DOI: https://dx.doi.org/10.1109/ICCC49849.2020.9238897]

232. Abeywickrama, H.V.; He, Y.; Dutkiewicz, E.; Jayawickrama, B.A.; Mueck, M. A Reinforcement Learning Approach for Fair User Coverage Using UAV Mounted Base Stations Under Energy Constraints. IEEE Open J. Veh. Technol.; 2020; 1, pp. 67-81. [DOI: https://dx.doi.org/10.1109/OJVT.2020.2971594]

233. Hajijamali Arani, A.; Azari, M.M.; Hu, P.; Zhu, Y.; Yanikomeroglu, H.; Safavi-Naeini, S. Reinforcement Learning for Energy-Efficient Trajectory Design of UAVs. IEEE Internet Things J.; 2022; 9, pp. 9060-9070. [DOI: https://dx.doi.org/10.1109/JIOT.2021.3118322]

234. Yu, J.; Zhu, Y.; Zhao, H.; Cepeda-Lopez, R.; Dagiuklas, T.; Gao, Y. Dynamic Coverage Path Planning of Energy Optimization in UAV-enabled Edge Computing Networks. Proceedings of the 2021 IEEE Wireless Communications and Networking Conference Workshops (WCNCW); Nanjing, China, 29 March 2021; pp. 1-6. [DOI: https://dx.doi.org/10.1109/WCNCW49093.2021.9419992]

235. Deng, C.; Xu, W.; Lee, C.H.; Gao, H.; Xu, W.; Feng, Z. Energy Efficient UAV-Enabled Multicast Systems: Joint Grouping and Trajectory Optimization. Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM); Big Island, HI, USA, 9–13 December 2019; pp. 1-7. [DOI: https://dx.doi.org/10.1109/GLOBECOM38437.2019.9013786]

236. Cui, Y.; Deng, D.; Wang, C.; Wang, W. Joint Trajectory and Power Optimization for Energy Efficient UAV Communication Using Deep Reinforcement Learning. Proceedings of the IEEE INFOCOM 2021—IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS); Virtually, 9–12 May 2021; pp. 1-6. [DOI: https://dx.doi.org/10.1109/INFOCOMWKSHPS51825.2021.9484490]

237. Ruan, L.; Wang, J.; Chen, J.; Xu, Y.; Yang, Y.; Jiang, H.; Zhang, Y.; Xu, Y. Energy-efficient multi-UAV coverage deployment in UAV networks: A game-theoretic framework. China Commun.; 2018; 15, pp. 194-209. [DOI: https://dx.doi.org/10.1109/CC.2018.8485481]

238. Jia, M.; Gao, Q.; Guo, Q.; Gu, X. Energy-Efficiency Power Allocation Design for UAV-Assisted Spatial NOMA. IEEE Internet Things J.; 2021; 8, pp. 15205-15215. [DOI: https://dx.doi.org/10.1109/JIOT.2020.3044090]

239. Zhang, H.; Zhang, J.; Long, K.; Nallanathan, A.; Leung, V.C.M. Resource Allocation for Energy Efficient NOMA UAV Network under Imperfect CSI. Proceedings of the ICC 2020—2020 IEEE International Conference on Communications (ICC); Dublin, Ireland, 7–11 June 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICC40277.2020.9148694]

240. Wu, C.; Shi, S.; GU, S.; Zhang, N.; GU, X. Energy Efficient Resource Allocation and Trajectory Design for Multi-UAV-Enabled Wireless Networks. Proceedings of the 2021 IEEE International Conference on Communications Workshops (ICC Workshops); Montreal, QC, Canada, 14–23 June 2021; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICCWorkshops50388.2021.9473827]

241. Zhang, T.; Liu, G.; Zhang, H.; Kang, W.; Karagiannidis, G.K.; Nallanathan, A. Energy-Efficient Resource Allocation and Trajectory Design for UAV Relaying Systems. IEEE Trans. Commun.; 2020; 68, pp. 6483-6498. [DOI: https://dx.doi.org/10.1109/TCOMM.2020.3009153]

242. Xu, K.; Zhao, M.M.; Cai, Y.; Hanzo, L. Low-Complexity Joint Power Allocation and Trajectory Design for UAV-Enabled Secure Communications With Power Splitting. IEEE Trans. Commun.; 2021; 69, pp. 1896-1911. [DOI: https://dx.doi.org/10.1109/TCOMM.2020.3042462]

243. Cai, Y.; Wei, Z.; Li, R.; Kwan Ng, D.W.; Yuan, J. Energy-Efficient Resource Allocation for Secure UAV Communication Systems. Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC); Marrakesh, Morocco, 15–18 April 2019; pp. 1-8. [DOI: https://dx.doi.org/10.1109/WCNC.2019.8885416]

244. Cai, Y.; Wei, Z.; Hu, S.; Ng, D.W.K.; Yuan, J. Resource Allocation for Power-Efficient IRS-Assisted UAV Communications. Proceedings of the 2020 IEEE International Conference on Communications Workshops (ICC Workshops); Dublin, Ireland, 7–11 June 2020; pp. 1-7. [DOI: https://dx.doi.org/10.1109/ICCWorkshops49005.2020.9145224]

245. Wang, X.; Feng, W.; Chen, Y.; Ge, N. Power Allocation for UAV Swarm-Enabled Secure Networks Using Large-Scale CSI. Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM); Big Island, HI, USA, 9–13 December 2019; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOBECOM38437.2019.9014165]

246. Yu, X.; Yao, J.; Ruan, L.; Yao, K.; Liu, D.; Chen, R. 3D Deployment of Multi-UAV for Energy-Saving: A Game-Based Learning Approach. Proceedings of the 2019 IEEE 5th International Conference on Computer and Communications (ICCC); Chengdu, China, 6–9 December 2019; pp. 1332-1337. [DOI: https://dx.doi.org/10.1109/ICCC47050.2019.9064450]

247. Masaracchia, A.; Nguyen, L.D.; Duong, T.Q.; Yin, C.; Dobre, O.A.; Garcia-Palacios, E. Energy-Efficient and Throughput Fair Resource Allocation for TS-NOMA UAV-Assisted Communications. IEEE Trans. Commun.; 2020; 68, pp. 7156-7169. [DOI: https://dx.doi.org/10.1109/TCOMM.2020.3014939]

248. Li, M.; Cheng, N.; Gao, J.; Wang, Y.; Zhao, L.; Shen, X. Energy-Efficient UAV-Assisted Mobile Edge Computing: Resource Allocation and Trajectory Optimization. IEEE Trans. Veh. Technol.; 2020; 69, pp. 3424-3438. [DOI: https://dx.doi.org/10.1109/TVT.2020.2968343]

249. Li, Z.; Wang, Y.; Liu, M.; Sun, R.; Chen, Y.; Yuan, J.; Li, J. Energy efficient resource allocation for UAV-assisted space-air-ground Internet of remote things networks. IEEE Access; 2019; 7, pp. 145348-145362. [DOI: https://dx.doi.org/10.1109/ACCESS.2019.2945478]

250. Ei, N.N.; Kang, S.W.; Alsenwi, M.; Tun, Y.K.; Hong, C.S. Multi-UAV-assisted MEC system: Joint association and resource management framework. Proceedings of the 2021 International Conference on Information Networking (ICOIN); Jeju Island, Republic of Korea, 13–16 January 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 213-218.

251. Ramzan, M.R.; Naeem, M.; Altaf, M.; Ejaz, W. Multicriterion Resource Management in Energy-Harvested Cooperative UAV-Enabled IoT Networks. IEEE Internet Things J.; 2021; 9, pp. 2944-2959. [DOI: https://dx.doi.org/10.1109/JIOT.2021.3094810]

252. de Paula Parisotto, R.; Klaine, P.V.; Nadas, J.P.B.; Souza, R.D.; Brante, G.; Imran, M.A. Drone Base Station Positioning and Power Allocation using Reinforcement Learning. Proceedings of the 2019 16th International Symposium on Wireless Communication Systems (ISWCS); Oulu, Finland, 27–30 August 2019; pp. 213-217. [DOI: https://dx.doi.org/10.1109/ISWCS.2019.8877247]

253. Zhao, N.; Liu, Z.; Cheng, Y. Multi-Agent Deep Reinforcement Learning for Trajectory Design and Power Allocation in Multi-UAV Networks. IEEE Access; 2020; 8, pp. 139670-139679. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.3012756]

254. Li, L.; Cheng, Q.; Xue, K.; Yang, C.; Han, Z. Downlink Transmit Power Control in Ultra-Dense UAV Network Based on Mean Field Game and Deep Reinforcement Learning. IEEE Trans. Veh. Technol.; 2020; 69, pp. 15594-15605. [DOI: https://dx.doi.org/10.1109/TVT.2020.3043851]

255. Yang, Z.; Pan, C.; Wang, K.; Shikh-Bahaei, M. Energy Efficient Resource Allocation in UAV-Enabled Mobile Edge Computing Networks. IEEE Trans. Wirel. Commun.; 2019; 18, pp. 4576-4589. [DOI: https://dx.doi.org/10.1109/TWC.2019.2927313]

256. Kim, H.; Park, J.; Bennis, M.; Kim, S.L. Massive UAV-to-Ground Communication and its Stable Movement Control: A Mean-Field Approach. Proceedings of the 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC); Kalamata, Greece, 25–28 June 2018; pp. 1-5. [DOI: https://dx.doi.org/10.1109/SPAWC.2018.8445906]

257. Han, D.; Chen, W.; Liu, J. Energy-Efficient UAV Communications Under Stochastic Trajectory: A Markov Decision Process Approach. IEEE Trans. Green Commun. Netw.; 2021; 5, pp. 106-118. [DOI: https://dx.doi.org/10.1109/TGCN.2020.3016266]

258. Xiao, L.; Xu, Y.; Yang, D.; Zeng, Y. Secrecy Energy Efficiency Maximization for UAV-Enabled Mobile Relaying. IEEE Trans. Green Commun. Netw.; 2020; 4, pp. 180-193. [DOI: https://dx.doi.org/10.1109/TGCN.2019.2949802]

259. Yang, H.; Xie, X. Energy-Efficient Joint Scheduling and Resource Management for UAV-Enabled Multicell Networks. IEEE Syst. J.; 2020; 14, pp. 363-374. [DOI: https://dx.doi.org/10.1109/JSYST.2019.2911895]

260. Koulali, S.; Sabir, E.; Taleb, T.; Azizi, M. A green strategic activity scheduling for UAV networks: A sub-modular game perspective. IEEE Commun. Mag.; 2016; 54, pp. 58-64. [DOI: https://dx.doi.org/10.1109/MCOM.2016.7470936]

261. Mkiramweni, M.E.; Yang, C. Energy Efficiency Optimization for Wireless Unmanned Aerial Vehicle Communication Networks: A Bargaining Game Approach. Proceedings of the 2018 10th International Conference on Wireless Communications and Signal Processing (WCSP); Hangzhou, China, 18–20 October 2018; pp. 1-6. [DOI: https://dx.doi.org/10.1109/WCSP.2018.8555719]

262. Luan, H.; Xu, Y.; Liu, D.; Du, Z.; Qian, H.; Liu, X.; Tong, X. Energy Efficient Task Cooperation for Multi-UAV Networks: A Coalition Formation Game Approach. IEEE Access; 2020; 8, pp. 149372-149384. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.3016009]

263. Yuan, Y.; Lei, L.; Vu, T.X.; Chatzinotas, S.; Ottersten, B. Actor-Critic Deep Reinforcement Learning for Energy Minimization in UAV-Aided Networks. Proceedings of the 2020 European Conference on Networks and Communications (EuCNC); Dubrovnik, Croatia, 15–18 June 2020; pp. 348-352. [DOI: https://dx.doi.org/10.1109/EuCNC48522.2020.9200931]

264. Yuan, Y.; Lei, L.; Vu, T.X.; Chatzinotas, S.; Sun, S.; Ottersten, B. Energy Minimization in UAV-Aided Networks: Actor-Critic Learning for Constrained Scheduling Optimization. IEEE Trans. Veh. Technol.; 2021; 70, pp. 5028-5042. [DOI: https://dx.doi.org/10.1109/TVT.2021.3075860]

265. Gangula, R.; Gesbert, D.; Kuelzer, D.F.; Franceschi, J.M. A Landing Spot Approach for Enhancing the Performance of UAV-Aided Wireless Networks. Proceedings of the 2018 IEEE International Conference on Communications Workshops (ICC Workshops); Kansas City, MO, USA, 20–24 May 2018; pp. 1-6. [DOI: https://dx.doi.org/10.1109/ICCW.2018.8403622]

266. Muntaha, S.T.; Hassan, S.A.; Jung, H.; Hossain, M.S. Energy Efficiency and Hover Time Optimization in UAV-Based HetNets. IEEE Trans. Intell. Transp. Syst.; 2021; 22, pp. 5103-5111. [DOI: https://dx.doi.org/10.1109/TITS.2020.3015256]

267. Manzoor, A.; Kim, D.H.; Hong, C.S. Energy Efficient Resource Allocation in UAV-based Heterogeneous Networks. Proceedings of the 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS); Matsue, Japan, 18–20 September 2019; pp. 1-4. [DOI: https://dx.doi.org/10.23919/APNOMS.2019.8892933]

268. Manzoor, A.; Kim, K.; Pandey, S.R.; Kazmi, S.M.A.; Tran, N.H.; Saad, W.; Hong, C.S. Ruin Theory for Energy-Efficient Resource Allocation in UAV-Assisted Cellular Networks. IEEE Trans. Commun.; 2021; 69, pp. 3943-3956. [DOI: https://dx.doi.org/10.1109/TCOMM.2021.3064968]

269. Pliatsios, D.; Sarigiannidis, P.; Goudos, S.K.; Psannis, K. 3D Placement of Drone-Mounted Remote Radio Head for Minimum Transmission Power Under Connectivity Constraints. IEEE Access; 2020; 8, pp. 200338-200350. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.3034881]

270. Wang, L.; Zhou, S. Energy-Efficient UAV Deployment with Flexible Functional Split Selection. Proceedings of the 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC); Kalamata, Greece, 25–28 June 2018; pp. 1-5. [DOI: https://dx.doi.org/10.1109/SPAWC.2018.8445959]

271. Basar, E.; Di Renzo, M.; De Rosny, J.; Debbah, M.; Alouini, M.S.; Zhang, R. Wireless Communications Through Reconfigurable Intelligent Surfaces. IEEE Access; 2019; 7, pp. 116753-116773. [DOI: https://dx.doi.org/10.1109/ACCESS.2019.2935192]

272. Huang, C.; Zappone, A.; Alexandropoulos, G.C.; Debbah, M.; Yuen, C. Reconfigurable Intelligent Surfaces for Energy Efficiency in Wireless Communication. IEEE Trans. Wirel. Commun.; 2019; 18, pp. 4157-4170. [DOI: https://dx.doi.org/10.1109/TWC.2019.2922609]

273. Li, S.; Duo, B.; Yuan, X.; Liang, Y.C.; Di Renzo, M. Reconfigurable Intelligent Surface Assisted UAV Communication: Joint Trajectory Design and Passive Beamforming. IEEE Wirel. Commun. Lett.; 2020; 9, pp. 716-720. [DOI: https://dx.doi.org/10.1109/LWC.2020.2966705]

274. Yang, L.; Meng, F.; Zhang, J.; Hasna, M.O.; Renzo, M.D. On the Performance of RIS-Assisted Dual-Hop UAV Communication Systems. IEEE Trans. Veh. Technol.; 2020; 69, pp. 10385-10390. [DOI: https://dx.doi.org/10.1109/TVT.2020.3004598]

275. Ranjha, A.; Kaddoum, G. URLLC Facilitated by Mobile UAV Relay and RIS: A Joint Design of Passive Beamforming, Blocklength, and UAV Positioning. IEEE Internet Things J.; 2021; 8, pp. 4618-4627. [DOI: https://dx.doi.org/10.1109/JIOT.2020.3027149]

276. Liu, X.; Liu, Y.; Chen, Y. Machine Learning Empowered Trajectory and Passive Beamforming Design in UAV-RIS Wireless Networks. IEEE J. Sel. Areas Commun.; 2021; 39, pp. 2042-2055. [DOI: https://dx.doi.org/10.1109/JSAC.2020.3041401]

277. Michailidis, E.T.; Miridakis, N.I.; Michalas, A.; Skondras, E.; Vergados, D.J. Energy optimization in dual-RIS UAV-aided MEC-enabled internet of vehicles. Sensors; 2021; 21, 4392. [DOI: https://dx.doi.org/10.3390/s21134392]

278. Long, H.; Chen, M.; Yang, Z.; Li, Z.; Wang, B.; Yun, X.; Shikh-Bahaei, M. Joint Trajectory and Passive Beamforming Design for Secure UAV Networks with RIS. Proceedings of the 2020 IEEE Globecom Workshops (GC Wkshps); Taipei, Taiwan, 7–11 December 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GCWkshps50303.2020.9367542]

279. Diamanti, M.; Tsampazi, M.; Tsiropoulou, E.E.; Papavassiliou, S. Energy Efficient Multi-User Communications Aided by Reconfigurable Intelligent Surfaces and UAVs. Proceedings of the 2021 IEEE International Conference on Smart Computing (SMARTCOMP); Irvine, CA, USA, 23–27 August 2021; pp. 371-376. [DOI: https://dx.doi.org/10.1109/SMARTCOMP52413.2021.00075]

280. Agrawal, N.; Bansal, A.; Singh, K.; Li, C.P. Performance Evaluation of RIS-Assisted UAV-Enabled Vehicular Communication System With Multiple Non-Identical Interferers. IEEE Trans. Intell. Transp. Syst.; 2021; [DOI: https://dx.doi.org/10.1109/TITS.2021.3123072]

281. Motlagh, N.H.; Bagaa, M.; Taleb, T. UAV-Based IoT Platform: A Crowd Surveillance Use Case. IEEE Commun. Mag.; 2017; 55, pp. 128-134. [DOI: https://dx.doi.org/10.1109/MCOM.2017.1600587CM]

282. Wu, G.; Miao, Y.; Zhang, Y.; Barnawi, A. Energy efficient for UAV-enabled mobile edge computing networks: Intelligent task prediction and offloading. Comput. Commun.; 2020; 150, pp. 556-562. [DOI: https://dx.doi.org/10.1016/j.comcom.2019.11.037]

283. Li, L.; Wen, X.; Lu, Z.; Pan, Q.; Jing, W.; Hu, Z. Energy-efficient UAV-enabled MEC system: Bits allocation optimization and trajectory design. Sensors; 2019; 19, 4521. [DOI: https://dx.doi.org/10.3390/s19204521]

284. Zhang, X.; Zhang, J.; Xiong, J.; Zhou, L.; Wei, J. Energy-Efficient Multi-UAV-Enabled Multiaccess Edge Computing Incorporating NOMA. IEEE Internet Things J.; 2020; 7, pp. 5613-5627. [DOI: https://dx.doi.org/10.1109/JIOT.2020.2980035]

285. Bai, T.; Wang, J.; Ren, Y.; Hanzo, L. Energy-Efficient Computation Offloading for Secure UAV-Edge-Computing Systems. IEEE Trans. Veh. Technol.; 2019; 68, pp. 6074-6087. [DOI: https://dx.doi.org/10.1109/TVT.2019.2912227]

286. Chen, L.; Zhao, R.; He, K.; Zhao, Z.; Fan, L. Intelligent ubiquitous computing for future UAV-enabled MEC network systems. Clust. Comput.; 2021; 25, pp. 2417-2427. [DOI: https://dx.doi.org/10.1007/s10586-021-03434-w]

287. Du, Y.; Yang, K.; Wang, K.; Zhang, G.; Zhao, Y.; Chen, D. Joint Resources and Workflow Scheduling in UAV-Enabled Wirelessly-Powered MEC for IoT Systems. IEEE Trans. Veh. Technol.; 2019; 68, pp. 10187-10200. [DOI: https://dx.doi.org/10.1109/TVT.2019.2935877]

288. Faraci, G.; Grasso, C.; Schembra, G. Design of a 5G Network Slice Extension With MEC UAVs Managed With Reinforcement Learning. IEEE J. Sel. Areas Commun.; 2020; 38, pp. 2356-2371. [DOI: https://dx.doi.org/10.1109/JSAC.2020.3000416]

289. Zhou, F.; Wu, Y.; Hu, R.Q.; Qian, Y. Computation Rate Maximization in UAV-Enabled Wireless-Powered Mobile-Edge Computing Systems. IEEE J. Sel. Areas Commun.; 2018; 36, pp. 1927-1941. [DOI: https://dx.doi.org/10.1109/JSAC.2018.2864426]

290. Jeong, S.; Simeone, O.; Kang, J. Mobile Edge Computing via a UAV-Mounted Cloudlet: Optimization of Bit Allocation and Path Planning. IEEE Trans. Veh. Technol.; 2018; 67, pp. 2049-2063. [DOI: https://dx.doi.org/10.1109/TVT.2017.2706308]

291. Yi, B.; Wang, X.; Li, K.; Huang, M. A comprehensive survey of network function virtualization. Comput. Netw.; 2018; 133, pp. 212-262. [DOI: https://dx.doi.org/10.1016/j.comnet.2018.01.021]

292. Tipantuña, C.; Hesselbach, X.; Sánchez-Aguero, V.; Valera, F.; Vidal, I.; Nogales, B. An NFV-based energy scheduling algorithm for a 5G enabled fleet of programmable unmanned aerial vehicles. Wirel. Commun. Mob. Comput.; 2019; 2019, 4734821. [DOI: https://dx.doi.org/10.1155/2019/4734821]

293. Xu, Z.; Petrunin, I.; Tsourdos, A. Dynamic Spectrum Management with Network Function Virtualization for UAV Communication. J. Intell. Robot. Syst.; 2021; 101, 40. [DOI: https://dx.doi.org/10.1007/s10846-021-01318-0]

294. Nogales, B.; Sanchez-Aguero, V.; Vidal, I.; Valera, F. Adaptable and automated small uav deployments via virtualization. Sensors; 2018; 18, 4116. [DOI: https://dx.doi.org/10.3390/s18124116]

295. Sami Oubbati, O.; Atiquzzaman, M.; Ahamed Ahanger, T.; Ibrahim, A. Softwarization of UAV Networks: A Survey of Applications and Future Trends. IEEE Access; 2020; 8, pp. 98073-98125. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2994494]

296. Chen, Y.; Feng, W.; Zheng, G. Optimum Placement of UAV as Relays. IEEE Commun. Lett.; 2018; 22, pp. 248-251. [DOI: https://dx.doi.org/10.1109/LCOMM.2017.2776215]

297. Azari, M.M.; Rosas, F.; Chen, K.C.; Pollin, S. Ultra Reliable UAV Communication Using Altitude and Cooperation Diversity. IEEE Trans. Commun.; 2018; 66, pp. 330-344. [DOI: https://dx.doi.org/10.1109/TCOMM.2017.2746105]

298. Yin, S.; Zhao, Y.; Li, L.; Yu, F.R. UAV-Assisted Cooperative Communications With Time-Sharing Information and Power Transfer. IEEE Trans. Veh. Technol.; 2020; 69, pp. 1554-1567. [DOI: https://dx.doi.org/10.1109/TVT.2019.2956167]

299. Thammawichai, M.; Baliyarasimhuni, S.P.; Kerrigan, E.C.; Sousa, J.B. Optimizing Communication and Computation for Multi-UAV Information Gathering Applications. IEEE Trans. Aerosp. Electron. Syst.; 2018; 54, pp. 601-615. [DOI: https://dx.doi.org/10.1109/TAES.2017.2761139]

300. Laneman, J.; Tse, D.; Wornell, G. Cooperative diversity in wireless networks: Efficient protocols and outage behavior. IEEE Trans. Inf. Theory; 2004; 50, pp. 3062-3080. [DOI: https://dx.doi.org/10.1109/TIT.2004.838089]

301. Letaief, K.B.; Zhang, W. Cooperative Communications for Cognitive Radio Networks. Proc. IEEE; 2009; 97, pp. 878-893. [DOI: https://dx.doi.org/10.1109/JPROC.2009.2015716]

302. Ho, D.T.; Grøtli, E.I.; Sujit, P.; Johansen, T.A.; De Sousa, J.B. Performance evaluation of cooperative relay and Particle Swarm Optimization path planning for UAV and wireless sensor network. Proceedings of the 2013 IEEE Globecom Workshops (GC Wkshps); Atlanta, Georgia, USA, 9–13 December 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1403-1408.

303. Nguyen, K.K.; Khosravirad, S.; Da Costa, D.B.; Nguyen, L.D.; Duong, T.Q. Reconfigurable Intelligent Surface-assisted Multi-UAV Networks: Efficient Resource Allocation with Deep Reinforcement Learning. IEEE J. Sel. Top. Signal Process.; 2021; [DOI: https://dx.doi.org/10.1109/JSTSP.2021.3134162]

304. Huang, J.; Zhou, Y.; Ning, Z.; Gharavi, H. Wireless Power Transfer and Energy Harvesting: Current Status and Future Prospects. IEEE Wirel. Commun.; 2019; 26, pp. 163-169. [DOI: https://dx.doi.org/10.1109/MWC.2019.1800378]

305. Quyen, T.V.; Nguyen, C.V.; Le, A.M.; Nguyen, M.T. Optimizing hybrid energy harvesting mechanisms for UAVs. EAI Endorsed Trans. Energy Web; 2020; 7, e8. [DOI: https://dx.doi.org/10.4108/eai.13-7-2018.164629]

306. Le, W.; Xue, Z.; Chen, J.; Zhang, Z. Coverage Path Planning Based on the Optimization Strategy of Multiple Solar Powered Unmanned Aerial Vehicles. Drones; 2022; 6, 203. [DOI: https://dx.doi.org/10.3390/drones6080203]

307. Wang, H.; Li, P.; Xiao, H.; Zhou, X.; Lei, R. Intelligent energy management for solar-powered unmanned aerial vehicle using multi-objective genetic algorithm. Energy Convers. Manag.; 2023; 280, 116805. [DOI: https://dx.doi.org/10.1016/j.enconman.2023.116805]

308. Song, X.; Chang, Z.; Guo, X.; Wu, P.; Hämäläinen, T. Energy efficient optimization for solar-powered UAV communications system. Proceedings of the 2021 IEEE International Conference on Communications Workshops (ICC Workshops); Montreal, QC, Canada, 14–23 June 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 1-6.

309. Yin, S.; Zhao, Y.; Li, L.; Yu, F.R. UAV-Assisted Cooperative Communications With Power-Splitting Information and Power Transfer. IEEE Trans. Green Commun. Netw.; 2019; 3, pp. 1044-1057. [DOI: https://dx.doi.org/10.1109/TGCN.2019.2926131]

310. Liu, Q.; Li, M.; Yang, J.; Lv, J.; Hwang, K.; Hossain, M.S.; Muhammad, G. Joint power and time allocation in energy harvesting of UAV operating system. Comput. Commun.; 2020; 150, pp. 811-817. [DOI: https://dx.doi.org/10.1016/j.comcom.2019.12.009]

311. Lakew, D.S.; Na, W.; Dao, N.N.; Cho, S. Aerial Energy Orchestration for Heterogeneous UAV-Assisted Wireless Communications. IEEE Syst. J.; 2021; 16, pp. 2483-2494. [DOI: https://dx.doi.org/10.1109/JSYST.2021.3075316]

312. Yang, L.; Chen, J.; Hasna, M.O.; Yang, H.C. Outage Performance of UAV-Assisted Relaying Systems With RF Energy Harvesting. IEEE Commun. Lett.; 2018; 22, pp. 2471-2474. [DOI: https://dx.doi.org/10.1109/LCOMM.2018.2876869]

313. Shakeri, R.; Al-Garadi, M.A.; Badawy, A.; Mohamed, A.; Khattab, T.; Al-Ali, A.K.; Harras, K.A.; Guizani, M. Design Challenges of Multi-UAV Systems in Cyber-Physical Applications: A Comprehensive Survey and Future Directions. IEEE Commun. Surv. Tutor.; 2019; 21, pp. 3340-3385. [DOI: https://dx.doi.org/10.1109/COMST.2019.2924143]

314. Mehta, P.; Gupta, R.; Tanwar, S. Blockchain envisioned UAV networks: Challenges, solutions, and comparisons. Comput. Commun.; 2020; 151, pp. 518-538. [DOI: https://dx.doi.org/10.1016/j.comcom.2020.01.023]

315. Ye, J.; Zhang, C.; Lei, H.; Pan, G.; Ding, Z. Secure UAV-to-UAV Systems With Spatially Random UAVs. IEEE Wirel. Commun. Lett.; 2019; 8, pp. 564-567. [DOI: https://dx.doi.org/10.1109/LWC.2018.2879842]

316. Li, B.; Fei, Z.; Zhang, Y.; Guizani, M. Secure UAV Communication Networks over 5G. IEEE Wirel. Commun.; 2019; 26, pp. 114-120. [DOI: https://dx.doi.org/10.1109/MWC.2019.1800458]

317. Challita, U.; Ferdowsi, A.; Chen, M.; Saad, W. Machine Learning for Wireless Connectivity and Security of Cellular-Connected UAVs. IEEE Wirel. Commun.; 2019; 26, pp. 28-35. [DOI: https://dx.doi.org/10.1109/MWC.2018.1800155]

318. Sedjelmaci, H.; Senouci, S.M.; Messous, M.A. How to Detect Cyber-Attacks in Unmanned Aerial Vehicles Network?. Proceedings of the 2016 IEEE Global Communications Conference (GLOBECOM); Washington, DC, USA, 4–8 December 2016; pp. 1-6. [DOI: https://dx.doi.org/10.1109/GLOCOM.2016.7841878]

319. Iqbal, S. A Study on UAV Operating System Security and Future Research Challenges. Proceedings of the 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC); Virtual, 27–30 January 2021; pp. 0759-0765. [DOI: https://dx.doi.org/10.1109/CCWC51732.2021.9376151]

320. Fotohi, R.; Nazemi, E.; Aliee, F.S. An agent-based self-protective method to secure communication between UAVs in unmanned aerial vehicle networks. Veh. Commun.; 2020; 26, 100267. [DOI: https://dx.doi.org/10.1016/j.vehcom.2020.100267]

321. Hu, Y.C.; Perrig, A.; Johnson, D. Wormhole attacks in wireless networks. IEEE J. Sel. Areas Commun.; 2006; 24, pp. 370-380. [DOI: https://dx.doi.org/10.1109/JSAC.2005.861394]

322. Ren, J.; Zhang, Y.; Zhang, K.; Shen, X. Adaptive and Channel-Aware Detection of Selective Forwarding Attacks in Wireless Sensor Networks. IEEE Trans. Wirel. Commun.; 2016; 15, pp. 3718-3731. [DOI: https://dx.doi.org/10.1109/TWC.2016.2526601]

323. Liu, Y.; Ma, M.; Liu, X.; Xiong, N.N.; Liu, A.; Zhu, Y. Design and Analysis of Probing Route to Defense Sink-Hole Attacks for Internet of Things Security. IEEE Trans. Netw. Sci. Eng.; 2020; 7, pp. 356-372. [DOI: https://dx.doi.org/10.1109/TNSE.2018.2881152]

324. Sun, X.; Ng, D.W.K.; Ding, Z.; Xu, Y.; Zhong, Z. Physical Layer Security in UAV Systems: Challenges and Opportunities. IEEE Wirel. Commun.; 2019; 26, pp. 40-47. [DOI: https://dx.doi.org/10.1109/MWC.001.1900028]

325. Skorobogatov, G.; Barrado, C.; Salamí, E. Multiple UAV systems: A survey. Unmanned Syst.; 2020; 8, pp. 149-169. [DOI: https://dx.doi.org/10.1142/S2301385020500090]

326. Tortonesi, M.; Stefanelli, C.; Benvegnu, E.; Ford, K.; Suri, N.; Linderman, M. Multiple-UAV coordination and communications in tactical edge networks. IEEE Commun. Mag.; 2012; 50, pp. 48-55. [DOI: https://dx.doi.org/10.1109/MCOM.2012.6316775]

327. Chung, S.J.; Paranjape, A.A.; Dames, P.; Shen, S.; Kumar, V. A Survey on Aerial Swarm Robotics. IEEE Trans. Robot.; 2018; 34, pp. 837-855. [DOI: https://dx.doi.org/10.1109/TRO.2018.2857475]

328. Shen, C.; Chang, T.H.; Gong, J.; Zeng, Y.; Zhang, R. Multi-UAV Interference Coordination via Joint Trajectory and Power Control. IEEE Trans. Signal Process.; 2020; 68, pp. 843-858. [DOI: https://dx.doi.org/10.1109/TSP.2020.2967146]

329. Chung, T.H.; Clement, M.R.; Day, M.A.; Jones, K.D.; Davis, D.; Jones, M. Live-fly, large-scale field experimentation for large numbers of fixed-wing UAVs. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA); Stockholm, Sweden, 16–21 May 2016; pp. 1255-1262. [DOI: https://dx.doi.org/10.1109/ICRA.2016.7487257]

330. Chen, J.; Wu, Q.; Xu, Y.; Zhang, Y.; Yang, Y. Distributed Demand-Aware Channel-Slot Selection for Multi-UAV Networks: A Game-Theoretic Learning Approach. IEEE Access; 2018; 6, pp. 14799-14811. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2811372]

331. Abdelkader, M.; Güler, S.; Jaleel, H.; Shamma, J.S. Aerial swarms: Recent applications and challenges. Curr. Robot. Rep.; 2021; 2, pp. 309-320. [DOI: https://dx.doi.org/10.1007/s43154-021-00063-4]

332. Aggarwal, S.; Kumar, N. Path planning techniques for unmanned aerial vehicles: A review, solutions, and challenges. Comput. Commun.; 2020; 149, pp. 270-299. [DOI: https://dx.doi.org/10.1016/j.comcom.2019.10.014]

333. Zhou, Y.; Cheng, N.; Lu, N.; Shen, X.S. Multi-UAV-Aided Networks: Aerial-Ground Cooperative Vehicular Networking Architecture. IEEE Veh. Technol. Mag.; 2015; 10, pp. 36-44. [DOI: https://dx.doi.org/10.1109/MVT.2015.2481560]

334. Zear, A.; Ranga, V. Path Planning of Unmanned Aerial Vehicles: Current State and Future Challenges. First International Conference on Sustainable Technologies for Computational Intelligence; Springer: Berlin/Heidelberg, Germany, 2020; pp. 409-419.

335. Almulhem, A. Threat modeling of a multi-UAV system. Transp. Res. Part A: Policy Pract.; 2020; 142, pp. 290-295. [DOI: https://dx.doi.org/10.1016/j.tra.2020.11.004]

336. Li, H.; Fang, S.; Mukhopadhyay, S.; Saykin, A.J.; Shen, L. Interactive Machine Learning by Visualization: A Small Data Solution. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data); Seattle, WA, USA, 10–13 December 2018; pp. 3513-3521. [DOI: https://dx.doi.org/10.1109/BigData.2018.8621952]

337. Park, B.; Rao, D.L.; Gudivada, V.N. Dangers of Bias in Data-Intensive Information Systems. Next Generation Information Processing System; Springer: Berlin/Heidelberg, Germany, 2021; pp. 259-271.

338. Jordan, M.I.; Mitchell, T.M. Machine learning: Trends, perspectives, and prospects. Science; 2015; 349, pp. 255-260. [DOI: https://dx.doi.org/10.1126/science.aaa8415] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26185243]

339. Sun, Y.; Peng, M.; Zhou, Y.; Huang, Y.; Mao, S. Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues. IEEE Commun. Surv. Tutor.; 2019; 21, pp. 3072-3108. [DOI: https://dx.doi.org/10.1109/COMST.2019.2924243]

340. Deng, L. The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]. IEEE Signal Process. Mag.; 2012; 29, pp. 141-142. [DOI: https://dx.doi.org/10.1109/MSP.2012.2211477]

341. Bhola,; Lai, C.C.; Wang, L.C. The Outage-Free Replacement Problem in Unmanned Aerial Vehicle Base Stations. IEEE Trans. Veh. Technol.; 2021; 70, pp. 13390-13395. [DOI: https://dx.doi.org/10.1109/TVT.2021.3116701]

342. Menouar, H.; Guvenc, I.; Akkaya, K.; Uluagac, A.S.; Kadri, A.; Tuncer, A. UAV-Enabled Intelligent Transportation Systems for the Smart City: Applications and Challenges. IEEE Commun. Mag.; 2017; 55, pp. 22-28. [DOI: https://dx.doi.org/10.1109/MCOM.2017.1600238CM]

343. Nguyen, M.T.; Nguyen, C.V.; Truong, L.H.; Le, A.M.; Quyen, T.V.; Masaracchia, A.; Teague, K.A. Electromagnetic field based wpt technologies for uavs: A comprehensive survey. Electronics; 2020; 9, 461. [DOI: https://dx.doi.org/10.3390/electronics9030461]

344. Saha, B.; Koshimoto, E.; Quach, C.C.; Hogge, E.F.; Strom, T.H.; Hill, B.L.; Vazquez, S.L.; Goebel, K. Battery health management system for electric UAVs. Proceedings of the 2011 Aerospace Conference; Big Sky, MT, USA, 5–12 March 2011; pp. 1-9. [DOI: https://dx.doi.org/10.1109/AERO.2011.5747587]

345. Jaafar, W.; Yanikomeroglu, H. Dynamics of Laser-Charged UAVs: A Battery Perspective. IEEE Internet Things J.; 2021; 8, pp. 10573-10582. [DOI: https://dx.doi.org/10.1109/JIOT.2020.3048087]

346. Jin, K.; Zhou, W. Wireless Laser Power Transmission: A Review of Recent Progress. IEEE Trans. Power Electron.; 2019; 34, pp. 3842-3859. [DOI: https://dx.doi.org/10.1109/TPEL.2018.2853156]

347. Lu, M.; James, A.; Bagheri, M. Unmanned Aerial Vehicle (UAV) charging from powerlines. Proceedings of the 2017 IEEE PES Asia-Pacific Power and Energy Engineering Conference (APPEEC); Chiang Mai, Thailand, 6–9 December 2017; pp. 1-6. [DOI: https://dx.doi.org/10.1109/APPEEC.2017.8308912]

348. Saif, A.; Dimyati, K.; Noordin, K.A.; Shah, N.S.M.; Alsamhi, S.H.; Abdullah, Q. Energy-Efficient Tethered UAV Deployment in B5G for Smart Environments and Disaster Recovery. Proceedings of the 2021 1st International Conference on Emerging Smart Technologies and Applications (eSmarTA); Ibb, Yemen, 25–26 October 2021; pp. 1-5. [DOI: https://dx.doi.org/10.1109/eSmarTA52612.2021.9515754]

349. Chen, B.; Yang, M.; Chen, Y.; Ning, Q.; Xie, D. A novel energy harvesting scheme in interference networks with UAVs. Phys. Commun.; 2019; 33, pp. 259-265. [DOI: https://dx.doi.org/10.1016/j.phycom.2019.01.011]

350. Cansiz, M.; Altinel, D.; Kurt, G.K. Efficiency in RF energy harvesting systems: A comprehensive review. Energy; 2019; 174, pp. 292-309. [DOI: https://dx.doi.org/10.1016/j.energy.2019.02.100]

351. Choi, K.W.; Kim, D.I.; Chung, M.Y. Received Power-Based Channel Estimation for Energy Beamforming in Multiple-Antenna RF Energy Transfer System. IEEE Trans. Signal Process.; 2017; 65, pp. 1461-1476. [DOI: https://dx.doi.org/10.1109/TSP.2016.2637320]

352. Stöcker, C.; Bennett, R.; Nex, F.; Gerke, M.; Zevenbergen, J. Review of the current state of UAV regulations. Remote Sens.; 2017; 9, 459. [DOI: https://dx.doi.org/10.3390/rs9050459]

353. Liu, J.; Shi, Y.; Fadlullah, Z.M.; Kato, N. Space-Air-Ground Integrated Network: A Survey. IEEE Commun. Surv. Tutor.; 2018; 20, pp. 2714-2741. [DOI: https://dx.doi.org/10.1109/COMST.2018.2841996]

354. Yu, J.; Liu, X.; Gao, Y.; Shen, X. 3D Channel Tracking for UAV-Satellite Communications in Space-Air-Ground Integrated Networks. IEEE J. Sel. Areas Commun.; 2020; 38, pp. 2810-2823. [DOI: https://dx.doi.org/10.1109/JSAC.2020.3005490]

355. Arani, A.H.; Hu, P.; Zhu, Y. Re-envisioning Space-Air-Ground Integrated Networks: Reinforcement Learning for Link Optimization. Proceedings of the ICC 2021—IEEE International Conference on Communications; Virtual, 14–23 June 2021; pp. 1-7. [DOI: https://dx.doi.org/10.1109/ICC42927.2021.9500978]