1. Introduction
Traditionally, the testing of UEs (User Equipments) has been done with expensive equipment or by testing houses. This testing was focused on conformance testing but application performance was not covered. In addition, the profiles of the companies requesting these kinds of testing services were big companies that can afford the high prices of the certification procedures or equipment. These kinds of testing solutions were also out of the scope of research institutions looking for testing new prototypes, algorithms, or services. Nowadays, the profile of the telecommunication stakeholders has evolved towards smaller companies and startups that are ramping up, especially those that are oriented to the IoT sector. In this context, the cost of the state-of-the-art test equipment or the testing houses were not affordable. Test requirements are evolving to incrementally cover additional application level and quality of experience (QoE) measurements. TRIANGLE testbed is an attempt to bring the conformance like tests to SME and researchers, extending the testing to the application layer and quality of experience (QoE) metrics [1]. In [2], a very detailed set of NB-IoT performance measurements are provided and the emulated setup introduced in the paper is very close to the one used in the TRIANGLE testbed [3]. However, the setup introduced is not oriented to provide experimentation features to third parties, as opposed to the multiple specific features added in TRIANGLE to share the research infrastructure with the community. In [4], the authors also perform a detailed study of energy performance of LoRaWAN, different network setups are used, and the same instrument is used for measuring power consumption, but also, in this case, the test equipment is not oriented to offer testing features two third parties. Similarly, in [5], the authors perform power measurements, RTT, and the throughput test under different network conditions; in this paper, the tests are done in two commercial NB-IoT networks. The nature of tests conducted in commercial networks makes them less repeatable and thus not as suitable for benchmarking of different algorithms or implementations.
Moreover, all of this work referenced can be carried out in the TRIANGLE testbed. Thus, a key objective of this paper is to bring awareness to additional talented researchers without such equipment but interested in conducting experiments and benchmarking using this infrastructure. TRIANGLE offers a high level access with predefined scenarios via a web Portal introduced in [6] but also a low level access based, where all the parameters are configurable plugins as well as the control of mobile devices and applications running at the UEs. For the sake of better understanding of the potential of the testbed, we also illustrate measurement results on specific configurations; however, it is not the intention of the paper to focus on the specific results of the experiments but on the capabilities exposed to the experimenters in the IoT area with limited access to dedicated equipment.
The TRIANGLE testbed is focused on the testing of third party solutions looking for understand the behavior of the devices under different network conditions and validate the performance obtained at the application layer. The TRIANGLE testbed was initially focused on cellular wide-band technologies. However it has evolved to also include the testing of IoT applications, devices, and services as they are gaining more and more weight in the current telecommunication environment. Moreover, the high heterogeneity of IoT devices and solutions also increase the complexity of their testing, which makes an automated benchmarking reference more needed than in the past. This paper provides an overview of the features provided for the testing of NB-IoT solutions.
The main contributions of the TRIANGLE testbed in the context of NB-IoT testing are the possibility of designing realistic NB-IoT scenarios and the full automation of the testing of NB-IoT solutions. TRIANGLE NB-IoT testing features are introduced in these papers, as well as the key functionalities being provided by NB-IoT technology, which will enable better understanding the type of solutions that can take advantage of this technology.
The paper is organized as follows: Section 2 introduces the background on NB-IoT technology. Section 3 provides technical details of NB-IoT standard, which will enable to understand the particularities of the testing of NB-IoT solutions. Section 4 provides an overview of the evolution of NB-IoT focused on key features of this technology, which reveals, again, the complexity of the use cases covered by this technology and the need for exhaustive testing. The results obtained during the execution of a testing campaign are shown in Section 5 as well as the design of testing scenarios, which illustrate the advanced parameters that can be configured in the testbed. Finally, Section 6 summarizes key features provided by NB-IoT technology and the NB-IoT testing features provided by the TRIANGLE testbed.
2. NB-IoT Background
In the last few years, the evolution of 3GPP standards related to IoT has accelerated due to the pressure of cellular stakeholders who require a standard solution to go one step further to provide a real cellular and standard solution for LPWA (Low Power Wide Area) networks. Behind this pressure are the billions of potential new subscribers originated by the IoT use cases.
IoT covers a wide range of use cases, which can only be partially covered by other previous 3GPP technologies such as GSM or Machine Type Communication (MTC) introduced in Release 8, and new ones such as eMTC or EC-GSM and unlicensed LPWA such as Lora or Sigfox. In contrast, NB-IoT is a 3GPP effort to penetrate the ultra-low cost, power and throughput, extended coverage, and delay tolerant IoT marketplace, which, in 5G terminology, is the massive Machine Type Communication (mMTC) use case. This use case contains the greatest number of potential IoT subscribers, on the order of several billion.
5G NR (New Radio), the term coined for the global 5G standard new radio access technology, covers, in the first version, eMBB (enhanced mobile broadband) and URLLC (ultra-reliable, low-latency communication) use cases. mMTC will not be considered until Release 17. In the meantime, NB-IoT is providing the solutions to cover the 5G mMTC use case. This converts this technology into the foundation for NB-5G.
The first version of NB-IoT was released in July 2016, as part of Release 13 in TR 45.820 [7]. This version of the NB-IoT standard focused on improving indoor coverage, support for massive numbers of low throughput device sensitivity, ultra-low device cost, low device power consumption, and optimized network architecture. This translates to the comparative numbers in Figure 1.
The fundamentals of this technology have been analyzed in previous works, as well as its comparison with other LPWA technologies [8,9]. In this paper, we will update these analyses. As we will discuss, Release 14 and Release 15 introduce new features to cover new use cases. With the features provided in Release 13 (reduced complexity, lower power, deeper coverage and higher density), the use cases covered by NB-IoT are mainly related to sensors, but new features such as positioning and mobility enhancements (NB-IoT Release 13 only supported cell-reselection in the idle state) have extended its applicability to wearables and tracking services.
NB-IoT should not be confused with the new item introduced in Release 16, “New Radio Industrial Internet of Things” (NR-IIoT) [10], which is focused on factory automation use cases. The key technology enablers of NR-IIoT are the NR URLLC (short TTI, reliability) and the Time Sensitive Networking (TSN) (accurate reference time, QoS for wireless Ethernet, Ethernet compression, etc). 3GPP Release 17 also introduces a new device with reduced capabilities, called NR-Lite, to cover IoT use cases with higher requirements that can be not be provided by Nb-IoT and LTE-M, such as higher data rate, higher reliability, and lower latency.
3. Close-Up View of the NB-IoT Standard: NB-IoT Internals
NB-IoT has been integrated into LTE standards; however, it introduces a new radio interface for LTE, which is compatible with LTE deployments. This means that, although there is not backwards compatibility in all scenarios, it can coexist with current cellular deployments. In order to enable this coexistence, NB-IoT supports three different operation modes: in-band, guard-band, and standalone modes. In the in-band mode, NB-IoT can use one physical resource block within a normal LTE carrier; for guard-band mode, NB-IoT deployment can use the unused resources block within a LTE carrier’s guard-band and, in the standalone mode, a dedicated carrier is required from 2G/3G refarming.
As shown, the in-band and guard-band operation modes are compatible with standard LTE deployments; however, the new RF channel bandwidth of 200 kHz (180 kHz plus guard-band, this is why these modes use only one LTE physical resource block of 180 kHz) requires new control and data channels. Relevant modifications in the physical layer, other than those already mentioned to work with narrowband channels, are the half duplex operation in FDD (Frequency Divising Duplexing), to reduce the maximum modulation scheme supported to QPSK and the higher number of repetitions used to enhance the coverage, up to 1048 repetitions in downlink channels and 128 repetitions in uplink channels. MAC (Medium Access Control), RLC (Radio Link Control), PDCP (Packet Data Convergence Protocol), and RRC (Radio Resource Control) procedures are based on existing LTE procedures and protocols in addition to some optimization to support channelization and the selection of the physical layer.
Apart from the changes made in the radio interface, signaling protocols and architecture have also been optimized. In general, these modifications seek to simplify the signaling procedures to reduce the overhead introduced by them, and therefore provide simplified versions of S1, S1-lite, and NAS, SNAS (Simplified NAS), and a new element called C-SGN (Cellular IoT Serving Gateway Node), which provides combined C-plane and U-plane functions, e.g., aggregating some of the functions that traditionally reside in MME (Mobility Management Entity) and SGW (Serving Gateway). P-GW (Packet Gateway) is only used to support the roaming case. For the non-roaming case, SGi interface terminates on C-SGN, and C-SGN can send/receive data on SGi directly. Figure 2 shows the NB-IoT architecture proposed in 3GPP TR 23.720 [11].
The other key issue which has determined NB-IoT architecture is the efficient support of infrequent, small data transmission. To address this requirement, the 3GPP in 3GPP TR 23.720 [11] has proposed a control plane optimization called DoNAS (Data over NAS), as the mandatory solution for the transport of IP data, non-IP data, and SMS. In this solution, NAS messages are used to carry user data which prevents having to set up DRB (Dedicated Radio Bearer), S1-U bearers and AS (Application Server) security, and reduces the signaling procedures in the core network and also in the radio access.
As an optional implementation, a user plane solution has also been proposed, based on an improvement of RRC, introducing two new procedures, “RRC Suspend” and “RRC Resume” and the storage of AS (Access Stratum) information in the UE. All of these enable the signaling overhead to be reduced when the UE (User Equipment) transits from RRC Idle to RRC connected mode and also returns to RRC Idle mode faster. Initially, this second solution will only be applicable to IP data transport and SMS; however, it can be combined with other solutions proposed in 3GPP TR 23.720 [11] to support non-IP data.
4. NB-IoT Evolution
In Release 13, NB-IoT was specified as a new radio access which addressed the main requirements for covering the low-end segment of the use cases considered in IoT and, as shown in Figure 3, it has been evolving and including new features to support additional uses.
Release 14 (March 2017) added support for more LTE features to increase functionality and the number of use cases covered by NB-IoT. Features in Release 14 include support for the ECID (Enhanced Cell Identifier) and OTDOA (Observed Time Difference of Arrival) positioning method based on NPRS (Narrowband Positioning Reference Signal), mobility and service continuity enhancements, new power class with a reduced output power of 14 dBm, Single-cell Point-to-Multipoint (SC-PTM) for broadcast/multicast services based on MBMS architecture, peak throughput improvement in the downlink (144 kbps) and in the uplink (142.5 kpbs), and a new device category Cat-NB2 with higher values of TBS (Transport Block Size) in the uplink and in the downlink.
Release 15 introduced TDD (Time Division Duplexing) support, higher spectral efficiency, early data transmission, wake-up mechanism and on demand network access for UE power saving, NPRACH (NB-IoT physical random access channel) reliability and range enhancement, small cell support and LTE Device to Device, and UE to Network Relays for IoT and wearables.
Some of the latest additions to Release 15 (December 2018) were the coexistence of NB-IoT with 5G NR and the coexistence of NB-IoT with eMTC. As many NB-IoT devices are expected to live more than ten years after being deployed, they are expected to be compatible with future 3GPP releases. In addition, as suggested by multiple 3GPP contributions [12], both NB-IoT and eMTC are also expected to coexist and converge in the future towards mMTC, but that does not mean that every new feature will be added to both eMTC and NB-IoT. Instead, quite likely, a decision should be taken case by case depending on the needs of the specific use cases.
From this listing, arguably the two most important features, from the point of view of the potential to cover new use cases and increase the number of NB-IoT devices, are OTDOA (Observed Time Difference Of Arrival) positioning (Release 14) and Device-to-Device (D2D) communication (Release 15). These new features are introduced in detail next.
The introduced improvements in Release 16 (June 2020) are related on enhancements of previous features: improved DL/UL transmission efficiency and UE power consumption, scheduling enhancement, network management enhancement, improved multi-carrier operation, mobility enhancements, and the study of the coexistence with NR.
Finally, the planned features for Release 17 (standard freeze scheduled for late 2021) are to increase the peak data rate introducing the support of a higher modulation (16 QAM) in downlink and also in uplink, reduce RRC reestablishment time to another cell defining specific signaling for neighbor cell measurements and the corresponding measurement triggering before radio link failure, and support NB-IoT carrier selection depending on the coverage level and specific carrier configurations.
4.1. UE Positioning Accuracy Enhancements for LTE (Release 14)
There are many use cases where positioning brings significant value into the IoT space. In addition to tracking sensors and meters, locating wearables, vehicles, and goods is also of interest. In 3GPP Release 14, there has been a significant effort to add positioning for IoT. Although NB-IoT is assumed to cover the lowest complexity use cases, this feature opens new possibilities.
NB-IoT positioning is based on the same concept used for LTE positioning named Observed Time Difference Of Arrival (OTDOA). Specific radio signals named Narrowband Positioning Reference Signals (NPRS) are transmitted by multiple cells in configured periodic time opportunities. As shown in Figure 4, the device to be tracked measures and reports the relative timing of the signal received from each monitored cell.
The LPP (LTE Positioning Protocol) core network protocol [13] is used for the exchange of information between the IoT devices and the E-SMLC (Enhanced Serving Mobile Location Centre), which is a dedicated core network entity for positioning functions. The network can also make use of power (RSRP) and quality (RSRQ) measurement reports from the devices, and take benefit from the timing advance measurements when the devices are active as additional sources of information.
In more technical detail, the new Narrowband Positioning Reference Signals (NPRS) are defined in [14]. As also shown in Figure 5, different resource elements are used with incremental density depending on whether the signals are transmitted in-band 2 port, in-band 4 port, or in standalone and guard-band modes. Depending on the cell identity, up to six different pattern shifts are possible by applying offsets in frequency. It is important to note that NPRS patterns can be flexibly configured, and different neighbor cells could transmit simultaneously or interleaving/selectively muting patterns in time for reduced interference at the expense of larger acquisition periods. More information about positioning can be found in [15].
4.2. D2D Communication Support in NB-IoT (Release 15)
Release 15 studies the viability of using NB-IoT devices as remote UEs in D2D communications. Specifically, D2D communications can be used to establish short-range communications between devices which will significantly reduce energy consumption and increase the coverage in the cell edge. IoT devices with limited bandwidth represent a large category of devices which has as a main restriction the power consumption. Usually, these devices are in the proximity of more power capable devices as smartphones. A way of reducing power consumption would be to relay the transmissions through the smartphone. Thus, the smartphone would offer Proximity Services (ProSe) [16], which will enable wearables, and IoT devices in general, in order to gain global LTE connectivity using the smartphone as a relay. The relay nodes support different working modes: UL relay (DL is established directly to the network) or DL/UL relay; in both cases, the consumption is improved. It is clear that the communication through other devices with less battery constraints, such as a smartphone, can reduce power consumption significantly due to the more reduced range and also increase the coverage.
ProSe (Proximity Services) [16] is the technology introduced in Release 12 to enable direct Device-to-Device (D2D) communication. In R12, the ProSe standard was focused on public safety and critical applications to provide communication support when cellular networks are not available. In that first release, D2D communications were only supported when the UE was out-of the network coverage. R13 also includes in/partial network coverage D2D communications and ProSe UE-Network relay functionality. The relay UE uses the direct sidelink interface (PC5) to communicate with remote UEs while connecting to the network using the standard LTE Uu interface.
Moreover, the discussions within the 3GPP are going a step further. 3GPP Technical Specification Radio Access Network Working Group 1 (TSG RAN WG1) is also studying the possibility of using NB-IoT devices as relays [17,18,19,20]. The use of such bandwith limited UE-to-Network relay brings a significant improvement in coverage (20 dB) in challenging scenarios such as underground or cell edge deployments. In this context, the enhancements introduced by the use of NB-IoT devices as relays are twofold: (1) to meet the extreme coverage enhancements levels; and (2) to reduce power consumption of remote UE in enhanced coverage.
However, to support D2D communication in NB-IoT devices, with only 1 PRB (Physical Resource Block) available, different issues have to be tackled: current synchronization mechanism requires at least six PRBs, discovery message uses two PRBs, subcarrier spacing and number of tones (NB-IoT support different subcarrier spacings, some of then not supported in standard LTE), resource pool structure, which is the set of subframes and resources blocks assigned to the D2D communication, and synchronization accuracy (there are also separated efforts to improve the synchronization of NB-IoT devices). To deal with these issues, new formats are required for D2D channels, new discovery design options have been identified in [21], and several D2D communication enhancements for UEs with six PRBs can be applied to UEs with one PRB bandwidth limitations as proposed in the 3GPP TSG RAN WG1 meeting held in August 2017.
Merging these two enabling technologies, NB-IoT and D2D communication, will allow for reducing the consumption of wearables and further extending the coverage in extreme IoT deployment scenarios and will increase the value of Proximity Services (ProSe).
4.3. TCP Performance in High Latency Scenarios
One of the conclusions of the study carried out in TR 45.820 [7] about Cellular system support for ultra-low complexity and low throughput Internet of Things is that the current E-UTRAN/EPC architecture based on the S1 interface presents an inefficient performance when transporting infrequent messages. In addition, one of the architectural enhancements identified by the study carried out in 3GPP TR 23.720 [11] to support ultra-low complexity, power constrained and low data rate, is the efficient support of infrequent small data transmissions. As a result, the Non-IP Data Deliver (NIDD), a term coined to refer to this modification in the standard, has been included in the general speciation of the LTE radio interface 3GPP TS 23.401 [22]. Concretely, two different mechanisms are supported:
Delivery using SCEF (Service Capability Exposure Function). SCEF was introduced in Release 13 as part of AESE specification for the provision of service capabilities to 3rd parties. In this proposal, the SCEF is connected directly to the MME and provide the delivery of NB-IoT services. The service provided by SCEF can be based on IP or non-IP protocol.
Delivery using a Point-to-Point (PtP) SGi tunnel.
The mechanisms proposed don’t guarantee the delivery in the order of the data packages, a higher protocol layer should ensure if it is required. However, the security, a key factor in IoT because the entire amount of private and sensible data that is transported is improved, is also due to control plane integrity mechanisms that are also applied to the data.
The third alternative for the transport of the Non-IP data is the SMS service. SMS service is supported in the NB-IoT specification, and can be used simultaneously with Non-IP data and IP-data transport.
Data can also be delivered on a user plane like in LTE. In this case, data are delivered in order, but there is a higher overhead due to the signaling.
4.4. NB-IoT in Release 16
As commented before, NB-IoT specifications for Release 16 [23] are focused on network operation and efficiency improvements:
Mobile-terminated (MT) early data transmission (EDT) for user data.
UE-group wake-up signals (WUS). The aim of UE grouping for WUS is reducing the false alarm probability.
Transmission in preconfigured resources
Scheduling enhancement: scheduling multiple DL/UL transport blocks with or without DCI (Downlink Control Information) for SC-PTM (Single-Cell Point-To- Multipoint) and unicast to support separate/share SC-TMCH (single cell multicast traffic channel) transmission.
Network management tool enhancement: Including SON (Self-Organizing Networks) support for ANR (Automatic Neighbour Relation), random access performance, and RLF (Radio Link Failure) report.
Inter-RAT cell selection: NB-IoT network may indicate frequency identifiers of neighboring eMTC/LTE/GERAN carriers to assist inter-RAT selection. eMTC/LTE network may indicate NB-IoT assistance information for inter-RAT cell selection.
5. End-to-End NB-IoT Testing
This section describes the new IoT scenarios specified in the TRIANGLE testbed for testing of NB-IoT solutions and demonstrates its potential explaining initial results. The purpose of this section is to demonstrate the potential offered by the TRIANGLE testbed to reproduce realistic NB-IoT network configurations, execute tests, collect measurements, and analyze the performance of the solution under testing.
5.1. Design of NB-IoT Network Scenarios for Testing
As commented previously, repetitions are used to improve the coverage, providing an increase of 20 dB compared to GPRS. Specifically, the testing environment introduced in this paper enables the configuration of repetitions for NPDCCH (Physical Downlink Control Channel), NPDSCH (Narrowband Physical Downlink Shared Channel), and NPUSCH (Narrowband Physical Uplink Shared Channel).
NPRACH is the channel used for uplink preamble transmission for the RACH (Random Access Channel) procedure. A repetition scheme is applied in this channel to increase the reliability of the procedure. The number of repetitions of the preamble is also configurable, in particular, for the robust scenario, four repetitions have been configured.
NPRACH periodicity is also a key parameter in NB-IoT, and it specified the amount of time between consecutive random access transmission attempts. For FDD, NPRACH periodicity can be configured to 40 ms, 80 ms, 160 ms, 240 ms, 320 ms, 640 ms, 1280 ms, and 2560 ms. For the scenarios defined in this paper, 40 ms has been applied.
The NPDCCH (Narrowband Physical Downlink Control Channel) conveys control information on where and how the data transmissions will happen between the eNB and the UE. The number of repetitions is increased as needed to protect this control information, as the UE would miss the actual data if it does not receive the control information in advance.
NPDSCH and NPUSCH repetitions increase significantly the overall signal energy to improve the probability correct reception in for robust (×2) and extreme (×16) coverage for data transmissions, and are significantly more redundant for initial messages to securely establish the connections. The number of repetitions for HARQ ACK/NACK is also modified to preserve the integrity of the feedback information.
In addition to the repetitions, additional protection is added to the transmissions in robust and extreme scenarios by reducing the Modulation and Coding Scheme (MCS) from 10 to 1 and 0, respectively.
Finally, the power transmission of the base station has been configured to 23 dBm.
5.2. TRIANGLE NB-IoT Capabilities
As explained, NB-IoT is a technology that seeks to adapt mobile networks to the connectivity requirements of low cost devices such as sensors. The key point is that NB-IoT provides direct connectivity, without intermediaries, which opens a new and wide range of scenarios and opportunities for the services and devices that will appear in the future and have not been yet imagined today.
This technological revolution will not only bring great advantages, but it will present important challenges, at very different levels, to the all actors involved in the Internet of Things. From the point of view of new device manufacturers or of the operators that have to provide connectivity of these new NB-IoT, there is a need to verify the radio frequency performance, check baseband processing, ensure correct interoperability of the NB-IoT protocols, verify the stability of the devices, and measure power consumption and longevity of the batteries. For researchers looking to provide a better understanding of the applicability of NB-IoT and propose future improvements, it is important to understand the dependencies between the different parameters and be able to adjust the models characterizing the impact of the network configuration on the services that make use of NB-IoT networks.
To that end, NB-IoT technology is one of the technologies integrated into the TRIANGLE testbed [3]. TRIANGLE testbed is an end-to-end testing platform for the benchmarking of applications and devices which provides communication equipments for emulating multiple mobile technologies and realistic radio conditions, mechanisms for conducting repeatable experiments, measurement probes, and instruments for measuring energy consumption. As we will show later in this section, the actual consumption of NB-IoT devices will vary greatly depending on the network configuration.
Figure 6 shows how a NB-IoT device is connected via radiated or conducted RF links to a multi-technology base station emulator and a power analyzer to measure power consumption during end to end data connections. The NB-IoT network emulator enables IP data exchange between an external data server and a NB-IoT device through either radiated or conducted RF connections using real mobile logical and physical protocols. The figure illustrates both the logical control from the test framework as well as the protocol stacks involved in the data path. A full description of the testbed and its usage is provided in [1,3]. In this paper, we focus on NB-IoT testing and the extension of the TRIANGLE testbed to support it. The eNodeB emulator is equipment from Keysight Technologies which has been upgraded to support NB-IoT and eMTC CAT-M radio technologies. This equipment is able to reproduce different network configurations including configurations with multiple NB-IoT cells, and radio propagation impairments such fading and interferences. Figure 7 shows how the transmitted signal from each cell can pass through a channel emulation module and then can be additionally impaired with White Gaussian Noise signals. There is accurate control on the power and timing of the signals received by the device being tested, which makes this ideal for repeatable testing. The Antenna Configuration (MIMO scheme) and port mapping can be also defined to adapt to the device connectivity scheme. Signaling messages (see Figure 8) are used to establish connections between the device and the network, configure low level protocol parameters, and receive the power measurements reported by the device. Additionally, it is also possible to perform numerous RF and baseband measurements with the network emulator. Figure 9 shows an example of modulation analysis of the NPUSCH (NB-IoT Narrowband Physical Uplink Shared Channel) signal transmitted by an NB-IoT device. In addition to the IQ constellation, it is possible to observe the power spectrum and check multiple quality indicators, as well as to verify the transmitted power in the channel.
In the TRIANGLE testbed, the Test Automation Platform (TAP), from Keysight, acts as the coordination entity responsible for configuring and running the tests. TAP is based on plugins and test plans. Each testbed component is controlled through a TAP plugin which serves as a bridge between the TAP engine and the actual component interface. The plugins expose a set of commands to configure and control the components of the testbed. The test plans are based on the usage of the default test steps provided by TAP to control the test execution flow and the specific commands exposed by the plugins of the components. Three new plugins have been developed to support NB-IoT testing. A SSH (Secure Shell) plugin has been implemented to control NB-IoT devices. This plugin enables the remote control of NB-IoT solutions, which provides a remote access via SSH network protocol. One of the key components of the TRIANGLE testbed is the set tools used to stimulate the system under test. The testing of NB-IoT solutions covers a wide range of use cases based which can be based on private protocols with security restrictions, so its testing has been based on the reproduction of the traffic generated by the solutions under test. Common tools such as iPerf or Ping are not suitable for NB-IoT traffic generation because of the high latencies reached in NB-IoT cause the expiration of the timers that regulate packet transmission in these tools and therefore the cancelation of the traffic generation. Instead, Scapy, a packet manipulation and injection framework, has been used, which enables replaying the pre-recorded traffic capture and Nping tool, which allows for packet generation and selecting the protocol, the packet length, and packet interval among other options. Thus, traffic can be reproduced using traffic captures provided by the experimenters or can be simulated configuring properly the packet generation in the Nping tool.
5.3. NB-IoT Performance Results
In this paper, we have quantified the impact that the network configuration has on power consumption and round trip time. Two different scenarios have been defined, following the same approach as in [1]. Scenario 1 called “normal” provides a configuration that corresponds to a situation with good propagation conditions; the second scenario is called “robust” and represents an adverse network scenario which requires configuring a higher number of repetitions. The maximum number of NPDCCH Repetitions has been configured to 8 in both scenarios. This parameter delimits the number of DCI subframe repetitions to 1, 2, 4, or 8. The “normal” scenario has been configured with one DCI subframe and the “robust” scenario has been configured with eight repetitions. Regarding the coding rate, in the normal scenario, we have chosen an Index of Modulation and Coding Scheme (IMCS) of 10 that provides a Transport Block Size (TBS) of 680 and a coding rate of 0.6769 with limited redundancy. In the robust scenario, the coding rate is reduced significantly to enable detection under more impaired conditions. Using a IMCS of 1, a smaller amount of TBS of 256 is used, and an overall coding rate of 0.1346 is obtained with additional redundancy as targeted.
eDRX and PSM have not been configured in these initial experiments and will be the focus of future experiments.
Figure 10 shows the power consumption pattern and the round trip time (RTT) for a series of periodic PING messages that are sent from the UE to the network. In this case, a normal coverage scenario is reproduced by using a moderate redundancy in both time and frequency resource allocations, and a limited number of retransmissions. It can be observed that an average delay of hundreds of ms is obtained for the round trip time because of the repetitions, and that power consumption peaks below 1 W are detected. In this test, the UE is a NB-IoT modem (Hologram Nova, U-Blox SARA-R410) connected to a Next, Unit of Computing (NUC) and the traffic generated is higher than that expected from a sensor. In addition, background traffic is present, which explains the peaks detected.
Large delays are expected in NB-IoT as the downlink control information (DCI) transmitted in the control channel (NPDCCH) and the actual data transmission in the data channel (NPDSCH) are repeated over a number of subframes and not transmitted simultaneously unlike in LTE technologies. For each ping repetition, two bursts of power consumption can be observed, one initial narrow pulse, where the NB-IoT device transmits the ping request to the network, and a second burst where the ping reply is received by the UE and feedback is provided back to the network. In Figure 11, it is shown the equivalent power and RTT patterns for a “robust” configuration designed to address scenarios with worse propagation conditions. For that purpose, a higher number of retransmissions and more redundant resource allocations are used. Both the duration of the power burst and the average power during each burst increase in the robust scenario as shown in the figure. The reason is that the NB-IoT device needs to repeat the uplink transmission during a larger number of subframes. It can also be observed that the delays in the round trip back have increased because of the larger NPDCCH periods and the additional repetitions both for NPDCCH, NPDSCH, and NPUSCH.
In Figure 12, both patterns are superposed for easier comparison. The differences in the power, reaching up to 1.5 W peaks, can be appreciated, as well as the noticeable wider initial transmission and the duration of the power gap to receive the downlink data that matches the measured values on the order of 1 s.
Table 1 provides a comparative of power consumption for different radio technologies during continuous transmission. WLAN, GPRS, UMTS, and HSDPA results have been extracted from [24] and LTE measurements are from [25]. Continuous transmission doesn’t take advantage of eDRX and PSM, the mechanisms used in NB-IoT to reduce power consumption. The effect of these mechanisms are analyzed in [26]. The results obtained in this paper are for the worst case but enable comparing the consumption with other technologies when using continuous transmission. Table 1 shows that NB-IoT power consumption is still lower than the rest of the technologies. It is not the purpose of this article to provide a detailed comparative of power consumption, but demonstrate the versatility offered by the testbed to execute tests in atypical conditions in order to obtain non nominal values, looking for the borders of the applicability of the technology and the system under test.
As an important aspect, due to the extremely reduced capacity of NB-IoT, we have verified that any traffic, even a residual from equipment connected to a NB-IoT device, can be very harmful. Therefore, in scenarios where NB-IoT devices work as modems, we have identified the need to systematically use firewalls or other mechanisms to filter any undesired traffic. Another interesting lesson learned is that some of the traditional traffic generation test tools have problems operating in scenarios of a very low rate of transmission and particularly in the presence of very high peaks of delay as those present in some configurations NB-IoT, so it is advisable to use solutions which have been specifically verified in those scenarios.
6. Conclusions and Future Enhancements
The contributions of this paper are twofold. We have highlighted the recent standardization efforts to complete the value proposition around NB-IoT as a key enabler for massive cellular IoT. Additionally, we have introduced the advantages of TRIANGLE for repeatable experimentation and benchmarking in the heterogeneous IoT device ecosystem.
Although a new radio access technology was specifically designed for NB-IoT in 3GPP Release 13, the standardization work has continued adding multiple features to build a richer self-contained ecosystem around NB-IoT. We have added specific details and insights on positioning and D2D communications as two disruptive features that will open the door to a myriad of use cases.
Finally, we have introduced the new experimentation capabilities added to the TRIANGLE testbed to support NB-IoT testing. The key innovation in this testbed, even beyond the deep flexibility and control of NB-IoT settings and test conditions, relies on the open proposition and generality of the testbed. It welcomes the research community and NB-IoT commercial providers to experiment on applications, devices, and services. We have demonstrated the potential to analyze in detail NB-IoT signaling, while also studying the impact at both the application and power consumption levels of IoT devices using the TRIANGLE testbed. In particular, the TRIANGLE NB-IoT testing framework has been applied in the context of the European project NRG-5 in the performance analysis of smart energy grid solutions using NB-IoT connections. In this paper, we have evaluated the impact on power consumption and latency of NB-IoT communications in order to demonstrate the NB-IoT testing potential of the TRIANGLE testbed.
Author Contributions
Conceptualization, A.D.Z. and F.J.R.T.; methodology, A.D.Z., F.J.R.T. and P.R.; software, A.D.Z. and P.R.; validation, A.D.Z., F.J.R.T. and P.R.; formal analysis, A.D.Z. and F.J.R.T.; investigation, A.D.Z., F.J.R.T. and P.R.; resources, A.D.Z.; data curation, A.D.Z., F.J.R.T. and P.R.; writing—original draft preparation, A.D.Z.; writing—review and editing, A.D.Z. and F.J.R.T.; visualization, A.D.Z. and F.J.R.T.; supervision, A.D.Z.; project administration, A.D.Z.; funding acquisition, A.D.Z. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by H2020 Leadership in Enabling and Industrial Technologies Grant No. 762013, Grant No. 732638 and Grant No. 815178.
Acknowledgments
Work presented in this paper has been funded under the 5G Public Private Partnership (5G-PPP) framework and the EC funded projects H2020 NRG-5 (815178), Fed4Fire+ (732638) and 5GENESIS (815178).
Conflicts of Interest
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Figures and Table
Table 1
Power consumption comparative.
| Average Power Consumption during Continuous Traffic (W) | |
|---|---|
| WLAN | 1.204 |
| GPRS | 1.07 |
| UMTS | 1.503 |
| HSDPA | 1.781 |
| LTE | 2 |
| NB-IoT | 0.3 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Rodríguez, Pilar