% Intended LaTeX compiler: pdflatex
\documentclass[conference]{IEEEtran}
 \usepackage{hyperref}
\usepackage{booktabs}
\usepackage{subfigure}
\usepackage{graphicx}
\usepackage{xcolor}
\usepackage{multirow}
\usepackage{amsmath}

\author{\IEEEauthorblockN{Loic Guegan and
Anne-C\'ecile Orgerie}
\IEEEauthorblockA{Univ Rennes, Inria, CNRS, IRISA, Rennes, France\\
Email: \{loic.guegan, anne-cecile.orgerie\}@irisa.fr}
}


\hypersetup{
 pdfauthor={Anne-Cecile Orgerie},
 pdftitle={Estimating the end-to-end energy consumption of low-bandwidth IoT applications for WiFi devices},
 pdfkeywords={},
 pdfsubject={},
 pdflang={English}}



\title{Estimating the end-to-end energy consumption of low-bandwidth IoT applications for WiFi devices}
\begin{document}

\maketitle
\newcommand{\hl}[1]{\textcolor{red}{#1}}

\begin{abstract}
Information and Communication Technology takes a growing part in the 
worldwide energy consumption. One of the root causes of this increase 
lies in the multiplication of connected devices. Each object of the 
Internet-of-Things often does not consume much energy by itself. Yet, 
their number and the infrastructures they require to properly work 
have leverage. In this paper, we combine simulations and real 
measurements to study the energy impact of IoT devices. In particular, 
we analyze the energy consumption of Cloud and telecommunication 
infrastructures induced by the utilization of connected devices, and 
we propose an end-to-end energy consumption model for these devices. 
\end{abstract}

\begin{IEEEkeywords}
IoT devices, energy consumption, clouds, end-to-end model
\end{IEEEkeywords}

\IEEEpeerreviewmaketitle


\section{Introduction}
In 2018, Information and Communication Technology (ICT) was estimated
to absorb around 3\% of the global energy consumption~\cite{ShiftProject}. 
This consumption is estimated to grow at a rate
of 9\% per year~\cite{ShiftProject}. This alarming growth is explained
by the fast emergence of numerous applications and new ICT
devices. These devices supply services for smart building, smart
factories and smart cities for instance. Through connected sensors
producing data, actuators interacting with their environment and
communication means -- all being parts of the Internet of Things (IoT)
-- they provide optimized decisions. 

This increase in number of devices implies an increase in the energy
needed to manufacture and utilize them. Yet, the overall energy bill
of IoT also comprises indirect costs, as it relies on computing and 
networking infrastructures that consume energy to enable smart
services. Indeed, IoT devices employ Cloud computing
infrastructures to store, analyze and share their data.  

In February 2019, a report by Cisco stated that ``IoT connections will
represent more than half (14.6 billion) of all global connected
devices and connections (28.5 billion) by 2022"~\cite{Cisco2019}. This
will represent more than 6\% of global IP traffic in 2022, against 3\%
in 2017~\cite{Cisco2019}. This increasing impact of IoT devices on
Internet connections induces a growing weight on ICT energy
consumption.  

The energy consumption of IoT devices themselves is only the top of
the iceberg: their use induce energy costs in communication and cloud
infrastructures. In this paper, we estimate the overall energy
consumption of an IoT device environment by combining simulations and
real measurements. We focus on a given application with low bandwidth
requirements, and we evaluate its overall energy consumption: from the
device, through telecommunication networks, and up to the Cloud data
center hosting the application. From this analysis, we derive an
end-to-end energy consumption model that can be used to assess the
consumption of other IoT devices.

While some IoT devices produce a lot of data, like smart vehicles for
instance, many others generate only a small amount of data, like smart
meters. However, the scale matters here: many small devices can end up
producing big data volumes. As an example, according to a report
published by Sandvine in October 2018, the Google Nest Thermostat is
the most significant IoT device in terms of worldwide connections: it
represents 0.16\% of all connections, ranging 55th on the list of
connections~\cite{Sandvine2018}. As a comparison, the voice assistants
Alexa and Siri are respectively 97th and 102nd with 0.05\% of all
connections~\cite{Sandvine2018}. This example highlights the growing
importance of low-bandwidth IoT applications on Internet
infrastructures, and consequently on their energy consumption. 

In this paper, we focus on IoT devices for low-bandwidth applications
such as smart meters or smart sensors. These devices send few
data periodically to cloud servers, either to store them or to get
computing power and take decisions. This is a first step towards a
comprehensive characterization of the global IoT energy
footprint. While few studies address the energy consumption of
high-bandwidth IoT applications~\cite{li_end--end_2018}, to the best
of our knowledge, none of them targets low-bandwidth applications,
despite their growing importance on the Internet infrastructures.

Low-bandwidth IoT applications, such as the Nest Thermostat, often
relies on sensors powered by batteries. For such sensors, reducing
their energy consumption is a critical target. Yet, we argue that
end-to-end energy models are required to estimate the overall impact
of IoT devices, and to understand how to reduce their complete energy
consumption. Indeed, shifting computations to the cloud is often used
to reduce the consumption of IoT devices~\cite{offloading}, without studying the 
additional cost for the cloud infrastructure. Consequently, such
an energy-saving technique, from the IoT device point of view, can
result on an higher overall energy consumption.
Using end-to-end models could prevent these unwanted effects. 

Our contributions include:
\begin{itemize}
\item a characterization of low-bandwidth IoT applications;
\item an analysis of the energy consumption of a low-bandwidth IoT
application including the energy consumption of the WiFi IoT device
and the consumption induced by its utilization on the Cloud and
telecommunication infrastructures;
\item an end-to-end energy model for low-bandwidth IoT applications
relying on WiFi devices.
\end{itemize}

The paper is organized as follows. Section~\ref{sec:sota} presents the
state of the art. The low-bandwidth IoT application is characterized
in Section~\ref{sec:usec}, and details on its architecture are
provided in Section~\ref{sec:model}. Section~\ref{sec:eval} provides
our experimental results combining real measurements and
simulations. Section~\ref{sec:discuss} discusses the key findings an
the end-to-end energy model. Finally, Section~\ref{sec:cl} concludes
this work and presents future work. 



\section{Related Work}
\label{sec:orge831050}
\label{sec:sota}
\subsection{Energy consumption of IoT devices}
\label{sec:org77c2591}
The multiplication of smart devices and smart applications pushes the
limits of Internet: IoT is now used everywhere for home automation,
smart agriculture, e-health, smart cities, logistics, smart grids,
smart buildings, etc.~\cite{Wang2016,Ejaz2017,Minoli2017}. IoT devices
are typically used to optimize processes and the envisioned
application domains include the energy distribution and management. It
can for instance help the energy management of product 
life-cycle~\cite{Tao2016}. Yet, few studies address the impact of IoT itself on
global energy consumption~\cite{jalali_fog_2016,li_end--end_2018} or
CO2 emissions~\cite{Sarkar2018}.  

The underlying architecture of these smart applications usually
includes sensing devices, cloud servers, user applications and
telecommunication networks. Concerning the computing part, the cloud
servers can either be located on Cloud data centers, on Fog
infrastructures located at the network edge, or on home gateways~\cite{Wang2016}.
Various network technologies are employed by IoT
devices to communicate with their nearby gateway; either wired
networks with Ethernet or wireless networks: WiFi, Bluetooth, Near
Field Communication (NFC), ZigBee, cellular network (like 3G, LTE, 4G),
Low Power Wide Area Network (LPWAN),
etc.~\cite{Samie2016,Gray2015}. The chosen technology depends on the
smart device characteristics and the targeted communication
performance. The Google Nest Thermostat can for instance use WiFi,
802.15.4 and Bluetooth~\cite{Nest}. In this paper, we focus on WiFi as
it is broadly available and employed by IoT devices~\cite{Samie2016,ns3-energywifi}.  

Several works aim at reducing the energy consumption of the device
transmission~\cite{Andres2017} or improving the energy efficiency of
the access network technologies~\cite{Gray2015}. An extensive
literature exists on increasing the lifetime of battery-based wireless
sensor networks~\cite{offloading,Wang2016}. Yet, IoT devices present more
diversity than typical wireless sensors in terms of hardware
characteristics, communication means and data production patterns.

Based on real measurements, previous studies have proposed energy
models for IoT devices. Yet, these models are specific to a given kind
of IoT device or a given transmission technology. 
Martinez et al. provide energy consumption measurements for wireless
sensor networks using SIGFOX transmissions and employed for
smart-parking systems~\cite{Martinez2015}. Wu et al. implement an
energy model for WiFi devices in the well-known ns3 network 
simulator~\cite{ns3-energywifi}.

These models can be used to evaluate
the energy efficiency of communication protocols or computation
offloading techniques~\cite{offloading}. However, they do not
provide an overall view of the energy consumption of the entire
system architecture: from the IoT device to the cloud server. 
To the best of our knowledge, one previous work targets 
an end-to-end energy model for IoT devices~\cite{li_end--end_2018}. 
However, this work focus on high-bandwidth IoT devices with data
streaming-oriented applications. This study shows that, in this
case (high-bandwidth IoT applications), the cloud server hosting
the application consumes more energy per IoT devices than the 
device itself (an IP camera in the case study)~\cite{li_end--end_2018}.
In our context of low-bandwidth devices, conclusions could be the
opposite as the IoT devices' consumption is optimized since they
are often powered through batteries.


\subsection{Energy consumption of network and cloud infrastructures}
\label{sec:orga15491a}
IoT architecture rely on telecommunication networks and Cloud
infrastructures to provide services. The data produced by IoT devices
are stored and exploited by servers located either in Cloud data
centers or Fog edge sites. While studies exist on the energy
consumption of network and cloud infrastructures in general~\cite{Ehsan}, 
they do not consider the specific case of IoT devices.
To the best of our knowledge, no study estimates the direct impact of
IoT applications on the energy consumption of these infrastructures.

Most work focusing on energy consumption, Cloud architecture and IoT
applications tries to answer the question: where to locate data
processing in order to save energy~\cite{jalali_fog_2016}, 
to reduce the CO2 impact~\cite{Sarkar2018}, or
to optimize renewable energy consumption~\cite{li_end--end_2018}.   

In both cases, the network and cloud infrastructures, attributing the
energy consumption to a given user or application is a challenging
task. The complexity comes from the shared nature of these
infrastructures: a given Ethernet port in the core of the network
processes many packets coming from a high number of 
sources~\cite{jalali_fog_2016}. Moreover, the employed equipment is not power
proportional: servers and routers consume consequent amounts of
energy while being idle~\cite{mahadevan_power_2009,li_end--end_2018}.
The power consumed by a device is divided into two parts: a dynamic
part that varies with traffic or amount of computation to process, and
a static part that is constant and dissipated even while being idle~\cite{Ehsan}. 
This static part implies that a consequent energy cost
of running an application on a server is due to the device being
simply powered on. Consequently, sharing these static energy costs
among all the concerned users requires an end-to-end model~\cite{li_end--end_2018}.  

In this paper, we focus on IoT devices using WiFi transmission and
generating low data volumes. Our model, extracted from real
measurements and simulations, can be adapted to other kinds of devices
and transmission technologies. 



\section{Characterization of low-bandwidth IoT applications}
\label{sec:org1da7386}
\label{sec:usec}

In this section, we detail the characteristics of the considered IoT
application. While the derived model is more generic, we focus on a
given application to obtain a precise use-case with accurate power
consumption measurements.  

\subsection{IoT device side}

The Google Nest Thermostat relies on five sensors: temperature,
humidity, near-field activity, far-field activity and ambient 
light~\cite{Nest}. Periodical measurements, sent through wireless
communications on the Internet, are stored on Google data centers and
processed to learn the home inhabitants habits. The learned behavior
is employed to automatically adjust the home temperature managed by
heating and cooling systems.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.6\linewidth]{./plots/home.png}
  \caption{Overview of IoT devices.}
  \label{fig:IoTdev}
\end{figure}

Each IoT device senses periodically its environment. Then, it sends
the produced data through WiFi (in our context) to its gateway or
Access Point (AP). The AP is in charge of transmitting the data to the
cloud using the Internet. Figure~\ref{fig:IoTdev} illustrates this
architecture. Several IoT devices can share the same AP in a
home. We consider low-bandwidth applications where devices produces
several network packets during each sensing period. The transmitting
frequency can vary from one to several packet sent per minute~\cite{Cisco2019}. 


\subsection{Cloud server side}
We consider that the link between the AP and the Cloud is composed of
several network switches and routers using Ethernet as shown in 
Figure~\ref{fig:parts}. The number of routers on the path depends on the
location of the server, either in a Cloud data center or in a Fog site
at the edge of the network.

We assume that the server hosting the application data for the users
belongs to a shared cloud facility with classical service level
agreement (SLA). The facility provides redundant storage and computing
means as virtual machines (VMs). A server can host several VMs at the
same time.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=.85\linewidth]{./plots/parts2.png}
  \caption{Overview of the IoT architecture.}
  \label{fig:parts}
\end{figure}

The Cloud part of the application gathers the data sent by the IoT
devices. These data are treated either on the fly (e.g. threshold
detection) or periodically, and action commands are sent back to the
device if required. For instance, if the user has set a targeted 
temperature, the connected thermostat sends the measured
temperature regularly, and once the target is reached, the Cloud server detects
it, and sends back to the IoT device the command to pause the heater.

In the following, we describe the experimental setup, the results and
the derived end-to-end model. For all these steps, we decompose the overall
IoT architecture into three parts: the IoT device part, the networking
part and the cloud part, as displayed on Figure~\ref{fig:parts}. 


\section{Experimental setup}
\label{sec:orgb5f6554}
\label{sec:model}
In this section, we describe the experimental setup employed to
acquire energy measurements for each of the three parts of our
system model. The IoT and the network parts are modeled
through simulations. The Cloud part is modeled using real
servers connected to wattmeters. In this way, it is possible to
evaluate the end-to-end energy consumption of the system. 

\subsection{IoT Part}
\label{sec:orgeb67dd0}
In the first place, the IoT part is composed of several sensors connected to an Access Point (AP)
which form a cell. This cell is studied using the ns3 network
simulator. In the experimental scenario, we setup
between 5 and 15 sensors connected to the AP using WiFi 5GHz 802.11n. The sensors are placed
randomly in a rectangle of \(400m^2\) around the AP which corresponds
to a typical use case for a home environment.

All the cell sensors employ the default WIFI energy model provided by ns3.
This model comprises different power levels depending on the state of the WiFi device
(i.e. idle, transmitting, receiving). The power consumption of receiving and transmitting
states depends on the data rate of the device at a given time. In this paper, 
we consider only one data rate as the target is low-bandwidth devices in a home environment.
The different
energy values used by the energy model are provided in Table~\ref{tab:params}. These parameters
were extracted from previous work~\cite{halperin_demystifying_nodate,li_end--end_2018} on
IEEE 802.11n. Besides, we suppose that the energy source of each
sensor is not limited during the experiments. Thus, each sensor 
can communicate until the end of all the simulations.

As a scenario, sensors send 192 bits packets to the AP composed of: \textbf{1)} A 128 bits
sensors id \textbf{2)} A 32 bits integer representing the temperature \textbf{3)} An integer
timestamp representing the temperature sensing date. They are stored as time series. The data are
transmitted immediately at each sensing interval \(I\) that we vary from 1s to 60s. Finally, the AP is in
charge of relaying data to the cloud via the network part.

\begin{table}[htbp]
  \centering
  \caption{Simulations Energy Parameters}
  \label{tab:wifi-energy}
  \subtable[IoT part]{
    \begin{tabular}{@{}lr@{}}
      Parameter      & Value  \\ \midrule
      Supply Voltage & 3.3V   \\
      Tx             & 0.38A  \\
      Rx             & 0.313A \\
      Idle           & 0.273A \\ \bottomrule
  \end{tabular}}
  \hspace{0.3cm}
  \subtable[Network part]{
    \label{tab:net-energy}
    \begin{tabular}{@{}lr@{}}
      Parameter  & Value \\ \midrule
      Idle       & 0.00001W           \\ 
      Bytes (Tx/Rx)  & 3.4nJ           \\ 
      Pkt (Tx/Rx)    & 192.0nJ         \\ \bottomrule
    \end{tabular}
  }
  \label{tab:params}
\end{table}

\subsection{Network Part}
\label{sec:orgaeb55ca}
The network part represents the network section starting from the AP to the Cloud excluding the
server. It is also modeled into ns3. We consider the server to be 9 hops away from the AP with a
typical round-trip latency of 100ms from the AP to the server~\cite{li_end--end_2018}. 
Each node from the AP to the Cloud 
is a network switch with static and dynamic network energy consumption. The first 8
hops are edge switches and the last one is consider to be a core router as mentioned 
in~\cite{jalali_fog_2016}. ECOFEN~\cite{orgerie_simulation_2017} is used to model the energy
consumption of the network part. ECOFEN is an ns3 network energy module dedicated to wired
networks. It is based on an energy-per-bit and energy-per-packet
model for the dynamic energy consumption~\cite{sivaraman_profiling_2011,Serrano2015}, 
and it includes also a static energy consumption.
The different values used to instantiate the ECOFEN energy model for the
network part are shown in left part of Table~\ref{tab:params} and come from previous work~\cite{cornea_studying_2014-1}.

\subsection{Cloud Part}
\label{sec:orgfc9ea54}
Finally, to measure the energy consumed by the Cloud part, we use a real server from the large-scale
test-bed Grid'5000. Grid'5000 provides clusters composed of several servers which
are connected to wattmeters. The wattmeters provide 50
instantaneous power measurements per second and per server. This
way, we can benefit from real energy measurements. The server used
in the experiment embeds two Intel Xeon E5-2620 v4 processors with
64 GB of RAM and 600GB 
of disk space on a Linux based operating system. This server is configured to use KVM as
virtualization mechanism. We deploy a classical Debian x86\_64 distribution on the Virtual Machine
(VM) along with a MySQL database. We use different amounts of allocated memory for the VM namely
1024MB/2048MB/4096MB to highlight its effects on the server energy
consumption. The server only hosts this VM in order to easily isolate its
power consumption.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.6\linewidth]{./plots/g5k-xp.png}
  \caption{Grid'5000 experimental setup.}
  \label{fig:g5kExp}
\end{figure}

The data sent by the IoT devices are simulated using another
server from the same cluster. This server is in charge of sending
the data packets to the VM hosting the application in order to fill
its database. In the following, each data packet coming from an IoT
device and addressed to the application VM is called a request. Consequently, it is easy to vary the
different application characteristics namely: \textbf{1)} The number
of requests, to virtually
add/remove sensors \textbf{2)} The requests interval, to study the
impact of the transmitting frequency. Figure~\ref{fig:g5kExp} presents this simulation
setup. We consider here a simple IoT application able to store the sensed values
and provide them upon request. We do not include any data mining or machine learning
techniques as they are highly dependent on the targeted application and quality
of service. They can be added a posteriori to the derived end-to-end model
if they are known, or estimated from specific energy models.




\section{Evaluation}
\label{sec:org8201f68}
\label{sec:eval}

In this section, we analyze the experimental results. All the experiments
concerning IoT devices and network parts (Table~\ref{tab:sensorsSendIntervalEffects} 
and Figure~\ref{fig:sensorsNumber}) 
are based on simulations using ns3, 
while all the experiments on Cloud servers (Figures~\ref{fig:vmSize}, \ref{fig:sensorsNumber-server}, \ref{fig:sensorsFrequency}, 
and~\ref{fig:sensorsNumber-WPS})
are real measurements performed on 
the Grid'5000 experimental platform.

\subsection{IoT and Network Power Consumption}
\label{sec:org1d05c1b}
   In a first place, we start by studying the impact of the sensors' transmission frequency on their
   energy consumption. To this end, we run several simulations in ns3 with 15 sensors using
   different transmission frequencies. The results provided by 
   Table~\ref{tab:sensorsSendIntervalEffects} show that the transmission frequency has a very limited impact
   on the energy consumption of sensor and network parts, and on the average end-to-end application delay. 
This is due to the fact that in such a scenario with very small
   number of communications spread over the time, sensors do not have to contend for accessing to the
   WiFi channel. Note that for the network part, we include the dynamic power consumption due to the
   traffic generated by the sensors themselves, and we split the static power consumption of the routers
   according the the utilization ratio taken by the sensors. This model is detailed in Section~\ref{sec:discuss}.

% Please add the following required packages to your document preamble:
% \usepackage{booktabs}
%\begin{table*}[htbp]
%\centering
%\caption{Sensors transmission interval effects with 15 sensors}
%\label{tab:sensorsSendIntervalEffects}
%\begin{tabular}{@{}lrrrrr@{}}
%\toprule
%Transmission Interval        & 10s            & 30s            & 50s       & 70s       & 90s      \\ \midrule
%Sensor Power     & 13.517\hl{94}W & 13.517\hl{67}W & 13.51767W & 13.51767W & 13.517\hl{61}W \\
%Network Power     & 0.441\hl{88}W & 0.441\hl{77}W & 0.44171W & 0.44171W & 0.441\hl{71}W \\
%Application Delay & 0.09951s       & 0.10021s       & 0.10100s  & 0.10203s  & 0.10202s  \\ \bottomrule
%\end{tabular}
%\end{table*}

\begin{table}[htbp]
\centering
\caption{Sensors transmission interval effects with 15 sensors}
\label{tab:sensorsSendIntervalEffects}
\begin{tabular}{@{}lrrr@{}}
\toprule
Transm. Interval & Sensor Power & Network Power & Application Delay \\ \midrule
10s & 13.517\hl{94}W & 0.441\hl{88}W & 0.09951s \\
30s & 13.517\hl{67}W & 0.441\hl{77}W & 0.10021s \\
50s & 13.51767W      & 0.44171W      & 0.10100s \\
70s & 13.51767W      & 0.44171W      & 0.10203s \\
90s & 13.517\hl{61}W & 0.441\hl{71}W & 0.10202s \\ \bottomrule
\end{tabular}
\end{table}

   Previous work~\cite{li_end--end_2018} on a similar scenario shows that increasing application
   accuracy impacts strongly the energy consumption in the context of data stream analysis. However,
   in our case, application accuracy is driven by the sensing interval and thus, the transmission
   frequency of the sensors.
In our case with small and sporadic network traffic, these results show that with a reasonable
   transmission interval, the energy consumption of the IoT and the
   network parts are almost not affected by the
   variation of this transmission interval. In fact, transmitted data are not large enough to
   leverage the energy consumed by the network.

We then vary the number of sensors in the WiFi cell.
Figure~\ref{fig:sensorsNumber} represents the energy consumed by the sensor and the network (from the AP to the cloud) parts
according to the number of sensors. Similarly to the results of Table~\ref{tab:sensorsSendIntervalEffects}, the network part
is almost not affected by the number of sensors as their traffic is negligible compared to the network devices capacities.
Consequently, sensors energy consumption is dominant, as each sensor adds its own consumption.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.75\linewidth]{./plots/numberSensors-WIFINET.png}
  \caption{Analysis of the variation of the number of sensors on the IoT/Network part energy consumption for a transmission interval of 10s.}
  \label{fig:sensorsNumber}
\end{figure}


\subsection{Cloud Energy Consumption}
\label{sec:org9daa066}
In this end-to-end energy consumption study, cloud accounts for a huge part of the overall energy
consumption. According to a report~\cite{shehabi_united_2016-1} on United States data center energy
usage, the average Power Usage Effectiveness (PUE) of an hyper-scale data center is 1.2. This metric
accounts for indirect data center power costs, such as the cooling infrastructure and the power distribution losses. 

In our analysis, we use the PUE to account for these costs and all energy measurement on cloud servers use it. 
It means that the power consumption of the server is multiplied by
the PUE~\cite{Ehsan}.

\begin{figure*}[htbp]
\begin{minipage}[t]{0.65\textwidth}
  \centering
  \includegraphics[width=.9\linewidth]{./plots/vmSize-cloud.png}
  \caption{Server power consumption multiplied by the PUE (= 1.2) using 20 sensors sending data every 10s for various VM memory sizes}
  \label{fig:vmSize}
\end{minipage}
\hspace{0.5cm}
\begin{minipage}[t]{0.27\textwidth}
  \centering
    \includegraphics[width=1.\linewidth]{./plots/sensorsNumberLine-cloud.png}
    \caption{Average server power consumption multiplied by the PUE (= 1.2) for sensors sending data every 10s}
    \label{fig:sensorsNumber-server}
\end{minipage}
\end{figure*}


Firstly, we analyze the impact of the VM allocated memory on the server energy
consumption. Figure~\ref{fig:vmSize} depicts the server energy consumption according to the VM
allocated memory for 20 sensors sending data every 10s. Note that
the horizontal red line represents
the average energy consumption for the considered sample of energy values. We can see that at
each transmission interval, the server faces spikes of energy
consumption. However, the amount of allocated memory to the VM
does not significantly influence the server energy consumption. In
fact, simple database requests do not need any particular 
heavy memory accesses and processing time. Thus, remaining experiments are based on VM with 1024MB
of allocated memory. 

Here, for clarity's sake, we use VMs with one virtual CPU (i.e. one physical CPU core). 
The influence of the number of core on the server' energy consumption has been widely 
studied in the literature~\cite{heinrich_predicting_2017}. For a given application that scales
smoothly when adding cores, the relation between number of cores and power consumption is
linear. In other words, adding cores for the execution of a parallel application multiplies
accordingly the dynamic energy consumption on the server part.

Next, we study the effects of increasing the number of sensors on the server energy consumption.
Figure~\ref{fig:sensorsNumber-server} presents the results of the average server energy
consumption when varying the number of sensors from 20 to 500, while Figure~\ref{fig:sensorsNumber-WPS}
presents the average server energy cost per sensor according to the
number of sensors. These results show a clear linear relation between the number of sensors and
the server energy consumption.

Moreover, we can see that the more sensors we have per VM, the
more energy we can save. In fact, since the server's idle power
consumption is high (around 97 Watts), it is more
energy efficient to maximize the number of sensors per server. As shown on 
Figure~\ref{fig:sensorsNumber-WPS}, a significant amount of energy can be save when passing from 20 to
300 sensors per VM. Note that these measurements are not the row
measurements taken from the wattmeters: they include the PUE
but they are not shared among all the VMs that could be hosted on this
server. So, for the studied server, its static power consumption
(also called idle consumption) is around 83.2 Watts and we consider
a PUE of 1.2, this value is taken from~\cite{shehabi_united_2016-1}\}.
As traditionally cloud servers host several VMs at the same time, our 
model will in fact share the static power consumption of the server
among the VMs it can host, depending on their VM size (allocated CPU and
RAM). This model is detailed in Section~\ref{sec:discuss}.


A last parameter can leverage server energy consumption, namely
sensors transmission interval. In addition 
to increasing the application accuracy, sensors transmission frequency increases network traffic and
database accesses. Figure~\ref{fig:sensorsFrequency} presents the impact on the server energy
consumption when changing the transmission interval of 50 sensors
to 1s, 10s and 30s. We can see that, the lower sensors transmission
interval is, the more server energy consumption peaks
occur. Therefore, it leads to an increase of the server energy consumption.

\begin{figure*}[htbp]
\begin{minipage}[t]{0.65\textwidth}
  \centering
  \includegraphics[width=0.9\linewidth]{plots/sendInterval-cloud.png}
  \caption{Server power consumption multiplied by the PUE (= 1.2) for 50 sensors sending requests at different transmission interval.}
  \label{fig:sensorsFrequency}
\end{minipage}
\hspace{0.5cm}
\begin{minipage}[t]{0.27\textwidth}
  \centering
    \includegraphics[width=1.\linewidth]{./plots/WPS-cloud.png}
    \caption{Average sensors power cost on the server hosting only our VM with PUE (= 1.2) for sensors sending data every 10s}
    \label{fig:sensorsNumber-WPS}
\end{minipage}
\end{figure*}


In the next section, we use the hints detailed here and extracted from the
real and simulated experiments in order to provide an end-to-end energy
model that can be used for low-bandwidth IoT applications.

\section{End-to-End Consumption Model}
\label{sec:orgfd3b6ae}
\label{sec:discuss}

To have an overview of the energy consumed by the overall system, it is important to consider the
end-to-end energy consumption.
We detail here the model used to attribute the energy
consumption of our application for each part of the
architecture. 

For a given IoT device, we have:
\begin{enumerate}
\item For the IoT part, the entire consumption of the IoT device
belongs to the system's accounted consumption.
\item For the network part, the data packets generated by the IoT
device travel through network switches, routers and ports that
are shared with other traffic.
\item For the cloud part, the VM hosting the data is shared with
other IoT devices belonging to the same application and the
server hosting the VM also hosts other VMs. Furthermore, the
server belongs to a data center and takes part in the overall
energy drawn to cool the server room.
\end{enumerate}


Concerning the IoT part, we include the entire IoT device power
consumption. Indeed, in our targeted low-bandwidth IoT application,
the sensor is dedicated to this application. From Table~\ref{tab:params}, one can
derive that the static power 
consumption of one IoT sensor is around 0.9 Watts. Its dynamic part
depends on the transmission frequency. So the power consumption of an IoT device:

\begin{footnotesize}
\begin{align*}
P^{IoTdevice} & = P_{static}^{IoT} + P_{dynamic}^{IoT}\\
               & = \frac{P_{idle}\times (T - t_{RX}-t_{TX})}{T} + \frac{P_{RX}\times t_{RX} + P_{TX}\times t_{TX}}{T} 
\end{align*}
\end{footnotesize}

where \(P_{static}^{IoT}\) and \(P_{dynamic}^{IoT}\) are respectively the static
and dynamic power consumption of the IoT device, $t_{RX}$, $t_{TX}$, and $t_{idle}$ are
the duration spent in each mode (receiving, transmitting and idle) and $P_{RX}$, $P_{TX}$, and $P_{idle}$ the
respective power consumption of each mode, and $T$ is the transmission interval between
two communications from the IoT device to the cloud server.

Concerning the sharing of the network costs, for each router, we
consider its aggregate bandwidth (on all the ports), its average
link utilization and the share taken by our IoT application. For a
given network device, we compute our share of the static energy
part as follows: 

\begin{footnotesize}
\[P_{static}^{netdevice} = \frac{P_{static}^{device} \times Bandwidth^{application}}{AggregateBandwidth^{device}
\times LinkUtilization^{device}}\]
\end{footnotesize}

where \(P_{static}^{device}\) is the static power consumption of the
network device (switch fabrics for instance or gateway),
\(Bandwidth^{application }\) is the bandwidth used by our IoT application,
\(AggregateBandwidth^{device }\) is the overall aggregated bandwidth of the
network device on all its ports, and \(LinkUtilization^{device}\) is the
effective link utilization percentage. The \(Bandwidth^{application }\)
depends on the transmission frequency in our use-case.
The formula includes the
link utilization in order to charge for the effective energy cost
per traffic and not for the theoretical upper bound which is the
link bandwidth. Indeed, using such an upper bound leads to greatly
underestimate our energy part, since link utilization typically
varies between 5 to 40\%~\cite{Hassidim2013,Zhang2016}.

Similarly, for each network port, we take the share attributable to
our application: the ratio of our bandwidth utilization over the
port bandwidth multiplied by the link utilization and the overall
static power consumption of the port. Table~\ref{tab:netbidules}
summarizes the parameters used in our model, they are taken from~\cite{mahadevan_power_2009,Hassidim2013}. These are the parameters
used in our formula to compute the values that we used in the
simulations and that are presented in left part of Table~\ref{tab:params}. 


\begin{table}[htbp]
  \centering
  \caption{Network Devices Parameters}
  \label{tab:netbidules}
    \begin{tabular}{l|l}
      Device      & ~Parameters  \\ \midrule
      \multirow{2}{*}{Gateway} & Static power = 8.3 Watts\\
       & Bandwidth = 54Mbps\\
       & Utilization = 10\%   \\ \hline
      \multirow{2}{*}{Core router} & Static power = 555 Watts\\
       & 48 ports of 1 Gbps\\
       & Utilization = 25\% \\ \hline
      \multirow{2}{*}{Edge switch} & Static power = 150 Watts\\
       & 48 ports of 1 Gbps\\
       & Utilization = 25\% \\
      \bottomrule
  \end{tabular}
\end{table}

The dynamic consumption of the network part includes a cost
per packet and a cost per byte for each network device as detailed in~\cite{orgerie_simulation_2017}:
\[P_{dynamic}^{netdevice} = \frac{P_{byte}^{device}\times NbBytes + P_{pkt}^{device} \times NbPkts}{T}\] 
with $NbBytes$ and $NbPkts$ respectively the number of bytes and packets sent by the application
during one transmission interval and \(P_{byte}^{device}\) and \(P_{pkt}^{device}\) the power consumption 
per network device for each byte and each packet respectively.



For the sharing of the Cloud costs, we take into account the number
of VMs that a server can host, the CPU utilization of a VM and the
PUE. For a given Cloud server hosting our IoT application, we
compute our share of the static energy part as follows: 

\[P_{static}^{Cloudserver} = \frac{P_{static}^{server} \times PUE^{DataCenter}}{HostedVMs^{server}}\]

Where \(P_{static}^{server}\) is the static power consumption of the
server, \(PUE^{DataCenter}\) is the data center PUE, and
\(HostedVMs^{server}\) is the number of VMs a server can host. This last
parameter should be adjusted in the case of VMs with multiple
virtual CPUs. We do not
consider here over-commitment of Cloud servers. Yet, the dynamic
energy part is computed with the real dynamic measurements, so it
accounts for VM over-provisioning and resource under-utilization.

In our case, the Cloud server has 14 cores, which corresponds to
the potential hosting of 14 small VMs with one virtual CPU each,
and each vCPU is pinned to a server core. We consider that for
fault-tolerance purpose, the IoT application has a replication
factor of 2, meaning that two cloud servers store its database.

The Figure~\ref{fig:end-to-end} represents the end-to-end system
energy consumption using the model described above while varying
the number of sensors for a transmission interval of 10
seconds. The values are extracted from the experiments presented in
the previous section. 

\begin{figure}[htbp]
  \centering
  \hspace{1cm}
  \includegraphics[width=.9\linewidth]{plots/final.png}
  \label{fig:end-to-end}
  \caption{End-to-end network energy consumption using sensors interval of 10s}
\end{figure}


Note that, for small-scale systems, with WiFi IoT devices, the IoT
sensor part is dominant in the overall energy consumption. Indeed,
the IoT application induces a very small cost on Cloud and network
infrastructures compared to the IoT device cost. But, our model
assumes that a single VM is handling multiple users (up to 300
sensors), thus being energy-efficient. Conclusions would be
different with one VM per user in the case of no over-commitment on
the Cloud side. For the network infrastructure, in our case of
low-bandwidth utilization (one data packet every 10 seconds), the
impact is almost negligible.

Another way of looking at these results is to observe that only for
a high number of sensors (more than 300), the power consumption of Cloud and
network parts start to be negligible (few percent). It means that,
if IoT applications handle clients one by one (i.e. one VM per
client), the impact is high on cloud and network part if they have
only few sensors. The energy efficiency is really poor for only few
devices: with 20 IoT sensors, the overall energy cost to handle these
devices is almost doubled compared to the energy consumption of the IoT devices
themselves. Instead of increasing the number of sensors, which
would result in a higher overall energy consumption, one should
focus on reducing the energy consumption of IoT devices, especially
WiFi devices which are common due to WiFi availability
everywhere. One could also focus on improving the energy cost of
Cloud and network infrastructure for low-bandwidth applications and
few devices.



\section{Conclusion}
\label{sec:org76c5125}
\label{sec:cl}

Information and Communication Technology takes a growing part in the 
worldwide energy consumption. One of the root causes of this increase 
lies in the multiplication of connected devices. Each object of the 
Internet-of-Things often does not consume much energy by itself. Yet, 
their number and the infrastructures they require to properly work 
have leverage.

In this paper, we combine simulations and real 
measurements to study the energy impact of IoT devices. In particular, 
we analyze the energy consumption of Cloud and telecommunication 
infrastructures induced by the utilization of connected devices.
Through the fine-grain analysis of a given low-bandwidth IoT device
periodically sending data to a Cloud server using WiFi,
we propose an end-to-end energy consumption model. 
This model provides insights on the hidden part of the iceberg: the
impact of IoT devices on the energy consumption of Cloud and network
infrastructures.

On our use-case, we show that for a given sensor, its
larger energy consumption is on the sensor part. But the impact on the
Cloud and network part is huge when using only few sensors with
low-bandwidth applications.
Consequently, with the
IoT exploding growth, it becomes necessary to improve the energy
efficiency of applications hosted on Cloud infrastructures and of IoT devices.

Our future work includes studying other types of IoT wireless
transmission techniques that would be more energy-efficient. We also
plan to study other
IoT applications in order to increase the applicability of our model
and provide advice for increasing the energy-efficiency of IoT infrastructures.


\section*{Acknowledgments}
Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by
Inria and including CNRS, RENATER and several Universities as well as other organizations (see \url{https://www.grid5000.fr}).

\bibliographystyle{IEEEtran}
\bibliography{references}
\end{document}
