path: root/2019-ICA3PP.org

#+TITLE: Estimating the end-to-end energy consumption of low-bandwidth IoT applications for WiFi devices

#+EXPORT_EXCLUDE_TAGS: noexport
#+STARTUP: hideblocks
#+OPTIONS: H:5 author:nil email:nil creator:nil timestamp:nil skip:nil toc:nil ^:nil
#+LATEX_CLASS: llncs
#+LATEX_HEADER: \usepackage{hyperref}
#+LATEX_HEADER: \usepackage{booktabs}
#+LATEX_HEADER: \usepackage{subfigure}
#+LATEX_HEADER: \usepackage{graphicx}
#+LATEX_HEADER: \usepackage{xcolor}
#+LATEX_HEADER: \author{
#+LATEX_HEADER:       Loic Guegan and
#+LATEX_HEADER:       Anne-Cécile Orgerie\\  
#+LATEX_HEADER: }
#+LATEX_HEADER: \institute{Univ Rennes, Inria, CNRS, IRISA, Rennes, France\\
#+LATEX_HEADER: Emails: loic.guegan@irisa.fr, anne-cecile.orgerie@irisa.fr
#+LATEX_HEADER: }





#+BEGIN_EXPORT latex
\newcommand{\hl}[1]{\textcolor{red}{#1}}
#+END_EXPORT

#+BEGIN_EXPORT latex
\begin{abstract}
Information and Communication Technology takes a growing part in the 
worldwide energy consumption. One of the root causes of this increase 
lies in the multiplication of connected devices. Each object of the 
Internet-of-Things often does not consume much energy by itself. Yet, 
their number and the infrastructures they require to properly work 
have leverage. In this paper, we combine simulations and real 
measurements to study the energy impact of IoT devices. In particular, 
we analyze the energy consumption of Cloud and telecommunication 
infrastructures induced by the utilization of connected devices, And 
we propose an end-to-end energy consumption model for these devices. 
\end{abstract}
#+END_EXPORT


* Introduction 
In 2018, Information and Communication Technology (ICT) was estimated
to absorb around 3% of the global energy consumption
\cite{ShiftProject}. This consumption is estimated to grow at a rate
of 9% per year \cite{ShiftProject}. This alarming growth is explained
by the fast emergence of numerous new applications and new ICT
devices. These devices supply services for smart building, smart
factories and smart cities for instance, providing optimized decisions
based on data produced by smart devices. All these connected devices
constitute the Internet of Things (IoT): connected devices with
sensors producing data, actuators interacting with their environment
and communication means. 
 
This increase in number of devices implies an increase in the energy
needed to manufacture and utilize all these devices. Yet, the overall energy
bill of IoT also comprises indirect costs as it relies on computing and
networking infrastructures that consume energy to enable smart
services. Indeed, IoT devices communicate with Cloud computing
infrastructures to store, analyze and share their data.  

In February 2019, a report by Cisco stated that ``IoT connections will
represent more than half (14.6 billion) of all global connected
devices and connections (28.5 billion) by 2022" \cite{Cisco2019}. This
will represent more than 6% of global IP traffic, against 3% in
2017 \cite{Cisco2019}. This increasing impact of IoT devices on
Internet connections induces a growing weight on ICT energy
consumption.  

The energy consumption of IoT devices themselves is only the top of
the iceberg: their use induce energy costs in communication and cloud
infrastructures. In this paper, we estimate the overall energy
consumption of an IoT device environment by combining simulations and
real measurements. We focus on a given application with low bandwidth
requirement and we evaluate its overall energy consumption: from the
device, through telecommunication networks, and up to the Cloud data
center hosting the application. From this analysis, we derive an
end-to-end energy consumption model that can be used to assess the
consumption of other IoT devices.

While some IoT devices produce a lot of data, like smart vehicles for
instance, many others generate only a small amount of data, like smart
meters. However, the scale matters here: many small devices can end up
producing big data volumes. As an example, according to a report
published by Sandvine in October 2018, the Google Nest Thermostat is
the most significant IoT device in terms of worldwide connections: it
represents 0.16% of all connections, ranging 55th on the list of
connections \cite{Sandvine2018}. As a comparison, the voice assistants
Alexa and Siri are respectively 97th and 102nd with 0.05% of all
connections \cite{Sandvine2018}. This example highlights the growing
importance of low-bandwidth IoT applications on Internet
infrastructures, and consequently on their energy consumption. 

In this paper, we focus on IoT devices for low-bandwidth applications
such as smart meters or smart sensors. These applications send few
data periodically to cloud servers, either to store them or to get
computing power and take decisions. This is a first step towards a
comprehensive characterization of the IoT energy footprint. While few
studies address the energy consumption of high-bandwidth IoT
applications \cite{li_end--end_2018}, to the best of our knowledge,
none of them targets low-bandwidth applications, despite their growing 
importance on the Internet infrastructures.

Low-bandwidth IoT applications, such as the Nest Thermostat, often
relies on sensors powered by batteries. For such sensors, reducing
their energy consumption is a critical target. Yet, we argue that
end-to-end energy models are required to estimate the overall impact
of IoT devices and to understand how to reduce their complete energy
consumption. Without such models, one could optimize the consumption
of on-battery devices at a heavier cost for cloud servers and
networking infrastructures, resulting on an higher overall energy 
consumption. Using end-to-end models could prevent these unwanted
effects. 

Our contributions include:
- a characterization of low-bandwidth IoT applications;
- an analysis of the energy consumption of a low-bandwidth IoT
  application including the energy consumption of the IoT device and
  the consumption induced by its utilization on the Cloud and
  telecommunication infrastructures;
- an end-to-end energy model for low-bandwidth IoT applications.

The paper is organized as follows. Section \ref{sec:sota} presents the
state of the art. The low-bandwidth IoT application is characterized
in Section \ref{sec:usec}, and details on its architecture are
provided in Section \ref{sec:model}. Section \ref{sec:eval} provides
our experimental results using real measurements and
simulations. Section \ref{sec:discuss} discusses the key findings an
the end-to-end energy model. Finally, Section \ref{sec:cl} concludes
this work and presents future work. 



* Related Work 
#+LaTeX: \label{sec:sota}
** Energy consumption of IoT devices
Smart apps and devices everywhere

Smart industry \cite{Wang2016} : archi with sensing devices, cloud
server, user applications and networks

IoT archi : devices, gateways, fog and clouds \cite{Samie2016}

Smart cities \cite{Ejaz2017}

Smart building \cite{Minoli2017}

home automation, smart agriculture, eHealth, logistics, smart grids

product life-cycle energy management \cite{Tao2016}


focusing on access network technologies \cite{Gray2015}, 

improving device transmission \cite{Andres2017}

modeling the energy consumption of WSN devices \cite{Martinez2015} or
the WiFi transmission \cite{ns3-energywifi}

on organizing wireless sensor communications to increase the network
lifetime \cite{Wang2016}

CO2 impact of IoT and fog computing architectures vs Cloud
\cite{Sarkar2018} 


Fog archi to use more renewable energy \cite{li_end--end_2018} or
reduce communication costs \cite{jalali_fog_2016}

** Energy consumption of network and cloud infrastructures
net models 
server models + VM sharing


* Characterization of low-bandwidth IoT applications
#+LaTeX: \label{sec:usec}



** Application Characteristic

   #+BEGIN_COMMENT
   The IoT part is composed of an Access Point (AP), connected to several sensors using WIFI. In the
   system, the IoT part is considered as the part where the system data are created. In fact, the
   data life cycle start when the sensors records their respective local temperature at a frequency
   $f$ and the local timestamp. Then, these data are transmitted through the network along with an
   arbitrary sensor id of 128 bits. Finally, the AP is in charge to transmit the data to the cloud
   using the network part.

   The network part is considered as the medium that link the IoT part to the cloud. It is composed
   of several network switches and router and it is considered to be a wired network.

   #+END_COMMENT

   
    #+BEGIN_EXPORT latex
    \begin{figure}
      \centering
      \includegraphics[width=0.6\linewidth]{./plots/home.png}
      \caption{Overview of IoT devices.}
      \label{fig:IoTdev}
    \end{figure}
    #+END_EXPORT
   
      
** Cloud Infrastructure


    #+BEGIN_EXPORT latex
    \begin{figure}
      \centering
      \includegraphics[width=0.85\linewidth]{./plots/parts2.png}
      \caption{Overview of the IoT architecture.}
      \label{fig:parts}
    \end{figure}
    #+END_EXPORT

* Experimental setup 
#+LaTeX: \label{sec:model}
  Our system model is divided in three parts. First, the IoT and the network parts are modeled through
  simulations. Then, the Cloud part is modeled using real servers connected to wattmeters. In this way,
  it is possible to evaluate the end-to-end energy consumption of the system.

** IoT Part
   In the first place, the IoT part is composed of several sensors connected to an Access Point (AP)
   which form a cell. This cell is evaluated using the ns-3 network simulator. Consequently, we setup
   between 5 and 15 sensors connected to the AP using WiFi 5GHz 802.11n. The node are placed
   randomly in a rectangle of $400m^2$ around the AP which corresponds to a typical real use case. All
   the cell nodes are setup with the default WIFI energy model provided by ns-3. The different
   energy values used by the energy model are provided on Table \ref{tab:wifi-energy}. These energy
   were extracted from previous work\cite{halperin_demystifying_nodate,li_end--end_2018} on
   802.11n. Besides, we suppose that the energy source of each nodes are unlimited and thus each of
   them can communicate until the end of all the simulations.

   As a scenario, sensors send 192 bits packets to the AP composed of: \textbf{1)} A 128 bits
   sensors id \textbf{2)} A 32 bits integer representing the temperature \textbf{3)} An integer
   timestamp representing the temperature sensing time to store them as time series. The data are
   transmitted immediately at each sensing interval $I$ varied from 1s to 60s. Finally, the AP is in
   charge of relaying data to the cloud via the network part.
   
   #+BEGIN_EXPORT latex
   \begin{table}[]
     \centering
     \caption{Simulations Energy Parameters}
     \label{tab:wifi-energy}
     \subtable[Wifi]{
       \begin{tabular}{@{}lr@{}}
         Parameter      & Value  \\ \midrule
         Supply Voltage & 3.3V   \\
         Tx             & 0.38A  \\
         Rx             & 0.313A \\
         Idle           & 0.273A \\ \bottomrule
     \end{tabular}}
     \hspace{0.3cm}
     \subtable[Network]{
       \label{tab:net-energy}
       \begin{tabular}{@{}lr@{}}
         Parameter  & Value \\ \midrule
         Idle       & 1W           \\ 
         Bytes (Tx/Rx)  & 3.4nJ           \\ 
         Pkt (Tx/Rx)    & 192.0nJ         \\ \bottomrule
       \end{tabular}
     }
   \end{table}
   #+END_EXPORT

** Network Part
   The network part represents the a network section starting from the AP to the Cloud excluding the
   server. It is also model into ns-3. We consider the server to be 9 hops away from the AP with a
   typical round-trip latency of 100ms from the AP to the server. Each node from the AP to the Cloud
   is assume to be network switches with static and dynamic network energy consumption. The first 8
   hop are edge switches and the last one is consider to be a core switch as mention in
   \cite{jalali_fog_2016}. ECOFEN \cite{orgerie_ecofen:_2011} is used to model the energy
   consumption of the network part. ECOFEN is a ns-3 network energy module dedicated to wired
   network. It is based on an energy-per-bit model including static energy consumption by assuming a
   linear relation between the amount of data sent to the network interface and its power
   consumption. The different energy values used to instantiate the ECOFEN energy model for the
   network part are shown in Table \ref{tab:net-energy} and come from previous work
   \cite{cornea_studying_2014-1}.

** Cloud Part
   Finally, to measure the energy consumed by the server, we used real server from the large-scale
   test-beds Grid5000 (G5K). In fact, G5K has a cluster called Nova composed of several nodes which
   are connected to watt-meters. In this way, we can benefit from real energy measurements. The
   server used in the experiment include an Intel Xeon E5-2620 processor with 64 GB of RAM and 600GB
   of disk space on a Linux based operating system. This server is configured to use KVM as
   virtualization mechanism. We deploy a classical Linux x86_64 distribution on the Virtual Machine
   (VM) along with a MySQL database. We used different amount of allocated memory for the VM namely
   1024MB/2048MB/4096MB to highlight its effects on the server energy consumption.

   The sensors requests are simulated using another server. This server is in charge to send hundred
   of requests to the VM in order to fill the database. Consequently, it is easy to vary the
   different requests characteristics namely: \textbf{1)} The number request, to virtually
   add/remove sensors \textbf{2)} The requests interval.

* Evaluation 
#+LaTeX: \label{sec:eval}
** IoT/Network Consumption
   In a first place, we start by studying the impact of the sensors position on their energy
   consumption. To this end, we run several simulations in ns-3 with different sensors position. The
   results provided by Table \ref{tab:sensorsSendIntervalEffects} show that sensors position have a very low impact
   on the energy consumption and on the application delay. It has an impact of course, but it is very
   limited. This due to the fact that in such a scenario with very small number of communications
   spread over the time, sensors don't have to contend for accessing to the Wifi channel.


   #+BEGIN_EXPORT latex
   % Please add the following required packages to your document preamble:
   % \usepackage{booktabs}
   \begin{table*}[]
   \centering
   \caption{Sensors send interval effects}
   \label{tab:sensorsSendIntervalEffects}
   \begin{tabular}{@{}lrrrrr@{}}
   \toprule
   Sensors Send Interval      & 10s      & 30s      & 50s      & 70s      & 90s      \\ \midrule
   Sensors Power Consumption  & 13.517\hl{94}W & 13.517\hl{67}W & 13.51767W & 13.51767W & 13.517\hl{61}W \\
   Network Power Consumption  & 10.441\hl{78}W & 10.441\hl{67}W & 10.44161W & 10.44161W & 10.441\hl{61}W \\
   Average Appplication Delay & 17.81360s & 5.91265s  & 3.53509s  & 2.55086s  & 1.93848s  \\ \bottomrule
   \end{tabular}
   \end{table*}
   #+END_EXPORT


   Previous work \cite{li_end--end_2018} on similar scenario shows that increasing application
   accuracy impact strongly the energy consumption in the context of data stream analysis. However,
   in our case, application accuracy is driven by the sensing interval and thus, the transmit
   frequency of the sensors. Therefore, we varied the transmission interval of the sensors from 1s
   to 60s. Some of these results are proposed on Table \ref{tab:sensorsSendIntervalEffects}.  In
   case of small and sporadic network traffic, these results show that with a reasonable
   transmission interval the energy consumption of the IoT/Network if almost not affected by the
   variation of this transmission interval. In fact, transmitted data are not large enough to
   leverage the energy consumed by the network.

  

   The number of sensors is a dominant factor that leverage the energy consumption of the
   IoT/Network part. Therefore, we varied the number of sensors in the Wifi cell to analyze its
   impact. The Figure \ref{fig:sensorsNumber} represents the energy consumed by each simulated part
   according the the number of sensors. It is clear that the energy consumed by the network is the
   dominant part. However, since the number of sensors is increasing the energy consumed by the
   network will become negligible face to the energy consume by the sensors. In fact, deploying new
   sensors in the cell do not introduce much network load. To this end, sensors energy consumption
   is dominant.
   
    #+BEGIN_EXPORT latex
    \begin{figure}
      \centering
      \includegraphics[width=0.6\linewidth]{./plots/numberSensors-WIFINET.png}
      \caption{Analysis of the variation of the number of sensors on the IoT/Network part energy consumption.}
      \label{fig:sensorsNumber}
    \end{figure}
    #+END_EXPORT


** Cloud Energy Consumption
   In this End-To-End energy consumption study, cloud account for a huge part of the overall energy
   consumption. According a report \cite{shehabi_united_2016-1} on United States data center energy
   usage, the average Power Usage Effectiveness (PUE) of an hyper-scale data center is 1.2. Thus, in
   our analysis, all energy measurement on cloud server will account for this PUE.

   In a first place, we analyze the impact of the VM allocated memory on the server energy
   consumption. Figure \ref{fig:vmSize} depict the server energy consumption according to the VM
   allocated memory for 20 sensors sending data every 10s. Note that horizontal red line represent
   the average energy consumption for the considered sample of energy values. We can see that at
   each sensing interval, server face to peaks of energy consumption. However, VM allocated memory
   do not influence energy consumption. In fact, simple database requests do not need any particular
   huge memory access and processing time. Thus, remaining experiments are based on VM with 1024MB
   of allocated memory.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \includegraphics[width=0.8\linewidth]{./plots/vmSize-cloud.png}
     \caption{VM size impact on the server energy consumption using 20 sensors sending data every 10s}
     \label{fig:vmSize}
   \end{figure}
   #+END_EXPORT

   Next, we study the effects of increasing the number of sensors on the server energy consumption.
   Figure \ref{fig:sensorsNumber-server} present the results of the average server energy
   consumption when varying the number of sensors from 20 to 500 while Figure
   \ref{fig:sensorsNumber-WPS} present the average server energy cost per sensors according to the
   number of sensors. These results show a clear linear relation between the number of sensors and
   the server energy consumption. Moreover, we can see that the more sensors we have per server, the
   more energy we can save. In fact, since the idle server energy consumption is high, it is more
   energy efficient to maximize the number of sensors per server. As shown on Figure
   \ref{fig:sensorsNumber-WPS}, a significant amount of energy can be save when passing from 20 to
   300 sensors per server.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \subfigure[Average server energy consumption]{
       \includegraphics[width=0.4\linewidth]{./plots/sensorsNumberLine-cloud.png}
       \label{fig:sensorsNumber-server}
     }
     \hspace{0.5cm}
     \subfigure[Average sensors energy cost on server]{
       \includegraphics[width=0.4\linewidth]{./plots/WPS-cloud.png}
       \label{fig:sensorsNumber-WPS}
     }
     \caption{Server energy consumption for sensors sending data every 10s}
     \label{fig:sensorsNumber-cloud}
   \end{figure}
   #+END_EXPORT

   A last parameter can leverage server energy consumption namely sensors send interval. In addition
   to increasing the application accuracy, sensors send interval increase network traffic and
   database accesses. Figure \ref{fig:sensorsFrequency} present the impact on the server energy
   consumption of changing the send interval of 50 sensors to 1s, 10s and 30s. We can see that, the
   lower sensors send interval is, the more server energy consumption peaks occurs. Therefore, it
   leads to an increase of the server energy consumption.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \includegraphics[scale=0.5]{plots/sendInterval-cloud.png}
     \label{fig:sensorsFrequency}
     \caption{Server energy consumption for 50 sensors sending request at different interval.}
   \end{figure}
   #+END_EXPORT
   
** End-To-End Consumption
   
   To have an overview of the energy consume by the system, it is important to consider the
   end-to-end energy consumption. The Figure \ref{fig:end-to-end} represents the end-to-end system
   energy consumption while varying the number of sensors. It is important to see that, for
   small-scale systems, the server energy consumption is dominant face to the energy consumed by the
   sensors. However, since we are using a single server, large-scale sensors deployment lead to an
   increasing consumption of energy in the IoT part. On the other side, network energy consumption
   is stable regarding to the number of sensors since the system use case do not required large data
   transfer. Thus, it is important to remember that, to save energy, we should maximize the number
   of sensors handle by each cloud server while keeping reasonable sensors request intervals.

   #+BEGIN_EXPORT latex
   \begin{figure}
     \centering
     \hspace{1cm}
     \includegraphics[scale=0.3]{plots/final.png}
     \label{fig:end-to-end}
     \caption{End-to-end network energy consumption using sensors interval of 10s}
   \end{figure}
   #+END_EXPORT
   


* Discussion 
#+LaTeX: \label{sec:discuss}

* Conclusion 
#+LaTeX: \label{sec:cl}

\bibliographystyle{IEEEtran}
\bibliography{references}


* Data Provenance :noexport:
  :PROPERTIES:
  :header-args: :eval never-export
  :END:
** Data Analysis (R Scripts)
*** Utils
**** R
      RUtils is intended to load logs (data.csv) and providing
      simple plot function for them.

      #+NAME: RUtils
      #+BEGIN_SRC R  :eval never
        library("tidyverse")

        # Fell free to update the following
        labels=c(time="Time (s)",sensorsSendInterval="Sensors Send Interval (s)", sensorsNumber="Number of sensors")
        PUE=1.2
        ns3SimTime=1800
        g5kSimTime=300

        loadData=function(path){
          data=read_csv(path)
          if("sensorsEnergy"%in%colnames(data)){ # If it is ns3 logs
            data=data%>%mutate(sensorsEnergy=sensorsEnergy/ns3SimTime)  # Convert to watts
            data=data%>%mutate(networkEnergy=networkEnergy/ns3SimTime)
            data=data%>%mutate(networkEnergy=networkEnergy+getSwitchesIDLE(sensorsNumber,sensorsSendInterval)) # Add Idle conso of switches
            data=data%>%mutate(totalEnergy=totalEnergy/ns3SimTime)
          }
          else{ # Log from g5k
            data=data%>%mutate(energy=energy*PUE) # Take into account PUE
            data=data%>%filter(time<=g5kSimTime) # Remove extrats values (theorical sim time != real sim time)
          }
        }


        getSwitchesIDLE=function(nbSensors, sendInterval){
          pktSize=192
          nEdgeRouter=8
          nCoreRouter=1

          EdgeIdle=4095
          EdgeMax=4550
          EdgeTraffic=560*10^9

          CoreIdle=11070
          CoreMax=12300
          CoreTraffic=4480*10^9

          # Apply 0.6 factor
          EdgeTraffic=EdgeTraffic*0.6
          CoreTraffic=CoreTraffic*0.6

          totalTraffic=pktSize/sendInterval*nbSensors

          EdgeConso=EdgeIdle*(totalTraffic/EdgeTraffic)
          CoreConso=CoreIdle*(totalTraffic/CoreTraffic)

          return(EdgeConso+CoreConso)
        }

                # Get label according to varName
        getLabel=function(varName){
          if(is.na(labels[varName])){
            return(varName)
          }
          return(labels[varName])
        }

        applyTheme=function(plot,...){
          palette<- c("#00AFBB", "#E7B800", "#FC4E07","#0abb00")
          plot=plot+
            theme_bw(...)+
              scale_fill_manual(values=palette)+
                scale_colour_manual(values=palette)
          return(plot)
        }

      #+END_SRC



**** Bash
***** Plots -> PDF
     Merge all plots in plots/ folder into a pdf file.
     #+NAME: plotToPDF
     #+BEGIN_SRC bash :results output :noweb yes
       orgFile="plots/plots.org"
       <<singleRun>> # To get all default arguments

       # Write helper function
       function write {
           echo "$1" >> $orgFile
       }

       echo "#+TITLE: Analysis" > $orgFile
       write "#+LATEX_HEADER: \usepackage{fullpage}"
       write "#+OPTIONS: toc:nil"
       # Default arguments
       write '\begin{center}'
       write '\begin{tabular}{lr}'
       write 'Parameters & Values\\'
       write '\hline'
       write "sensorsPktSize & ${sensorsPktSize} bytes\\\\"
       write "sensorsSendInterval & ${sensorsSendInterval}s\\\\"
       write "sensorsNumber & ${sensorsNumber}\\\\"
       write "nbHop & ${nbHop}\\\\"
       write "linksBandwidth & ${linksBandwidth}Mbps\\\\"
       write "linksLatency & ${linksLatency}ms\\\\"
       write '\end{tabular}'
       write '\newline'
       write '\end{center}'

       for plot in $(find plots/ -type f -name "*.png")
       do
           write "\includegraphics[width=0.5\linewidth]{$(basename ${plot})}"
       done

       # Export to pdf
       emacs $orgFile --batch -f org-latex-export-to-pdf --kill  
     #+END_SRC

***** CSVs -> data.csv (G5K)
     Merge all energy file into one (and add additional fields).

     #+NAME: G5K-mergeCSV
     #+BEGIN_SRC sh
       #!/bin/bash

       whichLog="last"

       logsLocation="logs/g5k"
       whichLog="${logsLocation}/${whichLog}"



       logFile="$(dirname $(readlink -f $0))"/$whichLog/simLogs.txt
       dataFile=$(dirname "$logFile")/data.csv


       getValue () {
           line=$(echo "$1" | grep "Simulation para"|sed "s/Simulation parameters: //g")
           key=$2
           echo "$line"|awk 'BEGIN{RS=" ";FS=":"}"'$key'"==$1{gsub("\n","",$0);print $2}'
       }

       ##### Add extract info to energy #####
       IFS=$'\n'
       for cmd in $(cat $logFile|grep "Simulation parameters")
       do
           nodeName=$(getValue $cmd serverNodeName)
           from=$(getValue $cmd simStart)
           to=$(getValue $cmd simEnd)
           vmSize=$(getValue $cmd vmSize)
           nbSensors=$(getValue $cmd nbSensors)
           simKey=$(getValue $cmd simKey)
           sendInterval=$(getValue $cmd sensorsSendInterval)
           csvFile="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}.csv"
           csvFileIDLE="$whichLog/${simKey}_${vmSize}VMSIZE_${nbSensors}NBSENSORS_${from}${to}_IDLE.csv"
           tmpFile=${csvFile}_tmp
           echo ts,energy,simKey,vmSize,nbSensors,time,state,sendInterval > $tmpFile
           minTs=$(tail -n+2 $csvFile|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
           minTsIDLE=$(tail -n+2 $csvFileIDLE|awk -F"," 'BEGIN{min=0}$1<min||min==0{min=$1}END{print(min)}') # To compute ts field
           tail -n+2 ${csvFile} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTs'",sim,"'$sendInterval'}' >> $tmpFile
           tail -n+2 ${csvFileIDLE} | awk -F"," '{print $0",'$simKey','$vmSize','$nbSensors',"$1-'$minTsIDLE'",IDLE,"'$sendInterval'}' >> $tmpFile
       done


       ##### Fill dataFile #####
       echo ts,energy,simKey,vmSize,nbSensors,time,state,sendInterval > $dataFile
       for tmpFile in $(find ${whichLog}/*_tmp -type f)
       do
           tail -n+2 $tmpFile >> $dataFile
           rm $tmpFile # Pay attention to this line :D
       done
     #+END_SRC

     #+RESULTS: G5K-mergeCSV

     #+RESULTS: mergeCSV



***** Log -> data.csv (ns3)
      logToCSV extract usefull data from logs and put them into logs/data.csv.

      #+NAME: NS3-logToCSV
      #+BEGIN_SRC bash :results output
        logsFolder="./logs/ns3/last/"
        csvOutput="$logsFolder/data.csv"

        # First save csv header line
        aLog=$(find $logsFolder -type f -name "*.org"|head -n 1)
        metrics=$(cat $aLog|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
        echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[1]);if(i<NF)printf(",");else{print("")}}}' > $csvOutput

        # Second save all values
        for logFile in $(find $logsFolder -type f -name "*.org")
        do
            metrics=$(cat $logFile|grep "\-METRICSLINE\-"|sed "s/-METRICSLINE-//g")
            echo $metrics | awk '{for(i=1;i<=NF;i++){split($i,elem,":");printf(elem[2]);if(i<NF)printf(",");else{print("")}}}' >> $csvOutput
        done
      #+END_SRC

      #+RESULTS: NS3-logToCSV



   
   
*** Plot Scripts
**** Random R Scripts

    Table sensorsSendInterval~Sensors+NetEnergyconsumption
    #+BEGIN_SRC R :noweb yes :results output
      <<RUtils>>

      data=loadData("logs/ns3/last/data.csv")
      sensorsE=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,sensorsEnergy)%>%arrange(sensorsSendInterval)
      delay=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,avgDelay)%>%arrange(sensorsSendInterval)
      netE=data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==15) %>%select(sensorsSendInterval,networkEnergy)%>%arrange(sensorsSendInterval)
      formatData=right_join(sensorsE,netE)%>%right_join(delay)%>%filter(((sensorsSendInterval/10)%%2)!=0)

      print(t(formatData))

    #+END_SRC

    #+RESULTS:
    :                         [,1]     [,2]     [,3]     [,4]     [,5]
    : sensorsSendInterval 10.00000 30.00000 50.00000 70.00000 90.00000
    : sensorsEnergy       13.51794 13.51767 13.51767 13.51767 13.51761
    : networkEnergy       10.44178 10.44167 10.44161 10.44161 10.44161
    : avgDelay            17.81360  5.91265  3.53509  2.55086  1.93848




    Figure Sensors Position ~ Energy/Delay
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsPosition-delayenergy.png
      <<RUtils>>
      tr=11 # Offset to center delay plot
      data=loadData("logs/ns3/last/data.csv")

      data=data%>%filter(simKey=="SENSORSPOS",sensorsNumber==9)

      p=ggplot(data,aes(y=sensorsEnergy,x=positionSeed,color="Energy"))+xlab(getLabel("Sensors Position Seed"))+ylab(getLabel("Sensors Power Consumption (W)"))+
      geom_line()+geom_point()+geom_line(aes(y=(avgDelay-tr),color="Delay"))+geom_point(aes(y=(avgDelay-tr),color="Delay"))+expand_limits(y=c(0,15))+
      scale_y_continuous(sec.axis = sec_axis(~.+tr, name = "Application Delay (s)")) +
      guides(color=guide_legend(title="Curves"))

      p=applyTheme(p)      
      p=p+theme(axis.title.y.right = element_text(margin = margin(t = 0, r = -8, b = 0, l = 10)))



      ggsave("plots/sensorsPosition-delayenergy.png",dpi=80, width=4, height=3.2)
    #+END_SRC

    #+RESULTS:
    [[file:plots/sensorsPosition-delayenergy.png]]



       Watt per sensor on server
    #+BEGIN_SRC R :noweb yes :results output
      <<RUtils>>
      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()


      data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
      data=data%>%distinct(nbSensors,.keep_all=TRUE)
      data=data%>%mutate(WPS=(avgEnergy/nbSensors))

      print(data%>%select(WPS,nbSensors))
    #+END_SRC



     Impact of vm size
     #+BEGIN_SRC R :results  graphics :file plots/vmSizeBar-cloud.png
       library("tidyverse")
       PUE=1.2
       # Load data
       data=read_csv("./logs/g5k/last/data.csv")

       data=data%>%filter(state=="sim",simKey=="vmSize")%>%mutate(energy=PUE*energy)%>%filter(time<=300)

       data=data%>%group_by(vmSize)%>%mutate(energy=mean(energy))%>%slice(1L)%>%ungroup()
       data=data%>%mutate(vmSize=as.character(vmSize))

       ggplot(data) + geom_bar(aes(x=vmSize,y=energy),stat="identity")+expand_limits(y=c(75,100))+ylab("Server Power Consumption (W)")+
       xlab("Experiment Time (s)")+scale_y_log10()


       ggsave("plots/vmSizeBar-cloud.png",dpi=90,height=3,width=6)  
     #+END_SRC

     #+RESULTS:
     [[file:plots/vmSizeBar-cloud.png]]



     #+NAME: ssiNet
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-net.png
       <<NS3-RUtils>>
       # Load Data
        data=read_csv("logs/ns3/last/data.csv")

       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=networkEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("networkEnergy"))+
       geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-net.png",dpi=80)
     #+END_SRC


     Effect of the number of sensors on the application delay
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/delay-nbsensors.png
       <<NS3-RUtils>>
       # Load Data
        data=read_csv("logs/ns3/last/data.csv")

       data%>%filter(simKey=="NBSENSORS") %>% ggplot(aes(y=avgDelay,x=sensorsNumber))+xlab(getLabel("sensorsNumber"))+ylab(getLabel("avgDelay"))+
       geom_line()+labs(title="For 20 sensors")
       ggsave("plots/delay-nbsensors.png",dpi=80)
     #+END_SRC

     #+RESULTS:
     [[file:plots/delay-nbsensors.png]]


   
     #+NAME: ssiWifi
     #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsSendInterval-wifi.png
       <<NS3-RUtils>>
       data=read_csv("logs/ns3/last/data.csv")
       data%>%filter(simKey=="SENDINTERVAL",sensorsNumber==20) %>% ggplot(aes(x=sensorsSendInterval,y=sensorsEnergy))+xlab(getLabel("sensorsSendInterval"))+ylab(getLabel("sensorsEnergy"))+
       geom_line() +     geom_line()+labs(title="For 20 sensors")
       ggsave("plots/sensorsSendInterval-wifi.png",dpi=80)
     #+END_SRC


      #+BEGIN_SRC R :results graphics  :noweb yes  :file plots/plot-final.png :session *R*
        <<NS3-RUtils>>
        simTime=1800

        # Network
        data=read_csv("logs/ns3/last/data.csv")
        data=data%>%filter(simKey=="NBSENSORS")
        dataC5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
        dataC10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy/simTime) %>%select(energy,sensorsNumber)
        dataNet=rbind(dataC5,dataC10)%>%mutate(type="Network")

        # Network
        dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
        dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy/simTime) %>%select(energy,sensorsNumber) 
        dataS=rbind(dataS5,dataS10)%>%mutate(type="Sensors")
      
        data=rbind(dataNet,dataS)%>%mutate(sensorsNumber=as.character(sensorsNumber))

        ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="Part"))
        ggsave("plots/plot-final.png",dpi=80)

      #+END_SRC
    
**** Plot In Paper

     Power sensors vs network
     #+BEGIN_SRC R :noweb yes :results graphics  :file plots/numberSensors-WIFINET.png :session *R*
       <<RUtils>>

       data=loadData("logs/ns3/last/data.csv")
       data=data%>%filter(simKey=="NBSENSORS")
       dataW=data%>%mutate(energy=sensorsEnergy)%>% mutate(type="Sensors") %>% select(sensorsNumber,energy,type) 
       dataN=data%>%mutate(energy=networkEnergy)%>% mutate(type="Network") %>% select(sensorsNumber,energy,type)

       data=rbind(dataN,dataW)
       data=data%>%mutate(sensorsNumber=as.character(sensorsNumber))
       data=data%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))
       data=data%>%filter(sensorsNumber%in%c(2,4,6,8,10))

       p=ggplot(data)+geom_bar(aes(x=sensorsNumber,y=energy,fill=type),position="dodge",stat="identity")+
       xlab(getLabel("sensorsNumber"))+ ylab("Power Consumption (W)") + guides(fill=guide_legend(title=""))
       p=applyTheme(p)+theme(text = element_text(size=15))

       size=5
      ggsave("plots/numberSensors-WIFINET.png",dpi=90,width=size,height=size-1)
     #+END_SRC

     #+RESULTS:
     [[file:plots/numberSensors-WIFINET.png]]


    Final plot: Energy cloud, network and sensors
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/final.png
      <<RUtils>>

      # Linear Approx
      approx=function(data1, data2,nbSensors){
        x1=data1$sensorsNumber
        y1=data1$energy

        x2=data2$sensorsNumber
        y2=data2$energy

        a=((y2-y1)/(x2-x1))
        b=y1-a*x1

        return(a*nbSensors+b)

      }


      # Load data
      data=loadData("./logs/g5k/last/data.csv")
      data=data%>%filter(state=="sim",simKey=="nbSensors")

      # Cloud
      data20=data%>%filter(nbSensors==20)%>%mutate(energy=mean(energy)) %>% slice(1L)
      data100=data%>%filter(nbSensors==100)%>%mutate(energy=mean(energy)) %>% slice(1L)
      data300=data%>%filter(nbSensors==300)%>%mutate(energy=mean(energy)) %>% slice(1L)
      dataCloud=rbind(data20,data100,data300)%>%mutate(sensorsNumber=nbSensors)%>%mutate(type="Cloud")%>%select(sensorsNumber,energy,type)


      # Network
      data=loadData("./logs/ns3/last/data.csv")
      data=data%>%filter(simKey=="NBSENSORS")
      dataN5=data%>%filter(sensorsNumber==5)%>% mutate(energy=networkEnergy) %>%select(energy,sensorsNumber)
      dataN10=data%>%filter(sensorsNumber==10)%>%mutate(energy=networkEnergy) %>%select(energy,sensorsNumber)
      dataNet=rbind(dataN5,dataN10)
      fakeNet=tibble(sensorsNumber=c(20,100,300))
      fakeNet=fakeNet%>%mutate(energy=approx(dataN5,dataN10,sensorsNumber),type="Network")

      # Sensors
      dataS5=data%>%filter(sensorsNumber==5)%>% mutate(energy=sensorsEnergy) %>%select(energy,sensorsNumber) 
      dataS10=data%>%filter(sensorsNumber==10)%>%mutate(energy=sensorsEnergy) %>%select(energy,sensorsNumber) 
      dataS=rbind(dataS5,dataS10)
      fakeS=tibble(sensorsNumber=c(20,100,300))
      fakeS=fakeNet%>%mutate(energy=approx(dataS5,dataS10,sensorsNumber),type="Sensors")

      # Combine Net/Sensors/Cloud and order factors
      fakeData=rbind(fakeNet,fakeS,dataCloud)
      fakeData=fakeData%>%mutate(sensorsNumber=as.character(sensorsNumber))
      fakeData=fakeData%>%mutate(sensorsNumber=fct_reorder(sensorsNumber,as.numeric(sensorsNumber)))
      fakeData$type=factor(fakeData$type,ordered=TRUE,levels=c("Sensors","Network","Cloud"))

      # Plot
      p=ggplot(fakeData)+geom_bar(position="dodge2",colour="black",aes(x=sensorsNumber,y=energy,fill=type),stat="identity")+ 
      xlab("Sensors Number")+ylab("Power Consumption (W)")+guides(fill=guide_legend(title="System Part"))
      p=applyTheme(p)+theme(text = element_text(size=16))
      ggsave("plots/final.png",dpi=90,width=8,height=5.5)

    #+END_SRC

    #+RESULTS:
    [[file:plots/final.png]]


    Impact of vm size
    #+BEGIN_SRC R :noweb yes :results graphics :noweb yes :file plots/vmSize-cloud.png
      <<RUtils>>

      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="vmSize")%>%filter(time<=300)

      data=data%>%mutate(vmSize=paste0(vmSize," MB"))
      data=data%>%group_by(vmSize)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
      p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~vmSize)+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)+expand_limits(y=c(0,40))+ylab("Server Power Consumption (W)")+
      xlab("Experiment Time (s)")

      p=applyTheme(p)

      ggsave("plots/vmSize-cloud.png",dpi=90,height=3,width=6)  
    #+END_SRC

    #+RESULTS:
    [[file:plots/vmSize-cloud.png]]


    Impact of sensors number
    #+BEGIN_SRC R :noweb yes :results  graphics :file plots/sensorsNumber-cloud.png
      <<RUtils>>
      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()

      data=data%>%mutate(nbSensorsSort=nbSensors)
      data=data%>%mutate(nbSensors=paste0(nbSensors," Sensors"))
      data$nbSensors=fct_reorder(data$nbSensors, data$nbSensorsSort)

      data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
      p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~nbSensors)+expand_limits(y=c(0,40))+ylab("Server Power Consumption (W)")+
      xlab("Experiment Time (s)")+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)

      p=applyTheme(p)
      ggsave("plots/sensorsNumber-cloud.png",dpi=90,height=3,width=6)  
    #+END_SRC

    #+RESULTS:
    [[file:plots/sensorsNumber-cloud.png]]
  
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/sensorsNumberLine-cloud.png :session *R:2*
          <<RUtils>>
          # Load data
          data=loadData("./logs/g5k/last/data.csv")

          data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()


          data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
          data=data%>%distinct(nbSensors,.keep_all=TRUE)
          data=data%>%mutate(WPS=(avgEnergy/nbSensors))

          p=ggplot(data,aes(x=nbSensors, y=avgEnergy)) + geom_point() +geom_line()+
          xlab(getLabel("sensorsNumber"))+ylab("Average server power consumption (W)")

          p=applyTheme(p)+theme(text = element_text(size=14))+ expand_limits(y=108)
          ggsave("plots/sensorsNumberLine-cloud.png",dpi=90,height=4.5,width=4)
    #+END_SRC

    #+RESULTS:
    [[file:plots/sensorsNumberLine-cloud.png]]
  
    #+BEGIN_SRC R :noweb yes :results graphics :file plots/WPS-cloud.png :session *R:2*
            <<RUtils>>
            # Load data
            data=loadData("./logs/g5k/last/data.csv")

            data=data%>%filter(state=="sim",simKey=="nbSensors")%>%ungroup()


            data=data%>%group_by(nbSensors)%>%mutate(avgEnergy=mean(energy))%>%distinct()%>%ungroup()
            data=data%>%distinct(nbSensors,.keep_all=TRUE)
            data=data%>%mutate(WPS=(avgEnergy/nbSensors))

            oldNb=data$nbSensors
            data=data%>%mutate(nbSensors=as.character(nbSensors))
            data$nbSensors=fct_reorder(data$nbSensors,oldNb)
            p=ggplot(data,aes(x=nbSensors, y=WPS)) + geom_bar(stat="identity")+
            xlab(getLabel("sensorsNumber"))+ylab("Server power cost per sensors (W)")

            p=applyTheme(p)+theme(text = element_text(size=14))+        theme(axis.title.y = element_text(margin = margin(t = 0, r = 8, b = 0, l = 0)))
            ggsave("plots/WPS-cloud.png",dpi=90,height=4,width=4)
    #+END_SRC

    #+RESULTS:
    [[file:plots/WPS-cloud.png]]

    #+BEGIN_SRC R :noweb yes :results  graphics :file plots/sendInterval-cloud.png
      <<RUtils>>
      # Load data
      data=loadData("./logs/g5k/last/data.csv")

      data=data%>%filter(state=="sim",simKey=="sendInterval")%>%ungroup()

      oldSendInterval=data$sendInterval
      data=data%>%mutate(sendInterval=paste0(sendInterval,"s"))
      data$sendInterval=fct_reorder(data$sendInterval,oldSendInterval)

      data=data%>%group_by(sendInterval)%>%mutate(avgEnergy=mean(energy))%>%ungroup()
          print(data)
      p=ggplot(data,aes(x=time, y=energy)) + geom_line()+facet_wrap(~sendInterval)+expand_limits(y=c(0,40))+ylab("Server power consumption (W)")+
      xlab("Experiment Time (s)")+geom_hline(aes(yintercept=avgEnergy),color="Red",size=1.0)

      p=applyTheme(p)
      ggsave("plots/sendInterval-cloud.png",dpi=120,height=3,width=6)  
    #+END_SRC

    #+RESULTS:
    [[file:plots/sendInterval-cloud.png]]

  
* Emacs settings :noexport:
  # Local Variables:
  # eval:    (unless (boundp 'org-latex-classes) (setq org-latex-classes nil))
  # eval:    (add-to-list 'org-latex-classes
  #                       '("llncs" "\\documentclass[conference]{llncs}\n \[NO-DEFAULT-PACKAGES]\n \[EXTRA]\n"  ("\\section{%s}" . "\\section*{%s}") ("\\subsection{%s}" . "\\subsection*{%s}")                       ("\\subsubsection{%s}" . "\\subsubsection*{%s}")                       ("\\paragraph{%s}" . "\\paragraph*{%s}")                       ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))
  # End: