@ -10,7 +10,7 @@ The base topology, as well as the specific components used is shown in Figure \r
\end{figure}
The source code of the \textit{ESPer} project is published into a \textit{Git} source code repository.
From there, the continuous integration system is responsible for automatically building and publishing the firmware binary files, as soon as updated source code is available.
From there, the continuous integration system is responsible for automatically building and publishing the firmware image files, as soon as updated source code is available.
It is also in charge of assembling and publishing meta-information (version and signature) required for the update process.
Both systems are described in detail in the following section.
Updates to the devices firmware is either triggered actively (manual or by the CI system) or on a regular schedule by the devices themselves.
@ -35,7 +35,7 @@ The according device type is provided as a string through a global constant at c
Device specific code is organized in a sub-folder for each device type.
To build the software, a \textit{Makefile}\cite{make} is used, which provides a simple way for reproducible builds.
Whenever a new build process is started, the build system scans for all device specific folders and calls the original build process for each of them.
After the build of the firmware has finished, the build system also creates a file for each device type, containing the build version and cryptographic signatures of the corresponding binary images.
After the build of the firmware has finished, the build system also creates a file for each device type, containing the build version and cryptographic signatures of the corresponding firmware images.
To avoid interferences between different build environments, and to roll out new versions as quickly as possible, the code has been integrated into a continuous integration (CI) system.
@ -50,7 +50,7 @@ The memory mapping mechanism of the MCU allows only a single page of 1 MB of fla
\label{fig:memory_layout}
\end{figure}
As the binary to download and flash possibly exceeds the size of free memory heap space, the received data must be written to flash directly.
As the image to download and flash possibly exceeds the size of free memory heap space, the received data must be written to flash directly.
In contrast, executing the code from the memory mapped flash while writing the same area with the downloaded update leads to unexpected behavior, as the executed code changes immediately to the updated one.
To avoid this, we chose the memory layout shown in Figure~\ref{fig:memory_layout}.
The flash is split into half to contain two firmware ROMs with different versions, one being executed and one which is being downloaded.
@ -64,11 +64,11 @@ In addition to the two firmware ROMs, the flash provides room for the bootloader
To ensure only valid firmware is running on the devices, a cryptographic signature of the firmware images is calculated and checked as part of the update process.
For the calculation of the signatures, the \textit{sha256} hashing algorithm \cite{RFC6234} and an elliptic curve cipher based on \textit{Curve25519}\cite{bernstein2006curve25519} are used, which are both considered modern and secure methods for software signing (see \cite{barker2016nist, bsi}).
The cryptographic signature for each of the two firmware ROMs is created by the continuous integration system during build time and is provided as meta-information along with the firmware binaries.
The cryptographic signature for each of the two firmware ROMs is created by the continuous integration system during build time and is provided as meta-information along with the firmware images.
Therefore, the CI system must be equipped with the private key used to create the signatures.
In contrast, the micro controller only needs to know the according public key, to be able to verify the cryptographic signature.
For the same reasons as stated in Section \ref{flashlayout}, the signature of the new firmware binary can not be verified before it is written to flash.
For the same reasons as stated in Section \ref{flashlayout}, the signature of the new firmware image can not be verified before it is written to flash.
Therefore, the calculation of the sha256 checksum required for the signature is done while the update is downloaded and written to flash.
After the download has succeeded, the signature is verified and the bootloader only gets reconfigured if the signature is validated successfully.
Otherwise, the bootloader will not be reconfigured and the system will not start the invalid firmware.
For deployment, only the master branch is considered.
After a successful build, all distribution files (the binary firmware and meta-data files) of all device types are copied to the repository server, where they are served by a HTTP server.
After a successful build, all distribution files (the firmware image and meta-information files) of all device types are copied to the repository server, where they are served by a HTTP server.
The configuration file (\texttt{Configurion.mk.maglab}) references this repository server as the source for updates.
Support for multiple devices of different type is implemented in both, the \textit{ESPer} framework itself and the build system.
@ -43,7 +43,7 @@ For this, a simple interface is specified by the framework, which must be implem
A single function \texttt{Device* getDevice()} must be defined exactly once in each device specific folder.
To implement this interface, a static instance of \texttt{Device} is created and returned.
Each \texttt{Device} is populated with device specific \texttt{Feature} instances.
While the \texttt{Feature}-API leverages common runtime polymorphism to share functionality between features, the initial \texttt{Device} creation uses compile-time polymorphism, which reduces the need for memory management and increases performance by avoiding virtual function tables.
While the \texttt{Feature}-API leverages common runtime polymorphism to share functionality between features, the initial \texttt{Device} creation uses compiletime polymorphism, which reduces the need for memory management and increases performance by avoiding virtual function tables.
Listing~\ref{lst:create_device_socket} shows the complete device specific code used for a simple power socket, which is mainly confined to the device type and its capabilities (i.e., the number of GPIO pins).
\begin{lstlisting}[caption={Device specific code for a socket driver.},
@ -67,12 +67,12 @@ The first one is a helper \textit{Makefile} built to accept a parameter for devi
In addition, the primary \textit{Makefile} scans a project subdirectory and uses each directory in there as a container for device specific code.
For each of these directories, the helper \textit{Makefile} is called and the subdirectories name is used as the value of the \texttt{DEVICE} parameter.
By splitting the build and recompiling the framework each time before intermixing it with the device specific code, the device type identifier can be used inside the shared framework code.
While building a devices firmware, the meta-information file used during updates is also created and stored beside the binary firmware image.
While building a devices firmware, the meta-information file used during updates is also created and stored beside the firmware image.
For development, each device can be build separately by using the device type identifier as \textit{Makefile} target.
In addition, the suffix \texttt{/flash} can be used to flash a specific firmware to the device.
While building the firmware binaries for a device, the build environment provides some constants which are baked into the resulting firmware image.
Beside the environmental configuration like the \textit{WiFi} credentials, \textit{MQTT} topics and other configurable tweaks, the current device and version identifiers are provided as compile-time constants.
While building the firmware images for a device, the build environment provides some constants which are baked into the resulting firmware image.
Beside the environmental configuration like the \textit{WiFi} credentials, \textit{MQTT} topics and other configurable tweaks, the current device and version identifiers are provided as compiletime constants.
In addition, the public key used to verify firmware signatures during updates is derived from the private key and provided as an linker object which is linked into each firmware image (~\ref{lst:public_key_object}).
This allows to use all the information inside the code without any overhead while being configurable during build time.
@ -8,12 +8,12 @@ Skipping the option results in the exclusion of the code for update management d
\subsubsection{Checking for updates}
Each device queries the update server regularly for the current firmware version and initializes the update process if remote and local versions differ.
To do so, the update server provides a file for each device type containing meta-information about the available firmware which consists of the version identifier and the signatures of both of the firmware binaries.
To do so, the update server provides a file for each device type containing meta-information about the available firmware which consists of the version identifier and the signatures of both of the firmware images.
These meta-information files are provided by the update server using \textit{HTTP 1.1}\cite{HTTP_1.1} under the following path pattern: \texttt{\$\{DEVICE\}.version} (whereas \texttt{\$\{DEVICE\}} is the device type name).
The meta-information file has a very simple line oriented, ASCII based format which is easy to generate and efficient to pars in the limited constrains of the embedded device.
The first line is the version identifier which can be an arbitrary string as the content is not interpreted semantically but only compared to the version identifier used during build time.
The other two remaining lines is the hexadecimal representation of the firmware signatures, one line for each firmware.
The other two remaining lines is the hexadecimal representation of the firmware signatures, one line for each firmware image.
Each device tries to fetch the meta-information file once every hour.
After the file has been downloaded successfully, the version identifier is extracted and compared to the version identifier provided during build time.
@ -27,9 +27,9 @@ This allows faster roll-outs of updates and finer control for manual maintenance
As the \textit{ESP-01s} is only equipped with 1 MB of flash, this means that the whole memory is mapped to a contiguous address space.
Therefore, the second ROM can not be re-mapped to have the same start address as the first ROM.
While the firmware is executed without any dynamic linking mechanism and the chip does not support position independent code, the addresses used in the ROMs are dependent to the offset at which the firmware is stored.
This arises the need for building two firmware binaries, one for each target location.
This arises the need for building two firmware images, one for each target location.
To do so, a linker script for each of the two ROM slots was created, which is used to create two variations of the same firmware, only differing in ROM placement.
The two resulting firmware binary files are both provided for download via \textit{HTTP 1.1} - which one to download depends on the target ROM slot and is selected by the device during the update process.
The two resulting firmware image files are both provided for download via \textit{HTTP 1.1} - which one to download depends on the target ROM slot and is selected by the device during the update process.
Listing~\ref{lst:linker_script} shows the only difference between the two linker scripts, where \texttt{\$\{SLOT\}} must be replaced with the slot number according to the current build.
\begin{lstlisting}[language=,
@ -74,9 +74,9 @@ if (bootconf.current_rom == 0) {
The firmware files provided on the update server are the exact same ones as used to initially flash the chip for the according version.
Using the same files for flashing and updating allows better debugging by eliminating errors related to the update process itself and makes development and initial installation very easy.
Listing~\ref{lst:choosing_rom} shows the algorithm used to determine the download address and reconfigure the bootloader.
The update server provides these files in the exact same way as it provides the meta-information files, but the path pattern differs: the suffixes \texttt{.rom0} and \texttt{.rom1} are used to provide the firmware binary files for the first and second slot respectively.
The update server provides these files in the exact same way as it provides the meta-information files, but the path pattern differs: the suffixes \texttt{.rom0} and \texttt{.rom1} are used to provide the firmware image files for the first and second slot respectively.
For installing a firmware update, the new firmware binary file is downloaded using an HTTP GET request.
For installing a firmware update, the new firmware image file is downloaded using an HTTP GET request.
While the image is being downloaded each chunk received in the download stream is used to update the \textit{SHA256} hash before it is written to the flash.
When the write has been finished, the next chunk is received and the process continues until all chunks have been received.
Implementing \textit{OTA} updates under the given requirements involves multiple components, which interact closely.
The continuous integration system is in charge of building the firmware from source, calculating cryptographic signatures, and publishing the built binary images.
The deployment infrastructure provides resources for downloading the binary firmware images and triggering the update on all devices.
The continuous integration system is in charge of building the firmware from source, calculating cryptographic signatures, and publishing the built firmware images.
The deployment infrastructure provides resources for downloading the firmware images and triggering the update on all devices.
Finally, the implementation of the update mechanism as a part of the firmware running on the embedded device is responsible for downloading and installing the updates.
