TinyContainer: Container Runtime Middleware Enabling Multi-tenant Microcontrollers with Built-in Security Bastien Buil
arXiv:2606.09225v1 [cs.OS] 8 Jun 2026
Orange Research Caen, France Cnam CEDRIC Paris, France [email protected]
Chrystel Gaber
Orange Research Caen, France [email protected]
Emmanuel Baccelli
Samuel Legouix
Orange Research Caen, France [email protected]
Samia Bouzefrane
Inria TRiBE Palaiseau, France [email protected]
Cnam CEDRIC Paris, France [email protected]
Abstract
Keywords
Software containerization technologies for resource-limited devices enable multi-tenant microcontrollers, which allow running multiple applications with different permission levels. However, current solutions lack run time configuration over various settings on container scheduling and container permissions to host resources. This limits the applicability of constrained containerization in dynamic and heterogeneous environments. This paper introduces TinyContainer, a lightweight software container management middleware designed for multi-tenant microcontrollers. TinyContainer provides per-container configurable scheduling and fine-grained access control to host resources through a metadata-driven approach, supporting multiple runtimes via a runtime abstraction layer. We analyze the performance of TinyContainer with a small WebAssembly runtime, CS4WAMR, and RIOT OS, a common RTOS. We report on experiments using popular IoT boards based on various Cortex-M microcontrollers. We show the endpoint system brought by TinyContainer allowing to regulate access of containers to host resources and provide host services to containers with an overhead of up to 4 ms per call. In particular, we showcase a TinyML use case, whereby containers retain data and model weights, while model inference is delegated to native host RTOS services.
Microcontrollers, WebAssembly, Containers, Cloud-to-IoT Continuum, IoT
CCS Concepts • Software and its engineering → Runtime environments; Embedded middleware; Virtual machines; Real-time systems software; • Computing methodologies → Machine learning; • Security and privacy → Software and application security; • Computer systems organization → Embedded systems.
Please use nonacm option or ACM Engage class to enable CC licenses This work is licensed under a Creative Commons Attribution 4.0 International License. WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2201-1/2026/06 https://doi.org/10.1145/3765613.3811689
ACM Reference Format: Bastien Buil, Chrystel Gaber, Samuel Legouix, Emmanuel Baccelli, and Samia Bouzefrane. 2026. TinyContainer: Container Runtime Middleware Enabling Multi-tenant Microcontrollers with Built-in Security. In Proceedings of the 19th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec ’26), June 30-July 03, 2026, Saarbrücken, Germany. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3765613.3811689
Acknowledgments The research leading to these results partly received funding from the MESRI-BMBF German-French cybersecurity program under grant agreements no ANR-20-CYAL-0005 and 16KIS1395K. The research leading to these results is also funded by ANRT Convention Cifre n°2024/0426. Some work was also partially financed by France 2030 via the PEPR Future Networks project FITNESS, and the PTCC project PQ-OTA. ANRT, MESRI and BMBF are not responsible for any use that may be made of the information this paper contains. This paper reflects only the authors’ views.
1
Introduction
Recent works, such as Aerogel [22] or CS4WAMR [10], have focused on software containerization on microcontrollers by using small WebAssembly (Wasm) runtimes. Containerization enables multiple stakeholders to share a microcontroller – a scenario called multitenancy. The challenges of multi-tenancy on microcontrollers are multiple. The first challenge is fitting the extremely small resource constraints of microcontrollers. To grasp this challenge, readers are referred to RFC7228 [9] which describes the characteristics of low-end Internet-of-Things (IoT) devices: a Random Access Memory (RAM) budget smaller than 256 KiB, Flash memory smaller than 500 KiB, and CPU clock speeds in MHz. Another challenge on multi-tenant devices is isolating the different applications running on the device. Isolating memory between
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
applications is common in existing containerization runtimes and is built-in by design in WebAssembly. However, temporal isolation and management of access to host resources are not common and remains difficult [11]. Very recently, IETF began fostering work in this domain, as demonstrated by the last T2TRG workshop on Composable Code for Things [13]. Prior work described in [27] studies I2C and USB peripheral drivers sandboxed in different Wasm components cooperating on Cortex-M devices (Raspberry Pi Pico). Another example is KaOS [25], which introduces a concept to retrofit software using multiple Wasm components on legacy IoT sensor hardware. Aerogel [22] tackles permissions and energy management for WebAssembly on microcontrollers by proposing to modify WebAssembly Micro Runtime (WAMR) to allow configuring access to peripherals of the device, Aerogel does not allow dynamic loading of permissions with the container and does not allow to have devices with multiple configuration of peripherals. None of these approaches tackle fine-grained permissions to varied resource types and per-container scheduling on containerization runtimes. This paper thus proposes a container management framework which fills this gap. Challenges — In more detail, we focus on two major concerns regarding runtimes for multi-tenant microcontrollers. 1. Scheduling of containers. Different applications often require different scheduling priorities. For instance, some applications require more execution time than others. Many runtimes, such as Aerogel [22] or CS4WAMR [10], use the underlying OS scheduler to schedule the different containers. This restricts the capacity to have custom scheduling configurations for each runtime. 2. Access control for host resources. When a resource (e.g. a native function) is exposed to one container, it is not possible to limit its exposure from other containers running in the same environment. Therefore, deploying containers with different permissions requires modifying the firmware and flashing the new version on the device to modify the access rights to the resources. This restricts the possibility to deploy co-located containers with different trust levels. Contributions — In this paper, we introduce TinyContainer, a new container management framework targeting microcontrollers. More specifically: (1) we design TinyContainer, which provides container lifecycle management, container scheduling, and host resource access control. TinyContainer is designed to support different runtimes through a runtime abstraction, support multiple peripheral types with a driver system, and be configurable with lightweight metadata provided with the containers. (2) we implement TinyContainer on top of RIOT and apply it to hosting small WebAssembly modules using CS4WAMR. TinyContainer implementation is available as open-source code1 .
1 Available
under LGPL-2.1 license at https://github.com/TinyPART/RIOT/tree/ tinycontainer/sys/tinycontainer
Bastien Buil, Chrystel Gaber, Samuel Legouix, Emmanuel Baccelli, and Samia Bouzefrane
(3) we perform benchmarks on popular IoT boards (Arduino and Nordic) based on common microcontrollers (Arm Cortex-M4 and Cortex-M33). (4) our measurements show that TinyContainer introduces an acceptable overhead to add crucial security mechanisms. (5) we apply TinyContainer to a tinyML use case which showcases the capabilities of TinyContainer. Especially, we highlight the endpoint system of TinyContainer which allows containers to access resources outside the container through an interface, and enables hosts to provide services to containers, such as TinyML services.
2
Background
Containerization on microcontrollers. WebAssembly (Wasm) is a compiled binary format intended for secure and isolated execution within WebAssembly virtual machines. WebAssembly (Wasm) is designed to be compiled from different programming languages. The binary produced after compiling code to WebAssembly is called a module. WebAssembly can run on microcontrollers through lightweight runtimes such as WebAssembly Micro Runtime (WAMR) [1] and Wasm3 [4]. On microcontrollers, WebAssembly Micro Runtime (WAMR) supports two execution modes: interpreter mode and AoT (Ahead of Time) compilation mode. In interpreter mode, WAMR interprets the Wasm bytecode on the device, causing overhead. AoT compilation involves translating the Wasm bytecode into machine code specific to the target hardware and deploying the AoT code on the IoT device. AoT compilation mode is used to achieve performance close to native execution while keeping isolation guarantees of WebAssembly. Multi-tenant microcontrollers use cases. Buil et al. [10] highlight commercial solutions like MicroEJ and Atym which offer containerization for microcontrollers. Atym focuses on containers to simplify debugging and development, provide application isolation for security, and support containerized Edge AI. MicroEJ offers a multi-application runtime for sandboxed environments, promoting servitization to allow third-party service development, akin to an app store for microcontrollers. This enhances the user experience and enables device manufacturers to monetize their products with premium features. MicroEJ’s applications span smart homes, smart grids, industrial uses, and medical devices. Fine-grained access control and temporal isolation. While Wasmtime [2], one of the main runtimes for WebAssembly, is not adapted to the most constrained devices, other runtimes like WebAssembly Micro Runtime (WAMR) [1] and Wasm3 [4] are specialized to such targets. Several propositions modify WAMR to add new functionalities, such as Aerogel [22] which adds energy and permission management, and CS4WAMR [10] which adds permission management as well as segmentation and control of memory usage. Existing propositions do not manage fine-grained access permissions on custom host resources. For example, CS4WAMR uses hard-coded permissions and Aerogel permission system focuses only on peripherals, uses hard-coded drivers, and does not allow loading container permissions along with the container. Moreover, existing Wasm runtimes do not manage the scheduling of container.
TinyContainer: Container Runtime Middleware Enabling Multi-tenant Microcontrollers with Built-in Security
Container
Untrusted containers
WASM Container 1
WASM Container 2
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
TinyContainer Service
Endpoint Driver
Open endpoint: open(endpoint_id) Check permissions & get peer_id for endpoint
TinyContainer Framework
Crypto Metadata Parser Memory Manager
WAMR Runtime & Adaptor
WAMR Runtime & Adaptor
Get a new endpoint file descriptor
TinyContainer Runtime
TinyContainer Runtime
Open endpoint: open(peer_id) Open endpoint and get new driver file descriptor
Scheduling & Control
Return driver file descriptor Return endpoint file descriptor
TinyContainer API
Endpoints
Write to endpoint: write(endpoint_fd, data, len)
Container Loading Code
Get driver_fd with endpoint_fd Write to endpoint: write(driver_fd, data, len) Return nb_bytes_written
Figure 1: Overview of TinyContainer architecture Return nb_bytes_written
We propose to use TinyContainer, the contribution of this paper, to integrate fine-grained scheduling and permission management on WebAssembly by integrating CS4WAMR as one of the runtimes. We choose CS4WAMR as it is based on WAMR runtime and proposes memory segmentation between containers.
3
TinyContainer: secure management of containers on multi-tenant microcontrollers
TinyContainer is a container management framework designed for remote management of multi-tenant microcontrollers with built-in security through a per-container configurable scheduler and an endpoint and permission system. TinyContainer is designed to be runtime independent, through an abstraction of the underlying runtime. TinyContainer uses metadata to control container scheduling and permissions, and checks the integrity and authenticity of containers. Figure 1 provides a high-level view of the architecture. The Scheduling & Control component manages containers and exposes its API for container for adding endpoints and managing containers. The runtime abstraction is ensured by the adaptor design pattern through the Runtime Adaptor and the TinyContainer Runtime. For TinyContainer to support a new runtime environment, only the Runtime Adaptor component needs to be developed to take into account the specificities of the runtime to be added. The TinyContainer Runtime component manages the lifetime of containers. Finally, the Scheduling & Control component uses multiple small components, such as the Metadata Parser component to parse metadata file, the Crypto component to verify integrity and authenticity of both containers and metadata, and the memory manager component to store containers and metadata. While TinyContainer supports multiple runtimes like CS4WAMR [10], rBPF runtime from FemtoContainer paper [28], and JerryScript, only CS4WAMR runtime adaptor currently supports all the described features. For example, other runtime adaptors currently lack the implementation of some features such as the endpoint system. This is why the paper will mainly focus on the usage of the CS4WAMR WebAssembly runtime. Containers can be dynamically loaded in TinyContainer from a container code file, like a WebAssembly module, a metadata file, and
Container
TinyContainer Service
Endpoint Driver
Figure 2: Connection of endpoints from containers up to drivers optionally from a data file accessible to the container. To remotely load containers, a typical solution is to use a CoAP server, installed on the device, as a container loading code on which clients can send container code and metadata. The container loading code can then load containers using the TinyContainer API. The responsibility for ensuring the integrity and authenticity of the code and metadata can then be delegated to TinyContainer, which will verify them during container loading.
3.1
Access management for containers to diverse resource types
TinyContainer allows containers to access resources outside the container through an endpoint mechanism. The endpoints can be peripherals, remote services, or services provided by the host to the containers. To control permissions to access endpoints, each container is deployed with metadata declaring container characteristics and the different endpoints accessible by the container. Container access permissions to endpoints are loaded by parsing the metadata manifest when loading the container. The endpoint system works similarly to Unix input-output system calls. Containers have access to multiple functions to interact with endpoints. Containers can open a file descriptor to an endpoint using the open() function and specifying in argument the target endpoint. When opening the file descriptor, a check is done to verify whether the container has access to the endpoint. If the container does not have access to the endpoint, the opening of the file descriptor fails and the open() function returns -1. This allows containers to provide fallback mechanisms when not having access to the endpoints due to lacking permissions or lacking peripheral. After obtaining a file descriptor, containers can read and write data to the endpoint using the read() and write() functions, and finally close the descriptor using the close() function. A typical interaction with a driver is illustrated in Figure 2. Multiple identifiers, which are integers, are used by the endpoint system.
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
Bastien Buil, Chrystel Gaber, Samuel Legouix, Emmanuel Baccelli, and Samia Bouzefrane byte[]
container_id container
enum
runtime_type
CWT
syscall_mask
uint
endpoint_id peer
endpoint_0 endpoints
- undefined - native - shell - bpf
enum
uint uint
loop_call_period
Figure 3: TinyContainer container states machine
security
code_token
peer-uid
- container - local - remote
uint
- in - out - both
uint
timing data_token
enum
byte[]
peer-id
start_exec_limit
endpoint_n
peer-type
Only for remote type
direction
endpoint_1
- wasm - javacard-applet - javascript - python
loop_exec_limit CWT CWT
uint
loop_max_calls
uint
stop_exec_limit
CWT
metadata_token
The endpoint_id is the identifier of the endpoint used by containers to communicate with it. The peer_id is the identifier of the driver used by TinyContainer. The endpoint file descriptor (endpoint_fd) and the driver file descriptor (driver_fd) are respectively the identifier for the communication between containers and TinyContainer Service, and the identifier for the communication between TinyContainer Service and endpoint driver. The link between the endpoint and its endpoint_id used by the container, and the link between driver and its peer_id used by driver code are defined in the metadata. This allows container code to be independent of the attribution of the endpoint identifier by the host system. The host integrating TinyContainer manages the endpoints, defines the peer_id associated with the different endpoints and provides drivers which implements the open(), read(), write() and close() functions to provide access to host services and peripherals to containers.
3.2
Per-container configurable scheduling
A specificity of TinyContainer is its ability to abstract the runtime environment, enabling the utilization of identical APIs for initializing, loading, starting, stopping a container, or checking its status irrespective of the runtime in use on a specific IoT device. This feature facilitates a remote provisioning platform in managing numerous devices with varying runtimes remotely. As depicted in Figure 3, the container lifecycle is managed through three distinct entry points corresponding to ON_START, ON_LOOP, and ON_STOP states. A container runtime adaptor must implement the associated entry point functions (on_start(), on_loop(), and on_stop()) to execute container code for the lifecycle. For instance, in our CS4WAMR implementation, WebAssembly code must expose start(), loop(), and stop() functions, and CS4WAMR runtime adaptor entry points call the respective Wasm functions. TinyContainer incorporates a control mechanism to control container execution time by implementing a semi-cooperative scheduling system. The frequency at which the on_loop() function is called is determined by the container’s metadata, using the loop_call_period value. It is also possible to limit the number of calls to the on_loop() function using loop_max_calls value, after which the container is stopped. The maximum execution time is also specified in the container’s metadata, with the start_exec_limit, loop_exec_limit, and stop_exec_limit values, ensuring that no single container can monopolize system resources indefinitely. Before executing a container function, a watchdog is armed to be triggered after associated execution time limit. When a container does not respect its execution time limit, the watchdog gets triggered
Figure 4: Container Metadata
and the execution is returned to TinyContainer, which kills the container by stopping its associated thread. This parameter can be adjusted to balance the needs of long-running tasks with the overall responsiveness of the system. These configurations allow for fine-tuning the container’s execution pattern based on its specific requirements and the overall system load. With the timing parameters from the container’s metadata, TinyContainer provides a flexible framework that can accommodate a wide range of container behaviors while preserving system integrity.
3.3
Metadata for multi-tenant microcontrollers
To build metadata ready for transfer over low-power networks and with built-in security in the format, we use CBOR (Concise Binary Object Representation) and COSE (CBOR Object Signing and Encryption) as used by the SUIT manifest [23], a state-of-the-art manifest format for remote firmware deployment on an Internet of Things devices. CBOR is a data format similar to JSON but designed to have a minimum size for constrained devices and network exchange. COSE is a specification to create signatures of CBOR data. CBOR Web Token (CWT) [19] allows creating CBOR-based tokens containing both a COSE signature and the signed data. TinyContainer metadata uses the CBOR format to encode metadata with minimum overhead and CWT to sign permissions of the container (syscall_mask), the hash of WebAssembly container, the hash of container data, and the hash of the metadata itself. As depicted in Figure 4, the metadata consists of three sections: containers description, endpoints description, and security-related description. The container description contains a container identifier, the type of container (e.g. Wasm, rBPF) and a signed token which contains a byte map describing the list of endpoint calls authorized for this container. The endpoint description contains a list of endpoints characterized by an ID, a type, and an access direction, that lists which endpoint functions are available to be used by the current container. As illustrated with the fields "...", the structure is flexible enough to add an unfixed number of fields to describe the endpoints. The security-related description contains a detailed description of limits related to the container lifecycle, as well as three tokens to verify the authenticity and integrity of data, code, and metadata. The Container Metadata contains four CBOR Web Tokens (CWT), each generated by different roles which may be involved in the
TinyContainer: Container Runtime Middleware Enabling Multi-tenant Microcontrollers with Built-in Security
Untrusted
Time & memory isolation between containers
WASM Container 1
WASM Container 2
Container 1 Metadata
Container 2 Metadata
Isolation of host memory & resources from containers
TinyContainer Framework
Protection from modified and unauthentic containers Untrusted Container remote loader code actors API
Peripheral Drivers Other apps on the device
Trusted
Figure 5: TinyContainer security model container’s lifecycle. The software developer creates the code token, while a service provider can add data to customize the code supplied by the developer and create the associated data token. The deployment infrastructure manager is responsible for generating the metadata token. Lastly, the owner of the microcontrollers produces the syscall mask token to authorize the use of specific resources on its device by the container. Managing this collection of tokens falls under the responsibility of the container management platform and is beyond the scope of this paper. It should be noted that all these roles can be carried out by a single actor or by multiple actors.
4
Security considerations
Security definition. TinyContainer provides sandbox isolation, as defined by [20], which consists of reducing container permissions to protect the host system, and the other containers. Contrary to technologies like ARM Trustzone, Sancus, TinyContainer does not provide safebox isolation, which consists of reducing the privilege of the system to access a container. Thus, TinyContainer allows the execution of mutually distrusting containers, which entrust the host and TinyContainer to provide sufficient resources for their execution. Containers having different trust levels means that containers are not trusted equally to access resources of the system and therefore have different permissions to access the systems. TinyContainer security model is represented in Figure 5. Security hypotheses. To maintain these security guarantees, multiple security hypotheses must be respected. The runtime used must provide both fault and memory isolation. WebAssembly was explicitly designed with these isolation guarantees in mind. However, WebAssembly bytecode interpreters require the bytecode to be validated. While some runtimes, such as WAMR [1], perform the validation on-device, others, like Wasm3 [4], assume that validation has already been carried out, potentially off-device. Some runtimes, like WAMR[1] and Wasmtime [2], support ahead-of-time compilation. To preserve equivalent security properties, AoT requires that the Wasm-to-native-code AoT compilation be done correctly by a trusted party. TinyContainer security holds as long as the Wasm security guarantees are upheld. Thus, TinyContainer is compatible with both interpreted and AoT, but for interpreters relying on off-device validation and for runtimes
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
using AoT, TinyContainer requires that the trusted entity signing the metadata both validates the Wasm bytecode, and in AoT case, performs the AoT compilation. Another security requirement is the correct implementation of the endpoint to maintain isolation. For instance, in the following section, we implemented a TinyML driver which allows performing inference from data and model weights while preserving isolation between containers. Attacker model. Our attacker model assumes that an attacker has full control over the container code as long as the container passes the validation step. Attackers can load untrusted containers, which can, for instance, try to access peripherals of the device or try to monopolize execution time. An attacker can communicate with the device over the network, for instance to load containers on the device. Attackers do not have control over the metadata of their container, as metadata are defined by a trusted entity that manages the device. Attackers breaking the cryptographic primitives used in metadata, notably to deploy invalid containers, is out of scope of this paper. Attackers do not have physical access to the hardware. Other applications trying to access information of the containers is not protected by TinyContainer, and thus, is out of scope of this paper.
5
Application on TinyMLaaS use cases
TinyContainer notably enables the host to provide services to containers and manage access control to them with metadata. We propose to test this mechanism of host-provided services with a use case of Tiny Machine Learning as a Service (TinyMLaaS).
5.1
Tiny Machine Learning as a Service (TinyMLaaS)
Doyu et al. [15] introduce TinyMLaaS, a solution involving a Cloud or Edge platform that uses model compilers for efficient ML model compilation and deployment on microcontrollers. WASI-NN [7] is a proposal for WebAssembly System Interface (WASI) allowing WebAssembly modules to send TinyML models to the host and run them in machine learning runtimes such as TensorFlow Lite [14] and OpenVINO [6]. Buil et al. [10] propose an architecture of TinyMLaaS with a WebAssembly container by creating AoTcompiled WebAssembly containers from machine learning model files for running model inference.
5.2
TinyMLaaS endpoint use case
CS4WAMR article [10] highlights the high overhead of Wasm bytecode interpretation for compute-intensive tasks such as TinyML, which makes interpreters impracticable for such tasks or requires the use of Ahead-of-Time compilation which does not have such overhead. To address this issue, we propose an architecture where the model weights are in the container, but machine learning inference is delegated to the host. The mechanism uses a predefined model structure, so that model code is already being present in the host, but containers can customize inference by changing the weights. Our proposition works by having an endpoint dedicated to machine learning which can first receive model weights from the container,
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
Container
Bastien Buil, Chrystel Gaber, Samuel Legouix, Emmanuel Baccelli, and Samia Bouzefrane
TinyML Driver
open(tinyml_endpoint_id)
automatically integrated with the ML service code in the TinyML endpoint.
fd write(fd, [SETUP], 1) write(fd, [WRITE_MODEL, PARAM_NUMBER, OFFSET, DATA...], write_len) write(fd, [WRITE_MODEL, PARAM_NUMBER, OFFSET, DATA...], write_len) write(fd, [WRITE_INPUT, IO_NUMBER, OFFSET, DATA...], write_len) write(fd, [PREDICT], 1) write(fd, [READ_OUTPUT, IO_NUMBER, OFFSET], write_len) read(fd, &output_buffer, read_len) Container
TinyML Driver
Figure 6: TinyML endpoint usage
then receive the input data for prediction, compute the result, and finally let the container read the result. This architecture allows to leverage the computation optimizations of the devices, not available when doing bytecode interpretation, but without needing AoT optimization which requires a trusted third party to compile the WebAssembly module to machine code and without requiring having a full model runtime, like LiteRT (former TensorFlow Lite), which uses a lot of memory and storage. This architecture allows to leverage models using the same structure, such as models that can be fine-tuned and brought by the container, and models using a standardized structure. A typical usage of the TinyML endpoint involves the device host either defining a structure for accepted models, e.g. a multi-layer perceptron with a defined size for each layer, or furnishing a model that can be customized and fine-tuned by container providers.
5.3
Implementation of TinyMLaaS on TinyContainer
TinyML service has been implemented as an endpoint for TinyContainer. The TinyML endpoint includes the ML service code to load and communicate with the thread responsible for machine learning processing, as well as the machine learning code, specific to the model structure, used to compute predictions. Communication between the TinyML endpoint and the container. Containers can interact with the TinyML endpoint through the endpoint system offered by TinyContainer. The sequence diagram in Figure 6 illustrates the communication steps. To load the model, two steps are necessary after establishing the connection between the container and the TinyML endpoint. Initially, the container must transmit the model weights to the TinyML endpoint. This enables the TinyML endpoint to reconstruct the model based on the parameters and model structure. Once the model is loaded into the TinyML endpoint, the container can perform predictions by sending a vector and receiving the predicted label. Generation of TinyML endpoint code. TinyML endpoint contains the ML code generated from a TensorFlow Lite model, and the ML service code to interact with the model. The machine learning code is generated from RIOT-ML [18], a machine learning toolkit which integrates a model compiler, based on Apache-TVM [12], to generate machine code. Using a script, the code produced is then
6 Evaluation 6.1 Methodology The evaluation was conducted on the 2025.07 release of RIOT-OS. All WebAssembly modules were built using clang version 18.1.3 with optimizations, avoiding the static linking of libraries like libc or those related to WASI. To tailor these modules for the limited memory of microcontrollers, linker flags were utilized to minimize the size of Wasm modules by setting the linear memory stack size to 1024 bytes and the global base, which determines the starting address of data in linear memory, to 16 bytes. Additionally, the compilation and linking processes employed the optimization flags O3 and link-time optimization. The RIOT-ML 82b1b5c commit version and Apache TVM v0.18.0 were utilized. All timing measurements presented in this document were performed using the device’s hardware timer. The benchmarks were conducted on two different boards. The first board is the Arduino Nano 33 BLE (rev1), which features an nRF52840 CPU utilizing an ARM Cortex-M4, running at 64MHz, with 256 kilobytes of SRAM and 1 megabyte of flash memory. The second board is the nRF9160-DK, equipped with an nRF9160 CPU based on an ARM Cortex-M33, also clocked at 64 MHz, and includes 256 kilobytes of SRAM and 1 megabyte of flash.
6.2
Performance evaluation
In this section, we evaluate the performance and memory impact of the TinyContainer framework. We compare the code running WebAssembly Micro Runtime (WAMR) [1], CS4WAMR [10] and our TinyContainer proposition with CS4WAMR runtime. To conduct our evaluation, we have created WebAssembly containers to evaluate the impact on performances of TinyContainer. First, we have created a polling container that reads data from an endpoint at each loop call to measure endpoint access performance. Then, we have created friendly containers executing NOP instructions for a small period of time and a malicious container monopolizing execution. Finally, we define a running minimal runnable container which is the smallest possible container which can run on the runtime to evaluate memory usage. 6.2.1 Metadata and access control overhead. Table 1 summarizes the performance overhead to load a polling container and execute it with WAMR, CS4WAMR and TinyContainer. The polling container uses a data endpoint to read two buffers of 1960 bytes each. The polling container works by reading 255 bytes at a time from the endpoint, hence, polling the endpoint to read the two buffers. Loading a container with TinyContainer is highly impacted by the loading of the metadata. This is due to the introduction of several additional steps, namely parsing CBOR objects, retrieving the public key, verifying the validity of the CWTs present in the manifest using the EdDSA algorithm, extracting the hashes of the CWT claimsets, and verifying that the hashes correspond to the metadata, data, and code. In particular, on the nRF9160-DK, the implementation of RIOT-OS does not activate the hardware cryptographic accelerator,
TinyContainer: Container Runtime Middleware Enabling Multi-tenant Microcontrollers with Built-in Security
Table 1: Execution time of WebAssembly runtimes and framework for running a polling container
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
Table 2: ROM and RAM usage in bytes of WebAssembly runtimes and framework for running minimal runnable container on Arduino Nano 33 BLE
Execution time (in ms) Runtimes WAMR Interpreter Load container Container execution CS4WAMR Load container Container execution TinyContainer with CS4WAMR (without cryptography) Load container metadata Load container Container execution TinyContainer with CS4WAMR Load container metadata Load container Container execution
1.61 0.91
1.67 0.98
2.21 0.97
2.16 0.99
19.50 3.54 63.23
19.33 6.04 60.10
91.15 3.61 74.0
8 810.16 6.05 63.54
Friendly A start Friendly B start
With WAMR
For 1 container
Per additional container
ROM
RAM
ROM
RAM
Wasm module 124 WAMR library 43817 Bootstrap code 706 CS4WAMR Wasm module 124 WAMR library 43821 CS4WAMR library 1710 Bootstrap code 454 TinyContainer with CS4WAMR Wasm module 124 WAMR library 43821 CS4WAMR library 1030 TinyContainer library 8798 Bootstrap code 2415
124 7056 304
+124 +0 +60
+124 +6969 +8
0 67 1032 10104
+124 0 +0 +0 +0 +0 +40 +10008
124 67 128 16870 739
+124 +0 +0 +156 +862
Arduino Nano 33 BLE nRF9160-DK
Runtimes WAMR Interpreter
+124 +0 +0 +9900 +534
Malicious container start
Friendly A start Friendly B start
With CS4WAMR
Malicious container start
Friendly A start Friendly B start Malicious
With CS4WAMR With TinyContainer container start
Malicious container killed
Figure 7: Representation of the scheduling of containers across WAMR, CS4WAMR, and TinyContainer with longrunning malicious task so all the calculations were done in software which greatly impacted the performance. The container execution is also impacted because each driver call must pass through TinyContainer endpoint system, which routes the communication to the correct driver. On the Arduino Nano 33 BLE, each read() endpoint call to read 255 bytes takes 4 ms with TinyContainer, while taking 11 us with CS4WAMR. 255 bytes per call is currently a limit of TinyContainer regarding the number of readable bytes at a time. 6.2.2 Scheduling overhead & guarantees. In this section, we evaluate the performance overhead of the scheduling mechanism. First, we have measured the performance overhead of the watchdog mechanism by measuring the time added by the mechanism. By running an example code on Arduino Nano 33 BLE, we observe that the setup of the watchdogs takes only 3 microseconds before the call of a container function and the killing of a container not respecting its contract takes 4 microseconds. We have also compared container scheduling with WAMR, CS4WAMR and TinyContainer with a container not respecting its time contract.
In order to do so, on each implementation, we have executed 3 containers: two friendly containers, and one malicious/dysfunctional container. The different scheduling of the containers are represented in Figure 7. The friendly containers execute for a few milliseconds and return the execution. The malicious or dysfunctional container executes but never returns the execution. As WAMR does not manage scheduling, we run one container after the other and see the malicious container monopolizing execution and blocking other containers from executing. With CS4WAMR, we observe that the malicious container is not killed but preempted, leaving a constant cost of running the malicious container each time it is called then preempted after scheduler timeout. With TinyContainer, the malicious container exceeds the execution time limit on its first call and is killed, leaving only the two friendly containers running. 6.2.3 Memory usage evaluation. The memory usage of WAMR, CS4WAMR and TinyContainer with CS4WAMR to run a minimal container is presented in Table 2. The major impact of TinyContainer either on RAM or ROM corresponds to the addition of the TinyContainer library for a single container. The overhead of TinyContainer for each additional container is mainly caused by the memory usage of CS4WAMR which requires a buffer given by TinyContainer. The overhead is also notably caused by the extra 485 bytes of metadata and the additional array elements to store containers and metadata.
6.3
TinyML approach comparisons
In the Table 3, we compare the inference time of our TinyMLaaS proposition using TinyContainer endpoint, as described in section 5, with the TinyML in the container proposed by Buil et al. [10]. Performing TinyML inference in-container with a WAMR interpreter
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
Bastien Buil, Chrystel Gaber, Samuel Legouix, Emmanuel Baccelli, and Samia Bouzefrane
Table 3: Model inference time comparison on Arduino Nano 33 BLE and nRF9160-DK boards Inference time (in ms) with DS-CNN Small model [29] Arduino Nano 33 BLE
nRF9160-DK
In-container WAMR AoT
442
497
In-container WAMR Interpreter
48 324
48 130
TinyContainer TinyML native endpoint
512
507
The inference time is the measured time of the execution of the container function running inference or the container function calling inference endpoint for TinyContainer. It includes all the inference time and the communication cost, and the Wasm container function starting cost. For TinyContainer part, it does not include the loading of the model from the container to the driver.
has a high cost on the inference time, while the overhead introduced by our solution to ensure the communication between the container and the driver is limited. Delegating computation to the host drastically reduces TinyML inference overhead without doing AoT compilation. Thus, delegating to the host compute intensive tasks is a solution to remediate the high overhead of WebAssembly interpreter. TinyContainer enables a way to expose such services from the host to the container.
7 Discussions 7.1 Limitation of endpoint model In the endpoint model used by TinyContainer, containers should be proactive in data exchange. A container cannot be called by an endpoint on the device. Therefore, to obtain updates, a container should pull updates from the endpoints. This might limit the possibility of the device being placed in sleep mode to save energy. One feature for future work is to allow containers to be put in sleep mode and be woken by endpoints. Then the container could pull from the endpoint to get information about the update. Another limitation of current implementation is the need for hosts to implement endpoints that could be used by containers, as the current implementation does not provide built-in endpoints to access device peripherals. One solution to fix this issue can be by adapting existing works that provide interfaces to containers like WASI systems.
7.2
Advantages and limitations of TinyContainer scheduling
The scheduling model proposed with TinyContainer is cooperative scheduling with a watchdog timer. This allows to have no unwanted context switching producing overhead as containers can choose when to return control to TinyContainer. TinyContainer scheduler automatically kills containers not respecting their contract defined in the container metadata. This prevents malicious or buggy containers from having a significant impact on other containers. The current scheduling model, while customizable per container, is constant over time. Consequently, it does not allow mechanisms
like Cloud bursting, which would enable containers to use unused resources when the device is under low utilization. Another challenge is the definition of loop execution time limit (loop_exec_limit) for each container, as a value too small puts the container at risk of being killed by the watchdog. Therefore, users might overprovision this value, and thus increase the maximum possible period of time that a container could wait to be executed.
7.3
Multi-tenant metadata overhead
TinyContainer metadata are conceived to be constructed with information from multiple actors. For example, one actor creates the container code and signs the code. Another actor personalizes the container by setting container data and signs the data. Finally, the entity managing TinyContainer aggregates all the signatures, produces container metadata, and signs the metadata. This system allows reducing possible interference between actors, allowing better understanding of the root cause of a problem in case of issue. For example, if a bug causes an issue in a container, TinyContainer gives the guarantee that the code of the container has not been modified by any other actor, as the code can be signed by the entity producing the code, and thus, the actor producing container code is potentially responsible. Another advantage is that if an actor uses a third-party provider using TinyContainer to remotely deploy its container on its device, the actor is assured that only containers that it has signed can be deployed on the device. One major drawback of this system is the need to verify multiple signatures when loading the container, which produces high overhead on boards not having a cryptographic processor. One alternative, if the metadata does not need to have the properties described above is to only keep the signature for the metadata_token or even to use only the metadata_token with Message Authentication Code (MAC) but with the requirement of using one unique key per device.
7.4
Advantages and limitations of host-delegated TinyML
Delegating TinyML to the host using TinyContainer allows for lighter machine learning implementations, containing only the required code, which is more suited for constrained microcontrollers compared to solutions like wasi-nn, which requires a model runtime like TensorFlow Lite which often means a non-negligible overhead. Additionally, deployed containers do not need Ahead of Time compilation to achieve good performance, compared to solutions such as [10], which require a trusted server for AoT compilation. However, this approach requires preinstalling machine learning code on the device, so a device can only support a single model structure at a time. Another constraint with this solution is the need to trust the host to give him the model weights, which can be a business value, to do the machine learning computation. This limitation can be acceptable as the containerization solutions used provide a sandbox, where the containers are protected from other containers but not from the host, and not a safebox, where the containers are protected from the host.
TinyContainer: Container Runtime Middleware Enabling Multi-tenant Microcontrollers with Built-in Security
8 Related Works 8.1 Runtimes WebAssembly (Wasm) is easily associated with containers as it is based on Wasm modules, which are runnable units of software, and runtimes like Wasmico [24] propose to manage WebAssembly modules as containers with features like containers pause and resume, test & development, delivery, and remote operation by exposing an HTTP server for remote management and monitoring. Several efforts focus on integrating additional control in WebAssembly runtimes on microcontrollers, either through permissions control [10, 22], energy control [22], and memory consumption control [10, 22, 24]. The WebAssembly component model [5] is a proposal for WebAssembly specification to implement a format that describes container permissions and container composition, making inter-container communication interface easier. But both the Component Model, which is only available on a component-aware runtime such as Wasmtime or on a component layer, such as wasm_component_layer [16], induce a costly overhead for constrained microcontrollers. Liu et al. [22] propose Aerogel to have an access-control specification provided with the container, as in the Wasm component model, but adapted to the most constrained devices. While runtimes can manage permissions to host resources, hosts should provide access to system resources to containers. To do that, WASI (WebAssembly System Interface) [17] defines a standard interface to access system resources. Van Kenhove et al. [27] propose a WASI proposal to allow containers to interact with I2C and USB, often used on IoT devices. Aerogel [22] proposes its own system to access memory-mapped peripherals. Runtimes using other bytecode or techniques exist on microcontrollers. For example, Toit [3] provides containers with a memorysafe language providing fault isolation for robustness and devopsstyle management of containers with over-the-air deployment. But, Toit is not made for mutually distrusting containers, as a malicious container can impact the execution of other containers, as any container can use the full Toit API, including the system API to control containers and networks. Femto-containers [28] uses rBPF-based containers that provide isolation and updatability with very small footprints on RIOT OS, VeloxVM [26] and Polyglot Cerberos [8] use custom bytecodes with custom schedulers to orchestrate containers on microcontrollers with timing control. At the operating-system level, Tock OS [21] combines language-based isolation for kernel “capsules” and hardware-enforced user processes using Memory Protection Unit (MPU), giving fine-grained isolation with predictable footprint. It uses a preemptive scheduler for processes (round-robin by default) and cooperative scheduling inside the single-threaded kernel event loop. We compare TinyContainer implementation with CS4WAMR with runtimes from state of the art in Table 4.
8.2
TinyML from containers on microcontrollers
Buil et al. [10] summarize existing approaches in MLOps and TinyMLOps while highlighting the benefits of using WebAssembly to enhance flexibility and modularity.
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
Prior to our proposition, two main methods existed to run machine learning models from WebAssembly containers on microcontrollers. The first one, WASI-NN [7], works by having the container provide the model file to the host that run inference using a model runtime, such as TensorFlow Lite Micro, OpenVINO, and ONNX Runtime. This method allows the device to run diverse model types, but requires a full model runtime on the device. The second one proposed by Buil et al. [10] compiles the model file into machine code and puts it in a WebAssembly container. This allows to not require any code from the host, but requires AoT compilation to have good performance, as bytecode interpretation has a significant overhead. Our proposition of having preinstalled machine learning code on the host is a mix of the two methods, having both the efficiency of the first method even with bytecode interpretation and the lightness of the second method. The limitations are that the model types on one device are fixed and offer less modularity to the container.
9
Conclusion
As IoT software embedded on microcontrollers becomes more complex, modularity becomes necessary, and multi-tenant scenarios become more prominent on such hardware. Recent work has focused on enabling the co-location of multiple miniature runtimes and/or virtual machines on a single microcontroller shared by multiple tenants. However, there was so far no adequate framework available for such multi-tenant scenarios to facilitate finer-grained aspects, including container lifecycle management, adequate container scheduling, and differentiated access control to host RTOS resources. In this paper, we thus introduce TinyContainer, an embedded framework designed for these purposes. We report on benchmarks using our implementation of TinyContainer, which we integrated in a common RTOS to manage small WebAssembly runtimes. We also showcase the advantages of TinyContainer in a TinyML use case whereby access to data and model weights can be isolated from the inference execution engine running in native code, thus benefiting from the best of both worlds: fast execution and isolation for assets that should be protected.
References [1] [n. d.]. Bytecodealliance/Wasm-Micro-Runtime: WebAssembly Micro Runtime (WAMR). https://github.com/bytecodealliance/wasm-micro-runtime. [2] [n. d.]. Bytecodealliance/Wasmtime. https://github.com/bytecodealliance/wasmtime. [3] [n. d.]. Toit - High-Level Software Platform for the ESP32. https://toit.io/. [4] [n. d.]. Wasm3/Wasm3. Wasm3 Labs. [5] [n. d.]. The WebAssembly Component Model. https://componentmodel.bytecodealliance.org/. [6] 2025. Openvinotoolkit/Openvino. OpenVINO™ Toolkit. [7] 2025. WebAssembly/Wasi-Nn: Neural Network Proposal for WASI. WebAssembly. [8] Sven Akkermans, Bruno Crispo, Wouter Joosen, and Danny Hughes. 2018. Polyglot CerberOS: Resource Security, Interoperability and Multi-Tenancy for IoT Services on a Multilingual Platform. In Proceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. ACM, New York NY USA, 59–68. https://doi.org/10.1145/3286978.3286997 [9] Carsten Bormann et al. 2014. Terminology for Constrained-Node Networks. RFC 7228. https://doi.org/10.17487/RFC7228 [10] Bastien Buil, Chrystel Gaber, Samia Bouzefrane, and Emmanuel Baccelli. 2025. TinyML as a Service on Multi-Tenant Microcontrollers. In Proceedings of the 2025 International Conference on Embedded Wireless Systems and Networks.
WiSec ’26, June 30-July 03, 2026, Saarbrücken, Germany
Bastien Buil, Chrystel Gaber, Samuel Legouix, Emmanuel Baccelli, and Samia Bouzefrane
Table 4: Feature comparison between containerization and runtime solutions
Runtimes
Permission Access-control Access to Configurable Support metadata multiple types of execution time AoT-compiled Bytecode management per container format host resources per container containers
FemtoContainers [28]
ebpfbased
VeloxVM [26] Polyglot Cerberos [8]
Custom Custom
Toit [3]
Custom
Aerogel [22]
Wasm
WAMR [1]
Wasm
Wasmico [24]
Wasm
CS4WAMR [10]
Wasm
TinyContainer Multiples
✓
✗
✓
✗
✗
✓ ✓
✓ ✓
✓ ✓
✓ ✓
✗ ✗
✗ ✓ ✗ ✓ ✓ ✓
✗ ✓ ✗ ✗ ✗ ✓
✓ ✗ ✓ ✓ ✓ ✓
✓ ✗ ✗ ✗ ✗ ✓
✗ ✗ ✓ ✗ ✓ ✓
[11] Bastien Buil, Chrystel Gaber, Sylvain Plessis, Emmanuel Baccelli, and Samia Bouzefrane. 2025. Shared Responsibility in Multi-Tenant Microcontrollers. In CNSM 2025-21st International Conference on Network and Service Management. [12] Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. In Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation (OSDI’18). USENIX Association, USA, 579–594. [13] Composable Code for Things. 2025. T2TRG Interim Meeting, Internet Engineering Task Force. https://github.com/t2trg/2025-10-composable-code. [14] Robert David, Jared Duke, Advait Jain, Vijay Janapa Reddi, Nat Jeffries, Jian Li, Nick Kreeger, Ian Nappier, Meghna Natraj, Tiezhen Wang, Pete Warden, and Rocky Rhodes. 2021. TensorFlow Lite Micro: Embedded Machine Learning for TinyML Systems. Proceedings of Machine Learning and Systems 3 (March 2021), 800–811. [15] Hiroshi Doyu, Roberto Morabito, and Martina Brachmann. 2021. A TinyMLaaS Ecosystem for Machine Learning in IoT: Overview and Research Challenges. In 2021 International Symposium on VLSI Design, Automation and Test (VLSI-DAT). 1–5. https://doi.org/10.1109/VLSI-DAT52063.2021.9427352 [16] Douglas Dwyer. 2025. DouglasDwyer/Wasm_component_layer. [17] Dan Gohman, Lin Clark, Alex Crichton, Andrew Brown, Sam Clegg, Pat Hickey, Yosh, Dave Bakker, Mendy Berger, Colin Ihrig, Peter Huene, Piotr Sikora, Jakub Konka, Bailey Hayes, Chris Dickinson, Mike Frysinger, Robin Brown, YAMAMOTO Takashi, Syrus Akbary, Sergey Rubanov, Josh Triplett, George Kulakowski, Eric Crosson, Denis Vasilik, Christian Clauss, Mark Christian, MaulingMonkey, Merlijn Sebrechts, Michiel Van Kenhove, and Nathaniel McCallum. 2025. WebAssembly/WASI: V0.2.4. Zenodo. https://doi.org/10.5281/ZENODO.14826680 [18] Zhaolan Huang, Koen Zandberg, Kaspar Schleiser, and Emmanuel Baccelli. 2024. RIOT-ML: Toolkit for over-the-Air Secure Updates and Performance Evaluation of TinyML Models. Annals of Telecommunications (May 2024). https://doi.org/ 10.1007/s12243-024-01041-5 [19] Michael B. Jones, Erik Wahlstroem, Samuel Erdtman, and Hannes Tschofenig. 2018. CBOR Web Token (CWT). Request for Comments RFC 8392. Internet Engineering Task Force. https://doi.org/10.17487/RFC8392 [20] Hugo Lefeuvre, Nathan Dautenhahn, David Chisnall, and Pierre Olivier. 2024. SoK: Software Compartmentalization. In 2025 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, 75–75. https://doi.org/10.1109/SP61157.2025.00075
[21] Amit Levy, Bradford Campbell, Branden Ghena, Daniel B. Giffin, Pat Pannuto, Prabal Dutta, and Philip Levis. 2017. Multiprogramming a 64kB Computer Safely and Efficiently. In Proceedings of the 26th Symposium on Operating Systems Principles. ACM, Shanghai China, 234–251. https://doi.org/10.1145/3132747. 3132786 [22] Renju Liu, Luis Garcia, and Mani Srivastava. 2021. Aerogel: Lightweight Access Control Framework for WebAssembly-Based Bare-Metal IoT Devices. In 2021 IEEE/ACM Symposium on Edge Computing (SEC). 94–105. https://doi.org/10.1145/ 3453142.3491282 [23] Brendan Moran, Hannes Tschofenig, Henk Birkholz, Koen Zandberg, and Øyvind Rønningstad. 2025. A Concise Binary Object Representation (CBOR)-Based Serialization Format for the Software Updates for Internet of Things (SUIT) Manifest. Internet Draft draft-ietf-suit-manifest-34. Internet Engineering Task Force. [24] Eduardo Ribeiro, André Restivo, Hugo Sereno Ferreira, and João Pedro Dias. 2024. WASMICO: Micro-containers in Microcontrollers with WebAssembly. Journal of Systems and Software 214 (Aug. 2024), 112081. https://doi.org/10.1016/j.jss.2024. 112081 [25] Karolina Skrivankova, Mark Handley, and Stephen Hailes. 2025. Why Are Smart Buildings Still Dumb: The Road Ahead. In Proceedings of the ACM SIGCOMM 2025 Posters and Demos. 91–93. [26] Nicolas Tsiftes and Thiemo Voigt. 2018. Velox VM: A Safe Execution Environment for Resource-Constrained IoT Applications. Journal of Network and Computer Applications 118 (Sept. 2018), 61–73. https://doi.org/10.1016/j.jnca.2018.06.001 [27] Michiel Van Kenhove, Maximilian Seidler, Friedrich Vandenberghe, Warre Dujardin, Wouter Hennen, Arne Vogel, Merlijn Sebrechts, Tom Goethals, Filip De Turck, and Bruno Volckaert. 2025. Cyber-Physical WebAssembly: Secure Hardware Interfaces and Pluggable Drivers. In NOMS 2025-2025 IEEE Network Operations and Management Symposium. IEEE, 1–7. https://doi.org/10.1109/ noms57970.2025.11073737 [28] Koen Zandberg, Emmanuel Baccelli, Shenghao Yuan, Frédéric Besson, and JeanPierre Talpin. 2022. Femto-Containers: Lightweight Virtualization and Fault Isolation for Small Software Functions on Low-Power IoT Microcontrollers. In Proceedings of the 23rd ACM/IFIP International Middleware Conference (Middleware ’22). Association for Computing Machinery, New York, NY, USA, 161–173. https: //doi.org/10.1145/3528535.3565242 [29] Yundong Zhang, Naveen Suda, Liangzhen Lai, and Vikas Chandra. 2018. Hello Edge: Keyword Spotting on Microcontrollers. https://doi.org/10.48550/arXiv. 1711.07128 arXiv:1711.07128 [cs]