LooseLeaf: Enabling Serverless GPUs for Computational Notebooks

Yusen Wu

Tingfeng Lan

Benjamin Carver

University of Virginia Charlottesville, VA, USA xrn4ub@virginia.edu

University of Virginia Charlottesville, VA, USA erc8gx@virginia.edu

George Mason University Fairfax, VA, USA benrcarver@gmail.com

Haoliang Wang

Geoffrey Fox

Yue Cheng

Adobe Research San Jose, CA, USA hawang@adobe.com

University of Virginia Charlottesville, VA, USA vxj6mb@virginia.edu

University of Virginia Charlottesville, VA, USA mrz7dp@virginia.edu

ABSTRACT Computational notebooks reserve a GPU for the whole session, yet the GPU computes only while a cell executes and sits idle otherwise. Serverless execution would release the GPU between cells, but reconstructing the accumulated notebook state on a fresh worker imposes substantial cost on every GPU cell. This paper presents LooseLeaf, a serverless GPU platform for interactive notebooks. LooseLeaf derives the state and import closures of each cell, the minimal variables and packages that the cell needs. Our analysis on 5,000 real-world notebooks shows that these closures are small, and user-cell editing leaves seconds to prepare them. LooseLeaf prepares a GPU worker with these closures while the user edits the cell. After execution, the worker saves the updated state within seconds, so the GPU is reclaimable after every cell. LooseLeaf starts 95–98% of GPU cells with no visible delay, matches the interactivity of a pinned session, and provisions 32–55% fewer GPU-hours than state-of-the-art notebook platforms.

1

INTRODUCTION

Computational notebooks are the de facto interface for data-driven programming. A notebook organizes a program into a sequence of cells. The user runs one cell at a time, and the output appears as soon as execution finishes. This immediate feedback suits the exploratory nature of data analysis and machine learning. Software such as Jupyter [41] brings this interface to millions of users across education, research, and industry [56, 58]. Every major cloud provider now offers a commercial notebook platform [2, 7, 10, 17]. Nowadays, AI/ML makes up a growing share of notebook workloads. A production study reports that interactive deep learning training has emerged as a dominant notebook use case [28]. The rapid progress of large language models (LLMs) pushes this trend further. Open-weight releases put billion-parameter pretrained models in the hands of individual users [37, 54, 66]. Fine-tuning or validating such a model for a task takes repeated trials [26, 33, 49], and notebooks fit this style of work. Every one of these steps executes on a GPU. GPU-backed notebooks are now a standard offering of these platforms [5, 7, 10, 13]. Platforms such as Colab [7] and Kaggle [10] bind a GPU to a notebook session for its entire lifetime. The session acquires the GPU when it starts and holds the GPU until it ends. A session, however, needs the GPU for only a small fraction of its lifetime. First, a notebook mixes GPU cells for training and inference with

Normalized GPU-hours (session-pinned = 100%)

arXiv:2610.33275v1 [cs.DC] 23 Oct 2026

LooseLeaf: Enabling Serverless GPUs for Computational Notebooks

better

Session-pinned

100 75

Modal Notebooks

50

NotebookOS LooseLeaf

25

Naive serverless

0 0

25%

50%

75%

100%

GPU-cell runs missing the 1s threshold (%)

Figure 1: Provisioned GPU-hours against interactivity for LooseLeaf and four baseline approaches on real Kaggle notebooks at 72 concurrent sessions (§4). The horizontal axis gives the fraction of GPU-cell runs that miss the onesecond interactive-start threshold. The vertical axis gives GPU-hours normalized to session-pinned reservation (e.g., Colab [7], Kaggle Notebooks [10]). The baselines trace a cost–interactivity frontier; LooseLeaf shifts this frontier toward the lower left, approaching the interactivity of sessionpinned execution while using fewer GPU-hours than all serverless baselines. CPU-only cells for data preparation and plotting. The GPU does no work while a CPU-only cell runs. Second, the user pauses between executions to examine the output, reason about the result, and edit the next cell. This development time far exceeds the execution time [28]. The reserved GPU thus computes only while a GPU cell executes and sits idle for the rest of the session. A production study reports that reserved GPUs are idle over 81% of the time [28]. Session-pinned reservation is therefore wasteful and prohibitively expensive at fleet scale. A natural remedy is to acquire a GPU only while a GPU cell executes. Serverless GPUs have recently emerged as a new type of cloud service and offer exactly this model [12, 18, 35]. In this model, code runs as self-contained functions. Each function invocation is served by a worker process that is created on demand on a GPU machine and reclaimed once the invocation finishes. A notebook could thus run each GPU cell as one invocation. The GPU would be released during the CPU-only cells and the pauses in between, and provisioning would follow actual demand.

2 BACKGROUND AND MOTIVATION 2.1 Interactive Notebooks and Serverless GPUs

A notebook cell, however, is not a self-contained function. Cells run in a long-lived kernel process and build on the variables and packages that earlier cells leave behind. A fresh worker holds none of this state and needs seconds to minutes per run to re-compute the whole of it, whether by re-running the earlier cells or by loading all previous variables. Such a start is far from interactive. Prior response-time studies suggest that responses should arrive within about one second for users to maintain cognitive flow [47, 51, 52]. We therefore use one second as the interactive-start threshold: a GPU-cell run misses this threshold if the cell begins executing more than one second after the user presses Run. Serverless GPUs for notebooks thus come down to a question of session state. A cell must receive the state it needs before it can run, and the time to provide that state decides whether the cell starts interactively. A cell does not need the whole state. A cell that runs inference, for example, reads the model, the tokenizer, and the inputs it evaluates. The dataframes and plots that earlier exploration left behind are irrelevant to it. What a cell needs is also knowable. Notebook code is organized in cells, so the variables and packages that a cell references can be read from its code while the user is still writing it. To verify these properties at scale, we characterized roughly 5,000 real AI/ML notebooks (§2.4). The characterization confirms that a GPU cell typically reads a small fraction of the session state, and that the packages it imports concentrate in a small common set that can be loaded once and shared. Notebooks also offer a unique window to exploit these properties. A user edits a cell for seconds to minutes before running it. What a GPU cell needs can thus be prepared during this window, before the user presses Run. This paper presents LooseLeaf, a serverless GPU platform built on these observations. While the user edits a cell, LooseLeaf derives from the code the variables and packages that the cell needs. We call these two sets the state closure and the import closure of the cell. LooseLeaf then prepares a serverless GPU worker with these closures ahead of the run. By the time the user presses Run, the worker already holds what the cell needs, and the cell starts at once. After the cell executes, the worker saves the state updates (incrementally) to a shared store within seconds, so the GPU is reclaimable right after every cell. LooseLeaf therefore provisions GPUs by actual demand. Figure 1 summarizes the key results on real Kaggle notebooks. Across all notebook solutions, LooseLeaf keeps user-visible start latency below one second at p95: with 72 sessions, more than 95% of GPU cells begin execution within one second after the user presses Run. In summary, this paper makes the following contributions: • We characterize roughly 5,000 real-world AI/ML notebooks and make three observations that motivate LooseLeaf. • We introduce per-cell state and import closures, two abstractions that reduce the preparation of a GPU cell from the whole session to what the cell needs. • We design a fast checkpoint store for the session state of GPU notebooks. On top of it, we build LooseLeaf, a serverless GPU platform that prepares a GPU worker for each GPU cell during editing. • We evaluate LooseLeaf on real Kaggle notebooks. Across loads, LooseLeaf provisions 43–70% fewer GPU-hours than session-pinned reservation and 32–55% fewer than the stateof-the-art NotebookOS.

Notebooks are widely used in data-driven programming, particularly in AI/ML workflows. They provide a user-friendly programming interface where a program is divided into cells, each containing a block of code. The intermediate outputs can be inspected immediately after cell execution. This immediate feedback enables an iterative workflow in which users write and execute code in a cell, examine the results, and then proceed to the next. Users can also revisit and re-execute earlier cells as they refine their analysis. To support this interactive workflow, each notebook runs within a session backed by a long-running process called the kernel. All cells execute in this kernel and share an implicit namespace: the variables and packages one cell creates persist and become available to later cells without re-execution. Each cell thus inherits the state left by earlier cells and adds its own, so the namespace grows as the session proceeds. With the development of AI, there is an increasing demand to run notebooks on GPU-equipped machines for deep learning applications (e.g., model fine-tuning, hyperparameter tuning, and inference) [28]. In existing notebook services such as Colab [7] and Kaggle [10], a notebook session retains the GPU throughout its entire lifetime. However, a typical notebook contains both CPUonly cells and cells that require GPU resources, and users often pause between cell executions to inspect outputs, reason about results, or step away. As a result, the GPU is needed by only a subset of cells for execution and sits idle for the rest of the time. In recent years, serverless GPUs have emerged as a cloud service model [12, 18, 35]. Users submit code as functions rather than rent GPU machines. Each function execution, or invocation, runs on a serverless GPU worker: a process created on demand on a GPU host and reclaimed after the invocation and any keep-alive interval [11, 60]. Users are typically billed for invocation execution rather than full machine reservation. A notebook can therefore run each GPU cell as one invocation, holding a GPU only during execution and releasing it during CPU-only cells and inter-cell pauses. Serverless GPUs are thus a natural fit for notebook workloads.

2.2

State and Import Closures

The stateful–stateless tension. Although serverless GPUs are a natural fit for notebook workloads, applying them introduces a fundamental tension. Notebook cells communicate through an implicit shared namespace, where each cell reads objects defined by prior cells and leaves its own updated objects for subsequent ones. The notebook session is therefore stateful. By contrast, a serverless invocation is stateless [39, 40]. To execute a GPU cell as a serverless function on a newly created runtime, the runtime must 1) prepare the execution environment by initializing the CUDA runtime and importing all packages that the kernel had previously loaded, and 2) restore all variables accumulated by prior cells. We call the end-to-end cost of making the worker ready to execute the cell its preparation latency. This cost includes both serverless cold-start overhead and notebookstate restoration, and is prohibitively high when all preparation begins after the user presses “Run.” 2

cell and its transitive dependencies require. For example, the function compute_map3 defined in Cell 4 calls torch.softmax and np.argsort, so torch and numpy must be available even though Cell 5 does not import them directly. Together with datasets and transformers , these four packages form the import closure 𝐼𝑐 . A GPU worker that receives only these two closures can execute Cell 5 without restoring the full session namespace.

[1]: import numpy as np, torch from datasets import load_dataset from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained(...) train_ds, val_ds = load_dataset(...).train_test_split(...) [2]: from transformers import (AutoModelForSequenceClassification, TrainingArguments, Trainer) model = AutoModelForSequenceClassification(...) [3]:

args = TrainingArguments(...) rank_w = [1, 1/2, 1/3]

[4]: def compute_map3(eval_pred): probs = torch.softmax(...) ... np.argsort(probs) ... * rank_w

2.3

[5]: ⋆ GPU target trainer = Trainer(model=model, args=args, train_dataset=train_ds, eval_dataset=val_ds, tokenizer=tokenizer, compute_metrics=compute_map3)

Dataset Collection and Statistics

The closures defined above narrow the two sources of preparation latency (§2.2) to per-cell scope: the import closure reduces environment preparation from all session packages to the minimal set a cell requires, and the state closure reduces variable restoration from the full namespace to the minimal set a cell references. Although the closures reduce the scope, this preparation work remains on the critical path if it begins only after the user requests execution. To understand the practical magnitude of each component, we characterize real-world AI/ML notebook workloads. We collected 5,004 notebooks from KagTable 1: Task composition of the gle [10] and public GitHub notebook corpus. repositories [6] by searching for AI/ML and GPU Task type Notebooks Share keywords such as llama, Training 2,597 52% lora, and pytorch. To Fine-tuning (PEFT) 1,016 20% Inference 988 20% improve labeling roData analysis and other 403 8% bustness, we indepenTotal 5,004 100% dently applied two large language models (LLMs), GPT-oss-120B [54] and Grok-code-fast-1 [23], to classify the device requirement of each cell. A cell was labeled as GPU-requiring when either model assigned a GPU to it, and was labeled as CPUonly otherwise. On the 5,004 notebooks processed by both models, the models agreed on 94.5% of the cell-level labels. Cohen’s 𝜅 was 0.75 [30], corresponding to substantial agreement under the Landis– Koch interpretation [42]. Table 1 summarizes the composition of the corpus. Most notebooks perform model training, fine-tuning, or inference, and 8% are devoted to data analysis or other tasks. Figure 3(a) shows the distribution of the number of cells per notebook. A typical notebook contains 17 cells. Notebooks are thus fairly long, which suggests that a session can accumulate substantial state as it advances through the cells. Figure 3(b) shows the fraction of GPU cells within each notebook. A GPU is never used in 37% of the notebooks, and among the notebooks that use one, GPU cells account for a median of 20% of all cells. Figure 3(c) gives a consistent view by cell type. Boilerplate, data handling, and visualization account for most cells, while model, training, and inference cells form a minority. Both views confirm the mismatch described in §2.1. A GPU bound to a whole session serves only a small fraction of its cells. Figure 3(d) shows the execution time of GPU cells. The distribution spans from milliseconds to hours, and the median GPU cell executes for only about eight seconds. Most GPU cells are thus short steps that demand the same interactivity as the rest of the session. The preparation latency of a GPU invocation must therefore be small.

trainer.train()

Sc state closure: { tokenizer, train_ds, val_ds, model, args, compute_map3, rank_w } Ic import closure: { datasets, transformers, torch, numpy }

Figure 2: Dependency graph of a five-cell model fine-tuning notebook. Cell 5 is the GPU target. Solid arrows are state dependencies, and dashed arrows are import dependencies. Both traces are transitive. The boxes at the bottom show the resulting closures 𝑆𝑐 and 𝐼𝑐 . Dependency-guided closures. The key observation is that notebooks expose program structure at cell granularity. By statically analyzing each cell, the runtime identifies the variables it defines and references, and these definition–use (def-use) relationships form a dependency graph over cells. An edge connects a reference to its definition. Because a name may be defined by more than one cell and cells may run in any order, the edge points to the definer that executed most recently. LooseLeaf uses this graph to compute each cell’s execution requirements precisely. For a target cell, we collect its referenced variables and perform a BFS over the graph to obtain the minimal state required to run it, and apply the same analysis to imported packages to obtain its minimal import requirement. This lets us model each cell as a serverless function 𝐹𝑐 (𝑆𝑐 , 𝐼𝑐 ) → (𝑆𝑐′ , 𝑂𝑐 ), where the state closure 𝑆𝑐 is the minimal set of live variables the cell must read, the import closure 𝐼𝑐 is the minimal set of packages it must import, 𝑆𝑐′ is the updated state, and 𝑂𝑐 is the output. Static analysis can determine both closures exactly, because every variable and package that a cell uses is named in its code or in the body of a function that it references. The only exception is code that reads the namespace dynamically through constructs such as exec or globals(), which notebook code rarely uses. The state closure and import closure are the two abstractions on which the rest of the paper builds. Figure 2 illustrates this analysis on a five-cell model fine-tuning notebook. Cell 5 is the GPU target cell that constructs a Trainer object and launches model training. By tracing the state dependencies (solid arrows), the runtime determines that Cell 5 directly references six variables defined by the preceding four cells: model , args , train_ds , val_ds , tokenizer , and compute_map3 . The trace also follows references inside function bodies. compute_map3 reads the list rank_w defined in Cell 3, so rank_w joins the closure even though Cell 5 never names it. These seven variables form the state closure 𝑆𝑐 . In parallel, the runtime traces import dependencies (dashed arrows) to identify the packages that the 3

0.50 All Kaggle GitHub

0.25 0.00 10

1

10

1.00

(b) GPU-cell share

0.75 0.50 37% never use a GPU

0.25 0.00

2

0.0

0.5

(c) Cell types

1.0

27 14 10 9 8 4

0

GPU-cell fraction per notebook

cells per notebook

42

Boilerplate Data Model Visualization Inference Training Evaluation

25

% of cells

CDF of GPU cells

0.75

CDF of notebooks

CDF of notebooks

(a) Notebook size

1.00

(d) GPU-cell duration

1.00 0.75 0.50 0.25

0.00 10−2

median 8.3s

10

0

102

104

per-cell execution time (s)

Figure 3: General statistics of the collected notebook corpus. (a) CDF of cells per notebook; a point (𝑥, 𝑦) means a fraction 𝑦 of notebooks contain at most 𝑥 cells. (b) CDF of the per-notebook fraction of GPU cells; the jump at zero corresponds to notebooks that never use a GPU. (c) Cell type distribution. (d) CDF of per-cell execution time over GPU cells, excluding definition-only cells; the dashed line marks the median.

0.50 0.25

22%

17%

11%

8%

4%

0.00 5-9

10-19 20-49 50-99 ≥100

namespace size before cell |Live|

60 40 20 0

1.00

CDF of cells

0.75

717 distinct pkgs

0.75

(c) Edit window synthetic typing JuNE measured AI-assisted

0.50 0.25 0.00

tr an sf torc or m h nu ers m pa py nd as to rc tqd hv m isi sk on le ar n da pe ta ft se ts m at PI pl L ot lib c Ot v2 he r

|Req| / |Live|

(b) Import frequency

median

% of GPU cells

(a) State closure

1.00

100

101

102

103

edit window (s)

Figure 4: Three properties of the corpus. (a) State-closure ratio against the number of live variables before each GPU cell. (b) Package frequency over GPU cells. (c) Per-cell editing-window distributions for manual typing and AI-assisted editing.

2.4

Implications The runtime should restore notebook state at variable granularity using the dependency-guided closure of the target cell.

Dataset Characterization

We then construct the dependency graph and compute the state closure and import closure for each cell. From this characterization, we derive the following three important findings: (1) the state a GPU cell needs is small (§2.4.1); (2) the packages it needs are concentrated (§2.4.2); and (3) editing exposes a window in which the preparation work can be hidden (§2.4.3).

2.4.2 Observation 2: GPU Cells Have a Head-and-Tail Package Distribution. For each GPU cell, we compute its import closure, which is the set of packages that must be available before the cell runs on a GPU. We aggregate these packages across GPU cells to obtain the package-frequency distribution in Figure 4(b). The distribution has a small common head and a long tail. Common AI/ML and data packages dominate the head. torch appears in 71% of GPU cells, followed by transformers, numpy, pandas, tqdm, torchvision, and sklearn. At the same time, the corpus contains 717 distinct packages, and 53.5% of GPU cells use at least one package outside the top 12. This distribution exposes a tradeoff in the preparation of GPU workers. Starting each worker from an empty process pays the same import cost for the head packages on every invocation. Preloading a fixed set of packages avoids this repeated cost, but the long tail means that any fixed set either misses the packages of many cells or loads hundreds of packages that most cells never use. This head-and-tail structure is similar to the package distribution reported by SOCK for Python serverless workloads [53]. SOCK addresses the same tradeoff with zygote processes [1]: workers fork from a process with the common packages already initialized and import the remaining tail packages after the fork. However, SOCK targets CPU-based serverless functions, while our workload requires GPU execution. The same mechanism cannot be applied

2.4.1 Observation 1: A GPU Cell’s State Closure Is Small. For each GPU cell 𝑐, we compare the state required by the cell with the named state accumulated before the cell executes. We define Live(𝑐) as the set of data variables available before 𝑐 executes, and Req(𝑐) as the cell’s state closure 𝑆𝑐 , i.e., the variables it must read. The ratio |Req(𝑐)|/|Live(𝑐)| measures the fraction of the live namespace that must be materialized for the invocation. Figure 4(a) plots this ratio against the namespace size before each GPU cell. The median ratio decreases as the live namespace grows. Among GPU cells with 5 to 9 live variables, the median cell reads 22% of its live variables. For cells with 20 to 49 live variables, the median ratio drops to 11%, and for cells with 100 or more live variables, it falls to 4%. These results show that the cost of restoring a full namespace grows with the accumulated session state, while the state required by a GPU cell grows much more slowly. This trend suggests that instead of reconstructing a full kernel namespace for every GPU invocation, the runtime should restore only the dependency-guided state closure of the target cell.

4

Frontend

directly because a forked child process inherits the CUDA context handles of the parent (see §3.3), but the underlying GPU driver resources are not duplicated by fork [4]. Still, the similar package distribution suggests that zygote-style process reuse is a promising basis for reducing package setup cost, as long as it handles the CUDA runtime correctly. Implications The runtime should reduce environment setup time with a GPUaware import cache.

Execution Recipe

Notebook DAG UI Daemon

1 Static Analysis

2 LooseLeaf Server 3 Dispatch CPU Cell Session A: Persistent Worker

2.4.3 Observation 3: Editing Provides a Short Preparation Window. The first two observations reduce the amount of state and environment work required by a GPU invocation. However, this work still increases interactive latency if it starts only after the user requests execution. Notebook editing provides an opportunity to move this preparation earlier. While a user edits a cell, the runtime can keep the dependency graph and the cell’s closures up to date. It can then start speculative preparation before execution begins. To estimate the size of this window, we used a typing-speed model from a large-scale keystroke study [32]. We validated this estimate with JuNE [63], a Jupyter notebook development trace dataset that records timestamped cell edits and executions. The JuNE dataset contains logs of code evolution in Jupyter notebooks, comprising over 100 hours of execution logs from 20 participants solving data science tasks. Figure 4(c) compares the synthetic typingtime distribution with measured pure-typing intervals. The two distributions closely match, with median times of 9.4s and 9.5s, respectively. This result shows that editing provides a real preparation window, but the typical window is only on the order of seconds. As LLMs mature, users increasingly rely on them to assist programming. In a notebook, the user asks an AI assistant for code and pastes the suggestion into a cell. Pasting takes far less time than typing, so this interaction could shrink the editing window. We examined this concern with the interaction log released by DSPM [45], a study of how students use an LLM assistant [36] for data analysis in computational notebooks. From the timestamped edit and execution events in the log, we measured the interval between each paste of generated code and the next execution. The distribution of this interval closely tracks the typing window, with a median of 10.7s (Figure 4(c)). The user thus still spends seconds reading and adapting the generated code before running it, and the editing window remains. Implications The runtime should prepare the GPU worker while the user edits the cell, and complete the preparation within the few-second editing window before the user requests execution, thereby keeping preparation off the interactive path.

State Closure Import Closure CPU/GPU Label Send Run/Prefetch req

Elastic GPU Host

GPU Cell

Import Cache

4 Fork

SessionB: Persistent Worker

Serverless Worker

Serverless Worker

CPU Host 5 Restore

Checkpoint Store

GPU Tensor

6 Checkpoint Dataset

Figure 5: Architecture of LooseLeaf and the path of a single request through it. As the user edits, the DAG daemon distills each cell into an execution recipe that holds the state closure, the import closure, and a CPU or GPU label. A CPU cell runs on the persistent worker of its session. A GPU cell runs on a worker forked from the import cache on a GPU host. Workers exchange notebook state through the checkpoint store. Circled numbers mark the steps described in the text. of the cell from the store and saves the updated variables back after execution. Therefore, LooseLeaf can reclaim GPU between cells. The remainder of this subsection describes the components of LooseLeaf and the path of a GPU cell through them. Architecture. As shown in Figure 5, LooseLeaf comprises five components. The frontend hosts the notebook UI and a DAG daemon that keeps the dependency graph current as the user edits and derives from it an execution recipe for each cell, which holds the state and import closures of the cell together with a CPU or GPU label. The server is the control plane that dispatches cells and manages workers. It does not require colocated GPUs and may run on the user’s local machine or on a remote host, provided that it can communicate with the frontend, worker hosts, and checkpoint store. A cell runs on one of two workers, a persistent CPU worker that hosts the notebook kernel of a session or an ephemeral GPU worker that runs a single GPU cell. The import cache is a per-host process pool from which each GPU worker is forked to reduce package import time. The checkpoint store holds notebook variables and moves them across workers and hosts. Workflow. A GPU cell passes through LooseLeaf in two phases. While the user edits the cell, the DAG daemon updates its recipe by static analysis ( 1 ). Once the cell is labeled as a GPU cell, the daemon sends the recipe to the server as a prefetch request ( 2 ). The server dispatches the request ( 3 ) and acquires a GPU worker for the session, which is either a warm worker or a new worker forked from the import cache ( 4 ). The fork provides the packages that the cache has already loaded. The worker then imports the remaining packages of the import closure and restores the variables of the state closure from the checkpoint store ( 5 ).

3 DESIGN 3.1 LooseLeaf Overview Based on the observations from §2.4, we design a serverless GPU platform for computational notebooks called LooseLeaf. For a GPU cell, LooseLeaf runs it as a serverless invocation and provisions a GPU worker on demand using the state and import closures of the cell. LooseLeaf keeps the session state in a checkpoint store shared across hosts (§3.2). A GPU worker restores the variables 5

GPU

When the user runs the cell, the run request carries the closures computed from the submitted code of the cell, and the worker completes any remaining preparation before it executes. After execution, the worker writes the updated variables back to the store ( 6 ) so that later cells can restore them. The worker then stays warm for the session until LooseLeaf reclaims it under the eviction policy (§3.4). The following subsections detail these mechanisms.

3.2

2MB

weights exp_avg

dirty unchanged

exp_avg_sq

2

GPU Tensor: Pinned-DMA

Payload Pool Checkpoint Store

weights exp_avg

Dependency-Guided Checkpoint / Restore

model

trainer

optimizer

dataset

exp_avg_sq

A GPU worker must hold the variables in the state closure of the cell before the cell can run. §2.4.1 shows that a GPU cell reads only a small fraction of the live namespace. LooseLeaf exploits this property by restoring state at the granularity of individual variables. For a target cell, LooseLeaf computes the state closure based on the dependency graph and restores only those variables. Between cells, these variables live in the checkpoint store. The rest of this subsection describes how a variable moves into the store and back. The problem with per-variable pickle. To restore variable-level state, the straightforward approach is to serialize each variable with ordinary pickle [14]. Pickle encodes the variable into a selfcontained stream carrying two kinds of content for every object, the structure and the payload. The structure describes how to rebuild the object. The payload is the raw data bytes that the object carries. With one stream per variable, the runtime can save and restore each variable independently, which is the granularity that the state closure requires. However, this approach is both incorrect and slow. The approach is incorrect because variables share objects. For example, an optimizer maintains parameter groups that refer to model parameters. However, pickle detects sharing only within the single stream that it writes. This is because pickle detects sharing through a memo that records every object written to the stream, and the memo is discarded when the stream ends. The model and the optimizer are serialized into two separate streams, so each stream carries a private copy of the shared parameters. Restore then materializes the two copies as distinct objects. As a result, a training step updates the copy held by the optimizer while the model computes with the other copy. The same failure arises for tensor views [19]. Slicing and reshaping can produce a view, which is a distinct tensor object that shares the storage of its base tensor. If the view and the base tensor are serialized into two separate streams, each stream again carries a private copy of that storage. The approach is also slow, because a pickle stream must materialize every payload byte inside it. Saving a variable copies the payload of each object at least twice, and restoring copies it back out. A GPU tensor pays two more transfers because its bytes cross the PCIe bus at each end. All of these copies run sequentially in one thread. The serialization cost of a variable therefore grows with its payload size. Unfortunately, almost all state bytes in a notebook sit in such payloads. On the notebooks used in our evaluation, GPU tensors hold 95.0% of all saved state bytes, and CPU buffers such as dataset and array payloads hold most of the remainder. The other objects are small control values and together hold under one percent. The cost of pickling a closure is thus the cost of moving its payload bytes. The sharing problem makes this even worse. Pickle cannot recognize a shared tensor across separate streams, so every stream that reaches the tensor moves its bytes again.

dataset buffers GPU Tensor

OOB Buffer

2 OOB Buffer: Parallel memcpy

3 commit

CPU live objects

model

trainer

optimizer

dataset

1 pickle Pickle stream trainer

.model

.optimizer

rootref rootref

model

reference structure

weights

objref

optimizer

params[0]

dataset

data buffer

exp_avg

exp_avg_sq

Figure 6: Checkpointing a cell’s state. Pickling captures the object structure and makes cross-variable sharing explicit through RootRef and ObjRef references ( 1 ). Large payloads leave the stream for a shared pool ( 2 ), and the structure blobs commit to the checkpoint store ( 3 ). Therefore, variable-level restore requires both a mechanism to preserve sharing across variables and a mechanism to keep the payload bytes out of the pickle stream. Once the payloads leave the stream, they can move on fast bulk paths. The stream shrinks to the object structure and the small control values that pickle serializes well. Figure 6 illustrates the two mechanisms on a training closure. Preserving cross-variable sharing. LooseLeaf preserves sharing by encoding cross-variable references explicitly. To detect sharing across variables, LooseLeaf retains the memo of every variable after serialization. The retained memo keeps holding the objects that it records, so the identity of a recorded object cannot be reused by another object. While serializing a variable, the serializer checks each object that it reaches against the retained memos of the other variables of the session. A match shows that another variable has already serialized the object. In this case, LooseLeaf emits a reference instead of serializing another copy of the object. A RootRef names another top-level variable, whereas an ObjRef names an object inside another serialized variable, such as a model parameter or a buffer. In Figure 6, trainer holds a RootRef to each of the top-level variables model and optimizer, and the parameter entry inside optimizer is an ObjRef to the weights serialized under model. Immutable objects are serialized by value because sharing

6

is unobservable for them. This mechanism allows variables to be serialized separately while preserving the sharing between them. Externalizing large payloads. To keep the payload bytes out of the stream, LooseLeaf intercepts the serialization of GPU tensors and CPU buffers. These two classes hold nearly all of the saved state bytes. When the pickler reaches such a payload, LooseLeaf moves the payload bytes out of the stream into a shared pool on the host. The pool is backed by huge pages [9], which reduces page fault time. In the stream, a reference to this pool location takes the place of the payload. The payload of a storage is written once, by the variable that first saves it. Any other variable that reaches the same storage references this pool location and writes nothing. The stream therefore keeps only the object structure and the references. Each payload class enters the pool on the bulk path that fits it. For a GPU tensor, the worker registers the pool regions with CUDA, so the tensor storage moves between device memory and the pool in one pinned DMA transfer. For a dataset or array buffer, LooseLeaf uses pickle protocol 5’s out-of-band (OOB) buffer mechanism [21] to separate the buffer payload from the pickle stream. LooseLeaf copies the resulting buffer into the payload pool using parallel memcpy. With every payload outside the stream, the pickle stream of each variable shrinks to a small blob of structure and references (Figure 6), and the memo that LooseLeaf retains for sharing detection covers only this structure. Each variable then commits this blob to the checkpoint store. The store keeps one entry per variable of the session and records on which host the payload bytes of the session reside. When the session moves to another host, LooseLeaf copies the payload bytes from the old pool into the pool of the new host (§3.4). Restore. At restore time, LooseLeaf starts from the state closure of the target cell. It unpickles the variables of the closure in dependency order. When unpickling reaches a RootRef or an ObjRef, LooseLeaf restores the referenced object and reconnects the reference to it, so a shared object is restored once. In Figure 6, restoring trainer pulls in model and optimizer through its RootRefs, and the ObjRef inside optimizer reconnects to the weights restored under model. For a payload reference, LooseLeaf fetches the payload bytes from the shared pool. The runtime therefore restores the DAG-selected variables together with their shared objects and reachable payload bytes. This restore is minimal, and the same bound applies to GPU memory. Only the tensors inside this reachable set move onto the GPU. When the session holds both a model and an optimizer and the target cell reads only the model, the closure names the model alone. LooseLeaf moves the weights onto the GPU and leaves the optimizer state in the checkpoint store. Incremental save. After a cell executes, the worker saves the updated state 𝑆𝑐′ . Besides the variables that the cell defines, 𝑆𝑐′ includes variables in the state closure that the cell changes in place, such as by appending to a list. Static analysis cannot see such changes, so the worker re-serializes every variable in the state closure and every variable reachable from it through references. The re-serialization rebuilds only the structure blobs, which are small because the payloads live in the pool. The payload bytes are updated by dirty tracking. For GPU tensors, LooseLeaf tracks dirty regions at 2 MB granularity. After a cell updates tensor values, LooseLeaf refreshes only the dirty regions in the pool instead of rewriting the full tensor payload (the dirty regions of weights and the optimizer

state in Figure 6). Because a variable is serialized again at every save that reaches it, the positions of its objects in the stream can change. An ObjRef therefore carries a permanent id of the object. The store maps the id to the current location of the object, and the owner updates this location at each save. State coherence across workers. The persistent CPU worker and the ephemeral GPU workers exchange variables only through the checkpoint store. After a cell executes, the worker saves the variables that the cell updated, and the next cell reads them from the store on any worker. The store keeps a version for every variable and advances it whenever a save changes the variable. A worker that already holds a copy of the variable refreshes the copy when the version has advanced.

3.3

GPU-Aware Import Cache

Restoring the state closure materializes objects whose classes are defined in packages, and executing the cell calls packages directly. Before the restore begins, the worker therefore needs to initialize CUDA and import the packages of the import closure. As §2.4.2 shows, GPU cells follow a head-and-tail package distribution. Many cells use the same common AI/ML and data packages, while a long tail of packages appears in only a small fraction of cells. If each worker imports packages from scratch, the common head causes worker creation to pay the same import cost repeatedly. LooseLeaf reuses redundant import packages for GPU cells through an import cache. A worker forked from the import cache inherits the loaded modules through copy-on-write and imports only the missing packages after the fork. The CUDA fork-safety problem. However, this fork-based method cannot be applied to GPU cells, because CUDA runtime state is not fork-safe [4]. A CUDA context is bound to the process that initializes it and cannot be used by a forked child, so a process that has initialized CUDA cannot serve as a parent for GPU workers. Therefore, every fork parent in the import cache must remain CUDA-clean and exclude packages whose imports touch CUDA. Cache hierarchy. LooseLeaf implements the import cache as a hierarchy of cache nodes (Figure 7). Each cache node is a live CUDAclean zygote process with a set of loaded modules. The base node holds the common packages from the head of the distribution, and child nodes extend it with tail packages. For a GPU cell, LooseLeaf chooses the cache node whose loaded modules best match the import closure of the cell ( 1 ). The worker forks from that node and inherits its loaded modules ( 2 ). After the fork, the worker initializes CUDA and then imports any remaining packages. Growing the cache. When a cell introduces imports that are absent from the selected cache node, the worker reports these imports to the import cache before executing the cell ( 3 ). The import cache then grows a new node asynchronously ( 4 ) by forking the selected node and importing the packages into it, so that later workers inherit them. Because this growth is asynchronous, it does not delay the execution of the cell. To keep every node CUDA-clean, LooseLeaf applies a CUDA-safety check and admits only packages whose import does not touch CUDA. A package that touches CUDA stays out of the cache, and each worker imports it after the fork, which is safe because the worker has already initialized CUDA. Each host caches a bounded number of nodes and evicts the least 7

Import Closure:

Algorithm 1: Per-cell worker acquisition. 𝐿 is the GreedyDual priority defined under Bounding GPU residency. Input: session 𝑠, cell 𝑐, per-host warm worker cache 𝑊 1 if 𝑠 has a warm worker 𝑤 in 𝑊 then 2 restore missing state into 𝑤 ; // hit 3 else // miss 4 ℎ ← most-loaded host with sufficient memory and a free GPU slot; 5 if no such host exists then 6 ℎ ← most-loaded host with an evictable idle worker; 7 evict the worker of session arg min𝑠 ′ 𝐿(𝑠 ′ ) from 𝑊 [ℎ] ; // GreedyDual 8 end 9 𝑤 ← fork from import cache on ℎ; 10 if ℎ ≠ previous host of 𝑠 then 11 migrate state of 𝑠 to ℎ; 12 end 13 restore state closure 𝑆𝑐 into 𝑤; 14 end

{torch, open_clip, timm, bitsandbytes} Import Cache Base Node {torch}

1 Best Match

{torch,open_clip}

4 Grow Node {torch,open_clip,timm}

2 Fork

{torch, datasets}

CUDA safety check {timm}- pass {bitsandbytes}- reject

GPU Worker

3 Report

Init CUDA -> import {timm, bitsandbytes}

Figure 7: The GPU-aware import cache. Cache nodes are live CUDA-clean zygote processes; the base node holds the common head and child nodes extend it with tail packages. For a cell whose import closure is {torch, open_clip, timm, bitsandbytes}, the worker forks from the best-matching node ( 1 , 2 ) and imports the rest after the fork. timm passes the CUDA-safety check, so the cache grows a node for it asynchronously ( 3 , 4 ); bitsandbytes touches CUDA and stays out of the cache.

enters the list when its dependencies include the CUDA runtime, and users can add further packages. Prefetch and worker acquisition. For a GPU-classified candidate cell, the frontend sends the current closures to the server as a prefetch request. The server then acquires a GPU worker for the session. If the session already has a warm worker, the server reuses it and restores only the missing or outdated variables, which constitutes a hit. Otherwise, the server creates a worker on a host chosen by consolidation (described below), which constitutes a miss. The new worker forks from the import cache (§3.3) and restores the state closure from the checkpoint store (§3.2). When the chosen host differs from the previous host of the session, the session state migrates first. Algorithm 1 summarizes the procedure. Bounding GPU residency. Edit-time preparation also creates a resource-control problem. A prefetched worker occupies a GPU device, so the system must decide how long the worker should remain resident. A fixed timeout provides no stable choice: a short timeout can discard useful preparation, while a long timeout lets a client hold a GPU with cheap prefetch requests. Instead of a fixed timeout, LooseLeaf manages idle warm workers as a cache. An idle worker can be seen as a cache entry: a later cell of the same session can hit the worker and skip its preparation. LooseLeaf manages the cache with GreedyDual [67], so eviction weighs recency together with restore cost. For a session 𝑠, let 𝑅𝑠 be the GPU state held by its warm worker. LooseLeaf computes ∑︁ 𝑐 (𝑠) = bytes(𝑟 ), 𝐿(𝑠) = 𝐻𝑠 + 𝑐 (𝑠),

recently used leaf node, and the base node stays resident to keep the common head of the distribution loaded.

3.4

Bounded Edit-Time Speculation

The two preceding mechanisms lower the cost of creating a GPU worker. The remaining question is when and where to create it. If the system waits until the user presses Run, the preparation latency falls on the critical path. Notebook editing, however, provides a short preparation window before execution (§2.4.3). LooseLeaf exploits this window by preparing GPU workers while the user edits a cell, and the DAG daemon keeps the def-use graph current with each edit for the preparation steps below. Routing cells to the GPU path. The frontend first needs to decide whether the candidate cell should use the GPU path. Running every cell on the GPU path would waste GPU resources, since many notebook cells are CPU-only. This decision can rely on static package dependencies, because notebook cells are Python code, and GPU work is invoked through Python packages that bind to GPU libraries such as PyTorch [55] and related AI/ML frameworks. LooseLeaf therefore identifies GPU cells by their package use. From the def-use graph, LooseLeaf computes the effective import set of each cell. The set contains every imported package that is reachable from the references in the top-level statements of the cell. In Figure 2, the set of Cell 5 contains transformers through the reference to Trainer and torch through the call in compute_map3 , whereas the set of Cell 4 is empty because its top-level statement only defines a function. LooseLeaf routes a cell to the GPU path only when its effective import set contains a package from a list of GPU framework packages. LooseLeaf builds this list both automatically and manually. A package installed on the worker hosts

𝑟 ∈𝑅𝑠

where 𝐻𝑠 is the value of the host’s clock when 𝑅𝑠 last changed; the clock never decreases. Under GPU pressure, LooseLeaf first evicts idle workers whose cell has completed. When no such worker exists, it evicts a speculatively prepared worker. Within each class, it evicts the worker with the smallest 𝐿(𝑠) and advances the clock to that value. 8

Speculative Phase

Frontend

LooseLeaf Server 1 Prefetch <Ic, Sc>

Import Cache

Checkpoint Store

time

2 Acquire Worker IF miss IF hit

fork reuse

GPU Worker

3 Prepare <Ic, Sc>

Authoritative Phase

Run Pressed 4 Run

Dispatch Output

5 Save Sc’

Figure 8: During editing, the frontend sends a prefetch request that carries the current closures ( 1 ). The server acquires a worker for the session ( 2 ). On a miss, the server forks a new worker from the import cache. On a hit, it reuses the warm worker of the session. The worker imports the remaining packages and restores the state closure from the checkpoint store ( 3 ). When the user presses Run ( 4 ), the worker completes any remaining preparation and executes the cell. After execution, the worker saves the updated state to the checkpoint store ( 5 ). The worker is later reclaimed and the state persists in the store. Table 2: Evaluation workload: 24 Kaggle notebooks. Each entry shows the realized notebook count, which equals the corpus-proportional target.

The policy bounds the residency that any client can obtain. Because 𝐻𝑠 never exceeds the clock, 𝐿(𝑠) lies at most 𝑐 (𝑠) above it, so the worker is evicted by the time the clock climbs that far unless 𝐿(𝑠) rises, and 𝐿(𝑠) changes only with 𝑅𝑠 . Apart from executing a cell, a client changes 𝑅𝑠 only through a prefetch, which loads variables that the worker does not yet hold, and every such variable comes from an earlier execution of the session. A repeated prefetch for the same closure therefore leaves 𝐿(𝑠) unchanged while the clock climbs past it. The residency of a session is thus bounded by the state bytes that its cells have already computed: cheap prefetch requests cannot hold a GPU, and an adversary obtains no more residency than an honest session that computed the same state. Worker placement and cluster provisioning. LooseLeaf places new workers by consolidation. Among the hosts with enough memory for the session state, the server picks the most loaded host with a free GPU slot, and when no such host exists it evicts an idle worker on the most loaded host under the policy above. A background controller scales the number of powered hosts with a target-utilization policy [8] and square-root staffing headroom [38], and it powers a host off only after its warm workers are evicted and the resident state migrates to another host. Life of a GPU cell. Figure 8 traces the protocol between the frontend and the server. The protocol has a speculative phase during editing and an authoritative phase after the user presses Run. During editing, the prefetch request carries the current closures of the cell, so the preparation of the worker overlaps with the edit window (§2.4.3). The speculation is safe because a prefetch only loads packages and variables and never executes the cell. The prepared worker may still become stale before the run. The run request therefore carries the closures computed from the submitted code, and the worker checks the versions of the restored variables (§3.2), refreshes the outdated ones from the checkpoint store, and completes any remaining preparation before execution. A stale prefetch thus costs only the uncovered preparation time and never affects correctness. After execution, the worker saves the updated state and stays warm under the eviction policy above.

Task type

small (≤300M)

medium (300M–1B)

large Total (≥1B)

training-only training+inference PEFT+inference inference-only

4 9 0 2

0 1 0 0

0 1 3 4

4 11 3 6

Total

15

1

8

24

4 EVALUATION 4.1 Experimental Setup Implementation. We implemented LooseLeaf for both Jupyter and marimo [25]. In both, the frontend maintains the def-use graph during editing and runs the closure computation and GPU labeling inside a Pyodide [15] web worker, and the two frontends share the same server and workers. The import cache is a per-host process tree with LRU eviction capped at 50 nodes. The checkpoint store keeps metadata in Redis [16] and payload bytes in a hugetlbfs pool managed by dlmalloc [20]. Payloads move across hosts through the zero-copy interface of Linux [31], and structure blobs synchronize through the object store of Ray [48]. The server exports the demand signal to a standard cluster autoscaler [8]. Testbed. All measurements run on a cluster of AWS p4d.24xlarge instances with 8 NVIDIA A100-40 GB GPUs, 96 vCPUs, and 1,152 GiB of RAM. Each node reserves a 512 GiB hugepage arena that backs the checkpoint payload pool (§3.2). Workload. We evaluate on 24 Kaggle notebooks sampled from the 2024–2025 competition corpus. The sample follows the tasktype and model-size distribution of the corpus by Hamilton apportionment [27] (Table 2). The notebooks cover training, training with inference, parameter-efficient fine-tuning, and inference-only pipelines. The models range from CNNs below 300 M parameters to 14 B-parameter LLMs. Inter-cell arrival time is drawn from the 9

(a) N = 24 sessions

(b) N = 48 sessions

(c) N = 72 sessions

GPUs

72 48 24 0 0

100

200

300

Time (min)

LooseLeaf Actual use

400 0

100

Session-pinned NotebookOS

200

Time (min)

300

Modal Notebooks Overprovisioned

400 0

100

Saved vs pinned Saved vs NotebookOS

200

300

400

Time (min)

Figure 9: Provisioned GPUs over time for 24, 48, and 72 concurrent sessions, against the GPUs actually executing cells. Provisioning acts at the whole-host granularity of eight GPUs. The shaded regions mark the GPU time that LooseLeaf saves against each baseline and the headroom that LooseLeaf holds above actual use. Adobe notebook traces [28]. The edit window of each cell is drawn from the paste-to-run distribution measured in DSPM [45] (§2.4.3). Baselines. We compare LooseLeaf against four baselines: • Session-pinned reservation is the model of Jupyter, Colab, and Kaggle: a session holds a GPU from its arrival until the session ends. It defines the latency ideal, since every cell finds a warm kernel. It also defines the provisioning ceiling, since the reserved GPU spans every idle gap between cells. We model it charitably: sessions are packed best-fit onto 8-GPU hosts, and a host powers off as soon as it becomes empty. • NotebookOS [28] keeps three persistent kernel replicas for each session, so an arriving GPU cell always finds a ready kernel to serve it. When a cell runs, NotebookOS assigns a free GPU to the replica that executes it. The replicas hold the session state in CPU memory, so a GPU cell must first move the accumulated model state onto the GPU before it executes. After a cell updates a model, the executing replica serializes the updated state and synchronizes the other replicas. This synchronization runs in the idle gap between cells. When it takes longer than the gap, the next cell waits for the remainder. • Modal [12] is a production serverless GPU platform that also provides a serverless notebook service. It keeps one kernel per session and reclaims the kernel after a default 10-minute idle timeout. The reclamation discards the session state. A later cell must reboot the notebook server and rerun all upstream cells. • Naive serverless runs each GPU cell as a plain serverless GPU function: it starts a fresh container, initializes CUDA, cold-imports its frameworks, and transfers the whole live namespace in; after execution, it writes the whole namespace back. This configuration equals LooseLeaf with the dependency graph, the checkpoint store, and the import cache all disabled. Metrics. For provisioning, we report GPU-hours, the time integral of provisioned GPUs at the whole-host granularity of eight GPUs, and the peak GPU count; both are compared with the GPUs actually executing cells. For interactivity, we report user-visible latency, from pressing Run until the cell begins execution, excluding cell execution time. Following §1, we use 1 s as the interactive-start threshold and assess p95 interactivity by whether at least 95% of cell runs begin within this threshold. To explain these results, we report preparation latency (§2.2), the time required to make a GPU worker

ready, relative to the edit window, and save time, the post-execution checkpoint cost, relative to the inter-cell interval. Save time also determines how quickly the worker can be reclaimed.

4.2

GPU Provisioning at Scale

Figure 9 reports the GPUs each baseline holds as the load grows from 24 to 72 concurrent sessions. Provisioning acts at whole-host granularity, so powering one host on adds eight GPUs. The green line gives the ground truth, the number of GPUs executing cells at each instant. Its peak stays at 9, 15, and 20 GPUs across the three loads, because sessions spend most of their time thinking and editing between cells. LooseLeaf binds a GPU to a cell rather than to a session. Between cells, it holds only warm workers, and a warm worker is evictable at any time because the session state persists in the checkpoint store. The worker-management policy (§3.4) concentrates these workers on a few hosts. Consolidation packs new workers onto the most-loaded hosts, so lightly loaded hosts drain. Square-root staffing boots hosts ahead of the arrival burst, and a drained host that stays idle is powered off. These policies appear as the steps up to 16 or 24 GPUs and back down to 8. As the load triples, the peak provisioning of LooseLeaf grows only from 16 to 24 GPUs. Session-pinned reservation scales with the number of sessions rather than with this demand. Every session holds one GPU from arrival until the session ends, so the reserved peak follows the session count, at 24, 48, and 72 GPUs. LooseLeaf provisions 43%, 63%, and 70% fewer GPU-hours across the three loads. The saving grows with the load, because the reservation cost grows linearly while LooseLeaf tracks actual use. NotebookOS holds the same 24 GPUs at every load. The three replicas of a session must reside on three distinct hosts, so three powered hosts form its floor. The replicas are resident in CPU memory and block scale-in, so the host count never falls. At 24 sessions most of this floor sits idle, and LooseLeaf holds 55% fewer GPU-hours. At 72 sessions the demand grows into the floor, and the saving narrows to 32%. Modal bounds each session reservation with the 10-minute idle timeout. A GPU returns only when a pause exceeds the timeout, so Modal recovers the long pauses and holds the GPU through the rest of the think time. Its peak reaches 24, 40, and 56 GPUs, 10

CDF (GPU cells)

0.75 0.50

interactive-start threshold

0.25 0.00 ≈0

0.1s

Session-pinned LooseLeaf

CDF (GPU cells)

keep the preparation within the window. For 79% of GPU cells, the session finds its warm worker in the worker cache, and the worker restores only the variables that changed since the last run. The remaining cells fork a fresh worker from the import cache and restore the state closure from the checkpoint store. This cold preparation is itself small, with a median below one second (§4.4). The residual tail comes from the cells whose edit window is shorter than this cold preparation. The user runs such a cell before the fork and restore complete, and the unfinished remainder is exposed. The second source is also absent in LooseLeaf: a GPU worker saves the updated variables immediately after execution, and the incremental save completes long before the next run (§4.6). NotebookOS suffers from both sources. Its preparation is reactive: no transfer can start before Run, and without a closure the executor moves every model the session has accumulated, so the exposed transfer grows with the session. The median is 0.33 s and p95 is 4.4 s. The second source dominates the tail. After each cell that updates a model, NotebookOS serializes the full model state and writes it out between cells. The save of LooseLeaf, in contrast, is incremental. Dirty tracking refreshes only the changed regions of the updated model, and the copy runs as pinned DMA into the checkpoint store (§3.2). On the notebooks that accumulate tens of gigabytes of state, this synchronization outlasts the gap between two runs, and the next run waits for the remainder. The tail reaches 63 s at p99 and 194 s at the maximum. Only 65% of its runs start within the threshold. Modal starts a GPU cell immediately while its container is warm, which covers 78% of GPU cells. After an idle shutdown, the next run reboots the endpoint and reruns all upstream cells. This recomputation takes 46 s at p90 and exceeds an hour in the tail. Naive serverless pays both sources on every cell. After Run, it starts a fresh container, initializes CUDA, imports the frameworks from scratch, and transfers the whole live namespace into the worker, for a median of 41 s. After execution, it writes the whole namespace back, and this write-back delays the next run in the same way. No run of naive serverless starts within the threshold. Each of these costs corresponds to one mechanism of LooseLeaf: the import cache removes the container start and the imports, the state closure shrinks the transfer, and the incremental save shrinks the write-back. LooseLeaf is the only design that keeps both sources near zero while keeping the GPU reclaimable between cells. The latency stays low as the load grows. As the sessions triple, the peak GPU count that LooseLeaf provisions grows only from 16 to 24 (§4.2). Three times the sessions share this nearly unchanged capacity, so warm workers are evicted more often and the hit rate falls from 79% to 48%. More GPU cells therefore prepare on a fresh worker. Figure 10(b) shows that the cost stays small. At 72 sessions, 86% of GPU cells still start immediately, and p95 stays at 0.88 s. Even at 72 sessions, 95.9% of runs start within the interactive-start threshold. The residual tail is bounded by the time to power on a node, which a demand spike occasionally pays before the autoscaler catches up. The GPU savings of §4.2 therefore do not come at the cost of interactivity.

(a) All designs, N = 24 sessions

1.00

1s

10s

NotebookOS Modal Notebooks

1m

10m

1h

Naive serverless

(b) LooseLeaf, N = 24 / 48 / 72 sessions

1.00

interactive-start threshold

0.75

N=24 (98% ≈0, p95 ≈0) N=48 (93% ≈0, p95 0.47s) N=72 (86% ≈0, p95 0.88s)

0.50 ≈0

10ms

0.1s

1s

10s

User-visible latency (log scale)

Figure 10: User-visible latency over the GPU cells (log scale). (a) All designs with 24 concurrent sessions. The curve of LooseLeaf coincides with the session-pinned ideal at the left edge. (b) LooseLeaf as the load grows to 48 and 72 sessions. The vertical dashed line marks the one-second interactivestart threshold. and LooseLeaf provisions 29%, 33%, and 47% fewer GPU-hours. A shorter timeout would recover more idle time, but every expiry discards the session state, and the next cell pays the recomputation that §4.3 reports. LooseLeaf avoids this tradeoff. The session state persists in the checkpoint store, so a warm worker is evictable. Among the four baselines, only the provisioning of LooseLeaf follows the actual use. Session-pinned reservation and Modal provision by the session count, and NotebookOS provisions a static floor. The remaining headroom of LooseLeaf above the ground truth comes from the whole-host granularity, the square-root staffing, and the warm workers held between cells. Next, §4.3 shows that this saving does not come at the cost of interactivity.

4.3

End-to-End Interactive Latency

Figure 10(a) reports the distribution of user-visible latency over the GPU cells, with 24 concurrent sessions under the AI-assisted edit window. In every design, the latency of a GPU cell comes from two sources. The first source is preparation that begins only after the user presses Run. The second source is unfinished work of the previous cell, which the new run must wait for. The five designs differ in how much of each source they expose to the user. LooseLeaf removes the first source by preparing during editing. The figure shows that the uncovered part is negligible: 98% of GPU cells begin executing immediately after Run. The exposed latency stays below 1.1 s at p99 and 2.0 s at the worst case. 98.9% of GPU-cell runs start within the interactive-start threshold. Two mechanisms 11

20k

bring-up 100%

19,558s

(b) Per-cell prep vs. window

import state

15k 5k

CDF (GPU cells)

Total cold path (s)

(a) Ablation

4,740s

50%

3,080s

Naïve +DAG-min +Checkpoint +Import Cache edit window

2k 449s

0

0 in int che ïve Na AG-m ckpo t Ca r e D + Ch mpo + I +

0.3s 1s 3s 10s

1m

1.00 0.75 0.50

LooseLeaf GCR image cuda-ckpt + CRIU

0.25 0.00 0.3s 0.5s

1s

2s

5s

10s

30s

Per-cell cold preparation (log scale)

Figure 12: Comparing LooseLeaf’s restore cost with state-ofthe-art transparent checkpoint/restore approaches.

5m

Per-cell time (log scale)

1.00

Figure 11: (a) Cold-path preparation summed over the workload as the mechanisms are enabled cumulatively, split into worker bring-up, module import, and state transfer/restore. (b) The per-cell distribution of the cold preparation at each step, against the distribution of the edit window.

CDF

0.75

0.25 0.00 10ms

DINOv3-L ESMC-300M Mistral-7B Qwen3-8B

4.4

0.1s

1s

10s

1m

10m

1h

Post-execution save time (log scale)

Table 3: Checkpoint and restore of four model states from the workload. The full save is the first save of the state and writes it whole. The incremental save is issued by the cells after the first and refreshes only the changed regions. Model

0.50

LooseLeaf NotebookOS

Naive serverless Inter-cell interval

Figure 13: Post-execution save time of each design against the inter-cell interval of the production trace.

State size Full save Restore Incr. save 1.2 GB 2.7 GB 14.5 GB 15.1 GB

0.18 s 0.41 s 1.57 s 1.63 s

0.29 s 0.48 s 2.03 s 2.04 s

the per-cell preparation distribution of each step against the distribution of the edit window, which every cell faces alike. Under the naive configuration most preparations outlast most windows, so the preparation cannot hide in the window. Each mechanism shifts the preparation distribution left. In the full configuration, preparation has a maximum of 6.9 s, while the edit-window distribution has a median of 10.7 s, indicating substantial opportunity to hide preparation during editing. Note it does not imply that every cold preparation is fully hidden: the figure does not pair each cell’s preparation time with its realized edit window, and a long preparation may coincide with a shorter window.

29 ms 47 ms 66 ms 19 ms

Why Preparation Fits the Edit Window

§4.3 shows that both latency sources stay near zero for LooseLeaf. The rest of the evaluation explains why. This section covers the first source, preparation. LooseLeaf performs it during editing, so it stays invisible only when it finishes within the edit window. The measurements release the GPU worker after every cell, so each GPU cell prepares on a fresh worker and pays the whole cold path. Figure 11(a) enables the three mechanisms one after another and reports the cold preparation of the whole workload at each step. The naive configuration spends 19,558 s, and 15,951 s of it moves the whole live namespace. Dependency-guided minimization restores only the state closure of each cell and imports only the packages the cell needs. It cuts the state term by 8.9× and the total by 4.1×. Enabling the checkpoint store moves the large state payloads at high speed, and the state term falls to 124 s. The import cache removes the repeated imports, so the import term falls 18× and the total reaches 449 s. The three steps shrink the cold path by 44×. Table 3 reports these costs on single model states. A restore is dominated by the DMA: reloading a 14–15 GB model takes about two seconds at 7.6–8.1 GB/s. The transfer runs over the host-toGPU PCIe path of the instance, whose measured pinned-memory bandwidth peak is 12.4 GB/s. The remaining time reconstructs the Python object structure. With all three mechanisms enabled, the cold preparation of a cell has a median of 0.71 s and a p95 of 3.8 s. Figure 11(b) places

4.5

Comparison with Transparent GPU Checkpoint/Restore

Dependency-guided restore is not the only way to prepare a GPU worker. Transparent checkpoint/restore offers an alternative. CRIU [3] with cuda-checkpoint [22] saves the kernel process together with its GPU memory and restores both on demand. GCR [68] is a recent GPU checkpoint system that restores GPU memory at close to the PCIe bandwidth. In both approaches, restoring a GPU cell first starts the container. The restore then loads the process memory and the GPU memory that the session has accumulated. We estimate this per-cell restore time from the measured memory sizes and transfer rates. Figure 12 shows the per-cell preparation time. The median preparation of LooseLeaf is 0.71 s. The GCR-based restore has a median of 2.50 s. The CRIU-based restore has a median of 4.38 s. The difference comes from the amount of state that each side restores. The checkpoint-based approaches restore the entire session. They also pay a fixed cost for the container start before any data moves. LooseLeaf restores only the state closure and the import closure of the target cell. The checkpoint-based approaches are faster only 12

at the tail. The slowest cell of LooseLeaf takes 6.9 s. Most of this time imports bitsandbytes, which is not a CUDA-safe package and cannot be optimized by the import cache.

4.6

systems are transparent. They capture the whole address space and cannot tell which part a cell needs, so a restore always carries the full session. LooseLeaf restores only the state closure of the target cell and preserves object sharing across the restored variables. The difference in preparation time is quantified in §4.5. Serverless data systems and cold start. Interactive data systems increasingly run on serverless platforms. Starling [57] and Lambada [50] run analytics on cloud functions, Cloudburst [61] colocates a cache with function executors, Vexless [62] keeps vector indexes in stateful functions, and Azure SQL Database Serverless resumes paused databases ahead of the next request [59]. General platforms reduce the start delay of any function with prebuilt snapshots [34, 64], predicted keep-alive windows [60], and snapshots of an initialized CUDA context [24]. These systems cannot determine the state that the next request requires until the request arrives, so they either hold the complete state ready or obtain it on the critical path. In a notebook, the code of the next cell is available before execution, and LooseLeaf prepares only the closure of that cell during editing, before the user requests execution.

Why the Save Fits the Interval

The second latency source of §4.3 is the save of the previous cell. After a cell executes, LooseLeaf saves the updated state to the checkpoint store, and this save must finish before the next run needs the state. The save of LooseLeaf is incremental at two levels (§3.2): it saves only the variables that the cell updated, and within a GPU tensor, dirty tracking refreshes only the 2 MB regions that changed since the previous checkpoint. An incremental save scans the state for the changed regions and transfers only those regions. Table 3 shows the effect: the full save of a 15 GB state takes 1.6 s at 9.6 GB/s of D2H DMA, and the incremental save of a later cell takes tens of milliseconds. NotebookOS saves the complete model and optimizer state on every synchronization. Naive serverless saves the whole live namespace after every cell. Figure 13 compares the three saves against the interval they must fit into, the time between consecutive runs in the production trace. Session-pinned reservation and Modal keep state only in the running kernel and write nothing back. The incremental save of LooseLeaf has a median of 29 ms and never exceeds 1.8 s. The smallest interval in the trace is 60 s, so no save of LooseLeaf delays the next run in this workload. The saves of NotebookOS and naive serverless cross into the interval distribution. The save of NotebookOS reaches 109 s at p95 and 254 s at the maximum, which is 177× the p95 of LooseLeaf, and it overflows 4.6% of the trace intervals. The save of naive serverless overflows 6.8%. These overflowing saves are the source of the latency tails in §4.3. A fast save also matters for provisioning. A warm worker is evictable only after its save completes, because eviction must not lose state. The save of LooseLeaf finishes within seconds of every cell, so every warm worker becomes evictable almost immediately. This enables the demand-following provisioning of §4.2.

5

6

CONCLUSION

This paper presented LooseLeaf, a novel computational notebook platform with serverless GPUs. A notebook cell needs only a small part of the session state, which can be derived from the code while the user edits the cell. LooseLeaf captures this part as the cell’s state and import closures. It prepares a GPU worker with these closures during editing and saves the updated state within seconds after execution. GPUs are therefore allocated to cells on demand rather than pinned to sessions, reducing idle GPU time and improving GPU utilization.

REFERENCES [1] [n.d.]. About the Zygote processes. https://source.android.com/docs/core/ runtime/zygote. [2] [n.d.]. Amazon SageMaker notebook instances. https://docs.aws.amazon.com/ sagemaker/latest/dg/nbi.html. [3] [n.d.]. CRIU: Checkpoint/Restore In Userspace. https://criu.org. [4] [n.d.]. CUDA C++ Programming Guide v11.8, §N.2.5.2: Using fork() with Managed Memory. https://docs.nvidia.com/cuda/archive/11.8.0/cuda-c-programmingguide/index.html#um-fork-managed-memory. [5] [n.d.]. Databricks notebooks. https://docs.databricks.com/en/notebooks/index. html. [6] [n.d.]. GitHub. https://github.com. [7] [n.d.]. Google Colab: A hosted Jupyter Notebook service. https://colab.research. google.com. [8] [n.d.]. Horizontal Pod Autoscaling. https://kubernetes.io/docs/tasks/runapplication/horizontal-pod-autoscale/. [9] [n.d.]. HugeTLB Pages. https://www.kernel.org/doc/html/latest/admin-guide/ mm/hugetlbpage.html. [10] [n.d.]. Kaggle Notebooks. https://www.kaggle.com/code. [11] [n.d.]. Modal documentation: Web Functions: How do Web Functions run in the cloud? https://modal.com/docs/guide/webhooks#how-do-web-functions-runin-the-cloud. [12] [n.d.]. Modal: High-performance AI infrastructure. https://modal.com. [13] [n.d.]. Modal Notebooks. https://modal.com/docs/guide/notebooks. [14] [n.d.]. pickle — Python object serialization. https://docs.python.org/3/library/ pickle.html. [15] [n.d.]. Pyodide: A Python distribution for the browser and Node.js based on WebAssembly. https://pyodide.org. [16] [n.d.]. Redis. https://redis.io. [17] [n.d.]. Run Jupyter notebooks in your workspace (Azure Machine Learning). https://learn.microsoft.com/en-us/azure/machine-learning/how-to-runjupyter-notebooks. [18] [n.d.]. Runpod Serverless GPU Platform. https://www.runpod.io/serverless-gpu. [19] [n.d.]. Tensor Views. https://pytorch.org/docs/stable/tensor_view.html.

RELATED WORK

NotebookOS [28] allocates GPUs to cells and is the strongest baseline in §4. This section covers the remaining related work. Notebook state management. ElasticNotebook [44] checkpoints session state for live migration and chooses between storing and recomputing each variable. Kishu [43] versions the session state and restores a past state through incremental checkout. Chipmink [29] partitions the object graph into cost-based pods and persists only the pods that changed. nbsafety [46] traces lineage across cells to warn about stale results. These systems manage the CPU state of a session that runs in one long-lived kernel, and their checkpoints are read back at a failure, a checkout, or a migration. LooseLeaf extends this line of work to GPU-resident state that moves between workers. The store sits on the execution path. Every cell restores its closure from the store and saves its updated variables back (§3.2). Transparent GPU checkpoint and restore. CRIU [3] with cudacheckpoint [22] snapshots a process together with its GPU state. PhoenixOS [65] checkpoints GPU applications concurrently with their execution. GCR [68] restores GPU memory near the PCIe bandwidth through incremental checkpoints of dirty regions. These 13

[20] 2000. A Memory Allocator. https://gee.cs.oswego.edu/dl/html/malloc.html. [21] 2018. PEP 574 – Pickle protocol 5 with out-of-band data. https://peps.python. org/pep-0574/. [22] 2024. CUDA Checkpoint/Restore Utility (cuda-checkpoint). https://github.com/ NVIDIA/cuda-checkpoint. [23] 2025. Grok Code Fast 1 Model Card. https://data.x.ai/2025-08-26-grok-codefast-1-model-card.pdf. [24] 2026. How we achieved truly serverless GPUs. https://modal.com/blog/trulyserverless-gpus. [25] Akshay Agrawal and Myles Scolnick. 2023. marimo: An open-source reactive notebook for Python. https://github.com/marimo-team/marimo. https://doi. org/10.5281/zenodo.12735329 [26] Saleema Amershi, Andrew Begel, Christian Bird, Robert DeLine, Harald C. Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. 2019. Software Engineering for Machine Learning: A Case Study. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2019). IEEE / ACM, 291–300. https://doi.org/10.1109/ICSESEIP.2019.00042 [27] Michel L. Balinski and H. Peyton Young. 2001. Fair Representation: Meeting the Ideal of One Man, One Vote (2nd ed.). Brookings Institution Press. [28] Benjamin Carver, Jingyuan Zhang, Haoliang Wang, Kanak Mahadik, and Yue Cheng. 2026. NotebookOS: A Replicated Notebook Platform for Interactive Training with On-Demand GPUs. In Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 (ASPLOS ’26). Association for Computing Machinery, New York, NY, USA, 183–202. https://doi.org/10.1145/3760250.3762230 [29] Supawit Chockchowwat, Sumay Thakurdesai, Zhaoheng Li, Matthew S. Krafczyk, and Yongjoo Park. 2025. Chipmink: Efficient Delta Identification for Massive Object Graphs. Proceedings of the VLDB Endowment 19, 4 (2025), 603–616. https: //doi.org/10.14778/3785297.3785303 [30] Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 20, 1 (1960), 37–46. https://doi.org/10.1177/ 001316446002000104 [31] Willem de Bruijn and Eric Dumazet. 2017. sendmsg copy avoidance with MSG_ZEROCOPY. In Netdev 2.1, The Technical Conference on Linux Networking. Montreal, Canada. https://netdevconf.org/2.1/papers/debruijn-msgzerocopytalk.pdf [32] Vivek Dhakal, Anna Maria Feit, Per Ola Kristensson, and Antti Oulasvirta. 2018. Observations on Typing from 136 Million Keystrokes. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174220 [33] Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020. Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping. arXiv:2002.06305 [cs.CL] https://arxiv.org/abs/2002.06305 [34] Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qixuan Wu, and Haibo Chen. 2020. Catalyzer: Sub-millisecond Startup for Serverless Computing with Initialization-less Booting. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’20). Association for Computing Machinery, 467–481. https://doi.org/10.1145/3373376.3378512 [35] Yao Fu, Leyang Xue, Yeqi Huang, Andrei-Octavian Brabete, Dmitrii Ustiugov, Yuvraj Patel, and Luo Mai. 2024. ServerlessLLM: Low-Latency Serverless Inference for Large Language Models. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, Santa Clara, CA, 135–153. https://www.usenix.org/conference/osdi24/presentation/fu [36] Gemini Team, Google. 2023. Gemini: A Family of Highly Capable Multimodal Models. arXiv:2312.11805 [cs.CL] https://arxiv.org/abs/2312.11805 [37] Gemma Team. 2026. Gemma 4 Technical Report. arXiv:2607.02770 [cs.CL] https://arxiv.org/abs/2607.02770 [38] Shlomo Halfin and Ward Whitt. 1981. Heavy-Traffic Limits for Queues with Many Exponential Servers. Operations Research 29, 3 (1981), 567–588. https: //doi.org/10.1287/opre.29.3.567 [39] Joseph M. Hellerstein, Jose M. Faleiro, Joseph Gonzalez, Johann Schleier-Smith, Vikram Sreekanti, Alexey Tumanov, and Chenggang Wu. 2019. Serverless Computing: One Step Forward, Two Steps Back. In 9th Biennial Conference on Innovative Data Systems Research (CIDR 19). www.cidrdb.org, Asilomar, CA. https://vldb.org/cidrdb/2019/serverless-computing-one-step-forward-twosteps-back.html [40] Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, Neeraja Yadwadkar, Joseph E. Gonzalez, Raluca Ada Popa, Ion Stoica, and David A. Patterson. 2019. Cloud Programming Simplified: A Berkeley View on Serverless Computing. arXiv:1902.03383 [cs.OS] https://arxiv.org/abs/1902.03383 [41] Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian E. Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica B. Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Damián Avila, Safia Abdalla, Carol

Willing, and Jupyter Development Team. 2016. Jupyter Notebooks – a publishing format for reproducible computational workflows. In Positioning and Power in Academic Publishing: Players, Agents and Agendas, 20th International Conference on Electronic Publishing. IOS Press, Göttingen, Germany, 87–90. https://doi.org/10.3233/978-1-61499-649-1-87 [42] J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 1 (1977), 159–174. https: //doi.org/10.2307/2529310 [43] Zhaoheng Li, Supawit Chockchowwat, Ribhav Sahu, Areet Sheth, and Yongjoo Park. 2024. Kishu: Time-Traveling for Computational Notebooks. Proceedings of the VLDB Endowment 18, 4 (2024), 970–985. https://doi.org/10.14778/3717755. 3717759 [44] Zhaoheng Li, Pranav Gor, Rahul Prabhu, Hui Yu, Yuzhou Mao, and Yongjoo Park. 2023. ElasticNotebook: Enabling Live Migration for Computational Notebooks. Proceedings of the VLDB Endowment 17, 2 (2023), 119–133. https://doi.org/10. 14778/3626292.3626296 [45] Qianou Ma, Kenneth R Koedinger, and Tongshuang Wu. 2026. Not Everyone Wins with LLMs: Behavioral Patterns and Pedagogical Implications for AI Literacy in Programmatic Data Science. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (Barcelona, Spain) (CHI ’26). Association for Computing Machinery, New York, NY, USA, Article 139, 22 pages. https: //doi.org/10.1145/3772318.3791283 [46] Stephen Macke, Hongpu Gong, Doris Jung-Lin Lee, Andrew Head, Doris Xin, and Aditya Parameswaran. 2021. Fine-Grained Lineage for Safer Notebook Interactions. Proceedings of the VLDB Endowment 14, 6 (2021), 1093–1101. https: //doi.org/10.14778/3447689.3447712 [47] Robert B. Miller. 1968. Response time in man-computer conversational transactions. In Proceedings of the AFIPS ’68 Fall Joint Computer Conference, Part I (AFIPS Conference Proceedings), Vol. 33. 267–277. https://doi.org/10.1145/1476589. 1476628 [48] Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, and Ion Stoica. 2018. Ray: A Distributed Framework for Emerging AI Applications. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, Carlsbad, CA, 561–577. https://www.usenix. org/conference/osdi18/presentation/nishihara [49] Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow. 2021. On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines. In 9th International Conference on Learning Representations (ICLR 2021). OpenReview.net. https://openreview.net/forum?id=nzpLWnVAyah [50] Ingo Müller, Renato Marroquín, and Gustavo Alonso. 2020. Lambada: Interactive Data Analytics on Cold Data Using Serverless Cloud Infrastructure. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 115– 130. https://doi.org/10.1145/3318464.3389758 A study on tolerable waiting time: how [51] Fiona Fui-Hoon Nah. 2004. long are Web users willing to wait? Behaviour & Information Technology 23, 3 (2004), 153–163. https://doi.org/10.1080/01449290410001669914 arXiv:https://doi.org/10.1080/01449290410001669914 [52] Jakob Nielsen. 1993. Usability Engineering. Morgan Kaufmann, San Francisco, CA. [53] Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-Optimized Containers. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 57–70. https://www.usenix.org/conference/atc18/presentation/oakes [54] OpenAI. 2025. gpt-oss-120b & gpt-oss-20b Model Card. arXiv:2508.10925 [cs.CL] https://arxiv.org/abs/2508.10925 [55] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019). 8024–8035. https://proceedings.neurips.cc/paper/2019/hash/ bdbca288fee7f92f2bfa9f7012727740-Abstract.html [56] Jeffrey M. Perkel. 2018. Why Jupyter is data scientists’ computational notebook of choice. Nature 563 (2018), 145–146. https://doi.org/10.1038/d41586-018-07196-1 [57] Matthew Perron, Raul Castro Fernandez, David J. DeWitt, and Samuel Madden. 2020. Starling: A Scalable Query Engine on Cloud Functions. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 131–141. https://doi.org/10.1145/3318464.3380609 [58] João Felipe Pimentel, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019. A Large-Scale Study About Quality and Reproducibility of Jupyter Notebooks. In Proceedings of the 16th International Conference on Mining Software Repositories (MSR 19). IEEE / ACM, Montreal, Canada, 507–517. https: //doi.org/10.1109/MSR.2019.00077 [59] Olga Poppe, Pankaj Arora, Sakshi Sharma, Jie Chen, Sachin Pandit, Rahul Sawhney, Vaishali Jhalani, Willis Lang, Qun Guo, Anupriya Inumella, Sanjana Dulipeta 14

Sridhar, Dheren Gala, Nilesh Rathi, Morgan Oslake, Alexandru Chirica, Sarika Iyer, Prateek Goel, and Ajay Kalhan. 2024. Proactive Resume and Pause of Resources for Microsoft Azure SQL Database Serverless. In Companion of the 2024 International Conference on Management of Data (SIGMOD ’24). 227–240. https://doi.org/10.1145/3626246.3653371 [60] Mohammad Shahrad, Rodrigo Fonseca, Iñigo Goiri, Gohar Irfan Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 205–218. https: //www.usenix.org/conference/atc20/presentation/shahrad [61] Vikram Sreekanti, Chenggang Wu, Xiayue Charles Lin, Johann Schleier-Smith, Joseph E. Gonzalez, Joseph M. Hellerstein, and Alexey Tumanov. 2020. Cloudburst: Stateful Functions-as-a-Service. Proceedings of the VLDB Endowment 13, 11 (2020), 2438–2452. https://doi.org/10.14778/3407790.3407836 [62] Yongye Su, Yinqi Sun, Minjia Zhang, and Jianguo Wang. 2024. Vexless: A Serverless Vector Data Management System Using Cloud Functions. Proc. ACM Manag. Data 2, 3, Article 187 (2024), 26 pages. https://doi.org/10.1145/3654990 [63] Sergey Titov, Konstantin Grotov, Cristina Sarasua, Yaroslav Golubev, Dhivyabharathi Ramasamy, Alberto Bacchelli, Abraham Bernstein, and Timofey Bryksin. 2026. Observing Fine-Grained Changes in Jupyter Notebooks During Development Time. Journal of Systems and Software 241 (2026), 112983. https://doi. org/10.1016/j.jss.2026.112983 Dataset: https://huggingface.co/datasets/JetBrainsResearch/JuNE. [64] Dmitrii Ustiugov, Plamen Petrov, Marios Kogias, Edouard Bugnion, and Boris Grot. 2021. Benchmarking, Analysis, and Optimization of Serverless Function

Snapshots. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’21). Association for Computing Machinery, 559–572. https://doi.org/10.1145/ 3445814.3446714 [65] Xingda Wei, Zhuobin Huang, Tianle Sun, Yingyi Hao, Rong Chen, Mingcong Han, Jinyu Gu, and Haibo Chen. 2025. PhoenixOS: Concurrent OS-level GPU Checkpoint and Restore with Validated Speculation. In Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles (SOSP ’25). Association for Computing Machinery, 996–1013. https://doi.org/10.1145/3731569.3764813 [66] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https://arxiv.org/abs/2505.09388 [67] Neal E. Young. 1994. The k-Server Dual and Loose Competitiveness for Paging. Algorithmica 11, 6 (June 1994), 525–541. https://doi.org/10.1007/BF01189992 [68] Shaoxun Zeng, Tingxu Ren, Jiwu Shu, and Youyou Lu. 2026. GPU Checkpoint/Restore Made Fast and Lightweight. In 24th USENIX Conference on File and Storage Technologies (FAST ’26). USENIX Association, Santa Clara, CA, 239–254.

15

Related documents

• LooseLeaf: Enabling Serverless GPUs for Computational Notebooks#1455756
• uGDS: A User-Space Development Kit for High-Performance GPU Direct Storage#1423986
• uGDS: A User-Space Development Kit for High-Performance GPU Direct Storage#1417808
• The Token-Energy Law of LLM Decode: Active Bytes and Realized Time Account for Joules per Token#1412633
• The Token-Energy Law of LLM Decode: Active Bytes and Realized Time Account for Joules per Token#1394205