LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data Debopam Sanyal
Hongjie Chen
Alexey Tumanov
Joshua Kimball
Georgia Tech
Dolby Labs
Georgia Tech
Dolby Labs
arXiv:2607.28880v1 [cs.DC] 30 Jul 2026
Abstract Modern multimedia machine learning workloads increasingly store large-scale datasets in cloud object storage services such as AWS S3. How these samples are physically organized in storage (i.e., storage layout) directly affects how quickly and cheaply they can be retrieved. Yet the benchmarks used to guide storage decisions today focus on database engines and query processing, and none systematically evaluates how different storage layouts perform for multimedia data retrieval. We present LayoutBench, the first benchmark designed to fill this gap. It evaluates three representative layout strategies: storing each sample as an individual object (L1), sequentially packing samples into tar archives (L2), and organizing samples as columns in Parquet files (L3). We measure retrieval time, data transferred, and monetary cost using 11 queries of varying result-set sizes on ImageNet across six AWS EC2 instance configurations that span different network bandwidth and memory tiers. Our experiments reveal that L2 achieves lower latency than L1 and L3 through connection reuse, but loses this advantage as retrieval sizes become very large. L3 is the fastest for very large retrievals but transfers substantially more data across all query sizes due to row-group granularity, and requires significantly more memory. Across all layouts, data transfer cost dominates total expenditure, with L3 costing an order of magnitude more than L1 or L2.
CCS Concepts • Software and its engineering → Software design techniques.
Keywords Storage Layouts, Benchmarks ACM Reference Format: Debopam Sanyal, Hongjie Chen, Alexey Tumanov, and Joshua Kimball. 2026. LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data. In The 6th Workshop on Machine Learning and Systems (EuroMLSys ’26), April 27–30, 2026, Edinburgh, Scotland Uk. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3805621.3807610
1
Introduction
Modern machine learning pipelines depend on large-scale datasets of unstructured data such as image, audio, and video [22, 26, 27, 33, 37]. These datasets are increasingly stored in cloud object storage services such as AWS S3 [4, 6, 41], which serve as the primary persistence layer for training and inference workloads [8, 17, 19, 32]. Before any model can train, the relevant data samples must be
This work is licensed under a Creative Commons Attribution 4.0 International License. EuroMLSys ’26, Edinburgh, Scotland Uk © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2605-7/26/04 https://doi.org/10.1145/3805621.3807610
retrieved from cloud storage and delivered to compute nodes, a process whose efficiency directly affects GPU utilization, pipeline throughput, and infrastructure cost. How samples are grouped, packed, and organized in cloud storage what we term the storage layout, fundamentally determines the retrieval performance and cost of this data loading step. Yet, practitioners today must choose among diverse storage layout strategies with no benchmark to systematically compare their retrieval performance, data transfer overhead, and cost under realistic workloads. Existing data systems benchmarks do not fill this gap. Benchmarks for OnLine Transaction Processing (OLTP) [35, 41], OnLine Analytical Processing (OLAP) [18], time-series databases [12, 14, 34], and other workloads [15, 24, 39, 40] measure query latency, throughput, and scalability under a fixed storage abstraction, assuming that the underlying data organization is either given or outside the scope of evaluation. These benchmarks target structured, tabular data and do not capture the distinct access patterns of multimedia ML workloads, where individual data samples are large binary objects (e.g., JPEG images, audio files) retrieved using metadata predicates rather than relational joins. As a result, the performance tradeoffs in storing and retrieving such data are governed by factors absent from existing benchmarks. Prior work has studied related but distinct aspects of cloud storage. Benchmarks such as COSBench [42] and CNSBench [20] evaluate storage services (e.g., throughput and latency of object stores) but do not vary the data layout within a given service. LavaStore [31] optimizes key-value storage layouts, but targets structured workloads rather than multimedia retrieval. Zeng et al. [38] compare columnar formats (primarily Parquet vs. ORC) but focus on relational analytics rather than multimedia retrieval. Pixels [7] optimizes storage layouts for relational data, and Delta Lake [5, 10] stores data as Parquet files with a co-located transaction log containing data-skipping statistics, but its primary concern is change tracking and ACID transactions rather than retrieval performance. BigLake [16] unifies storage and analytics at scale but does not benchmark alternative layout strategies against one another. On the ML systems side, WebDataset [2] and MosaicML Streaming [29] adopt sequential packing into shards, while Petastorm [1] and Lance [23] use columnar storage, yet none provides a controlled performance comparison of storage layout strategies for multimedia retrieval on cloud object storage with cost analysis. We propose LayoutBench, a benchmark framework for evaluating cloud storage layouts for multimedia ML datasets. LayoutBench targets data sample retrieval: fetching specific subsets of a dataset from cloud storage based on predicate conditions. Examples include retrieving training images of a particular class, loading audio segments that meet a size constraint, or selecting video clips matching resolution criteria. Such retrieval operations are pervasive in production systems, from serving image results on platforms like
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
Storage Layouts
Every storage layout must manage two kinds of information: metadata (attributes such as class label, file size, width, and height that are used to select samples, i.e., images) and the raw sample data itself (i.e., image bytes). The three layouts we benchmark differ in how they co-locate or separate these two concerns, and consequently in how retrieval is performed. Figure 1 illustrates the overall architecture. In all layouts, data resides in AWS S3 buckets and a local client EC2 instance runs a query engine that operates in two phases: ➀ a predicate search that consults metadata to identify which samples satisfy a query’s conditions, and ➁ a retrieval step that issues requests to S3 to fetch the matching samples. L1 and L2 perform these two phases separately: the predicate search returns sample locators (S3 URLs for L1, byte-range triplets for L2), and the retrieval step issues one HTTP request per locator. L3 combines both phases in a single DuckDB query that internally issues S3 API calls, requiring no separate metadata lookup. The bottom of the figure contrasts how each layout organizes data in S3: many individual objects (L1), fewer tar archives (L2), or a co-located Parquet table (L3). We describe each layout below.
2.1
Layout 1: Individual Sample Storage
Each sample is stored as an individual S3 object with a unique S3 URL. The client first queries the local metadata table to obtain the URLs of matching samples, then issues one HTTP GET request per URL to fetch each sample. Sec.A provides the query details for L1.
CLIENT EC2 INSTANCE
Metadata (Parquet) class, size, W, H, locators
Layout
L1
L2
L3
① Predicate Search (list of S3 URLs)
① Predicate Search (tar, offset, size)
①+② DuckDB Engine (predicate eval + sample fetch)
② HTTP GET
② RANGE GET
no separate metadata lookup
network boundary
AWS S3 BUCKET
Individual S3 Objects
Tar Archive Files
...
2
Queries (Q1 - Q11)
...
Pinterest to loading audio tracks on Spotify [25, 28]. Even with caching, indexing, and query optimization at higher layers, the organization of data at the storage layer directly affects retrieval latency, network transfer volume, and monetary cost [13, 21]. This paper makes the following contributions: (1) We propose an extensible benchmark into which new storage layouts, cloud backends, and datasets can be plugged. We validate it by implementing three representative layout strategies using AWS S3: individual object storage (L1), sequential packing into tar archives (L2), and columnar storage via Parquet (L3), spanning the design space from per-sample access to structured columnar retrieval. (2) Through experiments on ImageNet [9] with 11 representative retrieval queries across three dataset scales and six EC2 instance configurations, we provide an empirical characterization of the performance, data transfer, and cost tradeoffs of each layout. Our findings reveal that each layout is constrained by a different resource (L1 by per-request latency, L2 by network bandwidth, and L3 by memory), that data transfer accounts for over 98% of total cost, and that L2 offers the best latency-cost balance for image workloads. These results provide actionable guidance for ML practitioners configuring their storage layers.
Sanyal et al.
1 object per sample
packed into tar files
Parquet Table (metadata + samples) class
size
W
H
img_bytes row group (128 MB)
Figure 1: LayoutBench retrieval architecture. The dashed line marks the network boundary between the client EC2 instance and AWS S3. L1 and L2 perform a local metadata lookup (➀) followed by per-sample S3 requests (➁). L3 combines both steps in a single DuckDB query that internally issues S3 API calls. The bottom contrasts how each layout organizes data in S3: many individual objects (L1), fewer tar archives (L2), or a single co-located Parquet table (L3).
offset, and size. The client queries this table to obtain the (tar URL, offset, size) triplet for each matching sample, then issues one HTTP RANGE GET request per triplet, fetching only the relevant byte range from the corresponding tar file. Sec.B provides the query details for L2. Discussion: Other sequential formats such as WebDataset [2, 30, 36] and TFRecord prioritize streaming efficiency over random access retrieval. We select tar because it supports byte-range access, making it suitable for predicate-driven retrieval.
2.3
Layout 3: Samples in Columnar Storage
Unlike L1 and L2, Layout 3 stores the content bytes of each sample as a BLOB in a Parquet column alongside the metadata columns. The client issues SQL queries through a DuckDB connector, which internally performs predicate evaluation and sample fetching via low-level S3 API calls, i.e., no separate metadata lookup is needed. To DuckDB, all samples appear to reside in a single large Parquet table. Sec.C provides the complete SQL statements used in DuckDB.
Discussion: This is the most primitive storage layout. Existing literature [6, 7] has noted that retrieving individual files incurs high Time-to-First-Byte (TTFB) latency per request.
Discussion: Other columnar formats (e.g., ORC, Lance) and query engines (e.g., Apache Spark) can also serve this role [38]. We select Parquet and DuckDB due to their widespread adoption.
2.2
3
Layout 2: Sequential Sample (Tar) Storage
Samples are sequentially packed into tar files. The client-side metadata table additionally records each sample’s tar file URL, byte
Experimental Configuration
This section describes the configuration for datasets, queries, client instances and metrics. Additional details are provided in Sec.D.
LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data
22 1.36 GB
Client
Index File Size
810 KB
0
Server
Number of Files 128,066 28 28 File Size 13.91 GB 13.76 GB 13.57 GB
medium Client
Index File Size
Server
Number of Files 1,281,167 276 274 File Size 139.25 GB 137.73 GB 135.85 GB
Client
Index File Size
full
3.1
553 KB
4.4 MB
35.6 MB
6.8 MB
54.7 MB
0
0
Datasets
Our benchmarking is based on ImageNet-1K, one of the most widely used image datasets in the computer vision community, whose training subset contains 1.28 million images across 1,000 classes [9]. From this subset we derive three datasets at increasing scale: (1) mini, a 1% random sample of each class; (2) medium, a 10% random sample; and (3) full, the complete training subset. Table 1 summarizes the data statistics for each (dataset, layout) combination. Under L1, the number of files and total size correspond directly to the number of samples and raw dataset size, since each image is stored as an individual S3 object. Under L2 and L3, samples are packed into tar or Parquet files, with a per-file size limit of 64 MB for mini and 512 MB for medium and full. Note that the metadata information is stored in the client under L1 and L2.
3.2
Queries
We design 11 queries that reflect practical retrieval patterns in ML pipelines, where training typically involves filtering and selecting specific samples rather than scanning entire datasets [11]. Each query is a predicate search over one or more of four metadata attributes: image Class (C), file Size (S), Width (W), and Height (H). The queries fall into two groups, as summarized in Table 2. Atomic queries (Q1-Q6) apply a single predicate. Q1-Q5 filter on class using increasingly broad conditions: a single sample (Q1), a fixed batch (Q2), an entire class (Q3), a regex match across multiple classes (Q4), and a fan-out over three named classes (Q5). Q6, on the other hand, filters solely based on file size. The result sets for these queries range from one file (Q1) to 13,000 files (Q4) on the full dataset, spanning four orders of magnitude. Composite queries (Q7-Q11) combine two or more predicates. Q7 filters on both width and height; Q8 computes a cross-column ratio (width / height > 1.5), requiring evaluation over all width and height values; Q9 and Q10 combine class membership with a size constraint at different selectivities; and Q11 involves all four attributes. Result sets again span a wide range, from 32 files (Q11) to 229,710 files (Q8) on the full dataset. This design ensures coverage across three axes: predicate type (membership, comparison, regex, cross-column computation), predicate count (one through four), and result set size (1 to 229,710 files).
Client Instances
3.4
Metrics
We measure three metrics: (1) End-to-end Retrieval Time (𝑇 ): the elapsed time from the start of query execution on the client to the completion of all sample retrieval from S3. (2) Data Transferred (𝐷): the total bytes transferred from S3 to the client. (3) Cost Estimate (𝐶): the sum of S3 data transfer cost and EC2 rental cost, computed from AWS pricing [3] (detailed in §4.5).
4
Results
We present experimental results analyzing the three storage layouts across multiple dimensions: retrieval latency, data transfer efficiency, scalability, caching behavior, and cost. To account for run-to-run variability, experiments were repeated five times for the mini dataset and three times for the medium dataset, with error bars showing standard deviations. The full dataset was run once due to its scale.
Instance Types t3.medium
mini
23 1.38 GB
medium
Number of Files 12,756 File Size 1.39 GB
Server mini
Time T (in sec)
L3
full
L2
L1
L2
L3
c5.large
L3 (t3.large)
c8gb.large
4 3 2 1 0 4
×101
3 2 1 0
2.0 1.5 1.0 0.5 0.0
×102
Q1
Q2
Q3
Q4 Q10 Q11
Q1
Q2
Q3
Q4 Q10 Q11
Q1
Q2
Q3
Q4 Q10 Q11
L3
L3 (t3.large)
(a) Queries: Q1-Q4 & Q10-Q11. Instance Types ×101
t3.medium
L1
L2
c5.large
c8gb.large
3.2
mini
L1
2.4 1.6 0.8 0.0
medium
Attribute
3.3
We primarily evaluate on three types of EC2 instances: t3.medium, c5.large, and c8gb.large, due to their differences in network bandwidth. When memory becomes a bottleneck, we opt for three other EC2 instances, t3.large, t3.xlarge, and t3.2xlarge, which provide additional memory. Their specifications and prices are provided in Table 3.
Time T (in sec)
Dataset Location
Storage Layout Plan
×102 3.2 2.4 1.6 0.8 0.0 2.0
×103
1.5
full
Table 1: Dataset statistics at the storage server and client.
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
1.0 0.5 0.0
Q5
Q6
Q7
Q8
Q9
Q5
Q6
Q7
Q8
Q9
Q5
Q6
Q7
Q8
Q9
(b) Queries: Q5-Q9.
Figure 2: End-to-end retrieval time in seconds of three layouts (L1, L2, L3) on the EC2 instance types in Table 3 (per column) for three dataset scales (per row: mini, medium, full).
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
Sanyal et al.
Table 2: Summary of the queries. Each query performs a predicate search on one or more of four conditions: Image Class (C), Size (S), Width (W), and Height (H). Queries Q1–Q6 use a single condition, while Q7–Q11 involve two or more conditions. Q.
Predicate
Type C
Q1 Q2 Q3 Q4 Q5 Q6
One Sample Batch Fetch One Class Regex Match Fan-out File size Filter
Q7 Q8 Q9 Q10 Q11
Resolution Filter Cross-column Multi-label Filter Selective Filter All-column Filter
S
W
Query Description H
✓ ✓ ✓ ✓ ✓ ✓
✓ ✓ ✓
✓ ✓ ✓
✓ ✓
✓ ✓
✓
✓
Retrieval – Num. of Files mini
medium
full
one image of streetcar N images of lemon (𝑁 = 10(𝑚𝑖𝑛𝑖), 100(𝑚𝑒𝑑𝑖𝑢𝑚), 1000(𝑓 𝑢𝑙𝑙)) images of cello images whose labels are prefixed with snake, i.e., LIKE %snake images of Old_English_sheepdog, giant_panda, and water_buffalo images that are at least 500 KB
1 10 13 130 39 86
1 100 130 1,300 390 801
1 1,000 1,300 13,000 3,900 7,837
images whose width and height are both at least 1024 pixels images whose width / height ratio is greater than 1.5 images of canoe and starfish that are smaller than 100 KB images of tiger_cat that are smaller than 100 KB images of bagel under 200 KB with width and height at least 512 pixels
133 2,311 6 1 1
1,309 23,006 47 27 6
12,635 229,710 575 294 32
Table 3: Comparison of client EC2 instance types
per-request overhead compared to L1’s individual GET requests.
API Name
Network
Memory
vCPUs
Hourly Cost
t3.medium c5.large c8gb.large
Up to 5 Gigabit Up to 10 Gigabit Up to 20 Gigabit
4 GiB 4 GiB 4 GiB
2 vCPUs 2 vCPUs 2 vCPUs
$0.0416 $0.0850 $0.1185
Takeaway #1: L2 outperforms L1 for large retrievals, but L1 is faster for smaller retrievals.
t3.large t3.xlarge t3.2xlarge
Up to 5 Gigabit Up to 5 Gigabit Up to 5 Gigabit
8 GiB 16 GiB 32 GiB
2 vCPUs 4 vCPUs 8 vCPUs
$0.0832 $0.1664 $0.3328
L3 incurs a visible baseline overhead relative to L1 and L2, even for the smallest queries. In Fig. 2a, mini row, Q1 retrieves a single sample yet L3 takes 2-3 seconds while L1 and L2 complete in under 1 second. Q10 and Q11 show the same pattern, with L3 at 1-2 seconds versus sub-second for L1 and L2. For mid-range result sets (Q5-Q7, Q9 in Fig. 2b), L3 incurs substantially higher latency than L1 and L2 across all dataset scales and instance types. However, Q8 and Q4 are two notable exceptions: despite being the most data-intensive queries, L3 is faster than both L1 and L2. For example, on mini, L3 completes Q8 in approximately 5 seconds while L2 takes over 15 seconds and L1 takes over 30 seconds. This reversal occurs because Q8’s large result set (2,311 images on mini) causes L1 and L2’s perrequest overheads to dominate, while L3’s row group reads become relatively efficient. On the full dataset, L3 could not complete Q5-Q9 on the smaller instances, as indicated by × marks in the figures; larger-memory instances are required.
4.1
Layout Performance Comparison
Fig. 2 presents the end-to-end retrieval time across all three layouts (L1, L2, L3) for the 11 queries on four EC2 instance types (t3.medium, c5.large, c8gb.large, t3.large) and three dataset scales (mini, medium, full). Fig. 2a covers Q1-Q4 and Q10-Q11, while Fig. 2b covers Q5-Q9. Each row corresponds to a dataset scale and each column to an instance type; the y-axis scales differ across rows. A hatched bar (L3 with t3.large) appears on the medium and full rows where the default instances lacked sufficient memory for L3. We highlight three key takeaways. For queries with very small result sets (1-13 files), L2 is faster than L1. In Figure 2a, mini row, L2 bars are consistently shorter than L1 bars for Q1, Q2, Q3, Q10, and Q11 across all instance types. This is because L2’s RANGE GET requests to tar files incur lower per-request overhead than L1’s individual object GET requests at this extremely small scale. As the retrieval becomes slightly bigger, however, L1 slightly outperforms L2 for these same queries (Q1-Q3, Q10, Q11), visible in the medium and full rows of Figure 2a. When samples are distributed across a few hundreds of tar files, L2 cannot reuse connections effectively, and its advantage disappears. This is most visible on Q4: on mini (130 files), L1 is clearly faster than L2 across all instance types, but on full (13,000 files) L2 outperforms L1. The same pattern holds in Figure 2b: L2 achieves visibly lower latency than L1 for Q5-Q8, particularly on the medium and full datasets, where the four queries return hundreds to thousands of files. This advantage stems from L2’s use of HTTP RANGE GET requests to a small number of tar files, which enables TCP connection reuse and reduces
Takeaway #2: L3 carries a baseline overhead that makes it slow for small and mid-range retrievals, but it is the fastest for the largest retrievals. It is clear from Fig. 2 that the dominant factor in retrieval time is the number of samples returned, not the number or type of predicates involved. On the medium and full datasets, Q8 exhibits the highest latency for L1 and L2 because it returns the largest retrievals on medium and full (see Table 2). In contrast, Q11 involves all four predicate columns (class, size, width, height) yet produces small bars across all layouts because it returns only 32 files on the full dataset. Similarly, Q9 (575 files) and Q10 (294 files) complete quickly despite combining two predicates each. This ordering is consistent across all three instance types in both sub-figures. Takeaway #3: Result set (or retrieval) size, not predicate complexity, determines retrieval time.
LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data
4.2
Data Transfer Scalability
Fig. 3 shows the total data transferred from S3 to the client for each query across the three layouts and dataset scales. Fig. 3a covers Q1-Q4 and Q10-Q11 with units in MB, while Fig. 3b covers Q5-Q9 with units in GB. Each row corresponds to a dataset scale (mini, medium, full); the y-axis scales differ across rows. In Fig. 3, we report results using the smallest instance that could complete the given query in our experiments.
mini
L3 Data Transferred (in MB)
429.2 286.1 143.1 0
medium
L2
full
2288.8 1716.6 1144.4 572.2 0
L1
L2
Q5
Q6
L3
Data Transferred (in GB)
1.1 0.8 0.6 0.3 0 11.2 7.5 3.7 0
111.8 74.5 37.3 0
full
medium
mini
L1
57.2 38.1 19.1 0
Q1
Q2
Q3
Q4
Q10
Q11
(a) Queries: Q1-Q4 & Q10-Q11.
Q7
Q8
Q9
(b) Queries: Q5-Q9.
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
Takeaway #5: L3 transfers substantially more data due to row group granularity and fixed metadata reading cost, across queries and dataset scales.
4.3
Layout Scalability to Instance Types
Fig. 4 examines how each layout scales across different EC2 instance configurations. Unlike Fig. 2, which groups bars by layout within each instance type, this figure groups bars by instance type within each layout (one column per layout), making it easier to see how a given layout responds to increased resources. Each row corresponds to a dataset scale (mini, medium, full). Fig. 4a covers Q1-Q4 and Q10-Q11; Fig. 4b covers Q5-Q9. For L1 and L2, bars are colored by the three bandwidth-tier instances (t3.medium, c5.large, c8gb.large). For L3 on full, the legend changes to memory-tier instances (t3.medium, t3.large, t3.xlarge, t3.2xlarge), as the bandwidth-tier instances with 4 GB memory could not complete execution at this scale (× marks in the figures).
Figure 3: Data transferred in (a) megabytes and (b) gigabytes of three layouts (L1, L2, L3) for three dataset scales (per row: mini, medium, full) across all 11 queries.
L3
mini
×101 t3medium c5large c8gblarge
medium
Time T (in sec)
4 3 2 1 0
2.0 1.5 1.0 0.5 0.0
×102 t3medium t3large t3xlarge t32xlarge
Q1 Q2 Q3 Q4 Q10Q11
Takeaway #4: L1 and L2 achieve near-optimal data transfer, fetching close to the minimum necessary bytes.
Q1 Q2 Q3 Q4 Q10Q11
Q1 Q2 Q3 Q4 Q10Q11
(a) Queries: Q1-Q4 & Q10-Q11. Storage Layouts ×101
L1
L2
L3
mini
3.2 2.4 1.6 0.8
×102
medium
Time T (in sec)
0.0
t3medium c5large c8gblarge
3.2
2.4 1.6 0.8
0.0 ×103 t3medium t3large t3xlarge t32xlarge
2.0 1.5
full
L3 transfers more data than L1 and L2 across all queries in both sub-figures, because DuckDB’s Parquet reader fetches entire row groups containing the requested samples, reading adjacent rows even when only a subset matches the predicate. The overhead ratio depends on the result set size. For highly selective queries, the gap is extreme: Q1 (a single sample) transfers ∼ 57 MB under L3 on the mini dataset versus < 1 MB under L1 / L2 (Fig. 3a). Similar patterns hold for Q2, Q3, Q10, and Q11, where L3 bars dominate the chart while L1 / L2 bars are negligible. For queries returning large result sets, the gap reduces. For example, on Q4 and Q8, which return the largest result sets, the gap between L3 and L1 / L2 narrows relative to other queries but remains substantial (L3 transfers roughly 3-4× more data than L1 / L2 across all dataset scales on Q8). For all other large queries (Q5-Q7, Q9), L3 bars reach the top of the y-axis while L1 / L2 bars remain near the baseline, indicating that L3 transfers an order of magnitude more data than L1 / L2. L3 incurs higher data transfer volume than L1 / L2 primarily due to row group granularity and the fixed overhead of Parquet metadata reading.
L2
4 3 2 1 0
full
L1 and L2 transfer nearly identical bytes for most queries across both sub-figures, as both layouts fetch only the requested image bytes plus minimal protocol overhead. For Q1-Q3 and Q10-Q11 in Fig. 3a, the L1 and L2 bars are barely visible next to L3. Q4 is an exception: because it returns up to 13,000 files on the full dataset, L1 and L2 transfer visible amounts of data (>1,700 MB), though still less than L3. Data transfer under L1 / L2 scales linearly with result set size. For example, Q8 transfers approximately 0.3 GB, 3 GB, and 30 GB under L1 / L2 for mini, medium, and full datasets respectively (Fig. 3b), consistent with the roughly 10× scaling in sample counts across the corresponding dataset sizes (see Table 2).
Storage Layouts L1
1.0 0.5 0.0
Q5 Q6 Q7 Q8 Q9
Q5 Q6 Q7 Q8 Q9
Q5 Q6 Q7 Q8 Q9
(b) Queries: Q5-Q9.
Figure 4: End-to-end retrieval time 𝑇 in seconds of different EC2 instance types on layouts (per column: L1, L2, L3) for three dataset scales (per row: mini, medium, full). In the L1 column, higher-bandwidth instances visibly reduce retrieval time for data-intensive queries. The effect is clearest for
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
Sanyal et al.
Q4 and Q8 across all dataset scales. In Fig. 4a, L1 column, full row, Q4 retrieval time drops from ∼200 seconds on t3.medium to ∼85 seconds on c8gb.large, a reduction of more than 50%. A similar pattern holds for Q8 in Fig. 4b: on the full dataset, t3.medium takes ∼2,000 seconds while c8gb.large completes in ∼800 seconds. The L2 column shows a similar trend, though the reductions are less dramatic because L2 already benefits from connection reuse to fewer tar files. For queries with small result sets (e.g., Q1, Q10, Q11 in Fig. 4a), bars are uniformly short across all instance types in both L1 and L2, confirming that bandwidth is not the bottleneck when little data is transferred. Takeaway #6: L1 and L2 are both bandwidth-bound: a faster network reduces latency for large retrievals, but has little effect on small retrievals. The L3 column shows a different pattern. On the mini dataset, L3 runs on the same three bandwidth-tier instances as L1 and L2, and the bars remain similar across instance types in both figures, indicating that network bandwidth is not L3’s bottleneck. On the full dataset, the smaller instances could not complete execution, requiring the largest instance instead, for all queries Q5-Q9 in Fig. 4b. Among the memory-tier instances on the full dataset, the bars for smaller queries (e.g., Q1-Q3 in Fig. 4a, L3 column, full row) are similar across instance types, suggesting diminishing returns once DuckDB has sufficient memory to operate. For larger queries, on the full dataset (Fig. 4b, L3 column, full row), only the largest memorytier instance completed execution, with most smaller instances marked as failed. Takeaway #7: L3 is memory-bound rather than bandwidthbound, and requires progressively larger instances for larger retrievals. These scalability patterns have direct implications for instance selection. For large retrievals under L1 or L2, investing in c8gb.large ($0.1185/hr; Table 3) yields meaningful latency reductions over t3.medium ($0.0416/hr). For selective queries, t3.medium provides comparable performance, as the bars in Fig. 4a confirm. For L3 on the full dataset, the required t3.2xlarge (32 GiB, $0.3328/hr) costs 8× more per hour than the t3.medium that suffices for L1 and L2.
4.4
Cold vs. Warm Runs
To understand caching effects, we compare cold runs (first execution after instance launch) against warm runs (subsequent executions). Table 4 reports the ratio of cold-run to warm-run retrieval time, averaged across all queries, on t3.medium. A ratio above 1.0 indicates a cold-start penalty (cold runs are slower), while a ratio below 1.0 indicates that cold runs are faster than warm runs. Table 4: Cold-to-warm run- Table 5: Cost breakdown on time ratio on t3.medium across t3.medium (excluding Q8) across layouts. layouts and dataset scales. Dataset
L1
L2
L3
mini medium full
1.186 0.978 0.962
0.987 0.980 0.987
1.056 1.078 1.073
L1
L2
L3
𝐶 network 𝐶 compute
$0.795 $0.010
$0.795 $0.005
$9.183 $0.019
𝐶 (𝑡𝑜𝑡𝑎𝑙 )
$0.805
$0.800
$9.202
L2 exhibits the most stable behavior, with ratios between 0.980 and 0.987 across all dataset scales, i.e., essentially no difference between cold and warm runs. This is consistent with L2’s retrieval mechanism: each RANGE GET request is independent, and there is little client-side state to cache between runs. L1 shows a more varied pattern. On the mini dataset, L1’s ratio is 1.186, indicating a notable 19% cold-start penalty, likely reflecting TCP connection establishment and DNS resolution costs that are amortized in warm runs through connection pooling. However, on the medium (0.978) and full (0.962) datasets, L1’s ratios fall below 1.0, meaning warm runs are slightly slower than cold runs. This counterintuitive result may arise from increased resource contention on the client during warm runs or from the negligible role of connection-level caching when the number of GET requests grows to hundreds of thousands. L3 consistently shows ratios above 1.0 (1.056-1.078), indicating a modest 5-8% cold-start penalty across all dataset scales. This overhead is attributable to DuckDB’s one-time initialization costs: Parquet footer reading, metadata parsing, and query plan compilation. On warm runs, DuckDB benefits from cached metadata and pre-compiled execution plans, as well as cached columnar statistics (min/max values per row group) that enable more efficient predicate pushdown. Unlike L1, L3’s cold-start penalty is consistent across dataset scales, suggesting that the overhead is dominated by fixed initialization costs rather than per-sample factors. Takeaway #8: Caching effects are modest and dependent on layout and dataset scale.
4.5
Cost Estimates
We model the total cost 𝐶 for executing a set of queries as the sum of two components: 𝐶 = 𝐶 network +𝐶 compute , where 𝐶 network = $0.09×𝐷 is the S3 data transfer cost for 𝐷 GB of data downloaded, and 𝐶 compute = 𝑟 × 𝑇 is the EC2 rental cost at hourly rate 𝑟 for total retrieval time 𝑇 (in hours). The hourly rates for each instance type are listed in Table 3 [3]. We present the cost breakdown on the medium dataset using t3.medium, summed across all queries except Q8, in Table 5 . We exclude Q8 because L3 runs out of memory on t3.medium, as shown in Fig. 2b. Across all three layouts, 𝐶 network accounts for over 98% of total cost. Compute cost 𝐶 compute is negligible by comparison. This confirms that for cloud-based multimedia retrieval, the bill is determined almost entirely by how many bytes leave S3, not by how long the client runs. L1 and L2 incur identical transfer costs ($0.795) because both fetch only the requested image bytes (Takeaway 4). L2 achieves the lowest total cost ($0.800) thanks to its slightly faster retrieval, which reduces 𝐶 compute to half that of L1 ($0.005 vs. $0.010). L3’s total cost ($9.202) is 11.5× higher than L2, driven almost entirely by its excessive data transfer due to row group granularity (Takeaway 5). This means the storage layout choice is not just a performance decision but a significant economic one: running queries under L3 costs an order of magnitude more than under L1 or L2.
Takeaway #9: Data transfer cost dominates total expenditure, and L3’s row group overhead carries a steep price.
LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data
Table 6: Performance characterization by layout (L1, L2, L3). Each layout exhibits distinct resource constraints that dominate as the retrieval sizes become progressively larger. Characteristic Primary bottleneck Connection overhead Predicate pushdown Min. instance (full) Scales with bandwidth
L1
L2
L3
Latency High ✗ t3.medium ✓
Bandwidth Low ✗ t3.medium ✓
Memory N/A ✓ t3.2xlarge ✗
Overall Performance Characteristics: Table 6 summarizes the overall performance characterization of each layout. L1 is primarily bottlenecked by per-request latency, L2 by network bandwidth, and L3 by memory. L2 achieves a good latency-cost balance for ImageNet-1K: it matches or outperforms L1 across most medium to moderately large retrieval sizes, runs on the cheapest instance type, and incurs the lowest total cost. L3 is viable only for very large query retrievals where fast retrieval via predicate pushdown justifies its higher memory and data transfer costs.
5
Limitations and Conclusion
In our study, we evaluate only the image modality; performance tradeoffs may differ for larger binary objects such as video or audio, where per-sample sizes are orders of magnitude greater. Additionally, we benchmark read-only retrieval and do not measure write or ingest paths, which may favor different layouts. Finally, our experiments use a single cloud provider and a single query engine for L3; results may vary with other storage backends or engines. We presented LayoutBench, the first benchmark for systematically evaluating how cloud storage layouts affect multimedia data retrieval performance and cost. Our experiments across three layouts, 11 queries, three dataset scales, and six EC2 instance types reveal that L2 offers a good latency-cost balance for images, achieving low latency through connection reuse while incurring low total cost. LayoutBench is extensible by design: one can easily plug in new layouts, storage backends, datasets, and modalities. Future work includes benchmarking write and ingest paths, evaluating hybrid layouts that combine L2’s I/O efficiency with L3’s predicate pushdown, and extending to text and vector retrieval workloads.
References [1] 2022. Petastorm Documentation. https://petastorm.readthedocs.io/en/latest/ index.html. Accessed: 2026-02-23. [2] Alex Aizman, Gavin Maltby, and Thomas Breuel. 2019. High performance I/O for large scale deep learning. In 2019 IEEE International Conference on Big Data (Big Data). IEEE, 5965–5967. [3] Amazon Web Services, Inc. 2026. Amazon EC2 On-Demand Pricing. https: //aws.amazon.com/ec2/pricing/on-demand/. Accessed: 2026-02-22. [4] Panagiotis Antonopoulos, Alex Budovski, Cristian Diaconu, Alejandro Hernandez Saenz, Jack Hu, Hanuma Kodavalla, Donald Kossmann, Sandeep Lingam, Umar Farooq Minhas, Naveen Prakash, et al. 2019. Socrates: The new sql server in the cloud. In Proceedings of the 2019 International Conference on Management of Data. 1743–1756. [5] Michael Armbrust, Tathagata Das, Liwen Sun, Burak Yavuz, Shixiong Zhu, Mukul Murthy, Joseph Torres, Herman Van Hovell, Adrian Ionescu, Alicja Łuszczak, et al. 2020. Delta lake: high-performance ACID table storage over cloud object stores. Proceedings of the VLDB Endowment 13, 12 (2020), 3411–3424. [6] Alexander Arzhanov, Ilya Isaev, and Roy Allela. 2025. Applying Data Loading Best Practices for ML Training with Amazon S3 Clients. Amazon Web Services. https://aws.amazon.com/blogs/machine-learning/applying-data-loading-
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
best-practices-for-ml-training-with-amazon-s3-clients/ Accessed: 2026-02-22 AWS Machine Learning Blog. [7] Haoqiong Bian and Anastasia Ailamaki. 2022. Pixels: An efficient column store for cloud data lakes. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 3078–3090. [8] Zongzhi Chen, Xinjun Yang, Feifei Li, Xuntao Cheng, Qingda Hu, Zheyu Miao, Rongbiao Xie, Xiaofei Wu, Kang Wang, Zhao Song, et al. 2022. CloudJump: optimizing cloud databases for cloud storages. Proceedings of the VLDB Endowment 15, 12 (2022), 3432–3444. [9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248–255. [10] Rihan Hai, Christos Koutras, Christoph Quix, and Matthias Jarke. 2023. Data lakes: A survey of functions and systems. IEEE Transactions on Knowledge and Data Engineering 35, 12 (2023), 12571–12590. [11] Sasun Hambardzumyan, Abhinav Tuli, Levon Ghukasyan, Fariz Rahman, Hrant Topchyan, David Isayan, Mark McQuade, Mikayel Harutyunyan, Tatevik Hakobyan, Ivo Stranic, et al. 2022. Deep lake: A lakehouse for deep learning. arXiv preprint arXiv:2209.10785 (2022). [12] Yuanzhe Hao, Xiongpai Qin, Yueguo Chen, Yaru Li, Xiaoguang Sun, Yu Tao, Xiao Zhang, and Xiaoyong Du. 2021. Ts-benchmark: A benchmark for time series databases. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 588–599. [13] Zhaoxuan Ji, Zhongle Xie, Yuncheng Wu, and Meihui Zhang. 2024. Lbsc: A costaware caching framework for cloud databases. In 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 4911–4924. [14] Abdelouahab Khelifati, Mourad Khayati, Anton Dignös, Djellel Difallah, and Philippe Cudré-Mauroux. 2023. TSM-bench: benchmarking time series database systems for monitoring applications. Proceedings of the VLDB Endowment 16, 11 (2023), 3363–3376. [15] Bogyeong Kim, Kyoseung Koo, Undraa Enkhbat, Sohyun Kim, Juhun Kim, and Bongki Moon. 2022. M2bench: a database benchmark for multi-model analytic workloads. Proceedings of the VLDB Endowment 16, 4 (2022), 747–759. [16] Justin Levandoski, Garrett Casto, Mingge Deng, Rushabh Desai, Pavan Edara, Thibaud Hottelier, Amir Hormati, Anoop Johnson, Jeff Johnson, Dawid Kurzyniec, et al. 2024. BigLake: BigQuery’s evolution toward a multi-cloud lakehouse. In Companion of the 2024 International Conference on Management of Data. 334–346. [17] Guoliang Li, Wengang Tian, Jinyu Zhang, Ronen Grosman, Zongchao Liu, and Sihao Li. 2024. Gaussdb: A cloud-native multi-primary database with computememory-storage disaggregation. Proceedings of the VLDB Endowment 17, 12 (2024), 3786–3798. [18] Adrian Lutsch, Muhammad El-Hindi, Matthias Heinrich, Daniel Ritter, Zsolt IstvĂĄn, and Carsten Binnig. 2024. Benchmarking analytical query processing in intel SGXv2. arXiv preprint arXiv:2403.11874 (2024). [19] Yancan Mao, Ruohang Yin, Liyuan Lei, Peng Ye, Shengfu Zou, Shizheng Tang, Yunzhe Guo, Ye Yuan, Xiaochen Yu, Bo Wan, et al. 2024. Bytemq: A cloud-native streaming data layer in bytedance. In Proceedings of the 2024 ACM Symposium on Cloud Computing. 774–791. [20] Alex Merenstein, Vasily Tarasov, Ali Anwar, Deepavali Bhagwat, Julie Lee, Lukas Rupprecht, Dimitris Skourtis, Yang Yang, and Erez Zadok. 2021. { CNSBench } : A cloud native storage benchmark. In 19th USENIX Conference on File and Storage Technologies (FAST 21). 263–276. [21] Koyel Mukherjee, Raunak Shah, Shiv Saini, Karanpreet Singh, Harsh Kesarwani, Kavya Barnwal, Ayush Chauhan, et al. 2023. Towards optimizing storage costs on the cloud. In 2023 IEEE 39th International Conference on Data Engineering (ICDE). IEEE, 2919–2932. [22] Dan S Nielsen and Ryan McConville. 2022. Mumin: A large-scale multilingual multimodal fact-checked misinformation social network dataset. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval. 3141–3153. [23] Weston Pace, Chang She, Lei Xu, Will Jones, Albert Lockett, Jun Wang, and Raunak Shah. 2025. Lance: Efficient random access in columnar storage through adaptive structural encodings. arXiv preprint arXiv:2504.15247 (2025). [24] James Jie Pan, Jianguo Wang, and Guoliang Li. 2024. Survey of vector database management systems. The VLDB Journal 33, 5 (2024), 1591–1615. [25] Brian Regan, Desislava Hristova, and Mariano Beguerisse-Díaz. 2023. SemiAutomated Music Catalog Curation Using Audio and Metadata.. In ISMIR. 605– 611. [26] Debopam Sanyal, Jui-Tse Hung, Manav Agrawal, Prahlad Jasti, Shahab Nikkhoo, Somesh Jha, Tianhao Wang, Sibin Mohan, and Alexey Tumanov. 2023. Paretosecure machine learning (PSML): Fingerprinting and securing inference serving systems. arXiv preprint arXiv:2307.01292 (2023). [27] Debopam Sanyal, Anantharaman S Iyer, Alind Khare, Trisha Jain, Akshay Jajoo, Myungjin Lee, James Clayton Kerce, and Alexey Tumanov. 2026. KLAS: Using Similarity to Stitch Neural Networks for Improved Accuracy-Efficiency Tradeoffs. In The Fourteenth International Conference on Learning Representations.
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
[28] Raymond Shiau, Hao-Yu Wu, Eric Kim, Yue Li Du, Anqi Guo, Zhiyuan Zhang, Eileen Li, Kunlong Gu, Charles Rosenberg, and Andrew Zhai. 2020. Shop the look: Building a large scale visual shopping system at pinterest. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3203–3212. [29] The Mosaic ML Team. 2022. streaming. <https://github.com/mosaicml/streaming/ >. [30] Jordan Totten and Shane Hansen. 2021. Scaling deep learning workloads with PyTorch / XLA and Cloud TPU VM. Google Cloud. https://cloud.google.com/blog/topics/developers-practitioners/scaling-deeplearning-workloads-pytorch-xla-and-cloud-tpu-vm Accessed: 2026-02-22 Google Cloud Blog, Developers & Practitioners. [31] Hao Wang, Jiaxin Ou, Ming Zhao, Sheng Qiu, Yizheng Jiao, Yi Wang, Qizhong Mao, Zhengyu Yang, Yang Liu, Jianshun Zhang, et al. 2024. LavaStore: ByteDance’s Purpose-Built, High-Performance, Cost-Effective Local Storage Engine for Cloud Services. Proceedings of the VLDB Endowment 17, 12 (2024), 3799–3812. [32] Yifan Wang and Kenneth P Birman. 2025. Diagnosing and resolving cloud platform instability with multi-modal rag llms. In Proceedings of the 5th Workshop on Machine Learning and Systems. 139–147. [33] Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. [n. d.]. InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation. In The Twelfth International Conference on Learning Representations. [34] Zhiqi Wang and Zili Shao. 2022. Timeunion: An efficient architecture with unified data model for timeseries management systems on hybrid cloud storage. In Proceedings of the 2022 International Conference on Management of Data. 1418– 1432. [35] Siyang Weng, Qingshuai Wang, Luyi Qu, Rong Zhang, Peng Cai, Weining Qian, and Aoying Zhou. 2024. Lauca: A workload duplicator for benchmarking transactional database performance. IEEE Transactions on Knowledge and Data Engineering 36, 7 (2024), 3180–3194. [36] Xiang Xu and Rajesh Thallam. 2022. Efficient PyTorch training with Vertex AI. Google Cloud. https://cloud.google.com/blog/products/ai-machine-learning/ efficient-pytorch-training-with-vertex-ai Accessed: Google Cloud Blog, AI & Machine Learning. [37] Sukmin Yun, Rusiru Thushara, Mohammad Bhat, Yongxin Wang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, et al. 2024. Web2code: A large-scale webpage-to-code dataset and evaluation framework for multimodal llms. Advances in neural information processing systems 37 (2024), 112134–112157. [38] Xinyu Zeng, Yulong Hui, Jiahong Shen, Andrew Pavlo, Wes McKinney, and Huanchen Zhang. 2023. An empirical evaluation of columnar storage formats. Proceedings of the VLDB Endowment 17, 2 (2023), 148–161. [39] Chao Zhang, Guoliang Li, Leyao Liu, Tao Lv, and Ju Fan. 2025. CloudyBench: A testbed for a comprehensive evaluation of cloud-native databases. In 2025 IEEE 41st International Conference on Data Engineering (ICDE). IEEE, 1–13. [40] Chao Zhang, Guoliang Li, and Tao Lv. 2024. HyBench: A new benchmark for HTAP databases. Proceedings of the VLDB Endowment 17, 5 (2024), 939–951. [41] Jiashu Zhang, Wen Jiang, Bo Tang, Haoxiang Ma, Lixun Cao, Zhongbin Jiang, Yuanyuan Nie, Fan Wang, Lei Zhang, and Yuming Liang. 2023. Cdsben: Benchmarking the performance of storage services in cloud-native database system at bytedance. Proceedings of the VLDB Endowment 16, 12 (2023), 3584–3596. [42] Qing Zheng, Haopeng Chen, Yaguang Wang, Jian Zhang, and Jiangang Duan. 2013. Cosbench: Cloud object storage benchmark. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering. 199–210.
Sanyal et al.
LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data
A
SQL Statements for Retrieval in L1
(Q10) Selective Filter: Get images of tiger_cat that are smaller than 100 KB.
The SQL statements (Q1)-(Q11) are used in L1 experiments. (Q1) One Sample Preview: Get one image of streetcar. 1 2 3 4
1
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label = ' streetcar ' LIMIT 1;
2 3 4
2 3 4
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label = ' lemon ' LIMIT N ;
1 2 3 4
(Q3) One Class: Get all images of cello.
5 1 2 3
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label = ' cello ';
6
2 3
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label LIKE '% snake ';
1
1 2 3
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label IN ( ' Old_English_sheepdog ' , ' giant_panda ' , ' water_buffalo ') ;
(Q6) File Size Filter: Get images that are at least 500 KB. 1 2 3
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE filesize_bytes >= 500 * 1024;
(Q7) Resolution Filter: Get images whose width and height are both at least 1024 pixels. 1 2 3 4
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE width >= 1024 AND height >= 1024;
(Q8) Cross-column Predicate: Get images whose width / height ratio is greater than 1.5. 1 2 3 4 5
SELECT filename , width , height FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE CAST ( width AS DOUBLE ) / CAST ( height AS DOUBLE ) > 1.5;
(Q9) Multi-label Filter: Get images of canoe and starfish that are smaller than 100 KB. 1 2 3 4
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label IN ( ' canoe ' , ' starfish ') AND filesize_bytes < 100 * 1024;
L1 : Initialize S3 transfer with multithreading
2 3 4
(Q5) Fan-out: Get images of Old_English_sheepdog, giant_panda, and water_buffalo.
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label = ' bagel ' AND width >= 512 AND height >= 512 AND filesize_bytes < 200 * 1024;
The returned filenames are used to infer their S3 URLs in order to fetch the image samples.
(Q4) Regex Match: Get images whose labels are LIKE %snake. 1
SELECT filename FROM read_parquet ( '{ CLI_METADATA_L1_PATH } ') WHERE human_label = ' tiger_cat ' AND filesize_bytes < 100 * 1024;
(Q11) Composite Filter: Get images of bagel whose file size is smaller than 200 KB and width and height are both at least 512 pixels.
(Q2) Batch Fetch: Get N images of lemon. 1
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
For each file URL in parallel : Download file to local directory
B
SQL Statements for Retrieval in L2
The SQL statements (Q1)-(Q11) are used in L2. (Q1) One Sample Preview: Get one image of streetcar. 1
2 3 4
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label = ' streetcar ' LIMIT 1;
(Q2) Batch Fetch: Get N images of lemon. 1
2 3 4
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label = ' lemon ' LIMIT N ;
(Q3) One Class: Get all images of cello. 1
2 3
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label = ' cello ';
(Q4) Regex Match: Get images whose labels are LIKE %snake. 1
2 3
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label LIKE '% snake ';
(Q5) Fan-out: Get images of Old_English_sheepdog, giant_panda, and water_buffalo.
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
1
2 3
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label IN ( ' Old_English_sheepdog ' , ' giant_panda ' , ' water_buffalo ') ;
(Q6) File Size Filter: Get images that are at least 500 KB. 1
2 3
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE filesize_bytes >= 500 * 1024;
(Q7) Resolution Filter: Get images whose width and height are both at least 1024 pixels.
Sanyal et al.
1 2 3
L2 : Initialize S3 transfer with multithreading For each file in parallel : Download the byte range corresponding to shard , data_offset and data_size
C
SQL Statements for Retrieval in L3
The SQL statements (Q1)-(Q11) are used in DuckDB. (Q1) One Sample Preview: Get one image of streetcar. 1 2 3 4
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label = ' streetcar ' LIMIT 1
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE width >= 1024 AND height >= 1024;
(Q2) Batch Fetch: Get N images of lemon.
(Q8) Cross-column Predicate: Get images whose width / height ratio is greater than 1.5.
(Q3) One Class: Get all images of cello.
1
2 3 4
1 2 3 4
1 1
2 3
SELECT filename , shard , data_offset , data_size , width , height FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE CAST ( width AS DOUBLE ) / CAST ( height AS DOUBLE ) > 1.5;
(Q9) Multi-label Filter: Get images of canoe and starfish that are smaller than 100 KB. 1
2 3 4
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label IN ( ' canoe ' , ' starfish ') AND filesize_bytes < 100 * 1024;
(Q10) Selective Filter: Get images of tiger_cat that are smaller than 100 KB. 1
2 3 4
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label = ' tiger_cat ' AND filesize_bytes < 100 * 1024;
(Q11) Composite Filter: Get images of bagel whose file size is smaller than 200 KB and width and height are both at least 512 pixels. 1
2 3 4 5 6
SELECT filename , shard , data_offset , data_size FROM read_parquet ( '{ CLI_METADATA_L2_PATH } ') WHERE human_label = ' bagel ' AND width >= 512 AND height >= 512 AND filesize_bytes < 200 * 1024;
2 3
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label = ' cello '
(Q4) Regex Match: Get images whose labels are LIKE %snake. 1 2 3
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label LIKE '% snake ';
(Q5) Fan-out: Get images of Old_English_sheepdog, giant_panda, and water_buffalo. 1 2 3 4
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label IN ( ' Old_English_sheepdog ' , ' giant_panda ' , ' water_buffalo ')
(Q6) File Size Filter: Get images that are at least 500 KB. 1 2 3
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE filesize_bytes >= 500 * 1024;
(Q7) Resolution Filter: Get images whose width and height are both at least 1024 pixels. 1 2 3 4
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE width >= 1024 AND height >= 1024
(Q8) Cross-column Predicate: Get images whose width / height ratio is greater than 1.5. 1 2 3
The returned filenames, along with shard, data offset, and data size, are used to fetch the corresponding byte ranges from S3.
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label = ' lemon ' LIMIT N
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE CAST ( width AS DOUBLE ) / CAST ( height AS DOUBLE ) > 1.5
LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data
(Q9) Multi-label Filter: Get images of canoe and starfish that are smaller than 100 KB. 1 2 3 4
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label IN ( ' canoe ' , ' starfish ') AND filesize_bytes < 100 * 1024
(Q10) Selective Filter: Get images of tiger_cat that are smaller than 100 KB. 1 2 3 4
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label = ' tiger_cat ' AND filesize_bytes < 100 * 1024
(Q11) Composite Filter: Get images of bagel whose file size is smaller than 200 KB and width and height are both at least 512 pixels. 1 2 3 4 5 6
SELECT filename , image_bytes FROM read_parquet ( '{ Parquet_S3_PATH } ') WHERE label = ' bagel ' AND width >= 512 AND height >= 512 AND filesize_bytes < 200 * 1024
EuroMLSys ’26, April 27–30, 2026, Edinburgh, Scotland Uk
Notably, a separate download is not needed, as the image bytes are fetched directly within the SQL statements.
D
Additional Experimental Details
Cold and Warm Runs. After launching an instance, we iterate over all 11 queries and execute twice for each query, where the first execution corresponds to a cold run since there is no cache for the fetched samples. The second execution corresponds to a warm run, where samples may be cached in the system. Number of Threads. To fully utilize all resources provided by the client EC2 instances, we set the upper bound on the number of threads to 64 for the t3 family, 128 for c5.large, and 256 for c8gb.large under L1 and L2. In all cases, the upper bound is sufficiently large that the number of spawned threads does not hit the limit. Under L3, we configure the number of threads through DuckDB and set it to 32 by default. When experimenting with the full dataset, we adjust the number of threads to avoid out-ofmemory errors. Specifically, we set it as 8 for Q6 and Q9, 4 for Q7, and 2 for Q4. Storage on EC2 Clients. The storage provisioned on EC2 clients must be large enough to accommodate downloaded data. By default, we allocate 8 GB for the mini dataset and 32 GB for the medium and large datasets. For larger queries, we increase the storage capacity up to 64 GB as needed.