AlbumentationsX: One Augmentation Pipeline for Images and Related Annotations

AlbumentationsX: One Augmentation Pipeline for Images and Related Annotations Vladimir Iglovikov Albumentations LLC vladimir@albumentations.ai https://github.com/albumentations-team/AlbumentationsX

arXiv:2608.11123v1 [cs.CV] 11 Aug 2026

Abstract Augmentation can corrupt a training example when an image and its annotations receive different random changes. A crop must use the same coordinates for the image, mask, boxes, keypoints, stereo views, video frames, or volume. Code paths that choose these values separately can silently misalign the data. AlbumentationsX keeps the transform list, probabilities, annotation settings, and random seed in one Compose object. Each call chooses random values once and applies them to every supported part of the training example. The library keeps each object’s mask, box, and label together and lets projects add their own transforms. It can also save the pipeline definition, show what happened in one call, and run that call again. The examples place Compose after files have been decoded into arrays and before PyTorch groups examples into a batch. AlbumentationsX executes the declared transforms. Practitioners still decide whether a flip, crop, color change, or other operation preserves the correct label for their task.

1

Introduction

A crop applied to an image but not its mask corrupts a training example. The same failure appears when a box, keypoint, stereo view, video frame, or volume receives different crop, rotation, or resize values. Every related value must receive the same geometric change. Data augmentation reuses labeled examples by changing their appearance or geometry during training. A brightness transform can simulate different lighting. A crop changes which part of the scene is visible and where objects appear. Blur can simulate motion or a camera that missed focus. Collecting and labeling representative real data remains the stronger answer when those examples can be obtained. Every augmentation makes an assumption about the task: the change should leave the correct answer unchanged. A horizontal flip may be valid for road-scene segmentation because the same objects remain in the image. The same flip is usually wrong for optical character recognition because it mirrors the text. The dataset owner must choose transformations that match plausible variation and preserve the label. A computer-vision training example often contains more than an image and one class label. This paper calls the complete example a sample. Semantic segmentation uses an image and a pixel mask. Object detection uses an image, bounding boxes, and one class label per box. Pose estimation adds keypoints, such as the left wrist and right knee. A video sample contains an ordered sequence of frames. A 3D sample may contain a volume and a 3D mask. AlbumentationsX calls each named value that a transform knows how to handle a target. Images, masks, bounding-box arrays, and keypoint arrays are targets. In semantic segmentation, the image and mask are two targets in one sample. After a crop, the mask must still cover the same pixels as the object in the image. An augmentation policy is the transform list plus the rules that control it: order, probabilities, target settings, and random seed. AlbumentationsX stores this policy in one Compose object [1, 2]. One call chooses the random values and applies them to every target in the sample. A crop therefore uses one set of coordinates for the image, mask, boxes, keypoints, related images, and every frame in a video clip. A training input path usually reads a file, decodes it into arrays, augments one sample, and then groups several samples into a batch. AlbumentationsX performs the augmentation step while the image and annotations are still separate named values. The following sections show how to build one policy, keep related targets aligned, add a project-specific operation, and choose the record needed to inspect or repeat a random call.

1

2

Build One Pipeline

The example below crops every sample, flips half of them, and adjusts brightness and contrast or applies a gamma curve in 40% of calls. It then normalizes the image for the model. The argument p is the probability that a transform runs; p=1.0 means always. OneOf first decides whether its group runs and then chooses one child. In BboxParams, coord_format="pascal_voc" means that each box is stored as [x_min,y_min,x_max,y_max]. The setting label_fields=["labels"] keeps each surviving box paired with its class label after a crop. The value seed=137 starts the policy’s own random sequence. import albumentations as A policy = A.Compose( [ A.RandomCrop(height=384, width=512, p=1.0), A.HorizontalFlip(p=0.5), A.OneOf( [ A.RandomBrightnessContrast(p=0.5), A.RandomGamma(p=0.5), ], p=0.4, ), A.Normalize(p=1.0), ], bbox_params=A.BboxParams( coord_format="pascal_voc", label_fields=["labels"], ), seed=137, ) sample = policy( image=image, mask=mask, bboxes=boxes, labels=labels, )

The two children inside OneOf have the same p, so the group chooses each one equally often. Normalize(p=1.0) runs last, after the random image changes. A fixed transform list is enough for a simple policy. Larger policies must also express which steps are optional, which alternative to choose, and whether order is fixed or random. Table 1 maps each decision to the API object that expresses it. Need Ordered steps Optional step Choose one Choose several Choose and reorder Grouped steps

API Compose Transform p OneOf SomeOf RandomOrder Sequential

Result Run the declared sequence Run it with probability p Choose one child at random Choose a requested number at random Choose a subset, then apply it in random order Nest one ordered block

Table 1: These API objects keep order, optional steps, and random choices inside the policy. The Compose seed controls this policy’s sequence of random choices; global random and numpy.random seeds do not control it. Concurrent calls use separate random generators, so one call cannot consume another call’s random numbers. The declared order remains visible: crop first, change appearance second, and normalize last. Section 5 shows how to repeat one sample without depending on call order.

2

3

Keep Each Object with Its Mask and Box

Instance segmentation predicts each object separately. One object can have a mask, a bounding box, a class label, and keypoints. A geometric transform must update all of these annotations together. An object near the image edge can move completely outside the returned image after rotation and translation. Its returned box still has horizontal and vertical sides; this is an axis-aligned box. That box may overlap the image by a narrow strip even when the transformed mask has no pixels inside it. Figure 1 shows why the empty mask must remove the complete object.

Original

Returned sample

After Affine output frame apple

apple mug

mug box

apple

filter

Affine

box sliver inside

Figure 1: After the affine transform, the mug’s mask is outside the blue output frame, but its axis-aligned box still overlaps the frame by a narrow strip. AlbumentationsX 2.4.0 uses the empty mask to remove the mug’s mask, box, and class label together. The apple remains with aligned annotations. AlbumentationsX provides this behavior through instances: a list with one dictionary per object. Each dictionary keeps that object’s mask, box, and class label together. The bbox_labels dictionary holds the box fields named by BboxParams.label_fields. The value instance_binding=["masks","bboxes"] selects masks and boxes for joint processing. Compose collects each object’s mask and bbox, transforms them, and returns them to the same object dictionary. The example fixes the rotation at 30 degrees and the horizontal movement at 52 pixels. The values fill=0 and fill_mask=0 fill newly exposed image and mask pixels with zero. The box filters keep any box with at least one pixel of area, including the mug’s narrow strip. instances = [ { "mask": apple_mask, "bbox": apple_box, "bbox_labels": {"class_name": "apple"}, }, { "mask": mug_mask, "bbox": mug_box, "bbox_labels": {"class_name": "mug"}, }, ]

The policy binds those fields and applies the fixed affine transform:

3

instance_policy = A.Compose( [ A.Affine( rotate=(-30, -30), translate_px={"x": (52, 52), "y": (0, 0)}, fill=0, fill_mask=0, p=1.0, ), ], bbox_params=A.BboxParams( coord_format="pascal_voc", label_fields=["class_name"], min_area=1, min_visibility=0.0, ), instance_binding=["masks", "bboxes"], seed=137, ) sample = instance_policy( image=image, instances=instances, )

After the transform, the mug’s mask is empty, so AlbumentationsX removes the mug dictionary from sample["instances"]. The apple stays in the same per-object format with its transformed mask, box, and label. Other tasks use the same idea without the per-object container. Semantic segmentation passes image= and mask=. Detection and pose add bboxes= and keypoints=. A video clip uses images=frames; 3D data uses volume= and, when present, mask3d=.

3.1

Keep stereo images and depth maps aligned

Stereo vision uses two images of the same scene, one from the left camera and one from the right. Every allowed crop, resize, or other geometric transform must use the same values for both views. An RGB-D sample also contains a depth map, which stores the distance at each pixel. The depth map needs the same geometry as the color image, but it must skip brightness and contrast changes. The setting additional_targets={"right_image":"image"} tells Compose to process right_image like the main image: both receive the same geometry and the same image-only transforms. The depth map is passed as mask, so it receives the shared geometry but skips image-only transforms. The code renames that output back to depth after the call. Depth contains continuous measurements, so mask_interpolation=cv2.INTER_LINEAR selects linear interpolation. A segmentation mask contains class IDs and normally uses nearest-neighbor interpolation so resize does not invent new class values.

4

import cv2 paired_policy = A.Compose( [ A.Resize(height=384, width=512, p=1.0), A.RandomCrop(height=352, width=480, p=1.0), A.RandomBrightnessContrast(p=0.3), A.Normalize(p=1.0), ], additional_targets={ "right_image": "image", }, mask_interpolation=cv2.INTER_LINEAR, seed=137, ) paired = paired_policy( image=left_image, right_image=right_image, mask=depth_map, ) paired["depth"] = paired.pop("mask")

Both color images receive the same resize and crop, and both are normalized. The depth map receives the resize and crop but skips brightness, contrast, and normalization. The mask_interpolation setting applies to every value processed as a mask in this Compose. If one sample contains both continuous depth and a class-ID mask, one interpolation setting is not enough. Use a custom transform that resizes depth with linear interpolation and the segmentation mask with nearest-neighbor interpolation. Some stereo training methods need the same geometry but independent sensor noise in the two cameras. First use one pipeline for the shared geometry. Then use a separate pipeline for each view to change pixel values without moving pixels.

3.2

Apply one crop to the whole video clip

A video clip is an ordered sequence of frames. Cropping every frame independently would make the crop window jump around, creating artificial camera motion. Pass the complete video clip as images=frames. AlbumentationsX then uses the same crop and flip for every frame. clip_policy = A.Compose( [ A.RandomCrop(height=224, width=224, p=1.0), A.HorizontalFlip(p=0.5), A.RandomBrightnessContrast(p=0.3), A.Normalize(p=1.0), ], seed=137, ) augmented_clip = clip_policy( images=frames, )["images"]

Use image= for one multichannel image, such as a 9-channel satellite image. The channel count does not guarantee that every transform supports the array; some color transforms accept only one or three channels. Use volume= for a 3D volume and mask3d= for its 3D mask.

4

Add a Project-Specific Transform

A transform catalog cannot cover every camera and dataset. A project may need to simulate a specific gain error, stripe pattern, or sensor defect. A custom transform puts that operation inside the same Compose pipeline as the built-in transforms, so it uses the same probability and seed rules. The example below simulates sensor gain by multiplying every pixel by one number between 0.8 and 1.2. No pixel moves, so masks, boxes, and keypoints do not change. The class therefore derives from ImageOnlyTransform.

5

The sample_parameters method chooses the gain once for the call. The apply method receives that chosen number and performs only the pixel calculation. Compose supplies SamplingContext, whose random generators follow the policy seed. import numpy as np import albumentations as A from albumentations.core.invocation import SamplingContext class RandomGain(A.ImageOnlyTransform): def __init__(self, gain=(0.8, 1.2), p=0.5): super().__init__(p=p) self.gain = gain def sample_parameters( self, params, data, sampling: SamplingContext ): value = sampling.py_random.uniform(*self.gain) return {"gain": value} def apply(self, img, gain, **params): scaled = img.astype(np.float32) * gain return np.clip(scaled, 0, 255).astype(img.dtype)

Use sampling.py_random when the transform needs a few random numbers. Use sampling.random_generator when it needs a random array. Keep every random choice in sample_parameters; apply should only use the values it receives. Choose the base class by the values that the operation must change. Table 2 maps each project need to the corresponding API. Project need Pixel or sensor effect Spatial warp 3D volume operation Project-specific input name

Start from ImageOnlyTransform DualTransform Transform3D CustomTransformsApplyMixin

What changes Image values; spatial targets stay fixed Image, masks, boxes, and keypoints Volume and matching 3D targets apply_to_<key> receives the same chosen parameters

Table 2: Choose the custom-transform base class by the kind of data the new operation changes. First test the numerical operation without randomness; for example, verify that a pixel value of 100 becomes 120 when the gain is 1.2. Then place the transform in a fixed-seed Compose call and check every affected input. The project supplies the sensor- or task-specific calculation and reuses the surrounding pipeline code.

5

Repeat or Inspect a Random Call

When an augmented sample looks wrong, first decide what must be repeated. A policy seed repeats the whole sequence: a new Compose object with the same policy and seed makes the same random choices when samples arrive in the same order. If the order changes, a sample may receive a different crop or color change. Use invocation_seed when one sample must receive the same choices regardless of which other calls run first. Pass a stable integer with that sample: sample = policy( image=image, mask=mask, invocation_seed=1_000_001, )

A seed does not show which crop coordinates an earlier call used. Save the Compose definition when the goal is to rebuild the policy later. Set save_applied_params=True to record which transforms ran and the settings they exposed. Use ReplayCompose when the exact crop, flip decision, and other random values from one call must run again.

6

Question Repeat the ordered sequence? Repeat one sample despite call order? Rebuild the policy later? See which transforms ran? Run the exact random call again? Repeat the complete loader run?

Record to keep Policy definition, seed, and sample order Policy definition and invocation_seed Saved Compose definition and library version save_applied_params=True output and sample identifier ReplayCompose record for supported transforms Sample order, framework seed, worker count, and whether workers persist

Table 3: Keep the smallest record that answers the question. Table 3 gives the minimum record for each question. A complete data-loading run also depends on the training framework’s random state, the number of DataLoader workers, and whether those workers remain alive between epochs. The compact save_applied_params output is intended for inspection and may omit runtime values that ReplayCompose needs.

6

Call AlbumentationsX from the PyTorch Dataset

In PyTorch, Dataset.__getitem__ usually prepares one sample, and DataLoader later groups returned samples into batches. Call AlbumentationsX inside Dataset.__getitem__, after JPEG or other file bytes have become arrays. At that point, the image and annotations still have separate names, so Compose can update them together. The example below returns a fixed-size image and mask. import albumentations as A from albumentations.pytorch import ToTensorV2 from torch.utils.data import Dataset train_policy = A.Compose( [ A.RandomCrop(height=384, width=512, p=1.0), A.HorizontalFlip(p=0.5), A.RandomBrightnessContrast(p=0.3), A.Normalize(p=1.0), ToTensorV2(p=1.0), ], seed=137, ) class SegmentationDataset(Dataset): def __getitem__(self, index): image, mask = self.load_decoded(index) sample = train_policy( image=image, mask=mask, ) return sample["image"], sample["mask"].long()

This placement keeps the image and mask together until the last geometric operation. Image-only transforms receive only the image. Normalization and ToTensorV2 then produce the PyTorch tensors expected by the model. Because the crop returns a fixed size, PyTorch’s default batching step can stack the images and masks. Detection pipelines may need a custom batching function because different images can contain different numbers of boxes. Declare training and evaluation as separate policies. A validation policy usually contains deterministic resizing, padding, normalization, and tensor conversion. The two objects then answer two distinct review questions: which random variation trains the model, and which deterministic preprocessing defines evaluation?

7

Related Libraries

TorchVision v2, Kornia, and NVIDIA DALI can run supported image augmentations on GPUs; DALI can include decoding in the same pipeline [3–6]. In the cited releases, the public APIs expose 121 concrete 2D transform classes in AlbumentationsX, 61 in Kornia, and 38 in TorchVision v2 [2, 3, 5]. The count excludes base classes, pipeline and policy containers, format-conversion helpers, and transforms limited to 3D data or spectrograms. AlbumentationsX also keeps geometric changes aligned across images and related annotations.

7

8

Limits and Availability

AlbumentationsX cannot infer whether a transform preserves the task label. Rotation may be valid for aerial imagery and harmful for a digit classifier. Color changes may simulate different lighting, but they can remove the signal used by a model that classifies fruit ripeness. Practitioners make these decisions from knowledge of the task and deployment data. Target support varies by transform. For example, a color transform may accept an RGB image but reject a 9-channel array. Some arrays describe the camera rather than image pixels. A depth map can share the image’s crop and resize. A camera calibration matrix stores camera geometry, so a crop or resize must update its numbers with a separate rule. The examples use AlbumentationsX release 2.4.0 (commit e85171bc777f). The public repository is available under AGPL-3.0-only. The AGPL permits commercial use subject to its terms. Albumentations, LLC also offers separately negotiated commercial licenses with alternative rights defined by the applicable agreement. The source and documentation are available at the cited URLs [1, 2].

9

Conclusion

AlbumentationsX stores the transform list and its rules in one Compose object. Each call chooses random values once and applies them to the image and its related annotations. The instances input keeps each object’s mask, box, label, and keypoints together. The same pipeline can process stereo pairs, depth maps, video clips, 3D volumes, and project-specific transforms. It can also save the policy, show which transforms ran, or repeat one exact call. Use AlbumentationsX when a project needs one reviewable Python policy for decoded images and their related targets.

References [1] Albumentations Team. AlbumentationsX documentation. https://albumentations.ai/docs/, 2026. [2] Albumentations Team. AlbumentationsX source code, release 2.4.0. https://github.com/albumentations-team/AlbumentationsX/tree/2.4.0, 2026. Commit e85171bc777fbe7dfd0fa473cc2d3853c97437e5. [3] Kornia Contributors. Kornia augmentation exports, release 0.8.2. https://github.com/kornia/kornia/blob/v0.8.2/kornia/augmentation/__ init__.py, 2025. Commit 856fd1ae6becd4300646db2b44ea3507e4ab0066. [4] NVIDIA. NVIDIA DALI: Data loading and augmentation library. https://github.com/NVIDIA/DALI, 2026. [5] PyTorch Contributors. Torchvision v2 transform exports, release 0.26.0. https://github.com/pytorch/vision/blob/v0.26.0/torchvision/ transforms/v2/__init__.py, 2026. Commit 336d36e8db990a905498c73933e35231876e28bc. [6] Edgar Riba, Dmytro Mishkin, David Ponsa, Ethan Rublee, and Gary Bradski. Kornia: An open source differentiable computer vision library for PyTorch. In IEEE Winter Conference on Applications of Computer Vision, 2020. doi: 10.1109/WACV45572.2020.9093363.

8

Related documents

• SR-Edit: Region-Aware Image Editing via Self-Refinement#924381
• SR-Edit: Region-Aware Image Editing via Self-Refinement#920150
• Less Is More in the Long Tail: Stage-Adaptive Sample Selection for Annotation-Efficient Dense Prediction#1061478
• Multiclass Semantic Segmentation of Wildland Fire Images Using Context-Aware Centralized Copy-Paste Data Augmentation#1035207
• ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning#793156