trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy Judith Sáinz-Pardo Díaza,∗ , Álvaro López Garcíaa a Instituto de Física de Cantabria (IFCA), CSIC-UC, Avda. los Castros s/n, Santander, 39005, Spain
Abstract
Keywords: Data privacy microdata release differential privacy metric privacy open-source
trasgoDP is a modular, open-source, and easy-to-use Python framework for releasing tabular microdata under 𝜖-local differential privacy guarantees, as well as location data under geoindistinguishability assumptions, designed to be installed and integrated within standard data science workflows. The software enables systematic exploration of privacy-utility trade-offs across multiple mechanisms, data types, and 𝜖 values. While differential privacy has been extensively studied for aggregate data, its application to row-wise microdata release remains underexploited in terms of reusable software tools, a gap that is even more pronounced in the case of metric privacy and location-based data. trasgoDP implements local-DP mechanisms for numerical and categorical attributes (Laplace, Gaussian, Exponential, and Randomized Response), a geo-indistinguishability mechanism for location data, and a set of utility metrics, including a novel correlation-loss measure, to quantify information loss as a function of the allocated privacy budget. The objective of this work is to provide the research community with a reproducible, open-source baseline for evaluating tabular and location-based data publication methodologies under formal local differential privacy guarantees.
arXiv:2607.22230v1 [cs.CR] 24 Jul 2026
ARTICLE INFO
INTRODUCTION
The publication of open data is a fundamental pillar for building open science[5, 29]. More specifically, the three main pillars that compose open science are open data, open source, and open access. As data availability continues to grow, increasingly larger datasets are being released daily, allowing the development of a wide range of applications, including predictive models based on machine learning (ML) techniques. However, in certain domains, the publication of data is a complex process, especially when sensitive data or data that could identify individuals are involved and carry the risk of individual re-identification. In this sense, this challenge becomes even more complex in the context of microdata[8] releasing, where tabular data[12] presents an additional complexity: the anonymization guarantees applied to an early publication may be compromised by later updates that introduce new records. On the one hand, when we refer to microdata publication, we are dealing with raw data containing individual records on a group of studied individuals. This could be a hospital patient record, a municipal census, or data on public employees. On the other hand, when we talk about aggregated data, we are referring to statistics compiled from that microdata. In some cases, publishing aggregate data can be useful, but in others cases it is essential to have the complete dataset, in order to perform advanced tasks such as classification, clustering, or predictive modeling. In the same line, location data represents a particularly sensitive case of microdata. Each record typically contains precise latitude and longitude coordinates tied to an individual or event, and simple aggregation is often not sufficient to protect privacy, since even a small number of location points can reveal sensitive patterns such as home addresses, workplaces, etc. Usually, when releasing tabular microdata, the applied anonymization techniques focus on three types of attributes: identifiers (ID), which must be removed prior to publication; quasi-identifiers (QIs), which are anonymized through generalization; and sensitive attributes (SAs) which are protected by generalizing the QIs to avoid their association with an identified or identifiable individual. However, we can consider the case of releasing hospital patient records, where QIs such as age, gender, and demographic data are used as QIs, and the reason for admission is used as SA. Having this data is relevant for building a wide range concerning predictive and personalized health. However, what happens if we anonymize the microdata, publish it, and a week later a new (anonymized) release is published including new records? In this case, it is possible that an attacker could extract relevant information about the new records by analyzing that difference. [email protected] (J. Sáinz-Pardo Díaz); [email protected] (Á. López García)
ORCID (s): 0000-0002-8387-578X (J. Sáinz-Pardo Díaz); 0000-0002-0013-4602 (Á. López García)
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 1 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
In this sense, the idea of applying differential privacy (DP)[14] arises naturally, as it provides a formal definition for the privacy ensured with the guarantee that whatever additional information the attacker has, the level of privacy guaranteed by the privacy budget (𝜖) cannot be broken. DP is usually applied in its global form, i.e., we could think of applying DP to build aggregated data (when calculating statistics). This is evident if we consider the amount of noise we add in one case or the other (just taking into account the central limit theorem). Thus, although local differential privacy[35] has been studied theoretically, it is not widely implemented in the case of microdata publication, especially due to the high level of noise required to disturb the data. In local-DP, it is essential to strike a balance between the level of privacy applied and the amount of information we need to maintain so that the data remains statistically significant. In this line, with the aim of providing the community with tools that allow for the evaluation of such trade-offs, and whether the use of local-DP notions can be useful for such publication (taking into account the composition when applied to more than one column), we observe a lack of open tools for this purpose, especially if we think of those written in Python that allow us to easily incorporate it into data science, processing, visualization, and analysis flows. This gap is even more pronounced when working with location-based data and metric-privacy or geo-indistinguishability notions. This led us to the implementation of the open source Python library trasgoDP, which implements local-DP mechanisms, a geo-indistinguishability model for metric privacy, and utility metrics. It aims to complement pyCANON[30] and anjana[31], two Python libraries for anonymizing and checking the level of anonymity in tabular datasets, respectively. It forms the third part of a toolkit that seeks to enable the secure publication of microdata, because to promote open science, we need open data but also open source tools.
Related work When we refer to data privacy, we are referring to the protection of information from unauthorized use and access. In the current context, this is particularly relevant due to the massive generation and storage of data, which increases the risk of re-identification of individuals, data leaks, linking of data from different databases, etc[11]. As mentioned previously, anonymization methods are widely used for this purpose, particularly in accordance with the GDPR principle of data minimization, which states that anonymized data is no longer considered personal data related to an identified or identifiable individual. In this regard, there are different software products that enable the effective anonymization of tabular datasets, such as ARX[22] or anjana; for protecting statistical tables, such as 𝜏-argus or for creating microdata files, such as 𝜇-argus. However, in some cases, the goal is not to anonymize the data, but rather to apply other measures that protect it without resorting to the use of hierarchies, generalization or suppression methods. In this regard, we are interested in generating perturbed data (following the idea of synthetic data generation but starting from the raw data) using techniques based on differential privacy, thereby creating privatized versions of the original records. When it comes to software related to the implementation of mechanisms that ensure differential privacy, we must first review those that focus on global-DP, as these mechanisms are widely adopted due to their utility and numerous distributions are available. Specifically, we can highlight the following libraries: OpenDP[33], which is a modular Python library that implements a suite of statistical algorithms that satisfy the definition of differential privacy. pyDP[21] is a Python wrapper for Google’s Differential Privacy created by OpenMined. It provides differentially private algorithms, including statistics such as the mean, median, percentiles, etc., bounded by DP. The methods implemented use the Laplacian mechanism. In addition, diffprivlib[18] is the IBM differential privacy library written in Python, which implements the Laplace, Gaussian, Exponential, and randomized response methods for global DP, creating sanitized histograms, and for training ML models, including supervised and unsupervised learning. However, when we look for tools that allow us to apply DP mechanisms to raw data, the number of open-source solutions available is much smaller. Specifically, we can highlight three Python libraries: Multi-Freq-LDPy[3], which allows to perform multiple frequency estimation tasks under LDP guarantees; LDP-Toolbox[36], that explores utility and attacks trade-offs in local-DP, and pure-LDP[10], which provides simple implementations of state-of-the-art LDP frequency estimation algorithms. More specifically, regarding location-based data, the implications of introducing metrics and distance notions to the differential privacy paradigm was proposed in 2013 by Chatzikokolakis et al. in [7]. Then, the notion of geoindistinguishability was introduced in by Andrés et al with the idea of adding random noise to the user’s location from a planar Laplace distribution [2]. In addition, this research line has been extended with new algorithms that aim to solve the problems regarding the protection of isolated locations by the Laplace mechanism [6]. However, to the best of our knowledge, no actively maintained, general-purpose open-source Python library currently provides J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 2 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy DATA RELEASING Microdata releasing
Aggregate data releasing
Individual-level (per row) records are released
Summarized-level data are released (no individual records)
Example: census data in which each row represents an individual ID
Age
Gender
Country
Avg Income
ID1 ID2 ID3 ...
30 40 51 ...
Female Male Female ...
Spain France Germany ...
39,500€ 44,200€ 51,000€ ..
Advantages: • Flexible use for research. • More complete analysis. • Allows to capture variability. Disadvantages:
Example: table showing the average income per country Country
Avg. Income
Spain France Germany
40,000€ 45,000€ 50,000€
Advantages: • Stronger privacy. • Easy to release publicly. • Lower storage and management cost. Disadvantages:
• Privacy concerns.
• Less details for analysis.
• Risk of linkage and disclosure.
• Limited use for research and models training.
• Strong controls are required.
• Hidden variability.
Protection methods:
Protection methods:
• Pseudonymization.
• Global differential privacy.
• Anonymization.
• Cell aggregation.
• Local differential privacy.
• Data perturbation.
Figure 1: Difference between microdata releasing and aggregate data publishing.
an accessible implementation of such mechanisms integrated within a broader local-DP toolkit focused on tabular microdata releasing. In light of this, it was decided to implement trasgoDP, which implements algorithms for LDP applied directly to categorical and numerical data (taking into account the limitations in terms of the amount of noise to be added), a mechanism for geo-indistinguishability and thus metric privacy, as well as functions that allow us to measure and compare the utility of DP-protected data versus raw data, and to quantify the divergence between them. The goal is to build a comprehensive toolkit for data privatization alongside related tools such as anjana, which anonymizes tabular data, and pyCANON, which determines the level of anonymity of a dataset. Thus, trasgoDP forms the third component of this modular toolkit written in Python, which can be easily integrated into a data science pipeline.
Data releasing We have already discussed the difference between microdata and aggregated data, but it is important to elaborate on this point and compare both aspects to highlight why we are sometimes interested in real microdata releasing and thus why it is important when working on differential privacy to have tools not only for global-DP, but also for local-DP. Specifically, the differences between these two aspects are summarized in the diagram in Figure 1. In both cases, one method that can be used for data protection is differential privacy. As already mention, we can think on both local and global versions, depending on which step of the data processing pipeline we apply it. The workflows for these two approaches are compared in Figure 2.
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 3 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
Figure 2: Local-DP vs global-DP: schematic comparison.
Then, let’s start giving the formal definition for this theoretical guarantee. Here it is important to take into account that differential privacy is not a method or algorithm itself, but a mathematical definition.
Differential privacy As in this work we are focusing in local-DP instead of in global-DP, the following definitions will be given for the discrete case. Definition 1. Local 𝝐-differential privacy. A randomized algorithm , with domain and range , satisfies local 𝝐-differential privacy if for any inputs 𝑦, 𝑦′ ∈ and for any possible output 𝑟 ∈ it is satisfied that: ℙ[(𝑦) = 𝑟] ≤ 𝑒𝜖 ℙ[(𝑦′ ) = 𝑟], with 𝜖 ≥ 0. Definition 2. Local (𝝐, 𝜹)-differential privacy. A randomized algorithm , with domain and range , satisfies local (𝝐, 𝜹)-differential privacy if for any inputs 𝑦, 𝑦′ ∈ and for any possible output 𝑟 ∈ it is satisfied that: ℙ[(𝑦) = 𝑟] ≤ 𝑒𝜖 ℙ[(𝑦′ ) = 𝑟] + 𝛿, with 𝜖 ≥ 0 and 𝛿 ∈ [0, 1]. Concerning ensuring DP guarantees, different mechanisms can be considered. Specifically, let’s review the first ones implemented in trasgoDP in version v2.0.1 for both numerical and categorical attributes.
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 4 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
Numerical attributes The following mechanism adopted for numerical attributes are usually applied in global-DP
scenarios, but we can easily adopt them for the local case. Specifically, in the following definitions we need to define a proper notion of the sensitivity and we can consider the function 𝑓 (⋅) as the identity applied on each record. Then, is a widely known result that for ensuring 𝜖-global-DP, it is proven that the following mechanism (Laplace mechanism) can be adopted: Definition 3. Laplace mechanism. Given any function 𝑓 ∶ ⟶ ℝ𝑘 , we define the Laplace mechanism as: 𝐿 (𝑥, 𝑓 (⋅), 𝜖) ∶= 𝑓 (𝑥) + (𝑌1 , … , 𝑌𝑘 ), where 𝑌𝑖 , ∀𝑖 ∈ {1, … , 𝑘} are independent and identically distributed (i.i.d.) variables from the Laplace distribution with location 0 and scale Δ1 (𝑓 )∕𝜖 (equivalently 𝐿𝑎𝑝𝑙𝑎𝑐𝑒(0, Δ1 (𝑓 )∕𝜖)), being Δ1 (𝑓 ) the 𝑙1 -sensitivity. Concerning (𝜖, 𝛿)-global-DP guarantees the Gaussian mechanism can be applied: Definition 4. Gaussian mechanism. Given any function 𝑓 ∶ ⟶ ℝ𝑘 , we can define the Gaussian mechanism as: 𝐺 (𝑥, 𝑓 (⋅), 𝜖, 𝛿) ∶= 𝑓 (𝑥) + (𝑌1 , … , 𝑌𝑘 ),
where 𝑌𝑖 , ∀𝑖 ∈ {1, … , 𝑘}√are independent and identically distributed (i.i.d.) variables from the Gaussian distribution 𝑁(0, 𝜎 2 ), with 𝜎 =
Δ2 (𝑓 ) 2 log(1.25∕𝛿) , being Δ2 (𝑓 ) the 𝑙2 -sensitivity. 𝜖
Note that for moving to the local-DP case, and assuming that we are dealing with bounded numerical data (e.g. 𝑥 ∈ [𝑎, 𝑏]), then Δ1 = Δ2 = 𝑏 − 𝑎. In view of the above, it is important to note that the magnitude of the noise required differs substantially between the central (or global) model and the local model of differential privacy. In the case of global differential privacy (central model), when aggregated statistics such as the mean are privatized, sensitivity scales as (𝑏 − 𝑎)∕𝑛. Consequently, the noise added by Laplace or Gaussian mechanisms decreases proportionally to 1∕𝑛. This implies that as the sample size increases, the necessary perturbation decreases and statistical utility improves significantly. On the contrary, in the local differential privacy model, each individual perturbates their own data before sharing it, and in this case the sensitivity is 𝑏 − 𝑎, independent of the sample size. Therefore, the magnitude of the noise does not depend on n, which leads to a significantly higher variance compared to the global approach. This difference is the fundamental mathematical reason why mechanisms under LDP are considerably noisier than under the global assumption. In this line, we also have to note that, as the Gaussian mechanism provides (𝜖, 𝛿)-guarantees, in the local-DP approach we have to carefully select the value of the parameter 𝛿. Specifically, an initial approach should be to fix 𝛿 << 1∕𝑛, with 𝑛 the number of records of the dataset. However, as already stated, the local approach may be particularly relevant in scenarios where there is no central trusted entity, when users want to maintain direct control over their data, or in distributed systems for mass information collection. Likewise, LDP is particularly suitable for the construction of private histograms or frequency estimation, specifically in cases in which the large number of participants partially compensates for the increase in variance induced by individual disturbance.
Categorical attributes For categorical attributes, privacy preservation under the local model can be achieved
through mechanisms specifically designed for discrete domains. In this case, we can apply the Exponential mechanism or the Randomized Response one. Again, as in the numerical setting, we need to take into account the trade-off between privacy and utility, concerning the amount of noise added compared to the global approach, as in this setting each records is perturbed independently. Note that for the Randomized Response mechanism we can define two strategies depending weather or not we are dealing with binary attributes. For the non binary case we will call this method 𝑘-ary Randomized Response. These three mechanisms are proven to provide 𝜖-DP guarantees. Definition 5. Exponential mechanism. Be 𝐷 the set of inputs, the set of outputs, 𝑟 ∈ and Δ(𝑔) the sensitivity of the utility or score function 𝑔. The exponential mechanism outputs 𝑟 with probability ℙ[𝑟] defined as follows: ( ) 𝑒𝑥𝑝 𝜖⋅𝑔(𝐷,𝑟) 2Δ(𝑔) ℙ[𝑟] = ∑ ( ) 𝜖⋅𝑔(𝐷,𝑟′ ) 𝑟′ ∈ 𝑒𝑥𝑝 2Δ(𝑔) J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 5 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
Definition 6. Randomized Response (RR) mechanism for binary attributes. Be = {𝑦, 𝑦}) ̃ the data domain. Given 𝑒𝜖 the privacy budget 𝜖, be 𝑝(𝜖) = 𝑒𝜖 +1 , the randomized response algorithm for binary attributes for the value 𝑦 given 𝜖 returns 𝑦 as follows (note that 𝑦̃ = ¬𝑦): { 𝑦 with probability 𝑝(𝜖) 𝑦= . ¬𝑦 with probability 1 − 𝑝(𝜖) Note that if 𝜖 = 0, 𝑝(𝜖) = 0.5, and 𝑙𝑖𝑚𝜖→+∞ 𝑝(𝜖) = 1. Definition 7. k-ary Randomized Response (RR) mechanism. Be 𝑘 the number of different values in the data domain ( = {𝑦1 , … , 𝑦𝑘 }). The k-ary randomized response algorithm for the value 𝑦 given 𝜖 is the true value 𝑦 if 𝑏 = 0 and otherwise it is sampled from following an uniform distribution. In this approach we get 𝑏 ∼ 𝐵𝑒𝑟(𝑘∕(𝑒𝜖 + 𝑘 − 1)), with Ber the Bernoulli distribution. The three mechanisms described above are proposed as perturbation strategies applied independently to individual records of a categorical column in a dataframe or categorical array. Within the trasgoDP library, they are implemented as local privatization operators, enabling the transformation of categorical attributes into synthetic or noised representations while maintaining formal 𝜖-DP guarantees. Finally, it is important to note that when LDP mechanisms are applied to multiple columns of a dataset, the overall privacy guarantee degrades according to the sequential composition theorem. This means that if we apply local-DP with 𝜖𝑖 ∀𝑖 ∈ {1, … , 𝑛𝑐𝑜𝑙 } (with 𝑛𝑐𝑜𝑙 the number of columns in the database to be perturbed), the combined release ∑𝑛𝑐𝑜𝑙 𝜖𝑖 . This implies that practitioners must treat the privacy budget as a global resource to be distributed satisfies 𝑖=1 across columns, rather than as a per-column parameter. More details concerning composition theorems in DP can be found in [20].
Metric privacy When dealing with location-based data, we can naturally think of the notion of metric differential privacy (also known as d-privacy), and in the following referred to simply as metric privacy. The main objective of metric privacy is to take into account the geographical distance in the development of location privacy solutions and geoindistinguishability, in order to protect an individual’s location in location-based services. It is formally defined as follows: Definition 8. 𝝐-metric-privacy. A randomized algorithm , with domain and range , with provided with a metric 𝑑 ∶ 2 ⟶ ℝ≥0 , satisfies 𝝐-metric-privacy (see [6]) if for any inputs 𝑦, 𝑦′ ∈ and for any possible output 𝑟 ∈ it is satisfied that: ′ ℙ[(𝑦) = 𝑟] ≤ 𝑒𝜖⋅𝑑(𝑦,𝑦 ) ℙ[(𝑦′ ) = 𝑟], with 𝜖 ≥ 0. Note that we assume that is provided with a metric space. From the previous definition we can note that the inputs that are closer in relation to the given metric 𝑑 will be more indistinguishability to an attacker, while the ones that are more distant will be more easily discernible. Then, we can get the following definition for the concept of geo-indistinguishability as presented in [2]. Definition 9. Geo-indistinguishability. A mechanism provides guarantees of geo-indistinguishability if and only if for any radius 𝑥 > 0 we can ensure 𝜖-metric-privacy within the radius 𝑥. In trasgoDP, geo-indistinguishability is achieved by perturbing a true location (𝑙𝑎𝑡, 𝑙𝑜𝑛) through the addition of a two-dimensional noise vector drawn in polar coordinates. Concretely, an angle 𝜃 ∼ Unif[0, 2𝜋) is drawn together with a radius 𝑟 sampled from a Gamma distribution with shape 2 and scale 1∕𝜖, according to [2]. Then, the mechanism implemented is defined as follows: Definition 10. Geo-indistinguishability mechanism Given a location (𝑙𝑎𝑡, 𝑙𝑜𝑛) ∈ [−90, 90] × [−180, 180], a privacy budget 𝜖 > 0 and the Earth’s radius 𝑅⊕ (in meters), the mechanism draws 𝜃 ∼ Unif[0, 2𝜋) and 𝑟 ∼ Gamma(2, 1∕𝜖), and outputs the perturbed location (𝑙𝑎𝑡∗ , 𝑙𝑜𝑛∗ ) as: 𝑙𝑎𝑡∗ = 𝑙𝑎𝑡 +
𝑟 cos 𝜃 180 ⋅ , 𝑅⊕ 𝜋
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 6 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
𝑙𝑜𝑛∗ = 𝑙𝑜𝑛 +
𝑟 sin 𝜃 180 ⋅ , 𝑅⊕ cos(𝑙𝑎𝑡 ⋅ 𝜋∕180) 𝜋
returning the triplet (𝑙𝑎𝑡∗ , 𝑙𝑜𝑛∗ , 𝑟). Note that according with the previous definition, the amount of noise added does not depend on the sample size 𝑛, but here it is further determined by the geometry of the domain.
RESULTS Local differential privacy mechanisms: use examples As already stated, in the initial implementation of trasgoDP, three mechanisms for categorical attributes and two for numerical ones are implemented. In Table 1, these five methods are shown together with the name of the corresponding function in the library and the input expected for applying them. Note that in the following we show the function for the case of working directly with pandas dataframe. However, all these mechanisms have their corresponding functions for the cases in which numpy arrays or lists of values are introduced, as will be explained in the following. Note that for numerical attributes, the lower and upper bounds are parameters that must be fixed using public domain knowledge (e.g. a plausible age range) established independently of the dataset being sanitized, the same as the positive label for binary attributes. trasgoDP requires these bounds to be provided explicitly by the user, and does not infer them automatically from the data, as doing so would make the sensitivity data-dependent and would invalidate the formal privacy guarantees. Finally, the default 𝛿 parameter for the Gaussian mechanism, which is customizable, has been set to 1e-3. Method Laplace mechanism Gaussian mechanism Exponential mechanism RR mechanism (binary) RR k-ary mechanism
Function and input dp_clip_laplace(df, column, epsilon, lower_bound, upper_bound, new_column) dp_clip_gaussian(df, column, epsilon, lower_bound, upper_bound, delta, new_column) dp_exponential(df, column, epsilon, new_column) dp_randomized_response_binary(df, column, epsilon, new_column, positive_label) dp_randomized_response_kary(data, epsilon)
Table 1: Function which implement each LDP mechanism for the case of pandas dataframes. Following the examples conducted for the cases of both pyCANON and anjana, in the Example Code 1 we present an example of application of the five mechanisms to different columns of a pandas dataframe (one mechanism each time). This example is conducted for the classic adult dataset[4], which is an extraction of the 1994 Census database composed of 32,561 rows in the train set (used in this work). The data has been previously processed and it is available in the examples folder of the trasgoDP library. Note that, in the following example, in each case we create a new dataframe with the resulting column obtained when applying DP. import pandas as pd from trasgodp.numerical import dp_clip_laplace, dp_clip_gaussian from trasgodp.categorical import ( dp_exponential, dp_randomized_response_kary, dp_randomized_response_binary, ) # Read the data (already processed): data = pd.read_csv("adult_processed.csv") epsilon = 1 # Apply DP for the attribute age with the Laplace mechanism: lower_bound = 16 upper_bound = 100 df_lap = dp_clip_laplace(data, "age", epsilon, lower_bound, upper_bound, new_column=True) J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 7 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
# Apply DP for the attribute age with the Gaussian mechanism: df_gauss = dp_clip_gaussian(data, "age", epsilon, lower_bound, upper_bound, new_column=True) # Apply DP for the attribute workclass with the Exponential mechanism: df_exp = dp_exponential(data, "workclass", epsilon, new_column=True) # Apply DP for the attribute workclass with the k-ary Randomized Response mechanism: df_kary = dp_randomized_response_kary(data, "native-country", epsilon, new_column=True) # Apply DP for the attribute sex with the Randomized Response mechanism (binary): df_bin = dp_randomized_response_binary(data, "sex", epsilon, new_column=True)
Example Code 1: Example: applying the five implemented mechanisms to different columns in the case of the adult dataset. In case the functions presented in Table 1 should be applied directly to a list or a numpy array rather than to a pandas dataframe, Table 2 shows the functions implemented for that purpose. All of them return a numpy array containing the transformation of the original list by adding DP. Method Laplace mechanism Gaussian mechanism Exponential mechanism Randomized response mechanism (binary) Randomized response k-ary mechanism
Function and input dp_laplace_array(data, epsilon, lower_bound, upper_bound) dp_clip_gaussian(data, epsilon, lower_bound, upper_bound, delta) dp_exponential_array(data, epsilon) dp_randomized_response_binary_array(data, epsilon, positive_label) dp_randomized_response_kary_array(data, epsilon)
Table 2: Function which implement each LDP mechanism for the case of a list or numpy array.
Metric privacy via geo-indistinguishability As already introduced, trasgoDP implements the geo-indistinguishability mechanism presented in Definition 10 to sanitize pairs of latitude and longitude. Table 3 shows the function implemented in trasgoDP in this regard and for plotting the results, together with its expected input for the case of a pandas dataframe. Note that unlike the mechanisms for numerical and categorical data described above, this mechanism does not operate on a single column but jointly perturbs a pair of coordinates Method Geo-indistinguishability mechanism Plot the resulting map
Function and input metric_privacy(df, column_lat, column_lon, epsilon, new_cols, earth_radius_m, seed) plot_metric_dp_map(df_dp, column_lat, column_lon, save_file)
Table 3: Metric privacy related functions: applying the geo-indistinguishability mechanism for a pandas dataframe and plotting the resulting map with the original and privatized (lon, lat) coordinates and the radius. Coming back to Table 3, in the metric_privacy() function the inputs are as follows: df (the pandas dataframe with the data), column_lat (column with the latitude), column_lon (column with the longitude), epsilon (the privacy budget), new_cols (boolean value indicating whether two new columns will be added or whether the original columns will be replaced), earth_radius_m (radius of the Earth, set to 6,371,000 meters) and seed (random seed, by default 42). The output of this function is a new dataframe with new columns for latitude and longitude sanitized (or replacing the original ones) and a column with the calculated radius in each case (allowing users to inspect the magnitude of the perturbation associated with a chosen 𝜖). To visually inspect the effect of the mechanism, trasgoDP also provides plot_metric_dp_map(), which renders an interactive map comparing original and privatized locations, as illustrated below for the trip dataset. This function J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 8 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
allows to save the map in HTML format and has the following input values: df_dp (dataframe with the latitude and longitude columns privatized using the function metric_privacy() with the attribute new_cols set to true), column_lat (original column with the latitude), column_lon (original column with the longitude) and save_file (path for saving the map in HTML format). One code example for using these two functions is given in Example Code 2: import pandas as pd from trasgodp.geoindis import metric_privacy, plot_metric_dp_map # Read the data data = pd.read_csv("trip_data.csv") column_lat = "pickup_latitude" column_lon = "pickup_longitude" # Apply metric privacy creating new columns for lat and lon: epsilon = 1.e-3 data_priv = metric_privacy(data, column_lat, column_lon, epsilon, new_cols=True) # Plot and save the map: plot_metric_dp_map(data_priv, column_lat, column_lon, save_file="example_map.html")
Example Code 2: Example: applying the geo-indistinguishability mechanism to the NYC taxis dataset and plotting the resulting map.
Privacy-utility trade-off metrics With the aim of providing users with metrics to quantify the quality of the noised data generated using DP, particularly in terms of distributional consistency, we have implemented specific functions in trasgoDP. First, the most intuitive approach is to compute the divergence between the original column and the one obtained after applying DP, in order to quantify the information loss. To this end, different divergence metrics are calculated, including Total Variation Distance (TVD), Jensen-Shannon divergence (JS), and Kullback-Leibler divergence (KL). In addition, we have defined a novel metric to quantify correlation loss (expressed as a percentage), which is based on measuring how well correlations between features are preserved. The idea is to assess how the correlation between a given column and a set of other features changes after applying differential privacy. To do this, a set of features is selected with respect to which the correlation is computed using a chosen method. Then, the absolute error between the original and perturbed correlations is calculated and transformed into a relative error expressed as a percentage. In this way, large changes in correlations result in higher loss values. Since the metric is relative, it evaluates the error with respect to the “strength” of the original correlations. Thus, the procedure to compute the proposed metric is as follows:
Correlation loss (%) 1. Be 𝐷 ∈ ℝ𝑛×𝑑 the original dataset and 𝐷′ ∈ ℝ𝑛×𝑑+1 the privatized one (if one new column transformed with DP), or 𝐷′ ∈ ℝ𝑛×𝑑 (if the privatized column has been substituted). Let’s assume that 𝐷′ ∈ ℝ𝑛×𝑑 . Be 𝐹 = {𝑋1 , … , 𝑋𝑓 } the set of features selected, 𝑓 ≤ 𝑑. 2. For each categorical feature 𝑋𝑗 ∈ 𝐹 , we define a function 𝜙𝑗 ∶ 𝑗 → ℤ with 𝑗 the set of values observed in 𝐷 and 𝐷′ . Then we get the transformed datasets 𝐷̃ and 𝐷̃ ′ . ̃ ], 𝑋 ′ = 𝐷̃ ′ [𝐹 ]. 3. We extract the matrix based on the selected features: 𝑋 = 𝐷[𝐹 4. Be 𝜌(⋅, ⋅) a correlation method (Pearson, Spearman o Kendall). Then we calculate the correlation matrix as follows: 𝑅 = (𝜌𝑖𝑗 )𝑑𝑖,𝑗=1 with 𝜌𝑖𝑗 = 𝜌(𝑋𝑖 , 𝑋𝑗 ) and 𝑅′ = (𝜌′𝑖𝑗 )𝑑𝑖,𝑗=1 with 𝜌′𝑖𝑗 = 𝜌(𝑋𝑖′ , 𝑋𝑗′ ) 5. Remove autocorrelation: = {(𝑖, 𝑗) ∶ 𝑖 ≠ 𝑗, 1 ≤ 𝑖, 𝑗 ≤ 𝑑} J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 9 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
6. Difference between correlations: Δ𝑖𝑗 = |𝜌𝑖𝑗 − 𝜌′𝑖𝑗 |,
(𝑖, 𝑗) ∈ .
1 7. Mean of the difference between correlations: 𝜇Δ = || 1 ∑ matrix 𝜇𝑅 = || (𝑖,𝑗)∈ |𝜌𝑖𝑗 |.
∑
′ (𝑖,𝑗)∈ |𝜌𝑖𝑗 − 𝜌𝑖𝑗 |, and mean of the original correlation
𝜇
8. Correlation loss (%): = 100 ⋅ 𝜇Δ . 𝑅
The function for computing the correlation loss is available in the metrics subpackage, along with the functions for computing divergences. In the latter case, it is sufficient to provide the original and differentially private datasets, the sanitized column, and whether it has been replaced or newly created. In contrast, for the correlation loss, instead of specifying a single sanitized column, we provide the set of features over which we want to measure the correlation. In both cases, it is necessary to indicate whether the sanitized data has replaced the original column or has been added as a new one. A battery of examples on how to use this novel function, along with guidance on their interpretation, are presented below.
Tests and examples Adult dataset For conducting the test of the library, the adult dataset was used. In addition, we also use such dataset for illustrating the correlation loss function. First, in Table 4 we show an extraction of ten rows and eight columns that have been used during the testing phase. workclass
education
marital-status
occupation
sex
native-country
age
salary-class
Private Private Self-emp-not-inc Local-gov Self-emp-not-inc Self-emp-not-inc Self-emp-inc Local-gov Self-emp-not-inc Private
11th Bachelors HS-grad Some-college HS-grad 9th 5th-6th Bachelors HS-grad Bachelors
Never-married Never-married Married-civ-spouse Divorced Married-civ-spouse Married-civ-spouse Married-civ-spouse Divorced Married-civ-spouse Never-married
Machine-op-inspct Exec-managerial Other-service Adm-clerical Farming-fishing Craft-repair Transport-moving Adm-clerical Other-service Exec-managerial
Male Male Male Female Male Male Male Female Female Female
Puerto-Rico Germany Canada Mexico Cambodia Portugal Cuba Honduras Italy United-States
48 30 56 47 42 26 47 36 47 49
<=50K <=50K >50K <=50K >50K <=50K <=50K <=50K <=50K >50K
Table 4: Sample rows and columns extracted from the adult dataset with a selected subset of columns that can act as quasi-identifiers and sensitive attribute for data privatization tasks. In order to calculate the correlation loss and measure the impact of different values of the privacy budget, the results obtained for a numerical column (age) are shown below, using both the Laplace and Gaussian mechanisms; for a nonbinary categorical column (workclass), using the Exponential and Randomized Response mechanisms; and finally for a binary categorical column (sex), using both the Exponential and Binary Randomized Response mechanisms. Specifically, the following set of 𝜖 values were evaluated: 𝜖 ∈ {0.1 + 𝑘 ⋅ 0.3|𝑘 ∈ {0, 1, 2, 3}} ∪ {2 + 𝑘 ⋅ 0.5|𝑘 ∈ ℤ, 1 ≤ 𝑘 ≤ 20}, with 𝛿 = 10−6 in the case of the Gaussian mechanism. For the age, 16 and 100 were used as lower and upper bounds respectively. Concerning the features used for computing the correlation, all the columns presented in Table 4 were used. Finally, for each value of 𝜖, the procedure was repeated 10 runs to obtain the mean and standard deviation for each case. These results are shown graphically in Figure 3 for each variable and mechanism applied. From the graphs in Figure 3, it is immediately apparent that, as expected, a smaller value of 𝜖 increases the correlation loss, while a larger value substantially reduces it. Regarding the comparison of numerical mechanisms, it is observed that the flexibility provided by the delta parameter (by allowing the privacy budget to be exceeded with a certain probability) results in a correlation loss up to three times greater than with the Laplace mechanism for the same value of 𝜖. Thus, the curve corresponding to the Gaussian mechanism follows an approximately linear decreasing trend, while that of Laplace decreases exponentially, converging more rapidly toward low loss values. As for the results for the non-binary categorical variable (workclass), the Exponential mechanism exhibits a slower reduction in correlation loss than the k-ary Randomized Response, with both following an inverse sigmoid curve, although with different slopes. This behavior is also observed for the binary categorical variable (sex), where the correlation is close to zero with a privacy budget of approximately 𝜖 = 5 for the k-ary mechanism and 𝜖 = 10 for the J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 10 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
(a) Laplace mechanism. Column: Age.
(b) Gaussian mechanism. Column: Age.
(c) Exponential mechanism. Column: Workclass.
(d) k-ary randomized response mechanism. Column: Workclass.
(e) Exponential mechanism. Column: Sex.
(f) Binary randomized response mechanism. Column: Sex.
Figure 3: Utility loss calculated when varying 𝜖 during 10 runs, based on the differences between the correlation matrix. Comparison between mechanisms for numerical and categorical (binary and non-binary) attributes. Adult dataset.
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 11 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
Exponential mechanism. It is immediately noticeable how categorical variables, by taking a much smaller range of values, require a lower value of 𝜖 to maintain a reasonable trade-off between privacy and utility in terms of correlation, while numerical variables covering a wider range of values, such as age, require higher values of 𝜖 for the perturbed distribution to resemble the original.
Global cancer patients dataset We will now consider a second example, based on the application of DP mechanisms implemented to the global cancer patients dataset[16], which contains synthetic global cancer patient data reported from 2015 to 2024, including demographic variables such as age, gender, and country, as well as genetic risk, cancer type, and stage, in addition to treatment cost, survival rate, and severity. Even if this dataset doesn’t contain real-world patient data, it is inspired by statistics from the World Health Organization (WHO), data from the National Institutes of Health (NIH) and trends extracted from the Global Cancer Observatory (GCO) and Our World in Data. Table 5 shows an extract of 10 rows and the 10 columns mentioned above, selected from among the 15 columns and 50000 rows available. Note that this dataset is used here only as an illustrative example to demonstrate the applicability of the implemented mechanisms and metrics across two datasets with different statistical properties. Age
Gender
Country_Region
Year
Genetic_Risk
Cancer_Type
Cancer_Stage
Treatment_Cost_USD
Survival_Years
Target_Severity_Score
71 34 43 40 41 85 83 86 22 67
Male Male Female Female Male Other Other Male Male Female
UK China Brazil USA Canada India Russia Australia Germany China
2021 2021 2017 2018 2021 2016 2023 2015 2018 2015
6.4 1.3 5.1 6.4 5.1 2.8 3.2 8.6 9.5 2.6
Lung Leukemia Skin Colon Cervical Leukemia Prostate Skin Cervical Liver
Stage III Stage 0 Stage III Stage I Stage 0 Stage III Stage II Stage 0 Stage IV Stage II
62913.44 12573.41 77977.12 49619.66 9790.83 17158.84 38290.91 98333.43 33468.99 65060.21
5.9 4.7 2.9 0.4 1.0 3.0 1.1 0.1 9.5 9.7
4.92 4.65 3.62 6.03 5.05 4.55 4.09 4.15 5.98 3.85
Table 5: Sample rows and columns extracted from the global cancer dataset with a selected subset of columns that can act as quasi-identifiers and sensitive attribute for data privatization tasks. In this case, there are no binary variables available (the gender variable takes on three possible values), but we present the results for the numerical variable age, as well as the categorical variable country. Note that in both this example and the previous one, we focus on this kind of demographic variables rather than on sensitive attributes, since a priori, what we want is to protect against the possibility of identifying an individual in the database; therefore, we focus specifically on the QIs. Thus, Figure 7 shows the results obtained for these two variables using the Laplace and Gaussian mechanisms for age (again using 16 and 100 as lower and upper bounds) and the exponential and k-ary mechanisms for country. Concerning the features used for computing the correlation loss, in addition to the feature transformed with DP in each case, the following were used in this example: Genetic_Risk, Cancer_Type, Cancer_Stage, Treatment_Cost_USD, Survival_Years and Target_Severity_Score. Note that, once again, the same privacy budget values are used as in the previous example, and the experiments are repeated 10 runs in each case. In this case, the loss of correlation is significantly smaller than in the previous cases, although the mechanisms follow a similar trend curve: Laplace decreases more quickly than Gaussian, and k-ary more than exponential. Regarding the differences in the results obtained for the same epsilon values with both datasets, it is important to start by analyzing the distributions of the variables in each case. As for the number of unique values, the numbers are very similar: while the adult dataset had 73 unique values for age, the cancer dataset contains 70 unique values. For the country variable, this dataset has 10 unique values, compared to the 9 unique values for workclass in adult. With regard to column sex in the adult dataset, the value ’Male’ appears in the 66.92% of the cases times, compared to ’Female’ in the 33.08%. However, the most significant differences are found in the distributions of the variables. Starting with age, Figure 4 shows the age distribution in both cases-approximately uniform in the global cancer dataset and unimodal with moderate positive skewness (right-skewed) in the adult dataset. Specifically, applying a 𝜒 2 test to the age variable in the global cancer dataset comparing the observed frequencies with the expected frequencies under a uniform distribution (5,000 values per bin in the case of 10 bins), yields a p-value of 0.217, which does not allow us to reject the null hypothesis that the variable follows a uniform distribution. In contrast, in the case of adult, a normal test using the D’Agostino and Pearson’s tests confirms that the variable does not follow a normal distribution (with a p-value=0, as expected); given a skewness of 0.559, it is concluded that the J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 12 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
Figure 4: Distribution of the age feature in the adult dataset and in the global cancer dataset (10 bins).
Figure 5: Distribution of the age feature in the adult dataset and in the global cancer dataset (50 bins).
Figure 6: Distribution of the non-binary categorical features analyzed in the adult dataset (workclass) and in the global cancer dataset (country ).
distribution presents moderate positive skewness. The previous is verified when using 10 bins, as shown in Figure 4. However, Figure 5 shows the results obtained when considering 50 bins in each case. Under that approach, we can note (and verify with a hypothesis test), that for the global cancer dataset, the age doesn’t follow a uniform distribution. In addition, Figure 6, presents bar charts showing the distributions of the workclass and country variables. While the variable studied in the adult dataset has a very different distribution for each possible value, the variable studied in the global cancer dataset exhibits a distribution that follows a uniform one. The impact of these distributions is clearly J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 13 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
reflected in Figures 3 and 7, with the results obtained for the correlation loss in the the global cancer dataset being smaller than the ones for the adult dataset under a similar number of unique values and same noise multipliers.
(a) Laplace mechanism. Column: Age.
(b) Gaussian mechanism. Column: Age.
(c) Exponential mechanism. Column: Country_Region.
(d) k-ary randomized response mechanism. Column: Country_Region.
Figure 7: Utility loss calculated when varying 𝜖 during 10 runs, based on the differences between the correlation matrix. Comparison between mechanisms for numerical and categorical attributes. Global patients dataset.
Concerning the use of this metric and calculation of the divergence between the distribution of a column in the original and DP datasets, two functions are available in the trasgodp.metrics package: correlation_loss() and divergence_distributions(). The first one returns a float with the utility loss (%), and the second one returns a dictionary with the divergence metrics (TVD, JS, KL). Concerning the inputs, both require the original dataframe, the one with DP, the column to be analyzed and if a new column was created when applying DP. In addition, for the correlation loss the list of features to be included in the correlation analysis and the method used (Pearson, Kendall, or Spearman). Example Code 3 shows how to perform this calculation for the column age of the global cancer dataset, given the set of features used in the previous example. import pandas as pd import numpy as np from trasgodp import numerical, metrics # Read and process the data df_original = pd.read_csv("global_cancer_patients_2015_2024.csv")
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 14 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy # Select the features to be used when computing the correlation features = [ "Genetic_Risk", "Cancer_Type", "Cancer_Stage", "Treatment_Cost_USD", "Survival_Years", "Target_Severity_Score" ] # Apply DP with the Laplace mechanism column = 'Age' epsilon = 1 df_dp = numerical.dp_clip_laplace( df_original, column, epsilon, lower_bound = 16, upper_bound = 100, new_column=True ) # Get the correlation loss for the column 'Age': features_col = features + [column] corr_loss = metrics.correlation_loss( df_original, df_dp, features_col, new_column=True ) print(f'Correlation loss (%) for {column}: {np.round(corr_loss, 3)}')
Example Code 3: Example: calculating the correlation loss in a particular run for the column age of the global cancer dataset.
NYC taxis trip dataset Finally, to test the geo-indistinguishability mechanism and assess the metric-privacy guarantees provided by trasgoDP, we use a publicly available dataset containing information about the New York City taxi trips through 2013 [9]. This dataset gained popularity in the privacy research community after it was demonstrated that the pseudonymization applied to the original version (hashing taxi license and medallion numbers using the MD5 function) could be easily reversed, since the space of valid license numbers is small enough to be exhaustively searched, allowing for the complete re-identification of each individual trip. In addition, based on the de-anonymized dataset, different works cross-referenced the pickup and drop-off locations with time-stamped photographs of celebrities getting into or out of taxis in New York City, successfully inferring the exact fares for their trips, the tips, and, in several cases, their destinations, which are strongly correlated with their home or work addresses. This case illustrates how location microdata that seem harmless or that only allow only limited identification on their own, can reveal a great amount of sensitive information when linked to publicly available auxiliary information [13]. This reinforces the need for formal privacy guarantees, such as those provided by geo-indistinguishability. Specifically, a version of this data, available in the examples folder of the trasgoDP GitHub repository has been used. It contains 100,000 records, including information such as the latitude and longitude coordinates of departure and arrival, the duration, distance and date of the trip, the pick-up and drop-off time, number of passengers etc. Table 6 shows an extraction of the first ten rows, restricted to the subset of columns mentioned above and only departure coordinates for the sake of simplicity. Given this dataset, the geo-indistinguishability mechanism is applied to privatize the latitude and longitude values of the pick up, producing a new pair of coordinates for each record together with the corresponding perturbation radius. In addition, as already explained as shown in Example Code 2, users can render the resulting map as an interactive HTML file, which can be saved locally, allowing a direct visual comparison between the original and privatized locations. Figure 8 shows a screenshot of the map with an example of how one of the original longitude and latitude pick-up points has been perturbed using the algorithm presented in the Example Code 3 for this dataset. J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 15 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
pickup_datetime
dropoff_datetime
passenger_count
trip_time_in_secs
trip_distance
pickup_longitude
pickup_latitude
2013-12-05 22:45:00 2013-12-05 22:53:00 2013-12-05 22:51:00 2013-12-05 22:39:00 2013-12-05 22:54:00 2013-12-03 13:18:00 2013-12-03 13:16:00 2013-12-03 13:25:00 2013-12-05 23:49:00 2013-12-03 14:22:00
2013-12-05 22:54:00 2013-12-05 22:57:00 2013-12-05 22:55:00 2013-12-05 22:55:00 2013-12-05 22:58:00 2013-12-03 13:31:00 2013-12-03 13:29:00 2013-12-03 13:34:00 2013-12-05 23:55:00 2013-12-03 14:37:00
4 6 1 1 1 1 1 1 1 1
540 240 240 960 240 780 780 540 360 900
2.08 0.56 1.44 2.03 0.13 1.21 3.43 4.25 2.10 1.69
-73.973854 -73.982300 -73.989204 -73.974663 -73.985802 -73.990067 -74.014221 -73.972458 -73.983429 -73.994888
40.762615 40.766106 40.757675 40.763454 40.722168 40.745975 40.709648 40.746586 40.773666 40.750156
Table 6: Sample rows and columns extracted from the NYC taxis trip dataset with a selected subset of columns, including the latitude and longitude parameters of the pickup. Finally, for this third dataset, the geo-indistinguishability mechanism was run 10 times for each of three privacy budget values: 𝜖 ∈ {0.001, 0.01, 0.1}. Figure 9 shows the distribution of the mean perturbation radius obtained for each record across the 10 runs, for each of the three 𝜖 values considered. As expected, the perturbation radius decreases as 𝜖 increases, since larger values of 𝜖 correspond to weaker privacy guarantees and therefore less noise being added to the original coordinates. This result illustrates the practical importance of carefully calibrating the privacy budget for this type of data: on the one hand, an excessively small value of 𝜖 can result in perturbation ranges of several kilometers, which would seriously compromise the usefulness of the published data for tasks such as identifying areas with high demand for pickups at different times of the day; on the other hand, an 𝜖 that does not provide sufficient protection may leave pickup locations close enough to their original coordinates that, as discussed above, they can still be used to infer sensitive information (such as the home or work address of an individual) especially when cross-referenced with external data sources. Therefore, selecting an appropriate value for 𝜖 requires striking a balance between these two opposing objectives.
Figure 8: Example of a specific point (lat, lon) within the NYC taxis trip dataset, showing the original value and the value obtained by applying the geo-indistinguishability mechanism.
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 16 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
Figure 9: Distribution of the radius obtained for each record, mean after 10 runs with different seeds and with 𝜖 ∈ {0.001, 0.01, 0.1}.
DISCUSSION Conclusion: differential privacy in data science workflows When designing a data science workflow from zero, it is often assumed that the data lifecycle is in a controlled environment. However, when privacy constraints exist, it is necessary to incorporate additional mechanisms to prevent the extraction of sensitive information, whether directly from the data or indirectly through the trained models. When focusing on the privacy of the data, the first step typically involves removing or pseudonymizing identifiers, depending on whether they may need to be reconstructed in the future. Next, two main approaches can be considered: (1) first, anonymization through generalization, where QIs are transformed using generalization based on hierarchies to satisfy formal privacy definitions, such as 𝑘-anonymity, 𝓁-diversity, 𝑡-closeness, 𝛿-disclosure, etc. In this context, tools like anjana enable the anonymization process, while pyCANON facilitates the evaluation of the level of privacy achieved under different definitions. Both software products have been developed in an analogous way to trasgoDP, allowing for their easy integration within the same workflow. (2) Second, we may consider generating noisy data (following the idea of synthetic data generation but exploiting the initial raw data) using mechanisms based on local differential privacy or metric privacy, enabling microdata publication and thus analysis, while maintaining formal privacy guarantees. This is precisely the approach that trasgoDP aims to facilitate to incorporate in a complete data science pipeline. Both strategies have advantages and limitations. On the one hand, anonymization based on generalization can significantly reduce the usefulness of the data due to a loss of granularity; for example, an exact age provides more information than an age range. However, this approach accurately preserves the overall distribution of the original dataset within a generalization domain. On the other hand, LDP mechanisms maintain the original domain of the variables (for example, ages remain specific numerical values, and categorical variables retain their original categories). However, adding noise at the record level can significantly distort the data if it is excessive, or prove insufficient from a privacy point of view if it is limited. In this context, the main challenge involves obtaining an appropriate balance between privacy and utility. The same applies if the focus is exclusively on publishing microdata (both via anonymization or sanitization using LDP or metric privacy). Thus, from the perspective of training machine or deep learning (ML/DL) models on such data, it is important to note that certain anonymization techniques are designed to prevent inference attacks, which may conflict with modeling objectives. For example, in the case of 𝓁-diversity, the goal is for individuals with the same QIs to have different values for the sensitive attribute, which can make it difficult for the model to learn consistent predictive relationships. In addition, the approach achieved with LDP-based approaches to microdata releasing under formal privacy guarantees allows the data to be reused directly in analytical workflows in which models have been trained with data under the
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 17 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
same conditions, whereas if we deal with anonymized versions obtained using generalizations, such models could not be directly reused. Furthermore, focusing on privacy-preserving machine learning (PPML), in horizontal federated learning (FL) scenarios [15] where there are multiple clients with the same types of distributed data, the application of anonymization techniques requires ensuring consistency among the different participating nodes[31]. Differences in generalization levels (for example, using exact ages in one node and age ranges in another or lat-lon intervals instead of exact points) can lead to incompatible models and degrade the performance of the overall model. These types of limitations are partially mitigated by using LDP to generate homogeneous noised versions of the data instead of applying generalizations. In addition, DP could be added at the client level also to disturb local model updates [19]. It is worth noting that these approaches are not incompatible with the application of global differential privacy. For example, in statistical analysis, mechanisms such as Laplace or Gaussian can be applied to protect aggregated queries, while techniques such as k-ary Randomized Response allow for the private estimation of distributions. Similarly, during the training of deep learning models, differential privacy can be incorporated using algorithms such as DP-SGD, together with specific versions of different optimizers with DP (e.g. Adam) implemented in libraries like TensorFlow Privacy [1]. In the context of federated learning, differential privacy can also be applied during the aggregation phase of local models from the server side to prevent the extraction of sensitive information by potentially malicious participants. Overall, the implementation of trasgoDP enables its integration into a complete data science workflow, facilitating the generation of noised data with local DP and metric privacy guarantees, and even its combination with global differential privacy mechanisms during model analysis and training. In this way, such integration with these other solutions allows to protect not only the privacy of the input data but also that of the resulting model, both in inference scenarios and during production deployment. Whether to use this tool or anonymization ones depends on the specific use case, the application to be developed with that data (model development, publication, federated training, etc), the number of records, their distribution, the desired level of privacy, and the types of attacks that need to be prevented.
Limitations of the study Certain limitations of the current version of trasgoDP (v2.0.3) should be acknowledged. First, regarding local differential privacy for tabular attributes, the library currently implements a limited set of mechanisms (two for numerical attributes, three for categorical ones and one for location-based data), and does not yet support other data types available often in tabular dataset, such as free-text attributes. Second, the main limitation of applying LDP for microdata releasing remains its application to multiple columns of the same dataset: due to the sequential composition theorem, the overall privacy guarantee degrades according to the sum of the individual budgets. Since noise is added independently at the record level (local model), the reduction in utility loss observed as 𝜖 increases must be further explored. Thus, a more rigorous analysis of the privacy budget composition is left for future work. Third, in this same line, the correlation loss metric proposed in this work and implemented in trasgoDP, has only been validated empirically on two datasets, and its behavior on datasets with different sizes, dimensionalities, or feature correlation structures remains to be studied. Finally, as with any LDP-based approach, results are sensitive to the choice of the privacy budget 𝜖 (and 𝛿 for the Gaussian mechanism). No automated procedure is currently provided within trasgoDP to select these parameters based on a target utility level, and this limitation remains as future work.
METHODS Methodology and development conventions The development of the trasgoDP library has been conducted openly
on GitHub. Regarding the development methodology, the project adopts the Conventional Commits specification, in which each commit is categorized using standardized prefixes such as feat:, fix:, docs:, test:, and chore: among others. As discussed below, this convention enables the automatic generation of changelogs and facilitates semantic versioning by determining the appropriate version increment (major, minor, or patch). The project follows Semantic Versioning (SemVer), and the version referenced in this paper is v2.0.3. For this version, the package is classified under Development Status :: 5 - Production/Stable, targeting developers, educators, and scientific researchers, and it is distributed under the Apache 2.0 license.
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 18 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
Dependencies management and installation The software development of trasgoDP relies on Poetry [27] for
dependency management and packaging, with all configuration centralized in the pyproject.toml file. Dependencies are organized into separate groups according to their purpose: development environment (tox), unit testing (pytest[23] and pytest-cov), code linting and style enforcement (flake8[17]), code formatting (black[28]), static security analysis (bandit[25]), static type checking (mypy), undeclared dependency detection (pip-check-reqs), and package validation prior to publication (twine[26]). This modular structure ensures that only the dependencies strictly required for each task are installed, preventing unnecessary installation in production environments. Finally, the project uses tox to coordinate all testing and quality assurance environments. Testing covers five versions of Python (from 3.10 to 3.14), ensuring wide compatibility.
CI/CD and code coverage Continuous integration and delivery are managed through three dedicated GitHub
Actions workflows. The main pipeline (cicd.yml) runs all the tests on every push or pull request. A separate workflow (pypi.yml) handles automated publication of the package in PyPI upon the creation of a new release, while .codecov.yml reports code coverage metrics to Codecov [32]. Project documentation is automatically built using a dedicated pipeline and hosted via Read the Docs.
Unit testing As mentioned previously, unit testing has been conducted using the adult dataset. The unittest framework is employed to validate both edge cases and the correct behavior of the implemented functions concerning types of both inputs and outputs. The current test suite achieves over 97% code coverage in version v2.0.3. Release automation Although the software can be installed directly via pip + git and referring to the original repository, the release process has been fully automated through PyPI distribution. The package can therefore be installed using the command pip install trasgodp . This automation is enabled by the GitHub Actions workflow pypi.yml, together with the CI/CD pipeline that manages package validation and deployment via twine. Documentation The documentation of trasgoDP is published on Read the Docs and built automatically via the
aforementioned pipeline. It is generated using Sphinx[34], with furo as the documentation theme. The build process takes advantage of autodoc, extracting information from docstrings to describe function inputs, outputs, behavior, and expected types.
Requirements Finally, with respect to version v2.0.3, beyond the dependencies required for testing, documentation,
and code quality, the core runtime dependencies remain minimal, including the following: numpy (v2.0.2), pandas (v2.3.3), scipy (v1.15.3), typing_extensions[24] (v4.16.0), folium (v0.20.0). This lightweight dependency design enhances compatibility with a wide range of Python ecosystems and ensures support across Python versions 3.10 through 3.14.
RESOURCE AVAILABILITY Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Judith Sáinz-Pardo Díaz ([email protected]).
Materials availability References to the code generated from this study are given in the following section. Only synthetic versions of openly available datasets have been generated during this work.
Data and code availability The raw data used to conduct this work and to perform the unit tests are available in the examples folder of the library’s repository. The most relevant links concerning the availability of the source code, documentation and installation of the trasgoDP library are listed below:
• The code for the trasgoDP library is openly available in GitHub: https://github.com/IFCA-Advanced-Computing/trasgodp. • The documentation can be found in ReadTheDocs: https://trasgodp.readthedocs.org. J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 19 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy
• The installation can be managed using PyPI: https://pypi.org/project/trasgoDP/. • In order to ensure code reproducibility and availability, a Zenodo DOI has been created. This DOI stands for all the tagged versions, and always resolves to the latest one: https://zenodo.org/records/18754857. Any additional information required concerning the code reported in this paper is available from the lead contact upon request.
ACKNOWLEDGMENTS The authors would like to thank the funding through the EOSC SIESTA project “Secure Interactive Environments for Sensitive daTa Analytics”, funded by the European Union (Horizon Europe) under grant agreement number 101131957, and the support from the EOSC ARENA project “AI Research Enhancement through Networked Agents”, funded by the European Union (Horizon Europe) under grant agreement number 101292597.
AUTHOR CONTRIBUTIONS J.S-P.D. defined and conceived this work, performed the formal analysis, methodology, validation and software development. A.L.G. contributed to the definition and software development of this work and was responsible of supervision, project administration and funding acquisition. Both authors contributed to writing and reviewing the manuscript.
DECLARATION OF INTERESTS The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References [1] Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., Zhang, L., 2016. Deep learning with differential privacy, in: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308–318. [2] Andrés, M.E., Bordenabe, N.E., Chatzikokolakis, K., Palamidessi, C., 2013. Geo-indistinguishability: differential privacy for location-based systems, in: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, Association for Computing Machinery, New York, NY, USA. p. 901–914. URL: https://doi.org/10.1145/2508859.2516735, doi:10.1145/2508859.2516735. [3] Arcolezi, H.H., Couchot, J.F., Gambs, S., Palamidessi, C., Zolfaghari, M., 2022. Multi-freq-ldpy: multiple frequency estimation under local differential privacy in python, in: European Symposium on Research in Computer Security, Springer. pp. 770–775. [4] Becker, B., Kohavi, R., 1996. Adult. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5XW20. [5] Bertram, M.G., Sundin, J., Roche, D.G., Sánchez-Tójar, A., Thoré, E.S., Brodin, T., 2023. Open science. Current biology 33, R792–R797. [6] Biswas, S., Palamidessi, C., 2024. Privic: A privacy-preserving method for incremental collection of location data. Proceedings on Privacy Enhancing Technologies . [7] Chatzikokolakis, K., Andrés, M.E., Bordenabe, N.E., Palamidessi, C., 2013. Broadening the scope of differential privacy using metrics, in: international symposium on privacy enhancing technologies symposium, Springer. pp. 82–102. [8] Ciriani, V., De Capitani di Vimercati, S., Foresti, S., Samarati, P., 2007. Microdata protection, in: Secure data management in decentralized systems. Springer, pp. 291–321. [9] City of New York, 2026. TLC Trip Record Data. https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page. [Accessed 17-07-2026]. [10] Cormode, G., Maddock, S., Maple, C., 2021. Frequency estimation under local differential privacy [experiments, analysis and benchmarks]. arXiv preprint arXiv:2103.16640 . [11] Domingo-Ferrer, J., Sánchez, D., Soria-Comas, J., 2016. Database anonymization: privacy models, data utility, and microaggregation-based inter-model connections. Morgan & Claypool Publishers. [12] Domingo-Ferrer, J., Sánchez, D., Soria-Comas, J., 2022. Anonymization methods for microdata, in: Database Anonymization: Privacy Models, Data Utility, and Microaggregation-based Inter-model Connections. Springer, pp. 15–23. [13] Douriez, M., Doraiswamy, H., Freire, J., Silva, C.T., 2016. Anonymizing nyc taxi data: Does it matter?, in: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 140–148. doi:10.1109/DSAA.2016.21. [14] Dwork, C., Roth, A., 2014. The algorithmic foundations of differential privacy. Foundations and trends in theoretical computer science 9, 211–487. [15] El Ouadrhiri, A., Abdelhadi, A., 2022. Differential privacy for deep and federated learning: A survey. IEEE access 10, 22359–22380.
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 20 of 21
trasgoDP: An Open Source Framework for Releasing Noised Tabular Microdata under Local Differential Privacy [16] Feroze, Zahid, 2026. Dataset: Global Cancer Patients 2015-2024. https://www.kaggle.com/datasets/zahidmughal2343/ global-cancer-patients-2015-2024. [Accessed 09-02-2026]. [17] Flake8 developers, 2026. Github repository: flake8. https://github.com/pycqa/flake8. [Accessed 26-03-2026]. [18] Holohan, N., Braghin, S., Mac Aonghusa, P., Levacher, K., 2019. Diffprivlib: the IBM differential privacy library. ArXiv e-prints 1907.02444 [cs.CR]. [19] Hu, K., Gong, S., Zhang, Q., Seng, C., Xia, M., Jiang, S., 2024. An overview of implementing security and privacy in federated learning. Artificial intelligence review 57, 204. [20] Mironov, I., 2017. Rényi differential privacy, in: 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263–275. doi:10.1109/CSF.2017.11. [21] OpenMined, 2026. Github repository: pydp. https://github.com/OpenMined/PyDP. [Accessed 07-04-2026]. [22] Prasser, F., Eicher, J., Spengler, H., Bild, R., Kuhn, K.A., 2020. Flexible data anonymization using ARX - Current status and challenges ahead. Software: Practice and Experience 50, 1277–1304. doi:10.1002/spe.2812. [23] Pytest-dev, 2026. Github repository: pytest. https://github.com/pytest-dev/pytest. [Accessed 26-03-2026]. [24] Python, 2026. Github repository: typing_extensions. https://github.com/python/typing_extensions. [Accessed 26-03-2026]. [25] Python Code Quality Authority, 2026. Github repository: bandit. https://github.com/PyCQA/bandit. [Accessed 26-03-2026]. [26] Python Packaging Authority, 2026. Github repository: twine. https://github.com/pypa/twine. [Accessed 26-03-2026]. [27] Python-poetry, 2026. Github repository: poetry. https://github.com/python-poetry/poetry. [Accessed 26-03-2026]. [28] Python Software Foundation, 2026. Github repository: black. https://github.com/psf/black. [Accessed 26-03-2026]. [29] Ramachandran, R., Bugbee, K., Murphy, K., 2021. From open data to open science. Earth and Space Science 8, e2020EA001562. [30] Sainz-Pardo Diaz, J., Lopez Garcia, A., 2022. A python library to check the level of anonymity of a dataset. Scientific Data 9, 785. [31] Sáinz-Pardo Díaz, J., López García, Á., 2024. An open source python library for anonymizing sensitive data. Scientific data 11, 1289. [32] Sentry, 2026. Codecov. https://about.codecov.io/. [Accessed 26-03-2026]. [33] Shoemate, M., Vyrros, A., McCallum, C., Prasad, R., Durbin, P., Casacuberta Puig, S., Cowan, E., Xu, V., Ratliff, Z., Berrios, N., Whitworth, A., Eliot, M., Lebeda, C., Renard, O., McKay Bowen, C., . OpenDP Library. URL: https://github.com/opendp/opendp. [34] Sphinx developers, 2026. Sphinx documentation. https://www.sphinx-doc.org/. [Accessed 26-03-2026]. [35] Xiong, X., Liu, S., Li, D., Cai, Z., Niu, X., 2020. A comprehensive survey on local differential privacy. Security and Communication Networks 2020, 8829523. [36] Zhang, H., Mishra, A.K., Arcolezi, H.H., 2025. Demo: Exploring utility and attackability trade-offs in local differential privacy, in: Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, Association for Computing Machinery, New York, NY, USA. p. 4728–4730. URL: https://doi.org/10.1145/3719027.3760706, doi:10.1145/3719027.3760706.
J. Sáinz-Pardo Díaz & Á. López García: Preprint submitted to Elsevier
Page 21 of 21