A Formally Grounded ODRL Evaluator: Implementation and Comparison Jaime Osvaldo Salas , Paolo Pareti , Adeel Aslam, Christopher Maidens and George Konstantinidis
,
arXiv:2607.15987v1 [cs.AI] 17 Jul 2026
University of Southampton, University Road, Southampton, SO17 1BJ, UK Contact: [email protected], [email protected]
Abstract. The ODRL policy language is emerging as the de-facto standard for policy modelling data access and usage preferences, AI governance policies and data workflows in European dataspaces. The current standard has no mathematical formal semantics to describe how a system should implement policy evaluation. This has resulted in a variety of systems and tools that implement their own interpretation of the language, which limits interoperability and cannot guarantee consistent results. Based on an existing semantic model of ODRL, we formalise the problems of ODRL evaluation for the access control and monitoring scenarios, in both static and streaming settings, and we provide a novel, efficient algorithm and implementation. We present the first ODRL Evaluator with transparent formal semantics and supporting all rule types. We experimentally measure its performance, analysing different scalability dimensions related to policy complexity and size of the data on which a policy is evaluated. We compare our system with the state-of-the-art by providing a comparative review of existing ODRL evaluators, which highlights the differences in supported ODRL features and evaluation modes. Keywords: ODRL · computational policies · evaluation · monitoring · access control · streaming
1
Introduction
The Open Digital Rights Language (ODRL) is a W3C recommendation, currently in version 2.2, that provides a data model and a base vocabulary for the definition of computational policies. Based on the RDF data model, and an OWL vocabulary, ODRL is a natively Semantic Web technology. While many of its features can be seen as domain agnostic, such as the ability to define agreements, and the deontic rules of permissions, prohibitions and obligations, it is primarily designed to regulate the use of digital content and services. ODRL has recently gained attention as the de-facto policy language to regulate the use of data by AI systems and compliance-focussed initiatives like the European Dataspaces [22,7,13]. Since its inception, several studies have attempted to define a formal semantics for ODRL, namely a precise definition of how ODRL policies should
2
Salas et al.
be interpreted. Most notably, Pucella and Weissman formalised ODRL 1.1 to logical formulas [14], Steyskal and Polleres [20] defined semantics for ODRL 2.0 based on rule matching and logic programs, Bonatti et al. [1] proposed declarative semantics for ODRL 2.2, and Salas et al. [17] defined formal semantics for ODRL 2.2. based on query answering. A community report on ODRL semantics is currently being worked on, but it does not provide a formal semantics in the mathematical sense.1 In this paper, we reuse the semantics based on query answering defined by Salas et al., as it is the most recent and comprehensive semantics based on a mathematical model. Central to understanding ODRL semantics is the concept of an ODRL Evaluator, defined in the ODRL specification as “a system that determines whether the rules of an ODRL policy meet their intended action performance” [21]. Two scenarios that naturally occur in the context of ODRL evaluation are monitoring and access control, both of which have been studied in the literature [17,18] and are described in the community report on ODRL semantics. While monitoring typically refers to evaluating a policy on a state of the world, intuitively a log of past actions, to see if any violation occurred, access control evaluates policies primarily on access requests, which describe an intent to perform future actions. In this article we use the term ODRL Evaluator for a system that addresses one or both of the monitoring and access control scenarios, but note that the terms ODRL Monitor and ODRL Authoriser have been suggested to specify whether an ODRL processor targets the former or the latter [15]. A number of systems have been developed to evaluate ODRL. However, most only cover a limited subset of the ODRL language, lack formal semantics, and are based on a partial, and sometimes unspecified, interpretation of the evaluation problem. Motivated by this gap in the state-of-the-art, our first contribution is an extension of the ODRL semantics of Salas et al. [17] that models the decision problems for both policy evaluation scenarios in a single framework and shows how these two problems coincide. Moreover, to ensure the applicability of ODRL evaluation in a realistic setting, we extend the monitoring scenario to a streaming (online) mode of evaluation, where the state of the world changes frequently. Our second contribution is a novel algorithm for ODRL evaluation that covers all of the ODRL features in the chosen semantics. Notably, this algorithm performs a single-pass evaluation over the state of the world, allowing it to efficiently process incoming streams of events and access requests in constant time with respect to the size of the state of the world. As our third contribution we present OVAL, a new open source ODRL Evaluator that provides a full implementation of this algorithm. We provide an analysis of the performance of this system, showing its scalability with respect to the size of the inputs. Our fourth main contribution is to review existing ODRL evaluators and provide a structured comparison against OVAL based on features. For the most mature of such systems, we also assess performance relative to our system. 1
https://w3c.github.io/odrl/formal-semantics/
A Formally Grounded ODRL Evaluator: Implementation and Comparison
3
Our contributions provide a formal underpinning to ODRL evaluation, moving it from an underspecified and ad-hoc implementation task to a standardised and transparent method, proving clear connections between access control, offline monitoring, and online monitoring within a single semantic framework. Our implementation delivers a scalable system: OVAL exhibits linear growth with respect to both policy size and number of events, and produces results within milliseconds for existing datasets being up to two orders of magnitude more efficient than state-of-the-art.
2
Preliminaries
We now briefly introduce core concepts of ODRL semantics, and refer the reader to Salas et al. [17] for more details. As the main building blocks of a policy, ODRL rules are sets of constraints, which in turn are boolean combinations of conditions of the form (λ op ρ). The left operand λ is a property that is drawn from a known vocabulary, or Knowledge Graph, whose value is compared to the right operand ρ by the operator op. The operator op can be a binary operator between constants (=, >, ≥, <, ≤, ̸=), a binary operator between sets (≡, ∈, ∈, / ⊃, ⊂), or a special operator that checks class membership (=type ). These constraints are evaluated on sets of events, denoted states of the world, where each event e is a tuple of fixed length, whose indexes represent specific ODRL core components and/or left operands and thus allow the description of the event with respect to those features. For simplicity, we assume that the first value t of event e is always the timestamp of when the action has been performed, followed by the action value. With the exception of the indexes representing the timestamp and action, events can have null values. For example, assume a mapping between the first five indexes of an event and the features Party, Action, Asset and Page, respectively. Under this mapping, events eAlice : ⟨“2026-10-10T10:02:00”, Alice, Read, Book, 300⟩ and eBob : ⟨“2026-10-10T10:01:00”, Bob, Edit, Book, null⟩ represent, intuitively, the fact that Alice read page 300 of Book at datetime “2026-10-10T10:02:00”, and Bob edited it on “2026-10-10T10:01:00” (pages not specified), and a state of the world ω = {eAlice , eBob } denotes a record that both events have occurred. We say that a rule R matches an event e, denoted by match(R, e) if all its V constraints evaluate to true on e, that is when ( c∈R c)(e) is true. Thus, when evaluating a rule, all of its constraints are conjuncted with each other. ODRL policies are usually read as Rule for Party to Action Asset subject to Constraints and Refinements, e.g. Permission for Alice to Read a Book subject to the constraint the date is less than December 12th 2026 and the Book to be refined to its Subset of Pages 1-400. Such a rule could be formalised as the following set of constraints: pRead : {(P arty = Alice), (Action = Read), (Asset = Book), (Date < “2026-12-12T00:00:00”), (P ages ≥ 1), (P ages ≤ 400)}. An ODRL Lite policy is a tuple ⟨P, F, O⟩, where P, F and O are the sets of permission, prohibition and obligation rules, respectively. A state of the world ω is said to be valid with respect to a policy ⟨P, F, O⟩ if the following are all true:
4
Salas et al.
– for all events e ∈ ω, there exists a permission P ∈ P s.t. match(P, e) is true. – for all events e ∈ ω, there is no prohibition F ∈ F s.t. match(F, e) is true. – for all obligations O ∈ O, there exists an event e ∈ ω s.t. match(O, e) is true. ODRL 2.2 also allows for the definition of duties, remedies and consequences, which are obligations associated to permissions, prohibitions and obligations, respectively. A full ODRL policy is a 7-tuple Π = ⟨P, F, O, DP, DPC, FR, OC⟩ where DP, FR and OC are sets of pairs of permissions and duties, prohibitions and remedies, and obligations and consequences, respectively, and DPC is a set of triples of permissions, duties and consequences. In such a case, a state of the world ω is valid w.r.t. a full ODRL policy if it is valid w.r.t. ⟨P, F, O⟩ and: – for all permission-duty pairs ⟨D, P ⟩ ∈ DP and for all events e ∈ ω, if match(P, e) is true, then there exists an event e′ ∈ ω that occurred before e (t′ ≤ t) such that match(D, e′ ) is true, and – for all prohibition-remedy pairs ⟨F, R⟩ ∈ FR and for all events e ∈ ω, if match(F, e) is true, then there exists an event e′ ∈ ω that occurred after e (t′ ≥ t) such that match(R, e′ ) is true, and – for all obligation-consequence pairs ⟨O, C⟩ ∈ OC such that O contains a time constraint ct = (Datetime ≤ t), if there is no event e such that match(O, e) is true, then there must exist events e′ and e′′ that occurred after e (t ≤ t′ ∧ t ≤ t′′ ) such that match(C, e′ ) and match(O \ {ct }, e′′ ) are true, – for all permission-duty-consequence triples ⟨D, P, C⟩ ∈ DPC such that D contains a time constraint ct = (Datetime ≤ t), if there exists an event e such that match(P, e) is true and there is no event e′ such that match(D, e′ ) is true, then there must exist subsequent events e′′ and e′′′ such that match(C, e′′ ) and match(D \ {ct }, e′′′ ) are true. In our running example, state of the world {eAlice , eBob } would violate a policy that has permission pRead as the only rule, because the event eBob is not explicitly permitted, and thus it is implicitly prohibited. Note that the chosen semantics does not include reasoning, that is the inference of new facts, and constants are assumed to be disjoint from each other, following the unique name assumption.
3
Related Work
3.1
Formalisations
The earliest version of ODRL (1.0) was tailored for digital rights management, whereas currently it has been adopted for more general usage including the usage of content and services [8]. ODRL 2.0 added prohibitions and duties as rule types, resembling the concepts of deontic logic more closely. The current specification of ODRL (2.2) was formalised by Iannella and Villata [8]. Notably, it builds on ODRL 2.0 by adding additional temporal or conditional rules (e.g., consequences and remedies). Finally, constraints were structured to triples comprised of a left operand, an operator and a right operand.
A Formally Grounded ODRL Evaluator: Implementation and Comparison
5
We now review the most relevant pieces of work in the literature that address similar semantics problems. Garcı́a et al. [6] employed semantic web technologies such as OWL [11], to translate ODRL (in XML format) to OWL ontologies and RDF graphs, which allowed them to leverage the IPROnto ontology for digital rights management [4]. However, their approach was tailored for ODRL 1.1, which is deprecated, and lacks the richer tapestry of rule types such as prohibitions, and obligation. Pucella and Weissman [14] proposed a translation of ODRL 1.1 Policies to logical formulas. The normalisation steps discussed in this study could also be applied to their constraints, which would also allow logical normal forms. Steyskal and Polleres [20] earliest proposed semantics for ODRL 2.0+ based on rule-matching and logic programs; notably their semantics took into consideration the implicit dependencies between actions. Bonatti et al. [1] defined declarative semantics for ODRL 2.2 that covered the main components of the language, while also defining the notion of compliance with a policy, providing illustrative examples with complex policies that include duties, remedies and consequences. Salas et al. [17] defined formal semantics for ODRL 2.2 inspired by query evaluation over a state of the world that acts as a database. Moreover, they introduced the notion of policy comparison inspired by query containment. Slabbinck et al. [18] defined a vocabulary to describe interoperable ODRL Evaluation reports and also presented the SolidLab ODRL Evaluator, an ODRL Evaluator for access control scenarios. Kebede et al.[9] discuss the representational power of ODRL, reinforced by presenting use-cases and examples, and highlights some of the limitations of ODRL. For example, the ambiguous semantics of duties (as per the currently published recommendations) or the granularity of parties (e.g., certain members of an organisation). De Vos et al. [3] implement policy and compliance checking by translating policies into answer-set programs to check for compliance. The semantics of answer-set programming are ideal for applications that adopt a closed-world assumption. Their proposed model is ideal for representing real-world legal frameworks such as the GDPR. Kieffer et al. [10] propose an approach that computes the least restrictive license that complies to a set of policies. If such a license does not exist, one can assume that there are no events that are valid w.r.t. all policies simultaneously. Therefore, said approach could be used to verify policy overlap. However, the problems of containment and equivalence were not a focus of their work, and cannot be trivially derived.
3.2
ODRL Evaluation Systems
The SolidLab Evaluator [18] leverages the various standards defined by the W3C ODRL Community Group such as ODRL Evaluators and Compliance Reports as defined per the ODRL Information Model2 and supports reasoning tasks when evaluating policies using the EYE 3 reasoner. Its documentation specifies that the 2 3
ODRL Information Model 2.2: https://www.w3.org/TR/odrl-model/#terminology EYE reasoner: https://github.com/eyereasoner/eye
6
Salas et al.
only rule types that are currently supported are Permissions and Prohibitions, with a partial support of Duties through the State of the World object.4 The ODRE Framework [2] is a framework designed to provide enforcement capabilities to ODRL policies. They achieve this by adding more features to ODRL, creating layers with additional expressivity, which combine different technologies to enforce the features. For example, they add the ability to define data variables using the interpolated language Freemarker 5 . As a result, they produce enforceable policies through Python and Java. With regards to policy evaluation specifically, it supports only the odrl:dateTime left operand, and only numeric and symbolic ODRL operators [16]. Due to lack of documentation and problems in running the code, we were unable to test their system. However, we could access a demonstration that illustrated its functioning. ODRL-PAP6 is a mostly Java implementation that translates ODRL policies to equivalent rego7 expressions which are then fed into an Open Policy Agent (OPA). The result of the evaluation is given as input to the Policy Enforcement Point, which the user interacts with, and can actually enforce the policy. However, the implementation seemingly only takes into consideration a limited variety of constraints, tailored for a curated list of vocabularies (based on concepts from the DOME-Marketplace project8 ). For example, for ODRL, there are only translations for constraints with odrl:dayOfWeek, odrl:hourOfDay and odrl:currentTime. The ODRL Manager 9 is described as an ODRL Library for Node.js with functions to facilitate the parsing, validation and interpretation of ODRL 2.2 Policies. Upon a preliminary analysis, it appears to have the infrastructure to support ODRL Evaluation. However, we were unable to find a publication that formalises their semantics, and only accepts ODRL policies in JSON format with little space for deviations in notation. The MOSAICrOWN 10 is a Python implementation of a policy engine that aligns with Salas et al.’s semantics [17] insofar as ODRL policies that mention targets, assignees, actions and purposes. This makes sense since the implementation relies on a SPARQL query to traverse the graphs that represent the ODRL policies considering that the semantics are based on query evaluation. Access requests are modelled as SQL query requests. Gaia-X transforms contracts, policies and legislation into ODRL and performs reasoning using their Policy Reasoning Engine 11 Their engine requires 4
https://github.com/SolidLabResearch/ODRL-Evaluator/blob/main/ ODRL-Support.md, accessed 13/07/2026 5 https://freemarker.apache.org/docs/dgui misc alternativesyntax.html 6 ODRL-PAP: https://github.com/wistefan/odrl-pap 7 Open Policy Agent: https://www.openpolicyagent.org/docs/policy-language 8 DOME-Marketplace project: https://dome-marketplace.eu/ 9 ODRL Manager: https://github.com/Prometheus-X-association/odrl-manager 10 MOSAICrOWNpolicy engine: https://github.com/mosaicrown/policy-engine 11 GAIA-X Across [ https://gitlab.com/gaia-x/lab/policy-reasoning/odrl-vc-profile ] [ https://docs.gaia-x.eu/technical-committee/architecture-document/24.04/ Compliance and Sovereign Data Exchange work packages ]
A Formally Grounded ODRL Evaluator: Implementation and Comparison
7
a set of policies (expressed using Verifiable Credentials[19]), information about the consumer such as their location, and the consumer’s usage intentions expressed as an ODRL Request. Their policies employ alternative constraints to those of the ODRL standard, where the left operands are formulated as JSON Path queries. The policies are translated into RDF triples and loaded into a GraphDB instance, after which SPARQL queries are executed over the graph. If the queries return any results, then the system concludes that an agreement can be reached. This appears to align with the ODRL Lite semantics, but we were unable to find publications that describe their evaluation semantics formally to confirm this.
4
ODRL Evaluation Problems
In this section we formalise the monitoring and access control scenarios into a single semantic framework, and show under which conditions the chosen semantics allows translations from one to the other. Moreover, we define the monitoring scenario also under another dimension: streaming vs static states of the world. 4.1
Monitoring
The monitoring problem asks the question, given a compilation of events that have occurred (a state of the world) and an ODRL policy, whether the state of the world is valid with respect to the policy. As is normally the case for monitoring scenarios, the state of the world ω corresponds to a database containing a set of ordered events that have been logged by an external agent. While there is no strict data model that such log should adhere to, for simplicity we will assume that it is a table, with each row representing a distinct event, and each column representing a feature of these events. Given a policy, we assume there is a partial mapping between each column of the log and the features, such as the core components or the left operands, of the policy. We also assume that the first value in any event is the timestamp at which it occurred. If a column is not mapped to any feature of a policy, the information in that column is not used by the policy for evaluation. If a policy contains a feature that is not mapped to a column of the table, we assume that the value for that feature for any event contained in the policy is null.
Table 1: Example of a state of the world Datetime
Action
Actor
Asset
Resolution
Pages
2026-10-10T10:00:00 2026-10-10T10:01:00 2026-10-10T10:02:00 2026-10-11T09:00:00
Print Edit Read Print
Alice Bob Alice Bob
Book Book Book Picture
500 dpi null null 650dpi
300 null 300 null
8
Salas et al.
Table 1 shows an example of a state of the world. This state of the world contains four events about Alice and Bob performing actions on certain assets (a book and a picture). The property Resolution is a constraint associated with the Print action, while Pages is a constraint associated with the Book asset. We further distinguish two modes of monitoring evaluation. In log-based (offline) monitoring evaluation, a policy is evaluated against a single state of the world object. This corresponds directly to the definition of validity in Section 2. Definition 1. Determining whether a state of the world is valid with respect to a given ODRL policy is the offline monitoring evaluation decision problem. As an example, let Π be a policy with P = {pRead } as a permission and DP = {⟨dP rint , pRead ⟩} a permission-duty pair where dP rint is the rule {(P arty = Alice), (Action = P rint), (Asset = Book), (Resolution = 500dpi), (P ages > 200)}. This policy intuitively states that Alice is allowed to Read a Book if she has Printed the Book at a resolution of 500dpi and at least 200 pages. Under the monitoring scenario, the first event in Table 1 will be matched by dP rint , and the third event will be matched by pRead . Moreover, since the first event occurred before the third event, the permission-duty pair is satisfied. In stream-based (or online) monitoring, a policy is evaluated continuously against a stream of events ω, which is a time-ordered list of state of the worlds. Stream ω is time-ordered if, for every event e in batch ω i , the time event e is recorded to have happened is greater than the time recorded of every event e in any batch ω j such that j < i. Intuitively, each state of the world in ω could contain as little as a single event. Evaluating a policy over a stream of events should lead to the same result as evaluating the whole stream offline in a single run. Definition 2. A stream of events ω is valid w.r.t. an ODRL policy Π if and S only if the state of the world ω∈ω ω is valid w.r.t. Π. Definition 3. Determining whether a stream of events ω is valid w.r.t. an ODRL policy is the online monitoring evaluation decision problem. 4.2
Access Control
In an access control scenario, a requester sends an ODRL policy consisting of one or more permission rules that specify the intended actions by the requester over an asset. The evaluator must then take into consideration any internal ODRL policies and, potentially, a state of the world to decide if the request is granted. The access control scenario can be modelled as a monitoring scenario if we can extend the state of the world with a canonical event that simulates the requested action having taken place. Intuitively, an access request should be granted if the state of the world preserves its validity after adding to it an event that matches that request.
A Formally Grounded ODRL Evaluator: Implementation and Comparison
9
Definition 4. Given a timestamp t and a rule R where each constraint ci ∈ R is of the form (λi = ρi ), the canonical event at t etR is the event ⟨t, ρ1 , ρ2 , . . . , ρn ⟩, where n is the number of constraints in R. We can now define the access control decision problem in terms of the offline monitoring decision problem. Definition 5. Given a rule R, a policy Π,a timestamp t and a state of the world ω, an access request for R is accepted if the state of the world ω ∪ {etR } is valid w.r.t. Π. As an example, let the request R be the rule {(P arty = Alice), (Action = Read), (Asset = Book), (P ages = 250)} and let the policy Π be the running example. Alice sends the request R at “2026-10-12T15:00:00” to an endpoint with policy Π. Let us assume first a case where the state of the world ω is empty. In such a case, the evaluation will check if the state of the world consisting of the single event ⟨“2026-10-12T15:00:00”, Alice, Read, Book, 250⟩ is valid w.r.t. Π. Evidently, this is not true because the permission-duty pair ⟨pRead , dP rint ⟩ cannot be satisfied. Therefore, this request is not accepted. On the other hand, if we assume the state of the world is the one shown in Table 1 extended with etR , it is valid w.r.t. Π. Therefore, this request is accepted. In the access control scenario, the usage of consequence, remedy and obligation rules is less natural. If a required duty of a permission is not fulfilled, a request to exercise that permission will be denied, thus never triggering its consequence rule. The same can be said of forbidden actions, that cannot trigger remedies as they cannot be violated. When we consider a policy that does not contain consequences, remedies or obligations, we can notice that the access control scenario can directly model the streaming monitoring scenario. Theorem 1. Given a stream of events ω, such that every batch ω i ∈ ω contains a single event, and an ODRL policy Π that contains only permission, prohibition and duty rules, ω is valid w.r.t. Π if and onlySif, for every batch ω i ∈ ω, access request R is accepted over state of the world j<i ω j at time t, where t is the time stamp of the single event e ∈ ω i , and R is a permission that has e as the canonical event. Proof. ⇒ If stream ω is valid w.r.t S Π, but access request R of an event e in ω i is denied over state of the world j<i ω j , given the available rule types, it must be because 1) e is not permitted, 2) e is prohibited or 3) e is permitted, but one of the associated duties is not matched to any event that predates e. However any of those three conditions would contradict the validity of ω w.r.t Π. S ⇐ If every access request R is accepted, then by Definition 5 {e} ∪ j<i ω j is valid w.r.t. Π for each event e in ω. Then trivially, when considering the last event, the set of all events in ω is valid w.r.t. Π and thus the stream of events is valid w.r.t. Π too by Definition 2.
10
5
Salas et al.
The OVAL Evaluator
We now present the OVAL evaluator and discuss its improvements against the state of the art. In particular, we describe a novel algorithm to process all types of ODRL rules using a single pass over the events of a state of the world, thus enabling scalable processing of all types of evaluation, including streaming. The main features of OVAL are summarised in Table 2 along with a comparison with existing systems. OVAL is the first ODRL Evaluator to be grounded in formal semantics, as it implements nearly all of the semantics defined in [17] (consequences of obligations being the only feature not supported yet). Note that the chosen semantics does not support set and membership constraint operators, whose mode of evaluation is not fully specified in ODRL. This grounding in formal semantics is a fundamental property, which makes the evaluation nonambiguous. It enables precise testing, reuse and comparison with future systems, as the expected output of each policy evaluation is defined by the semantics.
OVAL ✓ SolidLab Evaluator ✓ ODRE Framework ✓ ODRL-PAP ✓ ODRL-Manager ✓ MOSAICrOWN ✓
Formal Semantics
Stream Processing
Ontologies
Time constraints
Count
Logical Constraints
Constraints
Monitoring
Reasoning
Remedies
Access Control
Consequences
Duties
Obligations
Prohibitions
System
Permissions
Table 2: Features comparison of different ODRL evaluators.
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
As a comprehensive implementation of ODRL, OVAL supports the basic rule types permissions, prohibitions and obligations, along with the more complex rules types of duties of permissions, consequences of permission duties, and prohibition remedies. It supports constraints and refinements. Notably, OVAL is the first evaluator to fully support all of the ODRL logic constraints, on all rule types, including nested constraints. Nested constraints allow for the expression of complex logic expressions by combining the and, or and xor logic operators. Time constraints are allowed through the odrl:dateTime left operand. Moreover, logs of events, or of exercised accesses requests are taken into account when evaluating the odrl:count constraint, which can be used to limit the number of times a permission can be exercised. OVAL is domain-agnostic, as it supports ontologies and custom vocabularies by allowing any term, identified by an IRI, to be used as an action, party, asset or left operand.
A Formally Grounded ODRL Evaluator: Implementation and Comparison
11
The function that OVAL uses to answer the evaluation decision problem takes as input, at a minimum, an ODRL policy and a state of the world. In the implementation, a state of the world is assumed to be a CSV file that contains column names odrl:Party, odrl:Action, odrl:Asset, that map to the three main components of an ODRL rule, along with an additional column with the IRI of λ as the header, for each left operand λ found in the constraints of the policy, additionally prefixed with odrl:Party/odrl:Action/odrl:Asset and a space if it is the refinement of the party/action/asset component, respectively. To support the streaming case of the monitoring scenario, as defined in Section 4, an OVAL evaluation outputs, along with the boolean result of the decision problem, an evaluation state object, which records key pieces of information which can be recorded to be used when processing future batches of events. To illustrate the use of this object, imagine a policy containing a prohibition to access a file, subject to the remedy to make a payment to the file provider, a first batch of events that contain a record of the file being accessed, followed by a second batch of events that contain a record of a payment to the file provider. The evaluation on the first batch detects a violation, as the forbidden action has been performed, but it also outputs a state of the evaluation object that records the need to perform the remedial action. Based on this information, when processing the second batch the evaluator will consider the stream compliant, and output an updated state of the evaluation where the remedial action is no longer needed. The evaluation state object includes the information contained in the original policy used for the evaluation, and extends it in the following two ways. Firstly, each rule is annotated with the number of events previously matched against this rule (matches count) and whether the policy requires an event to match this rule in the future as a condition for validity (required). Then, a global field is recorded: potential validity, which is set to 0 if the evaluation is invalid, and no future events can restore validity, for example if a prohibition has been violated, but no remedy for it exists. It is easy to see that the size of the evaluation state is constant with respect to the size of the policy. Therefore, for a policy of fixed size, and assuming all events are of fixed size, evaluating an additional event can be done in constant time, regardless of how many events have been evaluated previously, as all the inputs of the evaluation function remain constant in size. In a monitoring scenario, this property allows efficient processing of streams of events, and ensures that the complexity of evaluating of a log of events scales linearly with the size of the number of events, that is, with the size of the state of the world. The algorithm of the evaluation function of OVAL is described in Algorithm 2. This algorithm evaluates a state of the world object in a single pass from the oldest to the newest event. As each event is evaluated against all rules, the helper function in Algorithm 1 keeps track of how many times each rule matched an event through the matches count property, and disables the required flag of any matched rule which had such flag active. Intuitively, the matches count property records how past events matched rules and it is used, for example, to check if
12
Salas et al.
Algorithm 1 Rule matching Require: row x, rule R, evaluation state Σ (with R being a rule in Σ) 1: function CheckMatch(x, R, Σ) 2: if x satisfies all conditions of R 3: R.matches count ← R.matches count + 1 4: if R.required = 1 5: R.required ← 0 6: return True 7: return False
a duty of a permission had been fulfilled. The required flag instead defines the requirement of a rule to be matched by a subsequent event, for example to require a remedy to be eventually fulfilled. We provide an implementation of our evaluator along with a test suite of all of the core features of the evaluator, that contains many examples of policies, states of the world, and the expected results. This implementation also includes scripts to run the scalability tests presented next in Section 6.
6
Experimental Evaluation
To evaluate the performance of our evaluator we have developed two generators capable of creating synthetic ODRL policies and states of the world based on a number of parameters. A synthetic ODRL policy is created by instantiating a given number of permission, prohibition and obligation rules. Permission rules can be configured to have one or more duties. Specific actions, parties, assets and left operands are selected randomly from a pool of 10 constants each, chosen at random for each policy, with realistic values taken from the ODRL vocabulary. Rules are created by first defining the action it applies to, and then with a 50% chance each, specifying the asset and assignee components too. Between 0 and 2 refinements/constraints are then assigned to each component and to the rule itself. A synthetic constraint is generated by randomly selecting a left operand, a numeric operator (<, ≤, =, ̸=, >, ≥) and an integer between 0 and 100 for the right operand. A synthetic state of the world is generated from a policy by first extracting the set of features λ that appear in any condition (λ op ρ) in the rules of policy, and adding to it odrl:dateTime to ensure all events can be chronologically ordered. This set is then turned into a list to determine the columns of the state of the world table. A predefined number of rows is then created. Each row is generated by first selecting a random permission P , and then instantiating random values for each feature in the list, based on the conditions of P . If for a feature λ there is no (λ op ρ) in P , the field corresponding to that feature is left null with a 50% chance (if it is not odrl:Action), or it is otherwise assigned one of 100 values randomly. Otherwise, one such (λ op ρ) in P is selected, and the field is
A Formally Grounded ODRL Evaluator: Implementation and Comparison
13
Algorithm 2 ODRL Evaluation Algorithm Require: Policy Π, state of the world D, optional evaluation state Σ 1: function Evaluate(Π, D, Σ) 2: if Σ not provided //Initialise State of the Evaluation 3: initialise Σ with potential validity = 1; counts and required at 0 for rules 4: sort D by time ascending 5: validity ← Σ.potential validity 6: for each row x ∈ D 7: MP ← ∅, MF ← ∅ 8: for each permission P // Check all rules once on this row 9: run CheckMatch on all duties of P and their consequences 10: if CheckMatch(x, P, Σ) add P to MP 11: for each prohibition F 12: run CheckMatch on all remedies of F 13: if CheckMatch(x, F, Σ) add F to MF 14: for each obligation O 15: CheckMatch(x, O, Σ) 16: if MP = ∅ // Detect violations of Permissions 17: Σ.potential validity ← 0; validity ← 0; 18: for each P ∈ MP and its duties D // Enforce Duties and Consequences 19: if D.matches count = 0 20: if D has no consequences 21: Σ.potential validity ← 0; validity ← 0; 22: else 23: set D.required ← 1 and all consequences required 24: for each F ∈ MF 25: if F has no remedies //Detect violations of Prohibitions 26: Σ.potential validity ← 0; validity ← 0; 27: else//Enforce Remedies 28: set all remedies required 29: check obligations: if never matched ⇒ validity ← 0 30: check duties, consequences, remedies: if any still required ⇒ validity ← 0 31: return validity, Σ
populated with a value v such that v op ρ is true. For example, if op is > and ρ is 50, a value satisfying this condition is created by adding a random number between 1 and 100 to 50. This approach was chosen so that the generated rows will usually match existing permissions. While this does not guarantee validity, due to the complex ways the various randomly generated rules can interact with each other, it creates a more realistic test set by generating a significant amount of valid states of the world, alongside invalid ones. The results in Figure 1 show the scalability tests we performed on OVAL against different metrics, namely: the number of rules in the policy, the number of events in the state of the world, and the complexity of rules in terms of number of duties per permission, and constraints per rule. In Figure 1a), we examine the effects of the size and complexity of the policy on the runtimes. Here we can
14
Salas et al.
Runtime (seconds)
a) 0.2
b) 1
Permissions only Complex
30 rules 60 rules 90 rules
0.8
0.15
0.6 0.1 0.4 5 · 10−2 0
Runtime (seconds)
c) 1.5
0.2 0
50 Policy Size
0
100
100 events 300 events 500 events
0
200 400 600 800 1,000 Number of events
d) 0.4
100 events 300 events 500 events
1 0.2 0.5 0
0 5 10 Duties per Permission
0
0 50 100 Constraints per Rule
Fig. 1: Performance metrics of OVAL
see that the runtimes increase linearly with respect to the size of the policy, and this behaviour doesn’t change significantly even against policies with all rule types (complex policies). In Figure 1b), we examine the effect of the size of the state of the world (or number of events) on the runtimes. Once again, we can see a mostly linear behaviour, where the size of the policy alters the speed at which it increases. Then, in Figure 1c) we can also see that increasing the number of duties per permission increases the runtimes linearly. Finally, Figure 1d) shows a most notable result, where increasing the number of constraints per rule has almost no effect on the runtimes. This is most likely due to the early exit condition when checking multiple constraints, that terminates constraint evaluation after the first evaluation to false. In order to run a comparison between OVAL and the state-of-the-art SolidLab Evaluator, we used the test cases found in the SolidLab ODRL Test Suite 12 . The results in Figure 2 show a comparison of OVAL against the SolidLab Evaluator for the access control scenario against the test cases in the ODRL Test Suite. The runtimes are measured in milliseconds and shown in log scale because of the large difference. One can see that times are quite stable for both systems when the policies contain only permissions and constraints with datetimes, with an average of 30.32 ms for OVAL and 2595.87 ms for the SolidLab Evaluator. 12
ODRL Test Suite: https://github.com/SolidLabResearch/ODRL-Test-Suite
Runtime (ms)
A Formally Grounded ODRL Evaluator: Implementation and Comparison
15
104 103 102 OVAL
SolidLab Evaluator
Fig. 2: Runtimes of test cases in the SolidLab Test Suite in logscale.
However, there is a significant spike in three test cases, which are permissions with a large logical constraint (a disjunction of over 100 pairs of datetime constraints). For these cases, OVAL takes slightly over 150 ms, whereas the SolidLab Evaluator takes around 27 seconds. We note, however, that the SolidLab Evaluator also performs reasoning as part of its evaluation, which may explain part of the significant overhead. Regarding the actual result of the evaluation, the SolidLab Evaluator has three possible values: allowed, not allowed or “not enough information”, whereas OVAL can only respond true or false. For the purpose of comparison, we assumed both “not allowed” and “not enough information” are false in our system (prohibited-by-default). With this in consideration, our evaluators agreed on 54 out of 68 test cases. Most discrepancies were either because the SolidLab Evaluator performs reasoning while OVAL does not, or because OVAL supports duties while the SolidLab Evaluator does not. This suggests that the SolidLab Evaluator may be more well-suited to use cases that require reasoning, whereas OVAL for monitoring or other performance-sensitive scenarios. Regarding duties, the semantics adopted by OVAL interprets the ODRL specification that a “duty is fulfilled if all its constraints are satisfied and if its action, with all refinements satisfied, has been exercised” [8], as the existence of an event that matches the duty rule. This models in a natural way cases when a duty, such as the action of “anonymisation”, allows certain future actions, such as “sharing”, without setting a limit on how many times these actions can be performed. The SolidLab Evaluator, instead, interprets duties in a stricter sense, by requiring duty reports to map individual access requests with a specific fulfillments of a duty. This stricter interpretation models more naturally cases where each duty fulfilment enables an action to be performed just once, for example when each “payment” action allows a single “seat reservation” action.
7
Conclusion and Future Work
This paper presented OVAL, the first ODRL Evaluator grounded in a formal mathematical model of ODRL semantics. Moreover, we have formally defined the ODRL Evaluation problem for the access control, and offline and online monitoring, motivated by the different scenarios found in the literature where ODRL evaluation can be applied. We described the novel evaluation algorithm
16
Salas et al.
implemented by OVAL, that supports all of the different evaluation problems in a scalable way. We have validated the efficiency of our system experimentally demonstrating linear complexity with respect to increases in both policy complexity and size of the state of the world it is evaluated on. The experiments in this paper focused mostly on the scalability of the system against different metrics, while correctness is backed up by formal semantics. Currently, our evaluator is being used in major EU projects with real-world use cases and industrial partners. Future work on the evaluator will focus on extending it with reasoning capabilities, supporting set operators in constraints, and adapting it to leverage existing semantic technologies such as the ODRL Compliance Report Model [5]. Supplemental Material Statement: Source code, documentation and test scripts are available on Github 13 and are archived on Zenodo [12].
References 1. Bonatti, P.A., Fornara, N., Harth, A.: Towards a Formal Semantics of the Open Digital Rights Language (ODRL 2.2). In: Joint Proceedings of the ESWC 2025 Workshops and Tutorials. CEUR Workshop Proceedings, vol. 3977 (2025), https: //ceur-ws.org/Vol-3977/OPAL2025-4.pdf 2. Cimmino, A., Cano-Benito, J., Garcı́a-Castro, R.: Open Digital Rights Enforcement Framework (ODRE): from descriptive to enforceable policies (2024), https: //arxiv.org/abs/2409.17602 3. De Vos, M., Kirrane, S., Padget, J., Satoh, K.: ODRL Policy Modelling and Compliance Checking. In: Fodor, P., Montali, M., Calvanese, D., Roman, D. (eds.) Rules and Reasoning. pp. 36–51. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-31095-0_3 4. Delgado, J., Gallego, I., Llorente, S., Garcı́a, R.: IPROnto: An ontology for digital rights management. In: 16th Annual Conference on Legal Knowledge and Information Systems, JURIX. vol. 106 (2003) 5. Esteves, B., Slabbinck, W., Sellami, Y., Cimmino, A., Rodrı́guez-Doncel, V., Verborgh, R.: Capturing Requests and Context for ODRL-based Access and Usage Control. In: Joint Proceedings of the 16th Workshop on Ontology Design and Patterns and the 1st Workshop on Bridging Hybrid Intelligence and the Semantic Web (WOP-HAIBRIDGE 2025) colocated with the 24th International Semantic Web Conference (ISWC 2025) (2025) 6. Garcı́a, R., Gil, R., Gallego, I., Delgado, J.: Formalising ODRL semantics using web ontologies. In: Proc. 2nd Intl. ODRL Workshop. pp. 1–10 (2005) 7. Golpayegani, D., Esteves, B., Pandit, H.J., Lewis, D.: AIUP: an ODRL Profile for Expressing AI Use Policies to Support the EU AI Act. In: SEMANTiCS (Posters, Demos, Workshops & Tutorials) (2024) 8. Iannella, R., Villata, S.: ODRL Information Model 2.2. W3C Recommendation. World Wide Web Consortium. https://www.w3.org/TR/odrl-model (2018) 9. Kebede, M.G., Sileno, G., Van Engers, T.: A critical reflection on ODRL. In: International Workshop on AI Approaches to the Complexity of Legal Systems. pp. 48–61. Springer (2018) 13
https://github.com/DIPS-Tools/odrl-Engine/
A Formally Grounded ODRL Evaluator: Implementation and Comparison
17
10. Kieffer, M., Serrano-Alvarado, P., Bernelin, M.: Composing complex licenses to facilitate contextual resources reuse. In: 2025 Extended Semantic Web Conference Workshops & Tutorials (2025) 11. McGuinness, D.L., Van Harmelen, F., et al.: OWL web ontology language overview. W3C recommendation 10(10), 2004 (2004) 12. Pareti, P., Trejo, J.O.S., Aslam, A.: paolo7/odrl-engine: Oval evaluator (Jul 2026). https://doi.org/10.5281/zenodo.21415300, https://doi.org/10.5281/ zenodo.21415300 13. Petrova-Antonova, D., Pareti, P., Tomov, P., Yumusak, S., Maidens, C., Jiang, S., Konstantinidis, G., Roman, D.: Knowledge Graph-Driven Policy Enforcement in Urban Dataspace. In: International Joint Conference on Knowledge Graphs. pp. 406–421. Springer (2025) 14. Pucella, R., Weissman, V.: A formal foundation for ODRL. arXiv preprint cs/0601085 (2006) 15. Rodrı́guez-Doncel, V., Roman, N.: Towards conformance in ODRL 3.0. Joint Proceedings of the ESWC pp. 1613–0073 (2025) 16. Salas, J.O., Pareti, P.: ODRL Policy Comparison Through Normalisation (Mar 2026). https://doi.org/10.5281/zenodo.18862527 17. Salas, J.O., Pareti, P., Yumuşak, S., Gheisari, S., Ibáñez, L.D., Konstantinidis, G.: Evaluation and Comparison Semantics for ODRL. In: Knowledge Graphs. pp. 252–267. Springer Nature Singapore, Singapore (2026) 18. Slabbinck, W., Rojas Meléndez, J., Esteves, B., Colpaert, P., Verborgh, R.: Interoperable Interpretation and Evaluation of ODRL Policies. In: European Semantic Web Conference. pp. 192–209. Springer (2025) 19. Sporny, Manu, Longley, Dave, Chadwick, David, Herman, Ivan, Jones, Michael, Thibodeau, Ted, Cohen, Gabe: Verifiable Credentials Data Model v2.0. https: //www.w3.org/TR/vc-data-model-2.0/ (2025), W3C Recommendation, 15 May 2025 20. Steyskal, S., Polleres, A.: Towards formal semantics for ODRL policies. In: International Symposium on Rules and Rule Markup Languages for the Semantic Web. pp. 360–375. Springer (2015) 21. W3C: ODRL Information Model 2.2 (2018), https://www.w3.org/TR/ odrl-model/ 22. Yi, X.: Compliance-by-Design Micro-Licensing for AI-Generated Content in Social Commerce Using C2PA Content Credentials and W3C ODRL Policies. In: 2025 7th International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI). pp. 204–208. IEEE (2025)