I NTEGRABLE E LASTICITY VIA N EURAL D EMAND P OTENTIALS
arXiv:2605.22820v1 [cs.LG] 21 May 2026
Carlos Heredia∗ & Daniel Roncel† IAMM Research, Department of Applied Artificial Intelligence DAMM, Carrer del Rosselló 515, 08025 Barcelona, Catalonia, Spain
A BSTRACT We propose the Integrable Context-Dependent Demand Network (ICDN), a demand-first neural model for multiproduct retail demand. The model learns log-demand as a smooth, context-conditioned function of log-prices, allowing elasticities to be derived exactly from the learned demand surface. On the Dominick’s beer dataset, ICDN improves out-ofsample generalization over a directed log-log benchmark and yields more stable, economically plausible elasticity estimates, especially for weakly identified cross-price effects. Keywords: Deep Learning, Dynamic Pricing, Price Elasticity.
1
I NTRODUCTION
A central input to pricing decisions is a simple but difficult question: how does demand respond when prices change? In practice, this question is answered through elasticities—local log-sensitivities of demand with respect to price Asano (2012)—which are the natural primitives for scenario simulation, revenue optimization, and promotion planning. However, learning stable and reproducible elasticities at scale remains challenging. Retail demand is heterogeneous across stores and products, nonstationary over time, and shaped by promotions and assortment effects. At the same time, prices are often correlated across stock-keeping units (SKUs) and governed by business rules, making it easy for flexible models to fit demand while producing unstable or economically implausible elasticities. From a mathematical perspective, elasticities are derivatives of the demand map Anthony & Biggs (2024). As a result, small-scale irregularities in an estimated demand surface can be amplified into large fluctuations in elasticities. The challenge is more pronounced in the multiproduct setting, where the elasticity matrix couples all SKUs—or, in practice, at least the most relevant competitors. Additionally, cross-price effects are especially difficult to identify from observational price variation, as they are often weaker than ownprice responses and can be blurred by correlated pricing decisions, promotional activity, and other sources of variation. In applied work, elasticities are therefore typically computed from an estimated demand model, ranging from simple log-log OLS benchmarks Baddeley & Barrowclough (2009) to modern deep-learning predictors Acuña-Agost et al. (2021). From this perspective, it is natural to seek a single structured demand model that is differentiable and analytically tractable, so that elasticities can be obtained exactly as derivatives of the same underlying surface Berry et al. (1995); Banks et al. (1997). ∗ †
e-mail address: [email protected] e-mail address: [email protected]
1
Guided by this demand-first view, we introduce the Integrable Context-Dependent Demand Network (ICDN), a structured neural demand model that represents log-demand as a smooth function of log-prices and contextual covariates, so that own- and cross-price elasticities arise as exact derivatives of the same underlying surface. This demand-first construction is not unique to ICDN: any sufficiently smooth model of log-demand induces an integrable elasticity field when elasticities are defined as its Jacobian. The contribution of ICDN is to implement this principle in a form tailored to high-dimensional retail pricing, combining an explicit price basis, analytic spline derivatives, economic regularization, and sparse directed cross-price interactions. Structurally, ICDN decomposes price effects into interpretable linear components and flexible spline-based nonlinear refinements. Own-price responses are represented by a locally linear log-log term βii (x)ui together with a nonlinear spline component wii (x)⊤ Bi (ui ). Directed cross-price responses are modeled through a linear component βij (x)uj , a nonlinear univariate refinement wij (x)⊤ Bj (uj ), and a spline-spline interaction Bi (ui )⊤ U (ij) (x)Bj (uj ) that captures nonlinear relative price-positioning effects between products i and j. The coefficients attached to these price-response structures are not fixed globally: they vary with the observed retail context. A shared encoder maps the contextual information associated with each SKU into a product-specific latent representation hi . Own-price parameters are derived from hi , while cross-price parameters are constructed from ordered pairs (hi , hj ) and modulated by attention weights over selected product pairs. This ordered-pair construction does not impose symmetry: Eij and Eji are learned independently, allowing directional substitution or complementarity patterns across products. Because the spline bases admit closed-form derivatives, the resulting elasticities remain analytically computable without dense Jacobian or Hessian evaluations. In our empirical application, we evaluate ICDN on the Dominick’s Finer Foods scanner data for the beer category, a weekly retail scanner panel covering store-specific sales, prices, and promotional activity Kilts Center for Marketing (b;a). To make demand responses comparable across products, we express quantity in liters sold and prices in price per liter, and work in log-log coordinates. In this setting, own-price elasticities are expected to be predominantly negative, a pattern that is both standard for this class of products and confirmed in our cleaned Dominick’s sample Toro-González et al. (2014). Because the empirical application uses observational scanner data, the elasticities reported below should not be interpreted as fully identified causal effects of price interventions. They are local, context-conditioned derivatives of the fitted demand surface, evaluated under the current pricing regime. Accordingly, our empirical analysis focuses on predictive generalization, derivative coherence, and elasticity stability, rather than on causal identification. For causal approaches to pricing, see, for instance, Simchi-Levi & Wang (2023) and Schultz et al. (2024). Training balances predictive accuracy with economic regularity. We fit log-demand with a robust objective to reduce sensitivity to outliers and heteroskedastic noise, while regularizing the geometry of the learned demand surface to avoid artificial oscillations in the implied elasticities. In addition, we impose a soft penalty on positive own-price elasticities, reflecting the dominant empirical pattern of downward-sloping demand in this application, while still allowing local deviations when sufficiently supported by the data. Cross-price responses are also regularized through a soft elasticity-band penalty, which discourages unusually large cross-product effects unless they are supported by the demand-fit term. Our contributions are: • We formulate multiproduct elasticity estimation as a demand-first problem: log-demand is learned as a smooth surface and integrable elasticities are obtained as derivatives of that surface. • We present ICDN, a structured neural demand model designed to make this demand-first principle practical in high-dimensional retail settings. ICDN decomposes price responses into linear own2
price effects, directed cross-price interactions, and spline-based nonlinear refinements, allowing flexible but analytically tractable demand surfaces. • We exploit analytic derivatives of product-specific spline bases to compute own- and cross-price elasticities without relying on dense generic Jacobian or Hessian evaluations. This makes the derivative calculations transparent and scalable for multiproduct panels. • We introduce SKU-specific contextual conditioning together with sparse attention-modulated directed interactions. Each product is encoded through a shared contextual encoder, while crossprice effects are constructed from ordered product pairs, yielding heterogeneous and directional cross-price responses. • We evaluate the proposed architecture on retail demand data relative to a classical directed loglog benchmark, and analyze its implications for predictive performance, elasticity stability, reproducibility, and economic coherence. 1.1
R ELATED W ORK
Our work addresses various aspects of the demand-estimation literature. A first aspect consists of classical structural demand systems, which specify a system of demand equations and derive elasticities from the estimated parameters. Within this tradition, the Almost Ideal Demand System (AIDS) of Deaton & Muellbauer (1980) models budget shares across goods as functions of prices and real total expenditure. It provides a flexible first-order approximation to consumer demand and offers a theoretical framework in which the standard restrictions of demand theory—adding-up, homogeneity, and Slutsky symmetry—can be imposed and tested. It was introduced as a tractable alternative to earlier systems such as the Rotterdam Theil (1992) and translog models Christensen et al. (1975). Quadratic extensions such as QUAIDS further increase flexibility by allowing expenditure shares to be quadratic in log expenditure, while preserving a utility-consistent structure for analyzing price responses and welfare effects Banks et al. (1997). In parallel, the differentiatedproducts literature, exemplified by Berry et al. (1995), developed structural models of demand and supply in oligopolistic markets, where substitution patterns arise from heterogeneous consumer preferences regarding product attributes. Moreover, equilibrium prices are determined by firms’ marginal costs together with the markups implied by demand elasticities and strategic pricing. In this setting, elasticities are not estimated as free parameters, but are recovered from the estimated demand system. Building on this literature, we adopt the same demand-centered perspective. A related literature estimates price responsiveness using simple log-log demand specifications. We use these models as our main empirical benchmark, given their widespread use in applied pricing and retail analytics and their practical advantages: they are easy to estimate, computationally fast, and highly interpretable: the coefficient on the product’s own price is the own-price elasticity, while coefficients on other products’ prices can be read directly as cross-price elasticities. Their practical value is further strengthened by the ease with which additional regressors can be incorporated. Their main limitation, however, is that they impose globally constant, or otherwise highly restricted price effects, and therefore struggle to capture heterogeneous and state-dependent substitution patterns in multiproduct retail environments Baddeley & Barrowclough (2009). More recently, retail demand modeling has increasingly turned to machine-learning methods designed for high-dimensional settings. This shift reflects the idea that sales for individual SKUs are shaped by many nonlinear, lagged, cross-product, and store-specific effects that traditional models cannot adequately capture. In this spirit, Antipov & Pokryshevskaya (2020) propose a high-dimensional retail demand framework with nearly 500 predictors and show that Gradient Boosting Machines outperform both Random Forests and Elastic Net models, while Shapley-value decompositions provide interpretable feature attributions. This motivation is closely aligned with ours: the complexity of modern retail demand extends well beyond a small set of linear price and promotion variables. Our contribution, however, is different. Rather than treating elasticity analysis as a secondary or post-hoc exercise, ICDN preserves an explicit demand-as-a-function3
of-price representation, but allows the corresponding coefficients to depend on product characteristics and market context. Therefore, elasticities remain intrinsic, analytically recoverable objects of the model, while retaining the flexibility needed to accommodate high-dimensional retail environments. A more recent line of work uses deep learning models directly for elasticity estimation. Acuña-Agost et al. (2021) show that price elasticities can be recovered from deep-learning-based choice models through automatic differentiation of a trained predictor. This shares the same basic motivation as our approach, namely, treating elasticities as derivatives of a differentiable (deep-learning) model rather than as separate estimated objects. However, because elasticities are obtained through numerical differentiation, this strategy can become computationally costly at scale and offers only limited direct control over the implied demand structure and its derivatives. Our approach addresses this limitation by specifying a structured demand surface whose derivatives are analytically available as explicit functions of the neural-network parameters. Finally, our empirical setting connects to the literature on price elasticities in fast-moving consumer goods and, more specifically, beer demand. For aggregate beer demand, the empirical evidence consistently suggests that own-price elasticity is negative and inelastic. Nelson (2014) provides a meta-analysis of 191 beer price-elasticity estimates from 114 primary studies and reports an average elasticity of about −0.20 after correcting for heterogeneity, dependence, and publication bias. Similarly, Clements et al. (2022) summarize the existing evidence in their Table 1, reporting a mean of means of −0.47 and a mean of medians of −0.37 across relevant beer-demand studies, while their updated Australian estimates also indicate inelastic demand for the broad beer category. A related issue is the level of aggregation at which prices and quantities are measured. Using Universal Product Code (UPC) scanner data, Ruhm et al. (2012) show that more reliable grocery-store price measures imply a relatively low beer elasticity, probably around −0.3, although they aggregate UPC information into beer-category prices instead of estimating demand separately for individual SKUs. By contrast, Srivastava et al. (2015) estimate an AIDS model for 12 disaggregated alcoholic beverage types using monthly Nielsen data and obtain much larger own-price elasticities, with beer-type elasticities ranging roughly from −1.0 to −5.4. Closest to our empirical setting, Toro-González et al. (2014) use Dominick’s supermarket scanner data, which we also exploit, to estimate demand for differentiated beer products. Although their underlying data contain UPC-specific information, their reported elasticities are aggregated by beer type-mass-produced, craft, and imported beer. They find highly inelastic own-price elasticities of approximately −0.13, −0.21, and −0.22, respectively, and cross-price elasticities across beer types that are essentially zero. This evidence motivates our empirical prior that own-price effects should be negative, while cross-price effects are likely to be weaker, more heterogeneous, and should therefore be learned more cautiously from the data. Our contribution is to move beyond category- or beer-type aggregates by recovering own- and cross-price elasticities directly for individual SKUs/UPCs. Taken together, our work follows the demand-first logic of structural demand models, in which elasticities are derived from a single demand representation. We extend this logic to high-dimensional retail settings by specifying a flexible neural demand surface for individual SKUs/UPCs, where both own- and cross-price responses are allowed to vary with product characteristics and market context. Because the architecture is constructed so that these responses are analytically recoverable from the learned neural-network parameters, the resulting elasticities remain directly interpretable, regularizable, and suitable for large-scale multiproduct pricing applications. The remainder of the paper is organized as follows. Section 2 introduces notation and discusses elasticity computation from demand models. Section 3 presents our modeling approach, architecture, and training objective. Section 4 reports empirical results and diagnostic analyses. Section 5 concludes with directions for future research.
4
2
E LASTICITY F IELDS AND I NTEGRABILITY
In multiproduct pricing, demand is typically modeled as a function of prices and contextual information, while elasticities provide a compact summary of local price responses. Let q(p, x) ∈ Rn+ denote the demand vector at prices p ∈ Rn+ and contextual covariates x. The (point) price elasticity matrix is defined componentwise as ∂ log qi (p, x) Eij (p, x) = , ∂ log pj so that Eij measures how the demand of product i changes under an infinitesimal multiplicative perturbation of the price of product j. Elasticities are attractive in practice because they are scale-free, comparable across SKUs, and directly usable for reporting, and scenario evaluation. At the same time, E(p, x) is a differential object: it summarizes local derivatives, not the demand quantities q(p, x) themselves. This raises a basic consistency question that becomes central in high dimensions: under what conditions does an elasticity field E(·, x) correspond to a single, globally well-defined demand function? The term elasticity is used in several related but distinct ways. Depending on the context, one may distinguish between point elasticities, defined through derivatives, and arc elasticities, defined from finite changes; between own-price and cross-price elasticities; between Marshallian and Hicksian elasticities in structural consumer theory; and between elasticity and semi-elasticity interpretations in empirical models involving logarithmic transformations1 . In this paper, we focus on point log-price elasticities of observed demand conditional on contextual covariates, evaluated holding the context fixed. We do not interpret cross-price terms as Hicksian compensated effects, nor do we impose Slutsky symmetry. Instead, cross-price elasticities are interpreted as local directional responses in the fitted context-conditioned demand surface: Eij needs not coincide with the reverse effect Eji . This convention makes the elasticity matrix a Jacobian object. We now make this notation precise. Notation and elasticity matrix. Let p = (p1 , . . . , pn ) ∈ Rn++ be the vector of strictly positive effective prices for n SKUs, and let x ∈ X denote contextual covariates, such as store and SKU identity, time features, and promotion indicators. Demand, measured in volume, is modeled as a continuously differentiable map with respect to prices, v : Rn++ × X → Rn++ , (p, x) 7→ v(p, x), with components vi (·, x) ∈ C 1 (Rn++ ) for each fixed x. Throughout this section, derivatives are taken with respect to prices while holding the context x fixed. When x is fixed and clear from context, we omit it and write simply v(p). Definition 1. The point price elasticity matrix E(p, x) ∈ Rn×n is defined componentwise by Eij (p, x) :=
∂ log vi (p, x) pj ∂vi = (p, x), ∂ log pj vi (p, x) ∂pj
1 ≤ i, j ≤ n.
Equivalently, using elementwise logarithms, d log v(p, x) = E(p, x) d log p , where the differential is taken with respect to p, holding x fixed. In the one-dimensional case, n = 1, the elasticity matrix reduces to a scalar elasticity ε(p, x) = 1
d log v(p, x) , d log p
For further discussion of these distinctions, see: Poitras; Cameron (2022).
5
and, for each fixed context x, the elasticity relation becomes a separable ordinary differential equation in p. Given a baseline pair (p0 , v(p0 , x)) with p0 > 0 and v(p0 , x) > 0, demand at any p > 0 is given by Z p ds ε(s, x) v(p, x) = v(p0 , x) exp . s p0 Thus, in one dimension, specifying the local elasticity function ε(·, x) determines the entire demand curve up to the positive normalization v(p0 , x). In the multidimensional case, n > 1, the elasticity field E(·, x) specifies local derivatives of log-demand in several price directions. This introduces an additional consistency requirement that is absent in the onedimensional case. For each SKU i and fixed context x, define the 1-form on Rn++ ωi (p, x) :=
n X
Eij (p, x) d(log pj ).
j=1
If the elasticity field is induced by a differentiable demand function, then Definition 1 can be written componentwise as d log vi (p, x) = ωi (p, x), i = 1, . . . , n. The question is therefore whether the local responses encoded in ωi can be integrated into a single logdemand function log vi (·, x). In multiple dimensions, this is not automatic: the same final price vector may be reached through different paths of price changes, and a coherent demand representation requires the resulting change in log-demand to be path-independent. This is the integrability question addressed next. Importantly, this requirement is imposed row by row, for each demand component vi , and does not require the directional cross-effects Eij and Eji to be symmetric. Proposition 1. Fix a context value x ∈ X and a demand component i ∈ {1, . . . , n}. Let U ⊂ Rn++ be a simply connected open set and assume that Eij (·, x) : U → R is of class C 1 for every j = 1, . . . , n. Define the 1-form n X Eij (p, x) d(log pj ), p ∈ U. ωi (p, x) := j=1
If ωi (·, x) is closed on U , that is, if dωi (·, x) = 0
on U,
then there exists a function vi (·, x) : U → (0, ∞), unique up to a positive multiplicative constant, such that d log vi (p, x) = ωi (p, x), p ∈ U. Equivalently, the closure condition can be written as the commutation of cross-partials in log-price coordinates: ∂log pk Eij (p, x) = ∂log pj Eik (p, x), ∀ j, k ∈ {1, . . . , n}, ∀ p ∈ U. Proof. Introduce log-price coordinates uj = log pj and set V := log(U ) ⊂ Rn . Since the logarithm map is a diffeomorphism from Rn++ onto Rn , the set V is open and simply connected. Define the pullback of the elasticity 1-form to log-price coordinates by ω ei (u, x) :=
n X
Eij (eu , x) duj ,
j=1
6
u ∈ V.
The closure condition dωi (·, x) = 0 on U is equivalent, in log-price coordinates, to ∂uk Eij (eu , x) = ∂uj Eik (eu , x),
∀ j, k,
or equivalently de ωi (·, x) = 0 on V . By the Poincaré lemma Choquet-Bruhat & DeWitt-Morette (2004), every closed 1-form on a simply connected open set is exact. Hence there exists a function ψi (·, x) : V → R such that dψi (·, x) = ω ei (·, x). Now define vi (p, x) := exp ψi (log p, x) ,
p ∈ U.
Then vi (p, x) > 0. Moreover, n X ∂ψi d log vi (p, x) = d ψi (log p, x) = (log p, x) d(log pj ). ∂u j j=1
Since dψi = ω ei , we have and therefore
∂ψi (log p, x) = Eij (p, x), ∂uj n X Eij (p, x) d(log pj ) = ωi (p, x). d log vi (p, x) = j=1
This proves existence. Finally, suppose that vi and vei are two positive solutions. Then d(log vi ) = d(log vei ), so
vi d log = 0. vei Because U is connected, log(vi /e vi ) is constant on U . Hence vi /e vi is a positive constant, so the solution is unique up to a positive multiplicative normalization. Proposition 1 provides the precise link between an elasticity field and a globally defined demand map. The condition dωi = 0 is the standard closure condition for differential 1-forms. In differential-geometric terms, it is the condition under which the local object ωi can be integrated into a potential function; in the present setting, that potential is the log-demand function log vi (·, x). Thus, integrability is not an additional economic assumption, but a mathematical coherence requirement: the local price responses encoded by the i-th row of the elasticity matrix must be compatible with a single demand surface. To see the implication, fix a baseline p0 ∈ U and let γ : [0, 1] → U be any piecewise smooth path with γ(0) = p0 and γ(1) = p. If ωi is exact, then Z n Z X log vi (p, x) − log vi (p0 , x) = ωi (p̃, x) = Eij (p̃, x) d(log p̃j ), γ
and therefore
j=1
Z vi (p, x) = vi (p0 , x) exp γ
7
γ
ωi (p̃, x) .
Because exact forms have path-independent line integrals, the predicted demand vi (p, x) depends only on the final price vector p and on the normalization vi (p0 , x), not on the particular sequence of intermediate price changes used to reach p. If the closure condition fails, this coherence breaks down. Two different price-update paths ending at the same price vector can yield different values of Z ωi , γ
and therefore different reconstructed demands for the same final prices. One could restore determinism by imposing a canonical update path, for example by changing SKUs in a prescribed order, but this would make the forecast depend on an arbitrary convention rather than on a coherent demand function. This distinction is central for learning-based elasticity estimation. If elasticities are parameterized directly— for example, by a machine-learning model whose output is an elasticity-related quantity rather than a demand function, as in the GBM-based approach of Greenstein-Messica & Rokach (2020)—the resulting elasticity object will not, in general, satisfy dωi = 0. Consequently, integrating such elasticities can inherit path dependence, and the model may produce locally plausible elasticities that are not globally compatible with any demand map. By contrast, if one models log-demand directly and defines elasticities by differentiation, Eij (p, x) = ∂ln pj ln vi (p, x), then ωi = d ln vi holds by construction, and hence dωi = 0 follows automatically under the required smoothness conditions. This observation is general: it applies to any sufficiently smooth demand-first model, including neural networks, generalized additive models, spline models, or other differentiable demand representations. We therefore treat integrability not as a property uniquely delivered by ICDN, but as a design principle: the model should first define a smooth context-conditioned demand surface, and elasticities should then be obtained as its log-price derivatives. ICDN instantiates this principle with a structured spline-based price representation, analytic derivative formulas, and regularization terms designed to stabilize the resulting elasticity estimates in high-dimensional retail data. 2.1
E XAMPLE : CONSTANT ELASTICITY MATRIX
To illustrate the integrability condition, consider a simple multiproduct log-log specification at a fixed context value, which we omit from the notation. Suppose that the elasticity matrix is constant in log-price coordinates: Eij (p) ≡ cij (i ̸= j), Eii (p) ≡ εi , with cij , εi ∈ R. For SKU i, the associated 1-form is ωi (p) =
n X
Eij (p) d(log pj ) = εi d(log pi ) +
j=1
X
cij d(log pj ).
j̸=i
Since the coefficients are constant in log-price coordinates, this form is exact. Indeed, X ωi = d εi log pi + cij log pj , j̸=i
8
and hence dωi = 0. Therefore, the elasticity field is integrable and path-independent. Given a baseline price vector p0 ∈ Rn++ and a positive baseline demand vi (p0 ) > 0, a corresponding log-demand function is X log vi (p) = log vi (p0 ) + εi log pi − log pi0 + cij log pj − log pj0 . j̸=i
Equivalently, demand admits the closed-form representation cij εi Y pj pi . vi (p) = vi (p0 ) pi0 pj0 j̸=i
In retail pricing for fast-moving consumer goods, one typically expects εi < 0, so that lowering the own price increases demand. If prices are written relative to the baseline as pj = pj0 (1 − dj ),
dj ∈ [0, 1),
where dj denotes a price change, then vi (p) = vi (p0 ) (1 − di )εi
Y (1 − dj )cij . j̸=i
Under this convention, cij > 0 means that an increase in the price of product j raises the demand of product i, which is consistent with substitution. Equivalently, a discount on product j reduces the demand of product i, which is consistent with cannibalization. By contrast, cij < 0 means that an increase in the price of product j lowers the demand of product i, which is consistent with complementarity. This benchmark is analytically transparent and path-independent by construction. Given a baseline sales vector, for instance observed sales in a reference period, it provides a closed-form way to project demand under any proposed price configuration, including both own-price changes and changes in other products’ prices. It is therefore useful for scenario analysis, pricing simulations, and as an interpretable benchmark for multiproduct demand response. This type of log-linear scanner-data specification is also closely related to classical retail models for price and promotion response, such as SCAN*PRO and its extensions Wittink et al. (1988); van Heerde et al. (2002). Those models highlight the practical appeal of parsimonious, interpretable demand equations in retail environments. However, the constant-elasticity formulation remains restrictive: it imposes globally constant own- and cross-price effects and therefore cannot capture the state-dependent, context-dependent, and nonlinear substitution patterns that are often present in modern retail demand.
3
E LASTICITIES WITH N EURAL N ETWORKS
Section 2 shows that, in a multiproduct setting, elasticities should not be treated as arbitrary local objects: to define coherent demand responses, they must be compatible with a single log-demand surface. We now use this observation as the modeling principle for a neural demand model. The purpose of the neural network is not merely to forecast demand, but to learn a flexible and sufficiently smooth context-conditioned logdemand surface whose derivatives yield stable and economically interpretable elasticities. This distinction is important because small irregularities in the fitted demand surface can be amplified into large, noisy, or economically implausible price responses. Let p ∈ Rn++ denote the vector of effective prices and let x ∈ X denote contextual covariates, including store and SKU identity, seasonality, promotion indicators, and other market-state variables. We work in log-price coordinates u = log p ∈ Rn , 9
and define log-demand as
y(u, x) := log v(exp(u), x) ∈ Rn . In this formulation, the context x affects the entire price-demand surface, not only the baseline demand level. Consequently, price sensitivities may vary with the market state: the same price vector can imply different elasticities across stores, products, seasons, promotional regimes, or other contextual configurations. The elasticity matrix is therefore a local function of both prices and context. It is given by the Jacobian of log-demand with respect to log-prices: ∂y(u, x) ∈ Rn×n , Eij (u, x) = ∂uj log vi (exp(u), x). (1) ∂u This is equivalent to Definition 1, since uj = log pj and therefore ∂uj = ∂log pj . Thus, Eij (u, x) measures the local response of product i’s demand to the log-price of product j, evaluated at the specific price vector and contextual state (u, x). E(u, x) :=
We parameterize log-demand with a neural model y(u, x) = gθ (u, x) ≈ log v(exp(u), x),
(2)
where gθ represents the fitted context-conditioned log-demand surface. Demand predictions are therefore obtained by evaluating this surface directly at the target price vector and context. Elasticities are not modeled as separate outputs; instead, they are defined as the log-price derivatives of the same fitted surface: E(u, x) = Ju gθ (u, x),
Eij (u, x) = ∂uj gθ,i (u, x).
(3)
This construction also preserves the integrability property of Proposition 1. For each fixed context x, if every component gθ,i (·, x) is twice continuously differentiable in log-prices, then the induced row-wise elasticity 1-forms satisfy n X ∂uj gθ,i (u, x) duj = dgθ,i (u, x). ωi (u, x) = j=1
Hence they are exact, and the closure conditions hold automatically. We formalize this claim in Section 3.1. Building on this demand-first formulation, we introduce the Integrable Context-Dependent Demand Network (ICDN), a neural demand model that implements this demand-first principle through a structured differentiable map gθ (u, x). ICDN is designed for multiproduct retail panels, where price effects may be nonlinear, context-dependent, and sparse across products. Rather than using a generic neural predictor for log-demand, ICDN parameterizes the fitted demand surface in a form that keeps price effects explicit and elasticities analytically tractable. The integrability property itself is not specific to ICDN: any sufficiently smooth demand-first model would induce a row-wise integrable elasticity field if elasticities were defined as derivatives of predicted logdemand. The contribution of ICDN is instead to instantiate this principle through a structured log-price representation, combining linear price effects, spline-based nonlinear refinements, and directed cross-price interactions whose coefficients vary with the observed retail context. We detail this construction in the following subsection. This separation between an explicit price representation and context-dependent coefficients is also what makes derivative calculations tractable. Rather than relying on a generic multilayer perceptron and recovering Ju gθ or higher-order derivatives through generic automatic-differentiation calls, ICDN combines context-dependent coefficients with a smooth price basis whose derivatives are available in closed form. Elasticities can therefore be computed analytically from the fitted log-demand surface. This distinction is important for scalability: when the number of products is large, materializing dense Jacobians or Hessians can be memory-intensive and may become impractical on GPU. By exploiting analytic derivatives of 10
the structured price basis, ICDN avoids constructing full dense derivative operators and instead computes only the quantities required for training, regularization, and elasticity analysis. As a result, the architecture remains flexible enough to capture nonlinear and context-dependent price responses, while preserving derivative coherence and computational tractability in multiproduct demand estimation. 3.1
M ETHOD : T HE I CDN D EMAND S TRUCTURE
We now specify the ICDN demand structure that defines the fitted log-demand surface gθ (u, x). Throughout this subsection, we work in log-price coordinates u = log p ∈ Rn and treat the context x ∈ X as fixed when taking derivatives with respect to prices. ICDN models log-demand through a differentiable contextconditioned surface ybθ (u, x) = gθ (u, x) ∈ Rn , vbθ (p, x) = exp gθ (log p, x) , where the exponential is applied componentwise. Thus, predicted demand is positive by construction. The model-implied elasticity matrix is the Jacobian of this fitted log-demand surface with respect to log-prices: bθ (u, x) = Ju gθ (u, x), E
bθ,ij (u, x) = ∂u gθ,i (u, x). E j
Equivalently, for each SKU i, the associated model-implied elasticity 1-form in log-price coordinates is ω bθ,i (u, x) =
n X
∂uj gθ,i (u, x) duj .
j=1
ICDN parameterizes gθ by combining an explicit price representation with context-dependent coefficients. The nonlinear dependence on prices is represented through product-specific spline bases applied to the corresponding log-price coordinates. More precisely, all products share the same spline family and the same number K of basis functions, but the knot locations are adapted to the empirical price support of each SKU. Let us describe this spline-based construction in more detail. For each SKU i, let ui denote its log-price and let Qi be the empirical quantile function of the training observations of ui , after removing non-finite values. Given K spline basis functions, we choose uniformly quantile points2 τ1 < · · · < τK ,
τk ∈ [0.05, 0.95],
and define the product-specific knots in the original log-price scale as κ̃ik = Qi (τk ),
k = 1, . . . , K.
(4)
We also compute a product-specific centering and scaling, µi = Etrain [ui ],
σi = max{sdtrain (ui ), 0.2},
(5)
where the lower bound on σi avoids numerical instability for products with very limited price variation. Together, the quantile-based knot placement and the product-specific normalization ensure that the spline basis is both empirically localized and numerically comparable across SKUs. The spline basis is then evaluated in normalized log-price coordinates: zi =
ui − µ i , σi
κik =
κ̃ik − µi . σi
Putting these elements together, the spline basis is defined as ⊤ Bi (ui ) = Bi1 (ui ), . . . , BiK (ui ) ∈ RK , 2
We use the interior quantile range [0.05, 0.95] rather than the full [0, 1] range to avoid placing knots at extreme price observations.
11
with 3 3 ui − κ̃ik Bik (ui ) = ReLU(zi − κik ) = ReLU . σi Importantly, the knots, centering terms, and scales are fixed before training. Therefore, the neural network does not learn the knot locations; it learns the context-dependent coefficients multiplying these fixed productspecific basis functions. The advantage of this construction becomes clear when compared with a low-order global polynomial such as u2i . A quadratic term can introduce curvature, but only through a single global pattern over the entire log-price range. Truncated-power splines instead localize this curvature: each term activates after a productspecific knot, allowing the fitted log-demand surface to adjust its slope and curvature only in the price regions where the SKU has empirical support. Figure 1 illustrates this distinction.
(a) Global polynomial curvature.
(b) Local spline flexibility.
Figure 1: Comparison between a global quadratic price effect and a spline-based nonlinear price representation. A quadratic term such as u2i imposes a single curvature pattern over the full log-price domain. In contrast, truncated-power spline terms of the form Bik (ui ) = [ReLU((ui − κ̃ik )/σi )]3 activate after product-specific knots, allowing the slope and curvature of the fitted log-demand surface to vary across price regions while preserving differentiability. The cubic order is chosen to balance local flexibility with the regularity required by the integrability argument. In general, m ReLU(zi − κik ) ∈ C m−1 , so, within the truncated-power family, m = 3 is the lowest-order specification that yields a C 2 basis. This regularity is important because, as implied by the integrability condition in Proposition 1, closure of the elasticity 1-form requires derivatives of the elasticity field to be well defined. Since the basis is evaluated in normalized coordinates, its derivatives with respect to the original log-price ui are available in closed form. In particular, 3−r 3! 1 ui − κ̃ik (r) Bik (ui ) = ReLU , r ∈ {0, 1, 2}. (6) (3 − r)! σir σi Thus, nonlinear own-price effects and nonlinear cross-price interactions can be modeled flexibly, while their elasticity and curvature contributions remain analytically tractable. 12
In parallel, for each demand equation i, the neural context encoder maps the contextual state x into a baseline demand coefficient bi (x) ∈ R. In addition, for each ordered demand-price pair (i, j), with j = 1, . . . , n, the model produces price-response coefficients βij (x) ∈ R, wij (x) ∈ RK . The coefficient βij (x) captures the locally linear effect of the log-price of product j on the log-demand of product i, while wij (x) controls a product-j-specific nonlinear spline refinement of that effect. The case j = i corresponds to the own-price response of product i, whereas j ̸= i corresponds to directed cross-price responses. In the implementation, the own-price linear coefficient is sign-constrained. The network outputs an unconraw strained scalar βii (x), which is transformed as raw βii (x) = − softplus(βii (x)) .
(7)
Therefore, the linear own-price component is always non-positive by construction, consistent with the standard consumer-demand prior that own-price responses should be non-positive. This constraint is applied raw only to the own-price linear head βii (x). Cross-price coefficients βij (x), i ̸= j, are left unrestricted. For each ordered cross-product pair i ̸= j, the model also produces a nonlinear interaction matrix U (ij) (x) ∈ RK×K . This matrix governs the spline-spline interaction between the own log-price of product i and the log-price of product j in the demand equation of product i. Economically, it allows the cross-price relationship to vary across pricing regimes, so substitution or cannibalization effects need not be constant over the observed price supports of the two products. For example, a discount on product j may strongly cannibalize product i when i is priced near its regular range, but have a weaker incremental effect when i is already heavily discounted. Furthermore, since U (ij) (x) is defined for ordered pairs, these nonlinear cross-price effects are directional: the effect of product j’s price on product i’s demand is learned separately from the reverse effect3 . Taken together, the i-th component of the fitted log-demand surface is defined as n X X gθ,i (u, x) = bi (x) + βij (x)uj + wij (x)⊤ Bj (uj ) + Bi (ui )⊤ U (ij) (x)Bj (uj ). j=1
(8)
j̸=i
The first summation contains the direct price-response terms. For j = i, these terms represent the own-price component, including both a linear log-log effect and a nonlinear spline refinement. For j ̸= i, they represent directed cross-price effects, again decomposed into a linear component and a nonlinear univariate refinement in the price of product j. The second summation contains nonlinear pairwise interactions between the ownprice position of product i and the price position of product j. Since all coefficients are functions of x, both own-price and cross-price responses may vary across stores, products, seasons, promotional regimes, and other market settings. Elasticities follow by differentiating Eq. 8 with respect to log-prices. For i ̸= j, the cross-price elasticity of product i with respect to the price of product j is bθ,ij (u, x) = ∂u gθ,i (u, x) = βij (x) + wij (x)⊤ Bj′ (uj ) + Bi (ui )⊤ U (ij) (x)Bj′ (uj ). E j 3
This interaction need not be spline-based in principle. A simpler bilinear term such as ui uj could also capture price interactions, but it would impose a global and more restrictive functional form, as discussed in Figure 1.
13
The own-price elasticity is bθ,ii (u, x) = ∂u gθ,i (u, x) = βii (x) + wii (x)⊤ B ′ (ui ) + E i i
X
Bi′ (ui )⊤ U (ij) (x)Bj (uj ).
j̸=i
bθ (u, x) is available in closed form from the product-specific spline bases Bj , their Thus, each element of E derivatives Bj′ , and the context-dependent coefficients. bθ,ii (u, x) is not determined solely by the conIt is important to emphasize that the own-price elasticity E strained linear own-price coefficient βii (x) defined in Eq. 7. Although this coefficient is non-positive by construction, the total model-implied own-price elasticity also contains nonlinear spline and interaction derivative terms. Therefore, the sign constraint on βii (x) does not impose global monotonicity of demand with respect to own price. Local positive own-price elasticities may still arise if the nonlinear spline or interaction components dominate the constrained linear term. Beyond computational tractability, the closed-form elasticity representation also opens the door to further diagnostic and optimization analyses. Once the context x is fixed, the coefficients of the demand potential are fixed, and the implied elasticities become explicit functions of the log-price vector u. This makes it possible, in principle, to study how price sensitivity varies across the relevant price domain and to identify regions where own-price elasticities become substantially more negative or where the fitted demand surface changes curvature rapidly. These regions can indicate price ranges where demand becomes especially sensitive, so that further price increases may lead to disproportionately large volume losses. A full treatment of this question, however, requires specifying an explicit pricing objective, such as revenue, margin, profit, or constraints across the product portfolio, and is therefore beyond the scope of the present paper. We leave the use of ICDN for critical price-region analysis and price optimization to future research. Since the elasticity matrix is induced by differentiating the fitted demand potential gθ , the closure conditions from Proposition 1 hold by construction. The following proposition formalizes this property. Proposition 2. Fix a context value x ∈ X and let V ⊂ Rn be an open set in log-price space. Suppose that each component gθ,i (·, x) is of class C 2 on V . Define bθ,ij (u, x) := ∂u gθ,i (u, x), E j
ω bθ,i (u, x) :=
n X
bθ,ij (u, x) duj . E
j=1
Then ω bθ,i (·, x) is exact, and hence closed, on V . In particular, the row-wise closure conditions hold automatically: bθ,ik (u, x), bθ,ij (u, x) = ∂u E ∀ j, k, ∀ u ∈ V. ∂uk E j Proof. By construction, ω bθ,i (u, x) =
n X
∂uj gθ,i (u, x) duj = d gθ,i (u, x) .
j=1
Therefore ω bθ,i (·, x) is exact and hence closed. Since gθ,i (·, x) ∈ C 2 (V ), Clairaut’s theorem gives equality of mixed partial derivatives. Hence, bθ,ij (u, x) = ∂u E bθ,ik (u, x), ∂uk E j which proves the row-wise closure condition. Proposition 2 should be interpreted as a coherence result for demand-first modeling, rather than as a property exclusive to the ICDN architecture. The exactness of the elasticity 1-forms follows from defining elasticities 14
as derivatives of a smooth log-demand function. The distinctive feature of ICDN is therefore not the mathematical fact that a differentiated demand surface yields an integrable elasticity field, but the way in which the surface is parameterized: price effects remain explicit, nonlinearities are represented through spline bases with analytic derivatives, cross-price effects are modeled as sparse directed interactions, and, as detailed in the next subsection, regularization is applied directly to economically relevant quantities. 3.2
A RCHITECTURE OVERVIEW
Figure 2 summarizes the forward pass of ICDN. The architecture is built around three components. First, a context branch constructs one contextual token per SKU and maps these tokens into product-specific latent representations. Second, an attention-based sparse neighbor selector identifies a small set of relevant directed product interactions for each focal SKU. Third, a price basis branch evaluates product-specific spline basis functions and analytic derivatives at the observed log-prices. These components are combined in an integrable demand module that defines the differentiable log-demand map gθ (u, x), from which both demand predictions and elasticities are obtained. A key design choice is that ICDN does not compress the market state into a single global latent vector. Instead, the model constructs one token per SKU. For product i, the token combines global market information, such as store identity, calendar features, seasonality, and store-week promotional intensity, with information specific to each SKU, such as product metadata, lagged and rolling demand features, lifecycle variables, package-size information, missingness indicators, and local competitive descriptors. The current log-price vector is kept in the separate price branch, so the contextual tokens describe the market and product state excluding the price input used for differentiation. We denote this product token by τi (x) ∈ Rdtok ,
i = 1, . . . , n.
Stacking all SKU tokens yields τ (x) = τ1 (x), . . . , τn (x) ∈ Rn×dtok . A shared product encoder is then applied independently to each token: hi = ϕθ (τi (x)),
i = 1, . . . , n,
producing one latent representation for each SKU h(x) = h1 (x), . . . , hn (x) ∈ Rn×dh . This tokenized representation is primarily motivated by scalability. If the whole market state were compressed into a single global latent vector h(x), the model would not have access to product-specific representations hi and hj from which to construct directed pairwise price responses. In that case, cross-price effects would have to be generated as a dense n × n interaction structure directly from the global representation, with output dimension scaling as O(n2 ). By representing each SKU with its own latent vector, ICDN separates product representation from pair construction: own-price response terms are generated from hi , while cross-price response terms are generated from ordered pairs (hi , hj ). This factorization makes it possible to evaluate cross-price effects only on the top k selected neighbors of each focal product. Consequently, the cross-price component scales as O(nk) rather than O(n2 ), with k ≪ n, while still allowing heterogeneous and directional interactions across products. Own-price parameters are obtained from the latent representation of the corresponding SKU hi 7→ {bi (h), βii (h), wii (h)} , and cross-price response parameters are generated from ordered pairs of SKU representations: hpair := (hi , hj ) 7→ {βij (hpair ), wij (hpair ), U (ij) (hpair )}, 15
i ̸= j.
The ordered-pair construction is intentional: the effect of product j’s price on the demand of product i is learned separately from the reverse effect. Therefore, the architecture allows directional cross-price rebθ,ij ̸= E bθ,ji . sponses, implying E To make the cross-price component scalable, ICDN does not retain all off-diagonal product pairs. Instead, it uses a sparse neighbor selector based on scaled dot-product attention Vaswani et al. (2017). For each observation b = (s, t) in the current mini-batch, where s denotes the store and t the week, the latent SKU representations are projected into query and key vectors, (b)
(b)
(b)
(b)
qi = WQ hi , kj = WK hj . The relevance of candidate SKU j for focal SKU i is then scored as (b)⊤ (b)
qi kj √ + ξij , i ̸= j, datt where ξij is a fixed metadata-based bonus computed from SKU attributes such as brand, style, and packagesize similarity. For instance, package-size similarity is measured in log-space, so SKUs with similar liters per unit receive a larger bonus.4 (b)
sij =
The neighbor selector operates in two stages. The first stage determines the sparse support of directed cross-price interactions, while the second stage assigns observation-specific attention weights on the selected edges. During training, ICDN uses an online sparse-neighbor selection path. For each focal SKU i, the model (b) computes a relevance score sij for every candidate SKU j in each store-week observation b of the minibatch. The scores are then aggregated across the store-week observations in the mini-batch, 1 X (b) s̄B sij , ij = |B| b∈B
where B denotes the current mini-batch of store-week observations and |B| is the number of observations in that mini-batch. Self-pairs are masked out, and the top-ranked candidates according to s̄B ij are retained as temporary neighbors of i for the current forward pass. Since a focal SKU can interact with at most n − 1 other SKUs, the effective number of retained neighbors is keff = min{k, n − 1}, where n denotes the number of SKUs in the product set and k is the prescribed neighbor per focal SKU. The resulting online sparse edge set is denoted by P B ⊂ {(i, j) : i ̸= j}, |{j : (i, j) ∈ P B }| = keff . This online selection step is used only as a stochastic neighbor-exploration mechanism during training, while the contextual representations and attention parameters are being learned. It is not part of the final demand surface used for reporting. After training has converged, the sparse graph used for validation, testing, prediction reporting, and elasticity extraction is frozen. For each temporal split, all reported predictions and model-implied elasticities are computed using this frozen training-split graph, so they do not depend on the composition of the evaluation mini-batch. Specifically, after loading the final trained checkpoint, ICDN computes the average learned relevance score for each directed product pair over the training split, X (b) 1 s̄train = sij . ij |Dtrain | b∈Dtrain
Using query-key projections is more flexible than relying directly on the raw similarity h⊤ i hj . The latter imposes a more rigid and nearly symmetric notion of similarity. 4
16
The final frozen neighbor graph is then defined as P ⋆ = (i, j) : j ∈ TopKℓ̸=i s̄train , iℓ where TopKℓ̸=i denotes the set of the keff highest-scoring candidates among all ℓ ̸= i. Finally, it is stored with the final checkpoint and kept fixed for validation, testing, and reported elasticity estimates. The neighbor selector can optionally incorporate category prioritization. This option encodes the economic prior that the strongest substitution effects are often expected among SKUs within the same product category. However, it is not imposed by default, because the model should also be able to select economically relevant cross-category relationships, including weaker substitution, complementarity, or merchandising effects. When category prioritization is enabled, the first-stage ranking is modified so that same-category candidates are considered before cross-category candidates. Operationally, this is implemented by adding a large auxiliary ranking constant to the aggregated score used for graph selection. For a generic aggregation score s̄ij , train corresponding either to s̄B when freezing the graph, we define ij during online training or to s̄ij s̃ij = s̄ij + M 1{cat(i) = cat(j)},
M = 106 .
The value M = 106 is a numerical ranking device used to enforce priority of same-category candidates in the top-keff selection. It should not be interpreted as an economic magnitude or as an attention score. After self-pairs have been masked out, the top-keff candidates are selected according to s̃ij . If fewer than keff same-category candidates are available for a focal SKU i, the remaining slots are filled with the highestranked non-self candidates according to the original aggregated scores s̄ij . In the second stage, ICDN assigns observation-specific attention weights over the selected neighbors. During training, the weights are computed over the temporary online graph P B . During validation, testing, and elasticity extraction, they are computed over the frozen graph P ⋆ . Let P denote the selected graph used in the current forward pass, either P B during training or P ⋆ during evaluation. For each observation b, the (b) original logits sij , including the metadata bonus but excluding the category-priority ranking constant, are restricted to the selected neighbors and normalized over the candidate set of each focal product: (b)
(b)
aij = P
exp(sij )
(b) ℓ:(i,ℓ)∈P exp(siℓ )
,
(i, j) ∈ P.
Thus, the sparse graph controls which product pairs are evaluated, while the attention weights determine the observation-specific strength of each selected interaction. We would like to remark that the categorypriority constant M is used—if it is enabled—only for first-stage neighbor selection and is not included in the second-stage softmax. Consequently, category membership can affect which neighbors are selected, but it does not directly increase their final attention weights. In parallel, the price basis branch receives the log-price vector u = (u1 , . . . , un ) and evaluates, for each SKU, the product-specific cubic truncated-power spline basis, together with the analytic derivatives used to compute elasticities and the regularization terms introduced below: {Bi (ui ), Bi′ (ui ), Bi′′ (ui )}ni=1 . The knots, centering constants, and scaling factors are fixed before training, so these basis functions and derivatives are available in closed form, as defined in Eq. 6. The first derivatives Bi′ (ui ) enter the elasticity formulas, while the second derivatives Bi′′ (ui ) enter the own-price curvature penalty introduced in the next subsection. 17
The integrable demand module combines the own-price response terms, the selected attention-weighted cross-price response terms, and the product-specific spline basis values. Suppressing the mini-batch index for readability, the predicted output log-demand for product i defined in Eq. 8 becomes gθ,i (u, h; P) = bi (h) + βii (h)ui + wii (h)⊤ Bi (ui ) h i X + aij (hpair ) βij (hpair )uj + wij (hpair )⊤ Bj (uj ) + Bi (ui )⊤ U (ij) (hpair )Bj (uj ) .
(9)
j:(i,j)∈P
Output elasticities are obtained by differentiating this same fitted log-demand surface with respect to logprices. The latten-vector state h, the selected sparse graph P, and the attention weights aij (hpair ) are held fixed when taking price derivatives. The own-price elasticity is X bθ,ii (u, h; P) = ∂gθ,i (u, h; P) = βii (h)+wii (h)⊤ B ′ (ui )+ E aij (hpair )Bi′ (ui )⊤ U (ij) (hpair )Bj (uj ). i ∂ui j:(i,j)∈P
For each selected directed pair (i, j) ∈ P, with i ̸= j, the cross-price elasticity of product i with respect to the price of product j is h i bθ,ij (u, h; P) = ∂gθ,i (u, h; P) = aij (hpair ) βij (hpair ) + wij (hpair )⊤ B ′ (uj ) + Bi (ui )⊤ U (ij) (hpair )B ′ (uj ) . E j j ∂uj For non-selected directed pairs (i, j) ∈ / P, the corresponding cross-price effect is not evaluated, or equivalently is set to zero under the sparse approximation. 3.3
T RAINING OBJECTIVE AND REGULARIZATION
We train ICDN with a composite objective that balances predictive accuracy, smoothness of the fitted demand surface, and business plausibility of the implied elasticity matrix. Let O = (u, h, y ⋆ , m) denote a training observation, where u = log p ∈ Rn is the observed log-price vector, h ∈ H collects the contextual latten vector, y ⋆ = log v ∈ Rn is the observed log-demand vector, and m = (m1 , . . . , mn ), with mi ∈ {0, 1}, is an observation mask indicating which product demands are observed in the corresponding store-week5 . The training objective is h i L(θ) = EO Lfit (u, h, y ⋆ , m) + λsm Lsm (u, h, m) + λelast Lelast (u, h, m) .
(10)
In practice, the expectation EO is approximated by the sample average over mini-batches. The first term controls predictive fit on observed log-demands, the second term regularizes the geometry of the fitted demand surface, and the third term imposes a soft business-plausibility discipline on the implied own- and cross-price elasticities. For the fit term we use a robust loss on log-demand. Let n X Nm = mi i=1
denote the number of observed SKU demand entries in the current store-week observation. We define n 1 X Lfit (u, h, y ⋆ , m) = mi Huberδ (gθ,i (u, h) − yi⋆ ) . Nm i=1 5
This mask accounts for a common feature of retail panels: not every SKU is observed in every store-week.
18
Figure 2: ICDN forward pass. Context tokens are encoded into one latent representation per SKU, which generate own-price response terms, sparse attention-weighted cross-price response terms, and the parameters of the structured demand potential. Spline bases and analytic derivatives of log-prices are combined with these parameters to produce log-demand predictions and elasticities by exact differentiation. 19
The Huber loss is used to make the fit term robust to the residual heterogeneity that remains after cleaning. Store-SKU series differ in temporal support, price movements are often intertwined with promotions, and atypical price observations may still occur. A purely quadratic loss can therefore overreact to isolated or locally hard-to-explain weeks, whereas the Huber loss preserves sensitivity to small residuals while downweighting large deviations. Furthermore, normalizing by Nm makes the fit term an average over available products rather than a sum. This prevents store-weeks with more observed SKUs from receiving a mechanically larger weight in the objective. Because elasticities are derivatives of gθ , the objective cannot be understood as a pure forecasting loss. A model that closely fits observed log-demand but does so through a highly irregular surface may still produce unstable, noisy, or economically implausible elasticities. ICDN therefore seeks a balance between predictive fit and derivative stability: the fitted demand surface should explain observed demand while remaining smooth enough for its local derivatives to be interpreted as stable model-implied price responses. In this sense, the regularization terms are not auxiliary implementation details, but part of the elasticity-estimation objective. To control the geometry of the learned demand surface, we regularize own-price curvature, which is available analytically under the spline parameterization. Define κi (u, h) :=
∂ 2 gθ,i (u, h) ∂u2i
as the own-price curvature of the i-th log-demand component. When taking price derivatives, the contextualstate-latten vector h, the selected sparse graph Pθ (h), and the attention weights aij (hpair ) are held fixed with respect to prices. Under the ICDN parameterization in Eq. 9, the baseline and linear price terms do not contribute to this second derivative. Moreover, among the univariate spline terms, only the own-price spline component depends on ui . Hence, X κi (u, h) = wii (h)⊤ Bi′′ (ui ) + aij (hpair )Bi′′ (ui )⊤ U (ij) (hpair )Bj (uj ). j:(i,j)∈Pθ (x)
We define the smoothness penalty as Lsm (u, h, m) =
n 1 X mi κi (u, h)2 . Nm i=1
This term discourages excessive own-price curvature in the fitted log-demand surface, stabilizes the induced own-price elasticities, and acts as a complexity regularizer by favoring smoother price-response functions unless the data support sharper nonlinearities. Penalizing the full Hessian, including all cross-curvature terms, is possible in principle but substantially more expensive and less stable in high-dimensional multiproduct settings. We next impose a soft business-plausibility discipline directly on the model-implied elasticity matrix. Rather than imposing separate regularizers on individual cross- or own-price parameters (βij , wij and Uij ), we regularize the implied local elasticity. For each component Eij , let ℓij < rij denote a business-plausible range. In our empirical application, own-price elasticities are assigned the range [ℓown , rown ] = [−5, 0]. The upper bound reflects the empirical prior that own-price responses should be locally non-positive, while the lower bound prevents extremely negative local elasticities that may lead to unstable pricing simulations. The lower limit is inspired by the range of disaggregated alcoholic-beverage demand elasticities reported in Srivastava et al. (2015), where own-price responses can be substantially more elastic when estimated for disaggregated products rather than broad categories. 20
Cross-price elasticities are assigned the range [ℓcross , rcross ] = [−1, 1]. This interval allows both substitution and complementarity, since cross-price effects may have either sign, but discourages unusually large cross-product responses. The range is motivated by the empirical evidence in Toro-González et al. (2014), who find that cross-price effects across beer types in Dominick’s scanner data are weak and close to zero. Thus, r , i = j, ℓ , i = j, rij = own ℓij = own rcross , i ̸= j. ℓcross , i ̸= j, The elasticity penalty is evaluated only on economically observed and model-relevant entries. Define IE (u, h, m) = {(i, i) : mi = 1} ∪ {(i, j) : (i, j) ∈ Pθ (h), i ̸= j, mi = 1, mj = 1} , and let NE = |IE (u, h, m)|. The elasticity-band penalty is Lelast (u, h, m) =
1 NE
2 2 X bθ,ij (u, h) − rij + ReLU ℓij − E bθ,ij (u, h) . ReLU E
(11)
(i,j)∈IE
This term is a soft constraint. It does not force elasticities to remain inside the specified ranges, and it does not impose a sign restriction on cross-price effects. Instead, it discourages economically implausible local price responses unless the demand-fit term provides sufficient evidence for them. Taken together, the three components of the objective reflect the central design principle of ICDN: combining the flexibility of deep learning with business-informed economic structure. The neural demand surface provides the capacity to learn heterogeneous and nonlinear retail demand patterns, while the smoothness and elasticity-band penalties encode operational pricing knowledge about plausible local price responses. Thus, business theory is not used only as a post-hoc interpretation of the learned elasticities, but is embedded directly into the construction and training of the model.
4
N UMERICAL E XPERIMENTS
We evaluate ICDN on the Dominick’s Finer Foods (DFF) scanner dataset for the beer category, which contains weekly store-by-product observations on unit movement, prices, and promotional activity6 . The empirical goal is to assess whether the integrable neural demand representation introduced in Section 3 improves predictive generalization and yields more stable model-implied elasticity estimates than a classical store-SKU log-log benchmark. Since the data are observational, this evaluation should be understood as a comparison of fitted demand surfaces and their local derivatives, not as a causal identification exercise. To remain consistent with the notation used throughout the paper, we work in log-price coordinates and model log-demand. After preprocessing, let p denote the effective liter-normalized price vector and let v denote demand measured in liters. The model input is therefore u = log p,
y = log v.
Moreover, since products in the DFF dataset are indexed by UPC codes, we refer to the SKU unit as a UPC throughout the empirical analysis. 6
For details on the Dominick’s dataset and variable definitions, see Kilts Center for Marketing (a;b).
21
4.1
P REPROCESSING AND TARGETS
The raw DFF records contain multi-item deals and heterogeneous package sizes. Therefore, before training, both prices and quantities must be put on a common unit scale. Let total price denote the total price of a deal and let units per deal denote the number of UPC units included in that deal. We define the deal-corrected effective price per UPC as pupc =
total price . units per deal
(12)
Because package sizes vary substantially across products, for example 6/12OZ, 12/12OZ, and 750ML, quantities are normalized to liters. We transform pack size text into liters per UPC, denoted by ℓupc , allowing for both single containers and multipacks. The unit conversions are 1 oz ≈ 0.0295735 L,
1 gal ≈ 3.78541 L.
Liters sold and liter-normalized prices are then defined as qL = units sold · ℓupc ,
pL =
pupc . ℓupc
(13)
The target and main price input are expressed in log-log coordinates: y := log qL ,
u := log pL .
(14)
Observations with non-positive prices or quantities are discarded, since logarithms and elasticities are not well defined in those cases. Consequently, the fitted demand surface should be interpreted as a model of positive observed log-demand, conditional on an UPC being observed with positive sales, rather than as a full purchase/no-purchase, zero-sales, stockout, or assortment-availability process. We also exclude observations flagged by exclude flag, which identifies records that are potentially inconsistent or unreliable for modeling. This flag is used purely as a data-quality screen, not as an economic treatment indicator. The final dataset schema is organized by feature family in Tables 12-17. Specifically, Table 12 reports the identifier and product-descriptor variables, Table 13 summarizes the target, price, and promotion variables, Table 14 contains the temporal and lifecycle controls, Table 15 reports the demand-memory and missingness variables, Table 16 contains the dynamic competitive-neighborhood features, and Table 17 summarizes static assortment and neighbor-entry variables. The exploratory data analysis—see Appendix A.1—plays two roles in the empirical design. First, it determines the sample restrictions used for estimation. Second, it motivates the groups of covariates used as inputs to ICDN. The raw store-UPC panel is highly uneven and temporally irregular, with many series that are either too short or too fragmented to support reliable within-series elasticity estimation. In addition, although the panel contains substantial price movement overall, a non-negligible fraction of store-UPC series exhibits insufficient within-series price variation to identify own-price effects. Finally, price changes are often strongly linked to promotional status, creating promo-price confounding and weakening the interpretation of observational price responses. For these reasons, the modeling sample is constructed using identification-oriented filters. We retain only store-UPC series with at least 52 observed weeks, within-span coverage of at least 0.75, at least 3 distinct price values, at least 5 price changes, and a within-series log-price range of at least 0.15. To reduce promodriven confounding, we additionally remove series with absolute price-promotion correlation above 0.80 or with more than 80% of price changes coinciding with promotion-state switches. We then identify and remove log-price outliers within each UPC, since atypical price realizations are especially problematic for elasticity learning. Overall, the preprocessing pipeline reduces the sample from 22
1,966,147 basic-quality-filtered rows to 463,722 cleaned observed store-UPC-week observations. The resulting sample retains a negative (≈ −2) association between log price and log demand, while simple log-log OLS regressions still display only modest explanatory power. This fact is important for our application: it supports the presence of a meaningful own-price signal, but also indicates that price alone explains only part of the variation in demand. The same EDA also motivates the final feature design. Even after filtering, the panel exhibits temporal incompleteness, strong persistence in both demand and price, visible seasonal structure, lifecycle effects at both the UPC and store-UPC levels, and substantial variation in the local competitive environment. To construct temporal features coherently, we first complete each store-UPC history on a weekly calendar grid. Lagged, rolling, seasonal, and gap-aware covariates are computed on this completed panel and then projected back onto the observed economically valid rows used for estimation. All lagged and rolling variables are constructed using past information only, with backward-looking windows that exclude the current and future weeks and are recomputed within each training split to avoid validation- or test-period leakage. As a result, ICDN is trained exclusively on real store-UPC-week observations, while still benefiting from temporal features that account for internal gaps and globally missing weeks. The final covariate set therefore includes store, UPC, brand-family, style-segment, and category identifiers; current promotion status and store-week promotional intensity; explicit temporal controls; lifecycle variables; product-size information; autoregressive summaries of past demand; missingness indicators for lagged and rolling features; dynamic competitive-neighborhood variables; and static assortment measures. The variable log qL is retained as the supervised learning target, whereas log pL enters as the main economic price input from which elasticities are derived.
4.2
T RAINING PROTOCOL
ICDN is trained with a staged protocol designed to stabilize optimization before exposing the model to the full nonlinear demand specification. The implementation uses two operative phases. The first phase provides an economically sensible log-linear warm start on smoothed demand targets. The second phase starts from this checkpoint and trains the full nonlinear specification on the original targets. This design avoids fitting the most flexible version of the model from a random initialization. For each training split, spline knots—Eq. 4—and normalization constants—Eq. 5—are estimated using only the corresponding training fold. The product-specific spline bases are constructed from the empirical support of each UPC’s log-price, using quantile knots over the training observations. The model is then trained and validated on store-week observations in wide multi-product format. Since each store-week contains demand observations for only a subset of UPCs, the fit loss is evaluated only on observed entries through the observation mask. The wide log-price vector is completed separately from the demand observation mask. Starting from the store-week by UPC price matrix, missing prices are first imputed within each store by forward-filling and backward-filling along the weekly calendar. Any remaining missing entries, corresponding to UPCs with no usable price history in a given store, are filled with the overall column mean for that UPC. As a result, ICDN always receives a complete log-price vector u ∈ Rn , while the mask m only determines which UPC demand outcomes are observed and included in the fit loss. Consequently, a UPC that is not observed in a given store-week may still contribute an imputed price-state value to the price vector and, if selected in the sparse graph, to cross-price terms. These completed prices should therefore be interpreted as price-state inputs required to evaluate the multiproduct demand surface, not as direct evidence that the corresponding UPC was economically active or available in that store-week. Reported cross-price elasticity summaries are restricted to pairs for which both the demand-receiving UPC and the price-perturbing UPC are observed in the corresponding store-week. 23
Across both phases, training follows the same general loop. For each mini-batch, ICDN receives contextual covariates for each UPC, the completed log-price vectors, and the observation mask. The forward pass returns predicted log-demand, the elasticity terms required by the loss, and the auxiliary spline and interaction quantities entering the smoothness and elasticity-band regularizers. Optimization is performed with AdamW. Parameters are separated into decay and no-decay groups: standard network weights receive weight decay, whereas bias terms and the output layers that generate price-response coefficients are excluded from weight decay. Learning rates are reduced on validation plateaus, gradients are clipped to control instability, and the best validation checkpoint is retained. When a GPU is available, training uses automatic mixed precision. 4.2.1
P HASE 0: SMOOTHED LOG - LINEAR WARM START
The first phase trains a restricted version of ICDN whose purpose is to initialize the demand surface around a stable downward-sloping price response. In this phase, nonlinear spline parameters are frozen. In particular, the heads producing the nonlinear spline coefficients wij (h), as well as the spline-spline interaction matrices U (ij) (h), are kept fixed, so the model cannot yet use the full nonlinear spline-spline structure. The resulting specification behaves as a context-dependent log-linear warm start, so that baseline demand and linear price responses are learned before nonlinear flexibility is introduced. The own-price coefficient is initialized using the elasticity prior suggested by the EDA (Appendix A.1). Let βEDA = −2 raw denote the pooled descriptive own-price elasticity prior. The bias of the own-price head βii (x) is initialized through the inverse softplus transformation—due to Eq. 7—so that the initial value of the constrained coefficient satisfies approximately βii (x) ≈ βEDA
for all UPCs at the beginning of training. At the same time, the weights of the output layer that generates raw the own-price coefficient βii (x) are initialized to zero, so that the initial own-price slope is not driven by contextual variation before the model has learned a stable baseline. This gives the network an economically meaningful starting point rather than a random elasticity profile. Phase 0 is trained on smoothed demand targets. Specifically, the log-demand series are sorted by store, UPC, and week and replaced, for this initialization phase, by rolling averages with an eight-week window. This smoothing attenuates high-frequency noise and allows the model to first learn the broad scale and direction of the price-demand relationship. The loss in this phase combines the Huber fit term with the soft elasticityband penalty and the own-price smoothness penalty. The goal is therefore not to obtain the final elasticity estimates, but to produce a stable checkpoint with calibrated baseline demand and economically plausible own-price slopes. 4.2.2
P HASE 1: FULL NONLINEAR DEMAND MODEL
The second phase is initialized from the best checkpoint obtained in Phase 0. The model is then trained on the original, non-smoothed demand targets. At this point, the nonlinear price-response heads wij and Uij are unfrozen, allowing the model to learn own-price and cross-price spline effects, as well as nonlinear cross-price interactions. The same full ICDN loss remains active as in Phase 0, but it is now applied to the fully flexible nonlinear demand specification. The final checkpoint from Phase 1 is used for validation, test-time prediction, and elasticity extraction. Elasticities are computed only for observed UPC entries and, when cross-price elasticities are extracted, only for pairs in which both the demand-receiving UPC and the price-perturbing UPC are observed in the corresponding store-week observation. 24
4.3
H YPERPARAMETER OPTIMIZATION
The main hyperparameters of ICDN are selected with Optuna Akiba et al. (2019). The search is designed to optimize not only predictive performance, but also the economic plausibility of the derivative-based elasticities produced by the model. Each Optuna trial runs the complete two-phase training protocol described above and is evaluated using a tuning protocol based on three expanding temporal folds and three random seeds. In total, we run more than 100 trials under this tuning protocol. After selecting the best hyperparameter configuration, we keep these hyperparameters fixed and perform the final temporal validation and resampling analysis using a broader five-fold, five-seed evaluation protocol. Despite this repeated training procedure, the full hyperparameter search completed in less than one day on our computational setup, using an NVIDIA RTX 5070 Ti GPU. This indicates that the staged training pipeline remains practical for retail demand applications at this scale. The search space includes architectural and optimization choices such as the hidden-layer configuration, dropout, batch size, the number of spline knots, phase-specific learning rates, and the regularization strengths associated with the smoothness and elasticity-band penalties. We do not select hyperparameters solely by validation error. Instead, hyperparameter search is formulated as a multi-objective maximization problem with two validation objectives: predictive fit and elasticity plausibility. The first objective is predictive performance, measured by the global validation R2 on observed log-demand entries. For a validation set indexed by store-week observations b and UPCs i, with observation mask mbi ∈ {0, 1}, we compute P P 2 bbi ) b,i mbi (ybi − y b,i mbi ybi 2 R =1− P . ȳm = P 2, b,i mbi b,i mbi (ybi − ȳm ) Here ybi and ybbi are the observed and predicted log-demands for UPC i in store-week b, respectively, and ȳm is the masked validation mean computed over observed entries. The second objective is an elasticity score, designed to summarize the economic plausibility of the modelimplied price responses on validation data. This score is used only for hyperparameter selection; it is not an additional training loss. Let Eown denote the set of own-price elasticities evaluated on observed validation entries, and let Ecross denote the set of selected cross-price elasticities evaluated on observed validation product pairs. Recall that cross-price elasticities are included only when both the demand-receiving UPC and the price-perturbing UPC are observed. The own-price component of the elasticity score combines two complementary diagnostics: local plausibility and distributional alignment. The own-price in-range share is defined as X 1 1{−5 ≤ e ≤ 0}. Pown = |Eown | e∈Eown
The interval [−5, 0] is used as a soft plausibility region for own-price elasticities. It is inspired by the range of disaggregated beer-demand elasticities reported in Srivastava et al. (2015), consistently with the elasticity-band loss used in Eq. 11. However, Pown only measures the fraction of individual own-price elasticities that fall inside the plausibility band. It does not control whether the distribution as a whole is systematically shifted toward one side of the interval. For example, a model could achieve a high in-range share while producing own-price elasticities that are systematically too close to zero or systematically too negative. We therefore add a median-prior penalty that compares the center of the validation own-price elasticity distribution with the EDA-based 25
reference value βEDA = −2. Let eeown = median{e : e ∈ Eown }. The prior-deviation penalty is Pprior = min
max (0, |e eown − βEDA | − 0.3) ,1 . |βEDA |
The tolerance band of 0.3 allows moderate deviations from the EDA reference without penalty. Beyond this tolerance, the penalty increases with the absolute deviation of the median own-price elasticity from βEDA , and is capped at one for numerical stability. The own-price component of the elasticity score is then Sown = Pown (1 − Pprior ) , where Pown controls the plausibility of individual elasticity estimates, whereas Pprior controls systematic bias in the central tendency of the own-price elasticity distribution. For cross-price elasticities, the score discourages extreme values. Specifically, we compute X 1 Scross = 1{−1 ≤ e ≤ 1}, |Ecross | e∈Ecross
with Scross = 1 if no cross-price elasticity is available in a given validation evaluation.The interval [−1, 1], which is also used in the elasticity-band loss in Eq. 11, is motivated by the empirical evidence in ToroGonzález et al. (2014), where cross-price effects across beer types in Dominick’s data are found to be weak and close to zero. The final elasticity score combines own-price and cross-price plausibility: Selast = 0.7 Sown + 0.3 Scross . The larger weight on Sown reflects the fact that own-price effects are typically better identified and less noisy in observational retail data, whereas cross-price effects are weaker, more heterogeneous, and more difficult to estimate reliably. For each Optuna trial t, both R2 and Selast are computed across temporal folds and random seeds. Let Ht denote the set of fold-seed evaluations associated with trial t. We summarize each objective by a robust score that rewards high average performance while penalizing variability across folds and seeds: (t) 2,(t) Rrobust = R2 − 0.25 sdh∈Ht Rh2 , and
(t)
(t)
Selast,robust = S elast − 0.25 sdh∈Ht (Selast,h ) , where the bars denote averages over the fold-seed evaluations h ∈ Ht . After the search, we select the final configuration using a scalar robust ranking criterion. Let T denote the set of completed candidate trials considered for final selection. For each trial t ∈ T , we define (t)
2,(t)
(t)
Sselect = Rrobust + Selast,robust . The selected configuration is (t)
t⋆ = arg max Sselect . t∈T
Table 1 reports the selected hyperparameter configuration together with its validation performance. 26
Table 1: Selected hyperparameter configuration from the Optuna search. Validation performance is reported under the tuning protocol used during hyperparameter optimization, based on three temporal folds and three random seeds.
4.4
Hyperparameter / metric
Value
Selected trial Selection score Mean validation R2 Std. validation R2 Mean elasticity score Std. elasticity score Batch size Dropout Hidden layers Number of spline knots K LRP 0 LRP 1 λsm λelast
28 1.5826 0.6636 0.1020 0.9505 0.0240 256 0.2547 (256, 128, 64) 3 1.686 × 10−3 1.625 × 10−3 3.514 × 10−2 4.450 × 10−2
T EMPORAL VALIDATION AND RESAMPLING
After hyperparameter optimization, the selected ICDN configuration is kept fixed for the final evaluation. The final protocol follows the same expanding-window validation logic used during tuning, but is deliberately broader: instead of the three temporal folds and three random seeds used in Optuna, we evaluate the selected model over five expanding temporal folds and five random seeds. Each temporal fold trains on an initial block of weeks and validates on a later block, with the training window expanding across folds. Validation metrics are computed only on observed entries, using the same observation mask as in training. This expanded fold-seed design separates hyperparameter selection from final assessment. Temporal folds evaluate generalization to later time periods, while random seeds assess robustness to stochastic elements of deep-learning training, including parameter initialization, mini-batch ordering, dropout, and the online sparse-neighbor selection path. These stochastic training paths are not part of the reported ICDN demand specification; the repetitions are used only as evaluation diagnostics for predictive robustness and elasticity reproducibility. In addition, we use block bootstrap resampling over training weeks to assess the sampling stability of the elasticity estimates. Rather than resampling individual store-week observations independently, weeks are grouped into non-overlapping temporal blocks and sampled with replacement. This preserves short-run temporal dependence within each resampled block while generating alternative training samples. Repeating the training procedure over bootstrap samples produces a distribution of predicted elasticities, from which bootstrap standard deviations and confidence intervals are computed. The bootstrap runs are therefore used only as uncertainty and stability diagnostics for the estimated elasticities, not to define the demand model itself. 4.5
E VALUATION BENCHMARK
This benchmark is intended as a classical, interpretable econometric reference rather than as an exhaustive comparison against all possible machine-learning demand models. Accordingly, the empirical results be27
low should be interpreted as evidence relative to this directed log-log baseline, not as a claim of general superiority over alternative demand-first neural networks, or other flexible predictive architectures. Starting from the processed weekly dataset, each row corresponds to a (store code, upc code, week id) observation. We transform this long panel into a directed pairwise dataset. For each store-week, and for each ordered pair of distinct UPCs (i, j), we create one row containing the demand and own price of UPC i, together with the price of UPC j. Thus, for each directed pair, UPC i is the demand-receiving product and UPC j is the price-perturbing product. For each store s and directed UPC pair (i, j), we estimate the log-log regression own cross ⊤ log vsti = β0,sij + βsij log psti + βsij log pstj + zsti γsij + εsti .
(15)
Here vsti denotes liters sold for UPC i in store s and week t, psti is the price per liter of UPC i, and pstj is the price per liter of candidate UPC j in the same store-week. The term β0,sij is the intercept for store s and directed pair (i, j), capturing the baseline log-demand level in that local regression. The vector zsti contains the benchmark controls listed in Table 2—all defined for the demand-receiving UPC i—, and εsti denotes the regression disturbance, capturing residual variation in log-demand not explained by prices and controls. own The coefficient βsij is interpreted as the own-price elasticity of UPC i within this pairwise specification, cross while βsij is the directed cross-price elasticity measuring the local association between the price of UPC j and the demand of UPC i. The vector zsti contains a compact set of control variables designed to make the pairwise OLS benchmark interpretable and numerically stable. It is not formed by using all variables in the final deep-learning dataset reported in Tables 13–17. Using the full feature set would make the benchmark unnecessarily unstable, since many covariates are highly related by construction, including lagged and rolling demand summaries, seasonal controls, promotion variables, and competitive-neighborhood aggregates. This is especially problematic because the benchmark fits a separate regression for each store and directed product pair. In these smaller regressions, many collinear controls can make the estimates unstable. We therefore define zsti as a compact, collinearity-aware subset of controls selected from the final feature schema. To make inference less sensitive to heteroskedasticity across store-week observations, we report robust standard errors using the HC1 covariance estimator. Before fitting, rows with missing values in any required regressor are dropped. A regression is attempted only when the corresponding directed group is present in both the training and validation split, contains at least 30 clean training observations, and exhibits at least two distinct values for both the own log-price and the cross log-price. For each valid directed regression, we retain the estimated own-price elasticity, the estimated cross-price elasticity, their HC1 robust confidence intervals and p-values, together with the number of training and validation observations and the validation-period prediction metrics. In particular, we evaluate MAE, RMSE, and validation R2 for the predicted log-demand of UPC i. 4.6
R ESULTS
We now evaluate whether ICDN improves out-of-sample demand prediction relative to the log-log benchmark, and whether this improvement is accompanied by more stable and economically coherent modelimplied elasticity estimates. For clarity and comparability, the empirical evaluation is restricted to a fiveproduct subset. This subset is selected to maximize store-week overlap across UPCs, so that own- and cross-price elasticities can be compared on a common set of market instances and the resulting stability diagnostics are not driven by differences in product availability. The restriction is made for expositional and diagnostic purposes: the ICDN architecture itself is not limited to five products and can be extended to larger product sets through the same sparse interaction mechanism. 28
Table 2: Control variables used in the pairwise OLS benchmark. The benchmark uses a compact, collinearity-aware subset of the final feature set rather than all available covariates. Variable(s)
Definition and motivation
on_promo
Promotion indicator. Controls for the focal product’s promotional state, which is strongly correlated with price. Gap-free sequential time index. Captures smooth aggregate time trends in the weekly panel. Annual Fourier seasonality terms. Capture yearly demand patterns.
week_rank sin_52 cos_52 sin_13 cos_13 weeks_since_first_seen_store_upc lag_1_log_liters_sold lag_4_log_liters_sold miss_lag_1 miss_lag_4 promo_intensity_store_week n_neighbors_sw_cat neighbor_promo_share_sw_cat lag1_neighbor_mean_log_liters_sold share_new_neighbors_13w
Quarterly Fourier seasonality terms. Capture shorter recurrent seasonal patterns. Weeks since the UPC first appears in the corresponding store. Controls for local product lifecycle effects such as rollout and stabilization. Lagged log demand for the same store-UPC series. Captures short-run and monthlylike demand persistence. Missingness indicators for the retained lagged demand features. Share of UPCs on promotion in the same store-week. Controls for the store-wide promotional environment faced by the focal UPC. Number of other UPCs in the same store-week-category. Measures the size of the active category-level competitive set. Share of category neighbors on promotion. Captures promotional pressure from rival UPCs in the same category. Mean lag-1 log demand across valid category neighbors. Summarizes recent local category-demand conditions. Share of neighbors that are new in-store within the previous 13 weeks. Controls for changes in the competitive set due to product introductions or activations.
All results in this section are obtained after fixing the hyperparameters selected by Optuna and re-evaluating the selected configuration under the expanded five-fold, five-seed temporal validation protocol described above. The comparison is organized in four steps. First, we compare predictive generalization across temporal validation folds. Second, we analyze own-price elasticity stability. Third, we extend the analysis to cross-price elasticities. Finally, we evaluate resampling-based uncertainty diagnostics by comparing bootstrap intervals against fold-based estimates. Predictive generalization. Table 3 reports the out-of-sample comparison between ICDN and the benchmark. When results are summarized by temporal fold, ICDN improves validation R2 in all five comparable temporal folds. Because this comparison contains only five temporal folds, the paired t-tests are reported as descriptive diagnostics rather than as strong inferential evidence. The R2 differences favor ICDN, with t = 3.757 and p = 0.0198. The MAE and RMSE differences also point in favor of ICDN, but they are less conclusive when inference is based on the five temporal folds. The advantage of ICDN becomes clearer in the finer (store, UPC, fold) comparison. Across 1,241 matched triplets, ICDN attains lower MAE in 63.2% of cases and lower RMSE in 63.0% of cases. The median improvements are −0.0285 for MAE and −0.0332 for RMSE, where negative values indicate lower prediction error for ICDN. We report Wilcoxon signed-rank tests as descriptive matched-sample diagnostics, since the triplets are not independent due to repeated stores and UPCs, and temporal dependence across folds. Under this diagnostic comparison, both MAE and RMSE differences favor ICDN. Figure 3 visualizes this pattern: ∆R2 is positive in every fold, and the distributions of ∆MAE and ∆RMSE across matched (store, UPC, fold) triplets are shifted to the left of zero. Taken together, these descriptive diagnostics suggest that ICDN generalizes better than the log-log benchmark in the temporal validation comparisons considered here. Better out-of-sample generalization therefore 29
Table 3: Out-of-sample generalization comparison between ICDN and the benchmark. Negative ∆MAE and ∆RMSE indicate lower error for ICDN. Positive ∆R2 indicates better fit for ICDN. Metric / test Comparable folds Paired t-test on ∆R2 Paired t-test on ∆MAE Paired t-test on ∆RMSE Comparable (store, UPC, fold) triplets ICDN wins in MAE ICDN wins in RMSE Median ∆MAE Median ∆RMSE Wilcoxon on ∆MAE Wilcoxon on ∆RMSE
Value
Interpretation
5 t = 3.757, p = 0.0198 t = −2.452, p = 0.0703 t = −2.035, p = 0.1116
Matched temporal folds ICDN significantly better Favors ICDN; not significant at 5% Favors ICDN; not significant at 5%
1241 63.2% 63.0% −0.0285 −0.0332 p = 1.16 × 10−38 p = 1.83 × 10−38
Pair-level comparison Lower MAE than benchmark Lower RMSE than benchmark Favors ICDN Favors ICDN Favors ICDN Favors ICDN
Figure 3: Generalization comparison between ICDN and the benchmark. Left: fold-level ∆R2 , computed as ICDN minus benchmark. Middle and right: pair-level histograms of ∆MAE and ∆RMSE, where values below zero indicate lower prediction error for ICDN.
provides evidence that the learned surface is less driven by sample-specific noise, which in turn supports more stable derivative-based elasticity estimates. Own-price elasticity stability. Table 4 summarizes the stability diagnostics for own-price elasticities. In the bootstrap comparison over 212 matched store-UPC series, ICDN yields narrower confidence intervals in 83.5% of cases and lower bootstrap standard deviations in 85.4% of cases. The difference in dispersion is substantial: the median confidence interval width is 1.018 for ICDN and 2.683 for the benchmark, while the median bootstrap standard deviation is 0.252 for ICDN and 0.789 for the benchmark. The two approaches also display strong directional agreement. The same-sign rate is 99.1%, indicating that the models almost always agree on the sign of the own-price response. Moreover, ICDN satisfies the downward-sloping demand prior in all matched cases, with 100% negative own-price elasticities, compared with 99.1% for the benchmark. This is consistent with the sign-constrained own-price head in Eq. 7, together with the soft own-price elasticity-band penalty imposed during training. Temporal stability shows the same pattern. Among the 272 store-UPC series observed in at least three folds, ICDN is more stable across folds in 82.7% of cases. The median inter-fold standard deviation is 0.2557 for ICDN and 0.6026 for the benchmark. Figure 4 illustrates these diagnostics. The confidence-interval 30
Table 4: Own-price elasticity stability comparison between ICDN and the benchmark. Metric
ICDN
Benchmark
Matched series (bootstrap) Narrower CI rate Lower bootstrap std rate Same-sign rate Negative own-elasticity rate Median CI width Median bootstrap std
212 16.5% 14.6% 99.1% 100.0% 99.1% 1.018 2.683 0.252 0.789
Series with at least 3 folds More stable across folds Median inter-fold std
82.7% 0.2557
83.5% 85.4%
272 17.3% 0.6026
scatterplot is concentrated below the diagonal, indicating narrower ICDN intervals for most matched series; the point-estimate scatterplot shows strong sign agreement; and the inter-fold dispersion boxplot shows a tighter distribution for ICDN, while the benchmark exhibits a heavier upper tail.
Figure 4: Own-price elasticity stability diagnostics. Left: bootstrap confidence interval width for ICDN versus the benchmark. Middle: mean own-price elasticity estimates from both methods. Right: inter-fold standard deviation of own-price elasticities, showing lower temporal dispersion for ICDN and a heavier upper tail for the benchmark. These results support the interpretation that ICDN produces own-price elasticities that are economically plausible and substantially less sensitive to both bootstrap resampling and temporal partitioning. Cross-price elasticity stability. We next examine cross-price elasticities. This analysis is important because cross-price effects are typically weaker, noisier, and harder to identify from observational retail data than own-price effects. Table 5 reports the bootstrap and temporal-fold stability diagnostics for matched cross-price pairs. In the bootstrap comparison over 544 matched store-UPC-pair combinations, ICDN produces narrower confidence intervals in 79.8% of cases and lower bootstrap standard deviations in 83.3% of cases. The median confidence interval width is 0.650 for ICDN versus 1.824 for the benchmark, and the median bootstrap standard deviation is 0.159 versus 0.539. Thus, the cross-price effects implied by ICDN remain substantially more stable across bootstrap resamples than those of the benchmark, although the gain is less extreme than in the own-price case. 31
Table 5: Cross-price elasticity stability comparison between ICDN and the benchmark. Metric
ICDN
Benchmark
Matched cross-pairs (bootstrap) Narrower CI rate Lower bootstrap std rate Same-sign rate Positive cross-elasticity rate Median CI width Median bootstrap std Median cross-price elasticity
544 20.2% 16.7% 39.9% 91.0% 33.8% 0.650 1.824 0.159 0.539 0.302 −0.260
Cross-pairs with at least 3 folds More stable across folds Median inter-fold std
75.5% 0.1703
79.8% 83.3%
948 24.5% 0.2968
The average bootstrap elasticity estimates also reveal an important qualitative difference. The median crossprice elasticity is 0.302 for ICDN and −0.260 for the benchmark. Moreover, ICDN assigns positive crossprice elasticities in 91.0% of matched pairs, compared with 33.8% for the benchmark. This pattern is more consistent with a substitution interpretation among beer UPCs, although it should not be read as causal evidence of substitution. The same-sign rate for cross-price elasticities is 39.9%, substantially lower than for own-price elasticities, reflecting the greater difficulty of estimating cross-product responses and the fact that the two methods impose very different forms of regularization and interaction structure. Temporal stability reinforces the same general conclusion. Among 948 cross-price pairs observed in at least three folds, ICDN is more stable across temporal splits in 75.5% of cases. The median inter-fold standard deviation is 0.1703 for ICDN and 0.2968 for the benchmark. Figure 5 shows the same pattern visually: ICDN cross-price elasticities are more concentrated, its bootstrap intervals are generally narrower, and its inter-fold dispersion is lower for most matched pairs. Overall, the cross-price analysis suggests that ICDN does not merely stabilize own-price elasticities. It also regularizes the more difficult cross-price problem, producing cross effects that are more reproducible across resamples and temporal splits, while yielding a direction of cross-price response that is more consistent with substitution among related UPCs. Resampling-based uncertainty diagnostics. Narrow bootstrap intervals indicate stability across resampled training sets, but they do not by themselves show whether uncertainty is properly sized relative to the variation observed across temporal splits. We therefore compare nominal 95% bootstrap confidence intervals with fold-based elasticity estimates obtained from the temporal validation protocol. Table 6 reports the resulting uncertainty diagnostics for both own-price and cross-price elasticities. For own-price elasticities, empirical coverage is 99.4% for ICDN and 87.4% for the benchmark. Thus, ICDN exhibits over-coverage relative to the nominal 95% target, whereas the benchmark under-covers. The dispersion-ratio diagnostic gives a complementary view7 . The median ratio between bootstrap standard deviation and inter-fold standard deviation is 1.43 for ICDN and 1.35 for the benchmark, indicating that both methods are conservative in median dispersion for own-price elasticities. In this sense, ICDN’s own-price bootstrap intervals are not under-sized; rather, they are somewhat conservative relative to the fold-based variability observed in the final evaluation. 7
The dispersion ratio is the bootstrap standard deviation divided by the inter-fold standard deviation. Values near one indicate similar variability under bootstrap resampling and across temporal folds.
32
Figure 5: Cross-price elasticity diagnostics. Top left: distribution of bootstrap mean cross-price elasticities for ICDN and the benchmark. Top right: bootstrap confidence interval width comparison. Bottom left: cross-price elasticity point estimates from both methods. Bottom right: inter-fold standard deviation of cross-price elasticities. Figure 6 reports the own-price uncertainty diagnostics. The coverage distributions show that ICDN intervals frequently cover all fold-based estimates for a given series, while the benchmark has more under-covered series. The dispersion-ratio histogram shows both methods shifted above one, consistent with conservative median bootstrap dispersion for own-price elasticities. For cross-price elasticities, the picture is different. ICDN covers 83.6% of fold-based cross-price estimates, compared with 90.8% for the benchmark, so both methods under-cover relative to the nominal 95% target, with stronger under-coverage for ICDN. At the same time, ICDN’s median dispersion ratio is 1.02, very close to the ideal value of one, whereas the benchmark ratio is 1.79, indicating substantially more conservative bootstrap dispersion. These results suggest that cross-price uncertainty remains harder to size: ICDN matches the median dispersion across temporal folds well, but its bootstrap intervals do not fully cover the fold-based variation. Main takeaway. Overall, the results support three conclusions. First, ICDN appears to generalize better than the log-log benchmark across the temporal validation periods considered here. Second, this predictive 33
Table 6: Resampling-based uncertainty diagnostics for bootstrap elasticity intervals. Coverage compares nominal 95% bootstrap intervals against fold-based point estimates. The dispersion ratio is defined as bootstrap standard deviation divided by inter-fold standard deviation. Metric
ICDN
Benchmark
Own-price elasticities 95% CI coverage over fold estimates Number of fold estimates used Deviation from 95% target Median dispersion ratio
99.4% 1081 +4.4 p.p. 1.43
87.4% 1049 −7.6 p.p. 1.35
Cross-price elasticities 95% CI coverage over fold estimates Number of fold estimates used Deviation from 95% target Median dispersion ratio
83.6% 2714 −11.4 p.p. 1.02
90.8% 2698 −4.2 p.p. 1.79
Figure 6: Own-price resampling-based uncertainty diagnostics for bootstrap confidence intervals. Left: per-series coverage distributions relative to the 95% target. Middle: dispersion ratio, defined as bootstrap standard deviation divided by inter-fold standard deviation. Right: overall empirical coverage of nominal 95% intervals.
advantage is accompanied by substantially more stable own-price elasticities, both under bootstrap resampling and across temporal folds. Third, ICDN also improves the reproducibility of cross-price elasticities relative to the benchmark, although these effects remain more difficult to estimate and display weaker agreement across methods. The uncertainty diagnostics provide an additional qualification. Own-price bootstrap intervals are conservative relative to fold-based estimates, whereas cross-price intervals show under-coverage despite a nearideal median dispersion ratio. Thus, ICDN produces demand derivatives that generalize better, fluctuate less across comparable samples, and remain more consistent with economically plausible price-response patterns than those of the classical log-log benchmark considered here, but uncertainty calibration remains imperfect, especially for cross-price effects. These findings should nevertheless be interpreted carefully. Because the empirical design is based on observational scanner data, the diagnostics do not establish recovery of causal ground-truth price responses. 34
5
C ONCLUSIONS
This paper studied the problem of learning economically meaningful price elasticities in a high-dimensional multi-product retail setting. Rather than parameterizing elasticities directly, we adopted a demand-first perspective: log-demand is modeled as a smooth, context-dependent function of log-prices and covariates, and elasticities are obtained as exact derivatives of that same demand surface. This design addresses a central consistency issue in elasticity modeling. If elasticities are learned as independent local objects, they need not correspond to any globally coherent demand representation. By contrast, any sufficiently smooth demandfirst construction guarantees row-wise derivative consistency when elasticities are defined as derivatives of log-demand. To instantiate this idea, we introduced the Integrable Context-Dependent Demand Network (ICDN), a splinebased neural architecture that combines product-specific contextual conditioning, analytic price-basis derivatives, and sparse attention-modulated cross-price interactions. The model represents each SKU through a contextual token and a product-specific latent vector, generates own-price parameters from individual product representations, and constructs directed cross-price effects from ordered pairs of SKU representations. This yields heterogeneous and context-dependent elasticities while preserving analytic tractability. Importantly, ICDN does not impose Hicksian or Slutsky symmetry: cross-price elasticities are interpreted as directional local responses in the fitted demand surface, so the effect of product j’s price on product i’s demand need not equal the reverse effect. Empirically, the results show that ICDN improves out-of-sample generalization relative to the log-log benchmark. Across matched temporal folds, ICDN achieves higher validation R2 , and at the finer store-SKU-fold comparison it attains lower MAE and RMSE in a majority of matched cases. This predictive improvement is important because elasticities in ICDN are not separate fitted coefficients. A model that generalizes more robustly out of sample is therefore less likely to produce derivative estimates driven by sample-specific noise. The own-price elasticity results strongly support this interpretation. ICDN produces own-price elasticities that remain aligned with the economic prior of downward-sloping demand, while displaying substantially lower bootstrap dispersion and lower inter-fold variability than the benchmark in most comparable series. The cross-price results are more nuanced but still favorable to ICDN. Cross-price effects are weaker and harder to estimate from observational retail data, yet ICDN produces narrower bootstrap intervals, lower resampling variance, and lower temporal dispersion for most matched directed product pairs. In contrast to the benchmark, ICDN also yields predominantly positive cross-price elasticities, which is more consistent with a substitution interpretation among related beer SKUs. This sign pattern should nevertheless be interpreted cautiously, since the estimates are local derivatives of an observationally fitted demand surface rather than causally identified substitution effects. The resampling-based uncertainty diagnostics also call for a careful interpretation. For own-price elasticities, ICDN exhibits over-coverage relative to nominal 95% bootstrap intervals, with conservative median dispersion compared with fold-based variability. For cross-price elasticities, ICDN achieves a median bootstrapto-fold dispersion ratio close to one, but its intervals under-cover fold-based point estimates. Thus, the evidence supports improved stability, reproducibility, and economic coherence, but not perfect uncertainty calibration. More precisely, the results show that ICDN produces elasticities that are more reproducible across resamples and temporal splits; they do not prove that the model recovers unobserved ground-truth elasticities in an absolute causal sense. These conclusions are directly relevant for applied pricing. In operational settings, the usefulness of an elasticity model depends not only on point prediction accuracy, but also on whether the implied local responses are stable enough to support simulation, pricing diagnostics, and scenario analysis. By tying demand predictions and elasticities to the same differentiable potential, ICDN provides a principled way to improve derivative stability while preserving predictive performance. In this empirical setting, and relative to the log35
log benchmark, the resulting elasticity estimates appear better suited for pricing diagnostics and scenario analysis in which local price responses must be both interpretable and robust. Several directions remain open for future work. First, we could improve uncertainty quantification by modeling temporal changes in demand and dependence across stores, weeks, and SKUs more explicitly. For example, future demand responses may be affected by changes in inflation, category trends, consumer preferences or competitive assortment. The bootstrap diagnostics used here capture historical sampling variability, but they may not fully represent the uncertainty involved in using elasticities for future pricing decisions. We could also combine the demand-first architecture with explicit causal identification strategies, such as randomized price experiments, to distinguish predictive price sensitivities from causal price effects. Second, the empirical comparison could be broadened beyond the classical log-log benchmark considered here. In particular, we should compare ICDN with other demand-first flexible models, including generic neural predictors with automatic differentiation, and alternative spline-based demand specifications. Third, the cross-price structure learned by ICDN could be analyzed more deeply as a directed product-interaction network, allowing substitution, complementarity, and cannibalization patterns to be studied across individual SKUs. Fourth, the framework should be evaluated in operational pricing tasks, such as revenue optimization, margin-aware pricing, or promotion planning, to assess whether more stable elasticities translate into better operational decisions.
C ODE AND DATA AVAILABILITY The code and processed data used to reproduce the preprocessing pipeline, model training, and empirical results are available on GitHub.
ACKNOWLEDGMENTS CH: To my beloved Eva and my precious Daniela, thank you for being my endless inspiration and the driving force behind everything I do. DR: I would like to express my gratitude to Carlos Heredia for taking the time to teach me. Both authors gratefully acknowledge DAMM’s Artificial Intelligence Department (IAMM) for the valuable discussions on this topic and for the encouragement and motivation provided throughout this work, in particular, Daniel Losada and Lluis Pallarès.
AUTHOR C ONTRIBUTIONS Both authors contributed equally to this work.
F UNDING The authors declare that no funding was received for this work.
A
A PPENDIX
A.1
E XPLORATORY DATA A NALYSIS (EDA)
This appendix summarizes the exploratory analysis used to evaluate data quality, characterize the empirical price-demand relationship, and motivate the sample restrictions and feature design adopted for elasticity 36
estimation. Throughout the EDA, we work with x := log(price per liter),
y := log(liters sold).
The objective is to document the data conditions under which own- and cross-price model-implied elasticities can be estimated more stably and reproducibly. A.1.1
DATA QUALITY AND SAMPLE CONSTRUCTION
Before conducting the EDA, we apply two basic data-quality filters. First, we remove observations with units sold equal to zero, since these rows do not contribute information about positive realized demand and cannot be used in the log-demand specification. Due to this fact, the final estimation sample supports inference on positive observed log-demand only; it does not model the full process governing purchase/nopurchase outcomes, zero sales, stockouts, or assortment availability. Second, we exclude transactions with total price ≤ 0.05, which are likely to reflect recording errors or economically implausible price realizations. After these filters, the dataset is unique for each (store code, week id, upc code) combination: the number of rows exactly matches the number of unique store-week-UPC keys. We then evaluate temporal support at both the store-week and store-UPC observations. Table 7 summarizes the main sampleconstruction steps. The initial quality-filtered sample contains 1,966,147 observations and 21,004 observed store-week combinations out of 26,878 possible combinations, corresponding to a coverage rate of 78.15%. To ensure a minimum level of longitudinal support, we exclude stores with fewer than 150 observed weeks. This removes 19 stores and leaves a working sample of 70 stores. Table 7: Sample construction and main filtering steps. Step After basic quality filters Observed store-weeks Stores retained Store-UPC pairs before pair filters After identification-oriented filters After UPC-level log-price outlier removal
Rows / units retained
Comment
1,966,147 rows 21,004 / 26,878 70 stores 27,390 pairs 484,864 rows 463,722 rows
Unique at store-week-UPC level 78.15% coverage Excluding stores < 150 observed weeks Working panel after store filter 2,656 store-UPC pairs retained Final cleaned observed sample
After restricting to these stores, the panel contains 27,390 store-UPC pairs. Table 8 shows that temporal support is highly irregular. The median store-UPC pair is observed for only 41 weeks, and the median within-span coverage ratio is 0.647, meaning that a typical pair is observed in only about two thirds of the weeks between its first and last appearance. Internal intermittency is also substantial, with a median of 24 missing weeks within the active span. These diagnostics show that the raw store-UPC panel is both uneven in length and temporally irregular, which weakens elasticity identification within store-UPC series and motivates additional filtering before model training.. Table 8: Temporal support of store-UPC series before identification-oriented filtering. Metric
Mean
Median
p25
p75
p90
Observed weeks per store-UPC pair Within-span coverage ratio Missing weeks within active span
69.6 0.624 40.5
41 0.647 24
12 0.400 8
109 0.877 57
194 0.969 107
37
A.1.2
P RICE VARIATION , PROMOTIONS , AND IDENTIFICATION - ORIENTED FILTERING
Elasticity estimation requires sufficient within-series price movement. Table 9 reports the main pricevariation and promotion diagnostics. Although the raw panel contains meaningful price movement overall, a non-trivial fraction of series remains weakly informative: 3,859 store-UPC pairs, corresponding to 14.09% of the total, exhibit constant prices throughout their observed history. The median within-series log-price range is 0.252, implying a median max/min price ratio of approximately exp(0.252) ≈ 1.287. A more important identification concern is the strong link between prices and promotions. Promotional incidence is dominated by promo B, while promo S and promo C are rare. Promotional weeks are typically associated with lower prices, and the pair-level correlation between log-price and on promo is strongly negative. Moreover, a large share of price changes coincides with promotion-state switches. These patterns imply that naive price variation is not always cleanly separable from promotional activity. Table 9: Price variation and promotion-related diagnostics. Metric
Mean
Median
p25
p75
p90
Distinct log-price levels Price changes Within-series log-price range Price-promotion correlation Share of price changes at promo switch
6.53 21.8 0.233 -0.759 0.628
5 11 0.252 -0.810 0.714
2 2 0.134 0.526
9 32 0.337 0.835
15 65 0.406 1.000
Constant-price store-UPC pairs Promo B incidence Promo S incidence Promo C incidence Average promo-price gap Median promo-price gap Series with higher promo prices
3,859 pairs (14.09%) 27.1% of observations 0.5% of observations 0.02% of observations -0.160 log-price units -0.168 log-price units 1.58%
Taken together, these diagnostics show that not all store-UPC pairs are equally informative for elasticity estimation. We therefore construct the modeling sample using identification-oriented filters. We retain only store-UPC series with at least 52 observed weeks, within-span coverage of at least 0.75, at least 3 distinct price values, at least 5 price changes, and a within-series log-price range of at least 0.15. To reduce promodriven entanglement, we additionally require the absolute correlation between log price and on promo to be at most 0.80, and the share of price changes coinciding with promotion-state switches to be at most 0.80. This filtering step retains 2,656 store-UPC pairs and reduces the sample from 1,905,978 rows to 484,864 rows, yielding a panel with stronger temporal support, more meaningful within-series price variation, and less severe promo-price collinearity. A.1.3
O UTLIERS AND DESCRIPTIVE PRICE - DEMAND EVIDENCE
We next examine extreme observations in log price and log demand using the IQR rule. Table 10 summarizes the resulting outlier rates and descriptive price-demand evidence. Outliers are present, but they do not dominate the filtered panel. Under a global IQR criterion, 3.95% of observations are flagged as outliers in log price per liter, compared with 0.57% in log liters sold. When the diagnostic is computed within upc code, the outlier rate is 4.36% for log price and 0.49% for log demand. Since atypical price realizations are especially problematic for elasticity learning, we remove outliers computed within each UPC in log price per liter. This reduces the sample from 484,864 rows to 463,722 rows. 38
Table 10: Outliers and descriptive price-demand evidence in the cleaned sample. Diagnostic
Value
Global log-price outlier rate Global log-demand outlier rate Within-UPC log-price outlier rate Within-UPC log-demand outlier rate Rows after UPC-level log-price outlier removal
3.95% 0.57% 4.36% 0.49% 463,722
Spearman correlation: log price vs. log demand Within-store-UPC demeaned Spearman correlation Non-promotional Spearman correlation
-0.505 -0.451 -0.442
Pooled log-log elasticity Two-way fixed-effects log-log elasticity Pooled R2 Two-way fixed-effects R2
-1.98 -2.97 0.229 0.235
The cleaned sample retains a robust negative association between price and demand. The global Spearman correlation between log-price and log-demand is −0.505. After within-(store, UPC) demeaning, the correlation remains negative at −0.451, indicating that the inverse association is not driven solely by crossseries composition. Restricting to non-promotional observations yields a similarly negative correlation of −0.442, suggesting that the signal persists outside promotional regimes. As an additional descriptive benchmark, we estimate simple log-log OLS specifications relating log(liters sold) to log(price per liter). The pooled regression yields an elasticity of approximately −1.98, whereas a two-way fixed-effects specification, absorbing store-UPC heterogeneity and week effects, produces a more negative estimate of about −2.97. The explanatory power remains modest in both cases. These patterns support the existence of a robust negative own-price signal, while also showing that price alone explains only part of the variation in log demand. This motivates a richer nonlinear specification with contextual, temporal, promotional, and competitive controls. A.1.4
C ALENDAR COMPLETION AND TEMPORAL FEATURE CONSTRUCTION
After outlier filtering, the working panel still exhibits substantial temporal incompleteness. The cleaned observed sample contains 463,722 rows across 70 stores and 203 UPCs, spanning week id values from 91 to 399. Although this range contains 309 possible week identifiers, only 302 weeks are actually observed. Seven weeks are globally missing from the dataset altogether: week id ∈ {219, 262, 263, 264, 265, 284, 285}. To make temporal structure explicit and to compute lagged and rolling features on a coherent weekly grid, we expand each store-UPC history to its full weekly span. The inserted rows are marked as calendar-completion rows rather than treated as real economic observations. Table 11 summarizes the calendar-completion step. The expansion increases the panel from 463,722 observed rows to 532,965 rows by adding 69,243 synthetic rows. Of these, 15,816 correspond to globally missing weeks and 53,427 correspond to internal gaps within individual store-UPC series. Price, quantity, and promotion-related variables are intentionally left missing on synthetic rows; as a result, these economic fields exhibit 12.99% missingness in the calendar-completed panel. The completed panel also reveals strong temporal structure. When aggregated by week, week rank, a gap-free sequential time index constructed after calendar completion, is positively correlated with mean 39
Table 11: Calendar completion and temporal-gap structure. Quantity
Value
Cleaned observed rows before calendar completion Rows after calendar completion Synthetic rows added Rows from globally missing weeks Rows from internal store-UPC gaps Globally missing week IDs Missingness in economic fields after completion Observed weeks in cleaned panel Stores UPCs
463,722 532,965 69,243 15,816 53,427 219, 262-265, 284-285 12.99% 302 70 203
log-price, with correlation 0.82; negatively correlated with aggregate log demand, with correlation −0.74; and positively correlated with promotional intensity, with correlation 0.32. In addition, both demand and price display strong persistence: their autocorrelations are 0.96 and 0.98 at lag 1, respectively, and remain 0.32 and 0.33 at lag 52. These patterns motivate the temporal covariates used in the final model, including week rank, Fourier seasonality terms with periods 52, 26, and 13 weeks, lifecycle counters, lagged logdemand features, rolling summaries, missingness indicators, and store-week promotional intensity. A.1.5
F INAL MODELING SAMPLE AND FEATURE SET
Because temporal covariates are constructed on the calendar-completed panel, the final estimation sample is obtained by projecting those features back onto observed economically valid rows. Specifically, we exclude all rows created during calendar completion, remove observations belonging to globally missing weeks, and retain only rows with non-missing and finite values for both log(liters sold) and log(price per liter). ICDN is therefore trained exclusively on real store-UPC-week observations, while still benefiting from temporal features computed on the calendar-aware panel. The final schema is organized by feature family in Tables 12-17. It includes store and UPC identifiers; product descriptors such as brand family, style segment, category, and package size; the supervised target log qL ; the main price input log pL ; current promotion status and store-week promotional intensity; temporal and lifecycle controls; autoregressive summaries of past demand; missingness indicators for lagged and rolling features; dynamic competitive-neighborhood variables; and static assortment measures. This feature design follows directly from the EDA. The cleaned sample retains a robust negative pricedemand association, while demand and price display strong persistence and seasonality. Promotional activity remains sufficiently intertwined with price to justify explicit promotion controls, and the competitive environment varies meaningfully across store-week-category series. The variable log qL is retained as the supervised learning target, whereas log pL enters as the main economic price input from which elasticities are derived.
40
Table 12: Final dataset schema: identifiers and product descriptors. Variable(s)
Definition and motivation
store_code
Store identifier. Captures persistent store-specific heterogeneity in baseline demand, assortment, and local consumer behavior. Product identifier. Captures UPC-level heterogeneity in baseline demand, product appeal, and price sensitivity. Normalized brand-family label. Provides brand membership information and supports same-brand neighborhood features. Normalized style-segment label. Adds product-descriptor information beyond category, allowing the model to distinguish heterogeneous product styles. Product-category identifier. Defines higher-level product groupings and categorybased competitive neighborhoods. Liters contained in one sellable UPC unit. Controls for package-size differences and supports price and quantity normalization.
upc_code brand_family_norm style_segment_norm category_code liters_per_upc
Table 13: Final dataset schema: target, price, and promotion variables. Variable(s)
Definition and motivation
log_liters_sold
Log demand in liters. This is the supervised learning target y, retained for training and evaluation but not used as an input when predicting demand. Log price per liter. This is the main economic price input u, from which own-price and cross-price elasticities are derived. It enters only through the explicit price branch and is not included among the contextual covariates. Indicator that the UPC is on promotion in the corresponding store-week. Controls promotional status, which is strongly related to price and relevant for isolating price effects. Share of UPCs on promotion in the same store-week. Summarizes the store-wide promotional environment beyond the focal UPC’s own promotion status.
log_price_per_liter
on_promo
promo_intensity_store_week
Table 14: Final dataset schema: temporal, seasonal, and lifecycle features. Variable(s)
Definition and motivation
week_id
Raw calendar week identifier. Retained for ordering, traceability, and downstream joins. Sequential gap-free time index constructed after calendar completion. Captures smooth global time trends. Annual Fourier seasonality terms. Capture smooth yearly demand patterns without high-dimensional week fixed effects. Semiannual Fourier seasonality terms. Represent medium-horizon seasonal cycles beyond annual seasonality. Quarterly Fourier seasonality terms. Capture shorter recurrent seasonal patterns in demand. Weeks since the UPC first appears in the data. Captures global product lifecycle and product maturity. Weeks since the UPC first appears in a given store. Captures local lifecycle effects such as rollout, stabilization, or decline within store.
week_rank sin_52 cos_52 sin_26 cos_26 sin_13 cos_13 weeks_since_first_seen_upc weeks_since_first_seen_store_upc
41
Table 15: Final dataset schema: demand-memory features and missingness indicators. Variable(s)
Definition and motivation
lag_1_log_liters_sold lag_2_log_liters_sold lag_4_log_liters_sold rolling_mean_4_log_liters_sold rolling_mean_13_log_liters_sold miss_lag_1 miss_lag_2 miss_lag_4 miss_roll_4 miss_roll_13
Lagged log demand for the same store-UPC series at 1, 2, and 4 weeks. Captures short- and medium-run persistence in demand. Four- and thirteen-week rolling averages of log demand. Summarize recent and quarterly local demand levels while smoothing week-level noise. Missingness indicators for lagged demand features. Identify cases in which lagged demand information is unavailable. Missingness indicators for rolling demand features. Identify cases in which rolling demand summaries cannot be computed because the required historical window is unavailable.
Table 16: Final dataset schema: dynamic competitive-neighborhood features. Variable(s)
Definition and motivation
n_neighbors_sw_cat
Number of other UPCs observed in the same store-week-category. Measures the size of the active category-level competitive set. Share of category neighbors on promotion in the same store-week. Captures promotional pressure from rival UPCs within the category. Number of same-brand neighbors in the same store-week-category. Measures withincategory same-brand competitive intensity. Share of same-brand neighbors on promotion. Captures promotional pressure among close same-brand substitutes. Mean lag-1 log demand across valid category neighbors. Summarizes recent demand conditions in the local competitive set. Mean lag-1 log demand across valid same-brand neighbors. Captures recent demand conditions among same-brand neighboring UPCs. Mean four-week rolling log demand across valid category neighbors. Provides a smoother measure of recent category-level demand conditions. Missingness indicators for neighbor-based demand aggregates. Identify cases in which neighbor demand information is unavailable.
neighbor_promo_share_sw_cat n_same_brand_neighbors_sw_cat same_brand_neighbor_promo_share_sw_cat lag1_neighbor_mean_log_liters_sold lag1_same_brand_neighbor_mean_log_liters_sol d roll4_neighbor_mean_log_liters_sold miss_lag1_neighbor_mean_log_liters_sold miss_roll4_neighbor_mean_log_liters_sold miss_lag1_same_brand_neighbor_mean_log_liter s_sold
Table 17: Final dataset schema: static assortment and neighbor-entry features. Variable(s)
Definition and motivation
store_category_upc_count_static
Distinct UPC count in the same store-category. Controls for product range and fixed competitive scale at the store-category level. Same-brand UPC count in the same store-category, excluding the focal UPC. Measures within-category same-brand variety and potential same-brand substitution. Count and share of neighbors that are new in-store, defined as UPCs with at most 13 weeks since first appearance in that store. Capture changing competitive sets due to product introductions, rollouts, or activation.
same_brand_upc_count_store_cat_static n_new_neighbors_13w share_new_neighbors_13w
42
R EFERENCES Rodrigo Acuña-Agost, Emmanuel Thomas, and Alix Lhéritier. Price elasticity estimation for deep learningbased choice models: an application to air itinerary choices. Journal of Revenue and Pricing Management, 20(3):213–226, 2021. doi: 10.1057/s41272-021-00308-z. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A nextgeneration hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019. Martin Anthony and Norman Biggs. Mathematics for Economics and Finance: Methods and Modelling. Cambridge University Press, 2 edition, 2024. Evgeny A. Antipov and Elena B. Pokryshevskaya. Interpretable machine learning for demand modeling with high-dimensional data using gradient boosting machines and shapley values. Journal of Revenue and Pricing Management, 19(5):355–364, 2020. doi: 10.1057/s41272-020-00236-4. Akihito Asano. Differential calculus 2, pp. 124–146. Cambridge University Press, 2012. Michelle C. Baddeley and Diana V. Barrowclough. Using logs and estimating elasticities: Demand for air travel, pp. 64–85. Cambridge University Press, 2009. James Banks, Richard Blundell, and Arthur Lewbel. Quadratic engel curves and consumer demand. The Review of Economics and Statistics, 79(4):527–539, 1997. doi: 10.1162/003465397557015. Steven Berry, James Levinsohn, and Ariel Pakes. Automobile prices in market equilibrium. Econometrica, 63(4):841–890, 1995. ISSN 00129682, 14680262. A. Colin Cameron. Analysis of economics data: Chapter 9, models with natural logarithms. Course slides, November 2022. URL https://cameron.econ.ucdavis.edu/aed/traedv1_09. Accessed: 2026-05-10. Yvonne Choquet-Bruhat and Cecile DeWitt-Morette. Analysis, Manifolds and Physics, volume 1. NorthHolland, revised, 2nd edition, September 2004. ISBN 9780444860170. Laurits R. Christensen, Dale W. Jorgenson, and Lawrence J. Lau. Transcendental logarithmic utility functions. The American Economic Review, 65:367–383, 1975. Kenneth W. Clements, Marc Jim M. Mariano, George Verikios, and Berwyn Wong. How elastic is alcohol consumption? Economic Analysis and Policy, 76:568–581, 2022. doi: 10.1016/j.eap.2022.09.003. Angus Deaton and John Muellbauer. An almost ideal demand system. The American Economic Review, 70 (3):312–326, 1980. Asnat Greenstein-Messica and Lior Rokach. Machine learning and operation research based method for promotion optimization of products with no price elasticity history. Electronic Commerce Research and Applications, 40:100914, 2020. ISSN 1567-4223. doi: https://doi.org/10.1016/j.elerap.2019.100914. Kilts Center for Marketing. Dominick’s Dataset. Dataset (store-level scanner data), a. URL https: //www.chicagobooth.edu/research/kilts/research-data/dominicks. Academic research use. Partnership Chicago Booth–Dominick’s Finer Foods (1989–1994); store-level data on sales of 3,500+ UPCs. 43
Kilts Center for Marketing. Dominick’s Data Manual and Codebook. University of Chicago Booth School of Business, b. URL https://www.chicagobooth.edu/-/media/enterprise/centers /kilts/datasets/dominicks-dataset/dominicks-manual-and-codebook_kilt scenter. Created July 2013; updated October 2018. Jon P. Nelson. Estimating the price elasticity of beer: Meta-analysis of data with heterogeneity, dependence, and publication bias. Journal of Health Economics, 33:180–187, 2014. ISSN 0167-6296. doi: https: //doi.org/10.1016/j.jhealeco.2013.11.009. Geoffrey Poitras. Notes on point elasticity. Lecture notes. URL https://www.sfu.ca/˜poitras/4 17_elasticity.pdf. Accessed: 2026-05-10. Christopher J. Ruhm, Alison Snow Jones, Kerry Anne McGeary, William C. Kerr, Joseph V. Terza, Thomas K. Greenfield, and Ravi S. Pandian. What u.s. data should be used to measure the price elasticity of demand for alcohol? Journal of Health Economics, 31(6):851–862, 2012. doi: 10.1016/j.jheale co.2012.08.002. Douglas Schultz, Johannes Stephan, Julian Sieber, Trudie Yeh, Manuel Kunz, Patrick Doupe, and Tim Januschowski. Causal forecasting for pricing, 2024. URL https://arxiv.org/abs/2312.152 82. David Simchi-Levi and Chonghuan Wang. Pricing experimental design: Causal effect, expected revenue and tail risk. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 31788–31799. PMLR, 23–29 Jul 2023. Preety Srivastava, Keith R. McLaren, Michael Wohlgenant, and Xueyan Zhao. Disaggregated econometric estimation of consumer demand response by alcoholic beverage types. Australian Journal of Agricultural and Resource Economics, 59(3):412–432, 2015. doi: https://doi.org/10.1111/1467-8489.12095. Henri Theil. The Information Approach to Demand Analysis, pp. 627–651. Springer Netherlands, Dordrecht, 1992. ISBN 978-94-011-2408-9. doi: 10.1007/978-94-011-2408-9 3. Daniel Toro-González, Jill J. McCluskey, and Ron Mittelhammer. Beer snobs do exist: Estimation of beer demand by type. Journal of Agricultural and Resource Economics, 39(2):174–187, 2014. doi: 10.22004 /ag.econ.186564. Harald J. van Heerde, Peter S. H. Leeflang, and Dick R. Wittink. How promotions work: SCAN*PRO-based evolutionary model building. Schmalenbach Business Review, 54:198–220, 2002. doi: 10.1007/BF0339 6653. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. Dick R Wittink, Michael J Addona, William J Hawkes, and John C Porter. Scan* pro: The estimation, validation and use of promotional effects based on scanner data. Internal Paper, Cornell University, 1988.
44