Graphical-Probabilistic Modeling of Generative Flows in LLM-Native Software Systems Víctor A. Braberman Flavia Bonomo-Braberman [email protected] [email protected] Departamento de Computación, FCEN, Universidad de Buenos Aires / ICC, UBA-CONICET Buenos Aires, Argentina
arXiv:2606.15943v1 [cs.SE] 14 Jun 2026
Abstract Engineering LLM-native software remains a challenging and immature field. Current practice is largely exploratory, relying on experimentation and heuristic techniques such as prompting and context engineering. These, however, are low-level and lack the principled structure needed to support design-level reasoning or analysis. In contrast, traditional software engineering leverages modularity and abstraction to communicate and analyze system behavior. To bring similar rigor to LLM-native development, we propose methods for documenting generative flows and for stating properties of LLM-based software designs. Such methods must account for the stochastic, prompt-dependent behavior of large language models while remaining expressive enough to capture emergent phenomena. Our initial approach is based on graphical probabilistic models, tailored to capture phenomena characteristic of LLM-native systems. This framework—what we term Generation Networks—aims to provide a foundation for principled reasoning about generative interactions and system-level properties in LLMcentric software architectures.
CCS Concepts • Software and its engineering → System description languages; Software verification and validation; Extra-functional properties; • Computing methodologies → Probabilistic reasoning.
Keywords AI-Enabled Systems, LLMs, LLM-native applications, design specification, graphical probabilistic models, Generative Networks
1
Introduction
Engineering software systems that incorporate large language models (LLMs) remains a difficult and largely ad-hoc activity [20]. Development practices rely on heuristics—prompt engineering, context tuning, and workflow scripting—that work in isolation but provide little support for system-level reasoning or design documentation. The absence of abstraction mechanisms comparable to those in traditional software engineering hampers analysis, maintenance, and reuse. Recent conceptual work has been proposed viewing LLM-enabled applications—systems where LLMs perform essential computational roles—as collections of LLM-mechanized, potentially interleaved with external tool calls, orchestrated through arbitrary control flow (e.g., transformations [1, 5, 16]).
This paper extends that line of thought showing how probabilistic modeling can describe generative software designs. More concretely, we introduce graphical models tailored to LLM-based systems, to document data flows and express properties of interest. The examples highlight how conceptual variables, distributional parameters, and dependency structures can encode notions of correctness, robustness, and design improvement. Our aim is to show how these modeling constructs provide an engineering vocabulary for articulating the structure and behavior of LLM-enabled systems. By integrating software-engineering abstractions with the formalism of probabilistic modeling [17], the proposed approach—termed Generation Networks—offers a principled framework for communicating, documenting, and reasoning about architectures whose behavior emerges from stochastic, prompt-sensitive generative mechanisms. The paper is structured as follows. Section 2 reviews related work, and Section 3 shows the basics of Generative Networks and illustrates its use in documentation. Section 4 introduces a language for specifying design properties. Section 5 presents use cases that combine these constructs. Finally, Section 6 outlines future research directions, and Section 7 concludes by discussing the implications for LLM-native software development.
2
Related Work
Graphical probabilistic models [17] have been argued to represent neural inference at different levels of abstractions and for different communication goals. Hypothesis regarding internal mechanisms: Causal models, causal abstraction, and causal graphs has been applied to interpretability of neural networks activation [12]. There mechanisms underlying network inference for one requested task are hypothesized by using causal graphs. Hypothesis regarding prompt-code relationship: In [14] it is proposed a causal graph-based representation of the prompt and the generated code, which is established over the fine-grained, humanunderstandable concepts in the input prompts used specifically for code generation. Uniform blueprint of existing techniques: In [11] authors argue that existing LLM techniques like scratchpads [24], CoT [28], verifiers [9], STaR [31], selection-inference and tool use [10], can be expressed as particular probabilistic programs (cascades), and therefore represented in the language of graphical models with random variables whose values are complex data types such as strings.
,,
Víctor A. Braberman and Flavia Bonomo-Braberman
Current Incident Information
Historical Incidents VDB
Summarization
Summary
Relevant Summarized Incidents
Reactive::Root Cause ::Behavior Analysis
Thought(0)
Inc.ID × Question ⊕ Culprit (0)
Thought(1)
Similarity Retrieval
Detailed Incident (0)
Key-based Retrieval
Answer(0)
Open Q&A:: Text Analysis
Inc.ID × Question ⊕ Culprit (1)
Figure 1: An abstract execution of a hypothetical RCA tool modeled using transformation types as categorized by [5].
Uniform blueprint of LLM-native Software Designs: In [5], a systematic analysis of over one hundred LLM-based solutions from the software engineering literature was conducted to characterize the abstract functionalities assigned to LLMs. This was achieved by analyzing concrete prompt-based interactions to recover the underlying intent of each task (a.k.a, transformation). While that study primarily focused on identifying a typology of transformations and recurring composition motifs, it proposed—as a future research direction—the unification of LLM-native system descriptions through data-dependency graphs. Under this abstraction, the proposed “world model” provides a schematic representation of the system’s behavior in terms of its input, internal, and generated data variables. This perspective serves as the core inspiration for the approach adopted in this paper to represent and analyze LLM-integrated architectures.
3
Design Representation via Data-Dependency Graphs
A primary goal of this proposal is to communicate the central design decisions of a system—whether currently operational or under specification—by modeling some representative executions as Directed Acyclic Data-Dependency Graphs (DDGs). In these graphs, nodes denote conceptually typed data instances—comprising input, output, and internal variables—while edges reify the data transformations [5] that connect them. A transformation maps a set of parent variables values to a target child variable value. In LLMnative software, these mechanisms can be categorized into two primary types: (1) LLM-based transformations: These stochastic transformations abstract prompt-modulated interaction where the LLM produces the content of the target variable. Those prompts
typically would include instructions, examples, and stringlike representations of the parent variables values. The conditional probability distributions within these transformations (prompt continuations) are ultimately defined implicitly by the opaque process of token generation1 . (2) Algorithmic transformations: Typically deterministic processes, such as data projection, composition, or similaritybased retrieval from vector databases. Example. Figure 1 illustrates an abstract execution scenario of a hypothetical LLM-based Root Cause Analysis (RCA) tool inspired by [8]. This tool serves as a representative Retrieval-Augmented Generation (RAG) [19] agent designed for IT Incident Response. The system first summarizes the current report and log, retrieves similar historical cases via algorithmic similarity-search to establish context, and then iteratively decides whether to query a specific historical incident or identify the “Culprit” by following a ReActstyle reasoning–action loop [30] with tool-mediated actions. The representative execution depicted here is an execution that reaches a culprit after requesting and checking further details regarding what the LLM considers to be the closest registered incident. In this scenario, Current Incident Information serves as an input variable, while nodes such as Summary, Thought(i), and Answer(i) represent variables produced through transformations2 . Generative transformations—labeled according to types such as Summarization or Behavior Analysis [5]—yield stochastic content through LLM-mediated processes. Conversely, algorithmic transformations (e.g., similarity retrieval) produce deterministic content, denoted by double-circled nodes such as Detailed Incident (0) and Relevant Summarized Incidents. Consider the variable (Inc.ID × Question) ⊕ Culprit (1). Its realization depends on the initial incident report, the retrieved summaries, and the history of generated thoughts and obtained answers. Its distribution is governed by the Behavior Analysis transformation, which abstracts a prompt-based interaction instantiated with the values of these parent variables. Crucially, as an LLM-based transformation, it involves the autoregressive generation of Thought(1)—the verbalized Chain of Thought [28]—which also conditions the subsequent probability of the utterances for the final action. Thus, the network clarifies how the system leverages iteratively generative capabilities to reach a diagnostic conclusion, defining the precise generative contexts even without exhibiting concrete prompting details. From DDGs to BNs. The mapping from a DDG to a formal probabilistic model is firstly grounded in the nature of system execution. Given the intrinsic stochasticity of input and LLM-based transformations, each node in the DDG—comprising input, internal, and output data—can be formally regarded as a random variable. Moreover, for any fixed DDG, there exists a set of execution traces (i.e., sets of random variables values) consistent with its specific topology: the graph structure itself often encodes termination criteria or specific logical paths (e.g., a bounded unfolding of an iterative structure that actually concludes when culprit is yielded). 1 This is the emerging result of the specific network architecture [27], its fixed weights,
the effects of chosen generative/sampling adapters, and hyperparameters [22]. 2 As execution involves a trajectory of values for the same conceptual variable, names are postfixed with an instance number to identify each realization within the trace.
Graphical-Probabilistic Modeling of Generative Flows in LLM-Native Software Systems
,, Intent-labeled benchmark
This allows for the derivation of a corresponding Bayesian Network (BN) [17] to represent the distribution’s conditional independence structure for data compatible with the given DDG. In this framework, the variable dependencies define the causal relations [25] among variables.
Generative process sampling
Code + (Intent)0...1
SuS
4
Modeling Design Properties Golden intent
In addition to conveying structural task decomposition, we extend the representational and denotational scope of our notation, Generation Networks, to capture more nuanced design-level properties. This expansion is specifically tailored to support the rigorous documentation and analysis of LLM-based software architectures, moving beyond simple workflow visualization to a formal modeling of the interaction between deterministic logic and stochastic generation. Distribution-control and domain selector variables. In previous examples, details such as the specific model architecture, weights, or prompt instructions were left implicit within the underlying conditional distribution governing stochastic transformations. In some situations, however, it is useful to expose these factors explicitly—e.g., (typically latent) model parameters 𝜃 or controllable variables such as temperature, hyperparameters, or fixed demonstrations and instructions. Likewise, it can be convenient to introduce domain selector variables that modulate the distribution of exogenous variables, capturing which acquisition or data-source regime is being considered (e.g., benchmark, synthetic, or in-the-wild [7]). Such variables enable reasoning about acquisition shifts: situations where transformations must operate on data drawn from different representational or contextual distributions while addressing the same conceptual problem. For instance, when modeling a defect-detection transformation, one may distinguish between abstract categories of a defect (e.g., a class of memory-bound violations) and their concrete syntactic realizations across programming languages or styles. This distinction allows designers to articulate expected behavior under distributional variation, analyze robustness to representational changes, and document assumptions about training versus deployment conditions. Accordingly, we extend the notation with elements of metanetwork modeling [17], allowing selected distribution-control variables—whether affecting exogenous or generated variables—to be represented explicitly when relevant. This representation clarifies the use cases introduced later and generalizes naturally to modules implemented by other neural models beyond LLMs. On the Modeled World. Complementing the representational extensions above, we further expand the denotational scope of the framework. Beyond variables that denote observable inputs or data generated during execution, Generation Networks allow the inclusion of conceptual or latent quantities required to express asserted or expected system properties. For example, a diagram may introduce variables representing the (typically unobservable) groundtruth or aligned outputs of transformations, as well as virtual intermediate results—such as Boolean indicators capturing whether an outcome aligns with its expected value. These variables enable the specification of evaluative properties, including expected success
Code to summarize
Intent
≃
≃ Judgement
CodeCorresponding-Intent-Verbalization
OOD
OOD Detector for Github like comments
Figure 2: Prescription Use Case
rates, compositionality relations, and robustness conditions under given assumptions. Specifying Quantities. A key extension of Generation Networks is the ability to express first-order formulas defined over functional symbols that denote quantities derived from the joint probability distribution of the modeled variables. These formulas build on two classical operations from causal modeling [17]: probabilistic queries and intervention (causal) queries. Through them, Generation Networks can state relations between probabilistic quantities, enabling the formal expression of prescriptions, assertions, and search conditions based on the system’s joint distribution. A probabilistic query represents the posterior probability that variables 𝑌 take specific values given evidence 𝐸 = 𝑒: 𝑃 𝑁 (𝑌 | 𝐸 = 𝑒) An intervention query expresses the posterior distribution after enforcing a manipulation on certain variables, capturing causal reasoning in Pearl’s sense: 𝑃 𝑁𝑍 =𝑧 (𝑌 | 𝐸 = 𝑥) Thus, together, these specification-level constructs express design desiderata: descriptive, predictive, and prescriptive statements about LLM-native system designs modeled through Generation Networks. As said, these functionals denoting distributions can be used to express point-wise comparisons or divergence comparisons (e.g., KL [18] or Wasserstein Distance [2]) between those probabilistic quantities.
5
Use Cases
We now explore some potential uses of the modeling framework (some of them speculative) by combining the modeling features mentioned above. These use cases schematically illustrate framework’s expressive power and how it enables a uniform and principled way to treat a wide variety of design communication/inquiring operations. Due to a lack of space, we choose to illustrate some subset of uses.
,,
Víctor A. Braberman and Flavia Bonomo-Braberman benchmark regime
SuS Code + line
Code + Intent
SuS
CoverageTargetedInput-Generation (Td )
Code to summarize
S
CodeCorresponding-Intent-Verbalization
Actual params YTd
Input charact.
CodeReachabilityInput-Characterization (T1 )
C(YTd )
Actual params YT2 ◦T1
ConstraintSatisfyingInput-Generation (T2 )
Golden intent
Intent
C(YT2 ◦T1 )
≃
Figure 3: Exploration Use Case
5.1
Bimp
BNoRegr
BNoRegr :=
Prescribing Transformation Behavior
C(Td ) = True ⇒ C(YT2 ◦T1 ) = True
Bimp := C(YTd ) = False
∧ C(YT2 ◦T1 ) = True
Figure 2 illustrates how probabilistic prescriptions can formalize Figure 4: Improvement Use Case expected behavior of a transformation, in this case, the CodeCorre0...1 sponding-Intent-Verbalization task. The exogenous variable Code+Intent supplies a code snippet paired, optionally, with a reference (“golden”) a code snippet and its corresponding golden intent are available intent description. This input is deterministically projected into under a benchmark regime. We introduce a variable 𝑆 denoting these two elements. Generated random variables include: (i) an the set of few-shot examples [6] included in the prompt—along 𝐼𝑛𝑡𝑒𝑛𝑡 variable representing the verbalized intent produced by the with the current code snippet—to guide the model’s generation of transformation under prescription, and (ii) a Boolean variable ≃ the predicted Intent. Again, ≃ is a similarity judgment between the capturing the outcome of an LLM-based correctness judgment (true result and the golden intent. In this use case, the designer requests if the generated intent matches the golden reference). Additionoptimization with respect to available benchmark data3 . In our ally, an 𝑂𝑂𝐷 variable encodes the output of an out-of-distribution framework, exploring alternative few-shot configurations corredetection mechanism (e.g. [26, 29]), indicating whether the genersponds to performing interventions on 𝑆, yielding post-intervention ated intent resembles text sampled from an expected domain (e.g., distributions 𝑃 𝑁𝑆=𝑠 (≃) and the search for an optimal configuration intents from GitHub projects). A categorical domain selector, 𝐷, can be formalized as a model-based optimization problem: distinguishes between two regimes: (1) a benchmark regime, where 𝑠 ★ = arg max 𝑃 𝑁𝑆=𝑠 (≃= 𝑇𝑟𝑢𝑒). golden intents are available, and (2) a generative regime, where code 𝑠∈S snippets are sampled from a learned generative process. For each In this form, Generation Networks treat prompt configuration regime, we specify probabilistic prescriptions reflecting minimal and similar hyperparameters as interventional variables, enabling performance criteria: declarative specification of search, optimization, or tuning tasks [𝑃 (≃= True | 𝐷 = benchmark) ≥ 1 − 𝜀] [𝑃 (𝑂𝑂𝐷 = True | 𝐷 = generative) ≤ 𝛿] The first ensures near-correct behavior on benchmark data; the second enforces that generated intents remain within an acceptable distributional range (i.e., resemble legitimate comment-like text). These prescriptions exemplify how the proposed representation can encode behavioral expectations as constraints on conditional probabilities over transformation variables.
5.2
Requesting the Exploration of Mechanism Parameters
Having introduced probabilistic prescriptions that express expected behavior of a transformation, we now turn to the complementary question of how such behavior can be modulated through explicit mechanism parameters. Figure 3 illustrates how parameters influencing the behavior of neural-based transformations can be made explicit within the representational model. Consider again the CodeCorresponding-Intent-Verbalization transformation, where
within the same probabilistic framework used for descriptive and causal reasoning. In fact, conceptual prompt evaluation and calibration is another related use case that could be approached within our framework, similarly to what [14] does for code generation. Beyond prompt-level optimization, our formalism supports exploratory what-if analyses for design reasoning. For example, robustness to systematic component degradation or failures can be investigated by intervening on selected nodes of a design-level network, thus simulating alternative operational scenarios within the same representational language.
5.3
Asserting Design Improvement
Generation Networks can also express design improvements in realistic transformation pipelines through deterministic Boolean variables that depend on observable performance outcomes. Figure 4 sketches this use case. Consider a transformation (𝑇𝑑 ) performing CoverageTargeted-Input-Generation: given a parameterized code snippet and a target line, it attempts—within a single inference—to 3 This is akin to declarative specification frameworks such as DSPy [16].
Graphical-Probabilistic Modeling of Generative Flows in LLM-Native Software Systems
,,
produce an input that triggers execution of that line. A decomposed design instead realizes the same goal through two sequential transformations: (𝑇1 ), CodeReachability-Input-Characterization, which verbalizes a natural-language description of inputs expected to reach the target line; and (𝑇2 ), ConstraintSatisfying-Input-Generation, which generates a concrete input consistent with that description. For both designs, goal achievement is deterministically signaled by checking whether the target line is actually covered during execution—no golden reference is required. Let 𝑌𝑇𝑑 and 𝑌𝑇2 ◦𝑇1 denote the respective outputs (generated inputs) for the same code–line pair, and let a deterministic variable 𝐶 (·) indicate whether the resulting input achieves coverage. Then a Boolean variable capturing relative correctness between the two designs can be defined as: [𝐵 NoRegr := 𝐶 (𝑌𝑇𝑑 ) = True ⇒ 𝐶 (𝑌𝑇1 ◦𝑇2 ) = True ]
Advanced Use Cases: Compositional Analysis. A particularly promising direction concerns the use of Generation Networks to articulate verification desiderata involving distributional or representational compatibility between chained transformations. In this view, one could analyze whether the output characteristics of a transformation align well with predefined input (sub)domain manifold representations under which a subsequent transformation has been extensively verified—an attempt to capture forms of relative compositionality, non-regressive evolution of composed systems and hierarchical verification. Ideas from DLL testing based on latent spaces like [15, 23]4 in the setting of OOD for text (e.g. [29]) could be inspiring to define such mechanisms thus enable documenting and reasoning about how training, deployment, and interconnection assumptions affect overall system robustness and compatibility.
which is true whenever success of the direct design implies success of the decomposed one for the same instance. The marginal probability
This work introduces Generation Networks (GNs), a graphical probabilistic language for documenting and reasoning about LLM-native software architectures. Rather than defining a new inference mechanism, the contribution lies in a structured notation that captures how algorithmic and LLM-based transformations interact within generative workflows. By integrating probabilistic and causal notions, GNs make it possible to express stochastic dependencies and design-level properties. The examples illustrate how higherorder relations among outcomes can be encoded, enabling reasoning about alternative designs, regressions, and performance gains within a unified probabilistic setting. Looking ahead, GNs can serve as a foundation for design-level analysis and tool support in the emerging field of LLM-native software engineering. In this sense, the approach aspires to connect the abstraction principles of software engineering with the analytical rigor and toolsets of probabilistic graphical modeling—toward a principled, model-based engineering discipline for generative and learning-based systems.
[𝑃 𝑁 (𝐵 NoRegr = True) ≥ 1 − 𝜀] thus quantifies how frequently the two-step design performs at least as well as the one-step composition within the modeled domain—an “almost-sure” (minus 𝜀) notion of design improvement [3]. High probability values indicate that regressions (cases where the direct approach succeeds but the composed one fails) are rare in the selected domain. A complementary variable, [𝐵 imp := 𝐶 (𝑌𝑇𝑑 ) = False ∧ 𝐶 (𝑌𝑇2 ◦𝑇1 ) = True ] captures instances where the new design achieves coverage unattained by the original one. High values of 𝑃 𝑁 (𝐵 imp = True) therefore indicate measurable improvement in end-to-end capability, expressed within the same probabilistic representation that encodes behavioral prescriptions and causal relations among transformation components.
6
Future Work
Reasoning Backbone. The proposed language is primarily intended for communication and documentation rather than automated inference or verification. Nevertheless, its probabilistic foundations suggest natural paths toward analytical integration. Methods from probabilistic graphical modeling [17] and probabilistic programming [13] could support empirical or symbolic reasoning once exogenous variables and transformation implementations are specified. For example, if inputs were generated through controlled sampling (e.g., from a benchmark suite), forward sampling could enable approximate inference over queries defined within a Generation Network. Modeling Extensions. Future extensions will target the modeling of complex generation trajectories where the number of variables and interaction depth are not fixed a priori. We also plan to explore causal abstraction [4] and classical refinement [21] to enable hierarchical and modular representations, scaling Generation Networks from local patterns to full architectural designs.
7
Conclusions
Acknowledgments Amazon Research Award – Fall 2023 on Automated Reasoning; UBACyT 20020220300079BA and 20020190100126BA.
References [1] Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Daniel Klein, Matei Zaharia, and Omar Khattab. 2025. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457 [cs.CL] [2] Martín Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 214–223. http://proceedings.mlr.press/v70/arjovsky17a.html [3] Juan Manuel Baldonado, Flavia Bonomo-Braberman, and Víctor A. Braberman. 2025. Towards a Probabilistic Framework for Analyzing and Improving LLMEnabled Software. In IEEE International Conference on Software Testing, Verification and Validation, ICST 2025 - Workshops, Naples, Italy, March 31 - April 4, 2025. IEEE, 418–422. doi:10.1109/ICSTW64639.2025.10962470 [4] Sander Beckers and Joseph Y. Halpern. 2019. Abstracting Causal Models. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth 4 Note that variables denoting elements of latent spaces, encoding and decoding trans-
formations as required for a VAE-like ID detection or generation can be already naturally modeled in GN.
,,
AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. AAAI Press, 2678–2685. doi:10.1609/AAAI.V33I01.33012678 [5] Víctor A. Braberman, Flavia Bonomo-Braberman, Yiannis Charalambous, Juan G. Colonna, Lucas C. Cordeiro, and Rosiane de Freitas. 2024. Tasks People Prompt: A Taxonomy of LLM Downstream Tasks in Software Verification and Falsification Approaches. arXiv:2404.09384 [cs.SE] [6] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https://proceedings.neurips.cc/paper/2020/ hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html [7] Daniel C. Castro, Ian Walker, and Ben Glocker. 2020. Causality matters in medical imaging. Nat. Commun. 11, 3673 (2020). doi:10.1038/s41467-020-17478-w [8] Yinfang Chen, Huaibing Xie, Minghua Ma, Yu Kang, Xin Gao, Liu Shi, Yunjie Cao, Xuedong Gao, Hao Fan, Ming Wen, Jun Zeng, Supriyo Ghosh, Xuchao Zhang, Chaoyun Zhang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Tianyin Xu. 2024. Automatic Root Cause Analysis via Large Language Models for Cloud Incidents. In Proceedings of the Nineteenth European Conference on Computer Systems, EuroSys 2024, Athens, Greece, April 22-25, 2024. ACM, 674–688. doi:10.1145/3627703.3629553 [9] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems. arXiv:2110.14168 [cs.LG] https://arxiv.org/abs/2110.14168 [10] Antonia Creswell, Murray Shanahan, and Irina Higgins. 2023. Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. https://openreview.net/forum?id= 3Pf3Wg6o-A4 [11] David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-dickstein, Kevin Murphy, and Charles Sutton. 2022. Language Model Cascades. arXiv:2207.10342 [cs.CL] Presented as spotlight at the Beyond Bases workshop at ICML 2022. [12] Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. 2021. Causal Abstractions of Neural Networks. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.). 9574–9586. https://proceedings.neurips.cc/paper/2021/hash/ 4f5c422f4d49a5a807eda27434231040-Abstract.html [13] Andrew D. Gordon, Thomas A. Henzinger, Aditya V. Nori, and Sriram K. Rajamani. 2014. Probabilistic programming. In Future of Software Engineering, FOSE 2014, Hyderabad, India, May 31 - June 7, 2014, James D. Herbsleb and Matthew B. Dwyer (Eds.). ACM, 167–181. doi:10.1145/2593882.2593900 [14] Zhenlan Ji, Pingchuan Ma, Zongjie Li, Zhaoyu Wang, and Shuai Wang. 2025. Causality-Aided Evaluation and Explanation of Large Language Model-Based Code Generation. Proc. ACM Softw. Eng. 2, ISSTA (2025), 1374–1397. doi:10.1145/ 3728938 [15] Sungmin Kang, Robert Feldt, and Shin Yoo. 2020. SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation. In ICSE ’20: 42nd International Conference on Software Engineering, Workshops, Seoul, Republic of Korea, 27 June - 19 July, 2020. ACM, 521–528. doi:10.1145/3387940.3391456 [16] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2024. DSPy: Compiling Declarative Language Model Calls into State-of-the-Art Pipelines. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. https://openreview.net/forum?id= sY5N0zY5Od [17] Daphne Koller and Nir Friedman. 2009. Probabilistic Graphical Models - Principles and Techniques. MIT Press. http://mitpress.mit.edu/catalog/item/default.asp? ttype=2&tid=11886 [18] Solomon Kullback and Richard A. Leibler. 1951. On information and sufficiency. The Annals of Mathematical Statistics 22, 1 (1951), 79–86. doi:10.1214/aoms/ 1177729694 [19] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the 34th International Conference
Víctor A. Braberman and Flavia Bonomo-Braberman
on Neural Information Processing Systems (NIPS ’20). Article 793, 16 pages. [20] Jenny T. Liang, Melissa Lin, Nikitha Rao, and Brad A. Myers. 2025. Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts. Proc. ACM Softw. Eng. 2, FSE (2025), 1591–1614. doi:10.1145/3729342 [21] B. Liskov and J. Guttag. 1986. Abstraction and Specification in Program Development. MIT Press, Cambridge, MA. [22] Clara Meister, Gian Wiher, and Ryan Cotterell. 2022. On Decoding Strategies for Neural Text Generators. Trans. Assoc. Comput. Linguistics 10 (2022), 997–1012. doi:10.1162/TACL_A_00502 [23] Nusrat Jahan Mozumder, Felipe Toledo, Swaroopa Dola, and Matthew B. Dwyer. 2025. RBT4DNN: Requirements-based Testing of Neural Networks. arXiv:2504.02737 [cs.SE] https://arxiv.org/abs/2504.02737 [24] Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2021. Show Your Work: Scratchpads for Intermediate Computation with Language Models. arXiv:2112.00114 [cs.LG] [25] Judea Pearl. 2009. Causality (2 ed.). Cambridge University Press. [26] Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter J. Liu. 2023. Out-of-Distribution Detection and Selective Generation for Conditional Language Models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. https://openreview.net/forum?id=kJUS5nD0vPB [27] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 49, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998–6008. https://proceedings.neurips.cc/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract.html [28] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). http://papers.nips.cc/paper_files/paper/2022/hash/ 9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html [29] Qianhui Wu, Huiqiang Jiang, Haonan Yin, Börje F. Karlsson, and Chin-Yew Lin. 2023. Multi-Level Knowledge Distillation for Out-of-Distribution Detection in Text. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Canada, 7317–7332. doi:10.18653/v1/2023.acl-long.403 [30] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. https://openreview. net/pdf?id=WE_vluYUL-X [31] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. STaR: Bootstrapping Reasoning With Reasoning. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). http://papers.nips.cc/paper_files/paper/2022/hash/ 639a9a172c044fbb64175b5fad42e9a5-Abstract-Conference.html