C ONTEXT OPERATIONS TO ARCHITECTURE MODELLING OUTPUT FROM LARGE LANGUAGE MODELS AND EVALUATION CRITERIA FOR THEIR USE IN SYSTEMS ENGINEERING DESIGN
arXiv:2609.10132v1 [eess.SY] 9 Sep 2026
A P REPRINT Vinicius Kaster Marini∗ Department of Mechanical Engineering Centre of Technology Federal University of Santa Maria Santa Maria/RS, BR 97105-900, Brazil [email protected]
Petter Krus Section for Fluid Power and Mechatronics Department of Management and Engineering Linköping University SE 581 83, Linköping, Sweden [email protected]
September 10, 2026
A BSTRACT The development of generative artificial intelligence resources enables opportunities of speeding up systems and engineering design work. This contribution introduces a framework of formal operations for assembling context in LLM-based engineering design. This framework involves the assembly of modular context units, including policy prompts, reference units with persistence, and user questions with prompt vectoring. This approach enables the systematic structuring of interactions with generative models. A formal method for evaluating modelling-as-code LLM outputs is also presented, which enables the evaluation of compliance to intent from LLM answers and thereby asses the support from LLMs for systems architecture modelling. Keywords Large language models · Systems engineering · Context operations · Prompt engineering · Generative AI
1
Introduction
The complexity of automation in mobility systems draws to human-machine interfaces in operations as a matter of concern for systems engineering (Cummings [2004]). The implementation of artificial intelligence (AI) to augment control in novel vehicles has implications that extend back to the design process. The realization of gains from AI-enabled and connected smart devices entails added complexity and intricacy in the development of system designs as multi-domain system stacks (Törngren and Grogan [2018], Grogan [2021]). This contribution aims to demonstrate that human-machine collaboration through the system design process can be designed in a way that leverages generative artificial intelligence (GenAI) capabilities. Among many approaches of AI, tools with these capabilities make use of large language models (LLMs) (Brown et al. [2020], Vaswani et al. [2017]), which are pre-trained over a very large corpus of data over the internet to yield conversational abilities in answering questions. These tools have attracted the attention of system and design engineers (Krus [2024], Johns et al. [2024]) for the potential support to early design, a context where other design automation techniques – including those based on machine learning as well as LLMs – fell short. However, their use in engineering and design is subject to challenges regarding their probabilistic approach to content (Teubner et al. [2023]). While synthesis of assurance arguments makes a potential use case for LLMs upon the amount of paperwork involved, a research report by NASA analyses early explorations with LLMs and highlights their lack of matter-of-factness (Graydon and Lehman [2025]). Hence, the use of LLM-based tools in engineering and design requires careful review of the LLM outcomes towards design work products (Pradas-Gomez et al. [2024]). ∗
Adjunct professor at the Department of Mechanical Engineering, Centre of Technology, Federal University of Santa Maria, Brazil. You are welcome to contact me through the email above, or you can look for my profiles in LinkedIn and ResearchGate.
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Then, how to improve the accuracy of GenAI to design intent towards system architecture modelling? This contribution introduces the theme of LLMs in systems design and engineering within the use case of system architecture modelling, with developing from awareness to the state-of-the-art in LLMs (Marini et al. [2025]) towards the systematization of context input for that purpose. Related work involves the following use cases: assistance to systems modelling; assistance in system tools; example&rule assistance; complementary knowledge; assisted safety-driven methods; assisted reliability&safety methods; exploration of dependencies; and HMI for design assistance. **Assistance to systems modelling**: Explorations on using LLMs with systems models demonstrate the early use of developing language models along system modelling frameworks. Cámara et al. [2023] experiment with prompting at the language models with focus on a single system modelling task, supported by model templates intended to provide exemplars. The integration of modelling frameworks through exemplars helps at extracting useful information to proceed a with a significant part of model-building. **Assistance in system tools**: LLMs with chatbox tools can work within model-based systems engineering (MBSE) environments such as reported by DeHart [2024] and Johns et al. [2024], where the use of LLMs benefits from capturing the modelling framework in the MBSE environment and thereby enables the generation of system models from concept to architecture. This is also the focus in Timperley et al. [2025], who enable prompting at coding frameworks interface between the LLM tool and the modelling environment. Here, they add a design element ontology which enables their solution to provide significant support over the synthesis of design specifications. **Example&rule assistance**: Krus [2024] explores the use of LLMs within aircraft concept design with support of structured templates and domain-specific rules to generate system configurations onto prompting the LLM to compose and generate system models with considering these inputs. This approach evolves from early contributions by using a preliminary domain question refined to a systematic prompt with topic structure, aiming to convey design intent and composition rules on objects and their mutual relations in the intended model. **Complementary knowledge**: Balu et al. [2025] prompt LLMs to generate safety requirements; as they recognize the limitations by LLMs within their own pre-training, they elicit the aid of databases for the language model. They sample LLM responses at safety-focused prompts and figure the performance of agent-based RAG to generate better accurate responses against design intent. This is also the case with Hanke et al. [2025], who make use of RAG database support towards parsing unstructured content in design repositories towards structured content that can be leveraged onto system models. **Safety-driven methods**: The potential of LLMs on the generative synthesis of design information has not gone unnoticed by the safety and reliability community. Nouri et al. [2024] make use of LLMs to generate safety requirements for automotive applications, with proposing a pipeline of prompts designed to automate a hazard analysis and risk assessment (HARA) procedure. Another approach is proposed by El Hassani et al. [2024, 2025], who focus the processing of relationships in failure modes and effects analysis (FMEA) process with support from product-related data, through crafted prompts, RAG and model fine-tuning. **Assistance to reliability** Qi et al. [2025] make use of LLMs in an elaborate approach to performing systemstheoretical process analysis (STPA) by experimenting with prompt compositions and communication patterns between engineers and LLMs with different degrees of automation including stepwise review. The use of meta-structures to be supported by LLMs, is a characteristic in the application by Chen et al. [2025], where trustworthiness derivation trees (TDT) convey hierarchical dependencies between safety claims – generated by LLMs and curated of purpose-designed user interface – help the synthesis of assurance cases. **Exploration of dependencies**: Other approach for using LLMs in systems design and engineering is the exploration of dependencies within process models. Lipizzi [2025] looks to capture dependencies between information concepts through the synthesis of triplets and their vectoring to explore the design space and synthesize it into sentence-based network graphs. Another way to look at information dependencies examines the use of LLMs to generating design structure matrices (DSMs) representing the design space through connections between design objects (Koh [2025]). **HMI for design assistance**: Counter to the perception that the designer is to be automated out of the process, Marini et al. [2025] used concept maps to be parsed/splitted to enable sequenced prompt chains on mission design information for aircraft design. Krus [2025] explores ways of working with LLMs in different modes of operation: firstly, the direct generation of models from prompting; then, aircraft architecture models generated through on-the-fly generated application code that instantiates LLM-generated modules; then, through the embedding of LLM API calls in systems design applications. These use case propositions demonstrate the diversity of situations where LLM-supported GenAI tools could help systems design. At the same time, this contribution aims to further develop knowledge and practice on their use by providing an a formal structured approach to LLM operation, and an example on how it works.
2
Context operations with LLMs and evaluation criteria for systems engineering
2
A P REPRINT
Overview of LLM-based engineering approaches
The use cases presented in the introduction establish the field of operation towards positioning this contribution is positioned to support systems engineering and design. Figure 1 shows a morphological matrix characterizing the use cases (rows) and the workflow mechanisms (columns) of the approaches identified in the literature. The field overview from Figure 1 displays use cases that demonstrate the performance of LLMs in delivering outcomes within intended semantic and grammar approaches, often alongside sentence-based outputs. Here, we can see different characteristics of the use cases. 2.1
Use case design
**Intent**: The intent of using the LLM in each use case regards the proposition of the outcome towards the engineering design process. Design *requirements* state properties that the technical system under development shall meet or comply with over its lifecycle. System *architecture* assembles system characteristics/elements and their relations onto models conveying properties of the system. *X-ability* definitions regard the systematic processing of requirements and architecture to identify actual performance attributes of the technical system. **Context**: The activity intent involves context in which it addresses product characteristics and missions within various applications. The use of LLMs in engineering design can address *Product* development, when the focus of the approach is to develop a physical system that will be produced for use as part of a given operating context; or, it can address *Mission* design, when the focus is to develop the actual operation with physical systems being used within it, which means any physical system will be custom-produced for use within its context. **Meta-model**: The semantic meta-structure expressing the context information is seen to provide LLMs a route to achieving the intent towards certain engineering context. The formalization of design content for delegating knowledge processing to LLMs works through *Modelling language* related to the grammar and *Procedure model* related to the design representation. Then, *Boilerplating* uses clause/sentence structures to help automate requirement semantics and *Concept maps* provide visual representation of how elements and relationships are arranged. 2.2
Context workloading
**Method**: Because LLMs are language tools based on large-scale datasets, the approach to requesting the knowledge processing task bears significant influence on its output. The primary method in use by LLM approaches is by standard *Instructions to chatbox*, leveraging various prompting techniques in single questions or in conversations comprising a sequence of questions. There is the possibility of prompting with *Code to chatbox* as main conveyor of context in association to short requests. Then, the use of *Custom application* involves a purpose-specific application where LLM prompts are embedded within its workings and actions are performed by API calls. **Exemplar**: The use of exemplars is widely recognized as supportive to provide LLMs a better context in regards to the objectives of compliance with processing intent. Users can leverage the methods with exemplars. While *Zero-shot* involves no exemplars besides the core question, *One-shot* and *Few-shot* can involve one or more exemplars, respectively, which convey relevant grammar, semantics, and situations that are relevant to the query. The use of *RAG* also helps guiding LLM responses to make answers that are closer to query intent, by having the queried LLM to draw on a domain-specific vector database. **Prompt**: This regards how the demonstrated use cases frame user input to LLMs. *Instruction*-based prompts from the query provide general guidance to how the LLM shall process language to yield its output. Then, *Formulae* and *Rules* offer more structured control - through the relations between elements are still probabilistic - to get the LLM to yield in compliance to certain occurrence relationship from its training data. There are use cases with *Templates*, which show to be useful in providing detailed guidance about expected output formats and characteristics, and *Chained* instructions allow for sequenced responses from the buildup of context. 2.3
Use implementation
**Tool/environment**: These provide means for the user to interface with requesting information from the LLM, and understanding the outcome of its answer. Most use cases rely on *Chatbox* outputs which can be standalone developer-issued tools, or can interoperate with application environments such as *MBSE tools*. While standalone or *Standard* chatboxes provide incomplete and semi-compliant results, the *Custom* integration within applications through API-based routines helps with modelling the input contexts with favourable result to LLM outcomes. 3
Context operations with LLMs and evaluation criteria for systems engineering
Figure 1: Field of operation and positioning of this contribution
4
A P REPRINT
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
**Output**: The type of output determines the work product delivered by the LLM query workflow. Most use cases engage onto the generation of *Models*, which require an underlying grammar the LLM will attend to under its processing; this is possible because developers usually train their LLMs with including language grammars with a large variety of examples. Then, *Sentences*, *Tables*, and *Spreadsheets* involve direct application of natural or coded languages to their processing.
3
Degraded attention and mitigating factors
Hence, the use of LLMs depends on adding task-related to obtain better work product quality. However, the transformer attention mechanism (Vaswani et al. [2017]) cannot equally attend to all tokens in a context workload. This means that forcing the LLM to process multiple sources through long sequences may drive the attention mechanism to lose focus (Liu et al. [2024]), thereby reducing the effectiveness in yielding a sufficiently accurate response to the main query. The lost in the middle problem is in display by Figure 2 regarding the context workload length.
Figure 2: Reliance upon model pre-training for LLM performance. Liu et al. [2024] verify the issue becomes more significant in proportion to the length of the assembled context CQ = {tk1 ..tkn }, considering the actual context capacity of the model - tokens around tk1 and tkn by both ends of the context workload are more likely to be attended to than those tokens in the middle of the context workload CQ . This triggers the need to understand the factors influencing the attention mechanism and how to mitigate them, which motivates use to approach the topic of degraded attention and mitigating factors in the next sections. 3.1
Degraded attention factors
Language processing literature diagnoses a few factors for transformer-based models to lose performance about processing the context workload. While there is no reference concerning how these factors play out in systems design and engineering tasks, any technique that uses LLMs as processing resource is liable to these: **Workload length to context capacity**: the attention mechanism of a specific model has a limit on the number of tokens {tk1 ..tkn } in the context workload CQ it can process, a context capacity determined by the pre-training context intake length (Chen et al. [2023]). Besides this predetermined overall limit, Gupte et al. [2025] figure a common characteristic among experimented models, a degradation in recall probability P [tki ∈ AM ] as the workload length increases over certain proportion to the total context capacity of a given model. **Token distance from main query**: besides the fact that LLMs lose focus on the ’middle’ of the context workload CQ , Zhang et al. [2024] also demonstrate context loss (low P [tki ∈ AM ]) on tokens tki whose position is far from the main query. LLMs run on positional embeddings with higher weights for tokens close to the main query, and lower ones for those farther away from it (Naveed et al. [2025]). If the main query is located close to the middle, then the attention mechanism will mostly depend on the information by the ends of the workload (Zhang et al. [2026]). **Ambiguity across context units**: The presence of ambiguous information across different context units along the workload CQ can lead to confusion and misinterpretation by the LLM (Zhang et al. [2026]). This ambiguity, especially when it involves multiple context units around the middle of the workload, can arise from overlapping or conflicting information and further reduce P [tki ∈ AM ] because the model has difficulty in determining the desired response. This is also the case when context units are not clearly defined about elements that relate it to the main query. 5
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
**Evidence complexity**: Zhang et al. [2024] found that the complexity of the evidence at hand within the context workload CQ also plays a role in the degradation of context processing P [tki ∈ AM ] towards the answer. Complex context require the so-called ’multi-hop’ connection between units across different positions tki , which is a known challenge for LLMs (Baker et al. [2024]). This characteristics means complex evidences require more effort at reasoning about specifics within a main query with respect to the context, which can be challenging for LLMs to maintain and process effectively. **Order-to-structure incoherence**: The structural incoherence of the context can also lead to degradation in the performance of LLMs. This occurs when the context units are not organized in a coherent manner regarding the work product intent (Li et al. [2025]). This is explained by the behaviour of the attention mechanism, which processes the context workload through sequence-dependent concatenation CQ(i) = {tk1 ..tkn } = ||ni=1 tki . Here, a context workload with units that are misordered on their use to the intended answer can lead to misinterpretation by the attention mechanism, and thereby degrade the response generated by the LLM. 3.2
Mitigating degraded attention
The sources presented in this paper about LLM use cases demonstrate that when context units present information that is complementary and related to the main query. While there are factors inducing attention degradation, others make the effect to mitigate such mechanism, such as: **Placement about the context ends**: The attention mechanism of LLMs over the context workload CQ = ||ni=1 tki is subject to the following tendencies about how the LLM reads it (Liu et al. [2024]): primacy, for the tokens in the beginning tki→1 of the context workload determine the processing of all subsequent tokens; and recency, for the tokens in its end tki→n point out at the direction to which the model shall attend to. Zhang et al. [2024] also point out at the placement of duplicates of relevant context by the ends of the workload to avoid losing that information as reference. **Relevance to the main query**: The relevance of context units to the main query is a critical factor in mitigating attention degradation. Li et al. [2025] demonstrate that when a higher relationship to the main query from within the pretraining data, atracts attention to specific context. Zhang et al. [2024] experiment with placing relevant context units closer to the main query, which they find helps the attention mechanism to focus on those units when looking to improve the quality of the response from the LLM. **Role-based context positioning**: this approach involves the strategic placement of context units based on their roles in relation to the main query, with taking advantage of the primacy and recency tendencies upon the sequential processing of the context workload (Liu et al. [2024]). Guo et al. [2024] demonstrate that the placement of context units based on their roles - policy and rules in tki→1 by the beginning, taking advantage of primacy; then exemplars in tki→n by the end of the workload, taking advantage of recency - is beneficial to the quality of the answer. **Attention-trigger context tagging**: this approach involves the use of tags or markers to highlight important context units. Attention triggers to context units can involve styling or semantic cues to indicate the importance of a context unit, including its relation to the main query. These can help the attention mechanism to focus on context units about their relationship to the main query, and improve the capability of the LLM to process that particular tagged unit as reference for the answer (Zhang et al. [2026]). **Reduction of context workload**: Reducing the number of context units in the workload can help mitigate the effects of attention degradation Gupte et al. [2025], because it reduces the competitive demand for the attention capacity by the model. A less diverse context mitigates degraded attention on both context length and multi-hop requirement effects (Baker et al. [2024]) because a less diverse context workload entails less complexity in terms of the number of context units and the relationships between them.
4
Context assembly levels in LLM use cases
A study performed by Graydon and Lehman [2025] on specific use cases of LLMs in safety engineering finds resulting specifications come incomplete and inaccurate. This led our interest in the factors affecting the quality of the work product AM delivered by the LLMs with basis in the context workload CQ . Authors such as Cámara et al. [2023], Crabb and Jones [2024] tried the use of LLMs with simple queries to elicit responses from the LLM, and found that examples counting on the query alone yielded insufficient quality regarding the work product delivered by the LLM. Pradas-Gomez et al. [2024] and Timperley et al. [2025], among others, demonstrate the role of context assembly into building work productas at reasonable quality, which is seen to reduce the amount of rework required to make it compliant to design intent. As section 3 explores influences to the attention mechanism of LLMs in generic terms, the following sections explore context composition for LLMs in systems design and engineering. 6
Context operations with LLMs and evaluation criteria for systems engineering
4.1
A P REPRINT
Context composition levels
As section 2 shows, use cases involve specifics to utilising and assembling supportive context towards the LLM. Besides the overview of working pinciples from Figure 1, these specifics are outlined along the following characteristics: **Conveyor formats**: Role, guideline, requirement and directive definitions can involve text prose or specificallycontrolled prose when working with safety- and traceability-critical information (El Hassani et al. [2025]); short database records with cross-related information such as requirements, benchmarking and traceable safety information take benefit of csv-, markdown- and other table formats (Geissler et al. [2024]); and, visual information requires a multi-modal or vision-capable model that can interpret the visual information and relate it to the main query. **Grammar formats**: These can include formal or informal structures. Natural language syntax is mostly used to convey directives and guidelines, yet can sometimes be used to convey requirements and exemplars (Dehn et al. [2025]). Formal grammars such as markup (XML, YAML, JSON, etc.) and programming languages (Python, Rust, C++, etc.) in codeblocks (Pradas-Gomez et al. [2024], Krus [2026]), and programming languages, which convey the relationships between functions and their parameters in a program. **Application adapters**: These adapters enable the assembly of context and the communication to the LLM and back. Most MBSE use cases leverage application-specific APIs to enable the assembly of context units within the modelling environment and then thits communication to the LLM and back (Dehn et al. [2025], Timperley et al. [2025]); another way of assembling context workloads is the embedding of custom code and interfaces in applications through internal routines, which enable LLM calls within their own working environment (Chen et al. [2025]). **Downstream interpreters**: In this approach, the forwarding of the context workload to the LLM is carried out by the means of LLM-provider APIs, which get the LLM to generate work products from the exemplars in the same modelling language (Krus [2025]). The use of modelling-as-code interpreters can include a single foundation exemplar or a set of exemplars (Krus [2024]) that convey the relationships between elements and their properties in the intended work product. Conveyor and grammar formats enable machine-readable context workload so that the LLM can process it, along application adapters and downstream interpreters that enable LLM outputs to be actionable for a modelling environment. 4.2
Context composition strategies
Some use cases obtain acceptable work products from modelling-as-code context workloads, which enable modelbuilding by downstream interpreters, or by using application adapters within MBSE modelling environments. Both approaches share the following strategies: **Unit composition:**: The use cases display a diversity of strategies: counting on the pretraining context of the specific LLM (Cámara et al. [2023]); the cumulation of downstream question-answer turns at the LLM for in-context learning (Von Heissen et al. [2024]); the use of formal grammar snippets as exemplars supportive to the model-building task (Krus [2024]) along with the option to alternate interaction between context workloads for LLM and generated deterministic model-builing code (Krus [2026]). **Unit order**: The placement of references by along the main query (Guo et al. [2024]) is driven by the primacy and recency biases. Leveraging the tendencies related to context memory and attention degradation (Liu et al. [2024]), an ordered approach to context assembly involves setting rules, precedents, findings and other contextual information following a logical and methodic sequence; examples such as Dehn et al. [2025] and Krus [2025] demonstrate the use of ordered context units - from rules to examples -to improve the quality of the work product delivered by the LLM. **Modelling-as-code**: This technique involves sequencing the context workload with starting directives that precede µ-template codeblock elements (Krus [2025]), and by allowing the modular composition/intake of separate context units (Marini et al. [2025]), both techniques making use of models-as-code around the main query. The use of code-based modelling languages is also enabled in examples like those from Timperley et al. [2025] and Dehn et al. [2025] that compose the context to LLMs within modelling environments by the means of application adapters. **Context memory**: The use of context memory involves persistent context units being reused across multiple turns. This can be achieved through the use of RAG databases (Balu et al. [2025]) or by storing context units in a structured format that can be easily retrieved and reused (Hanke et al. [2025]). This helps to reduce the amount of context workload required for each query, thereby mitigating attention degradation and improving the quality of the work product delivered by the LLM. The effectiveness of context composition is determined by how directly each element of the workload conveys information to the LLM, and by how well the context units scaffold the LLM onto processing the work product. 7
Context operations with LLMs and evaluation criteria for systems engineering
5
A P REPRINT
Model generation parameters
This contribution is part of ongoing development on the use of LLMs as generative AI resource (GenAI) in systems engineering. The understanding of human-machine collaboration with GenAI makes the context, whereas the proposition of knowledge, strategy and applications to leverage the use of GenAI provide directive viewpoint. This study departs from fundamentals in the introduction to this paper and in previous contributions of our own (Krus [2024], Marini and Krus [2025]). Then, we aim at a formal approach to context operations in LLM-based systems engineering and design. 5.1
Choice of LLM function
The approach to context operations in this paper first involves formalizing the choice of LLM function and its parametrisation, to configure the LLM emdpoint that will receive the context workload. Then, the context workload assembly is formalized with modedls that present the context units and the operations that enable their composition onto the context workload. To understand the context intake functionality of large language models (LLMs), we formalize its workings as a function ′ class ′ LLMX , where each specific model plays the role of a function; ′ X ′ may be replaced by any particular LLM at the discretion of the user. Here, we use equation clauses to present the functionality and parameters of LLMs, and denote all references to them as LLMX . Equation 1 approaches LLMs in representing the ability of the professional user to choose one particular model among several available:
LLMX = ∨{MGP Tm , MClaudem , MGeminim , ..., MDeepseekm , MQwenm , MKimim , ..., MM istralm , NCommandm , ...}
(1)
The first engagement is the choice of operator – LLMX function –, which is done with the application by choosing a given individual model operator to answer an intended request. Here, M refers to each being a model, and m refers to one of several versions in a specific model lineup. One can then understand M as an instance of LLMX which can perform advanced language processing operations, including design and engineering tasks, where LLMX involves the processing of any chosen M at answering to a request by the user. 5.2
Parameters of LLM function
Considering the model choice from those in display, each option for LLMX from equation 1 has parameters that can be worked upon. The combination of such parameters in LLMX can define how the context will be processed. Our contribution considers the following parameters to LLMX : • Mlm : _model_size_ the size of LLMX on the amonut of parameters in the probability matrix from training, – Dtr : _dataset_size_ the diversity of sources into the calculation of Mlm on which LLMX is trained, – Tllm : _temperature_ the index to how random the LLMX will predict content in its response to a request, and, – CM (i)len : _context_length_ the intake capability of LLMX to receive a context length towards a request, – AM (i)len : _answer_length_ the answer length capability of LLMX to yield content in response to a request, • CQ(i)len : _context_workload_ the length of the context assembled towards the call to LLMX . Then, LLMX can be expressed by Equation 2 as a function of the parameters each call relays to the model. The parameters of main interest to our operations are: the context length capacity CM (i)len , the answer length capacity AM (i)len , and the assembled context workload CQ(i)len ; the first two are properties of LLMX and the last one is determined from the context operations by the user. 8
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
tuning parameters
model choice
}| { z }| { z LLMX = f ( Mlm , Dtr , topp , topk , Tllm , ... (2) CM (i)len , AM (i)len , CQ(i)len ) | {z } | {z } | {z }
context capacity
CM (i)len ≥ { CQ(i)len + AM (i)len }
answer length context intake
Each API library has a specific syntax for calling the function and determining its parameters. Specific language models - instances of LLMX - will also have specific parameter setting requirements for Tllm , topp and topk . On our main interest, settings to both CM (i)len and AM (i)len are also model- and provider- specific: within the condition set above, some models set limits at both whereas others allow any proportion between them.
6
Context operations
We have engaged onto formalizing the context assembly operations that enable a complete LLM call. Here, we use equation clauses to present context units as to their role onto supporting the LLM’s context intake. Our intent is to demonstrate the context-building components towards the enhanced chatbox. Before proceeding with the example, the following context operations are introduced: 6.1
Generic context formulation
Context assembly can work through assembling several context units around the main query, whose sequenced combination is intended for ingestion by LLMX as a single context package. The operator will assemble the context units as available so that LLMX will perform its internal processing towards the intent. Then, the overall context workload CQ(i) assembly for the ’i’ call will include the following components: • CQ(i) : _context_workload_ the aggregate operated context for an LLMX function call. – Cup(k,i) − _upstream_context_ in section 6.2.1: the workload part that is added before the main query to calling LLMX at each question. – QP (i) − _core_question_ from section 6.2.3 the question statement as elected by the user to call LLMX . – Cdn(m,i) − _downstream_context_ in section 6.2.2: the workload part included after the main query towards LLMX , and, • AM (i) : _answer_yield_ the answer yield from LLMX to yield content in response to a request, The context workload components for CQ(i) will be assembled before - Cup(i) - and after - Cdn(i) - the main query, for calling the LLMX function to the intent expressed in the QP (i) query. The context workload CQ(i) to the LLMX function results from assembling context units onto a message; the model internals in LLMX will process the workload CQ(i) onto an answer AM (i) . Then, equation 3 displays the formulation of each single call to LLMX with the individual terms to the call.
Cup(i) | {z }
upstream
QP (i) | {z }
question
Cdn(i) = | {z }
CQ(i)
downstream
(3)
AM (i) = LLMX { CQ(i) } Here, Cup(i) is placed first for the upstream context component to set the initial context for the call. Then, the core question unit QP (i) will express the intent of the context workload. The Cdn(i) component will convey imported reference units to provide complementary reference information. Whereas equation 3 only displays context unit grouping per position, there is the opportunity of allowing the intake of several references per context group. 9
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
The user will first compose/import the components he expects to forward to LLMX ; here, he can add a single component either upstream or downstream of the query, or can add a composition of context units positioned around the main query. Equation 4 represent the context assembly. Here, ′ k ′ represents the number of upstream context units and ′ m′ represents the number of downstream context units relative to the main query QP (i) . K
Cup(i) =
M
k=1
Cup(k,i)
Cdn(i) =
m=1
Cdn(m,i)
k, m ∈ N (4)
Cup(k,i)
QP (i)
Cdn(m,i) =
CQ(i)
Then, the user can import or compose several context units to be positioned before and after the main query, and the application will assemble them into a single context workload CQ(i) to be sent to LLMX as in display by Figure 3, which displays the context assembly process. The model will process the workload CQ(i) onto an answer AM (i) from its internals.
Ancillary context units and query statement: Cup(k,i) , QP (i) , Cdn(m,i) ∈ CQ(i) User
import
Cup(k,i)
QP (i)
Upstream units Cup(k,i)
||K k=1 Cup(k,i)
> CQ(i) (I)
eq. (4)
Query statement Core question QP (i)
|| QP (i) →
|| QP (i)
> CQ(i) (II) import
LLMX
References
|| Cup(k,i) →
compose
∈ CQ(i)
Cdn(m,i)
eq. (4)
References Downstream Cdn(m,i)
|| Cdn(m,i) →
||M m=1 Cdn(m,i)
> CQ(i) (III)
eq. (4)
Figure 3: Generic assembly operations of ancillary context units for a single LLMX call. The assembly of context units from string variables results from the concatenation of individual strings into a single string variable that will carry the full context workload to LLMX . This concatenation of context units is order-dependent upon the alignment between grammar structures in the input and language-processing capabilities by generativce resources such as LLMs. However, the attention mechanism within LLMs has a characteristic of reading the unified context workload from the beginning and from the end of the string, with a tendence to losing attention to the content by the middle of the string. The lost in the middle problem as identified by Liu et al. [2024] . 6.2
Role-focused units onto single-call
Role-focused units contain tokenized content attending to distinct purposes in relation to the main query, whose sequenced combination is intended for ingestion by LLMX as a single context package. The operator will assemble the context units as available so that LLMX will perform its internal processing towards the intent. Then, the overall context workload CQ(i) assembly for the ’i’ call will include the following components: 10
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
• CQ(i) : _context_workload_ the aggregate operated context for an LLMX function call. – RP (i) − _policy_rulework_ in section 6.2.1: the resulting rulework prompt with directives to calling LLMX at each question. – PP (i) − _context_reference_ in section 6.2.2: a context unit to recur at every call within ’r’ calls of the context memory Cmem , and, – QP (i) − _core_question_ from section 6.2.3 the question statement as elected by the user to call LLMX . • AM (i) : _answer_yield_ the answer yield from LLMX to yield content in response to a request, The context workload components for CQ(i) shall follow certain order in order to maximise the compliance by the LLMX function to the intent of the query. The context workload CQ(i) to the LLMX function results from assembling these context modules onto a message to the model. The model will process the workload CQ(i) onto an answer AM (i) from its internals. Then, Equation 5 displays the formulation of each single call to LLMX .
PP (i) | {z }
RP (i) | {z } policy
reference
QP (i) = | {z }
CQ(i)
question
(5)
AM (i) = LLMX { CQ(i) }
Role-based context units and query statement: CQ(i) User
import
RP (n,i)
PP (p,i)
QP (q,i)
Policy units RP (n,i)
||N n=1 RP (n,i)
> CQ(i) (I)
eq. (5)
Query statement Reference units PP (p,i)
|| PP (p,i) →
||P p=1 PP (p,i)
> CQ(i) (II) import
LLMX
References
|| RP (n,i) →
compose
∈ CQ(i)
eq. (5)
References Question units QP (m,i)
|| QP (q,i) →
||Q q=1 QP (q,i)
> CQ(i) (III)
eq. (5)
Figure 4: Role-focused assembly operations to context units for a single LLMX call. Here, RP (i) is placed first for the policy component to set rules over the whole of the call. The PP (i) component will convey imported reference units to provide complementary guidelines towards the answer. Then, the core question unit QP (i) can include exemplars such as µ-Templates (Krus [2026]) and specifics besides the actual question, and then take advantage of recency (Liu et al. [2024]) to maximise the influence of intent-related statements in the query. 11
Context operations with LLMs and evaluation criteria for systems engineering
6.2.1
A P REPRINT
Rule and policy operations
One can set policy context units to help steer the workings of LLMX function to yield an answer to a closer approximation to the query intent. These context units will work as rules of engagement which will drive the LLMX function to process the core question along the given context of composed and imported units. These context units can be individually composed by the user at anytime regarding its intent towards operating the LLMX function. Then, the professional user can compose the following string units: • RP (i) : _policy_rulework_ the rulework prompt with directives and guidelines to calling LLMX . – Gsp : _global_prompt_ a recurring prompt to every call that provides directives of engagement, – Bsp(i) : _boundary_prompt_ a context memory prompt with complementary guidelines, and, – RP (n,i) : _policy_context_ additional policy context units for a single call to LLMX . The user will first compose/import the RP components he expects to forward to LLMX ; here, he can add policy components with n policy context units towards setting the policy component RP . A context operation regarding policy assembles the RP (i) component with several units RP (n,i) , and sends it to LLMX at the request of the user. The user can compose/import the global prompt Gsp to request the LLM to follow a set of rules, without adding a boundary prompt. Here, the call to LLMX takes place by the (I) message line, and Gsp will be forwarded at all times.
Bsp(i) ∈ RP (i) | {z }
Gsp ∈ RP (i) |{z} global
(6)
boundary
The user can also compose/import the boundary prompt Bsp as complementary guidance to LLMX alongside Gsp to the RP (i) policy component. Here, the call to LLMX takes place by the (II) message line, and Bsp context component can be forwarded along the Gsp global prompt. Here, RP (i) can carry the single global prompt, or both global and boundary prompts, and can include further units RP (n,i) intended for a similar role. These complement each other regarding the order in which they are assembled: Gsp conveys role, background, attitude and style directives for LLMX , whereas Bsp(i) forwards guidelines to how LLMX shall assemble the answer such as work process, topic structure, stylesheet and generic codeblock format as applicable. Then, the user can compose/import several ′ n′ policy units RP (n,i) to complement the policy component RP (i) for a single call to LLMX . 6.2.2
Imported references
One can import reference context units to provide basis for the workings of LLMX function, to guide the process at a closer approximation to the intended reasoning mechanism for the query. These context units will work as reference basis to support the reasoning process by the LLMX function to process the core question. In an ordered context workload, the references are intended for placement between the policy units and the question units. Then, the professional user can compose the following string units: • PP (i) − _reference_component_: the reference component added to the context intake towards LLMX . – Pp(i) − _single_reference_ : an individual reference unit that will be directly added to the context, – PP (p,i) − _several_references_ : several ′ p′ reference units that will be assembled together and added to the context, The user will first import the context unit(s) he expects to forward to LLMX ; here, he can add a single Pp(i) context unit, or can add several PP (p,i) reference units to the context workload. Here, PP (i) (I) considers the import of a single reference unit Pp(i) , whereas PP (i) (II) considers the import of several reference units PP (p,i) . The assembled context is then sent to LLMX . P
PP (i) = Pp(i) |{z}
PP (i) =
single
12
PP (p,i) p=1 | {z } ref. unit
∀p∈N
(7)
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
These context units can be individually composed/imported by the user at anytime before the call. Here, an imported context unit Pp(i) serves as reference towards the call ′ i′ . to LLMX , according to the equation 7 above. At the same ′ time, the reference component can include several ′ p′ context units towards the call ′ i′ , from the first to the final ′ Pth reference, each making a context unit PP (p,i) to LLMX . The user makes all considerations of purpose and ordering about importing the reference units, one or multiple at a time. The effects of the formulation of the reference component PP (i) are only constrained by the limit within the LLM context window as defined by the selected LLMX function; the context window limit affects the functionality considering references the answer AM (i) , with effects explained in the section 3 of this paper. 6.2.3
Core question operations
The context assembly to LLMX calls involves the crafting of a question for each request, in the form of a query statement that is intended to trigger attention by the model. This is a key component to steer the internals of the selected LLMX function, as it defines the object of inquiry and thus the focus of the process. These context units, composed through the process in the Figure 5 will work as reference basis to support the reasoning process by the LLMX function to process the core question. Then, the professional user can compose the following string units: • QP (i) : _core_question_ the question statement as elected by the user to call LLMX . – Qp(i) : _query_statement_ the query statement written by the user towards its intent for LLMX , – Ov(i) : _vectoring_operator_ a relationship operator clause to steer Qc(i) onto specifics, – Qv(m,i) : _prompt_vector_ an aspect clause that adds a specific to Qc(i) that LLMX shall process,
Question workflow: QP (i) Qp(i)
User
compose(i)
Ov(i)
Qv(m,i)
Query statement to QP (i)
Qc(i) ∈ QP (i)
QP (i) (I) >
compose(i)
Set vectoring operator Ov(i) Vectoring operator Ov(i) Set prompt vectors Qv(q,i) Set || Qv(q,i)
|| Qv(q,i) →
Set || Qv(Q,i)
|| Qv(Q,i) → QP (i) (II) >
eq. (9, left)
Set vectoring mode as combo with Qv(q,i)
Ov(i) → compose(q,i)
LLMX
Set Qp(i)
Qp(i) →
setQv(q,i)
QP (i)
||Q q=1 Qc(i) || Ov(i) || Qv(q,i) = QP (i)
eq. (9, right)
Figure 5: Assembly operations to core question QP (i) for a single LLMX call. 13
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
The user will first compose the question he expects to forward to LLMX ; here, he can add the query statement Qp(i) alone, or can set a prompt vectoring operation. with a vectoring operator Ov(i) with ′ m′ prompt vectors to request LLMX to answer the query statement in several specific aspects, even with the ability to associate Qx(i,m,km ) exemplars to each vector. Once being set about the core question QP (i) before calling LLMX , the user can proceed to perform the query. The user needs to compose the core question QP (i) towards calling LLMX . Here, the user can compose a single query statement Qc(i) as in display by Equation 8. For that purpose, the query statement will contain guidewords such as ’what’, ’where’, ’how’, or elaborated requests such as with including ’please explain’ or ’I need to know’ clauses. These words play the role of attention-triggers telling LLMX to focus on the specific aspect of the query statement, and to provide a response that is compliant with the intent of the user.
QP (i) = Qc(i) |{z} query
QP (i) =
Q Qc(i) q=1 |{z} query
{ Ov(i) |{z}
operator
Qv(q,i) } | {z }
∀q∈N
(8)
vectors
At the same time, the user may figure the query statement can be enhanced by prompt vectors. Here, the question QP (i) as shown by equation 8 aggregates the clauses for Qc(i) , and then the Ov(i) , repeated times to each Qv(i,m) elements, and then QP (i) will ask the LLMX function to provide an aggregate answer considering all vectors according to Equation 8, under guidance by the rulework first provided in RP (i) and with reference to imported context units within PP (i) . The prompt vectoring formulation in Equation 8 applies to the assembly of a single request to LLMX function, and enables comprehensive responses upon the capability of individual LLMX functions. The individual answer provided by LLMX will address all single specifics as defined in prompt vectors, because the query statement Qc(i) and the operator Ov(i) are replicated at all times along each prompt vector in the core question component.
6.3
Context assembly operations
The assembly of context towards engineering tasks, from system design context definition to model-building and implementation, requires flexibility and modularity in designing and handling systems information as context input for use with large language models, through performing the context operations formalized in section 6. Table 1 lists the role-focused context units in the context assembly process as defined in the previous section, and the intent for each. This setting of different context units with basis on roles enables the assembly of a context workload CQ to be forwarded to the LLMX function with mind to a certain intent towards a work product that shall be embodied by the means of the answer yield AM . However, the effectiveness of the context workload CQ depends on the assembly of the context units, which requires understanding the dependencies between them. The reason for this understanding lies in the dependency relations between context units as expressed by the context operation definitions from section 6: the joining and assembly of context unities takes place by means of concatenation; the transformer mechanism within large laguage models Vaswani et al. [2017] is such that the order of the context units in the workload matters, and the dependencies between them are relevant to the answer yield AM . This means dependencies between context units must respected when assembling the context workload CQ Table 2 presents a Design Structure Matrix (DSM) capturing dependency relations between context units. The context assembly takes place before the call to LLMX , from which the internals of the model generate the answer. The DSM displays the correspondence and the resulting order of context units, from the upstream policy units, through the imported references placed in between, and the core question units downstream. Three principles steer this ordering sequence: (i) the order of context units in the workload matters due to the transformer mechanism; (ii) the dependencies between context units are relevant to the answer yield AM ; and (iii) the memory of LLMX ends up prioritizing the beginning and the end of the context workload. A person in the role of systems design engineer curates and composes/imports the relevant context units, thus assembling the context workload, and then requests LLMX . The chosen LLM will process the context workload CQ with the parameters set in section 5.2 and yield its answer AM under directives and guidelines set within the policy rulework module RP , with basis on the PP imported references, and supported by the exemplars set within QP . 14
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Table 1: Context modules and units used in the assembly of the context workload forwarded to the LLM. Context unit Operation Intent
RP
→ Gsp
Policy rulework Global prompt
→ Bsp
Boundary prompt
→ PP
PP
Reference priors Full References
→ Pp
Other references
→ Qp → Ov
QP
Core question Query statement Vectoring operator
→ Qv
Prompt Vectors
CQ ⇒
Context workload
section 6.3
LLMX
Large language model Answer yield
section 5.2
⇒ AM
section 6.2.1 Defines engineering role, epistemic rules, and non-hallucination constraints Defines system scope to its intended application and characteristics.
section 6.2.2 Provides references with descriptions and state-of-the-art technology. Extracts condensed system and design constraints from imported sources.
section 6.2.3 Sets primary architectural inquiry. Field: Sets relationships between partial aspects and the main inquiry. Field(s): Defines specific aspects that the task shall consider. Context workload assembled from context operations.
LLM-generated content that considers the information in the context workload.
Table 2: DSM Representing context assembly, LLMX event horizon and answer yield with memory. Gsp
Bsp
Gsp
–
Bsp
•
–
RP
•
•
RP
Pr
Pr
PP
Qc
Ov
Qv
QP
CQ
LLMX | |
–
|
Pr
–
Pr
•
–
PP
•
•
| | –
|
Qp
–
Ov
•
–
Qv
•
•
–
QP
•
•
•
–
→
→
→
||
–
|
→
•
–
→
•
CQ
→
AM
→
||
→
→
||
|
LLMX AM LEGEND: • Row items succeeding column items.
→ Concatenation of items onto modules.
15
| | |
|| Operations on context modules.
(•)
Context operations with LLMs and evaluation criteria for systems engineering
7
A P REPRINT
Case study
This section demonstrates the outputs of system configuration requests enabled by the context operations formally defined in section 6. This study focuses the assembly of context units to support the processing of LLMX functions at building component architecture models of complex systems. To demonstrate how context operations work onto supporting these functions, this paper involves a modelling case regarding the component architecture of a hybrid SAR UAV, which is hereby denoted as Suthern-Cross as in display by Table 3. Table 3: Context operations demonstration assignments regarding systems characteristics Heavy SAR UAV - Suthern-Cross Assignment
Operation: Search-and-rescue operations to lift and recover victims from flash flood to medical care.
Key requirements
Station-keeping on hover, limited downwash over victims, low noise footprint, aircraft regulatory compliance.
Key constraints
Heavy rainfall at location, aggressive wind and gusts, timespan to victim acquisition, energy and power budget.
LLM Assignment
Help functional decomposition and architecture design under professional supervision towards defining system/concept architecture.
Picture source: https://youtu.be/QVyptRhdDgA?si=ygtpMefwQXCHfO54&t=84, accessed on 2026-06-05.
Table 3 includes a depiction of an as-current Search-and-Rescue operation over flooded area with an helicopter - the SAR UAV is expected to have similar capability of recovering victims. The architecture modelling task is defined with the goal of creating a functional and operational system, starting from a system scope of required functionality. The focus of this case study is the leveraging of the modelling-as-code paradigm, where the modelling language is used to generate a model from the answer yield AM provided by LLMX . 7.1
Context workload treatments
The context treatments involve selective use of context units onto requesting LLMX for the generation of the intended work product. The context units within the treatments follow the definitions set in section 6; the role of individual context units towards the assembly of input to LLMX is considered upon composing and importing information content, as in display by Table 4. Table 4: Context workload treatments. Qc Query
Ov ||Qv Vectors
Role directives, Modelling scope, Generated mission scenario, principles, rules, benchmarking requirements analysis. Validation
Work product request, and requirements.
µ-Template subsystem exemplars.
T2
•++>
•++>
T3
•++>
•++>
• • •> 127 Tk • • •> 2485 Tk • • •>
T4
•++>
•++>
•++>
• • •>
Input
Typed/pasted prose & topics, saveable.
Typed/pasted prose & topics, saveable.
Pasted source text import or DOCX, PDF
Typed/pasted or imported prose & topics
Gsp Global Content
Bsp Boundary
PP Reference
T1
16
•++> 7645 Tk •++> 20555 Tk Paste into or import
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
The treatments are designed to be cumulative, where each treatment adds more context units to the workload2 for LLMX to process. The cumulation of treatments is such that the first treatment T1 only includes the query statement Qc ; the second treatment T2 adds the global and boundary prompts Gsp and Bsp ; the third treatment T3 adds the prompt vectors Ov and Qv ; and the fourth treatment T4 adds the reference priors PP . The treatments are designed to evaluate the effects of different combinations of context units on the answer yield AM from LLMX , which are assembled with basis on the context operations defined in section 6. For that purpose, the experiment is based on examples of using plantUML as modelling language, considering LLM-aided modelling technique, such as by Cámara et al. [2023] and Krus [2024]. 7.2
Model use treatments
Besides the four context treatments, the experiment also includes five LLMX model treatments. The model assortment for this case study is designed to evaluate the effects of model characteristics on the answer yield AM from LLMX receiving the same context workload. The LLM assortment is designed to evaluate the effects of model characteristics on the answer yield AM from LLMX receiving the same context workload. The LLM assortment was defined with mind to considering processing resource: one model running on local GPU3 , two models from the cloud4 that can be run on high-end local desktop GPUs to 48Gb RAM, and two frontier models to be run from the cloud. Table 5 displays the specifications, their descriptions and their units. The API interface forwards the assembled context workloads to the selected LLMX models. Table 5: Model structure metrics - characterization of models.
Resource Architecture [Param.] Context [Tk] Answer [Tk]
M1
M2
M3
M4
M5
Qwen3 8b
ChatGPT OSS-20b
Nemotron3 Kimi super-120b-a12b K2.5
Claude Sonnet4.6
Yang et al. [2025]
OpenAI [2025]
NVIDIA [2026]
Ollama [2026]
Anthropic [2026]
Local/GPU Distilled 8 × 109 3, 27 × 104 4, 09 × 103
Cloud/Nvidia MoE 2 × 1010 1, 28 × 105 3, 27 × 104
Cloud/Nvidia MoE 1, 2 × 1011 1, 00 × 106 6, 40 × 104
Cloud/Ollama MoE 1 × 1012 2, 56 × 105 6, 40 × 104
Cloud/Anthropic Undisclosed 1, 00 × 106 6, 40 × 104
The following characteristics affect modelling performance: (i) local hosting is limited to the memory room in the local machine; (ii) a higher scale of parameter count usually enables more complex and targeted processing; (iii) a longer context length capacity means the maximum number of tokens that can be processed by an individual LLM; and (iv) the answer length limit is less about capability, and more about policy - especially with higher-end models. 7.3
Modelling corpus requirements
The case study proceeds with the assumption that the modelling response can provide a work product the interpreter is able to render. Context workload components involve the same content set by context composition treatment regardless of the model called, so the resulting models can be compared across treatments and LLMX models to evaluate the role of context operations and model scale. In the modelling-as-code approach, the context workload conveys to LLMX a request for a system model intended to represent the intended system architecture with sufficient detail and within a configuration that complies with required functionality. To enable the proper validation of the model outputs in the corpus of generated models across treatments, two sets of modelling requirements are defined: a) plantUML syntax convention; and b) architecture requirements defined in Table 6. 2 The context intake length in tokens [Tk] is the workload that utilizes context length capacity in each of the models called. Table 5 displays model characteristics regarding context length and answer length capacities for reference. 3 The local model runs with a 4Gb Nvidia Quadro T1000 GPU and a 2.7-4.0 GHz 12-core Intel Core-i7 CPU with 16Gb memory. 4 Cloud bandwidth intake takes place at 50 Mbps and answer yield takes place at 500 Mbps.
17
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Considering the context workloads from Table 4 submitted to the LLMX models from Table 5 include requests for plantUML cobeblocks as part of the answers AM , the evaluation of the capability by the models to process context workloads takes place through the manual checking and validation of the codeblocks inside the answer files. For that purpose, the model outputs in conversation codeblocks are exported onto plantUML model code files, to be verified and corrected individually in the following steps: 1. Verify the model m0 ∈ AM in the answer codeblock from LLMX with, duplicating it onto a m1 = m0 plantUML model file for syntax review with the interpreter; 2. verify the m1 model file for syntax errors and architecture requirements, iterate corrections until the model is fully correct, then save the corrected model m2 ̸= m1 to a model file with Corr1 suffix; 3. Verify the corrections made to m2 on architectural requirements, amend it to comply with the requirements and save the corrected model m3 ̸= m2 to a model file with Corr2 suffix. Then, the evaluation of the resulting corpus involves the review and validation of model-as-code units with metrics conveying quality requirements from Table 6. Table 6: Modelling requirements for system architecture models. plantUML syntax rules Model syntax
Unique component Unique port name_def definitions name_def definitions in entity syntaxes: in entity syntaxes: [component_name], port PT within element, or component "Name", or port "PT" under with unique as alias. component name_def.
Single-pair flows with Flow direction signs cp1 - cp2:src-tgt and [format] params source-sink definition between dashes and and matching port links no stray component with correct arrows to or port name_defs flow sink. outside entity syntaxes.
Traceable, consistent component and port name_def assignment throughout component entity definitions to flow statements.
Component architecture requirements Model elements
Single energy source with fuel specification and power plant with controls, power lines, drivetrain and endeffect components.
Power take-off and Safety and payload distribution with heat systems including exchange components, parachute, hoist, winch, end-effects linked to cabling, load cell structural elements and control with such as airframe/pylons. rescue equipment.
Flight control systems including sensory and processing components, along connectivity and controls to ancillary & end-effect actuators.
Flight support and, and onboard mission controls and data processing with communication systems and protocols.
Model Component name_def name_def flow links to There is no stray Flow linetype setting Single flow connections ports/flows shall have one or more same name_def ports name_def component within brackets in flow between connected pairs individual name_def ports within or under name_def statement.
in single component at source and in single component at sink.
and no port/flow name_def mismatch by either flow end.
definitions between single pair of source and sink components.
of name_def ports by source and sink name_def components.
Model Power system energy compliance flows from single
Control system flows from sensor elements through controller elements and actuator components.
One-way source-to-sink flow chains for control and power systems with no circular paths across energy & signal flows.
Control architecture across signal flows by acting components to ensure functionality and control modes.
Specific quantification of components about energy sources and end-effect assemblies and unique ports.
source to countable end-effect components and outputs.
The inclusion of exported plantUML model files enables the verification of the outputs regarding the requirements above, on the following order: raw → syntax → architecture. A model is considered correct if it complies with the syntax and architecture requirements, and it is considered incorrect if it does not comply with either of them. The variety of models from Table 5 will determine different capabilities regarding the satisfaction of the requirements, which means each pair context-model may yield different levels of compliance with the requirements. 7.4
Modelling answer verification
LLM limitations can produce a representation with insufficient detail and lacking compliance to system requirements. Sometimes, the feeding system model codeblocks to the plantUML interpreter fails to render the model correctly because of syntax errors. Moreover, incorrect model representations manifest in differences between raw models from LLMX output and corrected models to syntax. In this context, the metrics reflect findings on the architectural model about whether it misses any required element or whether it does not comply to rules. The metrics proposed for this case study come in display by Table 7. 18
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Table 7: Model metrics considered for architecture validation. Generative modelling outputs Model answer
Answer length [tk] AMlen ∈ AM
Context utilization [tk] CLlen /[tk] CMlen
Answer time [min:s] tA CQ → A M
Model length [lin] mlen ∈ AM
Model type [type] mtype ∈ plantUML
Flow syntax [n] ferr ∈ / [type] ∈ plantUML
Port syntax [n] perr ∈ / [type] ∈ plantUML
Dupl. name_def [n] ea = eb ≥ 2x ∈ AM
plantUML syntax rules Model syntax
Model versions [n] mx off AM
Element syntax [n] eerr ∈ / [type] ∈ plantUML
Component architecture requirements Model elements
Element count [n] ex ∈ AM
Element miss [n] ex ∈ DC ∈ / AM
Stray element [n] ex ∈ DC ∄fxin , fxout
Module miss [n] gx ∈ DC ∈ / AM
No end-effect [n] zx ∈ DC ∈ / AM
Model ports/flows
Flow count [n] ∈ AM
Flow miss [n] fx ∈ DC ∈ / AM
Port dangle [n] px ∈ DC ∄fxin , fxout
Port count [n] ∈ AM
Port miss [n] px ∈ DC ∈ / AM
Input misses [n] fxout ̸= pxin ∈ mx
Chain misses [n] {ex → ey → ez } ∈ / DC
Output misses [n] pxout ̸= fxin ∈ mx
Model compliance
The model answer metrics in the first row quantify characteristics of the answer provided by LLMX to certain context workload treatment. The other metrics are designed to quantify the characteristics of the model outputs from LLMX regarding syntax and architecture requirements. These include AM processing statistics by LLMX , then all model verification criteria - syntax and architecture - apply at the raw model codeblock m0 exported to the model file m1 . Equation 9 expresses the relationship between the context workload CQ and the answer yield AM from LLMX , where the answer length AMlen is less than or equal to the context length CQlen , and the model length mlen is less than or equal to the answer length AMlen .
m0 ∈ AM (i) | {z }
LLMX {CQ(i) } = AM (i) | {z } | {z } CMlen ≥
CQlen +AMlen = CLlen
mlen < AMlen
m1 = m0 | {z }
(9)
m1 model file
Once a m1 model file is available, its verification is carried out on the following basis: the syntax of the model is checked for errors, and then the architecture of the model is evaluated for compliance with the requirements. This is done with the checking the following assertions on the model: the assertion (I) in Equation 10 regards the existence of syntax errors in the model m1 regarding element, flow and port definitions, and the second condition regards the existence of duplicate name_def definitions for elements.
I:
merr = {∃ [eerr , ferr , perr ] ∈ / plantUML ∨ ∀ [a, b] → ea = eb } ∈ m1 | {z } | {z } syntax errors
(10)
duplicate names/aliases
Then, the assertion (II) in the first line of Equation 11 regards the architecture requirements, with the first condition being about any missing element, flow, port, module or end-effect in the model and the second condition being about any stray component or dangling port in the model. Then, the assertion (III) in the second line of Equation 11 is true if it does not comply with architecture requirements, which include any input and output mismatches in the model, or any chain of elements that is not present in the model. 19
Context operations with LLMs and evaluation criteria for systems engineering
II:
mmiss = { ∄ [ex , fx , px , gx , zx ] ∈ DC | {z }
element, flow, port, module or end-effect misses
A P REPRINT
∨ [ex , px ] ∄ [fxin ∧ fxout ]} ∈ mx {z } | stray component, dangling port
(11) III:
mshort = {∃ [fxout ̸= pxin ∨ pxout ̸= fxin ] ∨ ∃ [ex → ey → ez ] ∈ / DC } ∈ mx | {z } | {z } Input and output mismatches
Chain misses
The verification of architectural metrics is preconditioned by correct model syntax.This means that the model-building attempts from context workload may display different inconsistencies with requirements - different context workloads entail levels of specification to modelling-as-code LLM outputs regarding those criteria. Then, Equation 12 describes the conditions under which additional model files are needed.
I:
Corr1 = I ⇒ ∃ m2 ̸= m1
II:
Corr1 =
III:
∄ (Corr1, Corr2) :
∧ Corr2 = [I ∧ (II ∨ III)] ⇒ ∃ m3 ̸= m2
[¬I ∧ (II ∨ III)] ⇒ ∃ m2 ̸= m1 , ∄ m3
(12)
[¬I ∧ (¬II ∧ ¬III)] ⇒ ∄ m3 , m2
The first condition in Equation 12 applies to the case where the model m1 has syntax errors, which requires a new model file m2 ̸= m1 to be created with corrections to syntax and architecture. Here, the need to make additional model files m2 and m3 shall involve syntax and architectural refinements to satisfy the requirements; the corrections are ladder-staged ifrom m1 to m2 and then from m2 to m3 ̸= m2 . When m1 has correct syntax, then it undergoes architectural review of m2 to determine the need for m3 . The second condition in Equation 12 applies to the case where the model m1 has correct syntax, but it does not comply with architecture requirements, which requires a new model file m2 ̸= m1 to be created with corrections to architecture; no further model instances will be created. The third condition applies to the case where the model m1 has correct syntax and complies with architecture requirements, which means no additional model files are required, and the architectural review can be carried out directly to the first model off the answer codeblock.
8
Results
The results of the case study are presented in this section, with a focus on the modelling outputs from the generative modelling approach supported by modular context assembly. The analysis includes selected examples of model responses by LLMs, as well as a comparative analysis of model length and element counts across different treatments and models. The generative modelling approach supported by modular context assembly has been experimented on 20 parallel model runs along the workload treatments from section 7.1 and the models called in section 7.2. With including the corrections needed, then the modelling corpus expands to 54 - fifty-four - model instances. 8.1
Modelling examples
To demonstrate the results from the case study, this section first presents selected examples of model responses by LLMs. The purpose of characterizing these examples is to illustrate the effectiveness of the generative modelling approach, and its characteristics in proportion to the context workload. This section should present and describe the resulting models from the generative modelling approach with regard to the context workload treatments, and the LLMs called. Figure 6 displays the model produced by (M5) Claude Sonnet 4.6 for the first treatment (T1) with no ancillaries to the question. This is a class model with system packages. This model has 38 high-level components with specifications in class properties (+) and 46 flows mostly expressing overall interdependence between elements. While possibly displaying a design specification of a heavy a SAR UAV, it deviates from the intent of a component architecture. 20
Context operations with LLMs and evaluation criteria for systems engineering
Figure 6: Model requested from Claude Sonnet 4.6 for T1 context.
21
A P REPRINT
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
While the first treatment presents the question alone and therefore mostly counts on the weights from model pretraining to process the question intent, the quality of the first model is not sufficient; it does not include ports in components, neither it does specify component flows between them, then it is not a component architecture model and does not comply with the requirements in Table 6. Then, the second treatment (T2) adds policy elements besides the question to the context workload, which is expected to improve the quality of the model output. For that purpose, the T2 treatment includes a policy statement with declaring role and directives by a Gsp global policy component and supportive modelling guidelines by the means of a Bsp boundary prompt component. An example model out from the T2 treatment is displayed in Figure 7, which is a model produced by (M4) Nemotron3-super for the second treatment.
Figure 7: Model requested from Nemotron3-super for T2 context. This model has 28 individual elements and 32 flows; power transmission and distribution elements appear with two sets of soft ports, combining lower-level components. The raw model did not render correctly due to syntax errors; Figure 7 displays the corrected Corr1 model. Ports are represented as nested components, and flows are expressed as single declarations between port pairs. There are a few dangling ports, such as QM_in within the Airframe subsystem, and some combined components such as the generator, yet most of the model complies with the requirements in Table 6. To support that purpose, treatments (T3) and (T4) involve the use of µ-Templates Krus [2024] as exemplars embedded in prompt vectors, with examples in figure 8a, 8b and 8c. These are aggregated to the query statement as prompt vectors following equation 8 to steer the processing of the LLM. The µ-Templates carry modelling-as-code snippets of the system components, ports and flows - plantUML in this case - to provide a reference so that the LLM shall produce a model with similar characteristics.
(a) SAF Gas Engine.
(b) PTO and distribution.
Figure 8: System model errors from LLMX answer products. 22
(c) Hoist system.
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
The use of µ-Templates is a key aspect of the tretments T3 and T4 in our experimental approach. To present an idea on how their use support the modelling process, Figure 9 displays the raw model produced by ChatGPT-OSS-20b for the third treatment. This T3 treatment involved the policy and prompt vector ancillaries (T3). This model has 28 individual elements and 32 flows; power transmission and distribution elements appear with two sets of soft ports – combining lower level components. Soft port declarations are expressed within the component statements, and flows are expressed as single declarations between port pairs.
Figure 9: Model requested from GPT-OSS-20b for T3 context. Then, Figure 10 displays a corrected model with basis on the one above. This corrective treatment involves layout modifications to the powertrain components, whereas the hoist system, electronics and communication systems remained the same, including the port-combination components, and the single flow declarations between port pairs. This means the resulting model is acceptable regarding its compliance to the requirements in Table 6, and it is considered a valid component architecture model, worth considering for check and approval in due engineering review process.
Figure 10: Model corrected from that of GPT-OSS-20b for T3 context. The modelling results from the T3 treatment show that the use of µ-Templates as exemplars embedded in prompt vectors can improve the quality of the model outputs from LLMs. The models produced by LLMs with the T3 treatment have more complete and accurate representations of the system components, ports, and flows, and they comply better with the requirements in Table 6. Ultimately, the T4 treatment yields best results by a marginal difference with support of extra references; their effectiveness is somehow limited by the LLMs’ ability to process and integrate the additional information, considering the increased context utilization and the effect from the bias to missing the middle. 8.2
Modelling analysis
This section is intended to present the results of the modelling experiment over the whole corpus and discuss its outcome with basis on the metrics in Table 7. The analysis is carried out with respect to the context workload treatments and the LLMs called, by the means of a python tool that implements the assertions in Equations 10, 11 and 12. The tool is able to parse the model files and check for syntax errors, missing elements, flows, ports, modules, end-effects, stray components, dangling ports, input/output mismatches, and chain misses. The tool also counts the number of elements, flows, ports, modules, and end-effects in the model files. 23
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Figure 11 displays the result of model length from processing context workload. Here, policy (T2) has mostly increased the modelling output from LLMX with exception of the top frontier model (M5). The introduction of µ-Templates as prompt vectors into the question (T3) reduced the model length; the addition of reference (T4) did not have so significant effect, which means the added references were not so effective in steering the model output.
Figure 11: Model length as result of processing context workloads. Then, an increase in element count from figure 12a with the introduction of policy (T2) and prompt vectors (T3) corresponds to a decrease in sub-element count from figure 12b in these treatments. Sub-elements are counted when system component statements include several component names from the vocablulary in a single entity. Then, the counts did not change significantly from treatment (T3) to treatment (T4) showing the major effect of µ-Templates in steering the representation of system architecture.
(a) Element count by workload treatment.
(b) Subelement by workload treatment.
Figure 12: System model errors from LLMX answer products. The results from the plots above are detailed in pairs regarding context workload treatments T1 & T2 in Table 8 and treatments T3 & T4 in Table 9. The differences in these two sets of treatments lie in the context workload; first, that T1 and T2 treatments do not include µ-Templates, while T3 and T4 treatments do; consequently, the context workload length is larger for T3 and T4 treatments as result of incorporating several µ-Templates as exemplars in the prompt vectors. Table 8 displays the metrics for evaluating the answer yields from LLMX on a workload that includes the single question preceded by the policy components, global and boundary prompt with different context workloads and models. Processing time differs sharply between GPU-ran and cloud-processed models, because model-processing is contrained to local GPU capacity whereas cloud-based models have access to more extensive computational resources. Then, the workload expands from 99 to 1164 tokens, thereby setting a higher load to each model’s context capacity. 24
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Table 8: Metrics for evaluating LLMX answer yields on single question, treatments T1 and T2. Qwen3 8b
ChatGPT OSS-20b
Nemotron3 super-120b-a12b
Kimi K2.5
Claude Sonnet4.6
T1 & T2 Context workloads
T1
T2
T1
T2
T1
T2
T1
T2
T1
T2
Answer [Tk]
1207
2038
1039
1718
1241
1790
1600
3080
4039
5185
Utilization [%]
4,07
13,81
3,56
12,83
2,09
6,52
0,67
2,17
0,42
0,77
Time [min: sec]
6:19
12:42
0:12
0:41
1:31
2:34
0:14
1:01
1:37
1:48
Model type/ Length [lines]
Cmp 89
Cmp 150
*
Cmp 141
Cmp 157
*
Cmp 81
Cmp 180
*
Cmp 122
*
Cmp 270
Cls 403
*
Element type/ count [:n]
Sub 12
Sub 12
Unit 39
Sub 13
Sub 32
Unit 21
Sub 35
Sub 44
Sub 48
Unit 50
Flow type/ count [:n]
Fu-Ct 22
n/a 43
n/a 32
Fw-Fu 32
Fu-Ct 24
Fw-Fu 40
Fu-Ct 30
Fw-Fu 41
Fu-Ct 46
Fw-Fu 101
Subelements/ count [:n]
-
-
-
Port 38
Spec 24
Port 58
Spec 29
Spec 36
Spec 145
Port* 170
Elements/ Ports/ Flows [:n]
12 0 22
12 0 43
39 0 32
13 38 32
32 0 24
21 58 40
35 0 30
44 0 41
48 0 46
50 170 101
Syntax misses [Types/:n]
none
none
none
Fl: 16
none
Fl: 40 Pt: 2
none
Fl: 30
none
none
Element misses [Types/:n]
23
22
14
13
10
16
12
15
7
12
Port/flow mismatches [Types/:n]
Ph: 1
none
Orph: 3
Dang: 38
Orph: 1
Dir: 7
Orph: 2 Ph: 1
Orph: 2
Orph: 1
Dang: 9
Cmp 374
Legend. Model type: Cmp – component diagram; * Cmp – packaged components (components grouped across multiple subsystem packages); Cls – class diagram. Element type: Sub – subsystem element; Unit – unit element; Spec – named property/rating; Port – connection in code; Port* – connection in element. Flow type: Fu – function; Ct – containment; Dv – deliverable; Fw – port-matching flow; Fu-Ct – mixed function/containment; n/a – unlabeled flows. Counts: length in code lines; Element/Subelement/Flow counts as [:n]. Audit. Syntax misses are render-blocking errors (those requiring a syntax-only Corr1 pass): Fl – flow statement, Pt – port statement, El – declaration. Element misses counted strictly vs the SAR reference checklist (per fine-grained element, of 32). Mismatches: Orph – declared element, no flow; Dang – declared, unwired; Dir – direction/port error; Ph – flow to/from undeclared id.
Answer size varies substantially, from 466 tokens (Qwen3 8B) to 1,984 tokens (Claude Sonnet 4.6), indicating different expansion behavior even under comparable prompting conditions. Model-output structure becomes richer in larger responses, as seen in line counts (31 to 195) and flow counts (12 to 56), suggesting greater elaboration of system interactions. Processing time also differs sharply, with local/smaller models taking longer in this setup (up to 12:42) while larger cloud models respond faster (down to 0:14). Besides computing hardware itself, the cloud processing time is determined by the scale of processing resource and the bandwidth of the cloud connection. Regarding the answer output, model-output structure becomes richer in larger responses, as seen in line counts (31 to 195) and flow counts (12 to 56), suggesting greater elaboration of system interactions. Overall, the results suggest that higher-capacity models improve output depth and structural detail, while all models remain lightly loaded in context usage for these treatments. This means that the answers depend more of the model pretraining than of the context workload, and thereby the model outputs are more diverse in structure and content. Table 9 displays the metrics for evaluating the answer yields from LLMX on a workload that includes the single question preceded by the policy components, global and boundary prompt with different context workloads and models; µ-templates are included in prompt vectors along the context workload in both T3 and T4, whereas T4 also includes additional context from references to previous flash-flood events and SAR resources involved. The workload expands to 7645 tokens and then to 20555 tokens, thereby setting a significantly higher utilization of each model’s context capacity by the incoming context workload. 25
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Table 9: Metrics for evaluating LLMX answer yields on single question, treatments T3 and T4. Qwen3 8b
ChatGPT OSS-20b
Nemotron3 super-120b-a12b
Kimi K2.5
Claude Sonnet4.6
T3 & T4 Context workloads
T3
T4
T3
T4
T3
T4
T3
T4
T3
T4
Answer [Tk]
1285
1245
1808
2095
1997
2425
3189
2964
3897
3379
Utilization [%]
27,25
66,53
28,85
69,12
14,71
35,06
4,23
9,19
1,15
2,39
Time [min: sec]
8:57
8:42
0:15
0:26
0:20
2:49
0:25
0:23
1:39
1:30
Model type/ Length [lines]
Cmp 60
*
Cmp 107
*
Cmp 118
*
Cmp 140
*
Cmp 132
*
Cmp 183
*
Cmp 240
*
Cmp 191
Cmp 262
*
Element type/ count [:n]
Unit 24
Unit 33
Unit 34
Unit 35
Unit 51
Unit 58
Unit 73
Unit 66
Unit 71
Unit 66
Flow type/ count [:n]
Fw-Fu 21
Fw-Fu 36
Fw-Fu 32
Fw-Fu 49
Fw-Fu 70
Fw-Fu 70
Fw-Fu 109
Fw-Fu 82
Fw-Fu 108
Fw-Fu 74
Subelements/ count [:n]
Port 23
Port 29
Port 28
Port 30
Port 46
Port 53
Port 66
Port 59
Port 68
Port 55
Syntax misses [Types/:n]
none
none
Fl: 8
Pre: 1
none
none
none
Fl: 2
none
none
Element misses [Types/:n]
16
13
13
9
2
none
none
none
none
none
Mismatches [Types/:n]
none
none
none
none
Ph: 3 Dup: 8
none
none
none
none
none
Cmp 217
Legend. Model type: Cmp – component diagram; * Cmp – packaged components (components grouped across multiple subsystem packages); Cls – class diagram. Element type: Sub – subsystem element; Unit – unit element; Spec – named property/rating; Port – connection in code; Port* – connection in element. Flow type: Fu – function; Ct – containment; Dv – deliverable; Fw – port-matching flow; Fu-Ct – mixed function/containment; n/a – unlabeled flows. Counts: length in code lines; Element/Subelement/Flow counts as [:n]. Audit. Syntax misses are render-blocking errors (those requiring a syntax-only Corr1 pass): Fl – flow statement, Pt – port statement, El – declaration. Element misses counted strictly vs the SAR reference checklist (per fine-grained element, of 32). Mismatches: Orph – declared element, no flow; Dang – declared, unwired; Dir – direction/port error; Ph – flow to/from undeclared id.
Processing time also differs sharply, with local/smaller models taking longer in this setup (up to 08:21) while larger cloud models respond faster (down to 0:30). The trends between local and cloud models are consistent with the overall performance patterns observed, yet the presence of examples and references seemed to enable better performance to timing of local models. Answer size is somehow correlated with the parameter complexity of the models in use, with Qwen3-8b yielding 1300 tokens and the frontier models, Kimi and Claude yielding 3000 tokens.Model-output structure becomes richer in larger responses, as seen in line counts (31 to 195) and flow counts (21 to 109), suggesting greater elaboration of system interactions and interconnections in larger models - the interdependencies between component functions may appear more pronounced from larger models. Overall, the results suggest that higher-capacity models improve output depth and structural detail, while all models remain lightly loaded in context usage for this experiment. At the same time, fromtier models manage to yield more complete and accurate representations of the system components, ports, and flows, and the results from the Table show that they perform better at leveraging context workload onto complying with the requirements in Table 6. The most significant gain in model quality is observed when µ-Templates are used in conjunction with policy and prompt vector ancillaries (T3), as compared to the baseline treatments (T1 and T2). Then, the lost-in-the-middle effect limits the effectiveness of the additional references in T4, as the model’s attention will concentrate on the policy by the beginning of the workload and on the µ-Templates along the core question by the end of the workload. There are significant effects from treatments regarding element misses and port/flow mismatches from LLMX yields. Element misses were consistently high on smaller LLMs, and policy and guidelines along the question (T2) made component misses to increase. Then, µ-Templates as prompt vectors practically annulled component misses in (T3) and (T4) treatments. 26
Context operations with LLMs and evaluation criteria for systems engineering
8.3
A P REPRINT
Discussion of results
This paper presented modelling samples towards architecting a heavy SAR UAV system, a set of requirements and associated assertions on which the models are evaluated, and results from a quantitative analysis over the generated models. This case study experiment involved our running 20 queries spanning context workloads and selected LLMs that were requested to generate system architecture plantUML codeblock models. The overall modelling answer corpus evolved from 20 treatment answer results to 55 individual model files, by counting the answer files plus the spawned models from the answer codeblocks upon the corrections required by the eval assertions. The analysis part involved setting up modelling metrics to verify the output of the LLMs; here, we focus element count and syntax requirements for the verification.
Figure 13: AI-generated Heavy SAR UAV rendering (source: Gemini). The whole dataset reveals the effect of the primacy and recency, characteristics of information units within a context workload to an LLM that were revealed by Liu et al. [2024] upon finding out on the limitations of LLMs under high utilization of their context length capacity. Here, smaller models such as Qwen3-8b (M1) and GPT-OSS-20b (M2), both with maximum 32767 Tk of context capacity, were significantly utilized – especially in treatments T3 and T4 with including µ-Template exemplars. These effects have something to do with context units that precede the question having a divergence effect, and context units that are within the question, which actually succeed the query statement – that carries the actual request and thereby the modelling intent – and make both convergence and enforcement effects to expected patterns in both coding and modelling. This means that the effects of primacy and recency as discussed by Liu et al. [2024] manifest themselves when the context workload in single question becomes significant towards context length capacity. While the performance of frontier models such as Kimi K2.5 (M4) and Claude Sonnet 4.6 (M5) in modelling architectures with support of µ-Templates – and that of similar-scale models such as Deepseek and ChatGPT-5.x from previous experience of ours in Marini and Krus [2025] – bears no remarks upon their higher scale and longer context capacities, µ-templates have a convergence effect to a desired model setup regarding its format, structure and compliance. In models such as Qwen3, GPT-OSS and Nemotron, the effects of context operations deserve attention. Their performance comes not without their responses coming briefly, either with subcomponents declared within component blocks – which deviate from the purpose of architecting – or with omitting components entirely yet complying with code syntax and modelling conventions. Here, one expects a topology of components, effect flows and ports that supports component-level specification and designing with mind to later integration tasks. At the same time, such responses can be fixed to completeness, accuracy and compliance with a few minutes of juggling model code. One significant matter of attention to be considered is the effect of different units onto the workload being forwarded to the LLMs. The workload units are the policy, the prompt vectors, the references and the question itself. The policy units are intended to set up a context for the modelling request, and they are expected to be processed by the LLMs as a primacy effect. The prompt vectors are intended to set up a context for the modelling request, and they are expected to be processed by the LLMs as a recency effect. 27
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Here, references in the **middle** of the context workload are expected to be processed by the LLMs as a convergence effect, yet their effectiveness is limited by the lost-in-the-middle effect. Then, this means that policy and modelling directives, as well as examples, are the key elements on which the LLMs focus their attention to produce a modelling result. Nevertheless, in-context learning could be a powerful technique regarding the build-up of reference for modelling requests, whose usefulness goes down to sorting out the token economics over searching through a knowledge database. The use of evaluation assertions to verify the modelling results from LLMs is a key element onto enabling the usefulness of a modelling corpus onto the development of a learning database for systems modelling with generative artificial intelligence. Along with context handling, evaluations enable setting convergence criteria for modelling loops and ultimately enable the potential for the use of hybrid deterministic-probabilistic modelling approaches to systems design, such as proposed by Krus [2025] when requesting a deterministic model builder algorithm from a context prompt. Understanding the capabilities of LLMs of a scale spectrum from giga to tera-scale enables proceeding to investigate and take advantage of novel context handling techniques to improve the capability of artificial intelligence onto systems modelling to design intent.
9
Conclusions
This contribution has presented a framework of context operations on which to prompt at large language models. Being implemented in a chatbox applicaiton as that shown by Marini and Krus [2025], this framework was successfully operated towards a case study of aircraft design, here being a heavy SAR UAV. This case study contributes with evolving from modelling examples towards the experimentation with modelling requests, and the verification of their modelling results through measuring their architectural properties. The evaluation of modelling quality by large language models from the synthesis of modular context workloads enabled us to discuss on the modelling capability of LLMs under a single query. The modelling results presented from the experiment are on par with current understanding about the workings of LLMs to system design. At the same time, these modelling results enable the development of a learning database on how to support systems modelling with generative artificial intelligence. Future work involves expanding the modular workload approach to conversation threading and agentic modelling, which in turn requires the expansion of the learning base to fine-tune model counting and validation rules.
10
Acknowledgments
This study has been carried out with the support of the CNPq-CISB grant no. 200944/2024-0 of the Brazilian National Council for Scientific and Technological Development (CNPq) and the Swedish-Brazilian Centre for Innovation, through collaborative research work carried out by the authors at the Federal University of Santa Maria (UFSM) in Brazil and at the Linköping University (LiU) in Sweden. The authors manifest their gratitude in advance to reviewers for their valuable comments and suggestions, which help improve the quality of this paper.
References Mary L. Cummings. Automation bias in intelligent time critical decision support systems. In : 1st AIAA Intelligent Systems Technical Conference. Chicago, IL: American Institute of Aeronautics and Astronautics, 2004. doi:10.2514/6.2004-6313. Martin Törngren and Paul T. Grogan. How to deal with the complexity of future cyber-physical systems? Designs, 2(4): 40 pp., 2018. ISSN 2411-9660. doi:10.3390/designs2040040. Paul T Grogan. Perception of Complexity in Engineering Design. Systems Engineering, 24(4):221–233, 2021. ISSN 1520-6858. doi:10.1002/sys.21574. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. URL https://proceedings.neurips.cc/paper_files/ paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html. 28
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract.html. Petter Krus. Large language model in aircraft system design. In : 34th Congress of the International Council of the Aeronautical Sciences, ICAS 2024, 2024. URL https://www.icas.org/icas_archive/icas2024/data/ papers/icas2024_0514_paper.pdf. Brian Johns, Kristina Carroll, Casey Medina, et al. AI systems modeling enhancer (AI-SME): Initial investigations into a chatgpt-enabled MBSE modeling assistant. INCOSE International Symposium, 34(1):1149–1168, 2024. ISSN 2334-5837. doi:10.1002/iis2.13201. Timm Teubner, Christoph M Flath, Christof Weinhardt, Wil Van Der Aalst, and Oliver Hinz. Welcome to the era of ChatGPT et al. Business & information systems engineering, 65(2):95–101, 2023. ISSN 1867-0202. doi:10.1007/s12599-023-00795-x. Mallory S. Graydon and Sarah M. Lehman. Examining proposed uses of LLMs to produce or assess assurance arguments. Technical memorandum, NASA/TM-2025-0001849, National Aeronautics and Space Administration, March 2025. URL https://ntrs.nasa.gov/api/citations/20250001849/downloads/NASA-TM-20250001849.pdf. Alejandro Pradas-Gomez, Petter Krus, Massimo Panarotto, and Ola Isaksson. Large language models in complex system design. In : DESIGN 2024 International design conference, volume 4, pages pp. 2197–2206. Cavtat, Croatia: Design Society, 2024. doi:10.1017/pds.2024.222. Vinicius K. Marini, Jens Alfredson, and Petter Krus. Context of collaborative human-machine systems architecture design for enhanced functionality awareness and balanced command and control authority. In Proceedings of the 12th Swedish Aersopace Technology Congress - FT2025. Stockholm, Sweden: Swedish Society for Aeronautics and Astronautics (FTF), 2025. doi:10.3384/ecp215.1192. Javier Cámara, Javier Troya, Lola Burgueño, and Antonio Vallecillo. On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML. Software and Systems Modeling, 22(3):781–793, June 2023. ISSN 1619-1374. doi:10.1007/s10270-023-01105-5. John K. DeHart. Leveraging large language models for direct interaction with SysML v2. INCOSE International Symposium, 34(1):2168–2185, 2024. ISSN 2334-5837. doi:10.1002/iis2.13262. Louis Richard Timperley, Lucy Berthoud, Chris Snider, and Theo Tryfonas. Assessment of large language models for use in generative design of model based spacecraft system architectures. Journal of Engineering Design, 36(4): 550–570, 2025. ISSN 0954-4828. doi:10.1080/09544828.2025.2453401. Balahari Vignesh Balu, Florian Geissler, Francesco Carella, et al. Towards automated safety requirements derivation using agent-based RAG. Proceedings of the AAAI Symposium Series, 5(1):299–307, May 2025. doi:10.1609/aaaiss.v5i1.35605. Fabian Hanke, Isaac Mpidi Bita, Oliver von Heißen, Weller Julian, Hovemann Aschot, and Dumitrescu Roman. AI-augmented systems engineering: conceptual application of retrieval-augmented generation for model-based systems engineering graph. Proceedings of the Design Society, 5:439–448, 2025. doi:10.1017/pds.2025.10058. Ali Nouri, Beatriz Cabrero-Daniel, Fredrik Törner, et al. Engineering safety requirements for autonomous driving with large language models. In 2024 IEEE 32nd International Requirements Engineering Conference (RE), pages 218–228. IEEE, 2024. doi:10.1109/RE59067.2024.00029. Ibtissam El Hassani, Tawfik Masrour, Nouhan Kourouma, Damien Motte, and Jože Tavčar. Integrating large language models for improved failure mode and effects analysis (fmea): a framework and case study. Proceedings of the Design Society, 4:2019–2028, 2024. doi:10.1017/pds.2024.204. Ibtissam El Hassani, Tawfik Masrour, Nouhan Kourouma, and Jože Tavčar. AI-driven FMEA: integration of large language models for faster and more accurate risk analysis. Design Science, 11:e10, 2025. ISSN 2053-4701. doi:10.1017/dsj.2025.7. Yi Qi, Xingyu Zhao, Siddartha Khastgir, and Xiaowei Huang. Safety analysis in the era of large language models: A case study of STPA using ChatGPT. Machine Learning with Applications, 19:no. 100622, 2025. ISSN 2666-8270. doi:10.1016/j.mlwa.2025.100622. 29
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Zezhong Chen, Yuxin Deng, and Wenjie Du. Trusta: Reasoning about assurance cases with formal methods and large language models. Science of Computer Programming, 244:103288, 2025. ISSN 1872-7964. doi:10.1016/j.scico.2025.103288. Carlo Lipizzi. From Text to Structure: Extracting and Validating Complex System Representations Using Large Language Models. In DS 141: Proceedings of the 27th International DSM Conference (DSM 2025), pages 145–153. Hoboken, NJ, USA: the Design Society, 2025. URL https://www.designsociety.org/publication/48684/from_text_to_structure_extracting_and_ validating_complex_system_representations_using_large_language_models. Edwin Koh. Retrieving Asymmetrical Indirect Links Through Large Language Models. In DS 141: Proceedings of the 27th International DSM Conference (DSM 2025), pages 5–8. Hoboken, NJ, USA: the Design Society, 2025. URL https://www.designsociety.org/download-publication/48688/retrieving_ asymmetrical_indirect_links_through_large_language_models. Petter Krus. Augmenting aerospace system design using large language models. In Proceedings of the 12th Swedish Aersopace Technology Congress - FT2025. Stockholm, Sweden: Swedish Society for Aeronautics and Astronautics (FTF), 2025. doi:10.3384/ecp215.1197. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the association for computational linguistics, 12:157–173, 2024. doi:10.1162/tacl_a_00638. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. ArXiv: 2306.15595, 2023. doi:10.48550/arXiv.2306.15595. Mihir Gupte, Eshan Dixit, Muhammad Tayyab, and Arun Adiththan. What works for ’lost-in-the-middle’ in llms? a study on gm-extract and mitigations. ArXiv: 2511.13900, 2025. doi:10.48550/arXiv.2511.13900. Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, and Zhangyang Wang. Found in the middle: How language models use long contexts better via plug-and-play positional encoding. ArXiV: 2403.04797, 2024. doi:10.48550/arXiv.2403.04797. Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models. ACM Transactions on Intelligent Systems and Technology, 16(5):1–72, 2025. doi:10.1145/3744746. Chuyifei Zhang, Hongyu Cui, Xiaowen Huang, and Jitao Sang. Positional failures in long-context llms: A blind spot in reasoning benchmarks. ArXiv: 2605.23170, 2026. doi:10.48550/arXiv.2605.23170. George Arthur Baker, Ankush Raut, Sagi Shaier, Lawrence E Hunter, and Katharina von der Wense. Lost in the middle, and in-between: Enhancing language models’ ability to reason over long contexts in multi-hop qa. ArXiV: 2412.10079, 2024. doi:10.48550/arXiv.2412.10079. URL https://arxiv.org/abs/2412.10079. Warren Li, Yiqian Wang, Zihan Wang, and Jingbo Shang. Order matters: Rethinking prompt construction in in-context learning. ArXiv: 2511.09700, 2025. doi:10.48550/arXiv.2511.09700. Qi Guo, Leiyu Wang, Yidong Wang, Wei Ye, and Shikun Zhang. What makes a good order of examples in in-context learning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 14892–14904, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi:10.18653/v1/2024.findings-acl.884. Erin Crabb and Matthew T. Jones. Accelerating model-based systems engineering by harnessing generative AI. In 2024 19th Annual System of Systems Engineering Conference (SoSE), pages 110–115, 2024. doi:10.1109/SOSE62659.2024.10620975. Florian Geissler, Karsten Roscher, and Mario Trapp. Concept-guided LLM agents for Human-AI safety codesign. Proceedings of the AAAI Symposium Series, 3(1):100–104, May 2024. doi:10.1609/aaaiss.v3i1.31188. Simon Dehn, Simon Schnürer, Georg Jacobs, and Gregor Höpfner. Generating sysml v2 models from natural language requirements using large language models. In 2025 IEEE International Symposium on Systems Engineering (ISSE), pages 1–7. IEEE, 2025. doi:10.1109/ISSE65546.2025.11369988. 30
Context operations with LLMs and evaluation criteria for systems engineering
A P REPRINT
Petter Krus. Using large language models for fluid power system design. JFPS International Journal of Fluid Power System, 19(2):74–79, 2026. doi:10.5739/jfpsij.19.74. Oliver Von Heissen, Fabian Hanke, Isaac Mpidi Bita, et al. Toward intelligent generation of system architectures. Proceedings of NordDesign 2024, pages 504–513, 2024. URL https://www.designsociety.org/publication/ 47646/toward_intelligent_generation_of_system_architectures. Vinicius Kaster Marini and Petter Krus. Synthesizing aircraft system specifications with document-managed Large Language Model outputs from one-shot system inquiry prompt chain. In 10th CEAS Aerospace Europe Conference, 28th AIDAA International Congress, 2025. doi:10.21741/9781644904251-99. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, et al. Qwen3 technical report. ArXiv:2505.09388, 2025. doi:10.48550/arXiv.2505.09388. OpenAI. gpt-oss-120b & gpt-oss-20b model card. ArXiv: 2508.10925, 2025. doi:10.48550/arXiv.2508.10925. NVIDIA. Nemotron 3 super: Open, efficient mixture-of-experts hybrid mamba-transformer model for agentic reasoning. ArXiv: 2604.12374, 2026. doi:10.48550/arXiv.2604.12374. URL https://arxiv.org/abs/2604.12374. Ollama. Kimi k2.5 model card. Ollama model card, 2026. URL https://ollama.com/library/kimi-k2.5. Accessed: 2026-06-04. Anthropic. Introducing Claude Sonnet 4.6. Anthropic Product Announcement, February 2026. URL https://www. anthropic.com/news/claude-sonnet-4-6.
31