arXiv:2604.08293v1 [cs.SE] 9 Apr 2026
CIAO - C ODE I N A RCHITECTURE O UT Automated Software Architecture Documentation with Large Language Models Marco De Luca
Tiziano Santilli
Domenico Amalfitano
University of Naples Federico II Naples, Italy [email protected]
University of Southern Denmark Odense, Denmark [email protected]
University of Naples Federico II Naples, Italy [email protected]
Anna Rita Fasolino
Patrizio Pelliccione
University of Naples Federico II Naples, Italy [email protected]
Gran Sasso Science Institute L’Aquila, Italy [email protected]
Abstract—Software architecture documentation is essential for system comprehension, yet it is often unavailable or incomplete. While recent LLM-based techniques can generate documentation from code, they typically address local artifacts rather than producing coherent, system-level architectural descriptions. This paper presents a structured process for automatically generating system-level architectural documentation directly from GitHub repositories using Large Language Models. The process, called CIAO (Code In Architecture Out), defines an LLM-based workflow that takes a repository as input and produces system-level architectural documentation following a template derived from ISO/IEC/IEEE 42010, SEI Views & Beyond, and the C4 model. The resulting documentation can be directly added to the target repository. We evaluated the process through a study with 22 developers, each reviewing the documentation generated for a repository they had contributed to. The evaluation shows that developers generally perceive the produced documentation as valuable, comprehensible, and broadly accurate with respect to the source code, while also highlighting limitations in diagram quality, high-level context modeling, and deployment views. We also assessed the operational cost of the process, finding that generating a complete architectural document requires only a few minutes and is inexpensive to run. Overall, the results indicate that a structured, standards-oriented approach can effectively guide LLMs in producing system-level architectural documentation that is both usable and cost-effective. Index Terms—Software Architecture Documentation, Architecture Recovery, Large Language Models, ChatGPT, GitHub Repositories, Automated Documentation
tion is frequently incomplete or informal in industrial and open-source projects, where time pressure and code-centric practices make its production difficult to sustain. Established standards provide guidance on how architectural information should be documented. ISO/IEC/IEEE 42010 defines core concepts and the relationship between architecture descriptions and stakeholder concerns, emphasizing that documentation should explicitly address the needs of its intended readers [5]. SEI’s Views & Beyond framework promotes a view-based, stakeholder-oriented approach in which different views capture complementary architectural structures [6]. In line with these principles, we focus on system-level architectural documentation aimed at developers who require a consolidated, high-level understanding of a repository. Recent years have seen the rapid adoption of Large Language Models (LLMs) in software engineering, with growing evidence of their usefulness in tasks such as program comprehension, code summarization, documentation generation, and automated repair [7]–[10]. Industrial analyses, such as GitHub’s Octoverse report, similarly highlight the increasing reliance on LLM-based assistants to help developers navigate large codebases [11]. Research has explored LLMs for generating fine-grained documentation artifacts, such as API descriptions [12], code explanations [13], test-case summaries [14], and requirement-like statements extracted from source code [15]. Other works explored LLMs for supporting higher-level reasoning, such as extracting UML diagrams, identifying design patterns, or reconstructing domain models from code [16]–[19]. These contributions highlight the potential of LLMs to support documentation and architecture recovery tasks. However, most approaches focus on specific artifacts or narrow tasks. The use of LLMs to produce system-level architectural documentation directly from repositories, following established documentation standards, remains largely unexplored.
I. I NTRODUCTION Software architecture documentation plays a central role in supporting system comprehension, communication, and long-term evolution. When documentation is missing, outdated, or inconsistent with the implementation, developers struggle to understand system decomposition, responsibilities, and dependencies, often leading to architectural drift, erosion, and ultimately architectural technical debt [1]–[4]. Despite its recognized importance, architectural documenta-
This paper addresses this gap by introducing CIAO (Code In Architecture Out), a structured process for automatically generating system-level architectural documentation from GitHub repositories using LLMs. CIAO defines a workflow that takes a repository as input and produces an architectural documentation following a template, which has been defined by taking inspiration from ISO/IEC/IEEE 42010, SEI Views & Beyond, and the C4 model [20]. The final output is an architectural documentation, which can be directly integrated into the target repository as a README file. Our study evaluates CIAO with 22 developers who analyzed the documentation generated for repositories they contributed to. The questionnaire addresses perceived value (RQ1), comprehensibility (RQ2), accuracy (RQ3), and identified limitations (RQ4). We also measure the generation time and computational cost (RQ5). The main contributions of this work are as follows: A standards-oriented template for system-level architectural documentation, based on ISO/IEC/IEEE 42010, SEI Views & Beyond, and the C4 model. • A structured LLM-based workflow that generates architectural documentation directly from GitHub repositories. • An open-source prototype implementing the proposed workflow, capable of producing ready-to-use architectural documentation that can be directly integrated into the target repository. •
The remainder of this paper is organized as follows. Section II reviews related work. Section III presents our documentation process. Section IV describes the experimental evaluation, and Section V reports the results. Section VI discusses the threats to validity. Finally, Section VII concludes the paper and outlines future work. The supplementary material, which includes the CIAO implementation, the generated documentation, and the full survey structure and results, is available at the following link: https://doi.org/10.5281/zenodo.18710540 II. R ELATED W ORK Reverse engineering comprises techniques for reconstructing the structure, behavior, and design intent of software systems from low-level artifacts such as source code and execution traces [21]. It is particularly valuable when documentation is missing or outdated, helping developers regain understanding of system decomposition, component relationships, and design decisions [22]. A central branch of this field is Software Architecture Recovery (SAR), which focuses on rebuilding the architectural organization of a system by identifying its main components, dependencies, and structural patterns [23], [24]. SAR is particularly relevant when architectural drift or erosion causes the implemented architecture to diverge from the intended one [1], [2], [4], [25]–[27]. When these inconsistencies are not addressed in architectural descriptions, they become architectural debt [3]. Existing SAR approaches include static analysis, dynamic analysis, and techniques using ranking, clustering, or machine learning to infer architectural boundaries [28]–[30]. Across these techniques, the goal is to
provide maintainers with a coherent architectural view that is otherwise implicit in large codebases. The rapid adoption of LLMs in software engineering has led to multiple documentation-related approaches that vary in input artifacts, abstraction level, and produced outputs. LLMs have been used for summarization and automated repair [9], [10], [31]–[33], with prompt-driven interaction enabling flexible, natural-language guidance [34]. A first group of studies uses source code as input to derive naturallanguage documentation or higher-level descriptions. Applications include extracting REST API endpoints for microservice systems [12], generating context-aware explanations during program comprehension [13], and producing requirement-like statements through the AUTO R EQ G EN pipeline [15]. Other contributions focus on code fragments, such as intent-oriented summaries [35] or explanations of legacy languages [36], and on test artifacts, where LLMs generate concise summaries to support navigation [14]. Recent studies have investigated whether LLMs can support higher-level reasoning, such as identifying design patterns, generating UML diagrams, or reconstructing domain models [17], [19], [37]. Hybrid approaches combine static analysis with LLM reasoning to recover structural views [16], while other studies compare LLM-based extraction with traditional MDRE techniques [38]. MDRE-LLM leverages RAG techniques to recover domain models at different granularities [18]. Additional work has explored architectural reasoning from requirements or code in microservice settings [39]–[41]. LLMs have also been used to generate textual descriptions from UML Use Case Diagrams [42] or to support modeling tasks in educational contexts [43]. A third line of work explores how LLMs can enrich or review existing documentation. Examples include enhancing API documentation with usage examples [44], tailoring documentation to different stakeholders [45], and supporting documentation review processes [46]. A recent contribution closely related to our work is CodeDocs-GenAI [47], which uses RAG and LLM-based summarization to generate README documentation for Git repositories. Like CIAO, it analyses the entire repository, including its structure and source code, to generate structured documentation. However, its focus remains on general-purpose project overviews rather than system-level architectural documentation grounded in established standards and guidelines. Overall, prior work shows that LLMs have been applied to many documentation-related tasks, mostly targeting specific artifacts such as APIs, code snippets, tests, requirements, or individual diagrams. By contrast, their use for full architecture recovery and end-to-end architectural documentation directly from source code remains largely unexplored. Our work addresses this gap by operating at a higher level of granularity: starting from a repository, we generate comprehensive architectural documentation across multiple views, stakeholders, and concerns within a unified template, extending the role of LLMs from local documentation support to system-level architectural documentation.
III. T HE D OCUMENTATION P ROCESS In this section, we present the automated documentation process implemented in CIAO, which generates system-level architectural documentation from GitHub repositories using an LLM. The process is built around a system-level architectural template that is lightweight enough to be provided as input to the LLM while remaining consistent with established standards and practitioner guidelines. Section III-A introduces the template and its expert validation, Section III-B describes the overall workflow, Section III-C details the prompt-engineering strategy, and Section III-D discusses the selection of the underlying LLM. A. The Proposed Template Software architecture documentation is commonly described as view-based [48]: the information to be presented and the appropriate level of abstraction should depend on stakeholders’ concerns and the architectural perspectives to be supported. In this work, we focus on system-level architectural documentation aimed at developers who need a consolidated, highlevel understanding of a repository to work with and evolve the system. The template must therefore remain aligned with established architectural practices while being lightweight and regular enough to be used as input to an LLM. Template Design. To determine which architectural information should be included and how it should be organized and presented, we adopted an expert-driven iterative design process. Three experts participated in this phase: one software engineering researcher with ten years of experience in architecture and documentation, and two industry software architects with five years of experience each. Through three separate focus groups, we elicited the architectural elements they deemed essential at system level. The sessions converged on a common set of needs: a clear definition of system scope and external context, a representation of deployable units and their responsibilities, an internal structural view of modules and their relationships, explicit links to code artifacts, and attention to cross-cutting concerns and quality-related rationale. The resulting synthesis was consolidated by two authors and validated by a third. These needs were then organized into a coherent structure drawing on three well-established sources. ISO/IEC/IEEE 42010 [49] guided the inclusion of concepts such as system scope, stakeholder concerns, external interactions, and architectural rationale. SEI’s Views & Beyond [48] informed the separation between structural, behavioral, and deployment perspectives. The C4 model [50], aligned with the Architecture-as-Code paradigm [26], provided a four-level hierarchy, Context (L1), Container (L2), Component (L3), and Code (L4), that is widely used for structuring developer-oriented documentation [51]. A revised draft was then discussed in a second round of focus groups, and a final joint session with all experts was used to resolve remaining ambiguities and validate the structure. The resulting template comprises eight sections, each addressing a distinct architectural concern and drawing consis-
tently on ISO/IEC/IEEE 42010, SEI’s Views & Beyond, and the four abstraction levels of the C4 model (Context, Container, Component, Code): 1) System Overview. Provides an entry point to the architectural description by summarizing the system’s purpose, scope, and main responsibilities. It establishes the systemof-interest, in line with ISO 42010, and offers a conceptual anchor for the architectural views that follow, helping readers quickly situate the repository before moving to more technical details. 2) Architectural Context. Characterizes the system’s external environment by identifying actors, interacting systems, APIs, and data sources. By clarifying boundaries and external dependencies, this section supports ISO’s emphasis on contextualizing the architecture and corresponds to the C4 Level 1 Context view. It provides the background necessary to understand integration points and the role of the system within a broader ecosystem. 3) Containers. Describes the system’s logical runtime organization, following the notion of “containers” in the C4 model, namely, the applications and data stores that must be running for the system to operate. For each container, the documentation summarizes responsibilities, exposed interfaces, key technologies, and interaction patterns. This section offers a coarse-grained view of how major building blocks collaborate at runtime, aligning with SEI’s Component-and-Connector perspective and ISO’s runtime-oriented concerns. 4) Components. Presents the internal logical structure of the system by identifying key modules, packages, or classes and the structural relationships among them. This section provides a technology-independent representation of architectural organization, complementing the container perspective by revealing how domain responsibilities are grouped and how subsystems interact. It aligns with SEI’s Module viewtype and corresponds to C4 Level 3. 5) Code-Level. Connects architectural abstractions to their concrete implementation by mapping components to source-code artifacts. It identifies relevant directories, files, entry points, and recurring design or architectural patterns. By making the realization of architectural elements explicit, this section supports traceability to implementation as recommended by ISO 42010 and corresponds to C4 Level 4. 6) Cross-Cutting Concerns. Summarizes concerns that influence multiple parts of the system, such as security, configuration, logging, testing, and monitoring, and describes how they manifest in the codebase. These aspects are critical to understanding system behavior beyond structural decomposition and align with SEI’s “beyondthe-views” guidance and ISO’s focus on capturing stakeholder concerns that transcend individual elements. 7) Quality Attributes and Rationale. Highlights the quality attributes (e.g., performance, maintainability, scalability, security) supported by the implementation and
synthesizes the rationale inferred from observable design choices. This section implements ISO 42010’s recommendation to document architectural rationale and supports readers in understanding why certain architectural decisions were made. 8) Deployment. Characterizes the system’s operational infrastructure by describing deployment artifacts (e.g., Dockerfiles and configuration files), execution environments, storage and compute nodes, and their relationships. It shows how software elements map to physical, virtual, or containerized resources, aligning with the SEI Allocation perspective and C4 deployment practices, and clarifies how the system runs in practice and how runtime responsibilities are distributed. B. CIAO Workflow CIAO is implemented as a Python-based tool that automatically generates system-level architectural documentation from GitHub repositories using LLMs. Figure 1 summarizes the end-to-end workflow. The process takes two primary inputs: (i) the target GitHub repository, which provides the source artifacts to be analyzed, and (ii) the documentation template, which defines the structure of the architectural document, the goal of each section, and the global writing guidelines. Based on these inputs, CIAO performs four automated steps: repository flattening, prompt generation, section generation, and final assembly with diagram rendering.
Fig. 1. Overview of the CIAO automated documentation workflow.
Repository Flattening: The repository is converted into a single textual artifact using R EPOMIX [52], which aggregates the project’s source code into one AI-friendly file. R EPOMIX can be configured to apply different filters; in our case, we remove comments and exclude files or directories that do not contain source code (e.g., binaries, build outputs, large test datasets, generated documentation), while retaining essential configuration artifacts (such as Dockerfile, container-orchestration descriptors, and dependency manifests like package.json or pom.xml). We also enable the option to enrich the output with a textual description of the repository structure. The resulting Flattened Repository representation captures both the project’s folder layout and the curated code base, and serves as input for the subsequent steps. Prompt Generation: For each documentation section, CIAO builds, in parallel, a composite prompt consisting of two parts: (i) a fixed global prompt, shared across all sections, and (ii)
a section-specific prompt, derived from the Documentation Template. Moreover, the Flattened Repository representation is appended to each prompt to ensure code-grounded generation. LLM-based Section Generation: For each prompt generated in the previous step, CIAO submits it to the LLM, processing all sections in parallel. This step produces the set of Generated Sections, each corresponding to a specific part of the template and adhering to its prescribed structure and constraints. Documentation Assembly: The Generated Sections produced in the previous step are assembled into a single document, referred to as the Intermediate Documentation. This yields a complete system-level architectural description that follows the structure defined by the template. Diagram Rendering: Since LLM-generated diagrams are provided in textual PlantUML format [53], CIAO renders them as images and replaces the textual definitions accordingly. Applied to the Intermediate Documentation, this step produces the final Generated Documentation, in which all diagrams are available in visual form. C. Prompt Engineering CIAO employs a structured prompt-engineering strategy to maximize architectural accuracy, limit hallucinations, and ensure consistency across sections. Each prompt consists of (i) a global prompt, shared across all sections, which defines the LLM’s role, target audience, writing style, and grounding requirements; and (ii) a section-specific prompt, instantiated from the documentation template and tailored to the goal and expected artifacts of that section. This separation reflects the fact that architectural documentation comprises heterogeneous elements that require different abstraction levels and extraction strategies. This design follows the task-decomposition prompting strategy proposed by Liu et al. [54], guiding the LLM to address a sequence of smaller, section-specific subtasks rather than generating the entire documentation in a single step. CIAO further follows Liu et al.’s taxonomy [54] by adopting both profile and instruction prompting, specifying who the model should act as and how it should perform the task. Along the profile dimension, role prompting [55] positions the model as a “Meticulous Software Architect”, helping to stabilize tone, terminology, and stylistic coherence across sections. Additionally, light motivational cues [56] encourage careful reasoning. On the instruction side, prompts specify evidencegrounded generation requirements and explicitly forbid inventing architectural elements not present in the repository. Selective few-shot examples [57] (e.g., small Markdown or PlantUML skeletons) provide structural scaffolding that guides formatting and level of detail. This combination supports coherent, template-aligned, and verifiable architectural documentation. The complete prompt is available in the supplementary material. D. Model Selection To select the LLM used in CIAO, we conducted an exploratory pilot study comparing four state-of-the-art models: GPT-5, Claude Sonnet 4.5, Gemini 2.5, and
Mistral Large 2. For three representative repositories provided by developers familiar with the projects, we generated documentation with each model and discussed the outputs in dedicated focus groups. The evaluation considered (i) accuracy of architectural elements and diagrams, (ii) consistency of terminology and relationships across sections, (iii) the presence of hallucinated or speculative content, and (iv) adherence to the template. Across models, GPT-5 consistently produced the most accurate, stable, and template-aligned documentation, particularly in sections requiring multi-level structural reasoning. It also yielded more syntactically correct P LANT UML diagrams and fewer speculative elements. Based on these observations, GPT-5 was selected as the default model in CIAO, although the tool remains configurable to support alternative cost–quality trade-offs. IV. E XPERIMENTAL E VALUATION The goal of this study is to evaluate the effectiveness of the proposed LLM-based process in generating system-level architectural documentation directly from source code while adhering to our standards-oriented template. We assess the perceived value, comprehensibility, and accuracy of the generated documentation, as well as the limitations and missing aspects identified by practitioners. To this end, we conducted a surveybased evaluation in which software developers provided realworld repositories and reviewed the documentation produced by our tool. A direct comparison with related approaches was not feasible, as the only closely related work, i.e., CodeDocsGenAI [47], does not provide an openly accessible implementation. The validation aims to answer the following Research Questions (RQs): RQ1: Do developers perceive the documentation as valuable enough to be integrated into their own projects? RQ2: To what extent is the documentation comprehensible? RQ3: To what extent is the documentation accurate with respect to the system’s source code? RQ4: What limitations or missing aspects do developers identify in the documentation? RQ5: What are the costs of generating the documentation? Metrics. To answer our research questions, we rely on two complementary sets of measurements. For RQ1–RQ4, we base our evaluation on the metrics derived from a questionnaire that we designed for this study. The questionnaire includes both Likert-scale items and open-ended questions, and the corresponding items for each research question are described in the following sections. For RQ5, we assess the cost of the documentation-generation process by measuring two quantitative indicators: (i) the time required to generate the complete architectural documentation for each repository, and (ii) the financial cost associated with the model’s API usage. These measurements allow us to evaluate both the technical and economic impact of the process. Questionnaire Design. To address our research questions, we constructed a questionnaire composed of closed-ended items measured on a 5-point Likert scale (1 = strongly disagree (SD), 2 = disagree (D), 3 = neutral (N), 4 = agree (A), 5 = strongly
agree (SA)) and open-ended questions designed to elicit richer qualitative insights. The introductory section collects background information about each participant and their relationship with the evaluated repository. Specifically, it records job role, application domain (e.g., web application, automotive, healthcare), prior familiarity with the codebase, and whether the participant had previously authored or maintained any repository documentation (e.g., READMEs, architectural descriptions, wiki pages, ADRs). When applicable, participants briefly described the type of documentation they had produced.This information helped contextualize their assessments and interpret perception differences across levels of experience and prior knowledge. The second part of the questionnaire is organized around the four research questions. Table II summarizes all items, including identifiers (Q1–Q30), wording, and type. Perceived value (RQ1) is examined through three Likert-scale items and one open-ended question (Q1–Q4), probing the usefulness and perceived contribution of the generated documentation. Comprehensibility (RQ2) is evaluated through four Likertscale items and one open-ended question (Q5–Q9) focusing on clarity, structure, terminology, and redundancy. Accuracy and consistency (RQ3) are assessed using the largest set of items (Q10–Q27), which combine global questions on architectural correspondence, correctness of responsibilities and relationships, and internal consistency with section-level items aligned with the proposed template (e.g., Containers, Components, Code-Level, Use Case Diagram, and Deployment Diagram), followed by an open-ended prompt for additional comments. Limitations and missing aspects (RQ4) are captured through three open-ended questions (Q28–Q30) targeting structural or conceptual weaknesses, missing information, and potential improvements. The questionnaire was implemented in Google Forms, which was used to collect all participants’ responses. Participants and Repositories. Participants were recruited through convenience sampling from the authors’ professional and academic networks. A total of 22 developers agreed to take part in the study by contributing one of their software repositories and completing the survey after reviewing the automatically generated documentation. All participants had prior software development experience and were familiar with the repositories they submitted, ensuring knowledgeable and informed evaluations. The sample consists of 15 researchers, 5 PhD students, and 2 software developers. Most respondents reported substantial development experience: 17 indicated more than five years of practice, while only two reported one year or less. Participants also expressed strong familiarity with software architecture concepts, such as UML, architectural styles, and architectural patterns, with an average self-assessed score of 4.45/5. Regarding their involvement with the analyzed repositories, 15 participants stated that they had authored or maintained documentation (e.g., READMEs, architecture descriptions, or user guides). Their free-text responses referenced onboarding materials, installation guides, usage documentation, and high-level architectural overviews. Overall, the sample reflects practitioners and researchers with solid soft-
TABLE I OVERVIEW OF ANALYZED REPOSITORIES WITH LANGUAGES AND CODE SIZE . Repo Group-Key-Phemapn [58] pyALS-RF-tmr [59] sa-bsn [60] ocean-lib [61] robethichor [62] Openjob [63] DSP-Frontend [64] DSP-Backend [65] QuFI [66] BoMoDT [67] m2dt [68] GymportalService [69] pyicub [70] treeco [71] TestingRobotChallenge-T4 [72] rse2025 [73] write2audiobook [74] BF-JITcompiled [75] visual novel pyarcade [76] SAM-CyFra [77] SCASS [78]
Domain IoT Machine Learning Healthcare Machine Learning Robotic Systems Web Application Cybersecurity Cybersecurity Quantum computing Digital Twins Digital Twins Web Application Healthcare Robotics Machine Learning Web Application Web Application Accessibility tools JIT Compiler Visual novel engine Network Security Cybersecurity
OSINT [79]
Cybersecurity
Lang. (LOC %) Files LOC C (17.0%), C++ (83.0%) 9 1061 C (0.9%), C++ (0.3%), CMake (1.6%), JSON (0.7%), Python (95.8%), Shell (0.7%) 85 9270 C (12.6%), C++ (75.5%), CMake (4.1%), Python (4.5%), Shell (0.5%), XML (2.9%) 175 9877 Python (80%), YAML (20%) 33 1208 CMake (1.7%), Dockerfile (1.6%), JSON (3.7%), Python (88.1%), XML (4.9%) 28 697 Dockerfile (0.6%), Java (65.3%), XML (32.3%), YAML (1.9%) 146 5386 CSS (10.4%), JS (88.7%), XML (0.9%) 447 238951 JS (100%) 38 6870 Python (100.0%) 8 851 CSS (0.1%), JSON (0.9%), Python (9.7%), XML (89.0%), YAML (0.4%) 55 50928 JSON (0.2%), Python (86.1%), XML (13.7%) 36 4040 Java (97.7%), XML (2.3%) 35 2458 JSON (31.2%), Python (64.3%), Shell (2.3%), XML (1.2%), YAML (0.9%) 123 8441 Python (100.0%) 61 6959 Batch (8%), Java (71%), Shell (11%), XML (7%), YAML (3%) 48 1660 JS (1.2%), YAML (98.8%) 4 81 Python (94.1%), YAML (5.9%) 12 1080 C (6.3%), C++ (93.7%) 3 668 JSON (7.0%), Python (93.0%) 18 1269 INI (0.1%), Makefile (0.1%), Python (91.5%), SQL (7.5%), XML (0.9%) 131 9796 Dockerfile (1.8%), JSON (33.8%), JS (0.2%), Makefile (0.1%), Perl (6.9%), Prolog (9.6%), Python (15.1%), 58 5088 Shell (1.5%), YAML (31.1%) CSS (1.0%), Dockerfile (0.4%), Go (3.6%), JSON (71.6%), Java (7.0%), JS (0.7%), Python (11.7%), Shell 172 23999 (1.1%), TypeScript (1.2%), XML (0.3%), YAML (1.4%)
ware engineering backgrounds and architectural knowledge. Many of them have direct responsibility for the documentation of the evaluated repositories. Table I summarizes the 22 repositories contributed by participants, reporting for each project the main implementation languages, number of source files, and non-comment lines of code (LOC). The repositories exhibit substantial variability in size and technology stack: the number of files ranges from 3 to 447, and LOC from 81 to 238,951. The sample includes code bases predominantly written in Python, Java, C/C++, or JavaScript, often complemented by configuration and infrastructure artifacts such as YAML, JSON, XML, Dockerfiles, and shell scripts. This diversity indicates that the generated documentation was evaluated on both small and relatively large, multi-language repositories.
Survey Execution and Data Analysis. Each participant provided a software repository of their choice, which was processed by our tool to generate the full architectural documentation. Participants were then asked to review the generated documentation and complete the questionnaire. No training phase was required, as the evaluation focused on the documentation itself rather than on the tool. Responses were collected individually. For the closed-ended questions, we grouped answers according to the Likert-scale values and computed their frequencies to describe the overall distribution of participants’ perceptions. For the open-ended questions, we performed a thematic analysis [80] following standard qualitative research practices: two authors independently coded all textual answers, iteratively identifying and refining recurring themes related to strengths, weaknesses, and missing aspects of the generated documentation. Discrepancies between coders were discussed and resolved with the involvement of a third author, leading to a consolidated set of themes and ensuring consistency in the interpretation.
V. R ESULTS In this section, we present the results of our empirical study by addressing each of the five research questions (RQ1–RQ5). For RQ1–RQ4, we report the quantitative findings from the questionnaire together with the qualitative insights derived from the open-ended responses, providing a comprehensive view of how developers evaluated the generated architectural documentation. For RQ5, we describe the time and cost required to automatically generate the documentation, characterizing the practicality of the proposed process. A. Answer to RQ1 Figure 2 shows the distribution of responses to Q1–Q4, indicating an overall positive perception of the LLM-generated architectural documentation. Q1, which probes whether the documentation provides valuable architectural insights, received predominantly positive ratings, with A(11) and SA(4), while a smaller group selected N(5) or D(2). A similar tendency appears in Q2, where respondents again expressed strong agreement, with A(9) and SA(8) accompanied by limited neutrality N(4) or disagreement D(1). This suggests that the documentation generally supports comprehension of the system’s architectural structure. Perceptions of maturity (Q3) were more varied yet remained broadly positive, with A(9) and SA(5). A subset of respondents selected N(3) or D(5), indicating that some additional refinement or domainspecific adjustments may be needed before full adoption. The open-ended responses further contextualize these findings by showing which sections developers found most valuable. The most recurrent theme was the usefulness of diagrams (13 mentions), which were appreciated for making structural relationships easier to understand, as reflected in comments such as “Diagrams [. . . ] visually clarify complex information” and “Class Diagrams [. . . ] make it easier to navigate dependencies”.
TABLE II OVERVIEW OF THE QUESTIONNAIRE STRUCTURE , QUESTION TYPES , AND MAPPING TO QUESTIONNAIRE PARTS . L EGEND : L = 5- POINT L IKERT SCALE ; O = O PEN - ENDED QUESTION . ID
Question
Type
Part 1 – Perceived value (RQ1) Q1 Q2 Q3 Q4
The LLM-generated architectural documentation provides valuable architectural insights about the system. The LLM-generated documentation is useful for understanding and maintaining the system’s structure and dependencies. The LLM-generated documentation is valuable enough that I would consider using it in my own project. Which sections or features of the LLM-generated documentation were most or least valuable to you?
L L L O
Part 2 – Comprehensibility (RQ2) Q5 Q6 Q7 Q8 Q9
The LLM-generated architectural documentation is clear, well-structured, and easy to follow. The LLM-generated architectural documentation uses appropriate terminology for software architecture. The LLM-generated architectural documentation contains excessive redundancy or unnecessary information. The explanations within the documentation are sufficiently detailed. Which parts of the documentation were most or least comprehensible to you?
L L L L O
Part 3 – Accuracy and consistency (RQ3) Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 Q23 Q24 Q25 Q26 Q27
The LLM-generated documentation accurately reflects the architecture implemented in the repository. The documentation is internally consistent across sections and diagrams. The content is complete and does not omit key architectural elements. The relationships and dependencies among components are well captured The component responsibilities are correctly described in the documentation I would use the documentation as a reliable reference for the system architecture. Section 1: System Overview: The purpose, scope, and key features described are correctly derived from the repository content. Section 2 Architectural Context: The external systems, APIs, data sources, and actors are correctly derived from the repository content. Section 2.1 Use Case Diagram: The diagram correctly represents actors and use cases derived from the repository content. Section 3 Containers: The described containers, technologies, and communication protocols match the repository content. Section 3.1 Component Diagram: The diagram correctly reflects the containers and externals as found in the repository. Section 4 Components: The identified components and interactions correspond to code-level relationships (imports, calls, dependencies). Section 5 Code-Level: The entry points, key modules, and patterns correspond to actual code structure (no speculative elements). Section 5.1 Code-Level Diagram: The diagram includes only elements and relationships explicitly present in the repository Section 6 Cross-Cutting Concerns: The concerns listed (e.g., security, logging, configuration) are correctly represented. Section 7 Quality Attributes & Rationale: The quality attributes and rationales are correctly represented (no speculative elements). Section 8 Deployment: The deployment information (nodes, environments) accurately reflects the repository’s infrastructure. Section 8.1 Deployment Diagram: The diagram includes only infrastructure elements and connections declared in the repository.
L L L L L L L L L L L L L L L L L L
Part 4 – Limitations and missing aspects (RQ4) Q28 Please describe any inconsistencies, inaccuracies, or unclear parts you noticed in the documentation. Which sections need improvement? Q29 What important architectural information is missing or insufficiently represented in the LLM-generated documentation? Q30 What improvements or changes would make the LLM-generated documentation more useful and trustworthy for your project?
2 5 Q1 (9%) (23%) 4 1 Q2 (5%) (18%) 5 3 Q3 (23%) (14%) SD (1) D (2)
9 (41%)
11 (50%) 9 (41%) N (3)
4 (18%) 8 (36%) 5 (23%) A (4) SA (5)
Fig. 2. Distribution of participants’ Likert-scale ratings for RQ1
A second prominent theme relates to the value attributed to the Components section (9 mentions), which several participants described as essential for understanding the system’s internal organization and for supporting modification tasks. As noted in the responses, “When I need to find which components I need to modify, I would check it here first” and “[It] highlighted aspects of the codebase that I was not aware of.” Other sections were mentioned less frequently but still appreciated. The Deployment (3 mentions) was valued for clarifying runtime and execution contexts, as reflected in “The deployment representation clearly identified dependencies”. Similarly, Cross-Cutting Concerns and Quality Attributes (4 mentions) were recognized for surfacing system-wide behaviors and architectural considerations, with comments such as “Cross-Cutting and Quality Attributes [. . . ] the list of APIs”. Based on this evidence, the perceived value of the generated documentation can be summarized as follows:
O O O
RQ1 Answer Developers generally evaluated the LLM-generated documentation positively, finding it valuable for understanding system structure and, in many cases, mature enough for integration into their projects. Architectural diagrams, codelevel details, and component views emerged as the most appreciated elements. B. Answer to RQ2 Figure 3 shows that the overall perception of comprehensibility is positive. Clarity and organization (Q5) received predominantly favorable evaluations, with A(7) and SA(4), while N(8) and a small number of negative ratings D(3) indicate that some parts of the documentation required additional effort to follow. Terminology (Q6) was viewed very positively, with A(9) and SA(7), and only limited N(4) or D(2). These results suggest that the architectural vocabulary used across the various sections, ranging from containers and components to code-level elements, was generally considered appropriate and consistent with standard architectural practice. Opinions on redundancy (Q7) were more evenly distributed. Several respondents noted the presence of repetitive content A(6), SA(4), while many remained neutral N(7), and fewer expressed disagreement D(4), SD(1). Overall, redundancy appears present but not sufficiently prominent to hinder comprehension. With respect to the level of detail (Q8), participants expressed positive perceptions, with A(12) and SA(6), and only limited N(3) or D(1). The documentation was considered
3 Q5 (14%) 2 4 Q6 (9%) (18%) 4 1 Q7 (5%) (18%) 3 1 Q8 (5%) (14%) SD (1)
8 (36%)
D (2)
9 (41%)
7 (32%) 12 (55%)
N (3)
7 (32%) 6 (27%)
A (4)
4 (18%) 7 (32%) 4 (18%) 6 (27%) SA (5)
Fig. 3. Distribution of participants’ Likert-scale ratings for RQ2
sufficiently detailed to support understanding of architectural relationships, responsibilities, and interactions across the different abstraction levels represented in the template. The open-ended responses highlight which parts of the documentation participants perceived as most comprehensible. The most frequently mentioned theme is the overall clarity of the documentation (4 mentions), with respondents noting that “comprehensibility is good with all parts” and “the entire documentation is easily readable and understandable”. A second recurrent theme concerns the clarity of high-level architectural sections (3 mentions), where system-level descriptions such as the Architectural Context and Containers were described as particularly easy to follow, as illustrated by comments like “Section 2 was the most comprehensible” and “Section 3 is the most comprehensible [. . . ] provides valuable information”. Among less frequent themes, some respondents pointed to Cross-Cutting Concerns and Quality Attributes (2 mentions) as especially clear, for example “Cross-Cutting Concerns” and “Quality Attributes and Rationale”. Finally, module-to-code and function mappings (1 mention) were also singled out as understandable, as in “Module-to-Code Mapping and Key Functions”. Overall, these observations indicate that participants generally perceived the documentation as comprehensible, particularly in its high-level narrative sections and in those parts that provide explicit structural mappings. Overall, these quantitative patterns and qualitative remarks can be summarized as follows: RQ2 Answer Developers generally find the documentation comprehensible: clarity, organization, terminology, and level of detail all received predominantly positive ratings, with only limited disagreement. Open-ended feedback reinforces this view, highlighting the overall readability of the documentation and, in particular, the clarity of high-level architectural sections.
Items Q13–Q15 confirm this trend: correctness of relationships (Q13) and responsibilities (Q14) was generally recognized, and perceived reliability (Q15), supported by A(8) and SA(3), suggests that many participants viewed the documentation as a usable architectural reference. A more detailed view of section-level accuracy is provided by Q16–Q27 and summarized in Figure 5. Sections closely grounded in observable code evidence received the strongest agreement. The Components section (Q21), describing the system’s internal logical structure, achieved the highest positive ratings with SA(10) and A(9). Cross-Cutting Concerns (Q24) showed a similarly strong pattern, with SA(8) and A(11). High levels of agreement also emerged for the Component Diagram (Q20) and the Containers section (Q19) (18 and 17 positive responses, respectively). The Code-Level section (Q22), capturing the concrete implementation of architectural elements in the source code, was likewise positively assessed, with 17 participants selecting A or SA. Collectively, these results indicate that sections reflecting architectural structure or explicit codebase artifacts were consistently perceived as accurate, whereas a smaller set of sections received more mixed evaluations. The System Overview (Q16) and Architectural Context (Q17) accumulated the highest numbers of negative ratings, D(4) and D(3), suggesting that high-level descriptions were more prone to inaccuracies or omissions. The Use Case Diagram (Q18) also showed weaker agreement, with several negative responses (D(3) and SD(1)). Additional structural diagrams also drew some negative feedback: the Code-Level Diagram (Q23) received D(4), while the Deployment Diagram (Q27) received D(1) and SD(1), indicating occasional mismatches with the underlying implementation or configuration.
2 4 8 Q10 (9%) (18%) (36%) 2 2 10 1 Q11 (5%) (9%) (9%) (45%) 4 4 5 Q12 (18%) (18%) (23%) 5 5 1 Q13 (5%) (23%) (23%) 3 10 1 Q14 (5%) (14%) (45%) 6 5 Q15 (27%) (23%) SD (1) D (2) N (3)
8 (36%) 7 (32%) 9 (41%) 11 (50%) 8 (36%) 8 3 (36%) (14%) A (4) SA (5)
Fig. 4. Distribution of participants’ Likert-scale ratings for RQ3 - General Part
C. Answer to RQ3 The responses to Q10–Q15, summarized in Figure 4, assess several dimensions of accuracy, including architectural correspondence, internal consistency, completeness, correctness of relationships and responsibilities, and perceived reliability. Architectural correspondence (Q10) received largely positive ratings, with A(8) and SA(8), while smaller groups selected N(4) or D(2). Internal consistency (Q11) showed a similar trend, with A(10) and SA(7), and only limited N(2), D(2), or SD(1). Completeness (Q12) also followed this pattern, with SA(9) and A(5), whereas N(4) and D(4) indicate that some omissions were noticed but did not dominate evaluations.
Based on this combination of high-level and section-level assessments, the answer to RQ3 can be expressed as follows: RQ3 Answer Developers perceived the LLM-generated documentation as accurate, indicating that it reflects the implemented architecture, is consistent across sections, and captures key structural elements. Accuracy was rated highest for code-related sections (Components, Containers, Code-Level), while more interpretive views (System Overview, Architectural Context, Use Case) received more mixed feedback.
4 3 8 7 Q16 (18%) (14%) (36%) (32%) 3 2 8 8 1 Q17 (5%) (14%) (9%) (36%) (36%) 3 6 7 5 1 Q18 (5%) (14%) (27%) (32%) (23%) 5 7 10 Q19 (23%) (32%) (45%) 3 8 10 1 Q20 (5%) (14%) (36%) (45%) 3 9 10 Q21 (14%) (41%) (45%) 4 8 9 1 Q22 (5%) (18%) (36%) (41%) 4 3 6 9 Q23 (18%) (14%) (27%) (41%) 3 11 8 Q24 (14%) (50%) (36%) 5 6 10 1 Q25 (5%) (23%) (27%) (45%) 7 8 7 Q26 (32%) (36%) (32%) 4 9 7 1 1 Q27 (5%) (5%) (18%) (41%) (32%) SD (1) D (2) N (3) A (4) SA (5) Fig. 5. Distribution of participants’ Likert-scale ratings for RQ3 - Section Part
D. Answer to RQ4 Coding of the open-ended answers identified eight limitation categories, summarized in Table III. Beyond Q28–Q30, we also reviewed the open-ended responses from the other RQs to capture additional limitations or missing aspects. The most prominent category was Diagram Errors (32 occurrences). Participants frequently noted incomplete or misleading diagrams, mentioning for example ‘the class diagram is truncated and omits some of the classes”, all the diagrams should be reviewed, and should be better presented [. . . ] it always seems a little too artificial”, and that some diagrams do not provide correct and useful information”. These remarks reinforce that diagrammatic views were perceived as the most fragile artifacts. A second major theme relates to deployment issues (14 occurrences), particularly in runtime and infrastructure views. Reported problems included unclear relationships, such as ‘Deployment Diagram [. . . ] the relation between components is not clear”, as well as incorrect or confusing arrows, for example Deployment (Local Workstation) has few arrows that should not exist”. Other responses indicated that the deployment view should specify not only the production environment but also include an artifact repository to manage dependencies” or that more details on the Deployment Diagram would be useful”. Less frequent but still relevant themes include Inconsistencies across sections (6 occurrences) and Missing Information (8 occurrences). Some participants observed that ‘the overall components are captured, however [. . . ] wrapping of some of them is not well captured” or noted that the LLM needs to be guided to avoid hallucinations TABLE III L IMITATION CATEGORIES FROM OPEN - ENDED RESPONSES .
Category Diagram errors Deployment issues Improvement suggestions Missing information Inconsistencies Redundancy
Occurrences 32 14 13 8 6 1
and inconsistencies”. Missing content was also highlighted, such as the class diagram is truncated and omits some of the classes”, it misses the use case tables”, or that respondents lack an in-depth description of this component”. Finally, several responses offered general Improvement Suggestions (13 occurrences), including ‘double-check with existing documentation if present”, human inputs would surely enhance the quality of produced documentation”, and requests for a shorter and better structured output to help users navigate it more easily”. Redundancy was mentioned only once, with one participant noting that all sections are complete, but some content is repetitive”. In summary, the themes emerging from the open-ended responses converge toward the following answer to RQ4: RQ4 Answer Developers primarily point to problems in the diagrams, which are often described as incomplete, unclear, or inaccurate. Deployment views are the second most common source of issues, with respondents noting missing details or confusing relationships. Other limitations include occasional inconsistencies and missing information, along with suggestions to improve structure, reduce verbosity, and complement the generated output with human review. E. Answer to RQ5 Table IV reports the minimum, maximum, and average values of execution time and monetary cost for generating architectural documentation across the 22 analyzed repositories. Generation times ranged from 1m 50s to 4m 25.85s, with an average of approximately 3m 1s, showing that a complete architectural description can be produced in only a few minutes. Monetary costs ranged from $0.35 to $2.48, with an average of $1.19 per repository, showing that an entire architecture document can be generated for roughly one to two dollars in API usage. These results suggest that the process is both time-efficient and economically lightweight compared to manual documentation effort. TABLE IV C OSTS OF G ENERATING THE D OCUMENTATION
Metric
Min
Max
Mean
Generation Time Financial API Cost
1m 50s $0.35
4m 25.85s $2.48
3m 0.90s $1.19
Overall, these measurements of time and monetary cost can be summarized in the following answer: RQ5 Answer Generating complete architectural documentation for a repository takes about three minutes on average, at an average API cost of $1.19, making the process time-efficient and costeffective compared to manual authoring. F. Additional Practitioner Interest Beyond the formal evaluation, three participants expressed interest in using CIAO to document internal closed-source
projects in their organizations, suggesting perceived practical value beyond the study setting. For example, one participant working on a safety-critical railway software component subject to strict regulations requested CIAO-generated documentation to support compliance activities, where architectural descriptions must be explicitly traceable to the source code. The participant found the structured representation of containers, components, and their dependencies particularly useful for SDLC activities such as test planning and execution. Another participant, responsible for an internal tool used for penetration testing of edge–cloud infrastructures, faced challenges due to significant misalignment between the system’s architecture and its implementation, an instance of architectural drift. Since the codebase had evolved faster than the documentation, CIAO was used to reconstruct an updated system-level description and restore architectural clarity. A third participant, a software developer working in a company that provides digital services for public administrations, highlighted the absence of structured architectural documentation in several internal systems. They employed CIAO to generate system-level descriptions aimed at facilitating onboarding and improving knowledge transfer within their development team. Taken together, these accounts illustrate early practitioner interest in deploying CIAO beyond open-source settings, particularly in contexts where regulatory compliance, architectural alignment, or team onboarding make structured systemlevel documentation especially valuable. VI. T HREATS TO VALIDITY In this section, we discuss the main threats to validity of our study and the mitigation measures adopted to address them. Internal Validity. A potential threat lies in the subjective nature of the evaluation. Participants assessed documentation generated for repositories they had personally developed or maintained. While this familiarity reduces the risk of misinterpreting the underlying architecture, it may also introduce positive or negative bias in judging accuracy, completeness, or usefulness. To mitigate this threat, the questionnaire combined Likert-scale items with open-ended questions that asked participants to justify their ratings through concrete observations, thus reducing reliance on overall impressions. Construct Validity. This study evaluates developers’ perceptions of value, clarity, and accuracy rather than objective architectural correctness. The results also depend on the prompting strategy and the documentation template adopted in CIAO. An additional threat concerns the template itself, which was iteratively designed with a limited number of experts and may therefore cover only a restricted range of architectural concerns or shape how they are represented. To mitigate this, the questionnaire explicitly asked participants to assess the clarity, completeness, and usefulness of each section, providing an indirect validation of the template’s suitability for architecturelevel analysis. External Validity. The study involved 22 participants, mostly from academic settings, and a diverse but not exhaustive set
of repositories. This limits the generalizability of the findings to other developer populations, software domains, and LLM configurations. To partly mitigate this threat, we included repositories varying in size, language, and domain, although broader studies, especially with industrial participants, are needed to further assess the applicability of the results. Conclusion Validity. The thematic analysis of the open-ended responses may introduce interpretive bias. To mitigate this threat, two authors independently coded all responses and resolved disagreements with a third author, following established qualitative analysis practices to improve consistency and reliability in theme identification. VII. C ONCLUSIONS AND F UTURE W ORK This paper introduced CIAO, a structured process for automatically generating system-level architectural documentation from complete GitHub repositories using LLMs. CIAO combines a standards-oriented template with an LLM-based workflow grounded in ISO/IEC/IEEE 42010, SEI’s Views & Beyond, and the C4 model. Our empirical study with 22 developers suggests that the generated documentation is generally perceived as useful, understandable, and consistent with the architecture of the analyzed systems. Participants particularly appreciated the narrative sections and views closely linked to source code, while the main limitations concerned diagrammatic and deploymentrelated artifacts. The qualitative analysis also indicates that most inaccuracies are localized rather than structural, making the documentation a potentially useful architectural reference. In addition, the results show that the process is practically feasible, with manageable generation time and cost when using GPT-5. Future work will focus on improving the reliability of diagrammatic views, which emerged as the weakest aspect of the generated documentation. Possible directions include integrating static or dynamic analysis and retrieval mechanisms tailored to architectural artifacts. We also plan to extend the evaluation to more system types, larger repositories, industrial contexts, and broader groups of developers. Finally, future studies may investigate how CIAO can support documentation and maintenance workflows, and whether human-in-the-loop refinement can further improve documentation quality and adoption. ACKNOWLEDGMENT This work was supported by the Italian Ministry of Research, under the complementary actions to the NRRP “Fit4MedRob - Fit for Medical Robotics” Grant (# PNC0000007).
R EFERENCES [1] D. Garlan et al., “Architectural mismatch: Why reuse is still so hard,” IEEE Software, vol. 26, no. 4, pp. 66–69, July 2009. [2] R. Verdecchia et al., “Architectural technical debt: A grounded theory,” in Software Architecture: 14th European Conference, ECSA 2020, L’Aquila, Italy, September 14–18, 2020, Proceedings. Berlin, Heidelberg: Springer-Verlag, 2020. [3] P. Kruchten et al., “Technical debt: From metaphor to theory and practice,” IEEE Softw., vol. 29, no. 6, p. 18–21, Nov. 2012. [Online]. Available: https://doi.org/10.1109/MS.2012.167 [4] D. M. Le et al., “An empirical study of architectural decay in opensource software,” in 2018 IEEE International Conference on Software Architecture (ICSA), 2018. [5] “ISO/IEC/IEEE 42010:2011 systems and software engineering— architecture description,” International Organization for Standardization, 2011. [6] P. Clements, F. Bachmann, L. Bass, D. Garlan, J. Ivers, R. Little, P. Merson, and J. Stafford, Documenting Software Architectures: Views and Beyond. Addison-Wesley, 2010. [7] Z. Fan, R. Liu, X. Chen, and H. Yu, “Large language models for software engineering: A systematic literature review,” ACM Computing Surveys, 2024. [8] F. Zhang, Q. Kang, and D. Yu, “A survey on large language models for software engineering,” arXiv preprint arXiv:2308.11679, 2023. [9] J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim, “A survey on large language models for code generation,” 2024. [10] M. Jin et al., “Inferfix: End-to-end program repair with llms,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2023. [11] GitHub, “The state of open source and ai: Github octoverse report,” https://octoverse.github.com/, 2023. [12] O. Chaplia and H. Klym, “Extracting rest api endpoints from microservices using llm agents,” in 2024 14th International Conference on Dependable Systems, Services and Technologies (DESSERT), 2024, pp. 1–7. [13] D. Nam et al., “Using an llm to help with code understanding,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New York, NY, USA: Association for Computing Machinery, 2024. [Online]. Available: https://doi.org/10.1145/3597503.3639187 [14] N. Djajadi et al., “Using large language models to generate concise and understandable test case summaries,” in 2025 IEEE/ACM 33rd International Conference on Program Comprehension (ICPC), 2025, pp. 322–326. [15] R. Xu et al., AutoReqGen: A Pipeline Approach for Automated Requirements Generation from Source Code, 07 2025, pp. 444–455. [16] H. A. Siala and K. Lano, “Using large language models to extract uml class diagrams from java programs,” in 2025 8th International Conference on Software and System Engineering (ICoSSE), 2025, pp. 70–74. [17] J. Miranda et al., “Towards an in-context llm-based approach for automating the definition of model views,” 2024. [18] A. Boronat and J. Mustafa, “Mdre-llm: A tool for analyzing and applying llms in software reverse engineering,” in 2025 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 2025, pp. 850–854. [19] K. Chen et al., “Automated domain modeling with large language models: A comparative study,” in 2023 ACM/IEEE 26th International Conference on Model Driven Engineering Languages and Systems (MODELS), 2023. [20] S. Brown, The C4 Model for Visualising Software Architecture. Leanpub, 2018. [Online]. Available: https://c4model.com [21] M. L. Nelson, “A survey of reverse engineering and program comprehension,” ArXiv, vol. abs/cs/0503068, 2005. [22] C. Stringfellow et al., “Comparison of software architecture reverse engineering methods,” Information and Software Technology, vol. 48, no. 7, pp. 484–497, 2006. [23] A. Telea et al., “An open visualization toolkit for reverse architecting,” in Proceedings 10th International Workshop on Program Comprehension, 2002. [24] J. Garcia et al., “A comparative analysis of software architecture recovery techniques,” in 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2013, pp. 486–496.
[25] “Controlling software architecture erosion: A survey,” Journal of Systems and Software, 2012. [26] A. Bucaioni et al., “Architecture as code,” in 2025 IEEE 22nd International Conference on Software Architecture (ICSA). IEEE, 2025, pp. 187–198. [27] R. Wohlrab et al., “Improving the consistency and usefulness of architecture descriptions: Guidelines for architects,” in 2019 IEEE International Conference on Software Architecture (ICSA), 2019, pp. 151–160. [28] T. Systä, “Static and dynamic reverse engineering techniques for java software systems,” 2000. [29] M. Pinzger et al., “Revealer: a lexical pattern matcher for architecture recovery,” in Ninth Working Conference on Reverse Engineering, 2002. Proceedings., 2002, pp. 170–178. [30] I. Şora, “Helping program comprehension of large software systems by identifying their most important classes,” in Evaluation of Novel Approaches to Software Engineering, L. A. Maciaszek and J. Filipe, Eds. Cham: Springer International Publishing, 2016, pp. 122–140. [31] T. Ahmed et al., “Automatic semantic augmentation of language model prompts (for code summarization),” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, 2024. [32] L. Zhang et al., “Evaluating large language models for real-world vulnerability repair in c/c++ code,” in Proceedings of the 10th ACM International Workshop on Security and Privacy Analytics, 2024. [33] M. Nejjar, L. Zacharias, F. Stiehle, and I. Weber, “Llms for science: Usage for code generation and data analysis,” Journal of Software: Evolution and Process, 2023. [34] P. Liu et al., “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Comput. Surv., vol. 55, no. 9, Jan. 2023. [35] Y. S. Nugroho et al., “Uncovering intention through llm-driven code snippet description generation,” in 2025 International Conference on Smart Computing, IoT and Machine Learning (SIML), 2025, pp. 1–6. [36] C. Diggs, et al., “Leveraging llms for legacy code modernization: Evaluation of llm-generated documentation,” in 2025 IEEE/ACM International Workshop on Large Language Models for Code (LLM4Code), 2025, pp. 177–184. [37] D. Amalfitano, M. D. Luca, T. Santilli, P. Pelliccione, and A. R. Fasolino, “Automated software architecture design recovery from source code using llms,” in Software Architecture. Cham: Springer Nature Switzerland, 2026, pp. 73–89. [38] H. Siala and K. Lano, “A comparison of large language models and model-driven reverse engineering for reverse engineering,” Frontiers in Computer Science, vol. 7, 2025, publisher Copyright: Copyright © 2025 Siala and Lano. [39] Q. Caballero et al., “Evaluating chatgpt’s proficiency in understanding and answering microservice architecture queries using source code insights,” SN Computer Science, 2024. [40] B. Gustrowsky et al., “Using generative artificial intelligence for suggesting software architecture patterns from requirements,” in Intelligent Systems and Applications, 2024. [41] J. Jahic and A. Sami, “ State of Practice: LLMs in Software Engineering and Software Architecture ,” in 2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C), 2024. [42] L. Naimi, et al., “Automating software documentation: Employing llms for precise use case description,” Procedia Computer Science, vol. 246, pp. 1346–1354, 2024, 28th International Conference on Knowledge Based and Intelligent information and Engineering Systems (KES 2024). [Online]. Available: https://www.sciencedirect. com/science/article/pii/S1877050924026176 [43] G. Garaccione et al., “Evaluating large language models in exercises of uml use case diagrams modeling,” in 2025 IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE), 2025, pp. 41–44. [44] A. Naghshzan, Automating API Documentation with LLMs: A BERTopic Approach. New York, NY, USA: Association for Computing Machinery, 2025, p. 1013–1015. [Online]. Available: https://doi.org/10.1145/3696630.3728615 [45] S. Bala et al., ChatGPT for Tailoring Software Documentation for Managers and Developers, 01 2025, pp. 103–109. [46] K. A. Demir, et al., “Generative ai efficiency and effectiveness in software project documentation review process,” in 2025 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA), 2025, pp. 1–7.
[47] A. Emerson, T. Meehan, M. Rogers, W. Cowen, and C. Darabos, “Codedocs: Genai to generate documentation from git repositories,” in Practice and Experience in Advanced Research Computing 2025: The Power of Collaboration, ser. PEARC ’25. New York, NY, USA: Association for Computing Machinery, 2025. [Online]. Available: https://doi.org/10.1145/3708035.3736102 [48] P. Clements, D. Garlan, R. Little, R. Nord, and J. Stafford, “Documenting software architectures: views and beyond,” in 25th International Conference on Software Engineering, 2003. Proceedings., 2003, pp. 740–741. [49] Software, systems and enterprise – Architecture description, International Standard ISO/IEC/IEEE 42 010:2022, November 2022. [Online]. Available: https://www.iso.org/standard/74393.html [50] https://c4model.info/\#app. [51] R. Jongeling, N. J. Strøm, L. P. T. Nissen, M. Kitchen, and J. Carlson, “Adopting the c4 model for lightweight architecture modeling - an experience report,” in Software Engineering and Advanced Applications, D. Taibi and D. Smite, Eds. Cham: Springer Nature Switzerland, 2026, pp. 393–409. [52] https://repomix.com. [53] https://plantuml.com/. [54] Y. Y. Liu, Z. Zheng, F. Zhang, and et al., “A comprehensive taxonomy of prompt engineering techniques for large language models,” Front. Comput. Sci., vol. 20, p. 2003601, 2026. [Online]. Available: https://doi.org/10.1007/s11704-025-50058-z [55] A. Kong, S. Zhao, H. Chen, Q. Li, Y. Qin, R. Sun, X. Zhou, E. Wang, and X. Dong, “Better zero-shot reasoning with role-play prompting,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), K. Duh, H. Gomez, and S. Bethard, Eds. Mexico City, Mexico: Association for Computational Linguistics, Jun. 2024, pp. 4099–4113. [Online]. Available: https://aclanthology.org/2024.naacl-long.228/ [56] C. Li, J. Wang, K. Zhu, Y. Zhang, W. Hou, J. Lian, and X. Xie, “Large language models understand and can be enhanced by emotional stimuli,” 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID: 260126019
[57] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 1877– 1901. [Online]. Available: https://proceedings.neurips.cc/paper files/ paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf [58] https://github.com/AntonioEmmanuele/Group-Key-Phemap. [59] https://github.com/SalvatoreBarone/pyALS-RF/tree/tmr. [60] https://github.com/rdinizcal/sa-bsn. [61] https://github.com/SaraPettinari/ocean-lib. [62] https://github.com/gianlucafilippone/robethichor. [63] https://github.com/gianlucafilippone/springboot-microservices-development. [64] https://github.com/DockerSecurityPlayground/DSP/tree/master/public. [65] https://github.com/DockerSecurityPlayground/DSP/tree/master/app. [66] https://github.com/QuTAM/QuFI. [67] https://github.com/alessandrasomma28/BoMoDT. [68] https://github.com/alessandrasomma28/m2dt. [69] https://github.com/AposVinc/GymportalService. [70] https://github.com/s4hri/pyicub. [71] https://github.com/francescodaghero/treeco. [72] https://github.com/Testing-Game-SAD-2023/A13/tree/main/T4/ gamerepo. [73] https://github.com/ykris152/rse2025-priv. [74] https://github.com/deangelisdf/write2audiobook. [75] https://github.com/deangelisdf/BF-JITcompiled. [76] https://github.com/deangelisdf/visual novel pyarcade. [77] https://github.com/VSecLab/SAM-CyFra. [78] https://github.com/NS-unina/SCASS. [79] https://github.com/NS-unina/OSINT-Collector. [80] V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qualitative Research in Psychology, vol. 3, pp. 77–101, 01 2006.