arXiv CSopen access

PRISMA-LLM: An Empirical Reporting Framework for AI-Assisted Systematic Reviews

Unknown · arxiv_cs

arXiv CS · Papers · License: Open Access

softwarearchitecture softwareengineering testing

software engineering, software architecture, testing

PRISMA-LLM: A N E MPIRICAL R EPORTING F RAMEWORK FOR AI-A SSISTED S YSTEMATIC R EVIEWS M ANUSCRIPT Miguel Zabaleta1 , Baihan Lin1,2,3,∗

arXiv:2609.11559v1 [cs.SE] 10 Sep 2026

Department of AI and Human Health, Icahn School of Medicine at Mount Sinai, New York, NY, USA 2 Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA 3 Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA ∗ Corresponding: [email protected]

September 11, 2026

A BSTRACT Large language models (LLMs) and AI-enabled software increasingly participate in systematic-review decisions, yet the information needed to audit these workflows is reported inconsistently. We analyze SciLitBench, a corpus of 888 review-automation papers with 14,726 annotations, to characterize changes in methods, review-stage use, evaluation and reported limitations. Automation has shifted toward LLM- and software-facing workflows, including stages that can alter the evidence base. Since 2023, 38.0% of software/product papers reported no evaluation, compared with 9.3% of LLM papers. Reporting coverage increased with LLM workflow complexity, yet 52% of positive-only LLM evaluations still reported an unmet reliability or performance requirement. From these patterns, we introduce PRISMA-LLM, an empirically grounded framework separating implementation disclosure from consequence-sensitive evaluation and limitation reporting. Keywords systematic reviews, large language models, PRISMA, reporting guidelines, evidence synthesis, review automation Systematic reviews help determine which evidence enters scientific syntheses and, in many fields, inform guidelines, policy, and clinical decisions. Computational systems have long assisted literature search and screening, but LLMs and AI-enabled software are now being used across a broader portion of the review workflow, including full-text screening, data extraction, quality assessment, and synthesis [1–3]. As automation moves from navigation toward decisions that can alter the evidence base, the reproducibility of a review increasingly depends on details of the computational workflow itself. This creates a reporting problem. An AI-assisted workflow can depend on model or product identity, version and access mode, prompts and system instructions, sampling settings, schemas, document parsing, chunking, retrieval, post-processing, thresholds, and human adjudication. For hosted or proprietary systems, behavior may also change over time without a corresponding change to the published review. Without sufficient disclosure, readers may be unable to determine what was automated, how an error could propagate, whether a decision was independently checked, or whether the workflow can be rerun or meaningfully audited [4]. Reporting guidelines turn general expectations of transparency into concrete information that readers can use. PRISMA 2020 specifies the information needed to understand how a systematic review was conducted and reported [5]; PRISMAS provides a close precedent by recognizing that technically consequential search procedures require dedicated reporting detail [6]. AI-assisted review workflows create a related reporting problem: without the model and workflow details, readers lack the information needed to know what was automated, how errors could arise, or whether the workflow can be reused. Several recent proposals have begun to address AI-related reporting, including PRISMA-AI, PRISMA-trAIce, Analysis code, publication-date data used in the analyses, PRISMA-LLM reporting instructions, a fillable checklist workbook, and the checklist evidence table are available at https://github.com/linlab/PRISMA-LLM.

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

PRISMA-DFLLM, and L-PRISMA [7–10]. What remains unclear is how reporting practice has evolved across the broader review-automation literature and which recurring dimensions of evaluation and limitation reporting can provide an empirical basis for more specific guidance. Here we analyze SciLitBench [11], a corpus of 888 papers on computational automation in systematic reviews. We ask three questions: how review automation has changed over time and across review stages; whether evaluation and limitation reporting differs across methods; and what these observed reporting patterns imply for transparent reporting of LLM-assisted reviews. We then introduce PRISMA-LLM as an empirically informed reporting framework aligned with the structure of PRISMA 2020. Its purpose is not to certify the quality or safety of an AI workflow, but to make the implementation choices, evaluation evidence, human oversight, and failure modes sufficiently visible for readers to interpret and audit the review. Because implementation complexity is not equivalent to methodological risk, we treat the proposed levels as disclosure tiers rather than risk tiers and explicitly allow consequential uses of otherwise simple tools to trigger stronger evaluation expectations.

Results

SciLitBench provides a longitudinal view of how computational systems are entering systematic-review practice. We first characterize changes in publication volume, technical approach, review-stage use, and model family, then examine whether evaluation and limitation reporting has changed with the methods being used. Finally, we use those empirical patterns to motivate a reporting framework. 1.1

Growth and shifts

Publication volume increased sharply over the observed period. Figure 1b shows that publication counts were low and relatively flat for most of the corpus history, began to rise around 2018–2019, and then steepened after the release of ChatGPT at the end of 2022. A log-linear fit to positive monthly counts from January 2020 through June 2025 corresponds to an estimated 4.7% monthly growth rate. The corpus spans several fields, with 61.7% of records coming from life sciences and medicine (Figure 1a). This concentration fits the broader history of evidence synthesis where biomedical research has generated a large systematic-review literature (PRISMA itself grew out of reporting guidance for reviews and meta-analyses of health-care interventions) [5, 12–14]. Engineering and technology and social sciences and management add 15.8% and 13.0%. Mixed-domain papers, natural sciences, and arts and humanities make up the rest. Figure 2 shows how review stages and methods have shifted over time. Earlier review-automation work was centered on custom model-building, BERT, deep learning, traditional machine learning, and rule-based systems. In recent years, we see a clear transition to LLMs and software products that use LLMs under the hood, like Elicit and DistillerSR [15, 16] (Figure 2a). Automation is also moving into review stages where errors can change the evidence base. We group these stages as discovery/navigation, screening/selection, and evidence construction. In recent records, screening/selection and evidence construction account for a substantial share alongside discovery/navigation (Figure 2b). We also examine how LLM adoption differs across fields. In life sciences and medicine, the share of papers using LLMs has risen, approaching 60% in recent months (Figure 2c). Engineering and technology and social sciences and management show more short-term variation, but both curves rise through the observed 2025 window after a late-2024 to early-2025 dip. Mixed-domain records reach 70–80% in several recent windows. We also explored whether disciplines differ in the language models they use. Supplementary Figure S1 shows that BERT-based papers have more domain specialization: biomedical variants dominate much of life sciences and medicine, while generic BERT, BERTopic, sentence embeddings, and other transformer variants appear more prominently outside that field. Most LLM papers use GPT/ChatGPT-family systems, consistent with prior reports [3, 17], with Claude, Gemini/Bard/PaLM, Llama, Mistral/Mixtral, and other families appearing less often. As for the use of open-weight versus proprietary models, the data shows that most LLM usage still relies on proprietary or hosted systems, while open-weight models appear much less often (84.1% proprietary, 11.0% mixed, and 4.9% open-weight). Even though we see more open-weight usage in some domains than others (for example, mixed-domain records include open-weight or mixed proprietary/open-weight use in 28.0% of cases, compared with 10.3% in engineering and technology), these differences are not statistically significant. Overall, the corpus shows a field changing on several fronts: review-automation papers are growing rapidly, the methods used have shifted toward LLM/software-facing workflows, and automation is increasingly present in stages that can affect which evidence enters a review and how it is interpreted. Next, we explore whether reporting practice has kept pace with these changes, and introduce reporting richness as a measure of how much evaluation and limitation detail each paper provides. 2

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

a) Corpus composition Corpus measure

Count

High-level domain

Paper-level records Annotation items Evaluation items Limitation items

888 14,726 6,562 2,896

Life sciences and medicine Engineering and technology Social sciences and management Mixed Natural sciences Arts and humanities

Papers, n (%) 548 (61.7) 140 (15.8) 115 (13.0) 62 (7.0) 21 (2.4) 2 (0.2)

b) Publication growth

Figure 1: Corpus scope and publication growth in review automation. a) SciLitBench contains 888 paper-level records and 14,726 annotation items; 61.7% of papers are from life sciences and medicine. b) Publication counts are shown from 2010 onward for legibility; five included papers predate 2010. The red curve is a log-linear fit to positive monthly counts from January 2020 through June 2025, aggregated to the displayed periods. The dashed continuation and grey region show a descriptive extrapolation through 2026, not a causal or calibrated forecast. The vertical dashed line marks the public release of ChatGPT.

1.2

Evaluation reporting differs by automation context and method complexity

Review automation is growing and moving into parts of the review where automated decisions can affect the evidence base. We therefore asked whether papers give researchers enough detail to understand what was automated, how it was evaluated, and what limits were reported. We summarize this with reporting richness, a descriptive index of paper-level coverage rather than a measure of study quality, and compare it across approach groups and LLM-era workflow types. Reporting richness is a descriptive 0–15 index of how much evaluation and limitation detail a paper reports. It combines five dimensions: performance evidence, comparisons, optimization or modification evidence, resource or feasibility information, and limitations or failure modes; the full scoring rule is given in the Methods (Section 3.4). Traditional, deep-learning, and BERT-oriented approaches decline as a share of recent papers while their mean reporting richness remains comparatively stable. LLM papers show increasing reporting richness after 2023, whereas software/product papers remain common but contain less paper-level evaluation and limitation detail (Supplementary Figure S3). Since 2023, software/product papers average a reporting-richness score of 3.3, compared with 6.3 for LLM papers and 6.0 for traditional, deep-learning, and BERT-oriented papers. The share of papers with no reported evaluation shows the same contrast: 38.0% of software/product papers report no evaluation, compared with 9.3% of LLM papers and 13.6% of traditional, deep-learning, and BERT-oriented papers. The same corpus also lets us look inside the methods used in LLM-era review automation. Rather than treating all LLM use as one category, we grouped papers by the kind of automation method reported: software/product-only use, prompt-only LLM use, structured or engineered LLM workflows, retrieval, adapted, or agentic LLM workflows, and non-LLM automation. Figure 3a shows the rolling 3-month share of these groups alongside three timing references: ChatGPT, GPT-4, and GPTs/Retrieval. Prompt-only LLM use first appears in 2023, structured or engineered workflows 3

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

a) Approach orientation

b) Review-stage composition

c) LLM adoption by domain

Figure 2: Shifts in method orientation, review-stage use, and LLM adoption. All panels use overlapping rolling 3-month windows. a) Exclusive approach orientation. The LLM/software-facing category includes LLM and softwareproduct papers; before ChatGPT, this category is therefore software-facing. The dotted line marks ChatGPT’s public release, and the solid line and triangle mark the first window in which LLM/software-facing work became the largest orientation category. b) Review-stage composition. Each paper contributes total weight one, divided equally across its assigned stage groups; screening/selection and evidence construction therefore represent the share of review-stage focus, not the share of unique papers. c) Share of papers using an LLM within each high-level domain, beginning with the first window containing LLM use. Mixed-domain papers span more than one domain and are shown separately. Adjacent points share observations and should not be interpreted as independent time points.

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

a) Method-complexity mix

b) Reporting by method complexity

Figure 3: LLM-era workflow complexity and evaluation reporting. a) Exclusive method-complexity categories in overlapping rolling 3-month windows. Reference dates for ChatGPT, GPT-4, and GPTs/Retrieval provide temporal context only and are not treated as causal breakpoints. b) Reporting outcomes among papers published from 2023 onward. Bars show mean reporting richness and the percentage of papers with no reported evaluation; n is the number of papers in each group. Reporting coverage increases across the observed LLM workflow categories, from prompt-only to engineered and retrieval/adapted/agentic methods. These are descriptive associations and may also reflect differences in study purpose, venue, task, and other characteristics.

become visible later that year, and retrieval, adapted, or agentic workflows appear in early 2024. In the latest rolling window, ending June 2025, the split is 34.0% prompt-only LLM, 29.2% software/product only, 14.2% engineered or structured LLM, 8.5% retrieval, adapted, or agentic LLM, 7.5% neural non-LLM, and 6.6% rule or traditional ML. Among papers published from 2023 onward, software/product-only papers have the lowest mean reporting-richness score (2.6), and 44.4% report no evaluation. Prompt-only LLM papers average 5.2, with 13.4% reporting no evaluation; engineered or structured LLM papers average 8.4, with 1.7% reporting no evaluation; and retrieval, adapted, or agentic LLM papers average 9.1, with no papers in this group annotated as reporting no evaluation (Figure 3). The monotonic pattern is descriptive rather than causal: more complex workflows are often themselves the object of method development, whereas off-the-shelf software may be used instrumentally in papers whose primary contribution lies elsewhere. Together, these results motivate PRISMA-LLM, a reporting framework designed to make implementation choices, human oversight, evaluation evidence, and failure modes more visible. 1.3

Usefulness and adequacy are distinct judgments

Overall evaluations of LLM-assisted review automation were often favorable, but favorable assessments did not imply that authors considered the systems adequate for unsupervised use. Among 118 LLM papers classified as positive only, 52% nevertheless reported at least one limitation indicating that the workflow remained below the reliability or 5

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

a) Overall impression among LLM papers over time

b) High-bar concerns by overall impression

Figure 4: Positive assessments frequently coexist with unmet reliability requirements. a) Shares of LLM papers assigned to four mutually exclusive overall-impression categories within rolling 3-month windows. Adjacent windows contain overlapping papers, and the earliest windows contain few records. b) Percentage of papers in each impression category that report at least one concern that the workflow remains below the reliability or performance bar required for its intended use; n denotes the number of papers in each category. High-bar concerns appear across all impression categories. Notably, 52% of papers classified as positive only still report at least one such concern.

performance bar required for its intended role. The corresponding share was 67% among 57 positive-with-caveats papers and 79% among 38 mixed/negative papers (Figure 4). These categories capture reported judgments rather than independent re-evaluation of the systems, but they reveal a recurring distinction between usefulness and adequacy for delegation. A workflow may save time or perform well on average while still requiring human verification at points where errors could change the evidence base. 1.4

PRISMA-LLM

PRISMA-LLM is an evidence-informed reporting framework for LLM-assisted and AI-enabled systematic-review workflows, organized to complement PRISMA 2020. Its empirical basis is the recurring evaluation and limitation dimensions observed in SciLitBench and the descriptive association between workflow type and reporting coverage [11]. Its implementation requirements additionally draw on the workflow choices encountered in SciLitBench and on PRISMA 2020, PRISMA-S, and adjacent AI-reporting proposals. PRISMA 2020 provides the baseline reporting structure for the review itself, while PRISMA-S demonstrates how a technically consequential component of the review can require dedicated reporting detail [5, 6]. PRISMA-AI focuses on reviews and meta-analyses of AI systems in healthcare; PRISMA-trAIce addresses AI-tool identification, human–AI interaction, performance evaluation, and limitations; PRISMA-DFLLM emphasizes domain-specific LLMs, fine-tuning, reproducibility, and ethical considerations; and L-PRISMA adds reporting for GenAI-assisted screening and summarization [7–10]. PRISMA-LLM differs from these proposals by deriving its graded reporting levels from observed patterns in a large corpus of review-automation papers. Its reporting scale is motivated by the relationship we observe between method type, reporting richness, and workflow complexity in SciLitBench. PRISMA-LLM is 6

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

intended to complement existing efforts by linking a detailed implementation-disclosure layer to reporting dimensions in an evidence-based approach. The framework has three layers (Figure 5). First, a universal checklist specifies the information that should be reported across LLM-assisted review workflows (Table 1). Second, five implementation-disclosure levels organize the additional technical detail needed to reconstruct workflows from off-the-shelf tools to coordinated multi-agent systems. Third, consequence-sensitive evaluation expectations determine when validation should exceed the minimum associated with a disclosure level. The levels are therefore disclosure tiers, not risk tiers. A technically simple workflow can still be highly consequential if it autonomously excludes records or supplies unverified extracted evidence; conversely, a technically elaborate workflow used only for non-substantive drafting may have limited influence on the evidence base. Task consequence, degree of human verification, system opacity and reversibility should therefore be reported explicitly and can justify stronger evaluation at any implementation level. The cumulative minimum reporting dimensions associated with the five implementation levels are summarized in Supplementary Figure S6. Layer 1: universal reporting checklist. Every LLM-assisted review should identify the system and task, describe inputs and outputs, report human oversight, disclose evaluation and failures, and provide reproducibility materials where possible (Table 1). Layer 2: implementation-disclosure level. Level

Workflow

Additional implementation detail

Off-the-shelf software

Prompt-only LLM

Structured/engineered

Retrieval/adapted/singleagent Multi-agent/swarm

Product/version, interface, role, settings, validation status and availability constraints. Level 1 plus model identity, prompts/system instructions, inference settings, repetitions and output constraints. Level 2 plus examples, schemas/rubrics, prompt chains, voting/ensembles, LLM-as-judge and post-processing. Level 3 plus retrieval sources, parsing/chunking, embeddings, adaptation/fine-tuning, thresholds, agent actions and failure handling. Level 4 plus agent roles, coordination, handoffs, termination, shared state, conflict resolution and cross-agent audit logs.

Layer 3: consequence-sensitive evaluation. Minimum evaluation breadth increases with implementation complexity, but implementation level is not a risk score. At any level, stronger validation is expected when AI outputs can change search retrieval, inclusion/exclusion, extracted evidence, risk-of-bias judgments, synthesis or conclusions; when human verification is limited; when the system is proprietary or mutable; or when errors are difficult to reconstruct or reverse.

Figure 5: The three-layer structure of PRISMA-LLM. The universal checklist specifies what every report should disclose; the implementation-disclosure level specifies the technical details needed to reconstruct the workflow; and task consequence, human verification, opacity and reversibility determine whether evaluation should exceed the minimum associated with the implementation level. This separates implementation complexity from methodological risk.

Discussion

Across 888 papers, review automation is becoming more common and moving into stages where computational outputs can determine which studies are retained and what evidence is extracted from them. At the same time, the information available to evaluate those workflows is uneven. Software/product papers were especially likely to omit paper-level evaluation, whereas LLM papers reported progressively more evaluation detail as workflows became more engineered. These findings identify a practical accountability gap: adoption can be straightforward even when the evidence needed to audit the adopted system is sparse. A second finding is that favorable assessments and unmet reliability requirements frequently coexist. More than half of LLM papers with positive-only overall evaluations nevertheless reported at least one concern that the workflow remained below a performance or reliability bar for its intended use. This distinction matters because “useful” is not the same claim as “adequate for delegation.” A tool may reduce workload, improve recall, or support extraction while still requiring human verification at exactly the points where errors could alter the review’s evidence base. Reporting should therefore make both utility and residual failure modes visible, together with who retained final responsibility for consequential decisions. The observed association between workflow complexity and reporting richness provides an empirical rationale for graded disclosure, but complexity should not be mistaken for risk. The present categories capture implementation 7

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

Section

Item

What to report

Identification

LLM-1: Title/abstract

Rationale Methods

LLM-2: Purpose LLM-3: System identification LLM-4: Task and disclosure level LLM-5: Inputs/document processing LLM-6: Prompts/settings

Identify substantive LLM or AI-enabled software use and the review stage(s) affected; distinguish review decisions/evidence processing from writing-only assistance. Explain why AI was used and which review burden or methodological problem it was intended to address. Report system/tool name, model/version, provider, access date, interface/API, deployment mode, and whether access was proprietary, open-weight, local, hosted or custom. Specify each review task, assign the PRISMA-LLM implementation-disclosure level, and state whether outputs could alter retrieval, inclusion/exclusion, extraction, appraisal, synthesis or conclusions. Describe records, abstracts, PDFs, tables, supplements, examples, labels and prior decisions supplied to the system, including parsing, OCR, section/table handling, chunking, context limits, truncation and retrieval. Report prompts and system instructions, examples, schemas/rubrics, inference settings, repetitions/sampling, output constraints and model-specific settings needed to reproduce the workflow. Describe generated labels, rankings, extracted fields, judgments or text; output schemas; parsing/format repair; thresholds; confidence scores; and how model outputs became review decisions or analysis data. Report consequential prompt, schema, retrieval or orchestration changes, the rationale for the final workflow, and important failed or alternative configurations that informed it. For screening, extraction, appraisal, annotation or LLM-as-judge use, report task-specific decision rules, reference standards, calibration, thresholds, edge-case handling and adjudication procedures. State who reviewed AI outputs, what proportion was independently checked, how disagreements/corrections were handled, and who retained final responsibility for consequential decisions. Pre-specify performance, comparative, modification/optimization, resource/feasibility, and limitation/failure-mode evaluation as applicable; strengthen evaluation when task consequence, limited verification, opacity or irreversibility warrants it. Report numbers of records, reports, fields, annotations, claims or judgments processed by AI, humans or both, including how automation altered the flow of records through the review. Report task-specific performance and required comparative/modification/resource evidence, plus false positives/negatives, hallucinations, extraction/parsing failures, disagreements, corrections and audit outcomes. Explain model/version dependence, prompt sensitivity, validation limits, proprietary opacity, remaining human workload and how observed or plausible failures could affect the evidence base or conclusions. Provide prompts, system instructions, settings, schemas, code, validation samples, labels, audit logs and raw/parsed outputs when legally and ethically possible; report data-sharing, copyright, privacy, vendor and terms-of-service constraints. Report sponsored access, credits, private model access, vendor involvement, material support and relevant competing interests.

Methods Methods Methods Methods Methods Methods Methods Methods

Results Results Discussion Availability

Disclosure

LLM-7: Outputs/post-processing LLM-8: Workflow development LLM-9: Task-specific workflow LLM-10: Human oversight LLM-11: Evaluation plan LLM-12: AI/human processing counts LLM-13: Evaluation and errors LLM-14: Limitations and implications LLM-15: Reproducibility materials LLM-16: Support and interests

Table 1: PRISMA-LLM reporting checklist for LLM-assisted and AI-enabled systematic reviews. This page-sized checklist is the primary user-facing framework and is intended to be used alongside PRISMA 2020 rather than as a replacement for it. The expanded item wording is provided in Supplementary Table S4. Implementation-level instructions, a fillable checklist workbook, and the checklist evidence table are available at https://github.com/ linlab/PRISMA-LLM.

choices needed to reconstruct a workflow; they do not measure the consequence of an error, the amount of human oversight, or the opacity of a proprietary system. An off-the-shelf product used to autonomously exclude records may require more stringent validation than a multi-agent system used only to draft text that is fully checked by authors. We therefore position the proposed PRISMA-LLM levels as disclosure tiers and add an explicit consequence override: workflows that can change the evidence base should report task-specific validation and human verification regardless of level. The study also has limitations. SciLitBench is concentrated in life sciences and medicine, covers publications through June 2025, and reflects the search and inclusion criteria of the parent systematic review. Reporting richness counts annotated evidence items and captures breadth with limited within-dimension depth; it is not a validated study-quality score. The observed complexity gradient is descriptive and may be confounded by study purpose, venue, task, publication year, or the fact that engineered workflows are more often themselves the object of method-development studies. Paper-level absence of evaluation also does not prove that no validation exists elsewhere, for example in a product report, protocol, repository, or prior publication. Finally, the proposed checklist combines empirical observations with workflow experience and prior reporting guidance; it has not yet undergone formal consensus development or prospective user testing. We introduce PRISMA-LLM as an empirically grounded reporting framework proposed in this study. It is not presently an official extension endorsed by the PRISMA Executive. Formal consensus development, independent usability testing and prospective evaluation can test and refine the framework as AI-assisted review practice evolves. When AI systems participate in evidence synthesis, the computational path from source documents to review decisions should be inspectable. This requires reporting what the system received and produced, what humans checked, where failures occurred, and how those failures could have affected the evidence base. PRISMA-LLM translates these 8

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

requirements into a checklist, implementation-disclosure scheme and consequence-sensitive evaluation framework for testing and revision.

Methods

3.1

Study design and corpus

This study is a secondary analysis of the 888 full-text papers included in the SciLitBench corpus [11]. SciLitBench was built from a systematic review of computational methods used to automate literature reviews. Its librarian-informed search covered PubMed, Semantic Scholar, and Scopus and yielded 42,981 candidate records after database export and deduplication. The corpus includes papers published through June 2025. The upstream screening workflow identified 1,819 title-and-abstract inclusions, retrieved 1,012 full texts, and produced 888 final inclusions after full-text screening and duplicate removal. Each included paper serves as one unit of analysis. The corpus contains six annotated fields: publication year, domain, review stage, computational approach, evaluation results, and limitations. Each paper has one high-level and one medium-level domain label. Review stage and computational approach are multi-label fields. The evaluation-results and limitations fields contain item-level evidence statements, including quotations preserved from the source paper. The harmonized analysis inventory contains 14,726 annotation items across the 888 papers: 888 year values, 1,776 domain labels, 1,350 review-stage labels, 1,254 approach labels, 6,562 evaluation-result items, and 2,896 limitation items (Supplementary Table S1). 3.2

Publication dates and temporal analysis

The source annotations provided publication year. We recovered publication month for all 888 papers using bibliographic databases, publisher records, and dates reported in source documents. The final date table contains one unique paper identifier and a recovered month and year. A separate override file documents 33 manually adjudicated dates and the evidence or decision basis for each. All temporal analyses end in June 2025, so the 2025 results represent January through June rather than a complete publication year. Monthly plots use rolling 3-month windows. For each displayed month, paper counts were summed over that month and the preceding two months before calculating proportions. Adjacent points therefore contain overlapping papers. Domain-level LLM adoption was calculated as the number of papers annotated with an LLM approach divided by the total number of papers in that domain and rolling window. For review-stage composition, each paper received a total weight of one, divided equally across its review-stage groups, before the rolling proportions were calculated. Approach-orientation and method-complexity plots use one exclusive group per paper, as defined below. Publication growth was calculated from unique papers per publication month. The exponential curve in Figure 1b was fitted to positive monthly counts from January 2020 through June 2025 by ordinary least squares on the log count, log(Nm ) = α + βm. The fitted monthly values were summed into 3-month calendar periods for display. The monthly growth rate was calculated as exp(β) − 1, and the 2026 extrapolation was calculated as the sum of the 12 fitted monthly values. This extrapolation was used only to summarize the recent trajectory; it was not treated as a calibrated forecast or evidence of a causal break at the release of any model or product. 3.3

Review-stage, method, and model groupings

Review-stage labels were grouped according to their role in the review. The discovery and navigation group includes search or retrieval, topic modeling or clustering, and text mining or information synthesis. The screening and selection group includes title-and-abstract screening, full-text screening, and screening used for ranking or prioritization. The evidence construction group includes data extraction, risk-of-bias or quality assessment, and claim verification. A paper may contribute to more than one review-stage group. We used both multi-label approach groups and an exclusive approach orientation. The multi-label groups are traditional/deep-learning/BERT, LLM, and software/product. Papers containing more than one approach can contribute to more than one of these groups. The exclusive orientation classifies a paper as custom ML/BERT/deep-learning/ruleoriented, LLM/software-facing, hybrid, or other/unclear. Hybrid papers contain at least one custom approach and at least one LLM or software/product approach. We used these definitions for approach trends and reporting comparisons since 2023. 9

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

High-level domain labels were used for domain comparisons. Under the final domain schema, papers spanning multiple substantially different domains or describing their corpus too broadly for one ASJC assignment were annotated as unknown / not reported. We display this group as Mixed in the analysis. Model-family values were normalized from the approach annotations. BERT-family labels were grouped as biomedical variants, generic BERT, BERTopic, scientific embedding variants, sentence embedding variants, general transformer variants, or other BERT-like models. LLM-family labels were grouped as GPT/ChatGPT, Claude, Gemini/Bard/PaLM, Llama, Mistral/Mixtral, other open-weight or open-family models, other proprietary or hosted models, or unclear. The model-family panels include papers from 2015 onward; the arts and humanities category was omitted because it contained only two papers, and unclear LLM families were omitted from the LLM-family panel. For the access analysis, models such as GPT, Claude, and Gemini were classified as proprietary or hosted, while models such as Llama, Mistral, and Qwen were classified as open-weight or open-family. Papers containing both groups were labeled mixed. Model access was coded only when the family could be identified. 3.4

Reporting measures and empirical method complexity

The five dimensions used for reporting richness were derived from SciLitBench’s evaluation-result and limitation annotations. The SciLitBench annotation team constructed starting schemas from 50 sampled papers and applied them to the full 888-paper corpus with Llama 3.3-70B. The team then reviewed source quotations and adjudicated the final categories and mappings. This process consolidated recurring evidence into four evaluation dimensions: performance, comparisons, modifications or optimization, and resources or feasibility, plus one dimension for limitations or failure modes. Full details of the annotation schemas, prompts, harmonization procedures, and annotation-quality checks are provided in the SciLitBench paper [11]. Reporting richness is a descriptive paper-level index with a maximum possible score of 15. It sums five dimension scores: performance evidence, comparative evidence, optimization or modification evidence, resource or feasibility evidence, and limitation or failure-mode reporting. Each dimension contributes 0 points for no items, 1 point for one item, 2 points for two or three items, and 3 points for four or more items. The score captures reporting breadth and a limited measure of within-dimension depth; study quality requires separate appraisal. Performance evidence is the number of items categorized as performance versus human annotation; comparative evidence is the number of comparative-result items; modification evidence is the number of feature- or modificationeffect items; and resources or feasibility is the number of runtime-and-cost items. The limitation dimension counts all limitation items except none reported. Overall qualitative impressions, none reported, and other evaluation items do not contribute to the score. Papers were classified as having no evaluation when their evaluation annotations contained the none reported category. For the supplementary sentiment analysis, overall-evaluation mappings were combined at the paper level. Papers with positive mappings and no mixed or negative mapping were labeled positive only. Papers with both positive and mixed or negative mappings were labeled positive with caveats. Papers containing mixed or negative mappings without a positive mapping were labeled mixed/negative; the remainder were descriptive or unclear. A paper was marked as reporting an unmet high bar when at least one limitation item was categorized as high bar needed but not achieved. The empirical method-complexity groups in Figure 3 were assigned from the approach categories and the normalized method labels attached to LLM annotations. The highest applicable LLM group took precedence. Retrieval, adapted, or agentic workflows required an explicit retrieval method such as RAG, model adaptation such as fine-tuning, or agentic orchestration. Engineered or structured workflows included techniques such as few-shot learning, structured prompting, voting or ensembles, and LLM-as-evaluator designs. Prompt-only workflows used generic or zero-shot prompting without one of these additional methods. Plain zero-shot use was therefore prompt-only, while zero-shot chain-ofthought was engineered. The prompt-only group was restricted to papers with an identified generic or zero-shot method; unresolved cases were labeled LLM method unclear. Non-LLM papers were labeled neural non-LLM when they contained BERT or deep learning, rule/traditional ML when they contained traditional machine learning or rule-based approaches, and software/product only when software was the only substantive approach category. Remaining papers were labeled other/unclear. These groups are analytical categories for the corpus and are separate from the five-level PRISMA-LLM proposal. The executable mapping is available with the analysis code. 3.5

Development of PRISMA-LLM

We developed PRISMA-LLM from three inputs: the reporting patterns observed in the SciLitBench corpus, the implementation choices encountered while building SciLitBench, and a comparison with PRISMA 2020 and adjacent reporting proposals for search methods and AI-assisted reviews. The empirical layer uses the five reporting dimensions 10

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

above and the observed tendency for more complex LLM workflows to report more evaluation and limitation detail. The implementation layer covers choices that can change the behavior of an automated review workflow, including model or tool identity, prompts and settings, schemas, document processing, retrieval and chunking, thresholds, post-processing, human oversight, adjudication, and reproducibility materials. We mapped each checklist item to the empirical findings, workflow experience, or prior reporting guidance that motivated it (Supplementary Table S2). The proposed implementation scale contains five levels. Level 1 covers off-the-shelf tools or software; Level 2 covers prompt-only LLM use; Level 3 covers few-shot, structured, engineered, voting, ensemble, or LLM-as-judge workflows; Level 4 covers retrieval-augmented, adapted, fine-tuned, or single-agent workflows; and Level 5 covers multi-agent or swarm systems. These levels summarize implementation complexity rather than methodological risk. The evaluation expectations provide a minimum floor that accumulates across levels, while consequential tasks can trigger stronger requirements regardless of level. Reporting richness remains a descriptive empirical index because no compliance cutoff has undergone prospective validation. The implementation checklist was mapped to PRISMA 2020 and informed by related reporting guidance, including PRISMA-S, PRISMA-AI, PRISMA-trAIce, PRISMA-DFLLM, and L-PRISMA. The framework is fully specified here; formal Delphi consensus development and prospective user testing are additional validation steps rather than prerequisites for scientific evaluation of the proposal. 3.6

Statistical analysis

The paper is the unit of analysis. Counts and percentages are descriptive unless a statistical test is named. Multi-label approach and review-stage percentages use papers as denominators and may sum to more than 100%. For reportingrichness comparisons, we report arithmetic means and the proportion of papers with no evaluation annotation. Where specified, comparisons by approach and empirical method complexity included only papers published from 2023 onward. Differences in proprietary versus open-weight LLM use across high-level domains were evaluated after excluding mixed-access and unannotated papers. A global chi-square statistic was compared with 5,000 label permutations using a fixed random seed of 20260630. Pairwise domain comparisons used two-sided Fisher’s exact tests, with Benjamini-Hochberg correction across domain pairs. Statistical significance was assessed at a two-sided α = 0.05. For exploratory analyses of reported performance evidence, metric names and nearby numeric values were parsed from performance-versus-human quotations. Sensitivity was normalized to recall. Percentages and values greater than 1 and no greater than 100 were converted to the 0–1 scale; fractions and values outside that range were excluded. Multiple values for the same paper, model class, stage group, and metric were collapsed to their median. We excluded LLM/BERT hybrid papers from direct model-class comparisons because paper-level annotations often leave the source model for individual values unresolved. Reported values were summarized by the median and interquartile range. We calculated two-sided Mann-Whitney U tests with tie correction only when each comparison group contained at least five paper-level values. These metric summaries are exploratory inventories of heterogeneous reported evidence rather than controlled benchmark comparisons, and no multiplicity correction was applied to them.

Code and materials availability Analysis code, the PRISMA-LLM reporting instructions, a fillable checklist workbook, and the checklist evidence table are available at https://github.com/linlab/PRISMA-LLM. Access to the underlying SciLitBench corpus and source-derived materials follows the terms described in the SciLitBench resource.

Acknowledgments BL is supported by the U.S. National Institutes of Health (NIH), the U.S. Department of Veterans Affairs (VA), the Brain & Behavior Research Foundation (BBRF), the Sidney R. Baer, Jr. Foundation, the Hasso Plattner Foundation, and the Windreich Family Foundation. No funding was specific to this work.

References [1] Tsafnat, G. et al. Systematic review automation technologies. Systematic Reviews 3, 74 (2014). URL https: //doi.org/10.1186/2046-4053-3-74. 11

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

[2] Marshall, I. J. & Wallace, B. C. Toward systematic review automation: A practical guide to using machine learning tools in research synthesis. Systematic Reviews 8, 163 (2019). URL https://doi.org/10.1186/ s13643-019-1074-9. [3] Scherbakov, D., Hubig, N., Jansari, V., Bakumenko, A. & Lenert, L. A. The emergence of large language models as tools in literature reviews: A large language model-assisted systematic review. Journal of the American Medical Informatics Association 32, 1071–1086 (2025). URL https://doi.org/10.1093/jamia/ocaf063. [4] National Academies of Sciences, Engineering, and Medicine. Reproducibility and Replicability in Science (National Academies Press, Washington, DC, 2019). URL https://doi.org/10.17226/25303. [5] Page, M. J. et al. The prisma 2020 statement: An updated guideline for reporting systematic reviews. BMJ 372, n71 (2021). URL https://doi.org/10.1136/bmj.n71. [6] Rethlefsen, M. L. et al. Prisma-s: An extension to the prisma statement for reporting literature searches in systematic reviews. Systematic Reviews 10, 39 (2021). URL https://doi.org/10.1186/s13643-020-01542-z. [7] Cacciamani, G. E. et al. Prisma ai reporting guidelines for systematic reviews and meta-analyses on ai in healthcare. Nature Medicine 29, 14–15 (2023). URL https://doi.org/10.1038/s41591-022-02139-w. [8] Holst, D., Moenck, K., Koch, J., Schmedemann, O. & Schuppstuhl, T. Transparent reporting of ai in systematic literature reviews: Development of the prisma-traice checklist. JMIR AI 4, e80247 (2025). URL https: //ai.jmir.org/2025/1/e80247. [9] Susnjak, T. Prisma-dfllm: An extension of prisma for systematic literature reviews using domain-specific finetuned large language models. arXiv:2306.14905 (2023). URL https://arxiv.org/abs/2306.14905. [10] Shailendra, S., Kadel, R., Sharma, A., Tahidul, I. M. & Saxena, U. R. L-prisma: An extension of prisma in the era of generative artificial intelligence (genai). arXiv:2603.19236 (2026). URL https://arxiv.org/abs/2603. 19236. [11] Zabaleta, M. & Lin, B. SciLitBench: Benchmark and design principles for LLM-powered systematic literature reviews. arXiv preprint arXiv:2609.05505 (2026). URL https://arxiv.org/abs/2609.05505. [12] Page, M. J. et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: A cross-sectional study. PLoS Medicine 13, e1002028 (2016). URL https://doi.org/10.1371/journal.pmed. 1002028. [13] Hoffmann, F. et al. Nearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000-2019. Journal of Clinical Epidemiology 138, 1–11 (2021). URL https://doi.org/10.1016/j.jclinepi.2021.05.022. [14] Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G. & The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The prisma statement. PLoS Medicine 6, e1000097 (2009). URL https://doi.org/10.1371/journal.pmed.1000097. [15] Byun, J. & Stuhlmüller, A. Elicit: Language models as research tools. In OECD, Artificial Intelligence in Science: Challenges, Opportunities and the Future of Research (2023). URL https://www.oecd.org/en/publications/artificial-intelligence-in-science_a8d820bd-en/ full-report/elicit-language-models-as-research-tools_fec8a6ab.html. [16] DistillerSR. The distillersr platform. https://www.distillersr.com/products/ distillersr-systematic-review-software. Accessed 2026-07-06. [17] Lieberum, J.-L. et al. Large language models for conducting systematic reviews: On the rise, but not yet ready for use—a scoping review. Journal of Clinical Epidemiology 181, 111746 (2025). URL https://doi.org/10. 1016/j.jclinepi.2025.111746.

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

Supplementary Information The supplementary material provides calculation details and supporting analyses for the empirical results, followed by the expanded PRISMA-LLM checklist and implementation-level details. S1

Supplementary methods

Analysis denominators and multi-label fields. All supplementary percentages use papers as the denominator. Approach, review-stage, limitation, and model-family analyses are multi-label unless an exclusive group is named. A paper can therefore contribute to more than one approach or review-stage group, and percentages across groups need not sum to 100%. The number of papers contributing to a cell is reported in the figure, caption, or accompanying table when needed to interpret the estimate. Overall impressions and high-bar concerns. The overall-impression analysis is restricted to papers annotated with an LLM approach. Item-level impression mappings were collapsed into four exclusive paper-level groups: positive only; positive with caveats; mixed or negative; and descriptive or unclear. Monthly values use the rolling 3-month procedure described in the Methods. The high-bar analysis uses the same paper-level impression groups and marks a paper as high-bar unmet when at least one limitation item states that the method did not meet the performance or reliability bar required for its intended use. The percentages in Figure 4b use all LLM papers in each impression group as the denominator. Limitation profiles and software reporting. Limitation prevalence was calculated as the share of papers in a group with at least one item in the named limitation category. Supplementary Figure S2 reports four recurring categories used in the reporting analysis: an unmet high bar, limited validation, parameter or prompt sensitivity, and small or narrow data. The categories are not mutually exclusive. The software-reporting analysis includes papers published from 2023 onward. Papers were grouped by approach and review stage using the multi-label definitions in the Methods. Substantive evaluation items include performance, comparative, modification, and runtime or cost annotations; qualitative impressions, none reported, and other items are excluded. Within each approach-stage cell, we calculated the mean number of substantive evaluation items per paper and the share of papers annotated with none reported. Reported performance evidence. Metric names and values were parsed from performance-versus-human annotations as described in the Methods. Supplementary Figure S5 and Supplementary Table S3 include only papers that reported both precision and recall for screening or data extraction. When a paper reported more than one value for the same approach, stage, and metric, values were collapsed to the paper-level median. The large diamonds in the figure mark the median precision and recall within each approach-stage group. S2

Supplementary analyses

Positive assessments and unmet reliability requirements. Overall impressions of LLM-assisted review automation are frequently positive, while limitation reporting gives a more qualified view of what the methods can do. Among the 118 LLM papers with positive-only overall evaluations, 52% also report at least one high-bar-unmet concern. The corresponding share rises to 67% among the 57 papers with positive evaluations and caveats and to 79% among the 38 papers with mixed or negative evaluations. Among the 35 descriptive or unclear papers, the share is 14% (Figure 4b). The rolling analysis shows positive assessments throughout the observed LLM period, alongside continuing caveated and mixed assessments (Figure 4a). Usefulness and adequacy therefore emerge as separate judgments in the literature: authors can find an LLM workflow promising while also reporting that it remains below the reliability required for its intended review role. Limitation profiles vary across methods. The kinds of limitations authors report also differ across automation approaches. Among LLM papers, 54% report an unmet high bar, 40% report parameter or prompt sensitivity, 32% report limited validation, and 45% report small or narrow data. Software/product papers report the same categories less often: 26%, 15%, 24%, and 34%, respectively. Earlier approaches show a different profile. Small or narrow data is reported by 47% of traditional-ML papers and 52% of BERT papers, while high-bar-unmet concerns appear in 33% and 40% (Supplementary Figure S2). The reported limitation profiles therefore differ by method. Prompt sensitivity appears especially often in LLM papers, whereas data scope remains prominent across both earlier and current approaches. The software reporting gap persists across review stages. The lower reporting observed for software/product papers appears within each review-stage group. In discovery and navigation, software/product papers report 3.0 substantive 13

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

evaluation items on average, compared with 5.2 for LLM papers and 4.8 for traditional, deep-learning, or BERT papers. Their no-evaluation rate is also higher at 32.5%, compared with 12.3% and 16.3%. The difference is largest in screening and selection: software/product papers average 3.2 evaluation items and 39.3% report no evaluation, whereas LLM papers average 7.3 items with a 3.8% no-evaluation rate. Evidence construction shows the same direction, with means of 5.3 for software/products, 6.2 for LLMs, and 9.0 for earlier custom approaches (Supplementary Figure S4). Stage-stratified comparisons preserve the software-reporting gap, showing that the aggregate difference is not explained solely by review-stage composition. Reported performance evidence is heterogeneous. Papers reporting both precision and recall show substantial variation within every approach group. For screening and selection, median recall is 0.821 for custom ML, deeplearning, BERT, or rule-based methods, 0.884 for LLMs, and 0.851 for software/products; median precision is 0.742, 0.695, and 0.667, respectively. For data extraction, median recall is 0.769, 0.890, and 0.778, while median precision is 0.776, 0.881, and 0.678 (Supplementary Figure S5 and Supplementary Table S3). The individual paper values span much wider ranges than the group medians suggest. Given this dispersion and variation in datasets, tasks, thresholds, and evaluation designs, these metrics provide a descriptive inventory of the evidence reported by the studies. The observed variation also supports reporting task-specific performance together with the evaluation context needed to interpret it. S3

Supplementary Figures

a) BERT-family composition

b) LLM model-family composition

Figure S1: Model-family use by field. Bars show within-field distributions of model-family mentions. Because a paper may mention more than one family, displayed n values are paper–family mentions rather than unique papers. Both panels include papers published from 2015 onward; arts and humanities is omitted because it contains only two papers. a) BERT-family composition. Biomedical variants are most prominent in life sciences and medicine, whereas other fields contain a broader mixture of generic BERT, BERTopic, embedding, and transformer variants. b) LLM-family composition among papers with an identifiable LLM family. GPT/ChatGPT-family systems dominate across fields; Claude, Gemini/Bard/PaLM, Llama, Mistral/Mixtral, and other families appear less often. The panels describe reported model use and do not compare model performance.

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

a) By approach

b) By high-level domain

c) By BERT family

d) By LLM family

Figure S2: Recurring limitations by approach, domain, and model family. Cells show the percentage of papers within each row group that report at least one item assigned to the corresponding limitation category. Panels group papers by a) approach, b) high-level domain, c) BERT family, and d) LLM family. All panels use the same 0–70% color scale, with higher prevalence shown in red. Limitation categories are not mutually exclusive, so a paper can contribute to multiple columns. Papers can also contribute to multiple approach or model-family groups; those panels therefore show within-group prevalence rather than mutually exclusive partitions of the corpus. Model-family estimates should be interpreted alongside group size because several families contain few papers.

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

a) Approach adoption over time

b) Reporting richness over time

Figure S3: Approach prevalence and evaluation reporting over time. a) Annual share of papers carrying each approach label. Approach categories are multi-label, so shares may sum to more than 100% across groups. b) Mean paper-level reporting-richness score by approach and publication year. The 0–15 index summarizes five dimensions of reported evaluation and limitations and should not be interpreted as study quality. LLM papers show increasing reporting coverage after 2023, whereas software/product papers remain common but report less paper-level evaluation and limitation detail. Values for 2025 include January through June only.

a) Mean evaluation items

b) No evaluation reported

Figure S4: Evaluation reporting by approach and review stage since 2023. Bars summarize papers published from 2023 onward within each approach–stage combination. a) Mean number of substantive evaluation items reported per paper. b) Percentage of papers reporting no evaluation. Approach and review-stage labels are multi-label, so papers can contribute to more than one group. Within each review-stage group, software/product papers report fewer evaluation items and are more likely to report no evaluation than traditional ML, deep-learning, or BERT papers and LLM papers. The aggregate difference is therefore not explained solely by differences in review-stage composition.

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

a) Screening and selection

b) Data extraction

Figure S5: Reported precision and recall for screening and data extraction. a) Screening and selection. b) Data extraction. Each small point represents one paper that reported both precision and recall, colored by approach group; legend values give the number of papers. Large diamonds mark the within-group medians for the two metrics. The dotted vertical and horizontal lines mark precision and recall of 0.8, respectively. Values come from heterogeneous studies using different datasets, tasks, thresholds, and evaluation designs.

Figure S6: PRISMA-LLM evaluation and limitation reporting map. Rows define five proposed levels of workflow complexity, and marked cells indicate the minimum reporting dimensions associated with each level. Expectations are cumulative: all workflows should report limitations or failure modes; prompt-only workflows add task-specific performance evidence; structured or engineered workflows add comparative and modification evidence; and retrieval, adapted, agentic, and multi-agent workflows cover all five dimensions. The levels reflect current review-automation practice and should be revised as the field changes. At any level, stronger task-specific validation is expected when AI outputs can alter retrieval, inclusion/exclusion, extracted evidence, appraisal, synthesis, or conclusions; when human verification is incomplete; when the system is opaque or mutable; or when errors are difficult to reconstruct or reverse. 17

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

Supplementary Tables

Field or domain

Record fields year domain review stage approach evaluation results limitations total

888 1776 1350 1254 6562 2896 14726

Domains Life sciences/medicine Engineering/technology Social sciences/management Mixed domain Natural sciences Arts/humanities

548 61.7% 140 15.8% 115 13.0% 62 7.0% 21 2.4% 2 0.2%

Annotation category

Items Papers

Evaluation results performance vs human annotation overall sentiment / qualitative impression comparative results feature or modification effects runtime and cost none reported

1788 1778 1149 959 722 163

395 663 386 353 320 163

698

373

642 445

295 229

393

196

256 199 238

146 118 238

Limitations small data, narrow scope, or limited data availability high bar needed but not achieved limited validation or benchmarking of AI outputs performance sensitive to parameters or prompts high human workload demands subjectivity and labeling bias none reported

Table S1: Corpus composition and annotation inventory. The left block reports item counts across the six annotated fields and the distribution of 888 papers across mutually exclusive high-level domains. The right block reports item counts for recurring evaluation-result and limitation categories and the number of papers contributing at least one item to each category. Papers can contribute multiple items and can appear in more than one category, so category-level paper counts are not additive.

PRISMA-LLM : An Empirical Reporting Framework for AI-Assisted Systematic Reviews

Basis

Contribution to the checklist

PRISMA-LLM: An Empirical Reporting Framework for AI-Assisted Systematic Reviews

Related documents