ATIBA: Grounded Integrity and Quality Checking for Research Papers

arXiv:2609.04123v1 [cs.SE] 3 Sep 2026

ATIBA: Grounded Integrity and Quality Checking for Research Papers Veli Karakaya∗

Semih Çağlar∗

Bilkent University Computer Engineering Ankara, Turkiye veli.karakaya@ug.bilkent.edu.tr

Bilkent University Computer Engineering Ankara, Turkiye semih.caglar@ug.bilkent.edu.tr

Yusuf Yiğit Korkmaz∗

Eray Tüzün✉

Bilkent University Computer Engineering Ankara, Turkiye yigit.korkmaz@ug.bilkent.edu.tr

Bilkent University Computer Engineering Ankara, Turkiye eraytuzun@cs.bilkent.edu.tr

Abstract Checking a manuscript’s reference integrity, its compliance with a target venue’s specific submission rules, and its adherence to community reporting standards is manual, repetitive, and different for every venue so in practice it is done inconsistently or skipped. We present ATIBA, a tool that runs five grounded integrity and quality checks on a manuscript: a reference-integrity check that verifies each citation against bibliographic sources and flags retracted or unfindable references; a venue/track compliance check that derives submission criteria directly from a venue’s own call-for-papers page and evaluates the manuscript against them, each verdict anchored to a verbatim quote from that page; an empirical-standards compliance check against the ACM SIGSOFT Empirical Standards, with a hallucination defence that discards any evidence quote it cannot locate verbatim in the manuscript; a multi-mode AI review (venue-specific, formal, and page-anchored annotation) powered by GPT-5.4 through Azure OpenAI; and a citation-suggestion feature that proposes candidate references for a manuscript and verifies each against bibliographic sources before it is shown to the user. All five checks are designed around the same principle: an LLM is only trusted to judge, never to invent the evidence it judges against. We evaluated ATIBA through a moderated user study with 13 nonauthor participants. Agreement across the six survey items ranged from 69% to 92%, with a mean of 85%, providing initial evidence of positive perceived usefulness across the evaluated workflows. These findings establish perceived usefulness; objective accuracy remains to be measured.

CCS Concepts • Applied computing → Publishing; • Software and its engineering; • Social and professional topics → Codes of ethics;

Keywords Research integrity, Research quality, Automated peer review, Reference hallucination, Large language models, Venue compliance ∗ These authors contributed equally to this work.

ATIQSER ’26, Munich, Germany 2026. ACM ISBN 979-8-4007-2999-7/2026/10 https://doi.org/10.1145/3844135.3845865

ACM Reference Format: Veli Karakaya, Semih Çağlar, Yusuf Yiğit Korkmaz, and Eray Tüzün. 2026. ATIBA: Grounded Integrity and Quality Checking for Research Papers. In Proceedings of the 1st International Workshop on Automated Techniques for Integrity and Quality in Software-Engineering Research (ATIQSER ’26), October 12–16, 2026, Munich, Germany. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3844135.3845865

1

Introduction

Submission counts at top-tier computing venues have grown sharply in recent years, placing increasing strain on traditional peer review [6, 11], a trend also reflected in ASE 2025 submission data [30]. Reviewing a submission for integrity and quality problems means checking several different things at once: are the references real and verifiable [10, 29], have any of them been retracted, does the manuscript follow the target venue’s specific formatting and anonymization rules, does it report its methodology the way the community expects, and is the contribution itself sound. Authors are expected to check relevant submission requirements before submitting, while venues perform editorial screening before manuscripts are sent for external review [9, 18]. These checks span formatting, scope, integrity, and other quality criteria, and the applicable requirements vary across publication venues [18]. In practice, carrying them out consistently imposes substantial effort, particularly because each venue defines its own aims, review standards, and publication requirements. Existing tooling addresses pieces of this problem but not the combination. Conference management systems such as HotCRP [16] and EasyChair [1] route submissions and assignments but perform no content-level integrity checking at all. General-purpose large language model (LLM) assisted review tools can produce plausiblesounding feedback on a manuscript [17], but an LLM asked to judge a paper’s references or its compliance with a venue’s rules is exactly as capable of inventing a plausible-sounding but false verdict as it is of catching a real problem [10]. ATIBA1 is built around a different default: every one of its five checks is grounded in something outside the LLM’s own output, such as a bibliographic lookup, a verbatim quote from the venue’s own page, or a verbatim quote from the manuscript itself. This 1 ATIBA is deployed and publicly available at https://atiba.onrender.com.

ATIQSER ’26, October 12–16, 2026, Munich, Germany

grounding allows a check to fail loudly, by returning no verdict or an explicitly low-confidence one, instead of fabricating an answer. The same manuscript, run once, gets a reference-integrity report, a venue-specific compliance report, an empirical-standards report, one of three styles of AI-generated review, and a set of verified candidate citations, all cached and re-runnable on demand. This paper makes the following contributions: (1) Grounded Integrity Pipeline: A multi-stage architecture that automates reference-integrity verification, venue/track compliance extraction, and empirical-standards checking against the ACM SIGSOFT Empirical Standards [21]. (2) Verbatim Hallucination Defence: A strict grounding mechanism that automatically discards LLM-generated feedback or compliance verdicts whenever supporting quotes cannot be located verbatim in the manuscript or venue source text. (3) Context-Aware Citation & Multi-Mode Review: A verified citation-suggestion engine and a venue-aware prompting layer providing three complementary AI review modes— venue-specific, formal, and page-anchored annotated review. (4) Empirical Evaluation: A moderated user study (𝑁 = 13) demonstrating high perceived usefulness across all five checks in real-world paper writing and internal peer-review workflows (Section 4).

2 Background and Related Work 2.1 Peer-Review Tooling Conference management platforms such as HotCRP [16] and EasyChair [1] handle submission routing and reviewer assignment, leaving all content-level integrity and compliance checking to human authors, reviewers, and chairs. AI-based review support has also been piloted directly within a venue’s own review pipeline, as at the AAAI-26 AI Review Pilot, where every AAAI-26 main-track submission received a system-generated review alongside its human reviews [6]. ATIBA’s AI review mode instead runs before submission, on the author’s side, alongside four independently grounded checks – reference integrity, venue compliance, empirical standards, and citation suggestion – that neither conference management platforms nor a reviewer-facing pilot provides.

2.2

Empirical Standards

The ACM SIGSOFT Empirical Standards project [21] defines, per research methodology (controlled experiment, case study, repository mining, and others), the essential, desirable, and extraordinary attributes a rigorous study of that type should report, replacing the piecemeal, methodology-specific guidance that preceded it [13, 23]. Today, checking a manuscript against these standards is a manual read-through.ATIBA’s empirical-standards check is the first to automate this specific checklist while grounding every verdict in a manuscript-verbatim evidence quote.

2.3

Reference and Citation Verification

Verifying that a cited work exists and is not retracted is a bibliographiclookup problem distinct from judging whether it is cited appropriately. Fabricated references are not a hypothetical risk: largescale studies of LLM-generated literature reviews have found that

Karakaya et al.

a substantial share of citations are entirely fabricated [29], and dedicated verification tools have begun to emerge in response [5]. ATIBA’s reference-integrity check delegates existence verification to a dedicated third-party tool, hallucinator-cli [28], invoked as an external subprocess. A similar strategy appears in CheckIfExist [5], which cascades queries across the same bibliographic sources ATIBA uses to verify its own citation suggestions (Section 3.5). ATIBA additionally adds its own LLM-based check for citation context – whether the citation is used in the right context – which existence-only verification cannot answer.

2.4

Citation Suggestion

Context-aware citation recommendation has long aimed to identify suitable references from the textual context surrounding a citation location. Jeong et al. [14] rank candidate papers for a citation placeholder using BERT-based context and citation-graph representations, and Jebari et al. [12] survey this broader line of local citation recommendation. More recently, Seo et al. [26] propose a retrieval-augmented approach matching manuscript sentences against a pre-collected reference repository, showing substantially higher accuracy than direct LLM prompting, which suffers from severe hallucination. ATIBA shares this goal but takes a different integrity-oriented approach: an LLM proposes candidate references, and each is independently checked against Crossref [7], Semantic Scholar [25], and OpenAlex [20] before being shown to the user.

2.5

LLM-Assisted Review and Hallucination Risk

LLMs can produce manuscript feedback that meaningfully overlaps with human reviewer comments [17], but LLM-based scholarly systems remain vulnerable to unsupported or fabricated evidence [10, 15]. WisPaper mitigates this by requiring verbatim source-text evidence for per-criterion judgments [15], while dedicated review-generation systems such as ReviewAgent [8] and ReviewerToo [24] pursue conference-aligned or persona-based feedback without the same evidence requirement. A parallel ecosystem of commercial pre-submission tools has emerged: PaperReview.ai grounds its critique in retrieved arXiv papers [27], PapeReview runs a comparable agentic workflow [3], and Review-it pairs feedback with a journal-recommendation feature [4], suggesting where to submit instead of checking compliance with a venue already chosen (Section 3.2). ATIBA follows the same evidence-grounding principle as WisPaper but enforces it at inference time – any supporting quote not locatable verbatim in its source is discarded together with its verdict – and, unlike the systems above, treats AI review (Section 3.4) as one of five independently grounded checks, not the tool’s sole output.

3

Proposed System: ATIBA

ATIBA combines a Django REST Framework backend with a frontend implemented in Next.js, React, and TypeScript. Checks run asynchronously, and manuscript results and venue/track data are cached for reuse. The service uses GPT-5.42 through Azure OpenAI. Figure 1 summarizes the five pipelines. 2 OpenAI GPT-5.4 model: https://openai.com/index/introducing-gpt-5-4/

ATIBA: Grounded Integrity and Quality Checking for Research Papers

ATIQSER ’26, October 12–16, 2026, Munich, Germany

Figure 1: ATIBA’s five manuscript-analysis workflows: (1) reference-integrity verification, (2) venue/track compliance checking, (3) empirical-standards checklist, (4) AI-assisted review in three modes, and (5) verified citation suggestion.

3.1

Reference-Integrity Check

Motivation. A fabricated or misattributed reference is one of the most concrete, checkable forms of research misconduct, yet verifying dozens of references by hand against bibliographic databases is time-consuming and repetitive. Design. ATIBA extracts the manuscript text and hands the PDF to hallucinator-cli [28], an external tool invoked as a subprocess, which verifies each reference against bibliographic sources and returns one of five per-reference statuses: verified (a matching record was found and the cited authors match), not_found (no matching record was found), author_mismatch (a matching record was found but the cited authors differ), retracted (the matching work has been formally retracted), or skipped (the citation could not be parsed, or the lookup timed out). With URL matching enabled (–url-match), unresolved references carrying a URL are also checked for liveness. Because bibliographic verification establishes only that a reference exists, not that it is used appropriately, ATIBA complements it with a citation-context check. For each numbered reference, the system locates bracketed in-text citation markers, including citations

appearing in groups, and extracts up to three sentences in which the reference is cited. Azure OpenAI then compares these citing contexts with the reference title and assesses whether the cited work plausibly supports, exemplifies, motivates, or is otherwise directly related to the claims being made. It returns a reference-level verdict of relevant, irrelevant, or uncertain, together with a short rationale. The uncertain verdict is used when the citing context or available bibliographic metadata is insufficient for a reliable judgment. As shown in Figure 1 (pipeline 1), this contextual assessment complements the bibliographic verdict in the reference-integrity report: a reference may therefore be verified as a real publication while still being flagged as potentially misapplied in the manuscript. Results are cached and exportable as a PDF report, with colourcoded statuses and per-reference details such as cited-versus-found author names, DOI or arXiv information, retraction notices, the extracted citation contexts, and the rationale for any contextual mismatch. Advantage. Splitting existence-verification (delegated to a dedicated, independently maintained tool) from context-relevance judgment (where an LLM’s ability to read intent is actually useful) means

ATIQSER ’26, October 12–16, 2026, Munich, Germany

ATIBA never asks the LLM to answer a question a database lookup can answer more reliably, and reserves the LLM step for the one judgment a lookup cannot make.

3.2

Venue/Track Compliance Check

Motivation. Every venue states its own submission rules – templates, page limits, anonymization policy, reference-desk-reject conditions – on its own call-for-papers page, in its own words, and authors routinely miss a rule stated on a track-specific page rather than the venue’s general homepage. Design. Given a track’s stored link (or a manually supplied callfor-papers URL), ATIBA scrapes that specific page using BeautifulSoup [22] and runs two separate extraction passes over an LLM: one for peer-review-rubric criteria (novelty, rigor, relevance), one for submission/format criteria observable from the PDF itself (page limits, template, anonymization, reference-integrity policy, openscience expectations). Several major publishers – the ACM Digital Library, Elsevier/ScienceDirect, IEEE Xplore – sit behind bot detection and reject automated requests outright, so ATIBA distinguishes “the page could not be fetched” from “the page loaded but stated no criteria”: in the former case it asks the user to paste the call-forpapers text in directly and runs the same extraction passes over that instead. Each extracted item is required to carry a verbatim quote from the scraped page as evidence and a source URL; items without a locatable quote are not returned. Before the manuscriptlevel pass, every extracted criterion first passes through an evidence gate: the user can edit the extracted requirement’s wording, accept it as-is, or reject a criterion that does not actually apply to their submission (Figure 1, pipeline 2). A third pass then checks each accepted criterion against the manuscript text and page count, returning a per-criterion result – criterion, requirement, observed value, pass/fail, and the originating evidence quote – not a single aggregate pass/fail verdict. Advantage. Deriving criteria from the venue’s own page, not from a fixed internal template, lets the same engine serve venues with very different rules without per-venue engineering, and the evidence-quote requirement ensures that only criteria stated in the source are presented. The evidence gate additionally keeps a human in the loop before any criterion is enforced, so an extraction error can be corrected before it silently fails the manuscript.

3.3

Empirical-Standards Checklist

Motivation. The ACM SIGSOFT Empirical Standards give the community a shared definition of what a rigorous study of a given methodology should report [21], but applying an eighteen-methodology checklist by hand to every submission does not scale. Design. ATIBA ships three checklist catalogues derived from the standards: a pre-submission checklist for authors, and one-phase and two-phase review checklists for reviewers at conferences and journals respectively. Against the selected catalogue, ATIBA first asks an LLM which of its eighteen methodologies (e.g. controlled experiment, case study, repository mining) the manuscript uses, together with a catch-all entry for studies that fit none of them and a one-sentence rationale for each selection. For each selected methodology, the corresponding checklist items are grouped into

Karakaya et al.

essential, desirable, and extraordinary attributes. A second pass then assigns a yes/partial/no status, together with a justification and a short evidence quote. Every evidence quote is checked for verbatim presence in the manuscript; a quote that cannot be located is discarded, while the verdict and justification are kept, and a no verdict with no justification is automatically softened to partial, so it never stands as an unsupported hard fail. The resulting checklist is then presented to the user for review: verdicts, justifications, and evidence quotes can be edited before the checklist is finalized (Figure 1, pipeline 3), so the automatically generated checklist remains a draft for the author or reviewer to confirm, and is never treated as a final judgment. Advantage. The hallucination defence means an ungrounded no – the verdict most likely to be disputed by an author – cannot survive on invented evidence, which matters most exactly where a wrong verdict would be most damaging.

3.4

AI Review

Motivation. Different reviewing moments call for different review shapes: a venue-specific read that adapts to a venue’s own rubric, a fixed, comparably-structured review useful for calibration across submissions, and a page-anchored review useful for revising a specific passage. Design. ATIBA offers three modes from one venue-aware prompting layer, all backed by Azure OpenAI [19] (Figure 1, pipeline 4). The venue-specific mode produces a decision, contribution summary, critical review, prioritized improvement list, and scope analysis, substituting the venue’s own extracted review criteria (Section 3.2) in place of a fixed rubric when they are available. The formal review mode keeps the same structure across venues – the venue and track are named in the prompt, but the sections and the scoring scheme do not change: a summary, strengths, weaknesses, section-level comments, questions for authors, an overall evaluation and a categorical recommendation from strong reject to strong accept, with an explicit instruction not to invent missing content. The annotated review mode processes the manuscript page by page and returns quote-anchored annotations – each tagged strength, weakness, or suggestion, with a severity and a verbatim quote from that page – alongside its own scores (soundness, presentation, contribution, and reviewer confidence) and recommendation, rendered as highlights directly on the PDF. Advantage. Sharing one prompting layer across three modes means a venue’s extracted criteria (Section 3.2) automatically improve the venue-specific review without separate integration work, while the formal and annotated modes give two different, independently useful views onto the same underlying model call.

3.5

Citation Recommendation

Motivation. Authors often know a claim needs a citation but not which one, and manually finding a fitting, real, non-fabricated reference is exactly the kind of task where a plausible-sounding but false suggestion is most damaging – a hallucinated suggested citation is no safer than a hallucinated existing one.

ATIBA: Grounded Integrity and Quality Checking for Research Papers

Design. Given a passage or claim in the manuscript, ATIBA asks an LLM to propose candidate citations. Each one is looked up against three bibliographic sources – CrossRef, Semantic Scholar, and OpenAlex – before it is shown (Figure 1, pipeline 5). Only a candidate that a source affirmatively verifies reaches the user; a candidate no source can confirm – whether because it does not exist, because the sources disagree with the cited year, or because a lookup failed – is discarded; it is never surfaced with a caveat. Advantage. Applying the same principle as the reference-integrity check (Section 3.1) – a citation is believed because a bibliographic record says so, never because the model asserted it – means citation suggestion cannot become a second, unguarded path for the same fabrication problem the rest of ATIBA is built to catch. A proposed citation is held to the same evidentiary bar as one already in the manuscript.

4

Evaluation

Before running the formal survey, we conducted an initial internal deployment by running ATIBA’s reference-integrity check on six manuscripts from the authors’ research group. Out of the 413 references processed, the check surfaced several citations for manual inspection, ultimately confirming seven distinct publication-year mismatches distributed across three of the manuscripts.

4.1

User Study

To complement this initial deployment experience, we conducted a moderated user study to assess the perceived usefulness of the broader system. Participants used the publicly deployed instance of ATIBA to review their manuscripts and subsequently completed a post-task survey. Participants. 13 non-author participants were recruited from the authors’ network via direct invitation: five undergraduate students, five master’s students, two PhD students and one research staff member. Four reported less than one year of research experience, seven reported one to two years, and two reported six to ten years. Self-reported effort gives a baseline for the manual work ATIBA targets. Reviewing a paper took most participants three hours or more (10 of 13, with three to five hours the most common answer), but checking that paper’s references took markedly less: 11 of 13 spent at most two hours on it, 6 of them less than one hour, and only 2 participants reported spending longer on references than on the rest of the review. The reference-integrity check therefore addresses the part of reviewing to which our participants devote the least time – which is also the part where an unverified citation is most likely to survive. 4.1.1 Procedure and Instrument. Each participant used ATIBA on their own manuscript, running the reference-integrity (hallucination) check, the venue/track compliance check, the empiricalstandards checklist, and the citation-suggestion feature, and reading the resulting AI review. Participants’ target venues for the venue/track compliance check included top software-engineering venues such as FSE, ICSE, and ASE. They then completed a survey. The instrument first collected the position, experience and effort items reported above, and then asked the six five-point agree/disagree

ATIQSER ’26, October 12–16, 2026, Munich, Germany

Strongly disagree Disagree Agree Strongly agree

Neutral

% of responses

Q1 Q2

1 2

2 2

% agree

6 3

77

4

69

6

Q3

1

6

6

92

Q4

1

6

6

92

Q5

2

5

85

Q6

1

6 4

25 0 25 n = 13; bar labels are response counts

92

8 50

75

100

Figure 2: Participant agreement with Q1–Q6 (𝑁 = 13), centred on neutral; bar labels show response counts and % agree combines Agree and Strongly agree. items below, quoted verbatim from the instrument and referred to as Q1–Q6 throughout this section and in Figure 2. Q1. The system helped me identify potential quality or integrity issues in the manuscript that I might otherwise have overlooked. Q2. The hallucination check helped me identify and verify potentially incorrect, unverifiable, or unjustified references through clear explanations. Q3. The venue criteria check helped identify whether the manuscript complied with the requirements of the selected venue and track. Q4. The AI-generated review accurately identified relevant strengths and weaknesses of the manuscript. Q5. The empirical checklist helped identify missing or insufficiently reported methodological information in the paper. Q6. I would use this system as part of my future paper-writing or internal peer-review workflow. The instrument closed with an explicit consent question; responses are reported anonymously. 4.1.2 Results. All 13 retained participants consented to the inclusion of their responses. Figure 2 reports agreement with each item. Agreement was high across all six items, with means between 4.0 and 4.5, and no participant selected strongly disagree anywhere in the instrument. The strongest item was adoption intent: 12 of 13 participants (92%) agreed or strongly agreed that they would use ATIBA in their future paper-writing or internal peer-review workflow (Q6, 𝑀 = 4.5), eight of them strongly. The venue-criteria check and AI-generated review each reached the same 92% agreement (Q3 and Q4, both 𝑀 = 4.4), and the empirical-standards checklist followed at 85% (Q5, 𝑀 = 4.2). For Q1, 77% agreed that ATIBA surfaced quality or integrity issues they might otherwise overlook (𝑀 = 4.0), with one disagreement and two neutral responses. The reference-integrity check drew 69% agreement (Q2, 𝑀 = 4.0, 𝑆𝐷 = 1.2), with two disagreements

ATIQSER ’26, October 12–16, 2026, Munich, Germany

and two neutral responses. Thus, all six items received majority agreement, with the greatest response variation on reference integrity.

5

Threats to Validity

Internal validity. The study is a moderated, single-session evaluation and measures participants’ perceived usefulness rather than objective accuracy. Moderator presence and the absence of controlled task comparisons may have influenced their responses. External validity. The evaluation involved 13 non-author participants from the authors’ research network, most with up to two years of research experience. The findings may therefore not generalize to more experienced researchers, reviewers, program chairs, or authors from other disciplines and institutions. Moreover, several ATIBA features rely on a particular LLM deployment, prompts, and configuration. Their behavior may vary across model providers, model versions, and future updates, limiting the generalizability and reproducibility of the observed results. Broader participant samples, multiple manuscript domains and LLM configurations, and controlled task-based evaluations are needed to establish generalizability and accuracy more firmly.

6

Limitations and Future Work

Evaluation coverage. Our study establishes perceived usefulness in a moderated setting. Measuring precision and recall on labeled manuscripts with known reference and compliance defects is the next step toward quantifying check accuracy. A controlled benchmark can vary the number and type of seeded reference defects, separating tool accuracy from the particular manuscript used in the moderated study. Venue-source coverage. Venue compliance is derived from the textual content available on the supplied page. Requirements contained only in linked PDFs, images, or separate documents are outside that input. When a page cannot be fetched, users can paste its text and run the same evidence-grounded pipeline. This fallback is useful for publisher pages that reject automated requests, but it still relies on the supplied text containing the applicable rule. Extending collection across linked materials would broaden coverage. LLM judgment. Grounding makes verdicts auditable by tying them to bibliographic records or source quotations, but context relevance, rubric-based review, and empirical-standard fulfillment still require model judgment. The displayed evidence and editable criteria keep these judgments open to user verification. Reference verification. Bibliographic results depend on database coverage and on hallucinator-cli [28]’s extraction of reference titles; we observed occasional title-extraction errors on IEEEformatted references and database timeouts, both of which can yield a not_found result for a legitimate publication. The interface therefore presents not_found as a prompt for inspection rather than proof of fabrication; future evaluation will quantify these cases and their effect on false-positive rates. Citation suggestion. Verification establishes that a suggested work exists and matches its bibliographic metadata; it does not by itself establish that the work is the best support for a particular

Karakaya et al.

claim. Evaluating claim–citation relevance is therefore an additional direction for the suggestion pipeline. Manuscript confidentiality. LLM-backed checks process manuscript text, or page images for annotated review, through Azure OpenAI, whose policy retains prompts and completions for up to 30 days for abuse monitoring rather than model training [2]. A locally hosted processing option would extend ATIBA to settings with stricter data-residency requirements.

7

Conclusion

Manual manuscript checking remains a repetitive and fragmented part of the publication process, particularly for reference verification, venue-specific requirements, and reporting standards. ATIBA brings these tasks together through five complementary workflows: reference-integrity verification, venue/track compliance checking, an empirical-standards checklist, multi-mode AI review, and verified citation suggestion. Across these workflows, the system is designed to ground model judgments in independently checkable evidence and to keep criteria, evidence, and outputs visible to the user. Our moderated study provides initial evidence that researchers find this approach useful, though broader evaluations with labeled defects and more diverse participants are still needed to establish objective accuracy. ATIBA does not aim to replace authors or reviewers; it aims to make routine integrity and quality checks more systematic, transparent, and easier to verify.

Data Availability ATIBA’s source code, the anonymized user study responses, and a demonstration video are available in our replication package3 .

Acknowledgments We thank Teams 3 and 8 for their contributions to the initial version of this project, developed as part of Bilkent University’s CS319 course in Spring 2026. The authors used OpenAI’s ChatGPT during the preparation of this manuscript to assist with drafting, editing, and improving the clarity and readability of the text. All generated content was carefully reviewed, revised, and validated by the authors. The authors take full responsibility for the accuracy, originality, and integrity of the final manuscript.

3 https://doi.org/10.6084/m9.figshare.33235254.v2

ATIBA: Grounded Integrity and Quality Checking for Research Papers

References [1] [n. d.]. EasyChair Conference System. https://easychair.org. Accessed 2026. [2] 2026. Data, privacy, and security for Azure OpenAI Service. https://learn. microsoft.com/en-us/azure/ai-services/openai/data-privacy. Accessed August 2026. [3] 2026. PapeReview: AI Peer Review & Detection for Research Papers. https: //papereview.com/. Accessed August 2026. [4] 2026. Review-it: AI Peer Review, Pre-Submission Review & Journal Finder. https: //review-it.ai/. Accessed August 2026. [5] Diletta Abbonato. 2026. CheckIfExist: Detecting citation hallucinations in the era of AI-generated content. arXiv preprint arXiv:2602.15871 (2026). [6] Joydeep Biswas, Sheila Schoepp, Gautham Vasan, Anthony Opipari, Arthur Zhang, Zichao Hu, Sebastian Joseph, Matthew Lease, Junyi Jessy Li, Peter Stone, Kiri L. Wagstaff, Matthew E. Taylor, and Odest Chadwicke Jenkins. 2026. AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot. arXiv:2604.13940 [cs.AI] https://arxiv.org/abs/2604.13940 [7] Crossref. 2026. Crossref REST API. https://www.crossref.org/documentation/ retrieve-metadata/rest-api/. Accessed August 2026. [8] Madhav Krishan Garg, Tejash Prasad, Tanmay Singhal, Chhavi Kirtani, Murari Mandal, and Dhruv Kumar. 2025. ReviewEval: An Evaluation Framework for AI-Generated Reviews.. In EMNLP (Findings). 20542–20564. [9] Alexander Goldberg, Ihsan Ullah, Thanh Gia Hieu Khuong, Benedictus Kent Rachmat, Zhen Xu, Isabelle Guyon, and Nihar B. Shah. 2024. Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS’24 Experiment. arXiv:2411.03417 [cs.CL] https://arxiv.org/abs/2411.03417 [10] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Trans. Inf. Syst. 43, 2, Article 42 (Jan. 2025), 55 pages. doi:10.1145/3703155 [11] Rajesh Jayaram, Drew Tyler, David Woodruff, Corinna Cortes, Yossi Matias, Vahab Mirrokni, and Vincent Cohen-Addad. 2026. Towards Automating Scientific Review with Google’s Paper Assistant Tool. arXiv:2606.28277 [cs.LG] https: //arxiv.org/abs/2606.28277 [12] Chaker Jebari, Enrique Herrera-Viedma, and Manuel Jesus Cobo. 2023. Contextaware citation recommendation of scientific papers: comparative study, gaps and trends. Scientometrics 128, 8 (June 2023), 4243–4268. doi:10.1007/s11192-02304773-8 [13] A. Jedlitschka and D. Pfahl. 2005. Reporting guidelines for controlled experiments in software engineering. In 2005 International Symposium on Empirical Software Engineering, 2005. 10 pp.–. doi:10.1109/ISESE.2005.1541818 [14] Chanwoo Jeong, Sion Jang, Eunjeong Park, and Sungchul Choi. 2020. A contextaware citation recommendation model with BERT and graph convolutional networks. Scientometrics 124, 3 (2020), 1907–1922. doi:10.1007/s11192-020-03561y [15] Li Ju, Jun Zhao, Mingxu Chai, Ziyu Shen, Xiangyang Wang, Yage Geng, Chunchun Ma, Hao Peng, Guangbin Li, Tao Li, Chengyong Liao, Fu Wang, Xiaolong Wang, Junshen Chen, Rui Gong, Shijia Liang, Feiyan Li, Ming Zhang, Kexin Tan, Junjie Ye, Zhiheng Xi, Shihan Dou, Tao Gui, Yuankai Ying, Yang Shi, Yue Zhang, and Qi Zhang. 2026. WisPaper: Your AI Scholar Search Engine. arXiv:2512.06879 [cs.IR] https://arxiv.org/abs/2512.06879 [16] Eddie Kohler. [n. d.]. HotCRP Conference Management Software. https://hotcrp. com. Accessed 2026. [17] Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin, Daniel McFarland, and James Zou. 2023. Can Large Language Models Provide Useful Feedback on Research Papers? A Large-Scale Empirical Analysis. arXiv preprint arXiv:2310.01783 (2023). [18] Jialiang Lin, Jiaxin Song, Zhangping Zhou, Yidong Chen, and Xiaodong Shi. 2023. Automated scholarly paper review: Concepts, technologies, and challenges. Inf. Fusion 98, C (Oct. 2023), 18 pages. doi:10.1016/j.inffus.2023.101830 [19] Microsoft. [n. d.]. Azure OpenAI Service Documentation. https://learn.microsoft. com/azure/ai-services/openai/. Accessed 2026 – gpt-5.4 deployment. [20] OpenAlex. 2026. OpenAlex API. https://developers.openalex.org/api-reference/ introduction. Accessed August 2026. [21] Paul Ralph, Nauman bin Ali, Sebastian Baltes, Domenico Bianculli, Jessica Diaz, Yvonne Dittrich, Neil Ernst, Michael Felderer, Robert Feldt, Antonio Filieri, Breno Bernard Nicolau de França, Carlo Alberto Furia, Greg Gay, Nicolas Gold, Daniel Graziotin, Pinjia He, Rashina Hoda, Natalia Juristo, Barbara Kitchenham, Valentina Lenarduzzi, Jorge Martínez, Jorge Melegati, Daniel Mendez, Tim Menzies, Jefferson Molleri, Dietmar Pfahl, Romain Robbes, Daniel Russo, Nyyti Saarimäki, Federica Sarro, Davide Taibi, Janet Siegmund, Diomidis Spinellis, Miroslaw Staron, Klaas Stol, Margaret-Anne Storey, Damian Tamburri, Marco Torchiano, Christoph Treude, Burak Turhan, Xiaofeng Wang, and Sira Vegas. 2020. Empirical Standards for Software Engineering Research. arXiv preprint arXiv:2010.03525 (2020). doi:10.48550/arXiv.2010.03525

ATIQSER ’26, October 12–16, 2026, Munich, Germany

[22] Leonard Richardson. 2026. Beautiful Soup Documentation. Accessed August 2026. https://beautiful-soup-4.readthedocs.io/en/latest/ [23] Per Runeson and Martin Höst. 2009. Guidelines for conducting and reporting case study research in software engineering. Empirical Softw. Engg. 14, 2 (April 2009), 131–164. doi:10.1007/s10664-008-9102-8 [24] Gaurav Sahu, Hugo Larochelle, Laurent Charlin, and Christopher Pal. 2025. Reviewertoo: Should ai join the program committee? a look at the future of peer review. arXiv preprint arXiv:2510.08867 (2025). [25] Semantic Scholar. 2026. Semantic Scholar Academic Graph API. https://www. semanticscholar.org/product/api. Accessed August 2026. [26] Young Hoon Seo, Byung Do Lee, Yujin Oh, Jin Seok Hong, Sung Mo Moon, Ju Young Jang, Min Seong Kim, Ji Sik Kim, and Kee-Sun Sohn. 2026. Contextaware citation suggestion: A retrieval-centric approach for academic writing. Engineering Applications of Artificial Intelligence 173 (2026), 114445. doi:10.1016/ j.engappai.2026.114445 [27] Stanford Machine Learning Group. 2026. PaperReview.ai. https://paperreview.ai/. Accessed August 2026. [28] Gianluca Stringhini. 2026. Hallucinator: Hallucinated Reference Detector. https: //github.com/gianlucasb/hallucinator. AGPL-3.0 licensed; invoked as a CLI subprocess, not vendored. Accessed 11 August 2026. [29] William H. Walters and Esther Isabelle Wilder. 2023. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports 13, 1 (sep 2023), 14045. doi:10.1038/s41598-023-41032-5 [30] Xovee Xu. 2026. Computer Science Conference Statistics. https://csconfstats. xoveexu.com/?conf=ASE. Accessed August 2026.

Received 2026-08-16; accepted 2026-09-01

Related documents

• Research Integrity#673189
• Working with Research Integrity—Guidance for Research Performing Organisations: The Bonn PRINTEGER Statement#778938
• Working with Research Integrity—Guidance for Research Performing Organisations: The Bonn PRINTEGER Statement#734541
• The Predatory Paradox: Ethics, Politics, and Practices in Contemporary Scholarly Publishing#177805
• The Aegis Solis Archive — Master Hash Manifest (v13.0 FINAL)#124961