arXiv CS open access

Can LLMs Test Terminal User Interfaces?

2026 · arxiv_cs

arXiv CS · Papers · 2026

software-architecture software-engineering testing

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Can LLMs Test Terminal User Interfaces?

Ruida Hu* Harbin Institute of Technology Shenzhen, China

Ajitha Rajan University of Edinburgh United Kingdom

chao.peng@acm.org

200111107@stu.hit.edu.cn

arajan@exseed.ed.ac.uk

Tegawendé F. Bissyandé University of Luxembourg Luxembourg

Jacques Klein University of Luxembourg Luxembourg

Cuiyun Gao Harbin Institute of Technology Shenzhen, China

tegawende.bissyande@uni.lu

jacques.klein@uni.lu

gaocuiyun@hit.edu.cn

arXiv:2608.03743v1 [cs.SE] 4 Aug 2026

Chao Peng∗ University of Edinburgh United Kingdom

Abstract Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated testing methodology. We survey 197 real-world TUI applications: only 12% of test code exercises the interface, and 45% of those tests never send input, checking a static frame instead. We turn these applications into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript, packaging each as an instrumented Docker image. We record line and widget coverage where reliable, rendered terminal states, and crashes. Under equal wall-clock budgets, we compare four frontier LLMs with random exploration. No model dominates. Random is a strong time-budgeted baseline, but its crash advantage comes from higher throughput: per interaction, LLM guidance is more efficient and uniquely reaches input-gated faults. Automatically deriving launch inputs yields the largest practical gain, enabling applications that otherwise never start. Line coverage poorly predicts crash discovery, weakening it as a proxy for test effectiveness. Automated TUI testing is feasible but far from solved, and honest baselines matter more than model choice. We release the coverage tool tuicov at https:// github.com/tui-testing/tuicov and the testing framework tuibot at https://github.com/ tui-testing/tuibot.

Testing loop: rendered TUI → tuibot → generate action → exercise TUI ⟲ until the budget is exhausted. Post-run: tuicov measures execution → coverage and crash report. Figure 1: Control flow of the released tuibot and tuicov toolchain.

∗ Chao Peng and Ruida Hu are co-first authors and contributed equally.

Preprint.

Command-Line Interface (CLI)

stdin

Graphical User Interface (GUI)

Terminal User Interface (TUI)

stdout graphics engine

input

output

Raw byte DOM tree

Stateless transducer metaphor.

stream

accessibility tree

Object-Oriented and Event-Driven Metaphor (accessibility tree).

Screen-oriented state management Trapped in a terminal.

Figure 2: A technical comparison of the three interface families. The CLI is a stateless transducer (argv/stdin in, stdout out); the GUI exposes state and events as addressable objects through an accessibility tree; the TUI exposes neither, presenting only a character grid rendered into a pseudoterminal driven by a raw byte stream.

Introduction

Terminal User Interface (TUI) applications combine properties of command-line (CLI) and graphical (GUI) software without belonging to either. Like CLI tools they run inside a terminal emulator and are launched from a shell. Like GUI applications they present a stateful, screen-oriented interface with windows, panes, focus management, modal dialogs, and keyboard- and mouse-driven navigation. This class is large and growing fast. File managers, system monitors, database clients, git frontends, and, most recently, LLM coding agents are routinely shipped as TUIs, built atop frameworks such as ratatui [15], bubbletea [3], textual [21], and ink [5]. Despite their prevalence, TUI applications have no dedicated testing methodology, and crashes in TUI software remain both common and uninvestigated. Existing research targets either CLI programs, treated as pure input/output transducers whose correctness is judged by exit codes and stdout, or GUI programs, driven through accessibility trees and pixel-level widget hierarchies [11, 12, 20]. Neither model transfers to a TUI. CLI techniques discard all stateful, screen-oriented behaviour, which is the very layer where TUI bugs manifest. GUI techniques assume rich accessibility metadata, namely widget identifiers, roles, and bounding boxes, that a character-grid terminal does not expose. There is no DOM, no accessibility tree, and no coordinate system beyond a two-dimensional array of cells rendered into a pseudo-terminal. A TUI’s entire observable state is a character grid, and its behaviour is driven by raw key, mouse, and resize events. A comparison of the three interfaces is shown in Figure 2. No existing fuzzing, exploration, or test-generation technique was designed for that interaction model. The consequence is a practical gap. TUI applications crash, users file bug reports, and no automated technique exists to find, reproduce, or prevent those failures. Two questions motivate this paper. First, how well are TUIs tested today? We surveyed the test suites of 197 real-world TUI applications and found the interface layer doubly under-tested: only about 12% of all test code exercises the terminal/widget layer at all, and of the tests that do, nearly half never send a single input event (Section 2). Second, do the LLM-driven exploration and test-generation techniques now reshaping CLI and GUI testing [6, 8, 17] transfer to TUIs? Our results suggest they do not transfer cleanly. The relationship between code coverage and crash detection in TUI applications diverges from the pattern observed in CLI and GUI domains, where higher coverage tends to predict more faults found [18, 19, 23]. In TUIs, coverage and fault-finding decouple. That finding undermines the standard practice of using line coverage as a proxy for test effectiveness and motivates a TUI-specific coverage metric that aligns with crash discovery. We focus on crash bugs. Silent logic errors and performance regressions matter, but they require ground truth that does not yet exist for TUIs, whereas crashes are unambiguous, automatically detectable, and, as our bug corpus shows, a substantial and underexplored failure mode in practice. 2

We make the following contributions: • An open, multi-language TUI benchmark. 197 real-world applications spanning the four dominant frameworks and languages (ratatui/Rust, bubbletea/Go, textual/Python, ink/TypeScript), each packaged as a per-instance, instrumented Docker image that builds and runs headlessly (Section 3). • A kill-resilient instrumentation toolkit. tuicov2 is a language-agnostic coverage tool that records both native line coverage and an experimental widget-coverage signal from an interactive application that must be terminated rather than allowed to exit (Section 3). • An empirical study and open-source test framework. tuibot3 implements the frontierLLM and exploration strategies evaluated on this benchmark, answering three research questions about model capability, technique comparison, and the coverage-fault relationship (Sections 4-5). Our findings counsel caution about the anticipated benefits of LLM-based TUI testing. No single model achieves consistent superiority across applications. A model-free random baseline remains competitive throughout. The principal determinant of practical effectiveness proves to be the correct derivation of launch inputs, without which input-dependent applications cannot be exercised at all. Most consequentially, conventional line coverage is a poor predictor of crash discovery in TUI software, which reinforces the case for a domain-specific coverage criterion. We release tuicov and tuibot as open-source tools to support TUI testing as an independent research area.

Motivation

Why TUIs need their own testing approach. To understand why existing testing techniques fall short for TUIs, consider a concrete behaviour from a file-manager TUI: pressing j moves the cursor down, and pressing Enter opens the highlighted entry. This single, familiar interaction exposes a fundamental mismatch with both CLI and GUI testing models, as illustrated in Figure 2. In the CLI model, the interaction cannot be expressed at all. CLI testing treats a program as a pure input/output transducer: a test fixes argv and stdin, captures stdout/stderr and an exit code, and compares the result against an expected value. A sequence such as “move cursor down, then open entry” has no representation in this model. It unfolds across time against a persistent, on-screen state that a single-invocation transducer cannot capture. In the GUI model, the same behaviour is straightforwardly testable, because a graphical application exposes an accessibility tree of named, queryable objects. A test driver locates the file list by its accessibility role, reads its selectedRow property, synthesises a keypress or mouse click to advance the selection, and then queries row.isSelected or awaits an activated event. State and events are first-class, named entities that the harness can inspect and subscribe to directly. In the TUI model, neither of these handles exists. The entire observable state of a TUI is a twodimensional grid of characters rendered into a pseudo-terminal; input is the raw byte stream "j\r". There is no queryable property for “which row is highlighted”, that information is encoded implicitly as the single line drawn with an inverted SGR (Select Graphic Rendition) attribute somewhere within an H × W grid of cells. To verify that the cursor moved, a test must write the bytes into the pseudo-terminal (PTY), wait for the screen to repaint, scan the full cell grid for the styled run, and decode the file path from its glyphs. To verify that an entry was opened, it must diff the entire frame against a prior state. There is no widget to query and no event to await, with only bytes in and a repainted screen out. This is precisely the layer that existing CLI- and GUI-oriented techniques bypass, and the layer that any automated TUI tester must be capable of driving directly. Figure 3 reinforces this point with a richer example: issuing an authenticated POST of a JSON user record, realised across all three interface families. TUIs are poorly tested in practice. TUI application frameworks provide testing harness and developers can write TUI-level tests with it. Figure 4 shows a representative example from twig, a JSON2 https://github.com/tui-testing/tuicov 3 https://github.com/tui-testing/tuibot

(a) CLI (curl).

(b) GUI (Postman).

Figure 3: The same task (issuing an authenticated POST to /users with a JSON body) across the three interface families. The CLI (a) encodes the whole request as one argv/stdin invocation; the GUI (b) exposes named fields and buttons as addressable objects; the TUI (c) renders the same fields as a character grid in a terminal, driven entirely by keystrokes.

explorer TUI in our benchmark, written against textual’s Pilot driver. The harness boots the app inside a headless pseudo-terminal (run_test), synthesises keystrokes (pilot.press), and, crucially, lets the test query live widget objects (query_one, screen.focused, OptionList.highlighted) instead of decoding glyphs from the screen buffer. This is exactly the “move cursor, then assert which row is highlighted” interaction from our running example, made expressible only because the framework re-exposes the named state and events that the terminal itself discards. Every TUI ecosystem ships some version of this affordance (ratatui’s TestBackend, bubbletea’s teatest, ink’s ink-testing-library); the question is how much developers actually use it. async with TwigApp(SAMPLE).run_test() as pilot: nav = app.query_one("ColumnNavigator") # type a path into the search bar and jump to it await pilot.press("/") await pilot.press(*’.regions["us-east-1"].vpcs[0]’) await pilot.press("enter") # assert on widget state, not on screen glyphs assert "col-" in str(app.screen.focused.id) # search "available", then step matches with n / N await pilot.press("/"); await pilot.press(*"available") await pilot.press("enter") first = nav_selected(nav) await pilot.press("n") # next match assert nav_selected(nav) != first await pilot.press("N") # previous match assert nav_selected(nav) == first

Figure 4: A TUI-level test from twig using textual’s Pilot harness (condensed from tests/test_integration.py). The harness drives the app through keystrokes and asserts on queryable widget state, which is the affordance that makes “press a key, check which row is highlighted" testable. To establish that this gap has real-world consequences, we surveyed the test suites of all 197 benchmark applications, classifying every test file (excluding vendored and dependency code) as either TUI-level (exercising the terminal or widget layer via the framework’s TUI test harness) or non-TUI, in the spirit of prior test-suite effectiveness studies [7]. Classification was LLM-assisted and verified by manual inspection; all counts are computed deterministically. Most applications barely test their interface. Of the 197 applications surveyed, only 76 (38.6%) include any TUI-level tests; 35.0% have tests that never touch the UI layer at all; and 26.4% ship no tests whatsoever. As shown in Table 1, only approximately 12% of all test code exercises the TUI itself. The remainder targets parsers, configuration, data layers, and business logic. Even textual, the ecosystem with the most mature TUI testing infrastructure, allocates only a quarter of its test files to the UI. The Median app share column computes each application’s own TUI-test fraction and takes the median across applications, describing the typical app rather than the aggregate. The three can diverge sharply, which is precisely why we report all of them: ink, for instance, shows 18.3% of files but a median of 0%, because its pooled figure is inflated by a single exhaustively tested outlier (gemini-cli) while the median ink application has no TUI test at all. Table 1: Fraction of test effort targeting the TUI layer, by framework. Framework Apps w/ TUI test TUI files / all Median app share ratatui 32% 8.9% 7.5% bubbletea 43% 10.0% 6.2% textual 41% 25.4% 15.6% ink 43% 18.3% 0.0% All 38.6% 12.3% — The same gap at the test-case level. Table 2 re-counts at the granularity of individual test cases (test functions). The 785 TUI-level test files resolve to 8,353 distinct TUI-level test cases (10.6 test cases per file in average) , of which only 5,213 (62%) are interactive. The TUI tests that do exist are shallow. Among the 785 TUI-level test files found across the 76 applications that include them, only 55% (430 test files) are genuinely interactive, that is, they feed key, mouse, or resize events and assert on the resulting state. A further 24% are render-only tests that draw a single static frame and diff the buffer, and the remaining 21% are snapshot-only tests 5

Table 2: TUI-level testing at the granularity of individual test cases (test functions), by framework. Interactive cases belong to a test that sends at least one key, mouse, or resize event. Framework TUI test cases Interactive Interactive % ratatui 1,376 511 37% bubbletea 3,820 2,841 74% textual 1,583 1,357 86% ink 1,574 504 32% All 8,353 5,213 62% Study design and measurement scope Corpus mining

Build filter

Experiment matrix

Completed run

649 awesome-tuis 601 GitHub repos 4 target frameworks

257 apps attempted 197 runnable images 23% attrition reported

4 settings 4 LLMs + random 2,561 scheduled jobs

2,290 executed 600 s per run same PTY harness

Coverage scope

Crash oracle

Reported faults

Line: Rust/Python usable TypeScript sparse; Go unavailable Widget: experimental pilot

Rendered-screen classifier filters usage exits, SIGINT, harness noise

1,033 raw crash events 179 valid TUI faults 45 affected apps

Figure 5: Overview of the completed study design and the measurement scope used in the paper. The benchmark starts from the mined TUI corpus, filters to 197 headlessly runnable applications, runs the four exploration settings under the same PTY harness, and reports only measurements that are valid in the completed run. that compare against a golden frame without sending any input. In total, 45% of TUI test files never send a single input event. Sixteen of the 76 applications (21%), including widely used tools such as bottom, bandwhich, kmon, and television, have TUI test suites that are composed entirely of static renders or snapshots. The interactive tests are also thin. The median interactive file uses only nine input actions, and 11% use just one or two. A file with one or two input actions typically presses a single key and checks a single outcome, exercising none of the multi-step navigation, focus transitions, modal dialogs, or edge-case key sequences that characterize real use. The TUI interface layer is therefore doubly under-tested: it receives a small fraction of overall test effort, and the majority of that effort is static. This is the gap that an automated TUI exploration approach is positioned to fill.

Benchmark and Infrastructure

Figure 5 gives a roadmap of the benchmark and the measurement scope used throughout the paper. The top row traces the construction and execution pipeline: we mine a corpus of real TUIs and resolve their frameworks, filter to the applications that build and run headless as per-instance instrumented images, schedule the four-setting experiment matrix across the model panel, and execute it. The bottom row records what the completed run can measure: the cross-language coverage scope, the rendered-screen crash oracle, and the resulting fault counts. 3.1

Frameworks and Applications

To ground our choice of frameworks in evidence rather than intuition, we first mined every TUI listed in awesome-tuis4 , an actively maintained collection of TUI applications with approximately 20,000 GitHub stars. In this collection, there are 649 projects across 13 categories. We fetched each repository’s evidence files (README, pyproject.toml, Cargo.toml, package.json, go.mod, 4 https://github.com/rothgar/awesome-tuis

. . . ), and asked three LLMs (Claude-Sonnet-4.5, Claude-Opus-4.7, GPT-5.4) to identify each project’s underlying TUI library from that evidence. The three models agreed on the same library for 76% of resolvable projects and never once named three different libraries, giving us confidence in the resulting tally. The result is a near-monolithic leader within each language: of projects whose library could be identified, Rust overwhelmingly uses ratatui (78%), Go uses bubbletea (the clear plurality), Python uses textual, and TypeScript uses ink. We therefore target exactly these four frameworks, one per mainstream language, as the dominant choice each ecosystem has converged on. From these ecosystems we assemble a suite of real-world applications spanning a wide popularity range, from 100k-star coding agents (codex, gemini-cli) down to niche hobby tools. 3.2

Per-Instance Instrumented Images

As there is no existing unified tool to measure how thoroughly a TUI’s interface is exercised across all four target languages and frameworks, we implement tuicov5 , an open-source, language-agnostic coverage toolkit that underpins the benchmark and the measurements throughout this paper. Given an application’s source, tuicov performs a four-stage instrument → build → run → report pipeline. It first parses the source and statically detects widget sites, the source locations where the framework’s UI components (e.g., ratatui, bubbletea, textual, and ink widgets) are constructed or rendered, and injects a lightweight runtime probe at each one. The instrumented app is then built under each language’s native line-coverage facility (cargo llvm-cov for Rust, go tool covdata for Go, coverage.py for Python, and c8 for TypeScript). At runtime the probe appends one record per widget hit to a log, while the native facility records executed lines. After a driven session, a languageagnostic reporter merges the two raw sources into a single unified report that pairs native line coverage with TUI-specific widget coverage per file. This dual signal (ordinary code coverage alongside an interface-aware one) is what lets us reason about interface exploration rather than mere code execution. The benchmark is realized as per-instance, tuicov-instrumented Docker images that run in a plain headless Linux environment. A single Dockerfile family (one per language) clones the app at the target commit, applies instrumentation in place, builds under native coverage when the runtime supports it, and ships a runnable image. Of 257 apps attempted at the most-recent commit, 197 produced a runnable image. A subset of arbitrary apps do not build or run headless, whether from missing system dependencies, display/daemon requirements, or compile failures. 3.3

Execution Harness and Coverage Instrumentation

Each run is driven by tuibot6 , our open-source automated TUI testing framework, through a pseudoterminal at a fixed terminal size. The harness waits for the rendered grid to stabilize after each action, records the character-grid state, and sends key, mouse, and resize events as raw terminal input. The same harness backs all four exploration settings we study: (1) random (model-free input as a baseline), (2) llm-guided (an LLM picks each action at runtime), (3) llm-guided-derived (llm-guided seeded with automatically derived launch inputs), and (4) llm-generated (an LLM reads the source and emits test scenarios that steer the explorer), defined in detail in Section 4. A key subtlety distinguishes TUI testing from ordinary unit testing. The exploration harness terminates each session when the time budget is used up, but most native coverage tools only flush on a clean exit. This makes line coverage unavailable for Go as it does not support extracting line coverage when the app is terminated by external signals. 3.4

A Content-Aware Crash Oracle

A consequential lesson from driving 197 apps is that “process exited non-zero” is not a usable crash oracle for TUIs. Across the capability run, 1,033 crash events were recorded, but only 179 (17.3%) are valid TUI-level faults after our examination. The remaining 82% are noise of a few distinct kinds (Section 5). These include an app printing a usage message and exiting because it needed required arguments, a graceful response to Ctrl+C, a docker run name clash, a PTY ioctl 5 https://github.com/tui-testing/tuicov 6 https://github.com/tui-testing/tuibot

failure, and, as the single largest category, a coverage-runtime emit failure that turns a clean exit into a non-zero one. An exit-code oracle counts all of these as crashes. We therefore classify each crash event on its rendered terminal screen rather than its exit code. A content-aware classifier scans the post-termination grid for a rendered traceback, an on-screen runtime exception, a Rust/Go panic, or a fatal signal (SIGSEGV/SIGABRT), and hard-excludes harness artifacts (coverage-emit failures, docker races, PTY errors, SIGKILL-at-budget, usage/–help exits). Events with no decisive on-screen signal are adjudicated individually: two of the authors independently labeled all 233 ambiguous crash events as valid TUI-level faults, harness or environment noise, or undecidable. The annotators reached 93% raw agreement, and the remaining disagreements were resolved through discussion until consensus. Per-app crash saturation, the fraction of an app’s runs that crash, together with the median number of steps to the crash, further separates input-invariant startup and environment failures (high saturation, crash at step 1) from interaction-reached faults (mid saturation, many steps). A direct implication for future TUI fault studies is that we adopt a rendered-screen oracle rather than counting raw exits.

Research Questions and Study Design

We investigate three research questions. Across all RQs we evaluate on TUI applications spanning multiple languages and frameworks (Rust/ratatui, Go/bubbletea, Python/textual, TypeScript/ink) to assess generalization, and all techniques drive each app through a pseudo-terminal, in the manner of end-to-end terminal testing tools [13], running through the same instrumented-container harness so their coverage and crash measurements are directly comparable. RQ1: How do frontier LLMs perform on testing TUI applications? We compare four frontier LLMs (Claude-Opus-4.8, GPT-5.5, Gemini-3.5-Flash, and DeepSeek-V4-Pro) using a fixed LLM-guided exploration strategy. We measure code coverage achieved, the crashes uncovered, and the token cost each model incurs, with budgets equalized in wall-clock time per session so every model drives each subject app for the same exploration window regardless of its latency or pricing. Because the budget is fixed in time rather than in steps, a model’s per-step throughput, namely how many actions it can issue before the clock runs out, is itself part of what RQ1 measures. RQ2: How do random exploration, LLM-guided exploration, and LLM-based test generation compare? We compare four settings: (1) random exploration as a model-free baseline, (2) LLMguided runtime exploration, (3) LLM-guided exploration augmented with automatically derived launch inputs, and (4) LLM-based test generation from source code. We measure coverage and crashes, again with budgets equalized in wall-clock time per session, and, for the crash comparison, report effectiveness normalized by interaction (valid crashes per unit of input) rather than by raw count, because a time-equalized budget gives each setting a very different number of steps. RQ3: Does code coverage correlate with fault-finding in TUI applications? We analyze the relationship between code coverage and the number of valid faults discovered across sessions and applications, using the adjudicated crash set (Section 5) as observed fault evidence and accounting for the session-truncation confound that a crash introduces by ending an episode. 4.1

Techniques (Settings)

The comparison axis comprises four settings. (1) random: Model-free random input, which serves as the baseline and is analogous to random/monkey testing in the GUI domain [4, 11, 20]. (2) LLM-guided: An LLM selects each action at runtime taking the current terminal screen (characters shown in terminal) as input. (3) LLM-guided-derived: The LLM-guided setting augmented with an LLM that derives CLI arguments and input fixtures so that input-dependent applications launch. (4) LLM-generated: An LLM reads source code and emits natural-language test scenarios that subsequently steer the LLM-guided explorer. 8

Each LLM setting runs across the model panel; random is model-independent. To account for the stochasticity of both random and LLM-guided exploration, every configuration is repeated three times per app, and all reported metrics are averaged over the three runs. Each run receives a 600 s wall-clock budget, equalized across settings. Exploration relaunches the app on exit/crash until the budget is spent, so coverage and crashes accumulate across episodes. To isolate each arm, the LLM settings employ no random fallback. A failed LLM call is retried and then fails the run rather than degrading silently to random, so that every step attributed to an LLM arm constitutes a genuine model decision. 4.2

Metrics

We record three signals per run. Line coverage and widget coverage are harvested from the unified report after each session (Section 3), but the results scope each signal to the runtimes where the completed run produced valid data. For crashes we do not report raw exit-non-zero counts, because, as Section 5 shows, those are dominated by environment and harness noise. We instead classify every crash event on its post-termination terminal screen into valid TUI-level faults and noise, and we report valid crashes both per run and per unit of interaction (per 1,000 steps), together with the count of unique (app, fault) signatures. For every LLM run we additionally record token usage (input tokens, output tokens, and number of model calls), so that effectiveness can be weighed against cost.

Results

Our study comprises a unified run of 197 applications under four settings and four models, with every configuration repeated three times per app and all reported numbers averaged over the three runs. Before answering the RQs, we scope the measurements. Crash results use the adjudicated valid-fault set rather than raw exit codes. Line coverage is reported for Rust, Python, and TypeScript, not for Go due to framework limitations as discussed in Section 3. 5.1

RQ1: No model dominates, and capability is decoupled from cost

Across the model panel, no single LLM leads on every metric. Within each setting×language cell, the best model on line coverage is rarely the best on crashes, and the four models are close on unique-fault discovery (Gemini 27, Claude 24, GPT-5.5 18, and DeepSeek 16 unique valid faults). The same non-dominance holds on coverage: broken down by model (Table 5), Gemini-3.5-Flash and GPT-5.5 lead line coverage in every setting while the costliest model, Claude-Opus-4.8, tops none. What separates the models sharply is token cost, not capability (Table 3). Under the same 600 s budget, the models differ by an order of magnitude in tokens consumed and in how many steps they manage to issue. Claude-Opus-4.8 spends ∼150k tokens/run over ∼53 model calls, while GPT-5.5 reaches a comparable outcome with ∼40k tokens, and DeepSeek-V4-Pro with only ∼24k. Because the budget is fixed in time, slow or verbose models take fewer steps before the clock expires, which, as RQ3 shows, is the dominant driver of measured effectiveness. Table 3: Per-model cost under the equalized 600 s budget (LLM settings, weighted by runs) Model Tokens/run Calls/run Tok/call Claude-Opus-4.8 150,289 52.7 2,853 Gemini-3.5-Flash 98,146 51.2 1,916 GPT-5.5 39,777 17.5 2,274 DeepSeek-V4-Pro 23,505 13.0 1,813 5.2

RQ2: Coverage and crash-finding by setting

Coverage. Table 4 summarizes per-app union line coverage (the union of lines covered across an app’s runs) on the usable line-coverage subset, together with per-run token cost and steps. Within that scope, LLM-guided-derived leads on line coverage (30.4% vs. 26–28% for the other arms). The gain is concentrated in getting input-hungry apps to start at all. Plain arms reach 0% on these apps that exit immediately without arguments or fixtures, and input derivation rescues them. Smarter per-step 9

reasoning alone (LLM-guided vs. random) yields little additional coverage. Random’s 28.4% is competitive with both non-derived LLM arms despite issuing only model-free input. The determining factor is input derivation, not action selection. Table 4: Coverage and cost by setting. Line % is per-app union coverage on the usable line-coverage subset; tokens are per LLM run. Setting Line % Tokens/run Steps random 28.4 — 784 LLM-guided 26.4 73,005 35 LLM-guided-derived 30.4 75,988 36 LLM-generated 26.6 94,371 30 Coverage by model. Table 5 breaks the same per-app union coverage down by model within each LLM setting, alongside the per-run token cost. Two patterns hold across all three settings. First, capability does not track cost: Gemini-3.5-Flash and GPT-5.5 reach the highest line coverage, yet GPT-5.5 does so at roughly a quarter of Claude-Opus-4.8’s token spend and Claude never leads a single cell despite being the most expensive. Second, input derivation helps every model: each of the four gains 3–8 points moving from LLM-guided to LLM-guided-derived, confirming that the derived-input effect is a property of the setting rather than of one strong model. Widget coverage. Alongside line coverage we collected the interface-level widget-coverage. There, widget coverage is flat across settings: 33.3% (random), 34.2% (LLM-guided), 37.1% (LLM-guided-derived), and 35.1% (LLM-generated). The ordering echoes line coverage, with input derivation improving coverage up a few points and action-selection strategy otherwise making little difference. Table 5: Line coverage by model within each LLM setting. Line % is per-app union coverage on the usable line-coverage subset; tokens are per run. The best model in each setting is bold. Setting Model Line % Tokens/run LLM-guided Claude-Opus-4.8 25.2 130,162 Gemini-3.5-Flash 28.2 91,149 GPT-5.5 28.5 32,967 DeepSeek-V4-Pro 23.2 26,521 LLM-guided-derived Claude-Opus-4.8 32.1 140,445 Gemini-3.5-Flash 33.7 91,376 GPT-5.5 30.7 35,323 DeepSeek-V4-Pro 26.1 25,989 LLM-generated Claude-Opus-4.8 25.8 169,283 Gemini-3.5-Flash 29.6 102,757 GPT-5.5 27.6 52,045 DeepSeek-V4-Pro 22.8 13,149 Raw crash counts are 82% noise. Driving 197 apps produced 1,033 crash events, but a contentaware classifier over the post-termination terminal screen, followed by per-case adjudication of every ambiguous exit (Section 3), reduces these to 179 valid TUI-level faults (17.3%) over 45 apps (Table 6).7 Noticeable crash sources include apps that printed a usage message and never entered their TUI (156) and graceful Ctrl+C exits (105). The ambiguous middle contained no reservoir of real bugs. Of 233 undecidable exits, adjudication identified only 4 genuine faults and 218 startup/environment failures. Exit-non-zero is not a crash oracle for this domain. Random wins per run, guidance wins per keystroke. On valid faults, random has the highest per-run yield (15.2%),8 ahead of LLM-guided-derived (9.0%), LLM-guided (8.1%), and 7 Five events remain undecidable as they exited without any error message or traceback. 8 Per-run yield is the fraction of runs (one run = one (app, setting, model, seed) session under the fixed 600 s budget) that surface at least one valid fault; a run that crashes several times still counts once. Random’s 15.2% is 30 of its 197 runs.

Table 6: Crash-event triage (1,033 events) Bucket Events Valid TUI-level fault 179 Noise (Go cov-emit, usage exit, SIGINT, . . . ) 849 Undecidable (Non-crash fault) 5

Share 17.3% 82.2% 0.5%

Raw exits are mostly noise

1,033 raw crash events

Valid TUI faults: 179 Noise: 849 Undecidable: 5 Figure 6: Most raw non-zero exits are not TUI faults. The rendered-screen oracle reduces 1,033 raw crash events to 179 valid TUI-level faults.

LLM-generated (4.1%). This lead is, however, a pure throughput artifact. Normalizing by interaction reverses the ranking completely (Table 7). Per 1,000 steps, the guided LLMs are ∼13× more crash-efficient than random. Random attains the higher per-run yield only because it issues ∼24× more inputs per run. It spends the entire 600 s emitting keystrokes (median 253 steps), whereas the LLM arms spend most of their wall-clock time awaiting API round-trips and take only a dozen steps. Random is not a better explorer. It is a faster one.

Table 7: Valid crashes per run vs. per unit of interaction Setting Med. steps Yield/run Valid / 1k steps random 253 15.2% 0.19 LLM-guided 12 8.1% 2.41 LLM-guided-derived 13 9.0% 2.57 LLM-generated 6 4.1% 1.42 11

In addition, truncating every arm to a matched step budget, at k ≤ 12 steps (the LLM arms’ median) LLM-guided (5.3%) and LLM-guided-derived (6.4%) both exceed random (3.0%), which overtakes only once granted its full-throughput keystroke budget. This confirms that exploration assisted by the LLM wins per keystroke.

Throughput vs. crash efficiency 3

Circle area encodes per-run yield

Valid faults per 1,000 steps

llm-guided-derived llm-guided

llm-generated 1 random 0 5

100

250

Median steps per run (log scale) Figure 7: The crash result is a throughput–efficiency trade-off. Random issues far more inputs within the fixed time budget, while LLM-guided settings find valid faults much more efficiently per interaction. Three factors explain the picture: (i) Crashes are shallow. The median valid crash is reached in just 5 steps (115 of 179 within 15 steps), so a few hundred blind keystrokes stumble onto shallow bugs without guidance, and the guidance premium would only show on deep, state-gated bugs, of which this corpus has few. (ii) The strategies are complementary, not ranked. Some crashing apps are reached only by an LLM arm (15 apps; e.g. a Rust panic in twitch-tui that fires only after completing an interactive OAuth wizard, and faults in posting, frogmouth, and Bagels that need a filled form or a valid query), while 10 are random-only (e.g. a kmon segfault reached by brute volume). (iii) LLM attrition shrinks the denominator, not the hit-rate. 71–83 runs per LLM setting never executed (API failures, timeouts), and LLM-generated quits within 5 steps in half of its non-crash runs (premature “task done”), so its 4.1% understates the explorer. Under a fixed time budget with shallow targets, random finds the most crashes, but per unit of interaction guidance is an order of magnitude more efficient and uniquely reaches input-gated bugs. The right design is therefore a hybrid that spends LLM calls to unlock state and random throughput to storm it. 5.3

Fault characteristics

The 179 valid faults collapse to 47 unique fault signatures over 45 affected applications. Python has the highest fault density (20.8 per 100 runs, vs. Rust 6.5, TS 8.6, Go 2.2), largely because Textual renders clean, attributable tracebacks (ScreenStackError, MarkupError, AttributeError). Its failures are legible, not necessarily more frequent. Rust contributes the most distinct panics and the only fatal signals (two SIGSEGVs), the highest-severity crashes in the set. Faults cluster on a 12

few primitive triggers, namely Enter (37), ArrowDown (14), ? (14), Tab (11), and Escape (11), with the help overlay (?) and field submission (Enter) disproportionately fatal. Opening a help pane or committing a field is where unhandled state lives. 5.4

RQ3: Coverage decouples from fault-finding

The setting LLM-guided found many crashes yet does not lead on line coverage, and random leads on neither coverage nor per-step crashes yet attains the highest per-run crash yield. Within the usable line-coverage subset, the coverage–fault relationship is therefore essentially flat, contrary to the conventional expectation that more line coverage should imply more faults found. This pattern is consistent with a session-truncation confound, in which a crash terminates the episode and caps the coverage that can accrue afterward, so that the runs which expose faults are penalized on coverage. Together with the observation that valid TUI crashes are shallow (median depth 5 steps) and broadly reachable, this implies that line coverage is a poor proxy for crash-finding in this TUI setting. The result echoes prior cautions that coverage and test effectiveness need not align [7, 14] and motivates a TUI-specific criterion keyed on reachable interactive states rather than executed lines.

Discussion

Terminal user interfaces sit at the heart of widely used software, from coding assistants and systemmanagement tools to database and network clients, yet to the best of our knowledge their automated testing has received almost no systematic study. We therefore take the first step toward closing that gap, and our results point to several directions for TUI testing research. The central lesson is that the testing problem is not only an action-selection problem. It is also a benchmark-design, measurement, launch-configuration, and oracle-design problem. Metric design is an important research topic. The completed run shows that raw exit counts and raw coverage numbers are easy to misread. Most non-zero exits are not valid TUI faults, and coverage availability differs sharply across runtime instrumentation paths, and even obtaining reliable line coverage for Go took substantial engineering effort. This difficulty is structural: whereas mobile GUI testing targets just two platforms (Android and iOS) with a small set of languages, TUI development is fragmented across many frameworks written in different languages, so a cross-framework study must solve instrumentation and measurement anew for each language rather than once. Future TUI benchmarks should therefore treat measurement validity as a first-class design goal: report valid faults rather than raw exits, and normalize crash-finding by interaction as well as by run. Crash-finding is a tractable first target. This paper focuses on failures that are visible from the terminal session: tracebacks, panics, fatal signals, and abnormal terminations with rendered evidence. That scope gives the field a concrete starting point. It makes automated triage possible and exposes real faults in widely used applications. The next step is to expand the oracle beyond crashes, toward visual misrenderings, incorrect state transitions, performance stalls, and usability defects. Doing so will require richer ground truth than exit status or stack traces, likely combining replayable interaction logs, screen-state differencing, and application-specific assertions. Hybrid exploration can be a practical path. Under a fixed wall-clock budget, random exploration remains strong because it emits many more inputs than an LLM-guided policy. At the same time, LLM guidance is much more efficient per interaction and reaches input-gated faults that random input misses. This argues against treating random and LLM-guided testing as mutually exclusive baselines. A practical TUI tester should use LLMs where semantic reasoning matters: deriving launch arguments, creating fixtures, filling forms, and unlocking states. Once the application is in a meaningful state, high-throughput exploration can still be valuable for stressing shallow or brittle transitions. Launch derivation deserves more attention. A recurring source of low coverage and false crashes is not poor navigation but failure to start the TUI in a meaningful mode. Many applications require files, command-line arguments, credentials, databases, or setup state before their interface appears. The llm-guided-derived setting shows that deriving these launch inputs can matter more than choosing the next keypress. Future work should study launch configuration as its own subproblem, with benchmarks that measure whether a testing system can infer required fixtures and enter the interactive UI before exploration begins. 13

Multimodal models are a natural next step. Our LLM-guided arms reason from textual representations of the terminal state. Modern multimodal models could instead consume rendered screenshots directly, potentially using layout, styling, alignment, and visual salience that are lost or flattened in a text-grid representation. Screenshot-based policies may be especially useful for TUIs whose state is encoded through color, highlighting, tables, charts, or spatial grouping. They should, however, be evaluated against the same baselines used here: a fixed wall-clock budget, per-interaction normalization, random throughput, and a content-aware crash oracle. The benchmark is a starting point. The 197 applications in this study form a practical, runnable, cross-framework slice of modern TUIs. They are not a complete census of terminal software. Future benchmark extensions should add other ecosystems, include applications that need controlled external services or accounts, and revive the historical-crash benchmark once buggy versions can be rebuilt and replayed systematically. The current benchmark establishes the measurement foundation on which those broader fault and correctness studies can build.

Threats to Validity

Construct validity. Our main construct validity includes coverage, interaction efficiency, and valid TUI-level crashes. Line coverage is an imperfect proxy for TUI behaviour because many terminalstate changes occur through framework rendering code or data-dependent UI state that may not map cleanly to newly executed application lines. Widget coverage is closer to the interface layer but the order of widgets being executed may matter more. Crash validity is also non-trivial: raw non-zero exits include usage errors, graceful interrupts, harness failures, and coverage artifacts. We mitigate this by classifying crashes from the rendered terminal screen and manually adjudicating ambiguous cases, with two of the authors independently labeling each ambiguous event and resolving disagreements by discussion until they reach agreement, but some valid faults may still be missed if they leave no recognizable on-screen evidence. Internal validity. The fixed wall-clock budget introduces a throughput confound. Random exploration issues hundreds of inputs per run, while LLM-guided exploration often issues only a dozen because of LLM reasoning time. We address this by reporting both per-run yield and valid faults per 1,000 steps. External validity. The benchmark targets four dominant frameworks: ratatui, bubbletea, textual, and ink. Results may not transfer to TUIs built with other frameworks and programming languages, and custom terminal engines. The corpus is also limited to apps that build and run headlessly in Docker. Applications that require live network services, user credentials, hardware, desktop integration, or unusual terminal features are underrepresented and hard to be included.

Related Work

Testing terminal user interfaces. Despite the prevalence of TUI applications, automated testing of the terminal interface itself has received almost no systematic study. The testing support that exists is framework-native and practitioner-built rather than the subject of published research. Each major framework ships a unit-level harness that renders the interface into an in-memory buffer and compares it against a stored frame, including the ratatui TestBackend [15], the bubbletea teatest model harness [3], the textual run_test Pilot API [21], and the ink-testing-library lastFrame probe [5]. End-to-end drivers that operate a real pseudo-terminal exist at the tooling level [13]. These harnesses share two limitations that our study documents quantitatively. They are exercised by hand-written tests whose coverage of the interface is shallow (Section 2), and they assume a clean process exit, which an interactive application under exploration does not provide (Section 3). No prior work establishes a cross-framework benchmark, an instrumentation method that recovers coverage from a terminated session, or an empirical account of how automated techniques perform on TUIs. This paper supplies all three, and in doing so treats TUI testing as a research problem in its own right rather than a downstream application of GUI or CLI methods. Automated GUI exploration. The body of work nearest to our exploration component is automated testing of graphical applications, where a driver issues input events and observes the resulting interface state. Random and model-free strategies set the baseline. Dynodroid generates input events through a guided observe-select-execute loop [11], and the random “monkey” tester remains the 14

reference point that more elaborate techniques are measured against. Choudhary et al. ran exactly that comparison and found that none of the surveyed academic tools reliably beat random exploration on coverage [4], a result that anticipates our own. Later work moved to model-based and search-based drivers, with Stoat building a stochastic model of the interface [20], Sapienz casting test generation as multi-objective search [12], and reinforcement learning treating exploration as a sequential decision problem [16]. Every one of these techniques relies on the addressability a GUI provides, namely a widget tree with stable identifiers, roles, and bounding boxes against which an action can be aimed and a result queried. A TUI exposes none of this. Its observable state is an H × W grid of styled character cells, and its input is a raw byte stream, so the perception and action interfaces these methods assume do not exist. We retain the random baseline these papers established and ask, for the first time, how it and its learned competitors behave when the only observation is a rendered screen. LLM-driven testing. A second line of work replaces hand-built exploration policies with large language models. For mobile GUIs, Liu et al. prompt an LLM to choose functionality-aware actions during testing [8] and to synthesise unusual text inputs that trigger crashes [9]. For library and unit testing, LLMs have been used as zero-shot fuzzers of deep-learning frameworks [6] and as generators of unit test suites [17]. These studies report that LLM guidance improves over weaker baselines on their respective targets, but each operates where the model can read structured artefacts such as an accessibility tree, a method signature, or library documentation. None addresses an application whose entire interface is a character grid driven by raw key events, and none reports the throughput cost that a per-step LLM call imposes under a fixed wall-clock budget. We evaluate LLM-guided exploration, LLM-derived launch inputs, and LLM-generated test scenarios on TUIs under a time-equalised budget, and we find that the per-keystroke advantage of guidance is real but is repeatedly outrun by the sheer throughput of random input, a trade-off prior LLM-testing work does not surface. Coverage and fault-finding. Whether code coverage predicts a test suite’s ability to find faults is a long-running question, and the evidence is mixed even within a single domain. Inozemtseva and Holmes report that coverage is not strongly correlated with suite effectiveness once suite size is controlled [7], and Papadakis et al. qualify the use of mutants as fault proxies at scale [14]. In GUI testing specifically, coverage and fault detection have been found to move together more often [19, 23], and coverage continues to serve as the default proxy in fuzzer benchmarking [2] and fault localisation [18]. Our results place TUIs on the sceptical side of this debate and explain why. A crash truncates the exploration session that found it, capping the coverage that run can accrue, so the very executions that expose faults are penalised on coverage. The decoupling we observe is therefore not noise but a structural property of interactive, crash-terminated testing, which is precisely the regime a TUI tester operates in, and it motivates a coverage criterion keyed on reachable interactive states rather than executed lines. Empirical “how far are we” studies. Our paper follows the empirical tradition that measures the real gap between a technique’s promise and its delivered behaviour rather than proposing a new tool. Choudhary et al. asked whether automated Android input generation had arrived and answered with a sober comparison against random [4]; Lukasczyk et al. subjected automated unit-test generation for Python to the same scrutiny [10]; and Tian et al. assessed how far a frontier LLM actually goes as a programming assistant [22]. Each of these studies derives its value from honest baselines, careful confound analysis, and an open artefact, and recent guidance for LLM-based software-engineering experiments codifies exactly these practices [1]. No such study exists for TUIs, because there has been no benchmark on which to run one. We provide that benchmark, hold the LLM arms to a model-free random baseline and a content-aware crash oracle, and report where automated TUI testing stands today.

Conclusion

Terminal User Interfaces are a large and growing class of software that is poorly tested in practice and lacks a dedicated testing methodology. We presented the first open, multi-language empirical study of automated TUI testing, over a benchmark of 197 headlessly-runnable, coverage-instrumented applications across the four dominant frameworks, driven by four frontier LLMs and a random baseline under an equalized wall-clock budget. No single model dominates, and capability is decoupled from cost. The largest gain comes from automatically deriving launch inputs, not from smarter per-step reasoning; and while random looks 15

competitive under a time budget, per interaction LLM guidance is an order of magnitude more efficient and uniquely reaches input-gated faults, arguing for hybrid strategies. Measurement is itself a research problem: raw exits are 82% noise, and once an oracle isolates the 179 valid faults, line coverage proves a poor proxy for crash-finding. Automated TUI testing is feasible but far from solved, and we hope this study establishes it as a research area in its own right. We release the cross-language coverage tool tuicov at https://github.com/tui-testing/tuicov and the automated TUI testing framework tuibot at https://github.com/tui-testing/tuibot.

References [1] Sebastian Baltes, Florian Angermeir, Chetan Arora, Marvin Muñoz Barón, Chunyang Chen, Lukas Böhme, Fabio Calefato, Neil A. Ernst, Davide Falessi, Brian Fitzgerald, Davide Fucci, Marcos Kalinowski, Stefano Lambiase, Daniel Russo, Mircea Lungu, Lutz Prechelt, Paul Ralph, Christoph Treude, and Stefan Wagner. Guidelines for empirical studies in software engineering involving large language models. arXiv preprint arXiv:2508.15503, 2025. [2] Marcel Böhme, László Szekeres, and Jonathan Metzman. On the reliability of coveragebased fuzzer benchmarking. In Proceedings of the 44th International Conference on Software Engineering (ICSE), pages 1621–1633. ACM, 2022. doi: 10.1145/3510003.3510230. Bubble tea: A powerful little TUI framework. [3] Charm. charmbracelet/bubbletea, 2024. Accessed: 2026-06-24.

https://github.com/

[4] Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso. Automated test input generation for Android: Are we there yet? In Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 429–440. IEEE, 2015. doi: 10.1109/ASE.2015.89. [5] Vadim Demedes. Ink: React for interactive command-line apps. https://github.com/ vadimdemedes/ink, 2024. Accessed: 2026-06-24. [6] Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pages 423–435. ACM, 2023. doi: 10.1145/3597926.3598067. [7] Laura Inozemtseva and Reid Holmes. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th International Conference on Software Engineering (ICSE), pages 435–445. ACM, 2014. doi: 10.1145/2568225.2568271. [8] Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. Make LLM a testing expert: Bringing human-like interaction to mobile GUI testing via functionality-aware decisions. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), pages 1–13. ACM, 2024. doi: 10.1145/3597503.3639180. [9] Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Zhilin Tian, Yuekai Huang, Jun Hu, and Qing Wang. Testing the limits: Unusual text inputs generation for mobile app crash detection with large language model. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), pages 1–12. ACM, 2024. doi: 10.1145/3597503.3639118. [10] Stephan Lukasczyk, Florian Kroiß, and Gordon Fraser. An empirical study of automated unit test generation for Python. Empirical Software Engineering, 28(2), 2023. doi: 10.1007/ s10664-022-10248-w. [11] Aravind Machiry, Rohan Tahiliani, and Mayur Naik. Dynodroid: An input generation system for Android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), pages 224–234. ACM, 2013. doi: 10.1145/2491411.2491450. [12] Ke Mao, Mark Harman, and Yue Jia. Sapienz: Multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA), pages 94–105. ACM, 2016. doi: 10.1145/2931037.2931054. 16

[13] Microsoft. TUI Test: End-to-end terminal testing framework. https://github.com/ microsoft/tui-test, 2024. Accessed: 2026-06-24. [14] Mike Papadakis, Donghwan Shin, Shin Yoo, and Doo-Hwan Bae. Are mutation scores correlated with real fault detection? a large scale empirical study on the relationship between mutants and real faults. In Proceedings of the 40th International Conference on Software Engineering (ICSE), pages 537–548. ACM, 2018. doi: 10.1145/3180155.3180183. [15] Ratatui Developers. Ratatui: A Rust library to build rich terminal user interfaces. https: //ratatui.rs, 2024. Accessed: 2026-06-24. [16] Andrea Romdhana, Alessio Merlo, Mariano Ceccato, and Paolo Tonella. Deep reinforcement learning for black-box testing of Android apps. ACM Transactions on Software Engineering and Methodology, 31(4):1–29, 2022. doi: 10.1145/3502868. [17] Max Schäfer, Sarah Nadi, Aryaz Eghbali, and Frank Tip. An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 50(1):85–105, 2024. doi: 10.1109/TSE.2023.3334955. [18] André Silva, Matias Martinez, Benjamin Danglot, Davide Ginelli, and Martin Monperrus. FLACOCO: Fault localization for Java based on industry-grade coverage. arXiv preprint arXiv:2111.12513, 2021. [19] Jaymie Strecker and Atif Memon. Relationships between test suites, faults, and fault detection in GUI testing. In 2008 1st International Conference on Software Testing, Verification, and Validation (ICST), pages 12–21. IEEE, 2008. doi: 10.1109/ICST.2008.26. [20] Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), pages 245–256. ACM, 2017. doi: 10.1145/3106237.3106298. [21] Textualize. Textual: A Rapid Application Development framework for Python. https: //textual.textualize.io, 2024. Accessed: 2026-06-24. [22] Haoye Tian, Weiqi Lu, Tsz On Li, Xunzhu Tang, Shing-Chi Cheung, Jacques Klein, and Tegawendé F. Bissyandé. Is ChatGPT the ultimate programming assistant – how far is it? arXiv preprint arXiv:2304.11938, 2023. [23] Xun Yuan, Myra B. Cohen, and Atif M. Memon. GUI interaction testing: Incorporating event context. IEEE Transactions on Software Engineering, 37(4):559–574, 2011. doi: 10.1109/TSE. 2010.50.

Can LLMs Test Terminal User Interfaces?

Analyze & explore related research in Conceptio

Related documents