Technology-Enhanced Tabletop Exercises for Cybersecurity Education: Lessons Learned

Technology-Enhanced Tabletop Exercises for Cybersecurity Education: Lessons Learned Jan Vykopal

Pavel Čeleda

Martin Horák

Valdemar Švábenský

arXiv:2607.28179v1 [cs.CY] 30 Jul 2026

Faculty of Informatics Faculty of Informatics Faculty of Informatics Faculty of Informatics Masaryk University Masaryk University Masaryk University Masaryk University Brno, Czech Republic Brno, Czech Republic Brno, Czech Republic Brno, Czech Republic [email protected] [email protected] [email protected] [email protected] 0000-0001-8546-280X 0000-0002-1835-6465 0000-0002-3425-0951 0000-0002-3338-2856

a web application designed to automate exercise delivery and enable data-driven evaluation. Unlike traditional pen-and-paper TTXs, IXP can automatically deliver scenario updates, facilitate team discussions, and collect interaction data for automated assessment. This approach enhances realism, reduces instructor workload, and offers actionable insights into student learning behaviors. Its uniqueness lies in combining experiential learning with analytics to improve teaching outcomes. This paper describes the innovation and lessons learned through designing, delivering, and evaluating 25 TTXs using the platform from October 2024 to March 2026. Automated data collection enabled analysis of team performance, communication patterns, and decision-making processes. The post-exercise feedback collected from trainees and our observations indicate improved student engagement and collaboration, and faster feedback compared to traditional teaching methods. This paper shares our recommendations for effective planning, development, delivery, and post-exercise reflection. We emphasize the importance of realistic, well-thought-out scenarios and of instructor support during TTX. Although our recommendations are based on experience from conducting TTXs in cybersecurity, they are not domain-specific and can be used for digital TTXs in other areas. I. I NTRODUCTION The paper is organized into six sections. Section II introduces the TTX format and recent related work. Section III presents A cybersecurity tabletop exercise (TTX) is a conversation the platform for the digital TTXs. Section IV summarizes the between participants who are responsible for fulfilling a phases of the exercise lifecycle, which provide the structure variety of roles during a cybersecurity incident [1], such as a for Section V, where we describe the lessons learned from phishing campaign, ransomware, or denial-of-service attacks. creating and conducting multiple TTXs using the INJECT The exercises are run by governments [2], [3], military [4], [5], Exercise Platform. Section VI concludes the paper and outlines [6], critical infrastructure [7], and private companies [8]. The future work. goal is to improve collaboration among roles during incident response, strengthen security awareness, reduce incident-related II. BACKGROUND AND R ELATED W ORK costs, or meet legal or contractual obligations and standards [9]. While TTXs are widely used in professional settings, they A tabletop exercise is an experiential and active learning remain underutilized in university courses [10], [11], [12]. This method, specifically, a form of simulation-based learning [14], paper addresses this gap by introducing a technology-enhanced [15] in small teams. Academic literature on cybersecurity TTXs approach to TTX delivery and evaluation. The innovative prac- and their practical delivery was reviewed by Vykopal et al. in tice involves implementing TTXs in cybersecurity courses using 2024 [10]. The following text provides a brief overview of the openly available INJECT Exercise Platform (IXP) [13] – the TTX structure and refers readers to the review for more

Abstract—This innovative practice full paper examines the integration of technology-enhanced tabletop exercises (TTXs) into computing education, focusing on cybersecurity curricula. The motivation is to better prepare students for complex, collaborative problem solving typical of incident response and IT governance, where coordination, communication, and timely decision-making are essential. Although TTXs are well-established in professional practice, they remain underused in universities. We address this gap by augmenting TTX delivery and evaluation through the INJECT Exercise Platform (IXP), a web-based environment that automates scenario flow and enables data-driven assessment. Our practice implements IXP to automatically deliver scenario updates, facilitate team discussions, and collect interaction data to support automated assessment. This combination enhances realism, reduces instructor workload, and provides actionable insight into student learning. From 2024 to 2026, we ran 25 exercises with 743 participants in multiple university courses and extracurricular events. We observed increased engagement and collaboration among students, and clearer visibility for instructors into how teams navigate complex scenarios. This paper shares 24 lessons learned from these exercises. Instructors and curriculum designers may benefit from concrete guidance for integrating technology-enhanced TTXs. We demonstrate that digital TTXs provide a scalable and replicable model for cybersecurity courses and others requiring team-based problem-solving. Index Terms—collaborative learning, cyber security, cyber exercise, tabletop exercise, TTX, incident response

©2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Cite this article as follows: J. Vykopal, P. Čeleda, M. Horák, V. Švábenský. Technology-Enhanced Tabletop Exercises for Cybersecurity Education: Lessons Learned. In Proceedings of the 56th IEEE Frontiers in Education Conference (FIE ’26). Paphos, Cyprus, 2026. DOI: TODO add after the proceedings publication.

comprehensive information. We also summarize the most recent literature that was not included in the review.

sectors. The participants found the TTXs valuable but too short. Lastly, Dwight [20] proposed a TTX format integrating the crime script analysis method used in crime investigation. A. TTX structure However, they did not provide any evaluation. TTXs start with a context-setting and rules briefing that Finally, the latest research on digital TTXs has been preexplains who the exercise participants are, their roles, re- sented in two papers. Watkins et al. [21] introduced a platform sponsibilities, mandate, tools, and communication channels. that transforms static, scripted TTXs into dynamic, decisionFor example, participants can be employees of a fictitious responsive simulations. The platform uses AI agents and organization, working as members of a cybersecurity incident knowledge bases such as MITRE ATT&CK to automatically response team responsible for coordinating incident handling. generate injects. The authors presented only a proposal for They must report to the chief security officer, follow the the evaluation metrics (surveys, system logs) because their organization’s processes, and comply with national laws regu- research has not yet been approved by their Institutional Review lating cyber incident response. They can task other simulated Board. Sumereder et al. [22] showcased digitalization of TTXs; participants, for example, IT administrators in the organization, however, for training emergency response outside cyberspace. when investigating an incident. The communication channel They introduced a proof-of-concept system utilizing cameras between exercise participants is a simulated email. and markers to capture physical modeling elements on a table. Once the context is set, the participants receive a number The approach was demonstrated using a road traffic accident of injects, inputs that stimulate further actions and discussions scenario to show how digital tools can facilitate faster data and advance the exercise [10]. Injects can be email messages, collection, objective evaluation, and remote participation. alerts from detection systems, a new report published by a III. INJECT E XERCISE P LATFORM trusted organization, or breaking news in the media. Participants The INJECT Exercise Platform is a lightweight, opendiscuss new information within their teams and decide what to do based on their roles and the provided organizational context. source web-based learning environment designed to deliver For instance, if the inject is a user report on phishing received and evaluate TTXs [13], [23]. It transitions the traditional penin their inbox, the participants should follow an explicitly or and-paper format into a digital format that automates repetitive tasks for instructors and provides insights into student behavior. implicitly defined process of handling such an incident. The platform delivers injects to teams based on time or the Injects may not always assign concrete, clear tasks to participants. Some injects may present only a piece of information completion of specific milestones to drive the scenario forward. that will be useful later in the exercise, while others may task Milestones are true/false conditions that track team progress participants to create an artifact, such as a situation report for and determine if a team has reached important situations in their management. Injects can also enable exploring incidents the scenario. Exercise scenarios are defined in human- and machinelasting for weeks in an exercise lasting only a few hours [1]. readable YAML files, enabling the same scenario to be deployed B. Recent related work repeatedly under consistent conditions. The platform supports Here, we summarize new works that were not covered in fully pre-scripted scenarios that do not require any exercise the literature review [10] from 2024. The most relevant and facilitators, as well as complex scenarios that evolve differently comprehensive is a book Cybersecurity tabletop exercises: based on trainees’ specific responses and inputs. In contrast to pen-and-paper TTXs, trainees can interact From planning to execution [1] that guides the reader through planning, developing, facilitating, and evaluating TTXs. It with simplified versions of real-world applications, such as provides example scenarios for different target audiences and a simulated email client to communicate with stakeholders, templates for reports from the exercise. Although the book a web browser to view in-exercise websites, or a firewall to covers tools for facilitating TTXs, it does not mention any block malicious network traffic. dedicated software, only generic polling software, remote The platform logs metadata and trainee actions, including presentation, and collaboration software. email threads, tool usage, and reached milestones. These logs Next, four papers focus on cybersecurity TTXs. Chowd- can be viewed in the built-in dashboard or exported to the hury and Gkioulos [16] proposed a lightweight framework JSONL format. Instructors and exercise designers can use for conducting TTXs, evaluated through exercises involving the dashboard to analyze team performance, such as the time industrial personnel and university students. The exercises were required to reach milestones and the order in which they were structured around the stages of the Lockheed Martin Cyber reached. This information reveals insights into how different Kill Chain. They can also run fully online, leveraging common teams approached the same TTX scenario. software tools and a simulator of a nuclear facility control room. Švábenský et al. [13] reported experience from using the Müller [17] developed and evaluated a TTX for ransomware open-source IXP to deliver a TTX in an undergraduate negotiations. The TTX lasted one hour and was delivered online cybersecurity course. Three runs of the course were described. via common software to five participants [18]. Kävrestad et The first exercise was delivered without a specialized platform, al. [19] held six TTXs lasting from 20 to 70 minutes on using shared text-based documents on Microsoft SharePoint. ransomware for a total of 90 decision-makers from various The second run used the prototype of the IXP, and the third

the very early version of the platform available in November outcomes, regardless of the quality of its technical execution. 2023. By automating data collection and analysis, IXP offered The phase also introduces the IXP’s exercise types, features, instructors pedagogical insights into student behavior that and instructor involvement models, providing the knowledge would be laborious or practically infeasible to capture in required for informed decision-making in later phases. traditional pen-and-paper TTXs. This paper reports lessons learned from creating, delivering, B. Specification phase This phase aims to produce a complete exercise specification and evaluating TTXs from 2024 to 2026, utilizing newer versions of the platform (v1–v5) released during this period [24]. before content is created. Separating conceptual design from IXP has evolved into a sophisticated training environment that implementation reduces the risk of a common failure: designers supports diverse multimedia injects and offers flexibility for who move directly to the platform tend to oscillate between both trainees and instructors. Key advancements include support narrative decisions and technical configuration, increasing the for on-demand exercises, allowing participants to initiate likelihood of an incoherent final product. To design effectively sessions at their convenience, and multi-tenancy, which enables at this stage, designers must connect two distinct competencies the simultaneous execution of multiple independent exercises – understanding what IXP enables, and translating learning for different groups. Furthermore, integrated dashboards now objectives into concrete learning activities and injects. The streamline delivery and post-exercise analysis for instructors. output is a specification detailed enough that the subsequent For content development, designers can utilize a built-in preparation phase becomes a matter of execution. exercise editor or a dedicated Visual Studio Code extension [25] for external development or fine-tuning exercises from the C. Preparation phase In this phase, exercise designers translate the exercise specieditor. Finally, architectural optimizations now permit the fication into a form that the platform can execute. Designers platform to support up to 100 teams running in parallel. implement their specification either by directly editing YAML IV. INJECT P ROCESS definition files or by using the platform’s visual editor. This The INJECT Process is a way to design, execute, and reflect encompasses two distinct tasks: implementing the chain of TTXs with the IXP. The Process leverages best practices for events through milestone logic and conditional triggers, and instructional design, TTXs, and features offered by the IXP. creating the content – inject texts, email templates, documents, It consists of five phases inspired by design thinking: under- and tool outputs – that trainees will encounter. Before delivery, standing, specification, preparation, execution, and reflection the exercise must be tested to verify that the implemented logic (Figure 1) [26]. The phases are similar to the phases of life matches the intended design. Designers can draw on exemplary cycles used for planning and conducting cybersecurity exercises, exercise definitions [31] that serve as practical starting points. such as [3], [27], [28], [29], [30]. However, none of these focus on TTXs in detail. The INJECT Process has been tailored for D. Execution phase digital TTXs with respect to the IXP. The execution phase is where the experience is delivered to trainees. The exercise may run synchronously or ondemand, in person or remotely, as a one-time event or repeated across multiple cohorts. Regardless of format, the quality of execution determines whether the investment in design and preparation translates into meaningful learning. The phase covers the full arc of delivery: onboarding first-time users of the platform, providing an initial briefing that sets the context and expectations, managing the exercise as it unfolds, supporting instructors in their real-time facilitation and evaluation tasks, and concluding with a structured wrap-up (so-called “hot wash”) that transitions the exercise into reflection. E. Reflection phase Fig. 1. INJECT Process phases [26].

A. Understanding phase The purpose of this phase is to establish whether the exercise is worth building at all – and if so, for whom and toward what end. Designers and instructors are guided through methods for identifying the need behind the exercise, characterizing the target audience, and mapping the constraints that will shape subsequent design decisions. Neglecting this groundwork increases the risk that the exercise fails to achieve its intended

The reflection phase closes the exercise lifecycle, but its purpose extends beyond evaluation. It operates on three levels: assessing trainee performance and connecting exercise data to actionable next steps for participants, reviewing the scenario to identify what worked and what did not, and reflecting on the overall organization and execution to improve future runs. The platform’s analytics – milestone timing, team decisions, scoring, and communication patterns – provide a richer empirical basis for this assessment than traditional TTX formats allow. The phase yields concrete recommendations for scenario

TABLE I E XERCISES DELIVERED FROM O CTOBER 2024 TO M ARCH 2026 THROUGH IXP VERSION 1 TO 5. D ISCUSSION EXERCISE = Ü. S IMULATION = ¨. Type ¨ ¨ ¨ ¨ Ü Ü ¨ ¨ ¨ ¨ Ü

Goal Respond to short reports as an incident response team Practice several phases of incident handling Practice writing an advisory and executive summary Practice response to a breach of personal data Train response to loss of a mobile device Train response to loss of a mobile device Introduce threat modeling in maritime security Train handling of a phishing incident Train handling of a phishing incident Train handling of a phishing incident Train response to an insider threat

Instructor Yes Yes Yes Yes No No No Yes Yes Yes Yes

improvement, identifies facilitation strengths worth preserving, and surfaces weaknesses to address before the next delivery.

Exercises Trainees 2 113 2 111 2 111 2 109 8 108 3 45 2 56 1 15 1 30 1 24 1 21 25 743

Target audience Cybersecurity students – one course, two years Cybersecurity students – one course, two years Cybersecurity students – one course, two years Cybersecurity students – one course, two years Undergraduate students – various disciplines Students of information studies – summer school Cybersecurity students from two countries – one remote Undergraduate computer engineering students Vocational school students and their teachers Finalists of the national cybersecurity competition Finalists of the national cybersecurity competition

b) Understand your target audience: Creating an exercise that is valuable for everyone is challenging. Even trainees enrolled in the same course, working in the same role, or V. L ESSONS LEARNED FROM EDUCATIONAL PRACTICE coming from the same organization differ in their backgrounds Here we present lessons from 25 technology-enhanced as well as in their levels of knowledge, skills, abilities, and TTXs we organized in 2024–2026 for a total of 743 trainees, competencies. Narrowing the target audience can therefore mostly undergraduate students as a part of their coursework, simplify and focus the exercise design. For example, if the but also high school students completing extra curricular target audience is young university students, incorporating activities. Table I details each exercise and its target audience. technologies and services they are already familiar with can In discussion TTXs, trainees respond to problems through make the scenario more accessible and engaging. questionnaires, decision tasks, and media inputs, aiming to c) The exercise format is a design decision: The platimprove communication. Simulations offer a realistic, process- form supports multiple exercise formats. Choosing the most focused experience featuring email and tool use. suitable format is crucial as it determines possible learning The lessons were derived from structured trainee feedback and instructor role. A simulation built around organizational systematically collected via post-exercise questionnaires and processes becomes counterproductive when participants share insights from instructor focus groups conducted after each event, no common background. A simulation relying on interaction attended by 8 instructors in total. The lessons are organized with facilitators cannot be delivered to numerous trainees according to the INJECT Process phases (see Section IV). without a corresponding number of facilitators. A discussionbased exercise loses its value in on-demand mode, where there A. Understanding phase is no facilitator to surface disagreement or slow down premature a) Digital format offers new possibilities: Using a digital consensus. Getting the format right requires understanding the platform enables simulating situations that would be impractical audience and the learning objective before opening the editor. or even impossible on paper. For instance, the platform can d) Lowering the barrier to building does not eliminate the branch the scenario based on team decisions – different choices need to understand first: IXP makes it easy to start creating. lead to different consequences, making the exercise responsive Designers can be tempted to substituting investigation of to what trainees do rather than following a single predetermined trainee needs and organizational context for personal intuition. path. Beyond this, instructors do not need to print injects and In our experience, exercises built this way tended to cover handouts for teams. The exercise can also incorporate videos topics the designers found interesting rather than gaps trainees or audio, or automatically and immediately request further faced. The result is a TTX that works technically but lands input based on trainees’ previous actions. This enhances the without impact. Identifying the need upfront does not slow the immersion and authenticity of the learning experience. Digital process down. Leveraging existing incident reports, competency TTXs have proven easy for trainees to participate in, without frameworks, or stakeholder conversations can redirect effort the need for lengthy briefings for first-time participants before toward something that will matter. the exercise. This represents an advantage over other, more complex team-based teaching formats such as POGIL [32]. B. Specification phase Finally, IXP accelerates the evaluation of trainees’ inputs, which would otherwise be submitted on paper or through standard a) Every activity in the exercise should lead to learning office software. When the TTX is well designed, trainees objectives: When specifying learning activities, designers and instructors can observe and assess team performance should use the learning objectives and the target audience immediately after the exercise. as a filter to decide whether an activity should be included.

Having more than one designer can help, as ideas proposed f) Exercise creation is difficult, even with platform support: by one can be challenged by another. IXP automates delivery, structures the scenario flow, and b) Milestone logic is the hardest cognitive shift for handles data collection. This removes substantial overhead, exercise designers: Designers naturally think in content and but it does not reduce the complexity of the underlying narrative: what happens next, what information trainees receive, task. Creating a good exercise requires connecting learning and how the story unfolds. The platform requires a different objectives, trainee context, narrative coherence, milestone logic, mode of thinking: states, conditions, and triggers. What action content quality, and difficulty calibration. These aspects interact: activates which milestone? What happens if the expected action a change to the scenario flow affects the milestone logic, which never occurs? This shift from storytelling to system logic is affects what feedback trainees receive, which affects whether unintuitive, and it is where specification most often breaks the learning objective is reached. Designers who approach the down. Designers who rush this step tend to produce exercises platform expecting the tooling to absorb this complexity tend that flow well on paper but freeze in practice. Trainees miss an to underestimate preparation time. A first exercise takes longer action the designer considered obvious, the expected milestone than anticipated – not because the platform is hard to use, but never activates, and the exercise stalls. Unlike traditional TTX because designing a TTX that works for a specific group of methodologies, which have no equivalent mechanism, IXP trainees is a complex task. makes this logic explicit and consequential. Getting comfortable with it during specification, before using the exercise editor, is C. Preparation phase an effective way to avoid rework later. a) Digital format and new features bring new challenges: c) Milestones enable adapting the speed of the exercise Although the exercise definition format is human-readable, to each team: Milestone logic allows transforming TTXs from creating multiple valid and interconnected YAML files remains a simple sequence of facilitator-driven injects delivered at challenging even for IT experts – and nearly impossible for predefined times into an exercise that adapts to each team’s pace. those without prior experience editing YAML. This barrier has For example, instead of sending an inject to all teams exactly been lowered by introducing two tools to IXP: an exercise ten minutes after the exercise begins, designers can trigger editor integrated into the platform and a Visual Studio Code different injects based on the answers to specific questions or extension [25]. The editor is a web-based wizard (see Figure 2) the use of certain tools. that allows non-technical users to define exercise elements and d) The Tools feature enhances simulation exercises: Simulink them together into a scenario. The Visual Studio Code lation exercises are built around processes that typically involve extension validates YAML definitions against their schema actions which i) produce information essential for subsequent and provides code snippets for common exercise components. steps, ii) require complex or real tools or infrastructure to execute, or iii) take significantly longer to perform than the Designers can begin building an exercise in the editor and duration of the exercise itself. Compared with the traditional later fine-tune details directly in YAML when needed. These discussion-based exercises, a digital platform enables these tools substantially simplify the work for novice designers and actions to be simulated through in-exercise tools. Such tools accelerate the workflow for experienced ones. b) AI assistants help with specific preparation tasks but automatically provide different textual or multimedia responses do not replace design judgment: Generative AI tools proved depending on the inputs submitted by trainees. For example, useful at specific, bounded points in the preparation process: a tool can simulate a firewall capable of blocking network rewriting emails to match a character’s voice consistently across traffic. When a trainee enters a specific IP address, the tool a scenario, generating plausible variants of inject content, and activates a corresponding milestone that triggers an inject or identifying gaps in milestone coverage when prompted with alters the overall progress of the exercise. Most exercises a scenario description. What they did not replace was the we conducted simulate an incident response process, which judgment required to decide what the exercise should achieve, includes several such tools. Feedback collected from trainees whether the milestone logic was sound, or whether the difficulty consistently indicates that these tools make the exercises more was calibrated correctly for the target audience. The useful realistic and interactive. framing is AI as a preparation accelerator for execution-level e) Specification and preparation are iterative: The INtasks – not as a design partner for decisions that require JECT Process presents the specification before preparation understanding the trainees and the learning objectives. because a solid specification makes content creation faster and more focused. In practice, however, the boundary between the c) Content reuse makes the model sustainable: Creating two phases is permeable. Implementation surfaces problems a TTX from scratch is time-consuming. Repeated delivery that were invisible at the design stage: a sequence of identical is viable by reusing and adapting existing scenarios. In our inject types that felt varied in a diagram becomes repetitive and experience, once a scenario is well-specified and implemented, fatiguing for trainees in the platform. A branching structure redeploying it for a new cohort requires minimal additional that seemed elegant on a whiteboard turns out to require more effort: adjusting contextual details, refining content based on instructor intervention than possible. These discoveries are a previous runs, and assigning participants to a new exercise part of the process. Specification reduces the cost of iteration, instance. The open-source exercise library provided with the but does not eliminate it. IXP [31] accelerates this further by giving new designers a

Fig. 2. Screenshot of the Editor in IXP depicting a part of the exercise with a learning activity with three injects connected through two milestones.

starting point. Sustainability of the model depends on treating scenarios as assets that accumulate value over time. D. Execution phase a) Two or three is a good team size: In our experience, teams of two are sometimes too small, while teams of four can be too large. Three members tend to form the optimal team size: two people may not provide enough diverse perspectives, and a third can help facilitate dialogue between members with differing views. Larger groups (more than three) often lead to reduced engagement from some participants. From a practical standpoint, forming more smaller teams may, in certain exercises, require additional instructors to manage and respond to their outputs effectively. b) Technology lowers the facilitation bar and lends the exercise legitimacy: The platform holds the exercise structure, can deliver injects automatically, and manages the scenario flow without requiring constant instructor intervention. In practice, this means that a less experienced facilitator can run a technically sound exercise. It also means that trainees enter the experience differently – the purpose-built environment signals that this is a structured activity. The technology itself confers credibility. The risk is that this scaffolding creates an illusion of coverage – a facilitator can follow the tool and miss what is happening in the room. Whether teams are engaging with the scenario, whether a group is stuck or just quiet, whether a critical moment deserves to be slowed down – these judgments remain human. The platform lowers the entry bar for execution and raises the perceived value of the exercise, but the ceiling of what it can achieve still depends on the person running it. c) Instructors should receive support: If the exercise requires interactions between trainee teams and instructors, designers should use platform features that support instructors during the exercise. For example, if email communication is involved, designers need to prepare not only for the expected workflow but also for alternative responses that trainees may send. To reduce instructor workload during the exercise, designers should provide predefined email templates covering a range of possible replies to trainees (see Figure 3). Another

Fig. 3. Screenshot of email templates for answering trainees’ mails in a TTX.

best practice is providing the instructor notes that detail actions and inputs that require instructors’ assistance. This information serves as a valuable guide for novice instructors and a refresher for those who have already instructed the exercise. d) Real-time view of trainees’ progress is invaluable for instructors during the exercise runtime: Instructors can immediately see the current state of the exercise for all teams. In particular, they can monitor which learning objectives have already been achieved by individual teams by completing the corresponding learning activities, each tracked through predefined milestones (see Figure 4). This real-time view provides the instructor with clear insight into whether the exercise is progressing as expected or whether any teams are struggling and may require additional support. e) Instructor evaluation is powerful but fragile under load: The ability for instructors to evaluate trainee responses in real time is one of the platform’s most distinctive features. The instructors can use predefined email templates, assess free-form submissions, or adjust milestone activation based on trainees’ responses. This enables a qualitative assessment that purely

Fig. 4. Real-time view showing the achievement of learning objectives and activities by all teams in an exercise.

Fig. 5. Analyst view showing team milestones grouped into three clusters (blue, green, and red) by activity similarity. Since the axes represent abstract dimensions, the key factor is the distance between the clusters, which guides the analyst’s focus.

automated systems cannot replicate. The fragility appears interfaces, seamless interactions. This is not an unreasonable at scale: when multiple teams reach an instructor-dependent expectation, but it is different, and it catches designers off guard. milestone simultaneously, some teams may wait while others Minor friction that would be invisible in a traditional format progress. This creates uneven pacing. One partial solution is – a slow page load, an unintuitive navigation step, a visual to decouple evaluation from execution – instructors review that does not render cleanly – becomes noticeable and affects and score responses after the exercise concludes rather than in perceived quality. The implication is not that the platform real time, removing the bottleneck from the critical path while needs to compete with consumer software, but that expectation preserving qualitative assessment. The tradeoff is that trainees management and interface quality carry more weight in digital lose immediate feedback during the exercise itself. A more delivery than experience with traditional TTXs would suggest. promising direction we are currently exploring is AI-assisted E. Reflection phase evaluation: the platform can surface relevant trainee responses a) The digital format enables detailed post-exercise analywith suggested assessments, which instructors confirm or adjust rather than generating from scratch. Early experience suggests ses for improving the exercise: All trainees’ inputs and actions this substantially reduces evaluation time without sacrificing are logged and processed by the platform, which provides valuable data and insights not only for reflection but also for the human judgment that makes the assessment meaningful. f) Remote exercise runs should be accompanied by an improvement of the exercise itself. This is particularly useful external communication channel: We recommend setting up after the first run of a new or updated exercise. We use the a synchronous communication channel outside the exercise provided data to answer the following design questions: • Did the majority of teams achieve all learning objectives platform, such as a video conference. The channel can be used or high score (see Figure 4)? If not, the exercise might for pre- and post-exercise briefings, as well as virtual breakout not match their proficiency and needs to be revised. rooms to support fully distributed teams. • Did the teams follow the anticipated progression, or were g) Fully automated exercises scale best but might lack there significant deviations? Figure 5 illustrates the analytflexibility: The platform makes it possible to deliver exercises ical results addressing this question. Do these anomalies that each team can start independently, at a time that suits them. represent valid, but unexpected solutions aligned with the Unlike synchronous exercises, which begin simultaneously for learning objectives, or do they expose weaknesses in the all teams, these on-demand exercises cannot rely on human fascenario design that require revision? The answer to this cilitators, as they are not continuously available. Consequently, question can be found in the causal graph of exercise on-demand exercises must be fully automated. While this events, see Figure 6. approach allows the exercise to be offered to hundreds of • What is the qualitative assessment from the participants? participants, it also restricts the design to exercise elements We conclude the exercise by asking trainees for feedback. that can be automatically evaluated, such as multiple-choice Having their answers in the platform allows for more questionnaires. These constraints may lead to less flexible convenient analysis of their experience compared to the scenarios, potentially reducing the trainees’ overall experience. feedback collected via external survey tools. h) Once the exercise is digital, trainees compare it to other digital experiences – not to TTX standards: A paperFurthermore, the ability to export exercise logs in JSONL based TTX is evaluated against other paper-based TTXs. A format is invaluable for external data processing. This feature digital one is evaluated against every other digital experience streamlines computing education research, which can, in turn, trainees have encountered – polished applications, responsive influence the practical implementation of digital TTXs.

Fig. 6. Analyst view showing a graph of selected exercise events for a team. Activating the left milestone triggered an inject and tool access. Tool use then activated another milestone that opens a form. The team made four submissions, and one of them activated the milestone on the right. For example, this view can be helpful for revealing unexpected progression that may not be intended by the scenario designer.

b) Multiple exercises for the same group of trainees bring removes significant problems: manual delivery, paper-based benefits: Once trainees receive their accounts in the platform, data collection, and coordination overhead. However, it does not they can be easily assigned to more exercises without additional remove the complexity of designing TTXs that achieve learning overhead of creating new accounts. What is more, instructors outcomes, facilitating experiences that trigger reflection, or can see and compare trainee performance across exercises. establishing the organizational conditions in which exercises c) The exercise is the trigger – reflection is where the lead to behavioral change. In each phase of the lifecycle, learning happens: Trainees engage with the scenario during we found that the platform’s potential was either realized or execution but the deeper processing comes afterward. This diminished based on decisions and practices outside the tool. includes understanding why their decisions were right or wrong, Ultimately, these insights underscore that the human-in-theconnecting the exercise to their organizational context, and loop remains an indispensable element in the exercise life cycle. identifying what they would do differently. All of this happens Our objective is therefore not to replace the educator, but to in the conversation that follows. In our experience, trainees provide a robust framework that supports and augments their consistently wanted more time for debriefing than the schedule expertise. In this view, technology serves as an enabler of, allocated. They wanted to compare their decisions with other rather than a substitute for, high-quality human instruction. teams, hear the reasoning behind the scenario design, and The lessons are relevant to educators integrating digital TTXs discuss whether their responses would hold in reality. IXP and developers of similar platforms. These insights highlight supports this: milestone timing, team decisions, scoring, and where developers can focus their efforts to address current communication patterns are all captured and available for limitations and enhance support for instructors and learners. structured review in ways that paper-based TTXs cannot match. The risk is that this data goes unused. Trainees leave with A. Future work Because exercise specification and preparation are complex, an experience rather than anything concrete to act on – no checklist or commitment to a specific next step. A structured demanding, and time-consuming phases, there is potential for debriefing that connects TTX data to actionable outputs would using language models to generate TTX scenarios or their substantially increase the probability of behavioral change of components, such as individual injects. Language models may trainees after the exercise. This value, however, depends on a also support the analysis of trainees’ textual responses during facilitator who can surface disagreement, slow down premature and after an exercise, as well as the generation of reports for consensus, and anchor the discussion in what happened. On- trainees and instructors. While large language models typically demand exercises sacrifice this. They are the most scalable achieve higher performance, smaller models may be more format, but scaling delivery at the cost of structured reflection suitable in cybersecurity education contexts, as they can be trades the exercise’s most durable benefit for its most convenient run locally without sharing sensitive data with third parties. Another direction for future work is enhancing post-exercise form. Treating reflection as a procedural closing step, or reflection. Both our experience and existing research [33], removing it entirely, leaves significant potential unrealized. [34] emphasize that reflection is a crucial phase. Yet it VI. C ONCLUSIONS AND F UTURE W ORK remains insufficiently addressed, as most attention is devoted to TTXs are widely used in professional practice but remain earlier phases: planning, preparation, and execution. Improving relatively uncommon in higher education. Based on our structured reflection processes and tooling could strengthen experience conducting 25 exercises with 743 participants, learning outcomes and support long-term skill development. we demonstrated that TTXs can be effectively integrated ACKNOWLEDGMENT into university-level teaching when supported by a dedicated This research was supported by the Open Calls for Security platform. We presented 24 lessons learned from two years of designing, developing, running, and evaluating these exercises Research 2023–2029 (OPSEC) program granted by the Ministry of the Interior of the Czech Republic under No. VK01030007 using the open-source INJECT Exercise Platform [23]. The lessons share a common theme: the value of technology- – Intelligent Tools for Planning, Conducting, and Evaluating enhanced TTXs is not determined solely by the platform. IXP Tabletop Exercises.

R EFERENCES [1] R. Lelewski and J. Hollenberger, Cybersecurity Tabletop Exercises: From Planning to Execution. San Francisco, USA: No Starch Press, 2025, ISBN: 978-1718503823. [2] T. Grance, T. Nolan, K. Burke, R. Dudley, G. White, and T. Good, “Guide to test, training, and exercise programs for it plans and capabilities,” NIST, Tech. Rep., 09 2006. [3] A. Zacharis, A. Sarri, C. Van Heurck, F. Fanourakis, G. Fernández, N. Christoforatos, and R. Arcus, “The ENISA Cybersecurity Exercise Methodology,” European Union Agency for Cybersecurity (ENISA), Technical Report, 2 2026. [Online]. Available: https://www.enisa.europa.eu/sites/default/files/2026-02/The% 20ENISA%20Cybersecurity%20Exercise%20Methodology.pdf [4] M. Preda, V. Popescu, C. Argint, N. Iancu, G. Raicu, and G. Ene, “Enhancing civil-military cyber resilience lessons from the ecybridge tabletop exercise.” International Journal of Information Security & Cybercrime, vol. 14, no. 1, 2025. [Online]. Available: https://www.ceeol.com/search/article-detail?id=1349715 [5] C. A. Ramezan, L. C. Schaupp, E. A. Vitullo, and W. J. Walker, “Simulating cyber-resilience: The strategic role of the locked shields exercise in enhancing international cyber preparedness,” Journal of Cybersecurity Education, Research and Practice, vol. 2026, no. 1, p. 5, 2026. [Online]. Available: https://doi.org/10.62915/2472-2707.1260 [6] S. L. Dorton, T. Fersch, E. Barrett, A. Langone, M. Seip, S. Bilsborough, C. B. Hudson Jr, P. Ward, and K. J. Neville, “The Value of Wargames and Tabletop Exercises as Naturalistic Tools,” in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 67, no. 1. SAGE Publications Sage CA: Los Angeles, CA, 2023, pp. 2454–2459. [Online]. Available: https://doi.org/10.1177/21695067231192617 [7] M. Bartnes and N. B. Moe, “Challenges in IT security preparedness exercises: A case study,” Computers & Security, vol. 67, pp. 280–290, 2017. [Online]. Available: https://doi.org/10.1016/j.cose.2016.11.017 [8] P. J. Lenk, S. Kines, E. Taube, M. Münzer, T. Kuusk, and S. Alberico, “Cyber-Security at OSI Layer 1: Defence-in-Depth for Energy Grids,” 2025, MP-SAS-190 Technical Evaluation Report. [Online]. Available: https://publications.sto.nato.int/publications/STO% 20Meeting%20Proceedings/STO-MP-SAS-190/MP-SAS-190-07.pdf [9] European Parliament and Council of the European Union, “Directive (EU) 2022/2555 of the European Parliament and of the Council of 14 December 2022 on measures for a high common level of cybersecurity across the Union,” Official Journal of the European Union, L 333/80, 2022. [Online]. Available: http://data.europa.eu/eli/dir/2022/2555/oj [10] J. Vykopal, P. Čeleda, V. Švábenský, M. Hofbauer, and M. Horák, “Research and Practice of Delivering Tabletop Exercises,” in 29th Conference on Innovation and Technology in Computer Science Education, ser. ITiCSE ’24. New York, NY, USA: ACM, 2024, pp. 220–226. [Online]. Available: https://doi.org/10.1145/3649217.3653642 [11] R. Ottis, “Light Weight Tabletop Exercise for Cybersecurity Education,” Journal of Homeland Security and Emergency Management, vol. 11, pp. 579–592, 12 2014. [Online]. Available: https://doi.org/10.1515/ jhsem-2014-0031 [12] G. Angafor, I. Yevseyeva, and L. Maglaras, “MalAware: A tabletop exercise for malware security awareness education and incident response training,” Internet of Things and Cyber-Physical Systems, vol. 4, pp. 280–292, 2024. [Online]. Available: https: //doi.org/10.1016/j.iotcps.2024.02.003 [13] V. Švábenský, J. Vykopal, M. Horák, M. Hofbauer, and P. Čeleda, “From Paper to Platform: Evolution of a Novel Learning Environment for Tabletop Exercises,” in Innovation and Technology in Computer Science Education. New York, NY, USA: ACM, 2024, pp. 213–219. [Online]. Available: https://doi.org/10.1145/3649217.3653639 [14] O. Chernikova, N. Heitzmann, M. Stadler, D. Holzberger, T. Seidel, and F. Fischer, “Simulation-Based Learning in Higher Education: A MetaAnalysis,” Review of educational research, vol. 90, no. 4, pp. 499–541, 2020. [Online]. Available: https://doi.org/10.3102/0034654320933544 [15] P. Hallinger and R. Wang, “The Evolution of Simulation-Based Learning Across the Disciplines, 1965–2018: A Science Map of the Literature,” Simulation & Gaming, vol. 51, no. 1, pp. 9–32, 2020. [Online]. Available: https://doi.org/10.1177/1046878119888246 [16] N. Chowdhury and V. Gkioulos, “A Framework for Developing Tabletop Cybersecurity Exercises,” in Computer Security. ESORICS 2022 International Workshops. Cham: Springer International Publishing,

2023, pp. 116–133. [Online]. Available: https://doi.org/10.1007/ 978-3-031-25460-4_7 [17] L. Müller, “Tabletop Exercise for Ransomware Negotiations,” in Augmented Cognition, D. D. Schmorrow and C. M. Fidopiastis, Eds. Cham: Springer Nature Switzerland, 2024, pp. 166–184. [Online]. Available: https://doi.org/10.1007/978-3-031-61572-6_12 [18] ——, “Tabletop exercise for ransomware negotiations,” Germany, February 2024, Bachelor’s thesis. [Online]. Available: https://www.researchgate.net/publication/381290646_Tabletop_ Exercise_for_Ransomware_Negotiations [19] J. Kävrestad, S. Johansson, and E. Bergström, “Using Tabletop Exercises to Raise Cybersecurity Awareness of Decision-Makers,” in Critical Information Infrastructures Security, G. Oliva, S. Panzieri, B. Hämmerli, F. Pascucci, and L. Faramondi, Eds. Cham: Springer Nature Switzerland, 2025, pp. 231–248. [Online]. Available: https://doi.org/10.1007/978-3-031-84260-3_14 [20] J. Dwight, “Collaborate, design, and generate cybercrime script tabletop exercises for cybersecurity education,” in International Conference on Computers in Education, 2023. [Online]. Available: https://library.apsce.net/index.php/ICCE/article/view/1406/1300 [21] T. Watkins, B. Davis, R. B. Ponnuru, and M. Azab, “AI-Driven Immersive Emulation for Tabletop Scenarios,” in 2026 IEEE 16th Annual Computing and Communication Workshop and Conference, 2026, pp. 234–240. [22] A. Sumereder, B. Bürger, and R. Woitsch, “Digitalization of TableTop Exercises: An Emergency Response Training Showcase,” in Information Technology in Disaster Risk Reduction, W. Seböck, T. J. Lampoltshammer, J. Dugdale, and I. Zeller, Eds. Cham: Springer Nature Switzerland, 2026, pp. 49–65. [Online]. Available: https://doi.org/10.1007/978-3-031-97115-0_4 [23] INJECT Team, “INJECT Exercise Platform,” https://inject.muni.cz, 2026, Open-source platform for tabletop exercises, accessed: March 20, 2026. [24] ——, “Platform Changelog,” https://docs.inject.muni.cz/changelog/, 2026, accessed: March 11, 2026. [25] ——, “IXP-Definition – MS Visual Studio Code Extension,” https://marketplace.visualstudio.com/items?itemName=inject-muni. ixp-definition, 2026, accessed: March 11, 2026. [26] ——, “INJECT Process,” https://docs.inject.muni.cz/INJECT_process/ intro/overview/, 2026, Documentation of designing tabletop security exercises with the INJECT Exercise Platform, accessed: March 20, 2026. [27] K. Scarfone, T. Grance, and R. Sexton, “NIST Special Publication 800-84: Guide to Test, Training, and Exercise Programs for IT Plans and Capabilities,” https://csrc.nist.gov/pubs/sp/800/84/final, Sep 2006, accessed: March 11, 2026. [28] J. Vykopal, M. Vizvary, R. Oslejsek, P. Celeda, and D. Tovarnak, “Lessons learned from complex hands-on defence exercises in a cyber range,” in 2017 IEEE Frontiers in Education Conference (FIE), 2017, pp. 1–8. [Online]. Available: https://doi.org/10.1109/FIE.2017.8190713 [29] Federal Emergency Management Agency, “Homeland Security Exercise and Evaluation Program (HSEEP),” https://preptoolkit.fema. gov/documents/1269813/1269861/HSEEP_Revision_Jan20_Final.pdf, jan 2020, accessed: March 11, 2026. [30] A. Brilingaitė, L. Bukauskas, and A. Juozapavičius, “A framework for competence development and assessment in hybrid cybersecurity exercises,” Computers & Security, vol. 88, p. 101607, 2020. [Online]. Available: https://doi.org/10.1016/j.cose.2019.101607 [31] INJECT Team, “Available Exercise Definitions,” https://docs.inject.muni. cz/INJECT_process/available-definitions/, 2026, accessed: March 11, 2026. [32] H. H. Hu, C. Kussmaul, B. Knaeble, C. Mayfield, and A. Yadav, “Results from a Survey of Faculty Adoption of Process Oriented Guided Inquiry Learning (POGIL) in Computer Science,” in Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, ser. ITiCSE ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 186–191. [Online]. Available: https://doi.org/10.1145/2899415.2899471 [33] G. Tembrevilla, A. Phillion, and M. Zeadin, “Experiential learning in engineering education: A systematic literature review,” Journal of Engineering Education, vol. 113, no. 1, pp. 195–218, 2024. [Online]. Available: https://doi.org/10.1002/jee.20575 [34] D. T. Rover, H. J. Duwe, M. Mina, N. D. Fila, P. H. Jones, and L. S. Sleeth, “Learning and Professional Development Through Integrated Reflective Activities in Electrical and Computer Engineering Courses,” in 2021 IEEE Frontiers in Education Conference (FIE), 2021, pp. 1–9. [Online]. Available: https://doi.org/10.1109/FIE49875.2021.9637478