SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemes Mengxiao Wang Texas A&M University [email protected]
Nitesh Saxena Texas A&M University [email protected]
can themselves be weaponized [10, 11], a single compromised agent can propagate market-wide harm. This is no longer a research-only concern: in mid-2026 the Bank of England warned publicly that autonomous AI agents could trigger a market meltdown [12], and a Wolters Kluwer survey found that 72% of U.S. banks lack AI-model kill switches or failure-reporting protocols [13]. U.S. legislators have already escalated: in June 2026, House Financial Services Democrats sent SEC Chair Atkins 13 questions on “correlated trading decisions” and “herding behavior” by AI trading agents on registered brokerages, with a 31 July 2026 response deadline [14]. As Figure 1 shows, the systems we study follow recent multi-role designs [15–18]: analyst, researcher, manager, and trader agents that jointly process market data and execute trades. Yet their behavior under extreme conditions is underexplored: even minor algorithmic reactions can cascade, as in the 2010 Flash Crash that briefly erased nearly one trillion dollars [19–21], and compromising even a small subset of the deployed LLM trading agents (e.g., 100k+ reported by Robinhood [8] alone) could deliberately induce the same effect through small perturbations in low-liquidity stocks. This scale of impact, where a fleet of compromised agents can destabilize an entire market, does not appear in the singlesession threat models that generic agentic-security studies assume. Recent work further shows that LLM trading agents systematically inherit and amplify herding, which can amplify flash crashes [22]. This gap frames our central question: how can we measure and ensure that financial LLM agents remain stable under normal market conditions, and resilient under adversarial ones? We approach it as a Systematization of Knowledge (SoK): a scheme-level evaluation of published trading-agent designs, aimed at clarifying the problem and mapping its attack surface before defenses are designed. Answering it starts with what makes these agents different. Their attacks tend to share five traits: they are Instant and Inconspicuous, rely on Unverified inputs, remain Adaptive across channels, and can set off Cascading effects across the market. Generic prompt-injection and jailbreak
Abstract Autonomous large language model (LLM) agents are moving rapidly into high-stakes domains, yet existing agentic-AI security studies remain largely domain-agnostic and overlook the distinctive, high-consequence attack surface such settings create. We examine this gap through financial trading agents, a representative case of high-stakes agentic security, where a single compromised agent has direct execution authority over real capital in an adversarial, reflexive market. To this end, we present FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Testing), a framework that performs scheme-level evaluation of financial LLM agents on two axes: robustness under market turbulence (including flash-crash-like scenarios), and security against three attack types: attacks on information sources, attacks on agents, and agent-as-attacker behaviors. Applying FARSIGHT to 15 representative academic schemes, we find that most overlook robustness and realistic adversarial threats: 80% fail at least one core robustness metric and 100% exhibit security vulnerabilities. These two failure modes are inseparable: a small misjudgment can cascade into a market-wide crash on its own, while an adversary can deliberately trigger the same collapse at minimal cost.
1
Introduction
Autonomous large language model (LLM) agents increasingly invoke tools, move data, and take consequential actions, and their security is now a serious concern. Recent systematizations [1–3] map agentic-AI attacks and defenses at large, but their domain-agnostic treatments cannot capture what changes once an agent operates in a high-stakes domain. Financial markets [4] are the sharpest such setting: actions are financially irreversible, adversaries are directly profit-motivated, and one agent’s output instantly becomes another’s input. Financial LLM agents [5] already manage real capital (e.g., RockAlpha [6], Composer [7]) and serve retail users (e.g., Robinhood [8], Nof1.ai [9]), and because financial systems 1
Figure 1: Overview of threat surfaces in financial LLM agent ecosystems. S1 denotes attacks on market-relevant information sources (e.g., data or news manipulation) that indirectly influence agent decisions; S2 represents direct attacks on LLM trading agents (e.g., prompt injection or model poisoning); and S3 captures scenarios where compromised agents act as attackers themselves: after being influenced via S1 or S2, they inject small perturbations into trading behaviors, which can propagate through market participants (bots, human traders, and market makers), collectively amplifying reactions and potentially triggering large-scale sell-offs or flash crashes. In addition to these security threat surfaces, our robustness assessment (R1–R4) evaluates whether agents can foresee market turbulence, react rapidly to shocks, contain cascading losses, and recover after stress events, reflecting the inherent risks of operating in volatile financial environments.
taxonomies do not model this combination, which our framework is designed to capture. We propose the FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Testing) framework to evaluate agents across robustness and security. Robustness is measured through four metrics: crash foresight, reaction, loss containment, and recovery. As shown in Figure 1, security includes three metrics: S1 (attacks on market-relevant information sources), S2 (attacks on agent logic and prompts, including prompt-injection, tool/Application Programming Interface (API) exploitation, and memory manipulation), and S3 (agent-as-attacker behavior). Our study reveals that current academic financial LLM agents lack both robustness and security by design: across the 15 systems, 80% fail at least one of the four robustness metrics and 100% carry at least one security weakness under realistic financial LLM–specific attacks. The recurring failures show that data, tools, and memory must be treated as attack surfaces enforced outside the model, not benign components trusted by it: safety must ride on cross-validated information, pre-committed risk and execution boundaries, and explicit containment for compromised agents. The adversary here is financially motivated and the agent holds real execution authority, so a single planted feed or hijacked tool can move real capital. Robustness here is itself a security concern rather than a separate reliability issue: in a reflexive, zero-sum market, an attacker can push an agent that mishandles a natural shock into a cascade at trivial cost, so robustness and security cannot be assessed apart. More broadly, we position financial trading agents as a sharp instance of high-stakes agentic security, a setting where
generic, domain-agnostic agentic defenses fall short and that the security community must confront as autonomous agents take on ever more consequential actions. To our knowledge, ours is the first work to jointly systematize market-robustness and security for the academic trading-agent literature. Our Contributions: Our main contributions are outlined below: 1. FARSIGHT: A Scheme-Level SoK Framework for HighStakes Agentic Security. We present FARSIGHT, an evaluation methodology (not a benchmark suite) that scores financial LLM agents on both robustness and security via a reproducible codebook (Appendix A), with dedicated threat models for the execution-authority, reflexive, and zero-sum conditions that make these agents high-stakes. 2. Characterizing the Unique Attack Surface of Financial Trading Agents. We systematically categorize financial LLM agents and identify five defining traits (Instant, Inconspicuous, Unverified, Adaptive, Cascading) that widen their exposure and separate them from agents in law, healthcare, or education. 3. Systematic Robustness and Security Evaluation of 15 Trading Schemes. Applying FARSIGHT (codebook in Appendix B) to 15 representative schemes, we find 80% fail at least one robustness metric and 100% exhibit at least one security weakness; a parallel LLM-based assessment disagreed on 53% of metric-level judgments, showing that fine-grained scoring still requires human expertise. Relevance to Security. (1) Existing agentic-AI security SoKs [1–3] are deliberately domain-agnostic, leaving highstakes application domains, where a single incident is financially or physically irreversible, outside the security lit2
erature. Our paper aligns with this SoK line and instantiates it in high-stakes trading domain. (2) FARSIGHT scores each scheme only on features that map to systemssecurity questions: source poisoning (S1), prompt/tool/memory compromise (S2), agent-as-attacker containment (S3), and adversarial-stress incident response (R1–R4), each with an established USENIX/S&P/CCS literature on non-financial substrates [23–28]. (3) This follows prior security SoKs (e.g., on ML security [29]) that systematize attack surfaces built on artifacts from adjacent, non-security communities.
2
quantitative data [50]. In practice, Alpha Miners often serve as signal generators for Trader Agents, and we treat them as a functional subset in this study.
3
This section introduces our FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Testing) framework, a structured methodology for evaluating the robustness and security of financial LLM agents. FARSIGHT is a scheme-level evaluation framework: each system is scored as a scheme, the design described in its published paper, not as a running instance. It is not a benchmark suite with fixed datasets and automated test harnesses. It provides structured criteria, a codebook-based scoring methodology, and a reproducible evaluation pipeline for expert assessment of agent architectures. This design choice reflects the heterogeneity of financial LLM agents: their diverse architectures, data sources, and deployment contexts make a single standardized benchmark insufficient for capturing the full spectrum of robustness and security concerns. FARSIGHT provides a reproducible foundation for assessing how current financial agents behave under extreme market volatility and adversarial conditions, aiming to bridge the gap between performance-driven agent design and security-oriented evaluation. To ensure replicability and transparency, we include a detailed codebook outlining the coding scheme, metric definitions, and annotation examples in Appendix B.
Background
2.1 Large Language Models and LLM Agents Large Language Models are transformer-based neural architectures [30] trained on large text corpora to capture linguistic and factual patterns. Through large-scale pretraining, they generalize across diverse language tasks, achieving state-ofthe-art performance in reasoning, summarization, and generation. Representative models include GPT-4 [31], LLaMA [32], Mixtral [33], and Claude [34]. Safety and alignment are typically enhanced via supervised fine-tuning and Reinforcement Learning from Human Feedback (RLHF) [35], while Retrieval-Augmented Generation (RAG) [36] improves factual grounding. Building on these foundations, LLM agents extend pretrained models with capabilities for autonomous reasoning, planning, and environment interaction. Unlike static LLMs, agents iteratively reason, act, and observe outcomes through feedback loops [37, 38], integrating modules such as planning, memory, and tool use [39, 40]. Modern frameworks (e.g., LangChain, AutoGPT, ReAct) enable multi-step task execution, including web searches, code execution, and data analysis.
2.2
Our FARSIGHT Framework
3.1
Workflow of FARSIGHT
Inspired by the SoK methodology presented by Akanda et al. [51], we developed FARSIGHT to assess the state of financial LLM agents through literature-grounded evaluation and metric-based analysis. Unlike prior frameworks that focus on implementation toolkits (e.g., LangChain, AutoGPT, ReAct) or benchmarking tasks (e.g., AgentDojo [52], StockBench [53]), FARSIGHT introduces a unified and reproducible evaluation pipeline that emphasizes robustness and security as primary concerns. Relation to general agentic-AI security SoKs. General agentic-AI security SoKs [1–3] map the attack and defense landscape at the domain-agnostic level. FARSIGHT is complementary: rather than re-deriving a generic taxonomy, it operationalizes the high-stakes threat model that distinguishes financial trading agents: execution authority over real capital, systemic reflexivity, and a zero-sum adversarial market. Concretely, it adds a robustness axis with no direct analogue in general agentic SoKs (crash foresight, shock reaction, loss containment, and post-crash recovery, R1–R4) and instantiates the security axis with finance-specific surfaces: manipulation of market-relevant information sources (S1) and agent-as-attacker market destabilization (S3), where a single compromised agent can trigger cascading, flash-crash-like
Financial LLM Agents
Ding et al. [5] define a financial LLM agent, often instantiated as an LLM trading agent, as an autonomous system that leverages large language model reasoning for trading tasks such as market prediction, portfolio optimization, and risk assessment. These agents integrate LLMs with financial data streams, market simulators, and trading APIs to enable datadriven investment decisions [41–43]. Unlike conventional algorithmic trading systems, they exhibit adaptability and contextual understanding through natural language processing, supporting applications such as sentiment-based forecasting [44], portfolio construction [45], high-frequency trading (HFT) [46], and multi-agent risk management [47]. Recent work further explores contest-based reinforcement [48] and decision auditing in investment management [42]. Two main paradigms have emerged: Trader Agents, which execute buy, hold, or sell decisions [15, 16, 49], and Alpha Miner Agents, which identify predictive signals (alphas) from textual and 3
Figure 2: Overview of the FARSIGHT framework. The framework supports both human-expert and LLM-based evaluations. effects. Where such SoKs synthesize the landscape at the survey level, we contribute an operational, evidence-based codebook (Appendix B) applied to 15 concrete trading systems, yielding reproducible per-agent ratings rather than a landscape overview. Closest to our setting is a concurrent SoK on the security of autonomous LLM agents in agentic commerce [54], which systematizes payment, negotiation, and transaction risks, including market manipulation, but at the protocol and landscape level rather than for concrete trading agents. We instead target the academic trading-agent literature and, to our knowledge, are the first to pair a marketrobustness axis (crash foresight, shock reaction, loss containment, and recovery) with the security axis in a reproducible per-agent codebook. As shown in Figure 2, our study follows the FARSIGHT framework to evaluate the robustness and security of financial LLM agents. We began by collecting a comprehensive corpus of academic works and real-world products related to financial LLM agents. From this corpus, we filtered and selected those directly relevant to trading and market-interacting systems, specifically agents that autonomously or semi-autonomously perform financial analysis, strategy generation, or trade execution. Each selected work was analyzed as an instantiated trading agent, forming the foundation of our evaluation. Based on this dataset, FARSIGHT introduces two categories of evaluation metrics: (i) Robustness Metrics, which measure an agent’s capacity to maintain operational stability under volatile or flash-crash-like conditions, and (ii) Security Metrics, which assess resilience against three major threat dimensions: attacks on the information environment, attacks on agents, and agent-as-attacker behaviors. The framework translates these metrics into a systematic evaluation process that traces the entire pipeline, from literature collection to simulated testing and insight generation. We applied these metrics to 15 representative financial LLM agents, identifying
common design weaknesses, overlooked risks, and insufficient consideration of real-world adversarial conditions. Our findings show that most existing systems fail to anticipate the impact of extreme market turbulence or targeted manipulation, leaving users and markets vulnerable to cascading financial losses. Finally, FARSIGHT provides actionable recommendations for building more robust and secure financial LLM agents, emphasizing that robustness and security must be embedded as core design principles rather than as postdeployment safeguards.
3.2
Robustness Metrics
Robustness in FARSIGHT refers to a financial LLM agent’s ability to maintain stable, safe, and rational behavior under sudden and extreme market disruptions (e.g., flash crashes, black swan events). We note that market volatility and price corrections are not inherently undesirable: classical economic theory recognizes rapid price adjustment as a feature of efficient markets [55]. Our robustness metrics do not target market volatility per se, but rather agent-amplified instability: scenarios where LLM agents, through delayed reaction, herding behavior, or cascading feedback loops, exacerbate market disruptions beyond what would occur with human-only participation. The distinction is between natural price discovery and artificial destabilization caused by poorly designed autonomous systems [56, 57]. We define four key robustness metrics that reflect different phases of a market crisis, from early detection to post-crash recovery. R1. Crash Foresight and Early Awareness. This metric assesses whether an agent can anticipate extreme market volatility or potential crashes in advance by identifying anomalous signals (e.g., spikes in the Chicago Board Options Exchange (CBOE) Volatility Index (VIX), correlated asset drops, or sentiment shifts). It measures the agent’s ability to perform 4
proactive risk sensing before catastrophic losses occur. R2. Rapid Reaction Capability under Market Shocks. This metric evaluates the timeliness and appropriateness of an agent’s response during sudden market disruptions. A robust agent should be able to temporarily halt risky orders, freeze trading, or switch to safe-mode strategies within milliseconds of shock detection. R3. Loss Containment and Stop-Loss Execution. This metric measures whether an agent can effectively contain losses and prevent cascading risk once a market crash begins. It captures the design of automated stop-loss mechanisms, exposure caps, and cool-off intervals that limit systemic damage. R4. Recovery and Post-Crash Adaptation. This metric captures the agent’s ability to recover from an adverse event and restore stable operation. It considers whether the agent can rebalance parameters, retrain on updated data, or adaptively recalibrate its strategy after a market crash. To ensure reproducibility, each FARSIGHT metric is assigned using an evidence-based checklist with deterministic scoring rules rather than subjective judgment. We detail this assignment methodology (together with the inter-rater agreement between our two independent evaluators) in Appendix A.
or broader systemic disruption, all without directly compromising the agent itself. This metric examines whether the agent relies on trusted, cross-validated information sources and whether it detects or filters unreliable inputs. In particular, we assess whether the agent validates news authenticity, aggregates signals across multiple feeds, or naively consumes highly manipulable sources such as social media, widely recognized as the easiest channel for coordinated misinformation. S2: Attacks on Agent Logic and Prompts. Threat Model: We assume the financial LLM agent operates in an interactive or API-accessible environment that accepts external inputs (chat requests, automated trading commands, tool-invocation APIs) without strict isolation between system prompts, user prompts, tool permissions, and persistent memory. The attacker can send crafted prompts or payloads to the agent, exploiting its instruction-following behavior, its ability to call tools or APIs, and its capability to write or retrieve long-term memory. The attacker aims to directly subvert the agent’s decision logic, induce unauthorized financial actions, or implant persistent malicious context that continues influencing behavior across sessions. Attack Flow / Objective: The attack proceeds by injecting crafted instructions that blend benign content with adversarial payloads. Once processed by the agent, these injections may override internal trading constraints, trigger unintended tool or API calls, or contaminate long-term memory with biased or adversarial instructions. FARSIGHT decomposes S2 attacks into the following categories:
3.3 Security Metrics and Underlying Threat Models This section presents the threat models associated with each security metric in FARSIGHT, outlining the underlying assumptions, adversary capabilities, attack flows, and the security properties each metric is intended to assess. S1: Attacks on Market-Relevant Information Sources. Threat Model: We assume the financial LLM agent continuously ingests market-relevant information (news feeds, sentiment data, social-media signals) without enforcing strong source authentication, provenance validation, or cross-source consistency checks. The attacker cannot access or modify the agent’s internal prompts or model parameters, but can compromise, forge, delay, or selectively manipulate upstream information sources that the agent relies on. By controlling these external data channels, the adversary can shape the agent’s perception of market conditions and influence its downstream trading decisions. Attack Flow / Objective: The attacker first compromises an external information provider or feed (e.g., a news API, sentiment stream, or social media source). Manipulated content (falsified news, selectively delayed signals, or biased sentiment) is then propagated through the agent’s subscribed channels. The agent ingests these deceptive inputs, integrates them into its decision-making context, and executes trades or reallocations based on corrupted market signals. The objective is to indirectly influence or steer the agent’s trading logic, causing misinformed actions, adversary-favorable outcomes,
• S2.1 Prompt-Injection and Instruction Override: Malicious text manipulates the agent’s reasoning chain or bypasses predefined trading rules through context hijacking or instruction override. • S2.2 Tool-Use and API Exploitation: Compromised reasoning leads to unauthorized execution of tools or APIs, enabling unintended trades, fund movements, or sensitive data exfiltration. • S2.3 Memory or Context Manipulation: The attacker injects persistent malicious content into memory, causing long-lived bias or harmful strategy drift that survives resets or retraining cycles. This metric evaluates whether the agent enforces prompt isolation, input sanitization, tool-use sandboxing, API permission boundaries, and memory-integrity protections, critical safeguards required to prevent direct logic-layer compromise. S3: Agent-as-Attacker Behavior. Threat Model: We assume that one or more financial LLM agents have already been compromised through S1 or S2 and continue to operate in a multi-agent or market-simulation environment where other agents trust their outputs. The attacker fully controls these compromised agents and can use them 5
to autonomously generate misinformation, propagate malicious prompts, or execute manipulative trades. The attacker aims to exploit the compromised agent’s trusted role to influence broader market dynamics and induce cascading or destabilizing effects. Attack Flow / Objective: A compromised agent behaves as an active adversary rather than a passive victim. It may disseminate false signals, coordinate harmful trades with other controlled agents, or manipulate liquidity or volatility to trigger cascading reactions. Because financial agents often treat peer outputs as actionable signals, a single compromised agent can rapidly propagate harmful behavior across the ecosystem. The attacker’s objective is to amplify systemic volatility, trigger coordinated sell-offs or flash-crash-like events, or exploit arbitrage gaps created by the disruption. This metric evaluates whether the system can detect malicious coordination, isolate compromised agents, and contain cascading impacts across interconnected agent networks, critical safeguards for preventing ecosystem-wide failures.
Screening. Two reviewers independently screened all 127 papers by title and abstract, applying the following inclusion criteria: (I1) the paper must propose, implement, or empirically evaluate a concrete financial LLM agent system; (I2) the agent must perform market-facing tasks (e.g., trading, portfolio optimization, or alpha discovery); (I3) the paper must be written in English and published or posted as a preprint between January 2023 and March 2026. Papers were excluded if they (E1) only mentioned LLMs tangentially without an agent implementation, (E2) focused exclusively on general LLM security without a financial application, or (E3) were surveys, editorials, or position papers with no system contribution. After screening, 87 papers were excluded, leaving 40 papers for full-text review. Eligibility and Inclusion. Both reviewers independently read all 40 full-text papers and assessed them against two additional eligibility criteria: (G1) the agent architecture must be described in sufficient detail to apply our FARSIGHT metrics (i.e., information sources, reasoning pipeline, and output actions are identifiable); and (G2) the system must include at least one form of autonomous or semi-autonomous decisionmaking rather than being purely a static prediction model. Of the 40 papers, 25 did not satisfy G2 (they are surveys, benchmarks, simulations, threat analyses, or defense frameworks with no autonomous decision-making) and are retained for background context in Section 4.2. The remaining 15 papers satisfy both G1 and G2, describe distinct, evaluable trading agent systems, and constitute our final evaluation set (Table 1).
Financial LLM Agent Selection and Categorization
This section presents our approach to identifying, collecting, and categorizing financial LLM agents. We describe the paper selection criteria and dataset construction, then propose a taxonomy that classifies agents by architecture, functionality, and evaluation purpose, highlighting the traits that distinguish financial LLM agents from other domain-specific systems.
Paper Collection and Selection Criteria
G
#Financial Agents
10
In Section 4.2, we categorize these studies by their agent architecture, system design, and evaluation methodology to illustrate how the research community conceptualizes and analyzes financial LLM agents.
This study focuses specifically on understanding and systematizing attacks targeting financial LLM agents, including financial LLM trading agents and finance-oriented LLM agents that employ large language models for market analysis, autonomous trading, or financial decision-making. We followed a structured selection process inspired by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [66]. Identification. We systematically searched for papers published between 2023 and 2026 using keyword queries including “financial LLM agents,” “LLM trading agents,” “LLM autonomous trading,” and “LLM financial decision-making.” Our search covered publications from the USENIX Security Symposium, the Network and Distributed System Security Symposium (NDSS), the ACM Conference on Computer and Communications Security (CCS), the IEEE Symposium on Security and Privacy (IEEE S&P), the Conference on Neural Information Processing Systems (NeurIPS), and the International Conference on Machine Learning (ICML), as well as relevant preprints on arXiv. We also performed backward and forward citation snowballing on highly relevant papers to cap-
5
4.1
PT 0 G 4 PT G PT -4 o -4 om i G ni PT -3 .5 G D P ee pS T-2 ee kV 3
4
ture additional works not returned by keyword search. This identification phase yielded 127 candidate papers.
Backbone LLM Model Figure 3: Distribution of backbone LLM models used across 15 financial LLM agents.
6
Table 1: Summary of LLM-based Financial Agents: Role, I/O, Mechanisms, and Evaluation Settings System / Paper T RADING GPT [58] F IN M EM [49] M ARKET S ENSE AI [59] F IN C ON [15] S TOCK AGENT [44] F INAGENT [60] F IN ROBOT [61] A LPHA -GPT 2.0 [50] Q UANTAGENT [46] T RAD E XPERT [62] T RADING AGENTS [16] FLAG-T RADER [63] C ONTEST T RADE [48] T RADING G ROUP [64] M ARKET S ENSE AI 2.0 [65]
Role
Ü
Typical Inputs
Outputs
Mechanism
Code
\ Î ÿ M \ Î M \ Î ÿ M
V @ V @ V @
: Z ¥ : ¥ Z 8
× ✓ ×
\ Î ÿ M ⌣ \ Î ÿ M \ Î M ¹ \ Î ÿ M ⌣ \ Î ÿ M Î \ Î M \ Î M ⌣ \ Î \ Î ÿ \ Î M ⌣ \ Î ÿ M
V @ V @ V @ V @ @ V @ V @ V @ V @ V @ V @ V @
: Z Z : : Z 8 ¹ Z Z Z Z Z 8
✓ ✓ × ✓ × ✓ × ✓ × ✓ × ×
Notes: \= News; Î= Stock price history; ÿ= Macro-market data; M= Financial statements/filings; ⌣= Market sentiment; ¹= Expert Opinions; V= Buy/Sell/Hold decision; @= Summary / Rationale; = Trader role; Ü= Alpha Miner role; := Uses memory module; Z= Multi-agent collaboration; ¥= Multiple profile/character settings; 8= RAG; = Chain-of-Thought.
4.2 Trading Agent Architectures and Ecosystem Context
Table 2: Reported jailbreak attack success rates (ASR) for major backbone LLMs used in financial LLM agents.
From the 40 papers retained during screening (Section 4), 25 are ecosystem studies that inform but are not themselves evaluable trading agents: surveys and SoK papers synthesize financial LLM tasks, inputs, and recurring designs [2,3,5,42,43,67–69]; benchmarks and evaluation suites probe agent-level attacks, financial metrics, and long-run robustness [17,70–73]; simulation testbeds provide controllable market environments [74–76]; and threat/defense analyses address domain-specific red teaming and architecture-level safeguards [77,78]. While these works shape the broader landscape, our evaluation targets the remaining 15 trading agent systems, agents that autonomously transform market data into executable trading decisions. We define trading agents as financial LLM agents that combine heterogeneous market inputs with LLM reasoning to produce trades and naturallanguage rationales. As detailed in Table 1, we organize them into six architectural paradigms. Single-Agent Memory & Persona Systems. TradingGPT [58] uses layered memory and task decomposition across cooperative roles for multi-horizon trading via GPT-4. FinMem [49] refines this with character design and adjustable cognitive span; its self-evolution mechanism enables adaptive risk profiling but also creates persistent attack surfaces. Analyst & Fundamental-Analysis Agents. Market-
Backbone LLM
Benchmark Name
GPT-4 GPT-4o GPT-4o-mini GPT-3.5-Turbo DeepSeek-V3
Do Anything Now (DAN) [25] HarmNet [79] GRAF [80] Do Anything Now (DAN) [25] CNSafe-RT [81]
Jailbreak ASR 95% 94.8% 95.0% 95% up to 100%
SenseAI [59] (and MarketSenseAI 2.0 [65]) applies chain-of-thought fundamental analysis over U.S. Securities and Exchange Commission (SEC) filings with minimal live API exposure. FinCon [15] formalizes a manager–analyst hierarchy with feedback-driven risk control. Event-Driven & Multimodal Agents. StockAgent [44] uses event-driven simulation with curated inputs. FinAgent [60] targets cross-asset generality via tool-augmented, multimodal reasoning with broad API integration. Platform & Alpha-Discovery Agents. FinRobot [61] provides a composable multi-LLM platform with modular design. Alpha-GPT 2.0 [50] focuses on alpha discovery with humanin-the-loop oversight and no live execution, one of the most secure designs. Low-Latency & Expert-Fusion Agents. QuantAgent [46] integrates classical technical features with a dedicated risk-monitoring agent for HFT-like settings, achieving the 7
strongest robustness profile. TradExpert [62] uses a mixtureof-experts architecture fusing specialized LLM outputs. Multi-Agent Collaborative Systems. TradingAgents [16] is the most comprehensive system (24.6k+ GitHub stars), scaling the firm metaphor with analyst/researcher/risk teams and debate–synthesis, but introduces >240 s decision latency. FLAG-Trader [63] blends LLM deliberation with reinforcement learning. ContestTrade [48] uses internal competition to mitigate overfitting. TradingGroup [64] probes group-scale collaboration with configurable risk parameters. Cross-Cutting Observations. Three convergences emerge across Table 1: (i) most systems ingest news, fundamentals, and price data to output trades with natural-language rationales; (ii) modular multi-agent structures and reflective memory modules dominate; (iii) RAG over SEC filings increasingly shapes model reasoning. To understand what makes financial LLM agents fundamentally different, we first examined how agentic LLM systems are structured across other high-stakes domains such as law, healthcare, and education. Prior studies show that legal agents emphasize traceability and precedent grounding [82, 83], healthcare agents prioritize safety, hallucination control, and clinical validation [84, 85], and educational agents focus on pedagogical reliability and personalized instruction [86]. Across these domains, risks usually stem from instruction-following errors, hallucinated reasoning, or misalignment with domain constraints. But none involve realtime market microstructure or the reflexive, latency-sensitive execution loops that trading agents operate in. By contrasting these domain patterns with our analysis of 15 major financial LLM trading systems, we identified a cluster of properties unique to financial agents, shaped by market speed, heterogeneous signal dependence, execution autonomy, and systemic reflexivity. As illustrated in Figure 4 and detailed in Appendix D, financial LLM agents exhibit five defining characteristics: (i) Instant millisecond-level reactions exploiting transient price dislocations; (ii) Inconspicuous micromanipulations that resemble legitimate market signals; (iii) Unverified ingestion of external data due to speed-first execution pipelines; (iv) Adaptive multi-source exploitation enabled by heterogeneous signal integration; and (v) Cascading amplification, where small perturbations propagate system-wide through inter-agent reflexivity. These properties combine into an attack surface that is fragile and easily amplified, and that differs fundamentally from other domain-specific agent systems. We also analyze the backbone LLMs underlying current financial agents to understand how their foundational vulnerabilities propagate into trading environments. As shown in Figure 3, the majority of financial LLM agents rely on the GPT-4 family (GPT-4, GPT-4o, and GPT-4o-mini), which serves as the backbone of twelve of the fifteen analyzed systems, with plain GPT-4 alone powering ten of them. For multi-model implementations, we consistently selected the most capable and
Figure 4: Distinct attack characteristics of financial LLM agents compared to other domain-specific agent systems.
up-to-date model used within each system. A few older backbones, such as GPT-3.5 and GPT-2, still appear in certain academic schemes, indicating limited model modernization. Two systems use open-weight backbones (GPT-2 and DeepSeekV3); the remaining thirteen depend on closed-source commercial APIs. As summarized in Table 2, jailbreak attack success rates (ASR) remain high across modern LLMs: both GPT4 and GPT-4o exceed 94% ASR on standard benchmarks, while GPT-4o-mini still exhibits a 95.0% ASR under adaptive multi-turn jailbreaking. Open models like DeepSeek-V3 reach jailbreak ASRs of up to 100% [81]. Because GPT-2 is an outdated model, it is not included in recent benchmark evaluations; however, its lack of alignment and defensive tuning would likely result in near-complete susceptibility. Overall, these findings indicate that despite rapid progress in capability, current backbone models remain highly exploitable under jailbreak attacks. We note that the ASR figures in Table 2 reflect base-model susceptibility under generic jailbreak benchmarks, not agentspecific exploitation rates. In deployed financial agents, system prompts, tool constraints, output filters, and multi-agent validation may reduce the effective ASR. We therefore treat these figures as upper-bound risk indicators rather than direct measures of financial-agent vulnerability. The gap between base-model ASR and actual agent-level exploitability remains an open research question.
5
Evaluation and Findings
This section presents the findings of our evaluation using the FARSIGHT framework from Section 3. We focus on two aspects: the robustness of financial LLM agents under market shocks and their security against diverse adversarial attacks. 8
Table 3: Robustness and security evaluation of financial LLM agents under the FARSIGHT framework. The table summarizes robustness and security levels across nine metrics: R1 (Crash Foresight and Early Awareness), R2 (Rapid Reaction Capability under Market Shocks), R3 (Loss Containment and Stop-Loss Execution), R4 (Recovery and Post-Crash Adaptation), S1 (Attacks on Market-Relevant Information Sources), S2.1 (Prompt Injection & Instruction Override), S2.2 (ToolUse & API Exploitation), S2.3 (Memory or Context Manipulation), and S3 (Agent-as-Attacker Behavior). Agent System
R1 R2 R3 R4 S1 S2.1 S2.2 S2.3 S3
TradingGPT FinMem MarketSenseAI FinCon StockAgent FinAgent FinRobot Alpha-GPT 2.0 QuantAgent TradExpert TradingAgents FLAG-Trader ContestTrade TradingGroup MarketSenseAI 2.0
❍ ❍ ❍ ❍ ◆ ❍ ◆ ◆ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ❍ ◆ ❍ ❍ ❍ ❍ ◆ ❍ ◆ ◆ ◆ ◆ ◆ ◆ ❍ ❍ ❍ ❍ ◆ ◆ ◆ ◆ ❍ ❍ ❍ ❍ ❍ ❍ ◆ ◆ ◆ ◆ ◆ ◆ ❍ ❍ ❍ ❍
❍ ❍ ❍ ❍ ❍ ◆ ❍ ◆ ◆ ◆ ❍ ◆ ◆ ◆ ◆ ● ◆ ◆ ❍ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ❍ ◆
◆ ◆ ● ◆ ◆ ◆ ◆ ● ◆ ◆ ◆ ◆ ◆ ◆ ●
❍ ❍ ● ◆ ◆ ◆ ◆ ◆ ◆ ● ◆ ◆ ◆ ❍ ●
live trading. QuantAgent includes a risk monitoring agent but lacks concrete metrics (e.g., VIX tracking, correlation dislocation). TradingAgents and TradingGroup assess market regime direction via multi-source sentiment, representing high-level awareness rather than quantitative foresight. None implement automated anomaly-detection pipelines comparable to traditional quantitative systems (e.g., VIX-based alerts, entropy divergence, principal component analysis (PCA)-based volatility clustering), revealing a consistent bias toward qualitative interpretation rather than measurable early warning. Analysis of Rapid Reaction Capability. Only QuantAgent, TradingAgents, and TradingGroup partially address rapid reaction. QuantAgent can trigger alerts upon abnormal volatility but is bottlenecked by multi-agent inference latency. TradingAgents includes a risk management team that can override risky behavior, yet acts through message-passing and debate rather than direct execution. TradingGroup offers configurable stop-loss and take-profit thresholds but lacks autonomous shock detection or circuit-breaking logic. All systems prioritize analytical deliberation over reflexive action; none implement fast-path interrupt mechanisms, safe-mode pipelines, or hardware-level latency reduction. As summarized in Tables 4 and 5, all 15 systems exhibit LLM inference delay and I/O latency, while 10 add multiagent communication overhead. The most widely used framework, TradingAgents, introduces over four minutes of total decision latency, orders of magnitude slower than algorithmic or high-frequency trading requirements. These latency gaps translate into critical vulnerabilities: (1) price staleness, where agents act on outdated information; (2) stop-loss failures, where delayed risk-control activation leads to uncontrolled drawdowns; and (3) slippage and execution drift, where I/O and API delays cause execution prices to deviate from targets.
❍ ❍ ◆ ◆ ❍ ◆ ● ● ❍ ◆ ❍ ◆ ◆ ❍ ◆
● (Fully Robust / Low Risk) ◆ (Partially Robust / Medium Risk) ❍ (Not Robust / High Risk) Per-agent justifications for all partial (◆) and fully robust / low-risk (●) scores are detailed in Appendix C. Scoring criteria follow the codebook in Appendix B (Table 8).
Table 4: Occurrence of major latency sources across 15 financial LLM trading agents.
5.1 Findings on the Robustness of Financial LLM Agents Table 3 summarizes the robustness evaluation of fifteen financial LLM-based trading systems under the FARSIGHT framework across four resilience dimensions (R1–R4). Overall, robustness remains weak: most agents show no built-in protection against market turbulence, and those that demonstrate partial robustness do so through ad-hoc or conceptual mechanisms without complete automation, verification, or closed-loop control. Analysis of Crash Foresight and Early Awareness. Five agents earn partial R1 scores, but all rely on qualitative or reactive mechanisms rather than structured quantitative foresight. FinMem adjusts trading aggressiveness based on realized drawdowns, purely reactive rather than predictive. AlphaGPT 2.0 uses offline human-in-the-loop oversight during research cycles, with no continuous anomaly surveillance in
Latency Source
Count
LLM Inference Delay Agent Communication Overhead I/O & API Latency
15/15 10/15 15/15
Analysis of Loss Containment and Stop-Loss Execution. Six agents earn partial R3 scores, but none formalize containment into deterministic enforcement. FinMem adapts risk modes after performance declines yet specifies no drawdown thresholds or liquidation triggers. Alpha-GPT 2.0 and QuantAgent acknowledge risk management roles but lack algorithmic stop-loss execution. TradingAgents achieves smaller drawdowns empirically through its risk management team, but without transparent liquidation logic. ContestTrade downranks poorly performing agents only after losses materialize, and TradingGroup offers configurable stop-loss settings 9
Table 5: Latency comparison between traditional trading styles and representative LLM trading agents. Trading Type/Agent
Typical tency
La-
Src.
High-Frequency Trading (HFT) Algorithmic Trading Intraday Trading Daily / Swing Trading TradingAgents
<0.0001 s [87, 88] <0.1s [89, 90] ∼1 s [89] ∼10 s [89] > 240 s
[R]
SIGHT framework. Overall, almost all systems reveal high exposure to misinformation, prompt manipulation, and compromise propagation. None exhibit end-to-end validation, isolation, or post-compromise containment, underscoring a systematic neglect of proactive security design in current financial LLM ecosystems. Analysis of Attacks on Market-Relevant Information Sources. Trading agents that continuously consume external market data or sentiment feeds are especially exposed to datalayer manipulation. TradingGPT, FinMem, MarketSenseAI, MarketSenseAI 2.0, FinCon, FinAgent, and TradExpert sit in the highest-risk tier: they aggressively ingest news, social sentiment, macroeconomic indicators, and fundamental documents from heterogeneous APIs and RAG pipelines, yet perform little explicit trust assessment or cross-source verification. Although many of these systems deliberately diversify inputs and stage multi-round debates to avoid dependence on a single feed, such mechanisms primarily filter noise and disagreement rather than targeted misinformation; a poisoned but plausible source can still be amplified into trading theses and written back into memory. The remaining agents, including multi-agent frameworks such as TradingAgents, TradingGroup, FinRobot, QuantAgent, FLAG-Trader, and ContestTrade, fall into a medium-risk tier: they also rely on diverse external streams, but combine them with stronger internal aggregation (for example, voting, ranking, or policy learning), which mitigates but does not eliminate the impact of corrupted feeds. Even more curated or closed-loop designs such as Alpha-GPT 2.0 and StockAgent remain vulnerable at the data layer, since simulated or human-filtered inputs reduce exposure but cannot fully defend against upstream manipulation of reference data. Overall, current financial LLM agents still treat information sources as essentially benign; without explicit provenance checks, crosssource consistency tests, or adversarial data screening, the data layer remains the primary attack surface. Analysis of Attacks on Agent Logic and Prompts. Promptlayer and logic-layer threats are pervasive but often underaddressed. In S2.1 (Prompt-Injection and Instruction Override), we rate TradingGPT and FinMem as high risk: both expose natural-language coordination channels that accept unverified inputs, enabling adversaries to override instructions or seed malicious reasoning. By contrast, Alpha-GPT 2.0 (an alpha-focused, idea-generation design) is low risk in S2.1 because it operates on curated inputs rather than open prompt channels; StockAgent is medium risk, since its simulated inputs reduce but do not eliminate exposure to promptlayer manipulation. In S2.2 (Tool-Use and API Exploitation), MarketSenseAI, MarketSenseAI 2.0, and Alpha-GPT 2.0 are assessed as low risk because their pipelines emphasize static document retrieval and analysis rather than live API-driven tool invocation, reducing the attack surface for real-time API spoofing or tool misuse. Systems that do invoke external tools remain more exposed, since injected prompts can escalate
[R] [R] [R] [E]
[R] = Reported in cited literature; [E] = Estimated from architecture description (multi-agent deliberation loop with iterative LLM inference, debate, and voting rounds). Specifically, the >240 s figure for TradingAgents is the median wall-clock latency we observed over five end-to-end runs of its open-source release under its fastest configuration (smallest supported backbone and shortest chain-of-thought), and therefore represents a lower bound. See Section 4.2 for architectural details.
without dynamic volatility-adjusted calibration. Across all systems, containment remains discretionary: none implement rule-based position limits, volatility-linked exposure scaling, or forced liquidation protocols, showing that current LLM agents treat risk control as a concept but rarely enforce it as a hard limit. Analysis of Recovery and Post-Crash Adaptation. R4 shows the most partial scores, with seven agents displaying forms of adaptive learning or reflection, yet all rely on continuous learning rather than crash-triggered recovery. FinMem updates memory representations over time but lacks event-triggered retraining to distinguish normal drift from tail events. FinAgent and Alpha-GPT 2.0 enable post-hoc behavioral adjustments (via self-reflection and human guidance, respectively) but provide no automated crash-recovery pipeline. QuantAgent’s two-loop optimization gradually integrates outcomes yet cannot accelerate recalibration after extreme losses; TradingAgents shares this limitation. ContestTrade promotes surviving agents via competition, but this selection process fails under systemic crashes that degrade all agents simultaneously. TradingGroup synthesizes data for continuous model updates, but recovery remains passive rather than actively triggered. None implement structured fault analysis, regime reclassification, or volatility-dependent retraining triggers. The key point is that these agents adapt by gradually accumulating experience rather than by responding to a specific crash: they keep learning continuously but do not recover in a targeted way.
5.2 Findings on the Security of Financial LLM Agents Table 3 summarizes the comparative security risk levels of fifteen financial LLM-based trading systems under the FAR10
into unauthorized API actions. In S2.3 (Memory or Context Manipulation), we identify TradingGPT, FinMem, and TradingGroup as the high risk group: their persistent memory and self-updating pipelines mean injected content can be stored, reinforced, and replayed across future decision cycles. Conversely, MarketSenseAI, MarketSenseAI 2.0, and TradExpert are low risk for memory attacks because they lack long-term, self-evolving memory layers. Overall, these assignments show that while some architectures reduce live API or memory exposure by design, the most dangerous failures arise when prompt, tool, and memory vulnerabilities interact, turning flexible agent capabilities into persistent attack vectors unless provenance, isolation, and sanitization controls are enforced. Analysis of Agent-as-Attacker Behavior. Once compromised, agents that carry order-placement authority pose the greatest threat. In S3 (Agent-as-Attacker), we identify TradingGPT, FinMem, StockAgent, QuantAgent, TradingAgents, and TradingGroup as high risk. Five interact with live markets through execution APIs or automated pipelines; StockAgent instead trades inside a multi-agent simulation, but through a shared order book with no isolation, so a compromised instance can still steer peer agents and downstream artifacts trained on that market. Once the internal logic or prompt layer of any of these agents is compromised, an attacker could issue real trades, manipulate orders, or coordinate malicious strategies across multiple sub-agents, amplifying both velocity and market impact. For instance, a poisoned TradingAgents cluster could synchronize trades to induce volatility, while a compromised TradingGroup could broadcast falsified intelligence through its collaborative communication channels. Agents such as FinRobot and Alpha-GPT 2.0 are comparatively low risk, as their design intentionally excludes any live execution or external trading connection; they operate strictly as analytical or simulation frameworks. The remaining agents, which can interface with market APIs but are not inherently autonomous, fall into a medium risk tier: their exposure depends on whether external tools or execution endpoints are activated at runtime. Overall, this category illustrates that financial autonomy amplifies security consequences: as agents transition from analysis to execution, their compromise shifts from a passive analytical error to an active, market-moving threat. Without authentication of trade intent, anomaly detection, and coordination containment, a compromised financial LLM agent can quickly shift from a victim (S1–S2) to an active source of market disruption (S3).
pendix F for additional details) to a 5-agent sub-sample (FinMem, TradingAgents, FinCon, QuantAgent, FLAG-Trader), we found that the model disagreed with human expert evaluations on 24 of 45 metric cells (a 53.3% disagreement rate, i.e., 46.7% agreement), while still identifying similar key financial LLM agents and repeatedly flagging the same recurring weaknesses. While its reasoning depth remains more limited than that of human experts, the approach offers high scalability, enabling rapid and consistent framework-guided evaluation as the literature continues to expand. A key insight from our analysis is that LLMs perform highly accurate evaluations for attacks that are already known, producing consistent and correct metric assignments. However, for newly proposed attack types introduced in this work, the model’s evaluation quality degrades significantly. In several cases, its metric predictions were inaccurate, with error rates reaching up to 100%, underscoring the limitations of relying solely on black-box LLM assessment for emerging or unseen threat classes.
5.4
Summary of Key Insights
Insights on Robustness. As summarized in Table 3, none of the evaluated agents achieved full robustness. Despite impressive analytical capabilities, this sophistication does not translate into resilience: most agents fail to anticipate market extremes, react with high latency, and lack automated recovery. Partial robustness increases slightly from R1 to R4, indicating limited attention to adaptation but a persistent absence of proactive mechanisms such as volatility-aware controls or circuit-breaker logic. Without explicit robustness design, financial LLM agents risk both failing to prevent catastrophic losses and actively accelerating them through feedback-driven trading loops. Insights on Security. The proportion of high-risk ratings decreases slightly from S1 to S3, but this apparent improvement is misleading: pervasive attack vectors (prompt injection, data poisoning, tool misuse) remain unaddressed across all systems. Financial LLM agents possess both decision authority and execution capability: once compromised, they become active threat actors capable of orchestrating market manipulation. Securing these agents requires defense-in-depth: authenticated data sources, sandboxed reasoning, restricted tool privileges, and multi-layer anomaly detection. Without these safeguards, automation only makes errors spread faster and at larger scale. Best Schemes. As summarized in Table 6, QuantAgent shows the strongest robustness via built-in risk monitoring and selfimprovement mechanisms. Alpha-GPT 2.0 achieves the highest security through human oversight and restricted API exposure, while TradingAgents offers the most balanced design, combining multi-agent coordination, risk control, and broad adoption (24.6k+ GitHub stars). However, these systems still have notable tradeoffs: Alpha-GPT 2.0 relies heavily on man-
5.3 Alignment Between LLM-based and Human Expert Evaluations We evaluated the utility of LLM-based analysis under the FARSIGHT framework to assess how effectively it mirrors human expert judgment. Applying ChatGPT Deep Research with predefined metrics, definitions, and scope (see Ap11
Table 6: Best-performing financial LLM trading agents by evaluation dimension. Dimension
Best Scheme
Robustness Security Overall Best
QuantAgent Alpha-GPT 2.0 TradingAgents
gap > 3σ in ≤ 1 minute, spread blow-up, or volatility regime jump), the agent must not launch a new reasoning round; it freezes new orders, cancels non-essential resting orders, and switches to capital-preservation mode via a rules engine that does not call the LLM. In multi-agent settings the risk agent outranks analyst/trader agents during shocks, and reactions should be hysteresis-based to resist transient spikes. R3: Loss Containment and Stop-Loss Execution. Agents consider risk but rarely enforce it: FinMem and Alpha-GPT 2.0 describe risk modes without algorithmic stops, ContestTrade downranks losers only after damage, and TradingGroup’s stops lack volatility-aware calibration. This recommendation targets R3, cross-covers S2.1 (risk invariants live outside the LLM) and S3 (hard caps bound compromisedagent damage). Designs should hard-code per-instrument, per-sector, and per-day loss limits that LLM output cannot override, with stop-losses evaluated on broker/exchange data rather than LLM-parsed feeds to remain intact under data poisoning [93, 94]. Containment must be multi-layered (positionlevel stop, strategy-level halt, portfolio-level circuit breaker) and volatility-aware (wider stops with smaller sizes in highvol regimes). Contest-style agents should feed stop-triggered events back as negative evidence. R4: Recovery and Post-Crash Adaptation. No surveyed agent triggers event-gated recovery: FinMem updates memory continuously, QuantAgent’s two-loop optimization cannot accelerate after tail events, and TradingGroup’s recovery is passive. This recommendation targets R4, reinforces S1 via cleandata replay and S2.3 via quarantine memory. Crisis economics consistently shows that liquidity spirals demand explicit postevent rebalancing rather than incremental drift [56, 95, 96]. A recovery pipeline should freeze trading and snapshot model/memory state, run a post-mortem against historical stress scenarios, downgrade aggressiveness (leverage, confidence thresholds, confirmation requirements), and replay the day with clean data to identify corrupted signals. Memory agents should route uncertain experiences to a quarantine memory revalidated before promotion, and no long-lived instruction should be accepted from a single event. S1: Attacks on Market-Relevant Information Sources. Every agent relying on RAG feeds or social sentiment (e.g., FinMem, FinCon, MarketSenseAI, MarketSenseAI 2.0) receives a high-risk S1 rating because none enforces provenance or diversity. Real incidents [91] and indirect prompt injection [23] show a single poisoned source is sufficient to coerce marketmoving actions. This recommendation targets S1 and reinforces R1 and R4 by ensuring both foresight and post-mortem replay run on verified inputs. Agents should tag every inbound item with provenance (domain, timestamp, authentication), require cross-source majority or diversity before acting, prioritize structured official feeds over unstructured social sources, maintain a denylist of easily spoofed channels, and surface single-source anomalies to the operator rather than silently trading on them.
ual verification, limiting scalability; both TradingAgents and QuantAgent, though adaptive, remain vulnerable to prompt and data-layer attacks due to weak isolation. Overall, the results highlight that greater agent autonomy expands the attack surface, while strict security constraints reduce autonomy. Further comparisons with passive-income schemes and active trading agents are provided in Appendix E.
6
Recommendations for Secure and Robust Financial LLM Agent Design
Building on Table 3, we give concise design recommendations, each tied to a specific failure pattern observed across the 15 surveyed agents and mapped to the robustness (R1–R4) and security (S1–S3) metrics of FARSIGHT. R1: Crash Foresight and Early Awareness. No agent in Table 3 receives a Full rating on R1: FinMem reacts only to realized drawdowns, while TradingAgents/TradingGroup infer regimes from sentiment rather than microstructure. This recommendation primarily targets R1 and additionally strengthens S1 via cross-source verification. History motivates the baseline: the 2013 AP Twitter hack erased $136 B in minutes because automated systems acted on a single unverified headline [91], and the 2010 Flash Crash was preceded by liquidity and toxic-flow signals no surveyed agent monitors [19, 20]. Agents should therefore source-rank and cross-check every market-moving input, downweighting any single-source headline until an independent high-reputation feed confirms it, and maintain a library of quantitative precursors that fire triggers independently of LLM reasoning: VIX, equity–credit correlation breakdowns, order-book flow toxicity [92], and exchange-traded fund (ETF)–underlying decoupling. Inputs arriving in thin-liquidity windows (pre-market, after-hours) warrant stricter confirmation. R2: Rapid Reaction Capability under Shocks. Table 3 shows zero Full ratings on R2, and Table 5 measures TradingAgents at over four minutes of decision latency, orders of magnitude slower than the circuit breakers instituted after the 2010 Flash Crash [19, 20]. This recommendation targets R2 and cross-covers S2.2 (by constraining which tools run during shock states) and S3 (by giving a supervisor override authority). Trading stacks should separate a thinking path (LLM reasoning, multi-agent debate) from a fast path (predefined, latency-bounded safety actions). On shock detection (price 12
countering the retrieval-poisoning and backdoor-implantation attacks documented in [27, 100]. These three layers together move the agent away from relying on model alignment alone and toward defense-in-depth, matching the assurance level that non-LLM trading infrastructure already provides outside the model. S3: Agent-as-Attacker Containment. Because financial LLM agents act in the real market, a compromised instance becomes an adversarial participant, the dynamic behind AIamplification concerns raised by regulators [4, 10, 19, 102]. In Table 3, six agents receive a high-risk S3 rating. This recommendation targets S3 and cross-covers R2 (supervisor override in shocks) and R3 (per-agent notional caps). Systems should assume a subset of agents will be compromised, and provide behavioral anomaly detection (sudden order-frequency, asset-class, or liquidity shifts), single-agent isolation without whole-system shutdown, and coordination guards so that multiple agents cannot place correlated large trades without risk-agent approval. Inter-agent messages are untrusted inputs, mediated by a policy engine. Finally, a human hard stop must always override agent behavior, so that operators can revoke the agent’s trading authority at any time.
S2: Prompt/Logic, Tool/API, and Memory/Context Attacks. A rapidly expanding body of work at top security venues establishes that LLM-level alignment cannot be the primary defense for autonomous trading. Jailbreak techniques bypass safety training through competing objectives and mismatched generalization [97]. They also spread through community-curated prompt repositories that stay effective across model generations [25], and they include transferable adversarial suffixes that compromise closed-source models with no white-box access [98]. Direct prompt injection via “ignore previous instructions”-style payloads [99] has been formalized and benchmarked at USENIX Security [24], while indirect prompt injection demonstrates that passively ingested third-party content alone is sufficient to coerce a fully aligned model [23]. Agent-specific studies sharpen the threat model for trading: AgentDojo [52] and ASB [70] quantify how tool-calling agents succumb to indirect injection; Yang et al. demonstrate backdoor implantation in LLM-based agents with high attack success and minimal trigger footprint [100]; AgentPoison weaponizes retrieval and long-term memory stores [27]; poisoning web-scale training corpora is shown to be both practical and inexpensive at IEEE S&P [26]; training-data extraction leaks proprietary signals embedded during fine-tuning [101]; and ToolEmu exposes cascading tool-misuse risks that alignment-only controls cannot prevent [28]. Because every one of these attack classes has been reproduced outside the laboratory and because financial agents simultaneously expose all three of the affected surfaces (natural-language prompts, external tool invocation, and persistent memory), trading invariants must be enforced outside the LLM and treated as security boundaries in the systems-security sense rather than as soft policies the model is asked to follow. This recommendation covers S2.1–S2.3 and reinforces R2 (fast-path safety cannot be stalled by an injected prompt) and R3 (position caps must sit beyond the prompt surface).
7
Discussion
Limitations. While FARSIGHT provides a systematic foundation for assessing the robustness and security of financial LLM agents, several limitations remain. First, our evaluation covers the most representative academic schemes, but not all proprietary financial agents were accessible for testing, which limits the generalizability of our findings. Second, the threat modeling and attack simulations were conducted under controlled, literature-grounded environments rather than live financial trading systems, potentially constraining external validity. Third, our codebook scoring is inherently evidencebased: while it ensures structured and reproducible assessment, it cannot substitute for empirical penetration testing or live adversarial red-teaming against running agent instances. Because our indicators are read from published papers, a ❍ rating means only that the control is not evidenced in the paper: it may under-count mechanisms present in released code or later versions but not written up. Future Work. Trading agents are rarely studied from a security or robustness perspective, yet the attacks are already real: fabricated news and prompts planted in web pages can steer many agents into the same trade and amplify market swings, and attributing such moves to a specific compromised agent is difficult. We see a need for a community-maintained, safetyfirst benchmark that standardizes a minimal core of agenttargeted attacks while remaining extensible to new threat models, and for empirical validation on running agent instances (including trading-API and broker-in-the-loop settings) that can complement the scheme-level findings reported here. Flash Crashes versus Normal Pullbacks. Since volatility
Concretely, three hardening layers must interlock so that the system degrades gracefully under partial compromise. At the logic layer (S2.1), immutable invariants (never exceed 2% of portfolio per new position, never trade restricted symbols, no market orders after hours) are enforced by an external policy engine, and every inbound text passes through a sanitizer that strips tool-invocation phrases, role reassignments, and hidden-channel payloads documented in published injection taxonomies [23, 24]. At the tool layer (S2.2), tool bindings expose fixed schemas, per-agent capability scopes separating analyst-tier read permissions from execution-tier write permissions [28,52], rate limits, and two-signal approval for high-notional or high-urgency orders, mirroring the dualcontrol discipline long used in financial operations. At the memory layer (S2.3), only trusted and market-consistent observations are promoted to long-term memory through explicit write-gates with provenance checks, while suspicious experiences remain in a revalidated short-term store, directly 13
is a normal feature of markets, our robustness metrics target adversarial flash crashes, not ordinary price moves. The two differ in explainability: a normal pullback follows identifiable fundamentals or news, such as an earnings miss or a rate decision, and is interpretable and, in principle, recoverable. An adversarial flash crash is deliberately manufactured, decoupled from fundamentals, and leaves participants unable to identify a cause. This lack of a clear cause is what makes adversarial flash crashes dangerous: an unexplained drop can invite panic selling, stop-loss cascades, and quant-bot liquidation, which feed back into trading agents and amplify the shock. Applicability to Deployed Systems. Our evaluation focuses on academic schemes, raising the question of whether production financial agents share the same vulnerabilities. While proprietary systems (e.g., those deployed by quantitative hedge funds) are not publicly accessible for security auditing, several observations suggest that the risks identified in this study are broadly applicable. First, production systems such as RockAlpha [6], Composer [7], and Nof1.ai [9] employ architecturally similar patterns (LLM-driven signal interpretation, API-based execution, and multi-source data ingestion) that expose the same attack surfaces documented in our S1–S3 analysis. Second, the U.S. SEC’s establishment of a dedicated AI Task Force [102] reflects regulatory recognition that LLMbased trading agents pose systemic risks that merit oversight. Third, the architectural weaknesses we identify (unverified data ingestion, lack of prompt isolation, absence of circuit breakers) are not artifacts of academic simplicity but fundamental design choices driven by the speed-first ethos of financial systems. Production systems operating under the same constraints are unlikely to have solved these problems without explicit security engineering. Practicality and Latency Considerations. A natural concern is whether our recommended defenses introduce unacceptable latency. We validated these defenses on a prototype trading agent, designing them around domain expertise and professional risk-hedging principles such as portfoliolevel risk budgeting, volatility-regime detection, and multileg hedging. Crucially, sub-second latency is not a binding constraint for the dominant use case of LLM-based trading agents: swing trading with multi-day holding periods. Our live-market testing shows that even when an agent processes a market-moving event 10 minutes after publication, sufficient alpha remains for profitable entries. The agent’s edge lies in continuous, around-the-clock information ingestion rather than in raw speed: whereas human traders cannot maintain constant vigilance, an autonomous agent systematically monitors and reasons over information streams without attentional lapses. Well-architected security therefore need not sacrifice practical trading capability, though precisely quantifying the latency–alpha trade-off across strategy horizons remains future work. Trading Agents in the Wild and the Urgency for Security
Considerations. Retail-built LLM trading agents are already operating with real capital. On X, Reddit, and Xiaohongshu (a major Chinese lifestyle and social-media platform), users share live returns yet never disclose architectures; most are built through “vibe coding” (lightly scaffolded, prompt-driven construction using tools such as Claude Code or Cursor) with minimal programming expertise, lowering the deployment barrier while expanding the S1 attack surface. Because free data sources are limited in coverage, most no-code agents converge on a narrow set of feeds, enabling supply-side poisoning: an adversary offering a free, high-quality-looking source could steer many agents simultaneously with zero knowledge of their internals, risking correlated trades and flash-crash amplification [21, 91]. The same broad execution authority that makes autonomous trading attractive also means that, under a poisoned data feed, an agent can propagate aggressive erroneous trades at the same speed as legitimate ones. This duality underscores an urgent need for a robustness and security codebook for trading agents operating under adversarial and stochastic market conditions.
8
Conclusion
Using FARSIGHT, we evaluated 15 representative financial LLM agents. They perform well at analysis and trading but pay little attention to robustness and security: most cannot anticipate market shocks, contain cascading losses, or resist adversarial manipulation. Future systems should treat verified data, strict safety boundaries, and built-in resilience as basic requirements for responsible deployment.
References [1] J. Kim, X. Liu, Z. Wang, S. Qiu, B. Li, W. Guo, and D. Song, “SoK: Attack and Defense Landscape of Agentic AI Systems,” in USENIX Security Symposium, 2026. [2] S. Wang, Y. Zhao, Z. Liu, Q. Zou, and H. Wang, “Demystifying LLM Supply Chain Vulnerabilities in the Wild: Distribution, Root Cause, and Real-World Impact,” Feb. 2025. [3] S. Raza, R. Sapkota, M. Karkee, and C. Emmanouilidis, “TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic MultiAgent Systems,” Jul. 2025. [4] N. Abbas, C. Cohen, D. J. Grolleman, and B. Mosk, “Artificial intelligence can make markets more efficient—and more volatile,” IMF Blog (October 15, 2024), 2024. 14
[5] H. Ding, Y. Li, J. Wang, H. Chen, D. Guo, and Y. Zhang, “Large Language Model Agent in Financial Trading: A Survey,” Jul. 2024.
[15] Y. Yu, Z. Yao, H. Li, Z. Deng, Y. Jiang, Y. Cao, Z. Chen, J. Suchow, Z. Cui, R. Liu et al., “Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making,” Advances in Neural Information Processing Systems, vol. 37, pp. 137 010–137 045, 2024.
[6] RockFlow Technology, “Rockalpha: Ai-powered autonomous trading agent,” https://rockalpha.rockflow.ai, 2025, accessed: 2026-04-01.
[16] Y. Xiao, E. Sun, D. Luo, and W. Wang, “TradingAgents: Multi-Agents LLM Financial Trading Framework,” Jun. 2025.
[7] Composer Technologies Inc., “Composer: Ai-powered automated trading strategies,” https://www.composer.t rade, 2025, accessed: 2026-04-01.
[17] W. W. Li, H. Kim, M. Cucuringu, and T. Ma, “Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?” Aug. 2025.
[8] Robinhood Markets, Inc., “Robinhood is now open to agents,” Robinhood Newsroom, https://robinhood.com/ us/en/newsroom/robinhood-is-now-open-to-agents/, 2026, accessed: 2026-08-24.
[18] Y. Chen, Y. Jiang, Z. Ma, Y. Cao, J. Keung, K. Kuang, L. Gan, F. Wu, J. Zhu, and J. Li, “MM-DREX: Multimodal-Driven Dynamic Routing of LLM Experts for Financial Trading,” Sep. 2025.
[9] Nof1 AI Inc., “Nof1.ai: Ai investment agent,” https: //nof1.ai, 2025, accessed: 2026-04-01. [10] T. C. W. Lin, “Financial weapons of war,” Minnesota Law Review, vol. 100, 2016, available: https://www.minnesotalawreview.org/wpcontent/uploads/2016/04/Lin_ONLINEPDF.pdf.
[19] A. Kirilenko, A. S. Kyle, M. Samadi, and T. Tuzun, “The flash crash: High-frequency trading in an electronic market,” The Journal of Finance, vol. 72, no. 3, pp. 967–998, 2017.
[11] N. Mulder, “The sanctions weapon,” IMF Finance & Development Magazine, June 2022, discusses the scale and spillover effects of modern sanctions and financial statecraft. [Online]. Available: https: //www.imf.org/en/Publications/fandd/issues/2022/06 /the-sanctions-weapon-mulder
[20] U.S. Commodity Futures Trading Commission and U.S. Securities and Exchange Commission, “Findings regarding the market events of may 6, 2010: Report of the staffs of the CFTC and SEC to the joint advisory committee on emerging regulatory issues,” https://ww w.sec.gov/news/studies/2010/marketevents-report.pd f, Sept. 2010.
[12] S. Breeden, “Agents of change,” Speech at the European Central Bank Forum on Central Banking (Sintra), https://www.bankofengland.co.uk/speech/2026/june /sarah-breeden-panel-at-the-european-central-bank-f orum-on-central-banking-2026, Jun. 2026, bank of England; Accessed: 2026-08-24.
[21] IT Brew, “The flash crash of 2010 offers warning as ai automates,” IT Brew, June 2025, accessed: 2026-04-01. [Online]. Available: https://www.itbrew.com/stories/2 025/06/16/the-flash-crash-of-2010-offers-warning-a s-ai-automates
[13] American Banker, “72% of banks lack AI model ‘kill switches,’ failure reporting,” https://www.americanba nker.com/news/72-of-banks-lack-ai-model-kill-swi tches-failure-reporting, 2026, reporting on the Wolters Kluwer U.S. Banking AI Risk and Governance Index, H1 2026 (survey of 230 U.S. banking professionals); Accessed: 2026-08-24.
[22] Y. Xia, P. You, T. Wang, F. Liu, H. Qi, X. Wu, and S. Zhang, “Agentic trading: When LLM agents meet financial markets,” 2026. [Online]. Available: https://arxiv.org/abs/2605.19337 [23] K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection,” in Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec ’23), 2023.
[14] P. Donachie, “Lawmakers press SEC on AI-Driven stock trading,” https://www.wealthmanagement.com/a rtificial-intelligence/lawmakers-press-sec-on-ai-tra ding-agent-oversight, Jun. 2026, wealth Management, 25 June 2026. House Financial Services Democrats (Reps. Foster and Sherman) letter to SEC Chair Atkins on “correlated trading decisions and “herding behavior from AI trading agents on registered brokerages; 13 questions with 31 July 2026 response deadline. Accessed 2026-09-03.
[24] Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and benchmarking prompt injection attacks and defenses,” in Proceedings of the 33rd USENIX Security Symposium, 2024. 15
[25] X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, ““Do anything now”: Characterizing and evaluating inthe-wild jailbreak prompts on large language models,” in Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2024.
[36] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” arXiv preprint arXiv:2312.10997, 2023. [37] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” 2023.
[26] N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramèr, “Poisoning web-scale training datasets is practical,” in Proceedings of the 2024 IEEE Symposium on Security and Privacy (S&P), 2024.
[38] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An open-ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291, 2023.
[27] Z. Chen, Z. Xiang, C. Xiao, D. Song, and B. Li, “AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases,” in Advances in Neural Information Processing Systems (NeurIPS), 2024.
[39] LangChain, “LangChain: Building applications with large language models,” https://github.com/langchain -ai/langchain, 2023. [40] Significant Gravitas, “AutoGPT: An autonomous GPT4 experiment,” https://github.com/Significant-Gravita s/AutoGPT, 2023.
[28] Y. Ruan, H. Dong, A. Wang, S. Pitis, Y. Zhou, J. Ba, Y. Dubois, C. J. Maddison, and T. Hashimoto, “Identifying the risks of LM agents with an LM-emulated sandbox,” in International Conference on Learning Representations (ICLR), 2024.
[41] A. Lopez-Lira, “Can large language models trade? Testing financial theories with LLM agents in market simulations,” arXiv preprint arXiv:2504.10789, 2025.
[29] N. Papernot, P. McDaniel, A. Sinha, and M. P. Wellman, “SoK: Security and privacy in machine learning,” in 2018 IEEE European Symposium on Security and Privacy (EuroS&P), 2018, pp. 399–414.
[42] P. Saha, J. Lyu, A. Saxena, T. Zhao, and D. Mehta, “Large Language Model Agents for Investment Management: Foundations, Benchmarks, and Research Frontiers,” Rochester, NY, Aug. 2025.
[30] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017.
[43] S. Mahdavi, J. Chen, P. K. Joshi, L. H. Guativa, and U. Singh, “Integrating Large Language Models in Financial Investments and Market Analysis: A Survey,” Jun. 2025.
[31] OpenAI, “GPT-4 technical report,” 2023.
[44] C. Zhang, X. Liu, Z. Zhang, M. Jin, L. Li, Z. Wang, W. Hua, D. Shu, S. Zhu, X. Jin, S. Li, M. Du, and Y. Zhang, “When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments,” Sep. 2024.
[32] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023.
[45] T. Guo, H. Shen, J. Huang, Z. Mao, J. Luo, B. Chen, Z. Chen, L. Liu, B. Xia, X. Liu, Y. Ma, and M. Zhang, “MASS: Muli-agent simulation scaling for portfolio construction,” Sep. 2025.
[33] A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch et al., “Mixtral of experts,” arXiv preprint arXiv:2401.04088, 2024.
[46] F. Xiong, X. Zhang, A. Feng, S. Sun, and C. You, “QuantAgent: Price-Driven Multi-Agent LLMs for High-Frequency Trading,” Sep. 2025.
[34] Anthropic, “The Claude 3 model family: Opus, Sonnet, Haiku,” Model Card, https://assets.anthropic.com/m/6 1e7d27f8c8f5919/original/Claude-3-Model-Card.pdf, 2024.
[47] X. Li, Y. Zeng, X. Xing, J. Xu, and X. Xu, “HedgeAgents: A Balanced-aware Multi-agent Financial Trading System,” Feb. 2025.
[35] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27 730– 27 744, 2022.
[48] L. Zhao, R. Sun, Z. Jiang, B. Yang, Y. Bai, M. Chen, J. Li, and Z. Bai, “ContestTrade: A Multi-Agent Trading System Based on Internal Contest Mechanism,” Aug. 2025. 16
[49] Y. Yu, H. Li, Z. Chen, Y. Jiang, Y. Li, D. Zhang, R. Liu, J. W. Suchow, and K. Khashanah, “FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design,” Dec. 2023.
[61] H. Yang, B. Zhang, N. Wang, C. Guo, X. Zhang, L. Lin, J. Wang, T. Zhou, M. Guan, R. Zhang, and C. D. Wang, “FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models,” May 2024.
[50] H. Yuan, S. Wang, and J. Guo, “Alpha-GPT 2.0: Human-in-the-Loop AI for Quantitative Investment,” Feb. 2024.
[62] Q. Ding, H. Shi, J. Guo, and B. Liu, “TradExpert: Revolutionizing trading with mixture of expert LLMs,” arXiv preprint arXiv:2411.00782, 2024.
[51] M. M. R. R. Akanda, A. Lacy, and N. Saxena, “SoK: Inaccessible and insecure: An exposition of authentication challenges faced by blind and visually impaired users in State-of-the-Art academic proposals,” in 34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 1393–1413.
[63] G. Xiong, Z. Deng, K. Wang, Y. Cao, H. Li, Y. Yu, X. Peng, M. Lin, K. E. Smith, X.-Y. Liu, J. Huang, S. Ananiadou, and Q. Xie, “FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading,” Feb. 2025. [64] F. Tian, F. D. Salim, and H. Xue, “TradingGroup: A Multi-Agent Trading System with Self-Reflection and Data-Synthesis,” Aug. 2025.
[52] E. Debenedetti, J. Zhang, M. Balunovic, L. BeurerKellner, M. Fischer, and F. Tramèr, “Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents,” Advances in Neural Information Processing Systems, vol. 37, pp. 82 895– 82 920, 2024.
[65] G. Fatouros, K. Metaxas, J. Soldatos, and M. Karathanassis, “Marketsenseai 2.0: Enhancing stock analysis through llm agents,” arXiv preprint arXiv:2502.00415, 2025.
[53] Y. Chen, Z. Yao, Y. Liu, A. Xin, J. Ye, J. Yu, L. Hou, and J. Li, “Stockbench: Can llm agents trade stocks profitably in real-world markets?” 2025. [Online]. Available: https://arxiv.org/abs/2510.02209
[66] M. J. Page, J. E. McKenzie, P. M. Bossuyt, I. Boutron, T. C. Hoffmann, C. D. Mulrow, L. Shamseer, J. M. Tetzlaff, E. A. Akl, S. E. Brennan, R. Chou, J. Glanville, J. M. Grimshaw, A. Hróbjartsson, M. M. Lalu, T. Li, E. W. Loder, E. Mayo-Wilson, S. McDonald, L. A. McGuinness, L. A. Stewart, J. Thomas, A. C. Tricco, V. A. Welch, P. Whiting, and D. Moher, “The PRISMA 2020 statement: an updated guideline for reporting systematic reviews,” BMJ, vol. 372, p. n71, 2021.
[54] Q. Mao, J. Wang, Y. Liu, L. Zhu, C. Ma, and J. Yan, “SoK: Security of Autonomous LLM Agents in Agentic Commerce,” arXiv preprint arXiv:2604.15367, 2026. [55] E. F. Fama, “Efficient capital markets: A review of theory and empirical work,” The Journal of Finance, vol. 25, no. 2, pp. 383–417, 1970.
[67] J. Liu, “A Survey of Financial AI: Architectures, Advances and Open Challenges,” Nov. 2024.
[56] H. P. Minsky, Stabilizing an Unstable Economy. Yale University Press, 1986.
[68] S. Joshi, “A Comprehensive Review of Gen AI Agents: Applications and Frameworks in Finance, Investments and Risk Domains,” International Journal of Innovative Science and Research Technology, pp. 1339–1355, May 2025.
[57] F. A. Hayek, Prices and Production. Routledge, 1931. [58] Y. Li, Y. Yu, H. Li, Z. Chen, and K. Khashanah, “Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance,” arXiv preprint arXiv:2309.03736, 2023.
[69] D. Batra, C. B. Hamill, J. Hartley, R. Okhrati, D. Seddon, H. Miller, R. Khraishi, and G. A. Cowan, “A Review of LLM Agent Applications in Finance and Banking,” SSRN, 2025.
[59] G. Fatouros, K. Metaxas, J. Soldatos, and D. Kyriazis, “Can large language models beat wall street? Evaluating GPT-4’s impact on financial decision-making with MarketSenseAI,” Neural Computing and Applications, pp. 1–26, 2024.
[70] H. Zhang, J. Huang, K. Mei, Y. Yao, Z. Wang, C. Zhan, H. Wang, and Y. Zhang, “Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents,” May 2025.
[60] W. Zhang, L. Zhao, H. Xia, S. Sun, J. Sun, M. Qin, X. Li, Y. Zhao, Y. Zhao, X. Cai et al., “A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist,” in Proceedings of the 30th Acm Sigkdd Conference on Knowledge Discovery and Data Mining, 2024, pp. 4314–4325.
[71] H. Li, Y. Cao, Y. Yu, S. R. Javaji, Z. Deng, Y. He, Y. Jiang, Z. Zhu, K. Subbalakshmi, G. Xiong, J. Huang, L. Qian, X. Peng, Q. Xie, and J. W. Suchow, “INVESTORBENCH: A Benchmark for Financial 17
Decision-Making Tasks with LLM-based Agent,” Dec. 2024.
[83] L. Zhu, X. Wang, and X. Wang, “Judgelm: Fine-tuned large language models are scalable judges,” arXiv preprint arXiv:2310.17631, 2023.
[72] Z. Chen, J. Chen, J. Chen, and M. Sra, “From Tasks to Teams: A Risk-First Evaluation Framework for MultiAgent LLM Systems in Finance,” in ICML 2025 Workshop on Reliable and Responsible Foundation Models, 2025.
[84] X. Tang, A. Zou, Z. Zhang, Z. Li, Y. Zhao, X. Zhang, A. Cohan, and M. Gerstein, “Medagents: Large language models as collaborators for zero-shot medical reasoning,” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 599–621.
[73] D. Emmanoulopoulos, O. Olby, J. Lyon, and N. R. Stillman, “To Trade or Not to Trade: An Agentic Approach to Estimating Market Risk Improves Trading Decisions,” Jul. 2025.
[85] S. Johri, J. Jeong, B. A. Tran, D. I. Schlessinger, S. Wongvibulsin, L. A. Barnes, H.-Y. Zhou, Z. R. Cai, E. M. Van Allen, D. Kim et al., “An evaluation framework for clinical use of large language models in patient interaction tasks,” Nature medicine, vol. 31, no. 1, pp. 77–86, 2025.
[74] C. Papadakis, G. Filandrianos, A. Dimitriou, M. Lymperaiou, K. Thomas, and G. Stamou, “StockSim: A Dual-Mode Order-Level Simulator for Evaluating Multi-Agent LLMs in Financial Markets,” Jul. 2025. [75] S. Gao, Y. Wen, M. Zhu, J. Wei, Y. Cheng, Q. Zhang, and S. Shang, “Simulating Financial Market via Large Language Model based Agents,” Jun. 2024.
[86] M. Li, Y. Cheng, L. Bai, F. Cao, K. Lv, J. Liang, and P. Lio, “Edullm: Leveraging large language models and framelet-based signed hypergraph neural networks for student performance prediction,” in Forty-second International Conference on Machine Learning, 2025.
[76] Y. Yang, Y. Zhang, M. Wu, K. Zhang, Y. Zhang, H. Yu, Y. Hu, and B. Wang, “TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets,” 2025.
[87] C. C. Moallemi and M. Saglam, “The cost of latency in high-frequency trading,” Working Paper, 2009. [Online]. Available: https://moallemi.com/ciamac/pa pers/latency-2009.pdf [88] J. Hasbrouck and G. Saar, “Low-latency trading,” Journal of Financial Markets, vol. 16, no. 4, pp. 646–679, 2013.
[77] G. Cheng, H. Jin, W. Zhang, H. Wang, and J. Zhuang, “Learning to Conceal Risk: Controllable Multi-turn Red Teaming for LLMs in the Financial Domain,” Sep. 2025.
[89] LuxAlgo. (2025) Latency standards in trading systems. [Online]. Available: https://www.luxalgo.com/blog/lat ency-standards-in-trading-systems/
[78] S. P. Shah and A. V. Deshpande, “Enforcing Web Security Constraints for LLM-driven Robot Agents for Online Transactions,” in 2024 International Conference on Distributed Systems, Computer Networks and Cybersecurity (ICDSCNC), Sep. 2024, pp. 1–6.
[90] CoinAPI. (2025) Crypto trading latency faq: 10 speed questions answered. [Online]. Available: https: //www.coinapi.io/blog/crypto-trading-latency-faq/
[79] S. Narula, J. Rafiei Asl, M. Ghasemigol, E. Blanco, and D. Takabi, “Harmnet: A framework for adaptive multi-turn jailbreak attacks on large language models,” 2025.
[91] B. GAMBINI, “With one false tweet, hack crash led to real panic,” University at Buffalo News, 2015, accessed: 2026-04-01. [Online]. Available: https://www.buffalo .edu/ubnow/stories/2015/05/karppi_hack_crash.html
[80] H. Tang, L. Yan, Y. Zhao, S. Wang, J. Huang, and D. Yin, “GRAF: Multi-turn jailbreaking via global refinement and active fabrication,” 2025.
[92] D. Easley, M. M. López de Prado, and M. O’Hara, “Flow toxicity and liquidity in a high-frequency world,” The Review of Financial Studies, vol. 25, no. 5, pp. 1457–1493, 2012.
[81] Z. Ying, G. Zheng, Y. Huang, D. Zhang, W. Zhang, Q. Zou, A. Liu, X. Liu, and D. Tao, “Towards understanding the safety boundaries of DeepSeek models: Evaluation and findings,” 2025.
[93] H. Markowitz, “Portfolio selection,” The Journal of Finance, vol. 7, no. 1, pp. 77–91, 1952. [94] F. S. Mishkin, The Economics of Money, Banking, and Financial Markets, 12th ed. Pearson, 2018.
[82] Z. Zhou, J.-X. Shi, P.-X. Song, X.-W. Yang, Y.-X. Jin, L.-Z. Guo, and Y.-F. Li, “Lawgpt: A chinese legal knowledge-enhanced large language model,” arXiv preprint arXiv:2406.04614, 2024.
[95] M. K. Brunnermeier, “Deciphering the liquidity and credit crunch 2007–2008,” Journal of Economic Perspectives, vol. 23, no. 1, pp. 77–100, 2009. 18
[96] C. M. Reinhart and K. S. Rogoff, This Time Is Different: Eight Centuries of Financial Folly. Princeton University Press, 2009.
S2.2 (tool/API exploitation), and S2.3 (memory manipulation), each scored independently, yielding nine ratings per agent. All 15 agents were independently scored by two evaluators. Evaluator A is a domain expert with in-depth knowledge of financial LLM agent architectures (including hands-on experience reproducing and extending TradingAgents-class systems). Evaluator B has a solid understanding of LLM security fundamentals and financial markets but is not a specialist in agent-level system design. Both evaluators applied the same codebook (Appendix B) without discussion during the scoring phase. Across all 15 agents × 9 metrics (135 individual ratings), the two evaluators reached identical scores on 126 ratings, yielding a raw agreement rate of 93.3% and a Cohen’s κ of 0.89, indicating near-perfect agreement [103]. The nine disagreements were concentrated in metrics R3, R4, and S2.2, where Evaluator B occasionally rated an agent as Not Robust (❍) while Evaluator A assigned Partial (◆). Root-cause analysis showed that these discrepancies stemmed from Evaluator B’s incomplete familiarity with certain agent-internal mechanisms, for example, implicit risk-adjustment loops in memory-based agents, or tool-permission scoping embedded in multi-agent communication protocols. Such mechanisms are not apparent from surface-level paper descriptions but are identifiable through deeper architectural reading. All disagreements were resolved through joint discussion, and final scores reflect the consensus ratings reported in Table 3.
[97] A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does LLM safety training fail?” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [98] A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” 2023. [99] F. Perez and I. Ribeiro, “Ignore previous prompt: Attack techniques for language models,” 2022. [100] W. Yang, X. Bi, Y. Lin, S. Chen, J. Zhou, and X. Sun, “Watch out for your agents! investigating backdoor threats to LLM-based agents,” in Advances in Neural Information Processing Systems (NeurIPS), 2024. [101] N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, Ú. Erlingsson, A. Oprea, and C. Raffel, “Extracting training data from large language models,” in Proceedings of the 30th USENIX Security Symposium, 2021, pp. 2633–2650. [102] U. Securities and E. Commission, “Sec creates task force to tap artificial intelligence for enhanced innovation and efficiency across the agency,” https: //www.sec.gov/newsroom/press-releases/2025-1 03-sec-creates-task-force-tap-artificial-intelligenc e-enhanced-innovation-efficiency-across-agency, Aug. 2025, press Release No. 2025-103.
B Codebook Used to Apply FARSIGHT Framework Metrics Following the approach of Akanda et al. [51], we operationalize each FARSIGHT metric into an evidence-based checklist of binary (present/absent) indicators, each anchored to concrete artifacts observable in a published paper (mechanism descriptions, system diagrams, experimental results, or source code). Ratings are assigned by deterministic scoring rules, ensuring that two independent evaluators examining the same paper will check the same indicators and arrive at the same score. Below we specify each indicator set and its scoring logic; Table 8 presents the full codebook in tabular form.
[103] M. L. McHugh, “Interrater reliability: the kappa statistic,” Biochemia Medica, vol. 22, no. 3, pp. 276–282, 2012.
A Metric Assignment Criteria To maximize reproducibility, we adopt an evidence-based checklist methodology for metric assignment. For each metric, we define a set of evidence indicators, concrete, binary (present/absent) features that evaluators identify in the agent’s published paper, including described mechanisms, system diagrams, and reported experiments. Three-level ratings (● Fully Robust/Low Risk, ◆ Partial, ❍ Not Robust/High Risk) follow deterministic scoring rules based on which indicators are satisfied, eliminating subjective judgment from the scoring step. The complete indicator definitions and scoring rules for all nine evaluation metrics (R1–R4, S1, S2.1–S2.3, S3) are provided in the codebook (Appendix B, Table 8). S2 is decomposed into three sub-metrics, S2.1 (prompt-injection),
Robustness Evidence Indicators. R1 (Crash Foresight). E1: Quantitative anomaly detection (e.g., VIX thresholds, statistical outlier tests, correlationbreakdown monitoring, PCA-based volatility clustering). E2: Market-regime awareness via qualitative signals (e.g., sentiment profiling, macro-trend assessment, multi-source news synthesis). E3: Automated threshold-based alert or trigger mechanism (programmatic, not dependent on LLM reasoning alone). Rule: ● = E1 ∧ E3; ◆ = ≥1 of {E1, E2, E3} present but ¬(E1 ∧ E3); ❍ = none. R2 (Rapid Reaction). E1: Fast-path interrupt mechanism ex19
Table 7: Comparison between LLM-based analysis ( ) and human expert (²) review under the FARSIGHT framework. Agent System
R1
R2 ²
FinMem TradingAgents FinCon QuantAgent FLAG-Trader Total #Diff / Ratio (per Metric) ● (Fully Robust / Low Risk)
❍ ● ● ❍ ◆
◆ ◆ ❍ ◆ ❍
5 / 100%
R3 ²
◆ ● ● ❍ ●
❍ ◆ ❍ ◆ ❍
5 / 100%
R4 ²
● ● ● ◆ ◆
◆ ◆ ❍ ◆ ❍
4 / 80%
◆ (Partially Robust / Medium Risk)
S1 ²
◆ ● ● ● ◆
◆ ◆ ❍ ◆ ❍
4 / 80%
S2.1 ²
❍ ◆ ❍ ❍ ❍
❍ ◆ ❍ ◆ ◆
2 / 40%
S2.2
² ❍ ◆ ◆ ◆ ◆
❍ ◆ ◆ ◆ ◆
0 / 0%
S2.3
² ❍ ◆ ◆ ◆ ◆
◆ ◆ ◆ ◆ ◆
1 / 20%
S3
#Diff
Ratio
❍ ❍ ◆ ❍ ◆
4 5 4 6 5
44.4% 55.6% 44.4% 66.7% 55.6%
2 / 40%
24
53.3%
² ❍ ◆ ◆ ❍ ◆
❍ ◆ ◆ ◆ ◆
1 / 20%
² ❍ ◆ ◆ ◆ ◆
❍ (Not Robust / High Risk)
ecutable without LLM inference (e.g., external rules engine, hardware-level circuit breaker, or safe-mode switch that fires before LLM dispatch). E2: Automated order freeze or cancellation under detected shock (programmatic action that may still pass through the LLM-mediated reasoning chain). E3: Risk-authority override protocol (risk agent can overrule other agents without multi-round debate). Rule: ● = E1 present; ◆ = ≥1 of {E2, E3} present but ¬E1; ❍ = none. R3 (Loss Containment). E1: Quantitative stop-loss parameters defined (specific drawdown thresholds, position caps, or percentage limits). E2: Risk-aware behavioral adjustment (exposure reduction, risk-mode switching, or loss-triggered strategy change). E3: Automated enforcement (stop-loss executes programmatically, independent of LLM reasoning or human approval). E4: Multi-layer or volatility-adjusted containment (position-level + portfolio-level, or dynamic calibration). Rule: ● = E1 ∧ E3; ◆ = ≥1 of {E1, E2, E3, E4} present but ¬(E1 ∧ E3); ❍ = none. R4 (Recovery). E1: Crash-triggered recovery protocol (distinct recovery activated by tail events, not merely continuous learning). E2: Structured post-mortem analysis (fault diagnosis, historical scenario comparison, or root-cause identification). E3: Explicit parameter recalibration post-crash (reduced leverage, wider confidence thresholds, strategy downgrade). E4: Experience replay or self-reflection mechanism that feeds past outcomes into future decisions. Rule: ● = E1 ∧ (E2 ∨ E3 ∨ E4); ◆ = ≥1 of {E1, E2, E3, E4} present but ¬(E1 ∧ (E2 ∨ E3 ∨ E4)); ❍ = none.
pendently scored sub-metrics, S2.1 (prompt-injection), S2.2 (tool/API exploitation), and S2.3 (memory or context manipulation). Each sub-metric uses three indicators with a uniform pattern: E1 captures an active defensive mechanism for that attack class, E2 captures strict design-level elimination of the corresponding attack surface, and E3 captures partial designlevel restriction. The same scoring rule applies to all three sub-metrics. S2.1 (Prompt-Injection and Instruction Override). E1: Active prompt-injection defense, system-prompt immutability with user/data-input separation, input sanitization or injection filtering, or multi-agent verification (debate, voting, role separation) that requires defeating multiple agents simultaneously. E2: Strict design-level absence of open prompt channels. Only curated, authenticated, or research-environment inputs feed the agent; no end-user free-form prompts processed. E3: Partial design-level restriction. Inputs are simulated, sandboxed, or otherwise constrained, but residual prompt-layer exposure remains. Rule: ● = E2; ◆ = ≥1 of {E1, E3} present but ¬E2; ❍ = none. S2.2 (Tool-Use and API Exploitation). E1: Active tool/API protection, role-based access control (RBAC), capability scoping separating read from execution permissions, dualauthorization, rate limiting, or risk-management agents that gate trade execution. E2: Strict design-level absence of live actuating tools. No trade-execution path exists, and the system operates as analysis-only or simulation-only. E3: Partial design-level restriction. Live tools exist but are limited to read-only or sandboxed endpoints, with actuating tools off by default or available only under explicit human trigger. Rule: ● = E2; ◆ = ≥1 of {E1, E3} present but ¬E2; ❍ = none. S2.3 (Memory or Context Manipulation). E1: Active memory protection, write-gating, quarantine memory, provenancechecked promotion, or feedback/risk-driven validation before observations enter persistent storage. E2: Strict design-level absence of long-term memory. The agent has no persistent or self-evolving memory state. E3: Partial design-level restriction. Memory is short-term, ephemeral across sessions, or otherwise limited in self-reinforcement. Rule: ● = E2; ◆ = ≥1 of {E1, E3} present but ¬E2; ❍ = none.
Security Evidence Indicators. S1 (Information-Source Attacks). E1: Source provenance tracking (source domain, timestamp, and authentication metadata are recorded). E2: Cross-source corroboration required before trade execution (≥2 independent sources must confirm a market-moving signal). E3: Structured quantitative aggregation (voting, ranking, or policy learning) across independent analysis channels. E4: Human curation or design-level scope restriction that limits raw external data exposure. Rule: ● = E1 ∧ E2; ◆ = ≥1 of {E1, E2, E3, E4} present but ¬(E1 ∧ E2); ❍ = none. S2 Sub-metric Structure. S2 is decomposed into three inde20
Note on S2.1–S2.3. The ● criterion (E2) is intentionally reserved for advisory-only or offline-analysis systems that do not expose the corresponding surface at all; executioncapable agents with strong active defenses (E1) reach at most ◆ by construction. The paper’s “100% exhibit at least one security weakness” finding should therefore be read as “no evaluated agent achieves the E2 (surface-absent) bar on every S2 sub-metric,” not as “every agent is trivially exploitable.” S3 (Agent-as-Attacker). E1: Behavioral anomaly detection (monitors for sudden changes in order frequency, asset class, or trade volume). E2: Agent isolation capability (can sandbox or disable a single compromised agent without systemwide shutdown). E3: Inter-agent message validation (peer messages treated as untrusted inputs; policy engine validates before execution). E4: Human hard-stop override channel that can immediately halt all agent actions. E5: Strict designlevel absence of autonomous market-action capability. The agent has no live execution path and operates as an analytical, advisory, or purely offline simulation-only framework. Multiagent simulations with a shared, unisolated order book do not qualify. E6: Partial design-level restriction of autonomous market-action capability. The agent can interface with markets but is not inherently autonomous (e.g., minimal live API exposure, requires explicit human triggers, or limited execution scope). Rule: ● = E5 ∨ (E1 ∧ E2); ◆ = ≥1 of {E1, E2, E3, E4, E6} present but ¬E5 ∧ ¬(E1 ∧ E2); ❍ = none. This evidence-based codebook provides structured, expertapplied scoring rules: given the same published paper, two evaluators who identify the same indicators will arrive at the same rating. Consistent identification of the indicators themselves still requires domain expertise, as our LLM-application study (Appendix F) confirms. It also enables automated or hybrid evaluation workflows: as demonstrated in Appendix F, we used large language models to assist in applying these indicators and to cross-validate human expert assessments. Furthermore, the codebook is extensible: new metrics require only defining additional indicator sets and scoring rules, and the modular design allows adaptation to other financial sectors (e.g., derivatives, cryptocurrency, international equities) or new risk dimensions (e.g., regulatory compliance, liquidity risk) without altering the existing evaluation logic.
alpha validation catches risky strategies offline (E2), but no continuous live surveillance (no E1, no E3). QuantAgent: dedicated risk monitoring agent provides qualitative regime awareness (E2), but lacks concrete quantitative metrics such as VIX tracking or correlation dislocation (no E1) and no programmatic threshold trigger (no E3). TradingAgents: Bull/Bear researcher agents assess market regime via semantic reasoning (E2), but rely on qualitative reasoning rather than statistical signals (no E1) and no threshold-based trigger (no E3). TradingGroup: multi-source news/sentiment synthesis (E2), but no threshold-based early warning (no E1, no E3). R2 (Rapid Reaction): QuantAgent: risk-monitoring component triggers automated alerts under abnormal volatility (E2), but bounded by LLM inference latency, no LLM-independent fast path (no E1). TradingAgents: risk management team can override (E3), but acts through message-passing rather than algorithmic immediacy (no E1). TradingGroup: configurable stop-loss/take-profit thresholds trigger automated cancellation (E2), but static and user-defined without an LLM-independent fast path (no E1). R3 (Loss Containment): FinMem: adaptive risk modes reduce exposure after performance declines (E2), but no defined drawdown thresholds (no E1) or programmatic liquidation triggers (no E3). Alpha-GPT 2.0: research-stage risk assessment (E2), but no execution-time enforcement (no E3). QuantAgent: position management functions within the risk agent (E2), but omits mathematical details on stop-loss parameters (no E1) and no programmatic enforcement (no E3). TradingAgents: coordinated loss containment via risk management team yields empirically smaller drawdowns (E2), but no transparent liquidation logic (no E3). ContestTrade: downranks poorly performing agents (E2), but adaptation occurs after losses materialize (no E1, no E3). TradingGroup: configurable stop-loss settings (E1), but without volatility-adjusted calibration (no E4) or fully autonomous programmatic enforcement (no E3). R4 (Recovery): FinMem: self-evolution mechanism updates memory over time (E4), but no event-triggered recovery distinct from continuous learning (no E1). FinAgent: dual-level reflection framework feeds outcomes back into future decisions (E4), but no explicit crash-triggered reparameterization (no E1). Alpha-GPT 2.0: human researchers guide iterative improvements (E3), but no automated post-crash recovery protocol (no E1). QuantAgent: two-loop structure (inner optimization, outer testing) integrates outcomes (E4), but adaptation is slow and not crash-triggered (no E1). TradingAgents: continuous-learning loop (E4) without crash-specific recovery triggers (no E1). ContestTrade: evolutionary resilience promotes surviving agents (E4), but cannot handle simultaneous system-wide crashes (no E1). TradingGroup: self-reflection and continuous data synthesis (E4), but recovery is passive rather than crash-triggered (no E1).
C Per-Agent Metric Justifications Below we provide justifications for partial (◆) and fully robust / low-risk (●) scores in Table 3, following the codebook criteria in Appendix B. (Items not listed are not-robust / highrisk ❍ entries, for which no satisfied indicator was identified in the source paper.) R1 (Crash Foresight): FinMem: sentiment-based risk profiling with adaptive memory switching satisfies E2, but no quantitative anomaly detection (no E1) and no programmatic threshold trigger (no E3). Alpha-GPT 2.0: human-in-the-loop
S1 (Information-Source Attacks): StockAgent: simulated market environment limits raw external feed exposure (E4); 21
Table 8: Evidence-Based Codebook for FARSIGHT Metrics. Each metric is decomposed into binary evidence indicators. Ratings follow deterministic rules: ● = Fully Robust / Low Risk; ◆ = Partially Robust / Medium Risk; ❍ = Not Robust / High Risk. Metric
ID
Evidence Indicator (what evaluators look for)
Observable Artifact in Paper
Scoring Rule
E1
Quantitative anomaly detection
E2
Market-regime awareness via qualitative signals
● = E1 ∧ E3 ◆ = ≥1 of {E1,E2,E3} but ¬(E1∧E3) ❍ = none
E3
Automated threshold-based trigger mechanism
VIX thresholds, statistical outlier tests, correlation monitoring, PCA clustering Sentiment profiling, macro-trend assessment, news synthesis Programmatic alerts not dependent on LLM alone
E1
Fast-path interrupt without LLM inference
E2
Automated order freeze/cancellation under shock
● = E1 present ◆ = ≥1 of {E2,E3} but ¬E1 ❍ = none
E3
Risk-authority override protocol
External rules engine, hardware-level circuit breaker, or safe-mode switch firing before LLM dispatch Programmatic cancel of resting orders (LLM-mediated allowed) Risk agent overrules without debate
E1
Quantitative stop-loss parameters defined
E2 E3 E4
Risk-aware behavioral adjustment Automated stop-loss enforcement Multi-layer or volatility-adjusted containment
Drawdown thresholds, position caps, % limits Exposure reduction, risk-mode switching Executes without LLM/human approval Position + portfolio levels, or dynamic calibration
● = E1 ∧ E3 ◆ = ≥1 of {E1..E4} but ¬(E1∧E3) ❍ = none
E1
Crash-triggered recovery protocol
E2 E3 E4
Structured post-mortem analysis Explicit parameter recalibration post-crash Experience replay or self-reflection mechanism
Distinct from continuous learning; tailevent activated Fault diagnosis, scenario comparison Reduced leverage, wider thresholds Past outcomes feed future decisions
● = E1 ∧ (E2∨E3∨E4) ◆ = ≥1 of {E1..E4} but ¬(E1∧(E2∨E3∨E4)) ❍ = none
Domain, timestamp, authentication metadata ≥2 independent sources confirm signal Voting, ranking, or policy learning across channels Curated input pipeline or restricted exposure
● = E1 ∧ E2 ◆ = ≥1 of {E1..E4} but ¬(E1∧E2) ❍ = none
Immutable system prompt, input sanitization, multi-agent verification, role separation No open prompt channels; curated/authenticated inputs only Simulated/sandboxed inputs with residual exposure
● = E2 ◆ = ≥1 of {E1,E3} but ¬E2 ❍ = none
RBAC, capability scoping, dual-auth, rate limits, risk-gated execution No live execution / actuating tools by design Read-only or sandboxed tools; actuating tools off by default
● = E2 ◆ = ≥1 of {E1,E3} but ¬E2 ❍ = none
Write-gating, quarantine memory, provenance/risk-checked promotion No persistent / self-evolving long-term memory Short-term/ephemeral memory only, no self-reinforcement
● = E2 ◆ = ≥1 of {E1,E3} but ¬E2 ❍ = none
Robustness Metrics R1: Crash Foresight
R2: Rapid Reaction
R3: Loss Contain.
R4: Recovery
Security Metrics S1: InfoSource Attacks
S2.1: PromptInjection
S2.2: Tool/API Exploit.
S2.3: Memory/ Context
S3: Agent as Attacker
E1
Source provenance tracking
E2 E3
Cross-source corroboration before execution Structured quantitative aggregation
E4
Human curation or scope restriction
E1
Active prompt-injection defense
E2
Strict design-level scope restriction
E3
Partial design-level restriction
E1
Active tool/API protection
E2
Strict design-level scope restriction
E3
Partial design-level restriction
E1
Active memory protection
E2
Strict design-level scope restriction
E3
Partial design-level restriction
E1
Behavioral anomaly detection
E2
Agent isolation capability
E3
Inter-agent message validation
E4 E5
Human hard-stop override Strict design-level absence of live execution
E6
Partial design-level restriction
Monitors order frequency, asset class, volume changes Sandbox/disable one agent without system shutdown Peer messages untrusted; policy engine validates Immediate halt of all agent actions Analysis-only, advisory, or purely offline simulation (excludes multi-agent sims with a shared order book) Minimal live exposure or nonautonomous execution
22
● = E5 ∨ (E1∧E2) ◆ = ≥1 of {E1..E4,E6} but ¬E5 ∧ ¬(E1∧E2) ❍ = none
no provenance tracking (no E1) or cross-source corroboration (no E2). FinRobot: structured aggregation across modular sub-agents (E3); no provenance or corroboration enforcement. Alpha-GPT 2.0: human-curated alpha-research inputs (E4); no provenance or cross-source corroboration before signal use. QuantAgent: structured fusion of technical features within the risk-monitoring agent (E3); no provenance or corroboration. TradingAgents: debate-and-synthesis aggregation across analyst/researcher teams (E3); no provenance or corroboration enforcement. FLAG-Trader: policy-learning aggregation over heterogeneous signals (E3); no provenance or corroboration. ContestTrade: real-time ranking and contestbased aggregation across parallel paths (E3); no provenance or corroboration. TradingGroup: collaborative aggregation across group communication channels (E3); no provenance or corroboration.
limits memory poisoning surface (E3); long-term memory persists (no E2). S3 (Agent-as-Attacker): FinRobot (●): composable analytical platform whose design intentionally excludes any live execution or external trading connection, E5 (strict designlevel absence of autonomous market-action capability) satisfied. Alpha-GPT 2.0 (●): alpha-discovery framework with no autonomous trade execution, E5 satisfied. MarketSenseAI, MarketSenseAI 2.0: minimal live API exposure with nonautonomous execution, E6 (partial design-level restriction) satisfied. FinCon, FinAgent, TradExpert, FLAG-Trader, ContestTrade: limited live capability with risk-management gating, contest filtering, or dual-control structure, E6 satisfied.
S2.1 (Prompt-Injection & Instruction Override): AlphaGPT 2.0 (●): operates on curated researcher inputs rather than open prompt channels, E2 (strict design-level absence of open prompt channels) satisfied. MarketSenseAI, MarketSenseAI 2.0: chain-of-thought analysis over curated SEC filings introduces multi-step structured reasoning (E1) and a partially constrained input pipeline (E3); no full design-level isolation (no E2). StockAgent: simulated event-driven inputs partially restrict prompt exposure (E3) plus role-decomposition across investor agents (E1); residual exposure remains (no E2). FinCon, FinAgent, FinRobot, QuantAgent, TradExpert, TradingAgents, FLAG-Trader, ContestTrade, TradingGroup: multiagent verification, role separation, or chain-of-thought structure constrains direct prompt overrides (E1); none achieve full design-level absence of open prompt channels (no E2).
D
Distinct Characteristics of Financial LLM Agents
As illustrated in Section 4.2 Figure 4, financial LLM agents possess attack characteristics distinct from those in domains like healthcare, law, or education. Their exposure to real-time market data, dependence on heterogeneous signals, and highfrequency autonomy make them highly sensitive to latency, trust, and reflexivity attacks. We summarize five defining traits below. Instant. Attackers exploit millisecond-level market reactions and transient price dislocations to execute hit-and-run orders before human or automated risk systems can respond. Financial markets’ ultra-fast feedback loops enable monetizable micro-windows absent in other domains.
S2.2 (Tool-Use & API Exploitation): MarketSenseAI, MarketSenseAI 2.0 (●): static document retrieval and analysis without live API-driven trade execution, E2 (strict designlevel absence of actuating tools) satisfied. Alpha-GPT 2.0 (●): alpha-discovery research framework with no autonomous trade-execution path, E2 satisfied. TradingGPT, FinMem, StockAgent: limited tool exposure via single-agent or simulation context (E3); no full design-level execution isolation (no E2). FinCon, FinAgent, FinRobot, QuantAgent, TradExpert, TradingAgents, FLAG-Trader, ContestTrade, TradingGroup: risk-management gating, multi-agent veto, or capability scoping constrains execution (E1); actuating tools remain reachable (no E2).
Inconspicuous. Tiny quote or tick manipulations appear legitimate, prompting agents to alter trading behavior while users notice no anomaly. Because financial agents act on minute price deltas, even subtle perturbations can trigger hidden but impactful responses. Unverified. Inputs such as fake news, spoofed APIs, or forged order messages are often processed without provenance checks due to speed-first system design. This lack of validation allows attackers to issue unauthorized actions or trigger auto-trades with minimal friction. Adaptive. By coordinating multi-source inputs (news, sentiment, technical feeds), attackers dynamically tailor strategies to exploit signal inconsistencies. The agents’ heterogeneous integration logic grants adversaries flexibility unmatched in more structured data domains.
S2.3 (Memory or Context Manipulation): MarketSenseAI, MarketSenseAI 2.0 (●): stateless analysis pipeline without persistent self-evolving memory, E2 satisfied. TradExpert (●): mixture-of-experts produces predictions without persistent self-evolving memory state, E2 satisfied. FinCon, FinAgent, FinRobot, Alpha-GPT 2.0, QuantAgent, TradingAgents, FLAG-Trader, ContestTrade: feedback-driven, validated, or risk-gated memory updates (E1); memory remains long-term and modifiable (no E2). StockAgent: simulated environment
Cascading. Minor perturbations, like brief rumors or small sell orders, can cascade through interconnected trading agents, causing flash crashes or liquidity shocks. Because markets are reflexive and interdependent, one agent’s output becomes another’s trigger, amplifying small manipulations into systemic events. 23
sources of the ChatGPT Deep Research system to better understand its literature analysis behavior. As shown in Figure 5, the model queried a diverse set of sources, including arXiv, ResearchGate, RePEc, and OpenReview, but also accessed several non-academic domains such as Medium, mql5.com, and DigitalOcean. Although the model could retrieve and summarize relevant papers, it frequently encountered reCAPTCHA and rate-limit restrictions that reduced its search coverage compared to human experts. Additionally, it was easily distracted by unrelated technical blogs and community sites, resulting in lower focus and precision. This analysis demonstrates that while LLM-based search can efficiently explore new research areas, its reliability and selectivity remain inferior to those of human-guided exploration. Overall, as shown in Table 9, our experiments indicate that while LLM-based literature reviews are effective for fast scanning and abstract summarization, they remain limited in deep reasoning, detailed analysis, and evidence-level verification.
E Comparison of Investment Schemes and Stock-Trading Agents Passive-income schemes operate with minimal intervention: once capital is invested, returns are generated automatically without active management. Fixed-income assets such as bonds offer the highest stability and predictable yields, with risk limited mainly to large-scale defaults or market collapse. Similarly, mutual funds, index funds, and ETFs embody the passive philosophy, requiring little investor action. They exhibit strong robustness and security due to limited market interaction and low exposure to real-time trading risks, though their annualized returns remain moderate by design. In contrast, LLM-based stock-trading agents actively engage with live markets to pursue higher returns, but this introduces latency, prediction errors, and exposure to adversarial manipulation. In summary, passive schemes provide stable, secure, and steady returns, while active stock agents offer higher potential gains at significantly greater risk, requiring investors to balance stability against growth ambition.
Table 9: Observed strengths and weaknesses of LLM-based review.
F LLM Literature Analysis Implementation We conducted an automated literature analysis using ChatGPT Deep Research through a structured prompt derived from our FARSIGHT framework. The process included supplying the framework definitions for robustness (R1–R4) and security (S1–S3) metrics, restricting the search to academic papers published between 2023 and 2026 that focused on “financial LLM agents” or “LLM trading systems,” and requesting structured summaries highlighting robustness gaps, attack surfaces, and mitigation strategies. The LLM outputs were then compared with our manually reviewed dataset to assess alignment accuracy, as shown in Table 7. In total, the LLM identified eight candidate papers labeled as “financial LLM agents,” but only five of them (FinMem, FinCon, QuantAgent, TradingAgents, and FLAG-Trader) were genuinely relevant. The remaining three were benchmark, survey, or simulation studies without real agentic designs. While the model produced strong summaries and coherent overviews, its detailed framework-based analysis was unreliable: approximately 53% of metric-level evaluations were incorrect. We attribute this to the model’s inability to precisely locate and interpret framework-specific evidence within long, multisection papers. In particular, the context window limitation caused the LLM to miss fine-grained details while maintaining fluent but shallow summaries. This observation reinforces that while LLMs are promising for rapid literature summarization, their analytical accuracy in structured scientific evaluation remains substantially lower than that of human expert review. From Table 7, we can conclude that while LLMs perform evaluations well on existing attacks, their accuracy significantly degrades on novel attacks we propose. We also examined the reasoning process and retrieved
Strengths
Weaknesses
Fast scanning Abstract summarization Pattern discovery
Deep reasoning Detailed analysis Evidence verification
Figure 5: Source frequency distribution from ChatGPT Deep Research.
24