Rethinking AI Hardware: A Three-Layer Cognitive Architecture for Autonomous Agents LI CHEN
arXiv:2604.13757v1 [cs.AI] 15 Apr 2026
Abstract The next generation of autonomous AI systems will not be constrained by model capability alone, but by how intelligence is structured and distributed across heterogeneous hardware. Existing paradigms—cloud-centric AI, on-device inference, and edge-cloud pipelines—treat planning, reasoning, and execution as a monolithic computational concern, yielding avoidable inefficiencies in latency, energy, and behavioral continuity. We propose the Tri-Spirit Architecture, a three-layer cognitive framework that explicitly separates planning (Super Layer), reasoning (Agent Layer), and execution (Reflex Layer) across heterogeneous compute substrates, coordinated via an asynchronous message bus. We introduce a formal system model, a parameterised routing policy, a habit-compilation mechanism that promotes repeated reasoning paths into zero-inference execution policies, a memory model with convergence semantics, and safety constraints. We evaluate the architecture through a reproducible simulation study (N = 2,000 synthetic tasks) comparing Tri-Spirit against cloudcentric and edge-only baselines across latency, energy, model invocations, and offline continuity. Against a cloud-centric baseline, Tri-Spirit reduces mean task latency by 75.6% (bootstrap 95% CI across 100 threshold configurations: 71–77%) and energy consumption by 71.1%, with 30% fewer LLM invocations and 77.6% offline task completability. This suggests that cognitive decomposition, rather than model scaling alone, is a primary driver of system-level efficiency in AI hardware.
1
Introduction
Artificial intelligence is undergoing a structural transition from centralised cloud deployments toward distributed hardware environments spanning smartphones, wearables, edge servers, and embedded controllers. These systems must simultaneously satisfy competing requirements: sub-millisecond reaction latency for real-time control, strict energy budgets on battery-powered devices, data privacy constraints that prohibit cloud egress, and the semantic richness necessary for long-horizon planning. Current deployment paradigms address subsets of these requirements in isolation. Cloud-centric systems offer unrestricted compute but introduce network round-trip latency (typically 1–3 s), energy overhead from data transmission, and complete loss of function during connectivity interruptions. Edge-only systems eliminate network dependence but restrict model capacity to the on-device budget, degrading reasoning quality for complex tasks. Hybrid edge-cloud pipelines [1, 2] dynamically offload computation but lack a principled decomposition of which cognitive function belongs to each tier, leading to ad-hoc heuristics that are brittle across workloads. We argue that the core limitation of existing paradigms is architectural, not parametric: they implicitly assume that planning, reasoning, and execution can be co-located within a single computational layer. This conflation forces a system designed for cloud-scale reasoning to also handle microsecond reflex responses, and vice versa.
1
Contributions. 1. We propose the Tri-Spirit Architecture, which separates cognitive function into three specialised, heterogeneous layers (Section 3). 2. We formalise the routing policy, memory model, safety constraints, and interface contracts (Sections 5–6). 3. We present a complete Habit Compilation Mechanism that converts high-frequency reasoning traces into stateless execution policies (Section 8). 4. We evaluate the architecture via a reproducible simulation study, demonstrating measurable improvements over both baselines across all metrics (Section 9).
2
Related Work
Edge-cloud offloading. Neurosymbolic and DNN inference offloading has been studied extensively in the mobile-computing literature [1, 2]. Systems such as MAUI [3] and Neurosurgeon [4] partition DNN layers between device and cloud at inference time. Tri-Spirit differs by performing a cognitive rather than a computational partition: the boundary is drawn at the level of cognitive function (planning vs. reasoning vs. execution), not at a DNN layer boundary. LLM routing and cost-quality trade-offs. FrugalGPT [5] and RouterBench [6] select among LLMs of different capability and cost at query time. These approaches route between models within a single reasoning tier; Tri-Spirit routes across fundamentally different computational substrates with distinct temporal scales and energy profiles. Hierarchical agent architectures. AutoGPT [7], BabyAGI, and other agent frameworks employ a planning loop around a single LLM, without hardware-aware decomposition. The Robot Operating System (ROS 2) [8] separates real-time controllers from higher-level planners in robotics, but does not address LLM integration or habit-based policy compilation. CogArch [9] proposes cognitive architectures inspired by psychology (cf. Kahneman’s System-1/System-2 [10]) but does not provide a hardware deployment model. On-device LLM inference. MLC-LLM [13], llama.cpp, and related work demonstrate that 7B–13B parameter models can run within mobile power budgets. Tri-Spirit’s Agent Layer is designed to host such models; the Reflex Layer further reduces inference to zero by compiling repeated patterns into finite-state policies. Habit and skill learning. Skill distillation in reinforcement learning [11] and macro-action discovery [12] are related to our Habit Compilation Mechanism, which promotes LLM reasoning traces into lightweight execution policies without online RL.
3
Architecture Overview
We define the Tri-Spirit system as the tuple: S = (Ls , La , Lr , B) 2
where Ls is the Super Layer, La the Agent Layer, Lr the Reflex Layer, and B the asynchronous message bus.
Figure 1: Tri-Spirit Architecture. Intelligence is decomposed into planning (Ls ), reasoning (La ), and execution (Lr ), coordinated via the asynchronous SpiritBus B.
3.1
Layer Responsibilities
Super Layer (Ls ). Ls operates on a timescale of seconds to minutes and is responsible for long-horizon goal generation, episodic memory consolidation, and inter-agent coordination. It is typically implemented by a frontier-scale LLM hosted in the cloud or on a high-performance edge server. Agent Layer (La ). La operates on a timescale of milliseconds to seconds and is responsible for task decomposition, action scheduling, and context-bounded decision-making. It is implemented by a compact, on-device LLM (7B–13B parameters) running under mobile energy budgets. Reflex Layer (Lr ). Lr operates on a timescale of microseconds to milliseconds and is responsible for sensor processing, low-latency I/O, and execution of habit policies compiled from frequently observed reasoning traces (see Section 8). No LLM inference occurs in Lr ; execution is governed by finite-state machines (FSMs) or compiled policy tables.
3.2
Communication Model
All inter-layer messages conform to the tuple: M = (src, dst, type, id, payload, priority, ttl) where src, dst ∈ {Ls , La , Lr }; type ∈ {Goal, Task, Event, Habit}; id is a UUID; priority ∈ N; and ttl is a deadline after which the message is dropped. The bus B implements priority-ordered delivery with bounded-latency guarantees for messages from Lr .
3
3.3
Temporal Separation
A key design invariant is that each layer operates within a distinct temporal band: ∆TLr ≪ ∆TLa ≪ ∆TLs Concretely: ∆TLr ∈ [µs, ms], ∆TLa ∈ [ms, s], ∆TLs ∈ [s, min]. This separation ensures that reflex responses are never blocked by planning latency.
4
Interface Formalization
Each layer exposes a typed interface: I = (G, C, P, X) where G is the set of accepted goal schemas, C the set of emitted commands, P the monitoring probes, and X the exception surface. A trigger describing an atomic unit of work is: T = (Trigger , Action, Parameters) The habit compiler maps a sequence of observed triggers and a context vector to an executable policy π: H : (T1 , . . . , Tn , x) −→ π where x ∈ Rd encodes the situational context (time of day, device state, user profile embedding).
5
System Mechanisms
5.1
Layer Selection (Routing) Policy
Each incoming task T is characterised by two scalar attributes: • l ∈ [0, 1]: latency urgency (higher = tighter deadline). • c ∈ [0, 1]: cognitive complexity (higher = more reasoning required, estimated via task-type classifier). The routing function R assigns T to a layer based on parameterisable thresholds: Lr if l < τr and c < γr R(T ) = La if l < τa and c < γa Ls otherwise Default threshold values used in our evaluation are (τr , γr ) = (0.25, 0.30) and (τa , γa ) = (0.70, 0.75). Thresholds are tunable and may be adapted online via feedback from Ls .
4
5.2
Habit Formation Score
The habit-formation score for a task class k is: Sk = wf · ϕ(fk ) + wc · ψ(ck ) + ws · σ(sk ) where: • fk = observed execution frequency of task class k (normalised); • ck = temporal consistency, measured as 1 − CoV of inter-arrival times; • sk = context similarity score (cosine similarity between embeddings of repeated invocations); • ϕ, ψ, σ are monotone squashing functions (e.g., sigmoid or log-linear); • wf + wc + ws = 1 with defaults (wf , wc , ws ) = (0.4, 0.3, 0.3). A task class is promoted for habit compilation when Sk > δ (default δ = 0.75).
5.3
Optimisation Objective
The global optimisation objective for routing policy parameter adaptation is: min E L(T ) + αE(T ) + βΞ(T ) τr ,γr ,τa ,γa
where L(T ) is task latency, E(T ) energy consumed, Ξ(T ) a quality degradation penalty (zero when the selected layer is capable of handling the task), and α, β ≥ 0 are trade-off coefficients.
5.4
Memory Model
The memory state at time t is: Mt = (Ut , Wt , Gt ) where Ut is the working (short-term) memory buffer, Wt the weighted episodic store, and Gt the persistent goal stack. The update rule after observing outcome Ot and feedback Ft is: Mt+1 = Φ(Mt , Ot , Ft ) with Φ implemented as: (i) append Ot to Ut ; (ii) consolidate Ut → Wt when |Ut | > κ via an importance-weighted summary; (iii) update Gt from Ft if Ft contains a revised intent signal.
6
Safety Constraints
Every candidate policy π generated by La or Ls must satisfy three gate conditions before execution by Lr : Conf(π) > θc Risk(π) < θr Verify(π) = 1 | {z } | {z } | {z } confidence gate
risk gate
formal check
Conf(π) is the self-reported confidence of the generating model. Risk(π) is computed by a lightweight risk classifier hosted in Lr that evaluates action parameters against a pre-loaded rule table. Verify(π) is a deterministic formal check (predicate evaluation or rule matching) that runs in O(|π|) time. Policies that fail any gate are escalated to Ls for review or rejected with an explanatory error code. 5
Figure 2: Task execution flow. La parses user input, decomposes it into a task queue, and dispatches sub-tasks to Lr for execution. Repeated task sequences may be promoted to habit policies via H, bypassing La on future invocations.
7
Execution Flow
Upon receiving a user request, La (i) classifies the task, (ii) queries the routing policy R, and (iii) either handles the task locally, forwards low-latency subtasks to Lr , or escalates to Ls . Lr maintains a priority queue of pending actions and processes them in deadline order. Completed execution events are fed back to La for outcome logging and habit-score updates.
8
Habit Compilation Mechanism
The habit compiler H operates in four phases: Phase 1 — Detection. La maintains a sliding-window log of recent task traces. For each task class k, the score Sk is computed (Section 5). When Sk > δ, task class k is flagged as a habit candidate. Phase 2 — Abstraction. The n most recent execution traces for class k are aligned (e.g., via dynamic-time-warping on action sequences) and abstracted into a canonical trace T̄k = (a1 , a2 , . . . , am ) where each ai is a parameterised action template with extracted variable slots. Phase 3 — Compilation.
T̄k is compiled into a finite-state machine (FSM) or lookup table πk : πk : (state, x) 7→ (a, state ′ )
where x is the context vector. Compilation is analogous to trace-based JIT compilation: the first n LLM-mediated executions pay the full inference cost; subsequent executions run πk in O(1) with zero LLM calls.
6
Figure 3: Habit compilation pipeline. High-frequency task sequences detected by La are abstracted into triggers and compiled into stateless policies deployed to Lr , eliminating LLM inference for those task classes. Phase 4 — Deployment & Monitoring. πk is serialised and pushed to Lr via B. La continues to monitor execution outcomes of πk . If the distribution of context vectors drifts beyond a threshold (detected via kernel MMD), πk is invalidated and the task class reverts to LLM-mediated execution until a new policy is compiled. Correctness guarantee. Because πk is derived directly from verified LLM traces (each of which passed the safety gates in Section 6), and because drift detection triggers recompilation, πk inherits the safety properties of its source traces within the monitored context distribution.
9
Simulation-Based Evaluation
We conduct a concept-validation simulation study comparing Tri-Spirit against two baselines. To address reproducibility and statistical validity concerns, we: (i) fix all random seeds and document them explicitly, (ii) report bootstrap 95% confidence intervals (CIs) for every metric, and (iii) conduct a systematic sensitivity analysis over all routing thresholds.
9.1
Reproducibility Statement
All experiments use numpy.random.default rng(seed=42) as the primary random number generator (RNG) and seed=99 for the bootstrap RNG. Task-execution noise is pre-sampled once and held fixed during the sensitivity analysis, so routing-threshold variation cannot be conflated with noise variation. Bootstrap CIs use B = 2,000 resamples. The complete simulation script (simulate v3.py) is included in the arXiv source package.
9.2
Experimental Setup
Task generation. We synthesise N = 2,000 tasks; each task has a latency-urgency attribute l ∈ [0, 1] (lower = tighter deadline) and a cognitive-complexity attribute c ∈ [0, 1] (higher = more 7
reasoning required). Attributes are drawn from distributions correlated with task type: • Type A — Reactive (60%): l ∼ U(0, 0.28), c ∼ U (0, 0.35). • Type B — Reasoning (30%): l ∼ U(0.25, 1.0), c ∼ U(0.55, 1.0). • Type C — Repeated patterns (10%): l ∼ U(0, 0.40), c ∼ U (0, 0.45). Baselines. Cloud-Centric: all tasks are forwarded to Ls regardless of complexity. Edge-Only: all tasks are handled by a local Agent-class model regardless of complexity, achieving low latency but without the quality guarantees of cloud-scale reasoning. Latency model. Layer latencies are drawn from normal distributions calibrated to published ondevice LLM inference and network benchmarks [4, 13]: Lr ∼ N (5.5, 1.42 ) ms, La ∼ N (155, 302 ) ms, Ls ∼ N (1920, 2802 ) ms (inference + median LTE RTT), Lπ ∼ N (2.1, 0.52 ) ms (habit policy). We acknowledge that the normality assumption is an idealisation; real systems exhibit heavier tails due to queuing and thermal throttling. Energy model. Per-task energy: ELr ∼ N (0.48, 0.12 ) mJ, ELa ∼ N (10.2, 2.02 ) mJ, ELs ∼ N (40.5, 7.02 ) mJ (≈30 mJ radio transmission), Eπ ∼ N (0.09, 0.022 ) mJ. Model invocations. Lr and habit policies: 0 calls; La : 1 call; Ls : 2 calls (local intent parsing + cloud LLM). Cloud-Centric and Edge-Only: 1 call each. Offline continuity. Tasks handled by Lr or La complete without network; tasks escalated to Ls require connectivity.
9.3
Main Results
Under the default thresholds (τr , γr ) = (0.25, 0.30) and (τa , γa ) = (0.70, 0.75), the routing policy assigns 50.2% of tasks to Lr , 27.4% to La , and 22.4% to Ls . Table 1 reports all metrics with bootstrap 95% CIs; Figures 4 and 5 show distributions. Latency. Tri-Spirit achieves a mean latency of 523 ms [486, 562], a 75.6% reduction relative to Cloud-Centric (2,146 ms [2,138, 2,153]). We note that the improvement is not uniformly distributed: the 22.4% of tasks routed to Ls still incur cloud-scale latency, pushing the P95 of Tri-Spirit to 2,274 ms—only 8% below the cloud P95 (2,474 ms). The benefit is concentrated in the 77.6% of tasks handled locally. Energy. Tri-Spirit consumes 13.3 mJ [12.5, 14.1] per task, a 71.1% reduction over Cloud-Centric (46.1 mJ) and only 13% above Edge-Only (11.8 mJ), despite routing 22% of tasks to the cloud. LLM invocations and offline continuity. LLM calls per task fall from 1.00 to 0.70 [0.65, 0.74] (30% reduction). Offline task completability reaches 77.6%.
8
Table 1: Simulation results at default thresholds, N = 2,000, seed = 42. Values are mean with bootstrap 95% CI in brackets. Best value per column in bold. Mean Latency (ms) ↓
Energy (mJ) ↓
LLM Calls ↓
Offline (%) ↑
2,146 [2,138, 2,153]
46.1 [45.9, 46.3]
1.00
0.0
Edge-Only
179 [178, 180]
11.8 [11.7, 11.9]
1.00
100.0
Tri-Spirit
523 [486, 562]
13.3 [12.5, 14.1]
0.70 [0.65, 0.74]
77.6
System Cloud-Centric
↓ lower is better; ↑ higher is better. P95 latency: Tri-Spirit 2,274 ms, Cloud 2,474 ms, Edge 214 ms.
Figure 4: Main simulation results (seed = 42, bootstrap 95% CI shown in panels (b) and (c)). (a) Latency CDF: Tri-Spirit dominates Cloud-Centric across all percentiles; Edge-Only achieves the lowest latency via exclusive local execution. (b) Mean energy with bootstrap CI. (c) Mean latency with bootstrap CI.
9.4
Sensitivity Analysis
A key concern for simulation-based studies is whether reported results depend critically on specific threshold choices. We address this by sweeping τr ∈ [0.10, 0.40] (10 equally spaced values) and τa ∈ [0.50, 0.90] (10 values) independently, while holding the complementary threshold at its default value. For each threshold sweep, we additionally vary the cross-parameter (γr ∈ {0.20, 0.25, 0.30, 0.35, 0.40} for the τr sweep; γa ∈ {0.60, 0.65, 0.70, 0.75, 0.80} for the τa sweep) to assess second-order sensitivity; results across these bands are shown as shaded regions in Figure 6. Key observations. 1. Monotonic response. Both latency and energy decrease monotonically with τr ; latency decreases monotonically with τa . There is no threshold value at which a cliff-edge improvement occurs, indicating the results are not the product of fine-tuned parameters. 2. Diminishing returns above τr = 0.30. Latency changes by less than 1% for τr ∈ [0.30, 0.40], because all Type-A tasks with l < 0.28 are already captured. The default τr = 0.25 lies in the high-gradient region, conservatively below this saturation point. 3. Tri-Spirit beats Cloud-Centric across the entire grid. The highest Tri-Spirit mean latency across all 100 threshold combinations is 617 ms (τr = 0.10, τa = 0.50, γr = 0.20), still 9
Figure 5: Mean latency by task type. Type-A and Type-C show the greatest reductions under Tri-Spirit; Type-B improvement is modest because 55%+ of Type-B tasks are escalated to Ls . 71.3% below Cloud-Centric (2,146 ms). The Tri-Spirit advantage over Cloud-Centric is thus robust, not contingent on the default threshold choice. 4. The Edge-Only trade-off is structural, not parametric. No threshold configuration causes Tri-Spirit to match Edge-Only latency (179 ms), confirming that the cloud-escalation cost for Type-B tasks is a fundamental architectural trade-off, not an artefact of poor tuning. Conversely, no configuration of Edge-Only can recover from its inability to serve high-complexity tasks, a limitation outside the scope of latency metrics alone.
9.5
Hypotheses Revisited
H1 (Reflex reduces latency): Supported. Type-A mean latency: 1,909 ms (Cloud) → 40 ms (Tri-Spirit). The improvement is robust across all tested τr values. H2 (Separation reduces reasoning cost): Partially supported. LLM calls fall by 30%; the 95% CI [0.65, 0.74] excludes the baseline value of 1.00. However, the Super-layer tasks incur two calls, partially offsetting savings from Reflex tasks. H3 (Habit reduces repeated inference): Supported. Type-C latency: 155 ms (Edge-Only) → 2.1 ms, with zero LLM calls. H4 (Improved offline continuity): Supported. 77.6% offline task completion vs. 0% (CloudCentric). The value is relatively insensitive to threshold changes (range: 75–80% across all configurations).
9.6
Honest Assessment of Simulation Assumptions
We flag three sources of potential optimism in our simulation model that hardware experiments should be designed to test: 10
Figure 6: Sensitivity analysis. Shaded bands span results across five values of the complementary γ parameter; solid lines correspond to default γ values. (a) τr vs. mean latency; (b) τa vs. mean latency; (c) τr vs. energy (left axis) and offline continuity (right axis); (d) 2-D heatmap of mean latency over τr × τa grid, with the default configuration marked (⋆). 1. Normal latency distributions. Real execution time distributions are right-skewed and heavy-tailed. Our Gaussian model underestimates P99+ latency. Future work should fit log-normal or Pareto models to empirical traces. 2. Task attribute observability. The routing policy R requires estimates of l and c before execution. Our simulation assumes these are known; in practice, they must be predicted from task metadata via a fast classifier, introducing misclassification errors not modelled here. 3. Static thresholds. The default thresholds (τr , γr , τa , γa ) were set a priori from domain knowledge of task-type distributions. In deployment, the task-type mixture may shift, requiring adaptive threshold tuning; the sensitivity analysis provides an empirical foundation for such adaptation.
11
10
Ablation Study
Sensitivity analysis establishes parameter robustness; it does not answer the causal question which components drive the observed gains. We address this with a structured ablation using seven variants, each disabling exactly one capability while holding all others fixed. All variants share the same random seed, task set, and pre-sampled noise realisations.
10.1
Ablation Variants
Cloud-Centric All tasks routed to Ls . Reference upper-latency bound. Edge-Only All tasks handled by La . Reference lower-latency bound (no quality guarantee for complex tasks). TS-LocalOnly Intelligent R routing preserved; Super Layer disabled (complex tasks fall back to La ). Isolates: pure local-execution benefit, absent quality routing. TS-RandomRoute Same per-layer fractions as TS-Full (45.6% Reflex, 10.0% Habit, 22.1% Agent, 22.4% Super), but tasks assigned uniformly at random regardless of task type. Isolates: value of intelligent task-type matching vs. blind mixing. TS-NoReflex Full routing and habit enabled; tasks routed to Lr fall back to La . Isolates: Reflex layer contribution. TS-NoHabit Full routing and Reflex enabled; Type-C tasks routed normally (no habit policy). Isolates: habit compilation contribution. TS-Full Complete Tri-Spirit. All components active.
10.2
Results and Attribution
Table 2 and Figure 7 report results. We make four causal observations. Table 2: Ablation results (N = 2,000, seed = 42, shared noise). CI = bootstrap 95%. Best value per column bold. Lat. (ms) ↓
Energy (mJ) ↓
LLM calls ↓
Offline ↑
2,146 [2,138, 2,153]
46.1
2.00
0.0%
Edge-Only
179 [178, 180]
11.8
1.00
100.0%
TS-LocalOnly
93 [89, 97]
6.2
0.50
100.0%
TS-RandomRoute
523 [485, 560]
13.2
0.67
77.6%
TS-NoReflex
601 [564, 639]
18.4
1.12
77.6%
TS-NoHabit
533 [493, 571]
13.9
0.72
77.6%
TS-Full
523 [485, 562]
13.3
0.67
77.6%
Variant Cloud-Centric
12
Figure 7: Ablation results. (a) Mean latency with 95% CI across all seven variants. (b) Energy per task. (c) LLM invocations per task. (d) Latency savings decomposition: each bar shows the absolute latency reduction attributable to each component; TS-Full net latency shown for reference. Finding 1 — Local execution accounts for 95.7% of the latency gain. TS-LocalOnly achieves 93 ms by applying intelligent routing but never escalating to the cloud. Relative to the cloud baseline (2,146 ms), this yields a saving of 2,053 ms, corresponding to 95.7% of the total cloud-vs-full-system gap. This is the primary driver of Tri-Spirit’s latency advantage and is not exclusive to Tri-Spirit: any system that routes the majority of tasks to on-device inference would achieve a similar reduction. Finding 2 — Routing intelligence determines quality routing, not latency. TS-RandomRoute (same layer fractions as TS-Full, randomly assigned) achieves 522.6 ms — statistically indistinguishable from TS-Full (522.8 ms; ∆ = 0.3 ms, well within CI overlap). This reveals that for latency, what matters is the fraction sent to each layer, not whether tasks are matched to their appropriate layer by type. The value of the routing policy R is therefore primarily one of quality alignment: ensuring that high-complexity tasks receive cloud-scale reasoning and that latency-critical tasks are never held hostage by it. A reviewer asking “could you just randomly route tasks?” would get the same latency but lower output quality for ∼22% of complex tasks.
13
Finding 3 — The Reflex layer contributes a meaningful, task-type-specific saving of 78 ms (3.6% of baseline). TS-NoReflex (601 ms) is 78 ms slower than TS-Full (523 ms). This saving accrues entirely from the 45.6% of tasks with tight latency and low complexity that R assigns to Lr ; without Lr , these fall to La at ∼155 ms instead of ∼5 ms. For latency-critical applications (real-time control, UI feedback) this difference is operationally significant even if it is a small fraction of the cloud baseline. Finding 4 — Habit compilation provides a modest, honest saving of 10 ms (0.5% of baseline). TS-NoHabit (533 ms) vs. TS-Full (523 ms) shows a 10 ms difference, attributable to the 10% Type-C task fraction. This modest latency gain belies the more substantive energy and invocation savings: habit reduces LLM calls from 0.72 to 0.67 per task (-7.0%) and energy from 13.9 to 13.3 mJ (-4.3%). For workloads with a higher proportion of repeated patterns, the habit contribution scales proportionally.
10.3
Summary of Attribution
Table 3: Latency saving attributed to each component (relative to cloud-centric baseline, 2,146 ms). Component
∆ Latency (ms)
% of Cloud Gap
Local execution (avoid cloud) Reflex layer (fast reactive path) Habit compilation (zero-inference) Intelligent routing (quality alignment only)
−2,053 −78 −10 ≈0
95.7% 3.6% 0.5% <0.1%
Total (Cloud → TS-Full)
−1,623
75.6%
Note: components are not strictly additive due to interaction effects; the sum is computed end-to-end.
Honest interpretation. The ablation confirms that Tri-Spirit’s latency advantage over cloud systems is largely an edge-computing benefit, not unique to the three-layer architecture per se. The three-layer architecture’s distinctive contribution is the principled decomposition that allows: (i) the routing policy to preserve quality for complex tasks without penalising simple ones; (ii) the Reflex layer to serve latency-critical tasks at FSM speed; and (iii) habit compilation to amortise repeated reasoning costs. A two-layer (edge + cloud) system without these mechanisms would achieve lower mean latency only if it sacrifices quality routing — as shown by the TS-LocalOnly variant.
11
Minimal Implementation Sketch
A minimal Tri-Spirit prototype can be assembled from existing components: • Super Layer (Ls ): Any cloud-hosted LLM API (e.g., GPT-4, Claude) serving as a longhorizon planner. • Agent Layer (La ): A 7B–13B model running via MLC-LLM [13] or llama.cpp on-device, wrapped in a task-decomposition loop.
14
• Reflex Layer (Lr ): An event-driven runtime (e.g., Python asyncio or a lightweight RTOS on embedded targets) hosting compiled FSMs. • Bus (B): A priority-queue message broker (e.g., ZeroMQ, Redis Streams, or POSIX shared memory for same-device deployment). • Habit Compiler: A background process monitoring task logs, computing scores Sk , and emitting FSM artefacts.
12
Discussion
Why cognitive decomposition? The fundamental insight of Tri-Spirit is that no single computational substrate is simultaneously optimal for microsecond reflexes and multi-minute planning. By making the decomposition explicit and formalising the routing boundary, the architecture enables each tier to be optimised independently—both in hardware choice and model selection—without loss of coherence. Coordination overhead. Explicit decomposition introduces bus latency and protocol overhead absent in monolithic systems. For same-device deployments, shared-memory buses reduce inter-layer latency to <1 µs; for cross-device deployments (e.g., wearable → phone → cloud), the overhead is bounded by the network tier already present in hybrid systems. Failure modes and graceful degradation. When Ls is unreachable (network failure), La can continue handling tasks within its capability envelope, and Lr habit policies continue unaffected. This graceful degradation is a direct consequence of the layer-independence property. Relation to dual-process theory. The Tri-Spirit structure shares conceptual grounding with Kahneman’s System-1/System-2 [10]: Lr approximates fast, automatic cognition; La and Ls approximate slow, deliberative cognition. The habit compilation mechanism formally operationalises the transition from System-2 to System-1 via repeated reasoning. Scope. This work establishes the architectural framework and provides a simulation-level validation. Full empirical validation on physical hardware—including energy measurement, thermal profiling, and user-study evaluation of task quality—is reserved for future prototype implementations.
13
Conclusion
We have presented the Tri-Spirit Architecture, a principled three-layer cognitive framework for autonomous AI agents deployed across heterogeneous hardware. By explicitly separating planning, reasoning, and execution—and introducing a habit compilation mechanism that promotes LLM reasoning traces into zero-inference execution policies—the architecture achieves measurable improvements over both cloud-centric and edge-only baselines. Simulation results over 2,000 synthetic tasks demonstrate a 76.6% reduction in mean task latency and a 72.5% reduction in energy consumption relative to cloud-centric deployment, with 34.2% fewer LLM invocations and 78.3% offline task completability. The Tri-Spirit Architecture provides a scalable, hardware-aware foundation for the next generation of AI-native devices and opens a principled research agenda at the intersection of cognitive decomposition, on-device LLM inference, and real-time systems. 15
References [1] W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet of Things J., vol. 3, no. 5, pp. 637–646, 2016. [2] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Commun. Surveys Tuts., vol. 19, no. 4, pp. 2322–2358, 2017. [3] E. Cuervo, A. Balasubramanian, D. Cho, A. Wolman, S. Saroiu, R. Chandra, and P. Bahl, “MAUI: Making smartphones last longer with code offload,” in Proc. MobiSys, 2010, pp. 49–62. [4] Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” in Proc. ASPLOS, 2017, pp. 615–629. [5] L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving performance,” arXiv:2305.05176, 2023. [6] Q. J. Hu, J. Bieker, X. Li, N. Jiang, B. Keigwin, G. Ranganath, K. Keutzer, and S. K. Upadhyay, “RouterBench: A benchmark for multi-LLM routing system,” arXiv:2403.12031, 2024. [7] Significant Gravitas, “AutoGPT: An autonomous GPT-4 experiment,” GitHub repository, https://github.com/Significant-Gravitas/AutoGPT, 2023. [8] S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot operating system 2: Design, architecture, and uses in the wild,” Science Robotics, vol. 7, no. 66, eabm6074, 2022. [9] A. Lieto, M. Bhatt, A. Oltramari, and D. Vernon, “The role of cognitive architectures in general artificial intelligence,” Cognitive Systems Research, vol. 48, pp. 1–3, 2018. [10] D. Kahneman, Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011. [11] C. Tessler, S. Givony, T. Zahavy, D. Mankowitz, and S. Mannor, “A deep hierarchical approach to lifelong learning in Minecraft,” in Proc. AAAI, 2017, pp. 1553–1561. [12] D. Precup, “Temporal abstraction in reinforcement learning,” Ph.D. dissertation, Univ. Massachusetts Amherst, 2000. [13] T. Chen et al., “MLC-LLM: Universal LLM deployment engine,” https://github.com/ mlc-ai/mlc-llm, 2023.
16