GuidaPA: Privacy-Preserving Chatbot for Public Administration via Federated Learning Daniel M. Jimenez-Gutierrez1 , Albenzio Cirillo2 , Raffaele Nicolussi2 , Alessio Beltrame2 , Andrea Vitaletti1 1
Sapienza University of Rome, Via Ariosto 25, 00185 Rome, Italy Fondazione Ugo Bordoni, Viale del Policlinico 147, 00161 Rome, Italy Emails: [email protected], [email protected], [email protected], [email protected],[email protected]
PA Client K
Client 1 Local model K
Local model 1 Local weights
Local weights
FL model Private Centralized model
Internet
Raw data transmission
Abstract—We present GuidaPA, a privacy-preserving chatbot for the Italian Public Administration (PA) trained via Federated Learning (FL) on documentation from two national PA platforms, SIGESON and SIDFORS. Our corpus includes approximately 8 pages of SIGESON manuals and 31 pages of SIDFORS manuals/FAQs; while this study uses public documentation as a safe proxy, the intended deployment extends to restricted internal sources (e.g., tickets, officer manuals, database extracts) that can not be centrally pooled due to regulatory and organizational constraints. GuidaPA integrates role-based access control, secure client-side preprocessing, explicit monitoring of non-IID effects, and parameter-efficient federated fine-tuning of large language models. Using QLoRA (4-bit) over 15 federated rounds with an 80/20 train–test split per client, we evaluate answer quality with ROUGE, BLEU-4, and METEOR. The best federated model achieves ROUGE-1/2/L of 61.10/55.77/59.44, BLEU-4 of 45.02, and METEOR of 63.94—close to private centralized fine-tuning while keeping data on-site. Compared to the general-purpose baseline, domain fine-tuning improves ROUGE-1 from 41.45 to 62.18 and BLEU-4 from 26.97 to 50.90. Overall, the results indicate that FL can deliver high-quality conversational AI for public services without centralized data sharing. Index Terms—federated learning, large language models, finetuning, distributed learning, non-IID data, public administration
Raw data transmission
arXiv:2606.01386v1 [cs.AI] 31 May 2026
2
Centralized model
I. I NTRODUCTION The rapid diffusion of Large Language Models (LLMs) is enabling wider adoption of NLP components based on Generative AI (GenAI) within public services [1], [2]. Public Administration (PA)—the set of governmental institutions responsible for delivering public services and enforcing regulations [3]—can benefit from LLM-based assistants to automate repetitive requests and improve access to information [4]. However, PA constraints make conventional centralized training impractical and often undesirable from a privacy and governance perspective [5], [6]. Figure 1 summarizes three deployment options. Internetcentralized solutions are technically convenient but require sending sensitive information to external providers, introducing privacy risks. A Private Centralized model keeps data within PA infrastructure and aligns with the principle that PAs should make data available to other PAs when required for institutional tasks, under legal and privacy constraints [7]. A centralized architecture within PA serves as a unified knowledge aggregator, effectively mitigating the inconsistencies generated by siloed workflows and ensuring seamless
Fig. 1: Alternative architectures for integrating an LLM within PA. Internet-centralized approaches transfer data to third parties; private centralized approaches keep data within PA but still require inter-entity sharing; Federated Learning (FL) exchanges only model updates, preserving data sovereignty. interoperability across all administrative sectors. Nonetheless, inter-office pooling still increases governance burden and exposure. FL strengthens privacy by keeping data local and sharing only model updates, enabling collaborative learning across institutional silos while preserving data sovereignty. A. Motivation Deploying chatbots in PA is challenging due to organizational and regulatory constraints [8], [9]. In our case study, these constraints arise in two operational PA platforms: SIDFORS and SIGESON. Their documentation includes end-user manuals, FAQs, and procedural guidance for administrative workflows such as enrollment, authorization requests, access
to company accounting positions, annual declarations, and payment of administrative fees. Although this study uses only public documentation as a safe proxy, these sources reflect the structured and procedural nature of PA knowledge and represent the broader institutional material that a deployed system may need to support, including internal manuals, support tickets, and curated database extracts. Key challenges include: (i) protecting sensitive data (C1 ), (ii) adapting foundation models to PA-specific procedures and terminology (C2 ), and (iii) integration with heterogeneous systems and operational scalability (C3 ). Fine-tuning and Retrieval-Augmented Generation (RAG) can improve domain knowledge [10], [11], but they do not remove the need for an infrastructure that preserves privacy and supports distributed governance. As noted by the Italian Ministry of Economy and Finance’s Treasury Department (MEF–DT) [12], AI in PA requires robust infrastructure and data management; yet interoperability remains incomplete [13], making centralized data pooling unrealistic. Fully local ad hoc solutions are possible but costly and fragmented, potentially leading to inconsistent behavior across offices. B. Research Objectives This study investigates whether FL can enable effective, privacy-preserving LLM-based chatbots in PA settings where data are distributed across institutional silos and cannot be centrally aggregated. We pursue: (i) designing a federated architecture with role-based access control (RBAC) and secure client-side data handling, (ii) evaluating whether federated fine-tuning matches private centralized quality while preserving locality, and (iii) assessing robustness under client heterogeneity and quantifying non-IID effects on global and local performance [14], [15]. C. Contribution We introduce GuidaPA, a privacy-preserving PA chatbot trained via FL to address C1 –C3 . Unlike prior FL applications that mainly focus on privacy-preserving model training alone, GuidaPA combines federated LLM adaptation with PA-specific deployment mechanisms that are directly relevant in practice: role-based access control, secure client-side preprocessing, and explicit monitoring of non-IID effects. Its main technical contribution, therefore, goes beyond applying FL to chatbot adaptation by integrating these components into a practical framework for institutional settings and showing empirically that this design can achieve near-centralized quality without centralizing the underlying documents. Using real documentation from the SIDFORS and SIGESON platforms, federated fine-tuning achieves quality comparable to a private centralized approach (Section VI-A) while keeping data local, and consistently improves over the general-purpose baseline (Table IV), demonstrating effective domain adaptation. Since PA documentation follows shared standards and regulatory structures, cross-client non-independent and identically distributed (nonIID) data is limited but non-zero, allowing federated training to converge while capturing realistic non-IID effects [14], [15].
More broadly, this work shows how privacy-preserving learning architectures can support controlled dissemination and access to institutional knowledge in public-sector information systems. D. Paper Structure Section II reviews related work. Section III describes the data sources and dataset. Section IV presents the architecture. Sections V and VI report the experimental setup and results, followed by discussion (Section VII), limitations (Section VIII), and conclusions (Section IX). II. R ELATED W ORK The use of chatbots in the context of PA remains relatively limited compared to commercial and open-domain applications, where large-scale conversational systems benefit from abundant centralized data and rapid iteration cycles. To the best of our knowledge, no prior work has applied FL to the development of LLM–based chatbots in the PA domain in Italy. Prior research spans (i) centralized chatbots for egovernment and public services, (ii) federated chatbot approaches in privacy-sensitive domains, and (iii) federated finetuning of LLMs, which provides methodological foundations for privacy-preserving adaptation. Centralized PA chatbots have been proposed for citizen support and information access, including Italian public-service conversational frameworks and domain assistants [16], [17], as well as systems for navigating governmental open data and improving transparency and responsiveness [18], [19]. While these works demonstrate the potential of conversational interfaces to reduce friction in citizen–government interactions, they typically assume that training data can be collected and maintained in a single location, which is often unrealistic in PA environments characterized by organizational silos and constrained data governance. In privacy-sensitive domains such as healthcare and education, FL has been used to train conversational agents without sharing raw conversations, showing that collaborative learning is feasible under data-locality constraints [20]–[22]. However, these studies do not address PA-specific requirements such as role-based access control, institutional governance, and cross-office procedural heterogeneity. Our work builds on this literature by focusing on these deployment constraints in the PA setting. Recent surveys and methods on federated LLM adaptation highlight key challenges and enablers—including communication efficiency, non-IID data, and parameter-efficient finetuning (PEFT) with low-rank adaptation and quantization— that motivate our design choices [23], [24]. PEFT reduces the computational and communication cost of federated tuning, while raising questions about stability and aggregation under client heterogeneity. These issues are especially relevant in PA, where local datasets may be small but authoritative, and compute resources are limited. GuidaPA addresses this gap by combining federated finetuning of LLMs with an application-level architecture tailored
to PA requirements, including RBAC, client-side secure preprocessing, and explicit monitoring of non-IID data effects. In doing so, this work bridges recent advances in federated LLM research with the practical needs of public-sector information services, providing empirical evidence that near-centralized quality can be achieved without centralizing the underlying documents. III. D OMAIN -S PECIFIC DATA S OURCES AND DATASET D ESCRIPTION FOR G UIDA PA GuidaPA was trained using domain documentation from two operational systems managed by the Ministry of Enterprises and Made in Italy (MIMIT) and developed with the support of Fondazione Ugo Bordoni (FUB), namely SIDFORS (Sistema per dichiarazione fornitura di reti/servizi di comunicazione elettronica) [25] and SIGESON (Sistema di Gestione delle Reti di Radiodiffusione sonora) [26]. We use their publicly available manuals and FAQs as training and evaluation sources. Public docs are used here as a safe proxy for restricted internal sources. SIDFORS enables sector operators to submit general authorization requests for electronic communications networks and services (e.g., telephony, Wi-Fi, satellite, fiber), request authorization to access a company’s accounting position, and pay the applicable fees [25]. SIGESON supports audio broadcasting operators in requesting authorization to access a company’s accounting position and paying the applicable fees [26]. In both cases, requests submitted by operators must undergo manual review and validation by public officers, and users often contact officers to clarify procedures and platform functionality. To reduce support effort, both platforms provide freely accessible documentation; for this study, GuidaPA is trained exclusively on this public material: • End-user manuals describing core procedures (e.g., enrollment, authorizations, payments, access to accounting positions) [25], [26]. • FAQs addressing common issues encountered by endusers [25], [26]. A. Dataset Composition and Statistics The datasets consist of Italian domain documentation from two PA platforms, each corresponding to one federated client and reflecting a distinct administrative vertical. Client 1: SIGESON (Radio Broadcasting). The SIGESON dataset is derived from the user manual “Guida rapida per l’operatore” (Version 1, October 2023; ∼8 pages) and the platform FAQ knowledge base [26]. It focuses on workflows such as access authorization to the accounting module, annual turnover declarations, and payment of administrative dues. Client 2: SIDFORS (Electronic Communications). The SIDFORS dataset includes the “Manuale d’uso del servizio SIDFORS” (Version 2.1, May 2024; ∼31 pages) and the platform FAQs [25]. It covers general authorization workflows (e.g., fiber, satellite, MNO/MVNO) and procedures to access the accounting module for managing pagoPA-related payments.
Data preprocessing. At each client, unstructured PDF/HTML sources are processed locally into an instructiontuning format via a lightweight pipeline: extract → clean → segment → FAQ-to-QA pairs. Table I reports dataset statistics; the larger SIDFORS documentation (31 pages vs. 8 pages) naturally induces a realistic non-IID imbalance. TABLE I: Dataset statistics per federated client. Metric
Client 1 (SIGESON)
Client 2 (SIDFORS)
Primary domain Source docs
Radio broadcasting Electronic communications Manual v1 (8 pages) + Manual v2.1 (31 pages) + FAQs FAQs Key procedures Turnover declaration; Network declarations; accounting-module access accounting-module access Samples 32 45 Avg. input length 10.0 10.19 (tokens) Avg. output length 27.29 25.13 (tokens)
All documents are written in Italian and reflect formal administrative language and platform-specific terminology. B. Roles and Permissions In PA environments, users interact with digital services based on defined institutional roles, which regulate access to information and procedures. GuidaPA uses RBAC to ensure context-appropriate responses and prevent disclosure beyond a user’s permissions. In our experimental setup, each domain’s public documentation is isolated to validate access control: SIDFORS roles receive answers derived only from SIDFORS content, while SIGESON roles receive answers derived only from SIGESON content. Accordingly, the following roles were defined: • Radio broadcasting operator/citizen: external user querying GuidaPA for SIGESON public documentation (manuals and FAQs). • Public electronic communications operator/citizen: external user querying GuidaPA for SIDFORS public documentation (manuals and FAQs). • SIDFORS Officer: MIMIT officer responsible for managing SIDFORS procedures. • SIGESON Officer: MIMIT officer responsible for managing SIGESON procedures. • Master: highest-level profile with access to all sources available in GuidaPA. The relationships between roles and access permissions are summarized in Table II. TABLE II: Access permissions across public documentation. Role Radio broadcasting operator/citizen Public electronic comm. operator/citizen SIDFORS officer SIGESON officer Master
SIDFORS
SIGESON
No Yes Yes No Yes
Yes No No Yes Yes
6
Global weights
Response generation
Client 1
3
Conversation manager
Server
LLM finetuning 1
Users
Role 2 authentication
7
Role 4 restriction
Performance component
Client 2
Local weights
... Client K
5
Federated Learning + Client data pipeline
Fig. 2: The GuidaPA proposed architecture.
IV. T HE G UIDA PA S YSTEM The overall architecture of GuidaPA, depicted in Figure 2, combines natural-language interaction with strong privacy guarantees through FL. It integrates four components—the Conversation Manager, the FL module, the (private) Client Data Pipeline, and a Performance Module—to enable operation across distributed PA data sources. Each PA office acts as a client that trains locally on its own data, while a central server orchestrates aggregation of model updates in a client– server FL workflow. We assume an honest-but-curious server that only aggregates updates; clients keep raw documents local; RBAC is enforced at the Conversation Manager. This setup mirrors the organizational reality of the PA, where data are siloed across offices while processes and regulations remain shared.
B. FL Module At the core of GuidaPA lies the FL module, which enables privacy-preserving collaborative training across distributed PA data sources. Each participating client trains a local model on its own dataset and periodically shares model updates—rather than raw data—with the central server. The server aggregates these updates into a global model, which is redistributed back to clients over multiple rounds. This enables the global model to benefit from collective knowledge while sensitive information remains local, reducing the need for centralized data collection and supporting privacy and regulatory constraints (e.g., GDPR principles). In this work, we do not train LLMs from scratch. Instead, we perform parameter-efficient fine-tuning of pretrained LLMs, which reduces computational cost and training time while adapting models to the PA domain. C. Client Data Pipeline Module
A. Conversation Manager Module The Conversation Manager is the entry point for user interaction and governs dialogue flow. It provides two safeguards: role restriction and controlled response delivery. When a user submits a query, the module validates credentials and maps the request to the sources authorized for that role. As shown in Figure 2, role restriction augments the query with the allowed sources while blocking access to restricted content (e.g., via prompt injection [27]), thus enforcing least privilege and organizational access policies. The resulting query is sent to the current global model, and the generated answer can be post-processed for conciseness, provenance logging, and safety checks before being returned.
The (private) Client Data Pipeline manages secure preparation and processing of local datasets before training or inference. Each client performs local text extraction, cleaning, and instruction-format conversion, ensuring sensitive data never leaves the client environment. The pipeline is extensible to additional sources, but in this study we use only public manuals and FAQs. D. Performance Module This module evaluates the effectiveness of the fine-tuned LLMs within the GuidaPA architecture. It reports quantitative metrics capturing the quality and reliability of generated answers after federated fine-tuning (see Section V), enabling monitoring of model behavior across federated rounds.
In addition to answer quality, the Performance Module measures heterogeneity across clients (non-IID data), which is relevant in PA settings where platforms contribute related but distinct procedures and terminology. We quantify non-IID effects using standard distributional measures such as Jensen– Shannon divergence (JSD) and Hellinger distance (HD) [28]. Joint analysis of quality metrics and heterogeneity supports assessing the robustness of federated fine-tuning under realistic data imbalance and domain shifts. E. Chatbot Interface Figure 3 shows the GuidaPA graphical interface. The active role (e.g., SIGESON OFFICER) determines which information sources are accessible. The interface illustrates two example queries and the corresponding behavior. In the first query, “Chi può richiedere l’abilitazione nel portale SIGESON?” (“Who can request access authorization in the SIGESON portal?”), the system identifies the request as SIGESON-related and returns an answer consistent with the role permissions. In the second query, “La mia società non è censita nel sistema SIDFORS” (“My company is not registered in the SIDFORS system”), the request is recognized as SIDFORSrelated. However, as defined by the role-based access policies (Table II), SIGESON officers cannot access SIDFORS resources; the system therefore returns a denial message and prompts the user to change role. This example highlights how the interface enforces rolebased access restrictions, ensuring that each profile can query only the sources it is institutionally authorized to consult, consistent with the least-privilege principle. V. E XPERIMENTS This section evaluates GuidaPA by assessing whether federated fine-tuning can effectively adapt LLMs to PA documentation under realistic data locality constraints. We describe the experimental setup (clients, training protocol, models, and metrics) and report global and local performance under FL. A. Experimental Design Testbed. Experiments were run on a single GPU workstation (RTX A6000, 48GB) using Flower [29] and Python 3.10. Data and Clients. Federated experiments use the two PA clients described in Section III (SIGESON and SIDFORS), each training locally on its own documentation. We adopt an 80/20 train–test split per client. The federation runs for 15 server rounds with one local epoch per round (Table III). Non-IID Data Quantification. To characterize client heterogeneity, we quantify lexical shift between SIGESON and SIDFORS using Hellinger distance (HD) and Jensen–Shannon divergence (JSD) over different n-gram sizes. Figure 4 shows moderate divergence, with higher differences for smaller ngrams and increasing overlap for longer n-grams, consistent with shared administrative language but distinct domain terminology. These measurements support analyzing FL robustness under realistic non-IID conditions (Section IV-D) [28].
Models and Baselines. We evaluate a focused set of Italian-centric and multilingual instruction-tuned LLMs commonly used in Italian NLP benchmarks [30], [31] and available via the Transformers ecosystem [32]. Specifically, we consider Italian-oriented conversational models and strong multilingual baselines: galatolo/cerbero-7b and galatolo/cerbero-7b-openchat; cosimoiaia/Loquace-7B-Mistral; mii-llm/ maestrale-chat-v0.4-beta; giux78/ zefiro-7b-beta-ITA-v0.1; Meta’s multilingual meta-llama/Llama-3.1-8B-Instruct; and finally the Mistral family (mistralai/ Mistral-7B-Instruct-v0.3 and mistralai/ Mistral-7B-v0.1). All models are adapted with the same federated parameter-efficient setup (Table III) using QLoRA and low-rank adaptation, following established best practices for efficient fine-tuning in resource- and communicationconstrained settings [24], [33]–[35]. We use FedAvg with weighted averaging by client dataset size. TABLE III: Hyperparameters used for LLM fine-tuning in FL. Hyperparameter
Value
Parameter-efficient method Quantization (bits) Gradient checkpointing LoRA rank (r) LoRA α LoRA target modules Sequence length Per-device batch size Gradient accumulation Learning rate LR scheduler Max training steps Num train epochs Logging steps Save steps / limit Local epochs (# client epochs) Rounds (# server rounds)
QLoRA 4 true 32 64 q proj, k proj, v proj, o proj 512 4 4 steps (effective batch = 16) 1 × 10−4 cosine 100 1 50 1000 / 10 1 15
Although the dataset size is intentionally small—serving as a safe, public proxy for restricted institutional sources —we actively mitigate the risk of overfitting through our architectural and hyperparameter choices. First, the use of PEFT via QLoRA restricts the trainable weights to a small set of low-rank adapters (with r=32). This significantly reduces the model’s capacity to memorize the training data, acting as a strong regularizer compared to full fine-tuning. Second, the federated training configuration strictly limits local adaptation to a single epoch per communication round across a total of 15 rounds. This strategy prevents the local models from overfitting their limited respective samples, while the central Server’s periodic aggregation enforces a regularizing consensus across the clients. Metrics. We evaluate answer quality using standard Natural Language Generation (NLG) metrics [36], [37], covering recall-oriented overlap, precision-oriented overlap, and seman-
Fig. 3: The GuidaPA user interface.
using a Private Centralized Model (see Figure 1). This configuration represents a conventional training scenario in which all available training data (public docs in this study) from SIGESON and SIDFORS are combined within a single, secure environment under PA control. The goal is to assess how much fine-tuning a general-purpose LLM on domain-specific documentation improves its ability to generate accurate and context-aware responses. This baseline serves as a reference point for evaluating the benefits and potential trade-offs of the federated approach introduced later. Fig. 4: Lexical shift (non-IID quantification) between SIGESON and SIDFORS using HD and JSD for multiple n-grams. tic similarity: • ROUGE-1/2/L: unigram overlap, bigram overlap, and longest common subsequence (recall-oriented). • BLEU-4: 4-gram precision overlap, rewarding fluency while penalizing omissions/extraneous content. • METEOR: synonym/stem-aware matching with wordorder sensitivity, capturing semantic adequacy.
TABLE IV: Centralized evaluation metrics for general-purpose and fine-tuned (on SIDFORS and SIGESON data) approaches. The model used is the one that yields the best results in the federated global results (see Table V). In bold, the highest value of each metric. Model cerbero-7b-openchat (general-purpose) cerbero-7b-openchat (fine-tuned)
ROUGE-1
ROUGE-2
ROUGE-L
BLEU-4
METEOR
41.45
29.97
38.41
26.97
41.83
62.18
55.37
60.09
50.90
64.17
VI. R ESULTS Before analyzing the federated setup, we establish a reference point using the Private Centralized model as the ideal case where all client data is available in a single secure environment. This baseline allows comparison between the federated and centralized settings. We report (i) local metrics on each client’s test set and (ii) global FL metrics as a datasetsize-weighted average across clients. Together, ROUGE measures content coverage, BLEU-4 exactness and fluency, and METEOR semantic similarity and linguistic flexibility.
Table IV reports the centralized evaluation metrics obtained with the best-performing model in the federated experiments (see Table V). The results show a substantial improvement across all metrics when the model is fine-tuned on PA-specific data (SIGESON and SIDFORS) compared to its generalpurpose version. ROUGE and METEOR scores increase by more than 20 points on average, while BLEU-4 rises by nearly 24 points, confirming that fine-tuning allows the model to better capture the specialized terminology and procedural language of the PA domain.
A. The baseline: the Private Centralized Model To address the issue of lack of domain-specific understanding highlighted in Section I-A, we first establish a baseline
B. FL Model This section presents the evaluation of the FL model implemented in GuidaPA (see Figure 1), first analyzing its
performance vs. that of the private centralized baseline, and then considering the performance of the local models running on the clients of the federated architecture. TABLE V: Global evaluation metrics across all clients with FL models. In bold, the highest value of each metric. Model cerbero-7b cerbero-7b-openchat Llama-3.1-8B-Instruct Loquace-7B-Mistral maestrale-chat-v0.4-beta Mistral-7B-Instruct-v0.3 Mistral-7B-v0.1 zefiro-7b-beta-ITA-v0.1
ROUGE-1
ROUGE-2
ROUGE-L
BLEU-4
METEOR
61.14 61.10 45.86 53.10 49.25 53.02 50.30 58.55
53.72 55.77 38.45 46.81 40.90 43.26 42.38 50.97
59.09 59.44 44.83 51.68 46.82 49.91 47.63 56.30
44.30 45.02 29.32 37.63 32.26 34.33 34.53 39.43
62.99 63.94 50.83 54.75 50.05 54.27 53.71 61.57
1) Comparison with the Private Centralized Baseline: We compare cerbero-7b-openchat, the best global federated model (Table V), against its centralized counterpart (Table IV). The comparison shows that federated metrics are close to those achieved under centralized training, indicating that federated fine-tuning can preserve model quality while keeping data local and maintaining data sovereignty. 2) FL Model Performance Evaluation: We report finalround values for comparison. Table V reports the final global metric values obtained at the last communication round for the eight models considered. Overall, Italian instruction-tuned models yield the strongest trade-off across recall-, precision-, and semantic-oriented metrics. These trends align with prior findings that instruction-tuned Italian models are strong on generation-based evaluation, but here we validate them in a PA deployment setting [38]–[40]. 3) Local Model Performance Evaluation: Table VI reports local evaluation metrics for each client (SIGESON and SIDFORS). Overall, most models achieve higher ROUGE, BLEU4, and METEOR on SIGESON, suggesting that SIGESON is more favorable for adaptation, whereas SIDFORS is more challenging. On SIGESON, cerbero-7b achieves the best results across all metrics (e.g., ROUGE-L = 67.16, BLEU-4 = 49.61, METEOR = 67.79), with zefiro-7b-beta-ITA-v0.1 also performing strongly (ROUGE-L = 64.43, METEOR = 65.95). On SIDFORS, cerbero-7b-openchat is the top model, leading all five metrics (ROUGE-1 = 59.45, ROUGE-2 = 53.87, ROUGE-L = 57.78, BLEU-4 = 45.33, METEOR = 63.17), while Mistral-7B-Instruct-v0.3 is the strongest remaining baseline among those reported (e.g., ROUGE-L = 52.59, METEOR = 58.74). These local differences highlight client non-IID data, while the global aggregation (Table V) supports a robust overall model. VII. D ISCUSSION AND I MPLICATIONS Our results show that federated, parameter-efficient adaptation of LLMs can support privacy-preserving conversational AI in PA while retaining strong answer quality. This is relevant because federated LLM deployment is often constrained by client heterogeneity, non-IID data, communication/efficiency limits in PEFT, and application-level security concerns [23], [27], [41]. In the PA setting, these challenges are further shaped by organizational silos and governance requirements.
A. Why it matters for PA Near-centralized quality without centralizing data. Across ROUGE, BLEU-4, and METEOR, federated finetuning achieves performance close to private centralized training, indicating that PA entities can collaborate to build highquality assistants while keeping documentation local. This provides a practical alternative to building centralized data repositories when data sharing is slow, expensive, or constrained by governance. Moderate non-IID supports stable aggregation in institutional domains. By quantifying lexical divergence and relating it to observed behavior (Section V), we find that cross-client differences remain moderate, consistent with PA documentation being produced under shared regulatory and procedural conventions. This structured non-IID setting appears conducive to effective global aggregation, reducing the need for heavy personalization mechanisms often required in highly divergent domains. Security and governance at the application layer. LLM-based systems are vulnerable to prompt injection and instruction-following attacks that can bypass intended policies if authorization is not explicitly enforced [27]. GuidaPA integrates RBAC and source restriction so different user categories (citizens, operators, officers) only access authorized information. Combined with monitoring of answer quality and non-IID divergence, it supports a governance workflow for deployment and maintenance under evolving local data [23], [41]. B. How this study differs from existing work To clarify novelty, GuidaPA differs from prior research in the following ways: • Domain novelty: This is the first federated LLM-based chatbot study grounded in an operational Italian PA context, rather than healthcare, education, or social platforms. • End-to-end deployment focus: Beyond model training, we integrate RBAC, client-side secure preprocessing, and monitoring of both quality and non-IID data as first-class components. • Empirical evidence under realistic PA non-IID data: We quantify cross-client divergence and show that federated tuning remains stable and near-centralized in this institutional domain, complementing recent federated PEFT work that targets extreme non-IID task regimes [24], [42]. Overall, GuidaPA serves both as empirical evidence that federated PEFT can be effective under realistic institutional constraints and as a practical blueprint for privacy-preserving conversational AI in information-intensive public services. VIII. L IMITATIONS This study has the following limitations. First, our empirical evaluation involves two PA platforms (SIGESON and SIDFORS) and uses public manuals/FAQs as a safe proxy for restricted institutional sources; consequently, results primarily reflect adaptation on small, authoritative documentation under moderate non-IID data. In particular, the amount of domain material used for adaptation is limited (approximately 8 and
TABLE VI: Local evaluation metrics for client 1 (SIGESON) and client 2 (SIDFORS). In bold, the highest value of each metric. SIGESON Model cerbero-7b cerbero-7b-openchat Llama-3.1-8B-Instruct Loquace-7B-Mistral maestrale-chat-v0.4-beta Mistral-7B-Instruct-v0.3 Mistral-7B-v0.1 zefiro-7b-beta-ITA-v0.1
SIDFORS
ROUGE-1
ROUGE-2
ROUGE-L
BLEU-4
METEOR
ROUGE-1
ROUGE-2
ROUGE-L
BLEU-4
METEOR
68.90 62.64 53.47 58.91 60.26 52.63 57.39 67.09
62.57 57.54 46.13 52.64 53.29 40.67 49.89 60.45
67.16 60.99 51.64 56.86 57.70 47.43 53.92 64.43
49.61 44.73 31.61 41.51 39.73 29.92 39.64 46.46
67.79 64.67 56.96 59.23 59.47 50.12 59.28 65.95
52.79 59.45 37.66 46.84 37.39 53.45 42.67 49.34
44.19 53.87 30.19 40.53 27.55 46.05 34.30 40.75
50.41 57.78 37.50 46.11 35.09 52.59 40.86 47.54
38.59 45.33 26.85 33.45 24.22 39.07 29.03 31.85
57.81 63.17 44.24 49.93 39.91 58.74 47.72 56.84
31 pages, respectively), which makes this setting more manageable than realistic deployments involving larger, noisier, and more heterogeneous institutional corpora. Therefore, the results should be interpreted mainly as evidence of feasibility rather than as evidence that the proposed framework will scale unchanged to broader PA settings. Second, evaluation is based on offline text-generation metrics (ROUGE, BLEU4, METEOR) on held-out splits and does not include human assessment [43], end-to-end task success, or live deployment measurements. Third, the federated training uses a standard aggregation baseline and does not compare alternative optimization strategies specifically designed to improve robustness under stronger non-IID conditions (e.g., proximal objectives or momentum-based server updates).
IX. C ONCLUSION AND F UTURE W ORK In conclusion, GuidaPA demonstrates the feasibility of a privacy-preserving chatbot for PA through federated finetuning of LLMs on siloed institutional data. The system combines RBAC, performance monitoring, and non-IID data quantification to produce accurate, context-aware, and policycompliant answers. Results show that, despite moderate nonIID data across clients, the federated setup yields robust models for both citizens and officers, with performance comparable to centralized models. Overall, this work shows that FL can balance scalability, data protection, and usability in AI-driven public services. Future work will extend GuidaPA in three directions. (i) Broader data and clients: incorporate additional PA entities and restricted sources (e.g., ticketing archives, officer manuals, curated database extracts) to study stronger nonIID settings, evolving documentation, and multi-agency governance. (ii) Non-IID robustness and optimization: evaluate federated optimizers for non-IID clients—such as FedProx [44] and momentum-based aggregation (e.g., FedAvgM [45])— and assess their impact on convergence and local/global quality under more diverse client distributions. (iii) Stronger privacy/security and evaluation: integrate secure aggregation and/or differential privacy, expand authorization testing via systematic red-teaming, and complement automatic metrics with human and task-based evaluation.
ACKNOWLEDGMENTS Daniel M. Jimenez-Gutierrez was partially supported by PNRR351 TECHNOPOLE – NEXT GEN EU Roma Technopole – Digital Transition, FP2 – Energy transition and digital transition in urban regeneration and construction. Andrea Vitaletti was supported by the project SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union - NextGenerationEU. R EFERENCES [1] F. Barthélemy, N. Ghesquière, N. Loozen, L. Matha, and E. Stani, “Natural language processing for public services,” Publications Office of the European Union, Luxembourg, Technical Report D02.01, Mar. 2022. [Online]. Available: https://interoperable-europe.ec. europa.eu/sites/default/files/inline-files/D02.01 Natural%20Language% 20Processing%20for%20Public%20Services 4.pdf [2] Agenzia per l’Italia Digitale (AGID) and T. per la Trasformazione Digitale, “Piano triennale per l’informatica nella pubblica amministrazione (2024–2026): Capitolo 5 – dati e intelligenza artificiale,” 2024, accessed April 2025. [Online]. Available: https://docs.italia.it/italia/piano-triennale-ict/pianotriennale-ict-doc/ it/2024-2026/capitolo-5 dati-e-intelligenza-artificiale/ intelligenza-artificiale-per-la-pubblica-amministrazione.html [3] European Commission, “European interoperability framework (eif): Promoting seamless public services across borders and sectors,” 2017, cOM(2017) 134 final. [Online]. Available: https://eur-lex.europa.eu/ legal-content/EN/TXT/?uri=CELEX:52017DC0134 [4] A. T. Oyewole, O. B. Adeoye, W. A. Addy, C. C. Okoye, O. C. Ofodile, and C. E. Ugochukwu, “Automating financial reporting with natural language processing: A review and case analysis,” World Journal of Advanced Research and Reviews, vol. 21, no. 3, pp. 575–589, 2024. [5] Y.-C. Tseng, W. Jarupreechachan, and T.-H. Lee, “Understanding the benefits and design of chatbots to meet the healthcare needs of migrant workers,” Proceedings of the ACM on Human-Computer Interaction, vol. 7, no. CSCW2, pp. 1–34, 2023. [6] S. Gökçearslan, C. Tosun, and Z. G. Erdemir, “Benefits, challenges, and methods of artificial intelligence (ai) chatbots in education: A systematic literature review.” International Journal of Technology in Education, vol. 7, no. 1, pp. 19–39, 2024. [7] Repubblica Italiana, “Codice dell’amministrazione digitale (d.lgs. 82/2005),” Codice dell’amministrazione digitale e successive modifiche, 2005, testo coordinato e aggiornamenti disponibili sul portale AgID. [Online]. Available: https://www.agid.gov.it/sites/default/files/ repository files/leggi decreti direttive/dl-7-marzo-2005-82 0.pdf [8] T. Chen, M. Gascó-Hernandez, and M. Esteve, “The adoption and implementation of artificial intelligence chatbots in public organizations: Evidence from u.s. state governments,” The American Review of Public Administration, vol. 54, no. 3, pp. 255–270, 2023, original work published 2024. [Online]. Available: https: //doi.org/10.1177/02750740231200522 [9] R. Dreyling, T. Koppel, T. Tammet, and I. Pappel, “Challenges of generative ai chatbots in public services: An integrative review,” SSRN, Social Science Research Network, Tech. Report 4850714, 2024. [Online]. Available: https://ssrn.com/abstract=4850714
[10] X. Lin, W. Wang, Y. Li, S. Yang, F. Feng, Y. Wei, and T.-S. Chua, “Dataefficient fine-tuning for llm-based recommendation,” in Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, 2024, pp. 365–374. [11] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel et al., “Retrievalaugmented generation for knowledge-intensive nlp tasks,” Advances in neural information processing systems, vol. 33, pp. 9459–9474, 2020. [12] Ministero dell’Economia e delle Finanze, Dipartimento del Tesoro, “Nota tematica n°1: Ai in public settings – status and perspectives,” Ministero dell’Economia e delle Finanze, Dipartimento del Tesoro, Roma, Italia, Nota Tematica 1, Feb. 2024, iSSN 1972-4128. [Online]. Available: https://www.dt.mef.gov.it/it/news/2024/nota tematica 21022024.html [13] G. Bellitti, C. Colasanti, and M. Fedeli, “Trasformazione digitale della pubblica amministrazione: Metodi per l’interoperabilità e lo sviluppo di e-service,” 2023, consulted April 2025. [Online]. Available: https://www.istat.it/wp-content/uploads/2024/01/ Trasformazione-digitale-della-Pubblica-Amministrazione-Ebook.pdf [14] D. M. Jimenez G, D. Solans, M. Heikkila, A. Vitaletti, N. Kourtellis, A. Anagnostopoulos, and I. Chatzigiannakis, “Non-iid data in federated learning: A systematic review with taxonomy, metrics, methods, frameworks and future directions,” arXiv e-prints, pp. arXiv–2411, 2024. [15] X. Ma, J. Zhu, Z. Lin, S. Chen, and Y. Qin, “A state-of-the-art survey on solving non-iid data in federated learning,” Future Generation Computer Systems, vol. 135, pp. 244–258, 2022. [16] A. Piizzi, D. Vavallo, G. Lazzo, S. Dimola, and E. Zazzera, “A natural language processing model for the development of an italian-language chatbot for public administration,” 2024. [17] V. Bellini, G. M. Biancofiore, T. Di Noia, E. Di Sciascio, F. Narducci, and C. Pomo, “Guapp: A conversational agent for job recommendation for the italian public administration,” in 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS). IEEE, 2020, pp. 1–7. [18] I. Cantador, J. Viejo-Tardı́o, M. E. Cortés-Cediel, and M. P. Rodrı́guez Bolı́var, “A chatbot for searching and exploring open data: Implementation and evaluation in e-government,” in Proceedings of the 22nd Annual International Conference on Digital Government Research, 2021, pp. 168–179. [19] A. Androutsopoulou, N. Karacapilidis, E. Loukis, and Y. Charalabidis, “Transforming the communication between citizens and government through ai-guided chatbots,” Government information quarterly, vol. 36, no. 2, pp. 358–367, 2019. [20] A. Ait-Mlouk, S. Alawadi, S. Toor, and A. Hellander, “Fedbot: Enhancing privacy in chatbots with federated learning,” arXiv preprint arXiv:2304.03228, 2023. [21] S. Puppala, I. Hossain, M. J. Alam, and S. Talukder, “Scan: A healthcare personalized chatbot with federated learning based gpt,” in 2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 2024, pp. 1945–1951. [22] S. D’Urso, F. Sciarrone, and M. Temperini, “Boulez: A chatbot-based federated learning system for distance learning,” in 2023 27th International Conference Information Visualisation (IV). IEEE, 2023, pp. 210–215. [23] Y. Yao, J. Zhang, J. Wu, C. Huang, Y. Xia, T. Yu, R. Zhang, S. Kim, R. Rossi, A. Li et al., “Federated large language models: Current progress and future directions,” arXiv preprint arXiv:2409.15723, 2024. [24] Z. Wang, Z. Shen, Y. He, G. Sun, H. Wang, L. Lyu, and A. Li, “Flora: Federated fine-tuning large language models with heterogeneous lowrank adaptations,” Advances in Neural Information Processing Systems, vol. 37, pp. 22 513–22 533, 2024. [25] Ministero delle Imprese e del Made in Italy, “Sidfors: Sistema per dichiarazione fornitura di reti/servizi di comunicazione elettronica,” Portale operativo per autorizzazioni generali in materia di reti e servizi di comunicazione elettronica, MIMIT – DGTEL, 2024. [Online]. Available: https://sidfors.mise.gov.it/ [26] ——, “Sigeson: Sistema di gestione delle reti di radiodiffusione sonora,” Portale operativo per la gestione amministrativa delle reti di radiodiffusione sonora, MIMIT – DGTEL, 2024. [Online]. Available: https://fm-dab.mise.gov.it/ [27] Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and benchmarking prompt injection attacks and defenses,” in 33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 1831–1847. [28] D. M. J. Gutierrez, A. Anagnostopoulos, I. Chatzigiannakis, and A. Vitaletti, “Fedartml: A tool to facilitate the generation of non-iid datasets in
a controlled way to support federated learning research,” IEEE Access, vol. 12, pp. 81 004–81 016, 2024. [29] D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y. Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusmão et al., “Flower: A friendly federated learning research framework,” arXiv preprint arXiv:2007.14390, 2020. [30] L. Moroni, S. Conia, F. Martelli, and R. Navigli, “Towards a more comprehensive evaluation for italian llms,” in Proceedings of the 10th Italian Conference on Computational Linguistics (CLiC-it 2024), 2024, pp. 584–599. [31] R. Orlando, L. Moroni, P.-L. H. Cabot, S. Conia, E. Barba, S. Orlandini, G. Fiameni, and R. Navigli, “Minerva llms: The first family of large language models trained from scratch on italian data,” in Proceedings of the 10th Italian conference on computational linguistics (CLiC-it 2024), 2024, pp. 707–719. [32] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: Stateof-the-art natural language processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Online: Association for Computational Linguistics, Oct. 2020, pp. 38–45. [Online]. Available: https: //www.aclweb.org/anthology/2020.emnlp-demos.6 [33] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022. [34] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,” Advances in neural information processing systems, vol. 36, pp. 10 088–10 115, 2023. [35] Y. Gao, M. R. Scamarcia, J. Fernandez-Marques, M. Naseri, C. S. Ng, D. Stripelis, Z. Li, T. Shen, J. Bai, D. Chen et al., “Flowertune: A crossdomain benchmark for federated fine-tuning of large language models,” arXiv preprint arXiv:2506.02961, 2025. [36] T. Hu and X.-H. Zhou, “Unveiling llm evaluation focused on metrics: Challenges and solutions,” 2024. [Online]. Available: https: //arxiv.org/abs/2404.09135 [37] D. Banerjee, P. Singh, A. Avadhanam, and S. Srivastava, “Benchmarking llm powered chatbots: Methods and metrics,” 2023. [Online]. Available: https://arxiv.org/abs/2308.04624 [38] L. Ragazzi, G. Moro, S. Guidi, and G. Frisoni, “Lawsuit: a large expertwritten summarization dataset of italian constitutional court verdicts,” Artificial Intelligence and Law, pp. 1–37, 2024. [39] G. Sarti and M. Nissim, “It5: Text-to-text pretraining for italian language understanding and generation,” arXiv preprint arXiv:2203.03759, 2022. [40] M. P. Priola, “Addressing hallucinations with rag and nmiss in italian healthcare llm chatbots,” arXiv preprint arXiv:2412.04235, 2024. [41] F. Piccialli, D. Chiaro, P. Qi, V. Bellandi, and E. Damiani, “Federated and edge learning for large language models,” Information Fusion, vol. 117, p. 102840, 2025. [42] J. Bai, D. Chen, B. Qian, L. Yao, and Y. Li, “Federated fine-tuning of large language models under heterogeneous tasks and client resources,” Advances in Neural Information Processing Systems, vol. 37, pp. 14 457–14 483, 2024. [43] S. Chaudhari, P. Aggarwal, V. Murahari, T. Rajpurohit, A. Kalyan, K. Narasimhan, A. Deshpande, and B. Castro da Silva, “Rlhf deciphered: A critical analysis of reinforcement learning from human feedback for llms,” ACM Computing Surveys, 2024. [44] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine learning and systems, vol. 2, pp. 429–450, 2020. [45] T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the effects of nonidentical data distribution for federated visual classification,” arXiv preprint arXiv:1909.06335, 2019.