Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions

arXiv:2608.04375v1 [cs.CR] 5 Aug 2026

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions JUNJIE XIONG∗† , Missouri University of Science and Technology, U.S. ZHENGYUAN JIANG∗ , University of South Florida, U.S. XIAORAN XU∗ , University of South Florida, U.S. CHI ZHANG∗ , University of South Florida, U.S. CHANGJIA ZHU, University of South Florida, U.S. NING WANG, University of South Florida, U.S. MINGKUI WEI, George Mason University, U.S. ZHUO LU, University of South Florida, U.S. YAO LIU, University of South Florida, U.S. LINGYAO LI † , University of South Florida, U.S. Large Language Models (LLMs) have emerged as powerful tools that impact information integrity on social media platforms. This comprehensive review examines the dual role of LLMs in both facilitating and mitigating various information integrity challenges, including misinformation, disinformation, fake news, social bots, and privacy concerns. We conduct a comprehensive review of the literature from 2019 to 2024, screening 1048 studies and performing an in-depth analysis of 215 representative papers. This systematic approach allows us to identify key patterns in how LLMs influence the information security in social media ecosystems. Through a systematic analysis of papers from multiple databases, our findings reveal that while LLMs can enhance detection capabilities for malicious content and enable sophisticated defense mechanisms, they simultaneously pose risks by enabling the generation of highly convincing, deceptive content. We categorize and analyze the potential and challenges across different dimensions of information integrity, examining technical capabilities, ethical implications, and privacy concerns. The study demonstrates critical gaps in current approaches, particularly in cross-lingual detection, real-time monitoring, and privacy-preserving implementations. We conclude by proposing future research directions and recommendations for stakeholders to leverage LLMs while mitigating risks in social media information integrity. CCS Concepts: • Security and privacy; • Computing methodologies → Artificial intelligence; Additional Key Words and Phrases: Large Language Models, Social Media, Security Implications, Misinformation, Disinformation, Fake News, LLM-enhanced Bots, Privacy

1

Introduction

Social media platforms have become indispensable tools for modern communication, connecting users around the world and facilitating real-time information sharing. However, these digital platforms remain vulnerable to critical information security threats, including misinformation, disinformation, and the propagation of biased or toxic content [67, 178]. These vulnerabilities not only erode public trust but also present substantial challenges in maintaining the integrity of digital systems. The growing risks associated with the dissemination of false information and cyber threats underscore the urgent need to safeguard online environments [181]. † Corresponding author.

Authors’ Contact Information: Junjie Xiong∗† , Missouri University of Science and Technology, Rolla, U.S., Junjiexiong@ mst.edu; Zhengyuan Jiang∗ , University of South Florida, Tampa, U.S., jz1@usf.edu; Xiaoran Xu∗ , University of South Florida, Tampa, U.S., xiaoranxu@usf.edu; Chi Zhang∗ , University of South Florida, Tampa, U.S., chiz@usf.edu; Changjia Zhu, University of South Florida, Tampa, U.S., changjiaz@usf.edu; Ning Wang, University of South Florida, Tampa, U.S., ningw@usf.edu; Mingkui Wei, George Mason University, Fairfax, U.S., mwei2@gmu.edu; Zhuo Lu, University of South Florida, Tampa, U.S., zhuolu@usf.edu; Yao Liu, University of South Florida, Tampa, U.S., yliu21@usf.edu; Lingyao Li † , University of South Florida, Tampa, U.S., lingyaol@usf.edu.

, Vol. 1, No. 1, Article . Publication date: August 2026.

2

Xiong et al.

Large Language Models (LLMs) have rapidly become embedded in today’s digital platforms. These AI systems, exemplified by GPT-style models, drive a range of applications from conversational agents to intelligent digital assistants and content creation tools. Social media platforms increasingly leverage LLM-driven bots to engage users or automate customer service, while writing assistants use them to generate articles, marketing copy, and code Lu et al. [113], Zhou et al. [213]. The integration of LLMs is reshaping how information is produced and consumed in digital systems. On the positive side, LLMs have shown potential in detecting and mitigating harmful content [99, 199]. LLM-powered applications can also help moderate content and flag policy violations at scale, complementing human moderators in keeping online communities civil [56, 73]. These applications show that LLMs carry significant implications for the quality of public discourse and the security of digital environments. Despite their capabilities, the integration of LLMs can also impact information integrity. A major concern is their tendency to “hallucinate,” generating text that appears authentic and authoritative but may be false or misleading [34, 158]. For example, recent studies show that LLMs can produce election-related misinformation that is almost indistinguishable from human-written content [20, 169]. Additionally, they have been known to generate misinformation or fake news, including hateful or offensive language, particularly when trained on toxic datasets or manipulated through jailbreak attacks [73, 106]. High-profile incidents, such as an AI chatbot coerced into spreading racist and offensive messages, illustrate this risk. Moreover, because LLMs learn from vast internet datasets, they can internalize and reinforce societal biases, associating certain groups with negative stereotypes and potentially harming marginalized communities [50, 74, 106]. When combined with social media’s amplification mechanisms, LLM-generated misinformation or harmful content can spread rapidly, misleading users on critical topics and further complicating efforts to maintain the integrity of digital discourse. To mitigate these issues, researchers and practitioners have proposed a variety of mitigation strategies to align LLM behavior with ethical and safety standards [74, 79]. One typical approach is the reinforcement learning from human feedback (RLHF) alignment pipeline, which fine-tunes models using human input to reinforce desirable behaviors Bai et al. [14], Kaufmann et al. [87]. By training on examples of preferred behaviors and using human judgments to reward desirable answers, LLMs can be steered toward more helpful and harmless responses. In addition, researchers have developed data curation pipelines to filter out misinformation from training corpora [170]. Such dataset-level interventions can preempt many issues before the model engages with users. Another typical strategy is the adversarial testing [56]. For example, adversarial prompts are Misinformation and Disinformation Generation

Prevention

Detection

Fake News Baseline LLM

Adapted LLM

LLM-powered applications

Simulation

Detection

Social Bot Social Media Information Integrity User

Embodied AI

Behavior Analysis

Social Media

User Privacy Information leakage Privacy preserving

Fig. 1. The Integration and Impact of LLMs in Social Media Information Integrity. , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions

3

designed to find weaknesses. By learning from these adversarial cases (e.g., either via additional fine-tuning or adjusting the model’s safety filters), developers can strengthen the model against malicious exploitation. These typical strategies can help ensure that known failure modes are addressed before online users encounter them. Even though recent research into LLM security, bias mitigation, and ethical AI has grown exponentially, existing literature surveys often prioritize static assessments of general model capabilities, leaving a significant void regarding the dynamic and interconnected risks within social media ecosystems. Specifically, leading surveys on adversarial security systematically categorize technical vulnerabilities like prompt injection [41] but overlook the “asymmetric co-evolutionary dynamics” inherent to social media environments [47]. For example, the attackers can iterate content mutations at low costs while defenders operate under high verification burdens and user-trust constraints. Similarly, comprehensive reviews of bias and fairness offer robust taxonomies for evaluation metrics [59] but primarily focus on “point-in-time” analyses, failing to capture how biases evolve in real-time alongside shifting social trends and user interactions [23]. These works also exhibit significant gaps in cross-lingual and cross-cultural robustness, often centering on Anglocentric contexts [51] while neglecting the specific harms faced by marginalized communities in diverse linguistic settings [1]. Furthermore, safety evaluation frameworks like “The Scales of Justitia” [107] typically focus on whether models follow instructions, yet they fail to address systemic “responsibility gaps” [154]—the ambiguous allocation of moral and legal accountability between developers, platform operators, and users, or the potential for second-order harms such as the “chilling effect” of automated moderation[18]. Our study addresses these systematic gaps by establishing a multi-dimensional framework that integrates the content layer (information disorder), agent layer (social bots), and infrastructure layer (privacy and ethics), redefining information integrity as a comprehensive “evidence-building” process designed to counter multimodal attacks [159] and resolve complex multi-stakeholder responsibility chains, motivating the following questions: • RQ1. What is the potential of LLMs in enhancing social media information integrity? • RQ2. How do LLMs challenge the detection and mitigation of information integrity issues? • RQ3. To what extent do ethical and security implications emerge from LLMs’ role in information integrity? In addressing these research questions, we propose a multi-dimensional framework for social media information integrity in the LLM era, systematically analyze LLM-driven potentials and risks, and outline future research and governance directions. 2 2.1

Backgrounds Concepts and Threat Model

In this survey, we treat information integrity as a multi-dimensional target rather than a self-evident synonym for factual accuracy. In social-media environments, integrity encompasses (i) veracity, namely whether claims are factually correct, (ii) grounding and sourcing, meaning that information is supported by traceable and credible evidence, (iii) authenticity of actors and context, including the absence of coordinated manipulation, impersonation, or synthetic amplification, and (iv) reliability under adversarial framing, capturing misleading presentation or selective distortion even when content is partially true. Correspondingly, integrity can be operationalized at a systemic level through measurable indicators such as exposure-weighted accuracy, calibration and uncertainty awareness, provenance and attribution quality, rates of coordinated inauthentic behavior, and downstream impacts on public trust and belief formation. This explicit operational framing clarifies the target variable for evaluating both threats and mitigation strategies throughout the survey. Information , Vol. 1, No. 1, Article . Publication date: August 2026.

4

Xiong et al.

security refers to protecting digital assets and systems under the confidentiality, integrity, and availability, including identity, access control, and system abuse. Safety refers to preventing harmful outcomes from model behavior or deployment, including misuse and unintended impacts. These concepts overlap but are not interchangeable: we focus on information integrity as the primary construct while treating security and safety as enabling conditions and adjacent risk domains. We study LLM-enabled threats in which attackers use automation to generate, personalize, and iteratively mutate deceptive content while coordinating large-scale dissemination and probing moderation systems to evade detection. Actors may range from operators to scammers or ideologically motivated groups. Defenders operate under resource and latency constraints and face asymmetric error costs: false negatives allow harms to scale, whereas false positives erode trust. We evaluate integrity using measurable targets such as reduced exposure to deceptive content, campaign disruption, and privacy protection. Users act as both amplifiers of harm and contributors to resilience, underscoring the need for both technical and user interventions. 2.2

What are the information security issues in social media?

The challenge of false information has become increasingly critical as social media platforms emerge as primary news sources, with reports indicating that 54% of US adults now obtain their news through platforms like Facebook and YouTube [27]. Our analysis of social media security issues reveals three interconnected categories that represent the primary challenges in maintaining information integrity. First, information disorder—encompassing misinformation, disinformation, and fake news—represents the content-level challenges. This category includes both unintentional spread of incorrect information and deliberate manipulation of facts, with studies showing that pseudoscience, conspiracy theories, lies, and deepfakes constitute significant threats [8]. Second, social bots represent the agent-level challenges, serving as primary vectors for amplifying and automating the spread of information integrity issues. These automated systems, enhanced by LLMs, can now generate highly convincing content while mimicking human behavior patterns [170]. Third, user privacy concerns represent the infrastructure-level challenges, where the need to protect personal information intersects with the requirements for effective content moderation and bot detection [129]. The advent of LLMs has significantly amplified these challenges across all three categories. For example, models like GPT-3 can generate highly convincing false information that humans find both comprehensible and compelling [169]. This capability becomes particularly concerning in critical domains such as health-related topics [21, 160], even in the presence of platform safeguards such as TikTok’s misinformation guard. We provide the complete definitions of these social media information integrity concepts in the Table 1. We also note that although we structure the survey around three primary domains, this taxonomy serves as a high-level organizational framework rather than a rigid or mutually exclusive partition, as cross-cutting dimensions such as multimodal analysis, cross-lingual evaluation, and low-resource settings span multiple categories and are therefore discussed within relevant sections. Information Disorder, defined in Shu et al. [166], Wardle and Derakhshan [190], includes false information intended to harm and factual information being used to manipulate. In the context of social media, we focus on the harmful aspects, known as misinformation, disinformation, and fake news. LLM can further complicate the information ecosystem. On one hand, LLMs can be exploited to automatically generate misleading content, impersonate users, or flood platforms with synthetic narratives, thereby intensifying the spread of information disorder. On the other hand, these models also offer potential countermeasures: they can be fine-tuned to detect falsehoods, identify manipulation patterns, and assist in fact-checking at scale. Misinformation and Disinformation represent two distinct but related threats to information integrity. Misinformation refers to false information shared without malicious intent [190], while disinformation involves deliberately , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions

5

created false content intended to cause harm [190]. The United Nations High Commissioner for Refugees (UNHCR) further distinguishes these two concepts by emphasizing that disinformation includes specific malicious content types such as hoaxes and propaganda [183]. Table 1. Definitions and Sources of Key Information Integrity Concepts in Social Media. Category

Author and Year

Definition

Origin

Misinformation

Altay et al. (2023)[8]

False and misleading information.

Academia

Wardle et al.(2017)[190]

Information that is false but not created with the intention of causing harm.

Academia

UNHCR (2021)[183]

Misinformation is false or inaccurate in- NGO formation. Examples include rumors, insults, and pranks.

Wardle et al.(2017)[190]

Information that is false and is knowingly shared to cause harm.

Academia

UNHCR (2021)[183]

Disinformation is deliberate and includes malicious content such as hoaxes, spear phishing, and propaganda. It spreads fear and suspicion among the population.

NGO

Cooke et al.(2017) [39]

False and often sensational information Academic disseminated under the guise of news reporting, yet the term has evolved over time and has become synonymous with the spread of false information

Allcott et al. (2017) [6]

News articles that are intentionally and verifiably false and could mislead readers.

Staab et al.(2023) [170] Lyu et al.(2023) [115]

Social bots capable of inferring and uti- Academic lizing multimodal capabilities, effectively processing and generating both textual and visual content.

Cloudflare (2025) [35]

Social bots are automated programs that mimic human users, operating partially or fully on their own, with many being used for deceptive or harmful purposes.

Industry

Helen et al.(2019) [129]

The right of users to control information flows and disclosures about themselves according to contextual norms and social expectations.

Academia

Facebook (2023) [120]

Providing users transparency and practi- Industry cal control over how their personal data is collected, used, and shared, consistent with regulatory compliance.

Disinformation

Fake News

Social Bot

User Privacy

Academic

Fake News has overlaps with misinformation and disinformation, but is distinguished through three key characteristics [6, 39]: its deliberate mimicry of legitimate news media formatting and style; its exploitation of established news distribution channels; and its intent to deceive by leveraging the , Vol. 1, No. 1, Article . Publication date: August 2026.

6

Xiong et al.

credibility traditionally associated with journalism. Fake news refers to objectively false information presented in various forms, including news articles, public statements, speeches, and social media posts [214]. Its core feature lies in the deviation from objective facts, regardless of the publisher’s motivation [186]. Whether created for malicious deception, entertainment, satire, or through unintentional dissemination, any content that contradicts factual truth falls under this category. This phenomenon has a long history of influencing political views and social discourse [98]. Social bots are automated programs that mimic human interaction on social media platforms, and their capabilities have significantly advanced with the integration of LLMs. While traditional bots relied on predefined scripts and rule-based automation [63, 70], LLM-enhanced bots leverage large-scale language models to generate highly contextual, personalized, and human-like content [100, 199]. These systems demonstrate sophisticated camouflage abilities, adaptive behavior through interaction learning [56], and multimodal processing of text and images [115, 170]. They can infer sensitive personal attributes with high accuracy and craft convincing phishing or social engineering content [37], substantially increasing the scale and subtlety of automated manipulation. At the same time, these advancements demonstrate opportunities to employ LLMs in more adaptive and context-aware bot detection and moderation frameworks [64]. User privacy encompasses individuals’ rights to control how their information flows within specific social contexts [129], extending beyond industry-focused notions of transparency and regulatory compliance [120]. The integration of LLMs into social media platforms introduces heightened privacy risks through powerful inference and memorization capabilities. Studies show that LLMs can deduce sensitive attributes from pseudonymized content and reconstruct detailed user profiles from fragmented data [170], while memorization and extraction attacks expose risks of training data leakage [46, 85]. Cross-platform linkage tools further amplify these risks by automatically connecting user identities across services [180], and model inversion or membership inference attacks can reveal participation in training datasets [25, 207]. These vulnerabilities become particularly concerning in real-time social media environments, where behavioral traces may unintentionally disclose personal habits, health status, or relationships. 2.3

Why does LLM raise information security issues in social media?

The rapid evolution of social media platforms has fundamentally transformed how information is created, shared, and consumed online [67, 178]. Within this landscape, LLMs have emerged as transformative technologies that significantly impact information security dynamics [170, 175]. Their deep integration into social media systems, from content moderation to user interaction, has created new opportunities and challenges that demand careful examination [25, 44]. This involvement stems from several fundamental factors: Technical Capabilities and Limitations: LLMs’ advanced natural language processing capabilities enable them to generate highly convincing content that can be indistinguishable from human-written text [199], while exhibiting tendencies to hallucinate [158]. This dual-use characteristic is reflected in prior studies that explore LLMs both as detection tools and as enablers of more evasive attacks. Under specific benchmarks and controlled experimental settings, some works report high detection accuracy (e.g., exceeding 98%) [64], while others document non-trivial evasion rates for LLM-enhanced bots (e.g., up to 29.6%) [56]. LLMs exhibit a distinctive dual-use profile: the same generative and reasoning capabilities that enable assistance, moderation, and detection can also be repurposed for scalable manipulation, evasion, and privacy-invasive inference. Rather than a symmetric “technological arms race”, the attacker and defender interaction is more accurately characterized as an asymmetric co-evolutionary system. Offense can often iterate at low marginal cost through automated content mutation, rapid probing of detection boundaries, and reuse of attack templates across platforms, while defense must absorb higher verification costs, manage , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions

7

false positives that erode user trust, and operate under latency and computational constraints. These asymmetries induce selective pressures: attackers are incentivized to maximize reach and evasion under limited risk, whereas defenders must balance harm reduction against robustness, transparency, and user experience. Scale and Privacy Implications: The integration of LLMs enables automated content generation at unprecedented scales [199], while raising substantial privacy concerns through their sophisticated inference capabilities. These models can extract and correlate personal information from seemingly unrelated data points, potentially revealing sensitive attributes such as location, demographics, and behavioral patterns [170]. The risk is further amplified by LLMs’ ability to process and analyze vast amounts of historical user data, where patterns in content creation and interaction histories may inadvertently expose private information. This creates fundamental tensions between leveraging LLMs’ powerful functionalities for content generation and ensuring robust privacy protection [139]. These characteristics make LLMs central to both the problems and solutions in social media information security. Understanding these fundamental factors is crucial for developing effective strategies to harness LLMs’ potential while mitigating their risks. In the following sections, we present a comprehensive scope review of both the opportunities and challenges that emerge from LLMs’ integration into social media information security systems. In this paper, we identify three critical areas that warrant in-depth analysis in social media security: information disorder, social bots, and privacy concerns. The first area examines misinformation and disinformation detection using enhanced LLM capabilities and automated systems. The second area focuses on LLM-enhanced detection techniques for identifying automated accounts and coordinated networks. The third area addresses privacy-preserving mechanisms while ensuring information integrity. These interconnected areas represent the key challenges where LLMs present both risks and opportunities in social media environments.

3.1

Data, Methods and Initial Findings Identification of new studies via databases and registers

Data preparation Identification

3

Records identified from openalex: Microsoft Academic, ORCID, DOAJ, PubMed, The ISSN International Centre, Internet Archive, and arxiv (n = 1048)

Records removed before screening: Duplicate records removed (n = 92) Records marked as ineligible by automation tools (n = 140) Records removed for other reasons (n = 5)

Screening

Records are retrieved via OpenAlex [131], which aggregates scholarly metadata primarily from Microsoft Academic Graph/Open Academic Graph and Crossref, with additional contributions from ORCID, DOAJ, PubMed, the ISSN International Centre, the Internet Archive, and arXiv. Given the rapid pace of LLM and information integrity research, we include a limited number of arXiv preprints to capture emerging results that have not yet appeared in archival venues. We explicitly distinguish preprint-only studies from peer-reviewed conference and journal publications and treat their findings as preliminary. We double-check cited preprints for updated peerreviewed versions and revised 14 references accordingly. To identify relevant literature, we query OpenAlex using keywords from three core categories: “social media,” “online platform,” and “large language model.” The full data preparation and filtering process is illustrated Fig. 2. PRISMA flow of the study screening. in Figure 2. We retrieve 1,048 papers matching our keyword criteria during this process. After removing duplicates—both identical titles and redundant formats (e.g., preprint and published versions of the same Records screened (n = 811)

Records match key questions: Q1 Facilitate information integrity Q2 Mitigate information integrity Q3 Ethical & security implications (n = 811)

Reports sought for retrieval (n = 804)

Reports not retrieved (n = 7)

Reports assessed for eligibility (n = 215)

Reports excluded: Reason 1 Filter 'LSTM'/'SVM' out-dated model (n = 40) Reason 2 Language requirement (n = 8) Reason 3 Insufficient methodological rigor or incomplete evaluation (n = 541)

Included

New studies included in review (n = 215) Reports of new included studies (n = 215)

Total studies included in review (n = 215) Reports of total included studies (n = 215)

, Vol. 1, No. 1, Article . Publication date: August 2026.

8

Xiong et al.

work)—we retain 956 unique records. Next, we apply a multi-stage screening process to ensure the remaining records align with the focus of our study. This involved three key filters: (1) Topic Matching: we retain papers addressing our core themes, including how LLMs facilitate information integrity, mitigate information integrity challenges, and ethical or security implications on social media field; (2) Outdated Model Filtering: is applied to exclude studies in which traditional machine learning or early deep learning architectures (e.g., SVM, standalone CNNs, or standalone LSTMs) served as the primary detection framework. Hybrid approaches that incorporated LLMs or Transformer-based models as the core component, while using auxiliary modules such as LSTMs for sequence aggregation or feature refinement, were retained, as these works fall within the scope of LLM-centered research; (3) Paper Quality Verification: we select papers that met our criteria of novelty and technical impact in addressing capabilities or challenges of LLMs in social media. 3.2

Data Description

To comprehensively understand the current landscape of LLMs in social media integrity research, our analysis focuses on four key dimensions: model adoption (types and scales of LLMs being utilized), architectural evolution (development in model structures and capabilities), research focus areas (primary integrity challenges being addressed), and platform influence (impact across different social media platforms). These dimensions are selected to capture both the technical advancement of LLM applications and practical impacts. Model Adoption and Evolution: The model popularity analysis (Figure 3) indicates that a large proportion of the sampled studies employ GPT-series models in social Fig. 3. Model popularity. Visualization of media integrity research. This preference may be attrib- the yearly prevalence of different LLMs. uted to several factors: GPT models’ accessibility through well-documented APIs, their extensive pre-training on diverse datasets, and their demonstrated capabilities in language understanding tasks. The growing adoption of alternative models like LLaMA and Bloom might reflect researchers’ interest in open-source alternatives and specialized architectures that can be fine-tuned for specific integrity tasks. This diversification in model suggests an evolving research landscape where different models serve complementary roles in addressing various aspects of information integrity. Architectural Trends: The temporal distribution of model architectures shown in Figure 4 indicates an overall increase in reported LLM usage across encoder-only, decoder-only, and encoder–decoder paradigms within the sampled studies. Decoder-only architectures, such as the GPT series, appear frequently in recent work, likely reflecting their versatility in both content generation and analytical tasks. Encoder-only models (e.g., BERT and RoBERTa) continue to be represented, particularly in classification- and detection-oriented settings, where computational efficiency remains important. Encoder–decoder architectures (e.g., T5 and BART) are also present, offering hybrid capabilities that support tasks requiring both structured understanding and controlled generation. These architectural patterns should be interpreted as descriptive observations drawn from the analyzed corpus rather than as evidence of causal shifts in modeling practice. Research Focus Distribution: The temporal analysis of research themes in Figure 5 highlights changes in the relative emphasis of different integrity-related topics over time within the sampled literature. While longstanding topics such as misinformation detection and content moderation continue to be widely studied, recent years show increased attention to emerging challenges, including advanced bot detection, multimodal deepfake analysis, and privacy-aware integrity mechanisms. , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions

9

These observed patterns reflect the evolving scope of integrity threats and corresponding research interests, though they are descriptive in nature and may be influenced by dataset availability, platform access, and broader shifts in the research ecosystem. Platform Distribution: The platform-level distribution illustrated in Figure 6 summarizes how LLM-based integrity research is represented across different social media platforms within the sampled studies. Twitter (now X) is the most frequently represented platform, which may be related to the availability of historical datasets and comparatively accessible APIs for studying misinformation and automated behavior. Other platforms, including Facebook and Reddit, maintain a steady presence, while platforms such as Discord and Mastodon appear less frequently but reflect emerging research interest in decentralized or community-driven social networks. These distributions should be interpreted in light of differences in data accessibility and platform policies rather than as comprehensive indicators of real-world platform impact. 3.3

Benchmark Datasets and Evaluation Metrics

Evaluation practices in LLM-based security research are shaped by the heterogeneity of tasks, ranging from misinformation detection and bot identification to contextual privacy auditing. Thus, there is no single universal benchmark or evaluation metric. Instead, prior work has converged on a set of datasets and indicators that define the evaluation landscape. This subsection summarizes the benchmark datasets and evaluation metrics. Information Disorder: Benchmark selection often matches the granularity of misinformation modeling. Claim-level veracity classification is commonly evaluated using LIAR [188], which provides short political statements with fine-grained truthfulness labels and stan- Fig. 4. Architectural trend. Temporal breakdardized splits, making it a frequent testbed for both tradi- down of encoder-only, decoder-only, and tional and prompting-based LLM baselines [60, 164, 165]. encoder-decoder LLM, reflecting the shiftArticle-level or discourse-level detection is often evalu- ing architectural choices. ated on FakeNewsNet [164], which combines textual content with social engagement and propagation metadata, enabling evaluation of retrieval- or context-augmented models but introducing variability in pre-processing and feature usage. For domain-specific settings, health misinformation studies frequently adopt CoAID [40], where robustness and domain transferability become central concerns. As misinformation increasingly mixes text and images, multimodal benchmarks have become important [108, 134]. Fakeddit is widely used due to its largescale paired text-image data and multiple labeling granularities [127], while newer datasets such as Mocheg emphasize cross-modal consistency rather than isolated classification [201]. Across these studies, metrics typically prioritize precision recall based indicators: accuracy is re- Fig. 5. Evolving research topics. Display of ported, but macro-F1 is often emphasized to address class the annual distribution of trending topics in imbalance [161, 163, 174]. When fact-checking includes social media integrity research. rationale generation, works commonly add generation-oriented scores such as ROUGE [76]. , Vol. 1, No. 1, Article . Publication date: August 2026.

10

Xiong et al.

Fig. 6. Temporal Analysis of LLM-based Social Media Integrity Research (2019-2024) Platform-specific research distribution visualized through bubble charts, where bubble size indicates study frequency. The visualization reveals temporal trends and platform-specific focus in LLM-driven integrity research.

Social Bot: Bot detection benchmarking is relatively standardized. TwiBot-20 [55] and TwiBot22 [54] are widely adopted as they provide large-scale labels together with behavior traces and graph-structured signals. Given strong class imbalance and deployment sensitivity, studies typically report precision, recall, and F1, often supplemented by AUC for ranking performance; accuracy alone is generally insufficient because small false-positive rates can translate into disproportionate moderation errors. Beyond correctness, some work also discusses reliability, using calibration metrics to reflect risks in threshold-based or human-in-the-loop pipelines. User Privacy: Privacy and contextual integrity studies depart from standard label prediction. Rather than optimizing accuracy/F1, they assess whether an LLM discloses sensitive information in contexts where disclosure violates normative expectations [24, 43, 102]. Benchmarks such as CONFAIDE define structured interaction scenarios and measure privacy failures as leakage or attack success rates under standardized prompting [179]. Importantly, these evaluations often report utility–safety trade-offs: mitigations are assessed by both leakage reduction and the degree of performance degradation on benign prompts, reflecting that success is defined by minimizing harm rather than maximizing predictive accuracy alone. 4

Structural Landscape and Emerging Challenges

As illustrated in Figure 7, the LLM-enabled social media integrity pipeline can be organized into four representative subdomain workflows, spanning misinformation detection, synthetic fake news analysis, bot governance, and privacy-preserving mechanisms. In the following, we examine these domains by distinguishing their emerging opportunities from their systemic challenges. 4.1

Landscape: Opportunities in Social Media Information Integrity

LLMs have delivered multi-dimensional gains for social-media information integrity, as illustrated in Figure 8. When enhanced through domain-specific fine-tuning or retrieval-augmented generation (RAG), they achieve significant improvements in multilingual misinformation screening recall, reduce fact-checking latency through automated source verification, identify covert bot networks through pattern analysis, and generate privacy-aware prompts that strengthen user media-literacy. At the same time, careful system design, such as contextual-integrity guards, federated fine-tuning, and cost-aware routing, can keep these advances affordable and policy-compliant. In what follows, we summarize the landscape of LLM opportunities across key dimensions of social media information integrity. , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 11

a

c Veracity

Verify

“Drinking hot water can cure COVID-19”

Explanation “According to the evidences … ”

LLaVA

Toxicity

Evidence

“Go back to your country.”

CLIP

Classifier

Image + Text

Veracity Support

Knowledge

Refute

User Profiles

Less Information

Toxicity Toxic

Memory

Perception

Perception

Goal Posts Comments

Misinformation and Disinformation

Server Aggregation

Llama

Action List

Goal

Action 2

Information Security

Relay

Prompt

Chatgpt

Goal Tasks …

Privacy and Information Intergrity

Role Actions

Action 1

d

Historical comments

Bot Agent

Social Network

NonToxic

Historical posts Memory

LLMs

Create

Post

Repost

Comment

Like

…

Decision Making

LLM-enhanced Bot Fake news SFT

Autograd Bridge

Client 1

Itermediate Results

RLHF

Autograd Bridge

Client 2

Privacy-perserving

Fake news

PEFT Client 3

rumors

Fine-tune method

… Llama Chatgpt

Forward pass Back propagation

b

Supervised Fine-tuning (SFT) Reinforcement Learning from Human Feedback (RLHF) Parameter-Efficient Fine-Tuning (PEFT) Fine-Tuning

Rumor detection LLM

…

Classifier

Prompt Gradients

Claude

Stemming Response

…

Real news

LLMs

Fig. 7. This figure illustrates four representative subdomain workflows (a–d) in information security research. (a)Misinformation and Disinformation: Lee et al. [99] propose a multimodal knowledge transfer approach using LLaVA and CLIP to improve fact-checking and rumor detection. (b)Fake News: Lai et al. [96] develops automated detection systems for identifying and combating synthetic fake news content. (c)LLM-enhanced Bot Detection: Liu et al. [109] demonstrates advanced techniques for identifying automated accounts and coordinated bot networks. (d)Privacy and Information Integrity: Su et al. [173] explores privacy-preserving mechanisms for protecting user data while maintaining information integrity in social media environments.

Detect & Moderate Misinformation. The proliferation of misinformation and disinformation on social media platforms represents one of the most pressing challenges in our digital age, with recent studies revealing that false information propagates as rapidly as truthful content [67, 178]. Within this landscape, the advent of LLM marks a critical turning point, introducing a complex dynamic where these models serve both as potential solutions and possible sources of misinformation. While LLMs demonstrate remarkable capabilities in detecting inconsistencies across multiple languages and formats [49, 195, 211], their sophisticated generative abilities simultaneously present unprecedented challenges by producing highly convincing synthetic content [4, 20, 158, 169, 198]. This dual nature of LLMs in the information integrity landscape requires careful examination of both their potential benefits and risks in social media environments, particularly as automated accounts continue to amplify the spread of both authentic and false information. The challenge of protecting sensitive data while maintaining model utility presents a fundamental tension in LLM deployments. Cai et al. [24] identify critical challenges in balancing analytical capabilities with privacy protection, particularly in mental health discussions. Traditional privacy-preserving techniques prove increasingly inadequate, as Patsakis and Lykousas [135] and Mattern et al. [118] show how LLMs can still infer user traits through subtle linguistic cues even after standard deidentification. This has led to exploration of more robust approaches, with Huang et al. [78], Li et al. [102], Wiseman et al. [191] investigating differential privacy strategies, while Su et al. [173] explores federated learning solutions for privacy-aware model updates. As Martin et al. [117] and Dickson and McCauley [43] emphasize, there is an urgent need to develop stronger ethical guidelines and resolve the fundamental tension between analytical utility and privacy preservation. Emerging evidence suggests that LLM-enhanced detectors may improve recall by 6 to 10 percentage points over baseline BERT methods, though these results remain preliminary and require validation across diverse datasets and languages. Koka et al. [94], Lazer et al. [98] report such gains on hate detector benchmark and misinformation detector benchmark, but the generalizability to , Vol. 1, No. 1, Article . Publication date: August 2026.

12

Xiong et al.

real-world social media (such as class distributions, temporal drift, and adversarial attacks) remains an open question. By modeling narrative intent, they distinguish inadvertent misinformation from deliberate disinformation, extending Wardle’s conceptual split [190]. This capability allows the same model to recognize both careless sharing and coordinated campaigns in diverse linguistic and cultural settings, even in low-resource languages [133]. In the context of this survey, "coordinated networks" and "campaigns" are operationalized through specific detection methodologies found in the technical literature: For the community detection, to identify clusters of coordinated accounts, graph-based studies typically employ modularity optimization algorithms. The Louvain method is classically used to detect dense subgraphs, while recent works[28, 54, 159, 199] increasingly adopt the Leiden algorithm to guarantee connected communities and improve clustering speed in largescale social graphs. For the coordination heuristics: We distinguish between two primary detection signals: (1) Link-farm heuristics[73, 100, 180], which identify coordination through deterministic patterns such as shared URLs, identical posting times, or co-retweet structures; and (2) Probabilistic cascade models, which utilize causal diffusion analysis [83, 110, 164] to infer latent coordination by modeling the probability of information spreading between nodes, distinguishing organic viral spread from artificial amplification. Recent audits further demonstrate that LLMs outperform traditional filters in identifying nuanced health and political falsehoods, particularly those embedded in contextual ambiguity, or cross-domain narratives that rule-based and conventional classifiers systematically overlook [56, 73].

Huntsman et al. (2024) Sangher et al. (2024) Shah et al. (2024) Sangher et al. (2024) Cooke et al.(2017) Allcott et al. (2017) Atzori et al. (2024) Lin et al. (2024)

Feng et al. (2024) Shafee et al. (2024)

Scalable Frameworks Fact-Checking

Community Engagement

Cooke et al. (2017) Allcott et al. (2017) Shah et al. (2024)

Lin et al. (2024) Heaton et al. (2024)

Contextual Verification

Potentials

User Education

Wardle et al. (2017) Haynes et al. (2021) Feng et al. (2024)

Helen et al. (2019) Staab et al. (2023) Duncan et al. (2023) Misinformation Moderation

Haynes et al. (2021) Feng et al. (2024) Panda et al. (2021)

Privacy Preserving

Staab et al. (2023) Lyu et al. (2023) Yang et al. (2024) Haynes et al. (2021)

Content Detection Bot Detection Bot Management

Fig. 8. Potentials in LLMs for social media information integrity. The figure illustrates key capabilities of LLMs in enhancing content verification, automated fact-checking, and content moderation, while also illustrating advances in privacy-preserving processing, user education, community engagement, and costeffective deployment frameworks. Each directional component references recent research (e.g., Wardle et al. (2017) - [190], Haynes et al. (2021) - [73], Panda et al. (2021) - [133], Cooke et al. (2017) - [39], Allcott et al. (2017) - [6], Shah et al. (2024) - [158], Atzori et al. (2024) - [11], Lyu et al. (2023) - [115], Yang et al. (2024) - [199], Duncan et al. (2023) - [50], Puppala et al. (2024) - [139], Nissenbaum et al. (2019) - [129], Lin et al. (2024) - [106], Heaton et al. (2024) - [74], Staab et al. (2023) - [170], Feng et al. (2024) - [56], Shafee et al. (2024) - [157], Panagiotou et al. (2021) - [132], Huntsman et al. (2024) - [80], Sangher et al. (2024) - [152])

, Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 13

Knowledge-augmented and retrieval-based detection pipelines differ substantially in how evidence is obtained and incorporated [99, 196, 198, 213]. In some settings, external evidence is provided in an oracle manner (e.g., curated knowledge or ground-truth supporting documents), while others rely on automated retrieval mechanisms such as BM25-style sparse search, dense vector retrieval, or API-based web search. These choices introduce trade-offs between evidence coverage and system latency, which are particularly important in real-time social media settings [30, 213]. Short-form content such as tweets or brief posts may also lead to failure modes where retrieval yields limited or ambiguous evidence. In addition, retrieval-augmented systems vary in reasoning style [31, 33, 196, 198]. Some adopt chain-of-thought generation to produce explanatory justifications, while others use verifier-style architectures that separately assess factual consistency. These design differences affect interpretability, computational cost, and deployment feasibility, highlighting that performance gains from knowledge-based detection should be understood within a broader system-level context rather than as purely model-level improvements. Beyond architectural and retrieval design, the effectiveness of zero-shot and few-shot LLM-based detectors is also highly sensitive to inference configurations. Factors such as prompt template design, role framing, temperature, nucleus sampling (𝑝), use of self-consistency, and tool-augmented reasoning chains can substantially influence model outputs. However, these settings are often inconsistently reported across studies, making direct performance comparisons difficult. We therefore argue that such inference parameters should be treated as part of the evaluation protocol rather than incidental implementation details. Explicit reporting of prompting and decoding configurations would allow future work to move toward more reproducible and head-to-head comparisons among LLM-based detection approaches. As shown in Table 3, LLM-generated misinformation presents two distinct categories of challenges. The first is unintentional generation or hallucinations [34, 158], which often occur due to the intrinsic properties of LLMs. These hallucinations are particularly challenging, as they appear highly plausible in fine-grained details such as dates, names, and numbers, making them difficult to detect even when models attempt to generate factual content. Shah et al. [158] explore this dual role of LLMs in both the combating and the potential generating of misinformation, examining their impact on social media consumption and various domain applications. Christensen et al. [34] further demonstrate these challenges through specific cases like tourism planning. The second category involves intentional generation - the deliberate misuse of LLMs for disinformation [16, 142]. This ranges from minor alterations to critical changes that dramatically alter meaning. Radivojevic et al. [142] demonstrate through experiments with GPT-4, LLama2, and Claude that humans can only identify the origin of such content 42% of the time, with persona’s influence exceeding human perception. Barman et al. [16] further explore LLMs’ potential in generating multimedia disinformation, emphasizing the need for ethical oversight and multi-stakeholder collaboration. These challenges are compounded by quality control issues, where verification must evaluate multiple aspects, including factual consistency and contextual alignment. The integration of these challenges with social media platforms creates additional complexities [49, 82, 198]. The scalability and automation capabilities of LLMs enable rapid generation of false content that can be automatically adapted across platforms. This technological capability, combined with sophisticated audience-targeting and platform-specific optimization, significantly enhances the potential for echo chamber reinforcement. These issues are further complicated by biased datasets, adversarial attacks, and the risk of users taking automated fact-checking outputs as definitive truth without proper judgment [49, 82]. Additionally, as noted by Yang et al. [198], data bias and topical bias in polarized comments remain underexplored challenges.

, Vol. 1, No. 1, Article . Publication date: August 2026.

14

Xiong et al.

Table 2. Representative Techniques of Fake News Detection. Technique

Models

Scenario

Strength

Limitation

References

Transformer Learning

BERT, RoBERTa, XLM-R

Text-only; cross-lingual

Strong semantic modeling

Domain shift; weak grounding

[29, 38, 42, 65, 116, 215]

Explainability

DistilBERT + SHAP

Interpretable support

Improved transparency

Limited gain

[114, 153]

Multi-modal Fusion

BDANN, FND-CLIP

Text + image detection

Cross-modal reasoning

Sensitive to image quality

[53, 156, 208]

Prompt-based Learning

ChatGPT, GenFEND

Zero-/few-shot

No task-specific training

Prompt & decoding sensitivity

[5, 77, 128]

Few-shot Learning

RumorLLM

Low-resource settings

Reduced labeling cost

Unstable with tiny data

[96]

Privacy-preserving

AugFake-BERT

Cross-organization

Privacy compliance

Performance trade-off

[90]

Domain Transfer

CT-BERT, RoBERTa

Region/domain adaptation

Better generalization

Needs domain pretraining

[116, 171]

accuracy

Table 3. Representative Techniques of Misinformation. Model / Method

Supervision

Knowledge Integra- Modality tion

Setting

References

ChatGPT / LLaMA

Zero-shot

Retrieval-augmented

Text + Vision

Online / Streaming

[30]

FACT-GPT / FactLLaMA

Fine-tuned

Knowledge graphs + Retrieval

Text

Online / Streaming

[31, 33, 99]

LEMMA

Few-shot

Retrieval / Tool-use

Text

Offline / Batch

[196, 198]

LLaVA

Few-shot

Retrieval or Transfer

Multimodal

Offline / Batch

[142]

Table 4. LLM Applications and Their Privacy Implications: A Comprehensive Overview of Privacy-preserving Techniques in Social Media Contexts Techniques

Methods

Description

References

Knowledge Distillation

Privacy-aware model compression and transfer

Minimize sensitive data exposure during training

[57, 212]

Differential Privacy

DP-SGD and privacy-preserving op- Formal privacy guarantees for model timization training

[78, 102]

Federated Learning

Decentralized model updates and training

Privacy-preserving distributed learn- [173, 202] ing

Adversarial Protection

Text perturbation and obfuscation techniques

Prevent unauthorized inference and identity linking

[44, 175]

Privacy-Preserving Attention

Modified attention mechanisms for privacy

Secure information processing in LLMs

[25, 135]

Political Economy and Platform Incentives The dynamics of misinformation spread cannot be fully understood from a purely technical perspective. These dynamics are influenced by platform business models, governance choices, and resource asymmetries among various actors. User engagement ranking algorithms prioritize content that generates clicks, shares, or increases user dwell time, creating selective pressure that favors sensational, polarizing, or emotionally charged content, which LLMs can efficiently generate or amplify [6, 182]. Furthermore, the high fixed costs of training and deploying state-of-the-art LLMs concentrate defensive capabilities in the hands of , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 15

well-resourced platforms and vendors, creating an asymmetry between attackers and defenders [17]. These structural dynamics mean that purely technical mitigation measures will be insufficient unless accompanied by changes in incentive structures. Therefore, current researchers call for interdisciplinary research integrating fields such as system design, economics, and governance to assess how different incentive structures alter the generation, detection, and dissemination of misinformation amplified by LLMs. Second-order Effects of LLM-based Moderation While LLMs can improve moderation coverage and responsiveness, deploying them at scale may introduce unintended societal harms. Over enforcement or low threshold automation can create a chilling effect by increasing false positives on legitimate but controversial speech, thereby narrowing the range of acceptable discourse. In addition, moderation systems trained primarily on dominant linguistic and cultural data may systematically mis-handle minority dialects, producing disparate error rates and cultural homogenization. These risks motivate safeguards such as calibrated thresholds and subgroup robustness auditing alongside conventional accuracy metrics. Multimodal Fact-Checking & Contextual Verification. Preliminary evidence indicates that coupling GPT-4 claim decomposition with retrieval augmented generation could reduce factchecker latency (reported as 35% improvement in Russo et al. [148]). However, this result is based on controlled experiments and may not generalize to real-world social media fact-checking pipelines, which face challenges such as API latency variability, content diversity, and heterogeneous crowd worker pools. Agentic decomposition approaches (e.g., FactAgent [56]) similarly show potential but require further evaluation regarding scalability, worker quality consistency, and cost effectiveness. Cross-platform context analysis uncovers temporal or source inconsistencies that headline-only systems overlook [158], and when these pipelines incorporate news format, presentation style, and narrative evolution tracking, they can flag emerging misinformation trends earlier in the news cycle [152]. Recent studies report macro-F1 scores around 0.82 on real-time news streams [65], surpassing traditional fact-checking workflows [6, 11, 39, 106]. Table 2 presents a systematic overview of fake news detection techniques, illustrating their evolution from basic transformer models to sophisticated multi-modal and domain-specific applications. Among these techniques, transformer-based learning forms the foundation of modern fake news detection, with models like BERT and RoBERTa demonstrating strong performance in text analysis. First, basic models like BERT, RoBERTa, and XLM-RoBERTa excel in text-based detection, with fine-tuned RoBERTa models specifically targeting COVID-19 misinformation [29, 116]. Second, cross-lingual detection capabilities are enhanced through XLM-RoBERTa-CNN combinations [65]. Third, hybrid models like BDANN integrate BERT with VGG-19 for initial multi-modal detection [208], while DistilBERT improves explainability [13]. Advanced multi-modal detection has evolved through three key innovations. First, FND-CLIP and EANBS process text and visual inputs simultaneously through contrastive learning and integrated BERT-CNN architectures [53, 156], enabling the detection of inconsistencies between textual claims and associated images. Second, GPT-4 enhances detection by identifying subtle cross-modal inconsistencies [115], particularly effective in cases where text and images have been manipulated to create misleading narratives. The model’s strong performance in cross-modal reasoning helps identify cases where images have been repurposed with false textual contexts or where subtle visual manipulations contradict textual claims. Additionally, generative frameworks have advanced detection capabilities in two ways: ChatGPT enables zero-shot classification in low-resource settings [5], allowing effective detection even without extensive training data, while GenFEND enriches training data through scenario simulation [128], creating diverse misinformation patterns. Domain Transfer and Cross-lingual Learning approaches address the challenge of adapting detection models across different domains and languages. CT-BERT demonstrates effective domain adaptation for COVID-19 misinformation [116],

, Vol. 1, No. 1, Article . Publication date: August 2026.

16

Xiong et al.

while RoBERTa-based models show promising results in cross-lingual transfer for Romanian political misinformation detection [171]. These domain transfer techniques are particularly crucial when dealing with emerging topics or low-resource languages. For instance, ChatGPT-based transfer learning has shown effectiveness in adapting bias detection across different cultural contexts [77]. Meanwhile, complementary techniques like active learning in RumorLLM [96] and federated learning in AugFake-BERT [90] enhance the efficiency and privacy of these domain adaptation processes. Beyond detection, LLMs advance fake news understanding through simulation and analysis. The Fake News Propagation Simulation framework studies diffusion patterns and virality factors [110], while Fact Agent models analyze dissemination patterns and develop real-time mitigation strategies [103]. Bot Detection, Analysis & Management. LLM-enhanced detectors jointly model textual semantics, image cues, temporal “digital-DNA” traces, and interaction graphs, yielding up to 9 percentage point gains in F1 score on the TwiBot-22 benchmark [54, 115, 170]. By analyzing behavioral patterns, content-generation fingerprints, and network dynamics, they surface coordinated, cross-platform inauthentic activity [6, 199]. These systems can also separate sophisticated socialengineering bots from legitimate automation [73], enabling fine-grained intervention policies that uphold overall platform integrity. Research into using LLM-enhanced bots for maintaining information integrity has evolved significantly, with substantial advances in detection and prevention capabilities. Early detection systems established fundamental approaches that continue to influence current solutions. Dukić et al. [48] and Heidari et al. [75] pioneered the integration of emoji embeddings and sentiment features with BERT models, demonstrating significant improvements in bot detection accuracy. These foundational works were extended by Xu et al. [194] through the combination of ALBERT with Bi-LSTM and self-attention mechanisms, establishing new benchmarks in bot spam detection performance. Multilingual detection capabilities have seen remarkable advancement. A series of studies demonstrated effective cross-lingual approaches. Panda et al. [133] developed models for detecting misinformation across English, Bulgarian, and Arabic; Garcia-Diaz et al. [61] achieved over 97% accuracy in Spanish language satire detection; and Milkova et al. [122] successfully detected value-expressive posts in Russian social media. These works collectively demonstrate the adaptability of LLM-based bot detection systems across different linguistic contexts. Advanced architectural approaches have significantly improved detection capabilities. Kumar et al. [95] and Guo et al. [69] developed hybrid frameworks combining BERT with graph convolutional networks, while Garcia-Silva et al. [62] demonstrated the superiority of generative transformers in bot detection tasks. Recent work by Ghanem et al. [64] and Raga et al. [143] has pushed performance boundaries further, achieving accuracy rates above 98% across different platforms. Although several studies report detection accuracies exceeding 98% across different platforms, such results should be interpreted with caution. Bot detection benchmarks are particularly sensitive to evaluation leakage, where structural or temporal overlap between training and test data may inadvertently inflate performance. For instance, shared retweet networks, synchronized activity bursts, or partially overlapping account clusters can allow models to learn dataset-specific signatures rather than generalizable bot behaviors. Real-time monitoring and prevention bot systems have emerged as crucial defensive tools. Panagiotou et al. [132] developed News Monitor, achieving high accuracy in rumor identification, while Kawintiranon et al. [88] and Catelli et al. [26] demonstrated effective approaches for context-specific bot spam detection. These systems have been enhanced by recent work from Ramamonjisoa et al. [144] and Bonechi et al. [19], who developed sophisticated automated moderation systems for bots. Specialized detection approaches have shown promising results in specific domains. Jones et al. [84] and Sallah et al. [151] developed bot systems specifically targeting child protection on social media, while Riyantoko et al. [147] demonstrated effective SMS bot spam classification using combined LSTM and BERT. , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 17

Privacy-Preserving Content Processing. The integration of LLMs into social media platforms has fundamentally reshaped the privacy landscape. With their ability to infer and memorize personal details across modalities, LLMs introduce privacy risks far beyond traditional NLP systems. Table 4 summary current privacy-preserving techniques and their applications in social media contexts. Privacy threat model: In social media platforms deploying LLMs, privacy risks arise from specific threat actors and attack surfaces. Key assets include user personal data, private messages, behavioral profiles, and sensitive group attributes. Adversaries may include external attackers probing model APIs, malicious insiders with training data access, or coordinated actors exploiting prompt-based leakage. Typical attack surfaces include training data collection pipelines, model outputs, retrieval components, and user facing conversational interfaces. Core threats include membership inference, model inversion, contextual integrity violations, and cross session leakage, with risk amplified under real time interaction and large scale personalization. To make privacy risks actionable in LLM-enabled social media systems, we map core threats to concrete controls. Membership inference and training data exposure motivate privacy-preserving training such as DP-SGD and data minimization. Differential Privacy solutions, such as DP-SGD, are considered the standard for providing (𝜖, 𝛿)-DP guarantees. Its efficacy in social media contexts is governed by the clipping normalization, typically calibrated between [0.1, 1.0], to bound the influence of outlier "heavy-hitter" users while minimizing the convergence floor of training loss curves. Similarly, Federated Learning (FL) solutions address decentralized storage but must incorporate Byzantine-robust aggregation. These developments address the need for context-aware privacy reasoning while maintaining a Pareto-optimal utility-privacy frontier. Model inversion and sensitive attribute inference require output filtering, regularization, and red-team leakage testing. Contextual leakage in conversational settings motivates contextual integrity constraints and prompt-level guardrails, though adaptive jailbreaks remain a residual risk. Cross-session leakage and personalization risks motivate isolation mechanisms for memory and retrieval, with continuous monitoring of leakage rates over long context interactions. These mappings point out that privacy governance must combine technical mitigation with ongoing auditing and deployment time enforcement. Grounded in Nissenbaum’s contextual-integrity theory [129], modern LLM pipelines aim to deliver personalized curation while respecting user privacy. Audits such as LLM-CI show that vanilla prompting can leak context-specific private attributes in a substantial fraction of cases (e.g., around 39%), underscoring the difficulty of enforcing contextual privacy guarantees in practice. Some studies explore embedding contextual-integrity–aware constraints as a mitigation strategy and report reduced leakage under specific experimental settings, though the generality of these gains and their utility trade-offs remain to be systematically validated [97]. Complementary safeguards include differential-privacy masking and other privacy-preserving analytics [170], transparent data-handling and audit trails [50], and consent-aware model updates via federated or split learning [139]. Together these measures enable platforms to offer fine-grained, trustworthy recommendations while upholding robust privacy guarantees. User Education & Media-Literacy Support. Instruction-tuned LLMs have been explored as tools for generating adaptive “trust cues,” critical-thinking scaffolds, and media-literacy interventions. In controlled user studies, one line of work reports improvements in misinformationrecognition accuracy on the order of 12% under specific experimental settings [74], though the generalizability and durability of these effects remain open questions. Beyond static tips, the models dynamically tailor explanations and verification checklists to a user’s demonstrated knowledge and engagement level, drawing on multilingual value-alignment work that transfers across languages with little performance loss [79]. They also incorporate practical guidance for source checking and claim verification [106], thereby empowering users to recognize manipulation techniques and exercise informed skepticism. Community Engagement & Collaborative Verification. , Vol. 1, No. 1, Article . Publication date: August 2026.

18

Xiong et al.

LLMs catalyze constructive dialogue by generating prompts and replies that nudge users toward evidence-based discussion [170]. In civic fact-checking forums, such prompts have doubled the volume of user-submitted evidence while avoiding alert-fatigue patterns [65]. Agentic pipelines like FactAgent decompose verification into micro-tasks that crowd workers validate asynchronously [56], yet still manage resource constraints [157] and platform-integration complexity [144]. 4.2

Landscape: Challenges in Social Media Information Integrity

Our landscape analysis also reveals four primary challenge domains in LLM-based social media information integrity, as illustrated in Figure 9. These findings emphasize critical concerns across technical implementation, ethical considerations, deployment feasibility, and user interaction. Detection and Verification Challenges Technical obstacles in LLM-based information integrity now center on model-specific detection evasion, inference performance bottlenecks, and hallucination issues. Detection evasion remains a foremost obstacle: adversaries continually refine prompt engineering tactics to bypass LLM integrity filters, with evasion success rates reported as high as 29.6% [56]. Performance bottlenecks in LLM inference and prompt processing continue to hinder timely and accurate integrity verification across high-volume social media streams [56, 152]. LLM cross-lingual capabilities remain limited, particularly in detecting misinformation across non-English or low-resource languages [133]. Most critically, model hallucination undermines trust in LLM-generated content, as spurious outputs can evade verification protocols and propagate misinformation [158]. Resource and Deployment Challenges LLM deployment for security and integrity applications faces significant challenges in resource management, system integration, and long-term

Panda et al. (2021) Feng et al. (2024) Cross-lingual Limitations

Haynes et al. (2021) Staab et al. (2023) Hallucination Issues

Shah et al. (2024) Christensen et al. (2024)

Yang et al. (2024) Haynes et al. (2021) Staab et al. (2023)

Privacy Vulnerabilities

Yang et al. (2024) Atzori et al. (2024) Cross-platform Security Risks

Coordinated Bot

Challenge

Feng et al. (2024) Sangher et al. (2024)

Lin et al. (2024) Duncan et al. (2023)

Detection Evasion Fairness and Accountability

Sangher et al. (2024) Panagiotou et al. (2021)

Puppala et al. (2024) Heaton et al. (2024) Resource and Deployment

Performance Bottlenecks

Shafee et al. (2024) Ramamonjisoa et al.(2024) Huntsman et al. (2024)

User Rights and Social Impact

Fig. 9. Challenges in LLMs for social media information integrity. The figure illustrates the key capabilities of LLMs in privacy vulnerabilities, cross-platform security risks, and fairness, as well as accountability, user rights, social impact, resource and deployment, performance bottlenecks, detection evasion, coordinated bots, and cross-lingual limitations. Each directional component references recent research (e.g., Feng et al. (2024) – [56], Sangher et al. (2024) – [152], Panda et al. (2021) – [133], Shah et al. (2024) – [158], Haynes et al. (2021) – [73], Staab et al. (2023) – [170], Yang et al. (2024) – [199], Atzori et al. (2024) – [11], Lin et al. (2024) – [106], Duncan et al. (2023) – [50], Heaton et al. (2024) – [74], Puppala et al. (2024) – [139], Shafee et al. (2024) – [157], Ramamonjisoa et al. (2024) – [144], Panagiotou et al. (2021) – [132], Huntsman et al. (2024) – [80]. ) , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 19

sustainability. The substantial computational demands of LLMs, combined with complex integration requirements, impede seamless deployment and maintenance at scale, particularly for systems that rely on continuous monitoring or multi-stage reasoning pipelines [80, 144, 157]. Beyond technical integration, recurring inference costs and energy consumption introduce additional economic burdens, which can limit the feasibility of sustained deployment for organizations with constrained resources [136, 172]. These challenges are further compounded by LLM-specific operational factors, including prompt optimization, model quantization, and the need to maintain multiple model variants across heterogeneous integrity tasks [132, 144]. Importantly, disparities in access to computational infrastructure and proprietary models can exacerbate global inequalities, making LLM-based defenses less accessible to institutions in low-income or Global South regions despite comparable security needs [185]. Privacy and Security Challenges LLM-specific security risks center on prompt injection vulnerabilities, training data privacy, and model extraction threats. The privacy issues are particularly concerning for LLMs, as these models may memorize and inadvertently leak sensitive information from their training data, while also being vulnerable to advanced prompt engineering exploits [73, 170, 193]. Furthermore, these challenges are exacerbated by issues related to data provenance, user consent, and content ownership, particularly when LLMs are trained on or applied to large-scale social media data collected through scraping. Even when the content on this social media may be publicly accessible, its reuse for model training or downstream security analysis raises concerns about permissible use, anonymization, and compliance with data protection regulations such as GDPR [184] and CCPA [130]. While privacy-preserving techniques such as Differential Privacy (DP) and Federated Learning (FL) provide important safeguards against attacks like model inversion and membership inference, they introduce non-negligible trade-offs in large-scale social media settings. DP mechanisms typically add noise to gradients or outputs, which can degrade model utility and require additional training iterations to maintain performance. Similarly, FL shifts computation to distributed clients and introduces communication overhead, synchronization delays, and potential model divergence across heterogeneous devices. These constraints are particularly relevant in the context of real-time social media monitoring, where high-throughput and low-latency processing are critical. LLM-based systems already face substantial computational bottlenecks; integrating DP or FL can further increase latency and resource consumption. Therefore, while these privacy-preserving approaches are promising, their deployment in real-time misinformation or bot detection systems requires careful balancing between privacy guarantees and operational performance. In addition, adversaries can leverage LLMs to generate highly convincing phishing content and automate social engineering attacks at scale [199]. The integration of LLMs across multiple platforms further complicates security and compliance, as attackers can exploit variations in model behavior, data handling practices, and regulatory enforcement across services [11]. Therefore, these challenges demonstrate that privacy-aware training pipelines, transparent data governance, and consent-sensitive deployment strategies are necessary when integrating LLMs into security-critical applications. Fairness and Accountability Challenges Ethical considerations specific to LLMs also encompass fairness, transparency, explainability, and accountability in safety-critical deployments. Challenges related to fairness stem from biases embedded in training data, model architectures, and evaluation benchmarks. These biases may cause LLMs to perpetuate or amplify societal prejudices during content generation and moderation, thereby disproportionately affecting marginalized groups [Lin2024-xa, Duncan2023-xk]. In social media contexts, such biases can lead to inconsistent moderation outcomes across different languages, dialects, and cultural backgrounds, directly influencing user participation, visibility, and inclusiveness. The opacity and probabilistic nature of LLMs further complicate accountability. Many LLM-based moderation and content integrity systems , Vol. 1, No. 1, Article . Publication date: August 2026.

20

Xiong et al.

operate as black boxes, making it difficult to trace decision-making processes or to explain why specific content is flagged, suppressed, or promoted. This lack of transparency can undermine user trust, hinder effective appeals, and complicate external audits by platform operators or regulatory bodies. Although post hoc explanation techniques—such as rationale generation or attention-based visualizations—show some promise in improving explainability, they may not faithfully reflect the underlying model behavior, raising concerns about the reliability of explanations and the risk of “explanation washing.” These challenges are magnified when LLMs are integrated into complex, multi-stage pipelines that involve multiple models, third-party APIs, and platform-specific policies. In such settings, accountability is diffuse: responsibility for erroneous or biased outcomes may be unclear, and tracing failures across components becomes difficult. Addressing these issues requires explicit accountability mechanisms, including decision traceability, transparent logging, and well-defined human-in-the-loop oversight [74, 139]. 5 5.1

Discussion Key Findings

Leading Models and Research Paradigms. Based on our review, the field of LLM-driven social media information integrity has crystallized around several influential models and research approaches. First, in fact-checking and misinformation detection, LLM-based models like FACT-GPT [33] and FactLLaMA [31] have emerged as industry standards. These models excel at automated verification but face a critical challenge: they are prone to hallucinations in zero-shot and cross-domain scenarios, particularly when verifying specific details like dates, numbers, or named entities [34, 158]. In the privacy and security domain, frameworks like ProPILE [93] and P-BERT [175], as well as federated learning approaches such as SocFedGPT [140], have emerged as leading solutions. While these LLM-based models have advanced detection capabilities, they also expose critical challenges in generalizability, interpretability, and robustness. Trends and Future Breakthroughs. The research frontier is increasingly oriented toward multimodal, cross-model ensemble, and human-in-the-loop systems. Under specific benchmarks and controlled experimental settings, some studies report that multimodal detection frameworks integrating text, metadata, and contextual signals yield accuracy gains on the order of 5–10% over text-only baselines [111]. Related work further explores ensemble-based verification strategies, reporting additional improvements (e.g., 8–12%) for certain classes of subtle misinformation, though these gains depend on datasets and evaluation protocols and are not directly comparable across studies. In privacy protection, differential privacy [102] and linguistic steganography [15] provide both theoretical and practical advances. Detection systems are evolving from single-model approaches to multimodal, ensemble, and explainable frameworks. By fusing text, images, and social network structures, multimodal systems can significantly improve the detection of complex and adversarial misinformation. Ensemble methods (e.g., model stacking, expert systems) can reduce individual model bias and enhance robustness against adversarial content. At the same time, explainability mechanisms provide transparent rationales for moderation decisions, fostering trust and usability for both moderators and end-users [210]. Privacy protection is shifting from passive anonymization to proactive defense and dynamic monitoring. Techniques such as differential privacy, knowledge distillation, and tag-based prediction reduce raw data exposure, while systems like P-BERT and ProPILE enable real-time detection and blocking of sensitive information. Federated learning enables privacy-preserving model training across distributed systems without sharing raw data, while linguistic steganography provides methods for secure information encoding in collaborative settings [15, 203]. These methods must balance protection strength, system overhead, and user experience, and their effectiveness against advanced attacks remains an open question. , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 21

5.2

Implications

Technical Implications. The technical implications of LLM deployment in social media manifest in three interconnected dimensions: content verification capabilities, adversarial content challenges, and platform security vulnerabilities. In terms of content verification, LLMs have transformed how platforms detect and combat false information. For example, Staab et al. [170] and Collier et al. [37] demonstrate how these systems can effectively identify coordinated misinformation campaigns by analyzing subtle patterns across posts and user behaviors. However, this enhanced detection capability comes with its own risks—for example, the same pattern recognition capabilities can be exploited to breach user privacy and reconstruct sensitive information from seemingly innocuous social media interactions [91]. The second dimension concerns the evolving nature of social media content manipulation. LLM-enhanced platforms face increasingly sophisticated forms of synthetic content and automated manipulation. As Duncan et al. [50] suggests, these platforms must balance aggressive content filtering with the risk of false positives that could suppress legitimate discourse. This challenge is particularly apparent in cross-cultural contexts, where content moderation systems must adapt to diverse cultural norms while maintaining consistent integrity standards [106]. The third dimension addresses platform security and scalability. While federated approaches [140] offer promising solutions for privacy-preserving content moderation, they introduce new vulnerabilities in cross-platform coordination. The emergence of sophisticated adversarial techniques targeting social media integrity systems[128] necessitates continuous adaptation of defense mechanisms. Societal Implications. Based on our review, we also summarize three interconnected dimensions in terms of the societal impact, including platform trust dynamics, information equity, and collective behavior shifts. In the trust dimension, LLMs are fundamentally reshaping how users verify and consume social media content. While these systems enhance automated fact-checking capabilities, they also introduce new challenges to platform credibility. One typical example is that user trust can fluctuate dramatically following high-profile misinformation incidents, particularly when LLM-based detection systems fail to catch sophisticated synthetic content [74]. This trust volatility is further complicated by cross-cultural variations [105]; that is, different societies exhibit varying levels of confidence in automated content verification systems, which further affect the overall effectiveness of platform integrity measures. The second dimension concerns information equity and accessibility. While LLM-powered content verification promises more democratic access to fact-checking tools, their deployment often creates new forms of information disparities. Specifically, the effectiveness of these systems varies significantly across languages and cultural contexts [50, 106], potentially marginalizing users from non-dominant linguistic and cultural backgrounds. Such uneven performance can undermine community-level trust in automated verification systems and exacerbate perceptions of exclusion among affected populations. For instance, fact-checking systems may perform well on Englishlanguage misinformation but fail to detect similar content in Swahili or Tagalog due to limited training data or weaker language modeling support, raising broader concerns about inclusion and the societal legitimacy of AI-mediated information governance. The third dimension addresses collective behavior adaptation in response to LLM-enhanced content moderation. Social media users are developing new strategies for information sharing and verification, shaped by their interactions with automated content analysis systems. This has been widely demonstrated in prior studies [89, 177], in which researchers have observed how communities adapt their communication patterns to either work with or circumvent LLM-based content filters, and reported the emergence of new forms of collaborative fact-checking that combine human expertise with LLM capabilities.

, Vol. 1, No. 1, Article . Publication date: August 2026.

22

5.3

Xiong et al.

Future Research

5.3.1 Information Disorder Detection. The future of information disorder detection and mitigation requires a comprehensive approach integrating technical innovations and adaptive systems. Technical Foundations and Detection Systems The cornerstone of future information disorder mitigation is the development of robust detection frameworks that fuse multiple data modalities and verification techniques. Multi-modal detection systems that integrate text, metadata, and contextual signals are an active area of investigation. Under specific benchmarks and controlled experimental settings, some studies report accuracy gains on the order of 5–10% over text-only baselines [111]. Related work explores cross-model verification via ensemble approaches that combine multiple LLMs, reporting additional improvements (e.g., 8–12%) for certain classes of subtle misinformation, though these results depend on datasets and evaluation protocols and are not directly comparable across studies [52].These architectures are further enhanced by data synthesis and augmentation techniques, exemplified by FACT-GPT [33] that improve detection capabilities through automated fact-checking and content verification. As adversarial tactics evolve, robust defense mechanisms become crucial. Recent advances in adversarial training and prompt-level hardening, such as Prompt Adversarial Tuning [125], demonstrate promising approaches to enhancing model robustness while maintaining legitimate task performance. These developments can be coupled with improved prompt engineering and detection mechanisms [5, 96, 104] to address evolving misinformation tactics effectively. Co-evolutionary dynamics and governance. The interaction between attackers and defenders can be modeled as a co-evolutionary process: attackers continuously adapt their generation and evasion strategies (e.g., prompt engineering, adversarial fine-tuning), while defenders improve detection, attribution, and policy intervention measures. Viewing this process as a resource-constrained game helps identify asymmetries (e.g., scale costs, latency constraints, false positive risks) and points to potential stable strategies. For example, increasing the marginal cost of large-scale abuse (e.g., rate limiting, attribution authentication), providing detection subsidies for smaller platforms, or implementing transparency requirements that increase the risk of attackers being detected. We suggest that future research should include: (a) formalizing attacker/defender payoffs, (b) simulating the arms race dynamics under different policy interventions, and (c) evaluating equitable governance mechanisms that can reduce systemic harm. Political economy of information disorder. Technical solutions alone cannot resolve LLM information disorder without addressing the structural incentives of the ecosystem. Engagementdriven platform business models (attention, retention, advertising) can systematically reward sensational, emotionally manipulative, and polarizing content—precisely the type of content that LLMs can now generate at scale and rapidly iterate through optimization and A/B testing. At the same time, defensive capacity is economically uneven: high-quality moderation and detection require compute, model access, and sustained human review, which can advantage large platforms while leaving smaller platforms initiatives with higher relative costs and weaker safety coverage. These political economic asymmetries imply that best technical defenses may not be deployed where they are most needed, and that effective governance should be evaluated not only by accuracy but also by how it reshapes incentives and access. Adaptive Learning and Cultural Context. Future systems must emphasize adaptability through continuous learning mechanisms that maintain accuracy on known patterns while quickly adapting to emerging threats. Meta-learning and parameter-efficient tuning approaches have shown potential in reducing annotation costs and accelerating responses to emergent narratives [205]. Integration of human feedback loops, through expert validation and model updates, can also help refine model accuracy and prevent the misclassification of benign content [197]. Cross-cultural

, Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 23

and linguistic adaptation represents another critical frontier. Language-agnostic, culturally aware embeddings have shown promise in reducing the gap between high- and low-resource settings [121]. By developing enhanced embeddings and culturally adaptive techniques [65, 146], detection systems can achieve effective detection across diverse linguistic and cultural environments. User Interface and Behavioral Analysis. The success of information disorder mitigation depends critically on effective user interaction and behavioral understanding. Future interfaces must clearly communicate detection confidence levels and provide transparent explanations for flagged content. The BiasX framework demonstrates the value of free-text explanations in content moderation, significantly aiding moderators in identifying subtle toxic content [210]. Understanding user behavior patterns, particularly through silent-user and propagation-path modeling [128], enables more effective intervention strategies. Simulation models analyzing propagation dynamics [103, 110] further enhance our ability to develop proactive mitigation approaches. Ground truth under synthetic reality. Current integrity systems implicitly assume a stable external ground truth against which claims can be verified Nissenbaum [129]. However, the proliferation of generated content challenges this assumption by lowering the marginal cost of producing multiple, contradictory yet plausible narratives at scale. In such settings, verification cannot rely solely on content plausibility or closed-loop model agreement: detectors trained on synthetic data risk reinforcing self-referential signals detached from external evidence. This motivates shifting from binary “true/false” framing toward integrity as an evidence construct, emphasizing provenance, source reliability, and verifiable supporting context. Future research should therefore study how to maintain shared epistemic reference points online when synthetic information saturates the environment, and how integrity interventions can prioritize traceable evidence, auditability, and calibrated uncertainty rather than assuming a universally ground truth. Intrinsic vs. extrinsic hallucinations. Recent work distinguishes between intrinsic hallucinations, where generated content directly contradicts the provided source or context, and extrinsic hallucinations, where the model introduces unsupported information that cannot be verified from the given evidence. Intrinsic hallucinations reflect failures of faithfulness, whereas extrinsic hallucinations reflect failures of grounding. Current LLM-based fact-checking systems are generally more effective at detecting intrinsic inconsistencies than resolving extrinsic claims that require trustworthy external knowledge. Future research should therefore prioritize scalable methods for addressing extrinsic hallucinations through robust retrieval, provenance-aware evidence integration, and verifiable grounding beyond model-internal consistency. Real-time, multimodal, and interactive disinformation. An emerging frontier of information disorder extends beyond static text and images toward real-time and immersive manipulation, including deepfake video, voice cloning, and AI-driven avatars in live streaming and social VR environments. These settings pose qualitatively different threats, where deception unfolds interactively, and personalized persuasion can be delivered on the fly through photorealistic agents or cloned voices of trusted figures. Current post hoc forensic detection paradigms are unlikely to suffice for such low latency, conversational contexts. Future research should prioritize new integrity mechanisms for real-time environments, including live authentication and provenance protocols, robust avatar and voice verification, proof-of-humanity signals, and safeguards that can operate during interaction rather than only after harmful content has spread. Toward More Generalized Intelligence for Information Integrity. Most existing approaches to information disorder detection are grounded in Artificial Narrow Intelligence (ANI), focusing on task-specific classification or verification under constrained threat models. While effective in controlled settings, such systems often struggle with long-horizon reasoning, cross-platform narrative tracking, and adaptive coordination against evolving adversaries. Emerging Artificial General Intelligence (AGI)-inspired capabilities, including richer world modeling, abstraction , Vol. 1, No. 1, Article . Publication date: August 2026.

24

Xiong et al.

across domains, and goal-aware reasoning, offer potential avenues to enhance future integrity systems. These advances could enable more holistic reasoning over social context, co-evolutionary attacker–defender dynamics, and long-term governance interventions, particularly in cross-lingual and low-resource settings. At the same time, increased generality raises challenges related to controllability and oversight. Future research should therefore explore incremental integration of AGI-inspired components while maintaining clear human oversight and accountability. 5.3.2 LLM-enhanced Social Bot Detection. The evolution of LLM-enhanced bot detection necessitates a new generation of detection technologies that are real-time and privacy-conscious. Advanced Detection Architectures The next generation of bot detection systems demands hybrid architectures that leverage multiple detection strategies. Research demonstrates that pipelines merging “digital DNA” temporal signatures with transformer encoders yield superior performance over traditional graph-only detectors [28]. Federated variants of these systems show particular promise in preserving privacy while enabling cross-platform intelligence sharing [200]. The integration of federated learning approaches, as demonstrated by Puppala et al. [139], represents a crucial advancement in enhancing detection capabilities while maintaining robust privacy standards. Cross-modal and Multilingual Capabilities As bot operations increasingly span multiple languages and media types, future research must prioritize comprehensive cross-lingual and multimodal detection capabilities. Evaluation frameworks such as ETS-MM are advancing joint text-audio-visual embedding analysis [101], while building on foundational work in cross-lingual detection by Panda et al. [133] and Ahmed et al. [2]. These developments enable more effective understanding and detection of malicious content across linguistic and cultural boundaries. The comprehensive evaluation frameworks pioneered by Gu et al. [68] provide essential benchmarks for assessing multimodal bot detection effectiveness. Cross-lingual and Low-resource Robustness: Cross-lingual integrity systems should be evaluated with stratified error analysis that reports precision/recall/false-positive rates by linguistic subgroup and stress tests known brittleness factors, including (i) script variation, (ii) morphological complexity and tokenization effects, (iii) code-switching and mixed language discourse common in global online speech, and (iv) domain drift that can differentially degrade performance across languages. Where primary studies do not report language by language breakdowns, we treat multilinguality as a key evidence gap rather than a resolved capability, and we explicitly call for standardized reporting of subgroup metrics and robustness ablations. We emphasize that multilingual transfer is not uniform across languages and is often evaluated on English and Twitter/X-centric benchmarks; therefore we treat cross-lingual robustness as an evidence gap unless stratified results are reported. We also highlight the need for broader evidence beyond Twitter/X and from non-Anglophone verification ecosystems to improve external validity in multilingual contexts. Real-time Monitoring and Response. The advancement of real-time monitoring capabilities represents a critical direction for future research. Streaming graph transformers have achieved significant breakthroughs in reducing moderation latency to sub-second levels, meeting crucial requirements for live platforms [209]. Recent innovations by Ramamonjisoa et al. [144] and Bonechi et al. [19] in automated moderation systems establish new benchmarks for real-time monitoring and response capabilities. These systems must balance immediate detection with accuracy and resource efficiency. 5.3.3 Privacy Preservation. The future of privacy preservation in LLM-enhanced social media environments demands a multi-layered framework integrating three key components: Advanced Privacy-Preserving Architectures. The foundation of future privacy protection lies in developing sophisticated architectural solutions that can safeguard user data while maintaining system functionality. Differential privacy approaches show particular promise, with recent work , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 25

by Coffey et al. [36] demonstrating how differentially private stochastic gradient descent during fine-tuning can effectively limit model memorization while preserving utility. Federated learning variants, as explored by Yao et al. [203], offer promising solutions for distributed privacy protection, though they must carefully balance privacy guarantees with system performance. These advances must be complemented by innovative anonymization strategies that can adapt to evolving threats and content sensitivity levels. Contextual Integrity and Inference Protection. A critical challenge in current LLM systems is maintaining contextual privacy across diverse interaction scenarios. The CONFAIDE benchmark reveals concerning levels of sensitive context leakage, with up to 57% of tested scenarios showing vulnerability [124]. Future research must focus on developing robust inference-time privacy guards that can dynamically adapt to different contexts while preventing unauthorized information extraction. The CONFAIDE framework [124] provides a foundation for standardizing privacy evaluations, but more sophisticated protection mechanisms are needed to address emerging threats while preserving model utility. Linguistic steganography techniques [15] and advanced PII protection methods [81] show promise in providing granular privacy controls across different linguistic. Real-time Monitoring and Compliance. As privacy threats evolve and regulatory requirements become more stringent, particularly with frameworks like the EU AI Act [168], real-time privacy monitoring and compliance systems become crucial. Systems like ProPILE [93] demonstrate the potential for proactive privacy protection through continuous monitoring and intervention. Future research must focus on developing explainable systems that can provide clear documentation of privacy measures while adapting to emerging regulatory requirements. 5.4

Ethical and Regulatory Perspectives

Ethical and regulatory frameworks provide crucial guidance for deploying large language models in social media environments. IEEE Ethically Aligned Design emphasizes transparency, accountability, and human oversight, while the EU AI Act classifies many AI-driven moderation and risk assessment systems as high-risk and mandates requirements for risk management, bias mitigation, and monitoring. UNESCO’s Recommendation on the Ethics of AI highlights inclusiveness, fairness, and respect for linguistic and cultural diversity. Together, these initiatives underscore the need to align technological advances in LLM-based systems with ethical principles and regulatory requirements. 6 Conclusions This comprehensive review examines the critical challenges and future directions in social media information integrity within the context of LLM applications. Our systematic analysis reveals four critical challenges that define this landscape: the escalating sophistication of information disorder, the emergence of LLM-enhanced social bots, persistent biases and uneven performance across languages and cultural contexts, and the growing complexity of privacy preservation and accountability in LLM-based systems. Through detailed examination of current approaches and limitations, we identified how LLMs simultaneously serve as powerful tools for detecting malicious content while potentially enabling more sophisticated forms of synthetic content generation. Future developments must focus on several key areas, each corresponding to the research gaps identified in this survey. To address the growing sophistication of information disorder, research should advance multi-modal and adaptive detection systems, including ensemble-based approaches that integrate heterogeneous signals under evolving threat models. To counter LLM-enhanced social bots and coordinated inauthentic behavior, future work must develop cross-platform coordination and real-time monitoring capabilities that capture propagation dynamics and adversarial adaptation at scale. To mitigate bias, multilingual performance gaps, and low-resource disparities, culturally aware models, stratified evaluation, and robustness analysis across languages and regions are , Vol. 1, No. 1, Article . Publication date: August 2026.

26

Xiong et al.

essential. Finally, to respond to privacy, transparency, and accountability challenges, robust privacypreserving mechanisms and governance-aware deployment practices are required to address contextual privacy leakage and ensure responsible system operation. For social media platforms, these challenges necessitate comprehensive strategies combining policy, technical, and user protection measures. Platforms should develop clear guidelines for LLMgenerated content, implement robust detection and moderation systems, and establish transparent content labeling mechanisms. Additionally, user education and protection should be prioritized through enhanced verification tools and effective reporting systems, particularly for communities that are disproportionately affected by biased or opaque automated decisions. Overall, the success of these efforts will depend on the integration of technical solutions with policy frameworks, while maintaining transparency and user trust across social media platforms. As LLM technology continues to evolve toward more generalized and adaptive capabilities, the approaches outlined in this review provide a foundation for systematically addressing existing research gaps and building more secure, privacy-respecting, and trustworthy social media environments. References [1] Kawsar Ahmed, Md Osama, Md Sirajul Islam, Md Taosiful Islam, Avishek Das, and Mohammed Moshiul Hoque. 2023. Score_isall_you_need at blp-2023 task 1: A hierarchical classification approach to detect violence inciting text using transformers. In Proceedings of the First Workshop on Bangla Language Processing (BLP-2023). 185–189. [2] Kawsar Ahmed, Md Osama, Md Sirajul Islam, Md Taosiful Islam, Avishek Das, and Mohammed Moshiul Hoque. 2023. Score_IsAll_you_need at BLP-2023 task 1: A hierarchical classification approach to detect violence inciting text using transformers. In Proceedings of the First Workshop on Bangla Language Processing (BLP-2023) (Singapore). Association for Computational Linguistics, Stroudsburg, PA, USA, 185–189. [3] Hani Al-Omari, Malak Abdullah, Ola Al-Titi, and Samira Shaikh. 2019. Justdeep at nlp4if 2019 shared task: propaganda detection using ensemble deep learning models. EMNLP-IJCNLP 2019 (2019), 113. [4] Hani Al-Omari, Malak Abdullah, Ola AlTiti, and Samira Shaikh. 2019. JUSTDeep at NLP4IF 2019 task 1: Propaganda detection using ensemble deep learning models. In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda (Hong Kong, China). Association for Computational Linguistics, Stroudsburg, PA, USA. [5] Jawaher Alghamdi, Yuqing Lin, and Suhuai Luo. 2024. Cross-domain fake news detection using a prompt-based approach. Future Internet 16, 8 (2024), 286. [6] Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. Journal of economic perspectives 31, 2 (2017), 211–236. [7] Sawsan Alshattnawi, Amani Shatnawi, Anas M R AlSobeh, and Aws A Magableh. 2024. Beyond word-based model embeddings: Contextualized representations for enhanced social media spam detection. Appl. Sci. (Basel) 14, 6 (March 2024), 2254. [8] Sacha Altay, Manon Berriche, Hendrik Heuer, Johan Farkas, and Steven Rathje. 2023. A survey of expert views on misinformation: Definitions, determinants, solutions, and future of the field. Harvard Kennedy School Misinformation Review 4, 4 (2023), 1–34. [9] Dimitris Asimopoulos, Ilias Siniosoglou, Vasileios Argyriou, Thomai Karamitsou, Eleftherios Fountoukidis, Sotirios K Goudos, Ioannis D Moscholios, Konstantinos E Psannis, and Panagiotis Sarigiannidis. 2024. Benchmarking advanced text anonymisation methods: A comparative study on novel and traditional approaches. In 2024 13th International Conference on Modern Circuits and Systems Technologies (MOCAST). IEEE, 1–6. [10] Hadi Askari, Anshuman Chhabra, Bernhard Clemm von Hohenberg, Michael Heseltine, and Magdalena Wojcieszak. 2024. Incentivizing news consumption on social media platforms using large language models and realistic bot accounts. PNAS Nexus 3, 9 (Sept. 2024), gae368. [11] Maurizio Atzori, Eleonora Calò, Loredana Caruccio, Stefano Cirillo, Giuseppe Polese, and Giandomenico Solimando. 2024. Evaluating password strength based on information spread on social networks: A combined approach relying on data reconstruction and generative models. Online Soc. Netw. Media 42, 100278 (Aug. 2024), 100278. [12] Navid Ayoobi, Sadat Shahriar, and Arjun Mukherjee. 2023. The looming threat of fake and llm-generated linkedin profiles: Challenges and opportunities for detection and prevention. In Proceedings of the 34th ACM Conference on Hypertext and Social Media. 1–10. [13] Jackie Ayoub, X Jessie Yang, and Feng Zhou. 2021. Combat COVID-19 infodemic using explainable natural language processing models. Information Processing & Management 58, 4 (2021), 102569. , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 27

[14] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022). [15] Calvin Bao and Marine Carpuat. 2024. Keep it Private: Unsupervised Privatization of Online Text. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 8678–8693. https://doi.org/10.18653/v1/2024.naacl-long.480 [16] Dipto Barman, Ziyi Guo, and Owen Conlan. 2024. The dark side of language models: Exploring the potential of llms in multimedia disinformation generation and dissemination. Machine Learning with Applications (2024), 100545. [17] Yochai Benkler, Robert Faris, and Hal Roberts. 2018. Network propaganda: Manipulation, disinformation, and radicalization in American politics. Oxford University Press. [18] Simone Bonechi. 2024. Development of an automated moderator for deliberative events. Electronics 13, 3 (2024), 544. [19] Simone Bonechi. 2024. Development of an automated moderator for deliberative events. Electronics (Basel) 13, 3 (Jan. 2024), 544. [20] Ali Borji and Mehrdad Mohammadian. 2023. Battle of the wordsmiths: Comparing ChatGPT, GPT-4, Claude, and bard. SSRN Electron. J. (2023). [21] Robert Brandt. 2023. AI-Assisted Medicine: Possibly Helpful, Possibly Terrifying. Emergency Medicine News 45, 7 (2023), 20. [22] Olivia Brown, Robert M Davison, Stephanie Decker, David A Ellis, James Faulconbridge, Julie Gore, Michelle Greenwood, Gazi Islam, Christina Lubinski, Niall G MacKenzie, et al. 2024. Theory-driven perspectives on generative artificial intelligence in business and management. British Journal of Management 35, 1 (2024), 3–23. [23] George-Octavian Bărbulescu and Peter Triantafillou. 2024. To each (textual sequence) its own: improving memorizeddata unlearning in large language models. In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML’24). JMLR.org, Article 121, 21 pages. [24] Yunna Cai, Fan Wang, Haowei Wang, and Qianwen Qian. 2023. Public sentiment analysis and topic modeling regarding ChatGPT in mental health on Reddit: Negative sentiments increase over time. arXiv preprint arXiv:2311.15800 (2023). [25] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Colin Raffel, et al. 2021. Extracting Training Data from Large Language Models. In Proceedings of the 30th USENIX Security Symposium. USENIX Association, 2633–2650. [26] Rosario Catelli, Hamido Fujita, Giuseppe De Pietro, and Massimo Esposito. 2022. Deceptive reviews and sentiment polarity: Effective link by exploiting BERT. Expert Syst. Appl. 209, 118290 (Dec. 2022), 118290. [27] Pew Research Center. 2024. Social Media and News Fact Sheet. https://www.pewresearch.org/journalism/factsheet/social-media-and-news-fact-sheet/ [28] Vaishali Chawla and Yatin Kapoor. 2023. A hybrid framework for bot detection on twitter: Fusing digital DNA with BERT. Multimed. Tools Appl. 82, 20 (Aug. 2023), 30831–30854. [29] Ben Chen, Bin Chen, Dehong Gao, Qijin Chen, Chengfu Huo, Xiaonan Meng, Weijun Ren, and Yang Zhou. 2021. Transformer-based language model fine-tuning methods for COVID-19 fake news detection. In Combating online hostile posts in regional languages during emergency situation: First international workshop, CONSTRAINT 2021, collocated with AAAI 2021, virtual event, February 8, 2021, revised selected papers 1. Springer, 83–92. [30] Canyu Chen and Kai Shu. 2024. Can LLM-Generated Misinformation Be Detected?. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=ccxD4mtkTU [31] Tsun-Hin Cheung and Kin-Man Lam. 2023. Factllama: Optimizing instruction-following language models with external knowledge for automated fact-checking. In 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 846–853. [32] Xiaoxiao Chi, Xuyun Zhang, Yan Wang, Lianyong Qi, Amin Beheshti, Xiaolong Xu, Kim-Kwang Raymond Choo, Shuo Wang, and Hongsheng Hu. 2024. Shadow-Free Membership Inference Attacks: Recommender Systems Are More Vulnerable Than You Thought. In IJCAI. [33] Eun Cheol Choi and Emilio Ferrara. 2024. Automated claim matching with large language models: empowering factcheckers in the fight against misinformation. In Companion Proceedings of the ACM Web Conference 2024. 1441–1449. [34] Jeff Christensen, Jared M Hansen, and Paul Wilson. 2024. Understanding the role and impact of Generative Artificial Intelligence (AI) hallucination within consumers’ tourism decision-making processes. Curr. Issues Tourism (Jan. 2024). [35] Cloudflare. 2025. What is a social media bot? https://www.cloudflare.com/learning/bots/what-is-a-social-media-bot/ Accessed: March 17, 2025. [36] Sean M Coffey, Joseph W Catudal, and Nathaniel D Bastian. 2024. Differential privacy to mathematically secure fine-tuned large language models for linguistic steganography. In Assurance and Security for AI-enabled Systems, Vol. 13054. SPIE, 160–171. [37] Henry Collier. 2024. AI: The Future of Social Engineering! Proc. Eur. Conf. Inf. Warf. Secur. 23, 1 (June 2024).

, Vol. 1, No. 1, Article . Publication date: August 2026.

28

Xiong et al.

[38] Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 8440–8451. https://doi.org/10.18653/v1/2020.acl-main.747 [39] Nicole A Cooke. 2017. Posttruth, truthiness, and alternative facts: Information behavior and critical information consumption for a new age. The library quarterly 87, 3 (2017), 211–221. [40] Limeng Cui and Dongwon Lee. 2020. Coaid: Covid-19 healthcare misinformation dataset. arXiv preprint arXiv:2006.00885 (2020). [41] Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2025. Security and privacy challenges of large language models: A survey. Comput. Surveys 57, 6 (2025), 1–39. [42] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. (2019), 4171–4186. [43] Jane Dickson and Thomas S. McCauley. 2023. The Ethics of AI in Mental Health: Safeguarding Patient Privacy in Online Settings. ACM Transactions on Internet Technology 23, 2 (2023), 1–24. [44] Dilara Dogan, Bahadir Altun, Muhammed Said Zengin, Mucahid Kutlu, and Tamer Elsayed. 2023. Catch Me If You Can: Deceiving Stance Detection and Geotagging Models to Protect Privacy of Individuals on Twitter. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 17. 173–184. [45] Yao Dou, Isadora Krsek, Tarek Naous, Anubha Kabra, Sauvik Das, Alan Ritter, and Wei Xu. 2024. Reducing Privacy Risks in Online Self-Disclosures with Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 13732–13754. https://doi.org/10.18653/v1/2024.acl-long.741 [46] Sunny Duan, Mikail Khona, Abhiram Iyer, Rylan Schaeffer, and Ila Rani Fiete. 2025. Uncovering Latent Memories in Large Language Models. In Proceedings of the International Conference on Learning Representations (ICLR). [47] David Dukić, Dominik Keča, and Dominik Stipić. 2020. Are you human? Detecting bots on Twitter Using BERT. In 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 631–636. [48] David Dukic, Dominik Keca, and Dominik Stipic. 2020. Are you human? Detecting bots on twitter using BERT. In 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA) (sydney, Australia). IEEE. [49] Chris Dulhanty, Jason L Deglint, Ibrahim Ben Daya, and Alexander Wong. 2019. Taking a stance on fake news: Towards automatic disinformation assessment via deep bidirectional transformer language models for stance detection. arXiv preprint arXiv:1911.11951 (2019). [50] Clay Duncan and Ian Mcculloh. 2023. Unmasking Bias in Chat GPT Responses. In Proceedings of the International Conference on Advances in Social Networks Analysis and Mining (Kusadasi Turkiye). ACM, New York, NY, USA. [51] Clay Duncan and Ian Mcculloh. 2023. Unmasking bias in Chat GPT responses. In Proceedings of the International Conference on Advances in Social Networks Analysis and Mining. 687–691. [52] Mohammed E. Almandouh, Mohammed F Alrahmawy, Mohamed Eisa, Mohamed Elhoseny, and AS Tolba. 2024. Ensemble based high performance deep learning models for fake news detection. Scientific Reports 14, 1 (2024), 26591. [53] Mengyi Wang Fangfang Shan, Huifang Sun. 2024. Multimodal Social Media Fake News Detection Based on Similarity Inference and Adversarial Networks. Computers, Materials & Continua 79, 1 (2024), 581–605. [54] Shangbin Feng, Zhaoxuan Tan, Herun Wan, Ningnan Wang, Zilong Chen, Binchi Zhang, Qinghua Zheng, Wenqian Zhang, Zhenyu Lei, Shujie Yang, et al. 2022. Twibot-22: Towards graph-based twitter bot detection. Advances in Neural Information Processing Systems 35 (2022), 35254–35269. [55] Shangbin Feng, Herun Wan, Ningnan Wang, Jundong Li, and Minnan Luo. 2021. Twibot-20: A comprehensive twitter bot detection benchmark. In Proceedings of the 30th ACM international conference on information & knowledge management. 4485–4494. [56] Shangbin Feng, Herun Wan, Ningnan Wang, Zhaoxuan Tan, Minnan Luo, and Yulia Tsvetkov. 2024. What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 3580–3601. https://doi.org/10.18653/v1/2024.acl-long.196 [57] Chiarello Filippo, Giordano Vito, Spada Irene, Barandoni Simone, and Fantoni Gualtiero. 2024. Future applications of generative large language models: A data-driven case study on ChatGPT. Technovation 133 (2024), 103002. [58] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, 1322–1333. [59] Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics 50, 3 (2024), 1097–1179.

, Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 29

[60] Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. 2023. Rarr: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 16477–16508. [61] José Antonio García-Díaz and Rafael Valencia-García. 2022. Compilation and evaluation of the Spanish SatiCorpus 2021 for satire identification using linguistic features and transformers. Complex Intell. Syst. 8, 2 (April 2022), 1723–1736. [62] Andres Garcia-Silva, Cristian Berrio, and Jose Manuel Gomez-Perez. 2021. Understanding Transformers for Bot Detection in Twitter. (2021). arXiv:2104.06182 [63] Razan Ghanem and Hasan Erbay. 2020. Context-dependent model for spam detection on social networks. SN Appl. Sci. 2, 9 (Sept. 2020). [64] Razan Ghanem, Hasan Erbay, and Khaled Bakour. 2023. Contents-based spam detection on social networks using RoBERTa embedding and stacked BLSTM. SN Comput. Sci. 4, 4 (May 2023). [65] Masood Ghayoomi. 2023. Enriching contextualized semantic representation with textual information transmission for COVID-19 fake news detection: A study on English and Persian. Digital Scholarship in the Humanities 38, 1 (2023), 99–110. [66] Erwin Gielens, Jakub Sowula, and Philip Leifeld. 2025. Goodbye human annotators? Content analysis of social policy debates using ChatGPT. Journal of Social Policy (2025), 1–20. [67] Nir Grinberg, Kenneth Joseph, Lisa Friedland, Briony Swire-Thompson, and David Lazer. 2019. Fake news on Twitter during the 2016 US presidential election. Science 363, 6425 (2019), 374–378. [68] Tianle Gu, Zeyang Zhou, Kexin Huang, Liang Dandan, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Yujiu Yang, Yan Teng, Yu Qiao, et al. 2024. Mllmguard: A multi-dimensional safety evaluation suite for multimodal large language models. Advances in Neural Information Processing Systems 37 (2024), 7256–7295. [69] Qinglang Guo, Haiyong Xie, Yangyang Li, Wen Ma, and Chao Zhang. 2021. Social bots detection via fusing BERT and graph convolutional networks. Symmetry (Basel) 14, 1 (Dec. 2021), 30. [70] Ankur Gupta*, School of Information Technology, RGPV, Bhopal, India., Yogendra P S Maravi, Nishchol Mishra, School of Information Technology, RGPV Bhopal, India., and School of Information Technology, RGPV Bhopal, India. 2019. Twitter Spam Detection using Pre-trained Model. International Journal of Recent Technology and Engineering (IJRTE) 8, 4 (Nov. 2019), 10620–10623. [71] Fouzi Harrag, Maria Dabbah, Kareem Darwish, and Ahmed Abdelali. 2020. Bert Transformer Model for Detecting Arabic GPT2 Auto-Generated Tweets. In Proceedings of the Fifth Arabic Natural Language Processing Workshop (WANLP). Association for Computational Linguistics, Barcelona, Spain (Online), 207–214. https://aclanthology.org/2020.wanlp1.19/ [72] Ehtesham Hashmi, Sule Yildirim Yayilgan, Muhammad Mudassar Yamin, Subhan Ali, and Mohamed Abomhara. 2024. Advancing Fake News Detection: Hybrid Deep Learning With FastText and Explainable AI. IEEE Access 12 (2024), 44462–44480. [73] Katherine Haynes, Hossein Shirazi, and Indrakshi Ray. 2021. Lightweight URL-based phishing detection using natural language processing transformers for mobile devices. Procedia Comput. Sci. 191 (2021), 127–134. [74] Dan Heaton, Jeremie Clos, Elena Nichele, and Joel E Fischer. 2024. “The ChatGPT bot is causing panic now – but it’ll soon be as mundane a tool as Excel”: analysing topics, sentiment and emotions relating to ChatGPT on Twitter. Pers. Ubiquitous Comput. (May 2024). [75] Maryam Heidari and James H Jones. 2020. Using BERT to extract topic-independent sentiment features for social media bot detection. In 2020 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) (New York, NY, USA). IEEE. [76] Xing Hu, Feifei Niu, Junkai Chen, Xin Zhou, Junwei Zhang, Junda He, Xin Xia, and David Lo. 2025. Assessing and advancing benchmarks for evaluating large language models in software engineering tasks. ACM Transactions on Software Engineering and Methodology (2025). [77] Hanjuan Huang, Hsuan-Ting Peng, and Hsing-Kuo Pao. 2023. Fake News Detection via Sentiment Neutralization. In 2023 IEEE International Conference on Big Data (BigData). IEEE, 5780–5789. [78] Jinyuan Huang, Zhao Song, Keliang Li, Shuang Zhang, Sitao Duan, Bo Li, and Haitao Zhao. 2020. InstaHide: Instance-hiding Schemes for Private Discourse on Public Training. In International Conference on Machine Learning (ICML). [79] Kexin Huang, Xiangyang Liu, Qianyu Guo, Tianxiang Sun, Jiawei Sun, Yaru Wang, Zeyang Zhou, Yixu Wang, Yan Teng, Xipeng Qiu, Yingchun Wang, and Dahua Lin. 2024. Flames: Benchmarking Value Alignment of LLMs in Chinese. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Association for Computational Linguistics, Mexico City, Mexico, 4551–4591.

, Vol. 1, No. 1, Article . Publication date: August 2026.

30

Xiong et al.

[80] Steve Huntsman, Michael Robinson, and Ludmilla Huntsman. 2024. Prospects for inconsistency detection using large language models and sheaves. arXiv preprint arXiv:2401.16713 (2024). [81] Sungsoon Jang, Yeseul Cho, Hyeonmin Seong, Taejong Kim, and Hosung Woo. 2024. The Development of a Named Entity Recognizer for Detecting Personal Information Using a Korean Pretrained Language Model. Applied Sciences 14, 13 (2024), 5682. [82] Shan Jiang, Miriam Metzger, Andrew Flanagin, and Christo Wilson. 2020. Modeling and measuring expressed (dis) belief in (mis) information. In Proceedings of the international AAAI conference on web and social media, Vol. 14. 315–326. [83] Weiqiang Jin, Ningwei Wang, Tao Tao, Bohang Shi, Haixia Bi, Biao Zhao, Hao Wu, Haibin Duan, and Guang Yang. 2024. A veracity dissemination consistency-based few-shot fake news detection framework by synergizing adversarial and contrastive self-supervised learning. Scientific Reports 14, 1 (2024), 19470. [84] Bianca Montes Jones and Marwan Omar. 2023. Detection of twitter spam with language models: A case study on how to use BERT to protect children from spam on twitter. In 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE) (Las Vegas, NV, USA). IEEE, 511–516. [85] Niket Tandon Kandpal, Samuel Bowman, Ethan Perez, and Colin Raffel. 2023. Quantifying Memorization Across Neural Language Models. In Proceedings of the International Conference on Learning Representations (ICLR). [86] Debanjana Kar, Mohit Bhardwaj, Suranjana Samanta, and Amar Prakash Azad. 2021. No rumours please! a multiindic-lingual approach for covid fake-tweet detection. (2021), 1–5. [87] Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. 2024. A survey of reinforcement learning from human feedback. Transactions on Machine Learning Research (2024). [88] Kornraphop Kawintiranon, Lisa Singh, and Ceren Budak. 2022. Traditional and context-specific spam detection in low resource settings. Mach. Learn. 111, 7 (July 2022), 2515–2536. [89] Indra Kertati, Carlos Y T Sanchez, Muhammad Basri, Muhammad Najib Husain, and Hery Winoto Tj. 2023. Public relations’ disruption model on chatgpt issue. J. Studi Komun. (Indones. J. Commun. Stud.) 7, 1 (March 2023), 034–048. [90] Ashfia Jannat Keya, Md. Anwar Hussen Wadud, M. F. Mridha, Mohammed Alatiyyah, and Md. Abdul Hamid. 2022. AugFake-BERT: Handling Imbalance through Augmentation of Fake News Using BERT to Enhance the Performance of Fake News Classification. Applied Sciences 12, 17 (2022). [91] M Mehdi Kholoosi, M Ali Babar, and Roland Croft. 2024. A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality. (2024), 107–117. [92] Myeong Gyu Kim, Minjung Kim, Jae Hyun Kim, and Kyungim Kim. 2022. Fine-tuning BERT models to classify misinformation on garlic and COVID-19 on Twitter. Int. J. Environ. Res. Public Health 19, 9 (April 2022), 5126. [93] Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2024. Propile: Probing privacy leakage in large language models. Advances in Neural Information Processing Systems 36 (2024). [94] Sahas Koka, Anthony Vuong, and Anish Kataria. 2024. Evaluating the Efficacy of Large Language Models in Detecting Fake News: A Comparative Analysis. arXiv preprint arXiv:2406.06584 (2024). [95] Shubham Kumar, Shivang Garg, Yatharth Vats, and Anil Singh Parihar. 2021. Content based bot detection using bot language model and BERT embeddings. In 2021 5th International Conference on Computer, Communication and Signal Processing (ICCCSP) (Chennai, India). IEEE. [96] Jianqiao Lai, Xinran Yang, Wenyue Luo, Linjiang Zhou, Langchen Li, Yongqi Wang, and Xiaochuan Shi. 2024. RumorLLM: A Rumor Large Language Model-Based Fake-News-Detection Data-Augmentation Approach. Applied Sciences 14, 8 (2024). [97] Guangchen Lan, Huseyin A Inan, Sahar Abdelnabi, Janardhan Kulkarni, Lukas Wutschitz, Reza Shokri, Christopher G Brinton, and Robert Sim. 2025. Contextual integrity in llms via reasoning and reinforcement learning. arXiv preprint arXiv:2506.04245 (2025). [98] David MJ Lazer, Matthew A Baum, Yochai Benkler, Adam J Berinsky, Kelly M Greenhill, Filippo Menczer, Miriam J Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, et al. 2018. The science of fake news. Science 359, 6380 (2018), 1094–1096. [99] Jaeyoung Lee, Ximing Lu, Jack Hessel, Faeze Brahman, Youngjae Yu, Yonatan Bisk, Yejin Choi, and Saadia Gabriel. 2024. How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 13060–13077. https://doi.org/10.18653/v1/2024.findings-emnlp.764 [100] Siyu Li, Jin Yang, and Kui Zhao. 2023. Are you in a masquerade? exploring the behavior and impact of large language model driven social bots in online social networks. arXiv preprint arXiv:2307.10337 (2023). [101] Wei Li, Jiawen Deng, Jiali You, Yuanyuan He, Yan Zhuang, and Fuji Ren. 2025. ETS-MM: A Multi-Modal Social Bot Detection Model Based on Enhanced Textual Semantic Representation. In Proceedings of the ACM on Web Conference 2025. 4160–4170.

, Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 31

[102] Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. 2022. Large Language Models Can Be Strong Differentially Private Learners. In Proceedings of the International Conference on Learning Representations (ICLR). [103] Xinyi Li, Yongfeng Zhang, and Edward C Malthouse. 2024. Large Language Model Agent for Fake News Detection. arXiv preprint arXiv:2405.01593 (2024). [104] Yufan Li, Zhan Wang, and Theo Papatheodorou. 2024. Staying vigilant in the Age of AI: From content generation to content authentication. arXiv preprint arXiv:2407.00922 (2024). [105] Ying Lian, Huiting Tang, Mengting Xiang, and Xuefan Dong. 2024. Public attitudes and sentiments toward ChatGPT in China: A text mining analysis based on social media. Technol. Soc. 76, 102442 (March 2024), 102442. [106] Luyang Lin, Lingzhi Wang, Jinsong Guo, and Kam-Fai Wong. 2025. Investigating bias in llm-based bias detection: Disparities between llms and human perception. In Proceedings of the 31st International Conference on Computational Linguistics. 10634–10649. [107] Songyang Liu, Chaozhuo Li, Jiameng Qiu, Xi Zhang, Feiran Huang, Litian Zhang, Yiming Hei, and Philip S Yu. 2025. The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs. arXiv preprint arXiv:2506.11094 (2025). [108] Xuannan Liu, Zekun Li, Pei Pei Li, Huaibo Huang, Shuhan Xia, Xing Cui, Linzhi Huang, Weihong Deng, and Zhaofeng He. [n. d.]. MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs. In The Thirteenth International Conference on Learning Representations. [109] Xiaohan Liu, Yue Zhan, Hao Jin, Yuan Wang, and Yi Zhang. 2023. Research on the classification methods of social bots. Electronics (Basel) 12, 14 (July 2023), 3030. [110] Yuhan Liu, Xiuying Chen, Xiaoqing Zhang, Xing Gao, Ji Zhang, and Rui Yan. 2024. From Skepticism to Acceptance: Simulating the Attitude Dynamics Toward Fake News. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Kate Larson (Ed.). International Joint Conferences on Artificial Intelligence Organization, 7886–7894. Human-Centred AI. [111] Yifan Liu, Yaokun Liu, Zelin Li, Ruichen Yao, Yang Zhang, and Dong Wang. 2025. Modality interactive mixture-ofexperts for fake news detection. In Proceedings of the ACM on Web Conference 2025. 5139–5150. [112] Ye Liu, Jiajun Zhu, Kai Zhang, Haoyu Tang, Yanghai Zhang, Xukai Liu, Qi Liu, and Enhong Chen. 2024. Detect, Investigate, Judge and Determine: A Novel LLM-based Framework for Few-shot Fake News Detection. arXiv preprint arXiv:2407.08952 (2024). [113] Zhuoran Lu, Sheshera Mysore, Tara Safavi, Jennifer Neville, Longqi Yang, and Mengting Wan. 2024. Corporate communication companion (CCC): An LLM-empowered writing assistant for workplace social media. arXiv preprint arXiv:2405.04656 (2024). [114] Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 30. 4765–4774. [115] Hanjia Lyu, Jinfa Huang, Daoan Zhang, Yongsheng Yu, Xinyi Mou, Jinsheng Pan, Zhengyuan Yang, Zhongyu Wei, and Jiebo Luo. 2025. GPT-4V(ision) as A Social Media Analysis Engine. ACM Trans. Intell. Syst. Technol. 16, 3, Article 50 (April 2025), 54 pages. https://doi.org/10.1145/3709005 [116] SreeJagadeesh Malla and PJA Alphonse. 2022. Fake or real news about COVID-19? Pretrained transformer model to detect potential misleading news. The European Physical Journal Special Topics 231, 18 (2022), 3347–3356. [117] J. Martin, P. Johnson, and D. Chang. 2022. AI and Mental Health: Privacy Challenges and Solutions. IEEE Transactions on Computational Social Systems 9, 3 (2022), 180–192. [118] Justus Mattern, Benjamin Weggenmann, and Florian Kerschbaum. 2022. The Limits of Word Level Differential Privacy. In Findings of the Association for Computational Linguistics: NAACL 2022, Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (Eds.). Association for Computational Linguistics, Seattle, United States, 867–881. https://doi.org/10.18653/v1/2022.findings-naacl.65 [119] Nikhil Mehta and Dan Goldwasser. 2024. Using RL to Identify Divisive Perspectives Improves LLMs Abilities to Identify Communities on Social Media. In Findings of the Association for Computational Linguistics: EMNLP 2024. 5371–5390. [120] Meta Platforms, Inc. 2023. Facebook Privacy Policy. Available at https://www.facebook.com/privacy/policy, accessed March 2024. [121] Zhongtao Miao, Qiyu Wu, Kaiyan Zhao, Zilong Wu, and Yoshimasa Tsuruoka. 2024. Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment. In Findings of the Association for Computational Linguistics: NAACL 2024, Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 3225–3236. https://doi.org/10.18653/v1/2024.findings-naacl.204 [122] Maria Milkova, Maksim Rudnev, and Lidia Okolskaya. 2023. Detecting value-expressive text posts in Russian social media. arXiv preprint arXiv:2312.08968 (2023). [123] Helen Milner and Michael Baron. 2023. Establishing an optimal online phishing detection method: Evaluating topological NLP transformers on text message data. Journal of Data Science and Intelligent Systems 2, 1 (July 2023), 37–45.

, Vol. 1, No. 1, Article . Publication date: August 2026.

32

Xiong et al.

[124] Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022. Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 8332–8347. https://doi.org/10.18653/ v1/2022.emnlp-main.570 [125] Yichuan Mo, Yuji Wang, Zeming Wei, and Yisen Wang. 2024. Fight back against jailbreaking via prompt adversarial tuning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [126] Robert T. Morris, Florian Tramer, and Nicholas Carlini. 2024. Language Model Inversion: Recovering Training Data from Language Models. In Proceedings of the International Conference on Learning Representations (ICLR). [127] Kai Nakamura, Sharon Levy, and William Yang Wang. 2020. Fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection. In Proceedings of the twelfth language resources and evaluation conference. 6149–6157. [128] Qiong Nan, Qiang Sheng, Juan Cao, Beizhe Hu, Danding Wang, and Jintao Li. 2024. Let silence speak: Enhancing fake news detection with generated comments from large language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 1732–1742. [129] Helen Nissenbaum. 2019. Contextual Integrity Up and Down the Data Food Chain. Theoretical Inquiries in Law 20, 1 (2019), 221–256. [130] State of California Department of Justice. 2026. California Consumer Privacy Act (CCPA). https://oag.ca.gov/privacy/ ccpa. [131] OpenAlex Help Center. 2024. Where do works in OpenAlex come from? https://help.openalex.org/hc/en-us/articles/ 24347019383191-Where-do-works-in-OpenAlex-come-from. Accessed: 2026-02-02. [132] Nikolaos Panagiotou, Antonia Saravanou, and Dimitrios Gunopulos. 2021. News Monitor: A framework for exploring news in real-time. Data (Basel) 7, 1 (Dec. 2021), 3. [133] Subhadarshi Panda and Sarah Ita Levitan. 2021. Detecting multilingual COVID-19 misinformation on social media via contextualized embeddings. In Proceedings of the Fourth Workshop on NLP for Internet Freedom: Censorship, Disinformation, and Propaganda (Online). Association for Computational Linguistics, Stroudsburg, PA, USA. [134] Stefanos-Iordanis Papadopoulos, Christos Koutlis, Symeon Papadopoulos, and Panagiotis C Petrantonakis. 2024. Verite: a robust benchmark for multimodal misinformation detection accounting for unimodal bias. International Journal of Multimedia Information Retrieval 13, 1 (2024), 4. [135] Constantinos Patsakis and Nikolaos Lykousas. 2023. Man vs the machine in the struggle for effective text anonymisation in the age of large language models. Scientific Reports 13, 1 (2023), 16026. [136] David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. 2021. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 (2021). [137] Protik Bose Pranto, Syed Zami-Ul-Haque Navid, Protik Dey, Gias Uddin, and Anindya Iqbal. 2022. Are you misinformed? a study of covid-related fake news in bengali on facebook. arXiv preprint arXiv:2203.11669 (2022). [138] Haritz Puerto, Martin Gubri, Sangdoo Yun, and Seong Joon Oh. 2025. Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models. In Findings of the Association for Computational Linguistics: NAACL 2025, Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 4165–4182. https://doi.org/10.18653/v1/2025.findings-naacl.234 [139] Sai Puppala, Ismail Hossain, Md Jahangir Alam, and Sajedul Talukder. 2024. FLASH: Federated Learning-Based LLMs for Advanced Query Processing in Social Networks through RAG. (2024), 281–293. [140] Sai Puppala, Ismail Hossain, Md Jahangir Alam, and Sajedul Talukder. 2024. SocFedGPT: Federated GPT-Based Adaptive Content Filtering System Leveraging User Interactions in Social Networks. In International Conference on Advances in Social Networks Analysis and Mining. Springer, 79–88. [141] Shengsheng Qian, Jinguang Wang, Jun Hu, Quan Fang, and Changsheng Xu. 2021. Hierarchical Multi-modal Contextual Attention Network for Fake News Detection. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 153–162. [142] Kristina Radivojevic, Nicholas Clark, and Paul Brenner. 2024. LLMs Among Us: Generative AI participating in digital discourse. Proceedings of the AAAI Symposium Series 3, 1 (May 2024), 209–218. [143] Sarika S Raga and Chaitra B. 2022. A bert model for sms and twitter spam ham classification and comparative study of machine learning and deep learning technique. In 2022 IEEE 7th International Conference on Recent Advances and Innovations in Engineering (ICRAIE) (MANGALORE, India). IEEE. [144] David Ramamonjisoa and Shuma Suzuki. 2024. Comments Analysis in Social Media based on LLM Agents. In CS & IT Conference Proceedings, Vol. 14. [145] Junaid Rashid, Jungeun Kim, and Anum Masood. 2024. Unraveling the Tangle of Disinformation: A Multimodal Approach for Fake News Identification on Social Media. In Companion Proceedings of the ACM Web Conference 2024

, Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 33 (Singapore, Singapore) (WWW ’24). Association for Computing Machinery, New York, NY, USA, 1849–1853. [146] V Rathinapriya and J Kalaivani. 2024. Adaptive weighted feature fusion for multiscale atrous convolution-based 1DCNN with dilated LSTM-aided fake news detection using regional language text information. Expert Systems (2024), e13665. [147] Prismahardi Aji Riyantoko, Tresna Maulana Fahrudin, Dwi Arman Prasetya, Trimono Trimono, and Tahta Dari Timur. 2022. Analisis Sentimen Sederhana Menggunakan Algoritma LSTM dan BERT untuk Klasifikasi Data Spam dan Non-Spam. PROSIDING SEMINAR NASIONAL SAINS DATA 2, 1 (Dec. 2022), 103–111. [148] Daniel Russo, Serra Sinem Tekiroğlu, and Marco Guerini. 2023. Benchmarking the Generation of Fact Checking Explanations. Transactions of the Association for Computational Linguistics 11 (2023), 1250–1264. [149] Marko Sahan, Vaclav Smidl, and Radek Marik. 2021. Active Learning for Text Classification and Fake News Detection. In 2021 International Symposium on Computer Science and Intelligent Controls (ISCSIC). 87–94. [150] Siva Sai. 2020. Siva at WNUT-2020 task 2: Fine-tuning transformer neural networks for identification of informative covid-19 tweets. In Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020) (Online). Association for Computational Linguistics, Stroudsburg, PA, USA. [151] Amine Sallah, El Arbi Abdellaoui Alaoui, Said Agoujil, Mudasir Ahmad Wani, Mohamed Hammad, Yassine Maleh, and Ahmed A Abd El-Latif. 2024. Fine-tuned understanding: Enhancing social bot detection with transformer-based classification. IEEE Access 12 (2024), 118250–118269. [152] Kanti Singh Sangher, Archana Singh, and Hari Mohan Pandey. 2024. LSTM and BERT based transformers models for cyber threat intelligence for intent identification of social media platforms exploitation from darknet forums. Int. J. Inf. Technol. 16, 8 (Dec. 2024), 5277–5292. [153] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019). [154] Filippo Santoni de Sio and Giulio Mecacci. 2021. Four responsibility gaps with artificial intelligence: Why they matter and how to address them. Philosophy & technology 34, 4 (2021), 1057–1084. [155] Ali Satvaty, Suzan Verberne, and Fatih Turkmen. 2024. Undesirable Memorization in Large Language Models: A Survey. In arXiv preprint arXiv:2410.02650. [156] Isabel Segura-Bedmar and Santiago Alonso-Bartolome. 2022. Multimodal fake news detection. Information 13, 6 (2022), 284. [157] Samaneh Shafee, Alysson Bessani, and Pedro M Ferreira. 2025. Evaluation of LLM-based chatbots for OSINT-based Cyber Threat Awareness. Expert Systems with Applications 261 (2025), 125509. [158] Siddhant Bikram Shah, Surendrabikram Thapa, Ashish Acharya, Kritesh Rauniyar, Sweta Poudel, Sandesh Jain, Anum Masood, and Usman Naseem. 2024. Navigating the web of disinformation and misinformation: Large language models as double-edged swords. IEEE Access (2024), 1–1. [159] Fangfang Shan, Huifang Sun, and Mengyi Wang. 2024. Multimodal Social Media Fake News Detection Based on Similarity Inference and Adversarial Networks. Computers, Materials & Continua 79, 1 (2024). [160] Filipo Sharevski, Jennifer Vander Loop, Peter Jachim, Amy Devine, and Emma Pieroni. 2023. Talking abortion (mis) information with chatgpt on tiktok. In 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, 594–608. [161] Karishma Sharma, Emilio Ferrara, and Yan Liu. 2022. Construction of large-scale misinformation labeled datasets from social media discourse using label refinement. In Proceedings of the ACM Web Conference 2022. 3755–3764. [162] Utkarsh Sharma, Prateek Pandey, and Shishir Kumar. 2022. A transformer-based model for evaluation of information relevance in online social-media: A case study of Covid-19 media posts. New Gener. Comput. 40, 4 (Jan. 2022), 1029–1052. [163] Sachin Ashok Shinde et al. 2025. NLP-Based Investigation of Textual and Semantic Cues in Fake News Identification. In 2025 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE). IEEE, 235–240. [164] Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2020. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data 8, 3 (2020), 171–188. [165] Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter 19, 1 (2017), 22–36. [166] Kai Shu, Suhang Wang, Dongwon Lee, and Huan Liu. 2020. Mining disinformation and fake news: Concepts, methods, and recent advancements. Disinformation, misinformation, and fake news in social media: Emerging research challenges and opportunities (2020), 1–19. [167] Utsav Shukla, Manan Vyas, and Shailendra Tiwari. 2023. Raphael at ArAIEval shared task: Understanding persuasive language and tone, an LLM approach. In Proceedings of ArabicNLP 2023 (Singapore (Hybrid)). Association for Computational Linguistics, Stroudsburg, PA, USA, 589–593.

, Vol. 1, No. 1, Article . Publication date: August 2026.

34

Xiong et al.

[168] Nathalie A Smuha. 2025. Regulation 2024/1689 of the Eur. Parl. & Council of June 13, 2024 (EU Artificial Intelligence Act). International Legal Materials (2025), 1–148. [169] Giovanni Spitale, Nikola Biller-Andorno, and Federico Germani. 2023. AI model GPT-3 (dis) informs us better than humans. Science Advances 9, 26 (2023), eadh1850. [170] Robin Staab, Mark Vero, Mislav Balunović, and Martin Vechev. 2024. Beyond Memorization: Violating Privacy via Inference with Large Language Models. In Proceedings of the International Conference on Learning Representations. [171] Andrei Stipiuc. 2024. Romanian Media Landscape in 7 Journalists’ Facebook Posts: A ChatGPT Sentiment Analysis. SAECULUM 57, 1 (2024), 20–46. [172] Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th annual meeting of the association for computational linguistics. 3645–3650. [173] Ningxin Su, Chenghao Hu, Baochun Li, and Bo Li. 2024. TITANIC: Towards Production Federated Learning with Large Language Models. In Proceedings of IEEE INFOCOM. [174] Megha Sundriyal, Harshit Choudhary, Tanmoy Chakraborty, and Md Shad Akhtar. 2024. Crowd Intelligence for Early Misinformation Prediction on Social Media. arXiv preprint arXiv:2408.04463 (2024). [175] B N1 Supriya and CB Akki. 2022. P-BERT: Polished Up Bidirectional Encoder Representations from Transformers for Predicting Malicious URL to Preserve Privacy. In 2022 IEEE 9th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON). IEEE, 1–6. [176] Yingjie Tian and Yuhao Xie. 2024. Artificial cheerleading in IEO: Marketing campaign or pump and dump scheme. Inf. Process. Manag. 61, 1 (Jan. 2024), 103537. [177] Ahmed Tlili, Boulus Shehata, Michael Agyemang Adarkwah, Aras Bozkurt, Daniel T Hickey, Ronghuai Huang, and Brighter Agyemang. 2023. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn. Environ. 10, 1 (Feb. 2023). [178] Christopher K Tokita, Kevin Aslett, William P Godel, Zeve Sanderson, Joshua A Tucker, Jonathan Nagler, Nathaniel Persily, and Richard Bonneau. 2024. Measuring receptivity to misinformation at scale on a social media platform. PNAS nexus 3, 10 (2024), pgae396. [179] Batuhan Tömekçe, Mark Vero, Robin Staab, and Martin Vechev. 2024. Private attribute inference from images with vision-language models. Advances in Neural Information Processing Systems 37 (2024), 103619–103651. [180] Ben Treves, Md Rayhanul Masud, and Michalis Faloutsos. 2023. RURLMAN: Matching Forum Users Across Platforms Using Their Posted URLs. In Proceedings of the International Conference on Advances in Social Networks Analysis and Mining. 484–491. [181] Milena Tsvetkova, Taha Yasseri, Niccolo Pescetelli, and Tobias Werner. 2024. A new sociology of humans and machines. Nature Human Behaviour 8, 10 (Oct. 2024), 1864–1876. http://dx.doi.org/10.1038/s41562-024-02001-8 [182] Zeynep Tufekci. 2017. Twitter and tear gas: The power and fragility of networked protest. Yale University Press. [183] UNHCR. 2021. Using Social Media in Community-Based Protection. Retrieved January, 2021 from https://www.unhcr. org/innovation/wp-content/uploads/2021/01/Using-Social-Media-in-CBP.pdf [184] GDPR European Union. 2026. Complete guide to GDPR compliance. https://gdpr.eu/. [185] Standford University. 2024. The 2024 AI Index Report. https://hai.stanford.edu/ai-index/2024-ai-index-report. [186] Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. science 359, 6380 (2018). [187] Nazmi Ekin Vural and Sefer Kalaman. 2024. Using Artificial Intelligence Systems in News Verification: An Application on X. İletişim Kuram ve Araştırma Dergisi 67 (2024), 127–141. [188] William Yang Wang. 2017. “Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL). 422–426. [189] Xiao Wang, Jinyuan Sun, Jinyuan Zhang, Neil Shah, and Bo Li. 2024. Prompt Inversion: Leveraging Attention-Based Inversion to Recover Prompts from Language Models. In Proceedings of the International Conference on Learning Representations (ICLR). [190] Claire Wardle and Hossein Derakhshan. 2017. Information disorder: Toward an interdisciplinary framework for research and policymaking. Vol. 27. Council of Europe Strasbourg. [191] Sam Wiseman, Stuart M. Shieber, and Alexander M. Rush. 2018. Learning Neural Templates for Text Generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 3174–3187. [192] Yunfei Xing, Justin Zuopeng Zhang, Guangqing Teng, and Xiaotang Zhou. 2024. Voices in the digital storm: Unraveling online polarization with ChatGPT. Technology in Society 77 (2024), 102534. [193] Junjie Xiong, Changjia Zhu, Shuhang Lin, Chong Zhang, Yongfeng Zhang, Yao Liu, and Lingyao Li. 2025. Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2025. [194] Guangxia Xu, Daiqi Zhou, and Jun Liu. 2021. Social network spam detection based on ALBERT and combination of Bi-LSTM with self-attention. Secur. Commun. Netw. 2021 (April 2021), 1–11.

, Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 35

[195] Junhao Xu, Longdi Xian, Zening Liu, Mingliang Chen, Qiuyang Yin, and Fenghua Song. 2024. The future of combating rumors? retrieval, discrimination, and generation. arXiv preprint arXiv:2403.20204 (2024). [196] Keyang Xuan, Li Yi, Fan Yang, Ruochen Wu, Yi R Fung, and Heng Ji. 2024. LEMMA: towards LVLM-enhanced multimodal misinformation detection with external knowledge augmentation. arXiv preprint arXiv:2402.11943 (2024). [197] Hongwei Yan, Liyuan Wang, Kaisheng Ma, and Yi Zhong. 2024. Orchestrate latent expertise: Advancing online continual learning with multi-level supervision and reverse self-distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 23670–23680. [198] Chang Yang, Peng Zhang, Wenbo Qiao, Hui Gao, and Jiaming Zhao. 2023. Rumor detection on social media with crowd intelligence and ChatGPT-assisted networks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 5705–5717. [199] Kaicheng Yang and Filippo Menczer. 2024. Anatomy of an AI-powered malicious social botnet. J. Quant. Descr. Digit. Media 4 (May 2024). [200] Yingguang Yang, Renyu Yang, Hao Peng, Yangyang Li, Tong Li, Yong Liao, and Pengyuan Zhou. 2023. FedACK: Federated adversarial contrastive knowledge distillation for cross-lingual and cross-model social bot detection. In Proceedings of the ACM Web Conference 2023. 1314–1323. [201] Barry Menglong Yao, Aditya Shah, Lichao Sun, Jin-Hee Cho, and Lifu Huang. 2023. End-to-end multimodal factchecking and explanation generation: A challenging dataset and models. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2733–2743. [202] Xin Yao, Tianchi Huang, Chenglei Wu, Rui-Xiao Zhang, and Lifeng Sun. 2019. Federated learning with additional mechanisms on clients to reduce communication costs. arXiv preprint arXiv:1908.05891 (2019). [203] Yuhang Yao, Jianyi Zhang, Junda Wu, Chengkai Huang, Yu Xia, Tong Yu, Ruiyi Zhang, Sungchul Kim, and Rossi. 2024. Federated large language models: Current progress and future directions. arXiv preprint arXiv:2409.15723 (2024). [204] Junshuai Yu, Qi Huang, Xiaofei Zhou, and Ying Sha. 2020. IARNet: An Information Aggregating and Reasoning Network over Heterogeneous Graph for Fake News Detection. In 2020 International Joint Conference on Neural Networks (IJCNN). 1–9. [205] Zhenrui Yue, Huimin Zeng, Yang Zhang, Lanyu Shang, and Dong Wang. 2023. MetaAdapt: Domain Adaptive FewShot Misinformation Detection via Meta Learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Canada, 5223–5239. https://doi.org/10.18653/v1/2023.acl-long.286 [206] Hanna Yukhymenko, Robin Staab, Mark Vero, and Martin Vechev. 2024. A synthetic dataset for personal attribute inference. In Proceedings of the 38th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS ’24). Curran Associates Inc., Red Hook, NY, USA, Article 3837, 45 pages. [207] Jinyuan Zhang, Xinyue Chen, Xuechen Li, and Cho-Jui Hsieh. 2024. Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration. In Advances in Neural Information Processing Systems (NeurIPS). [208] Tong Zhang, Di Wang, Huanhuan Chen, Zhiwei Zeng, Wei Guo, Chunyan Miao, and Lizhen Cui. 2020. BDANN: BERT-Based Domain Adaptation Neural Network for Multi-Modal Fake News Detection. In 2020 International Joint Conference on Neural Networks (IJCNN). 1–8. [209] Xiang Zhang, Yufei Cui, Chenchen Fu, Zihao Wang, Yuyang Sun, Xue Liu, and Weiwei Wu. 2025. Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming Perception. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 10185–10193. [210] Yiming Zhang, Sravani Nanduri, Liwei Jiang, Tongshuang Wu, and Maarten Sap. 2023. BiasX: “Thinking Slow” in Toxic Content Moderation with Explanations of Implied Social Biases. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 4920–4932. https://doi.org/10.18653/v1/2023.emnlp-main.300 [211] Yizhou Zhang, Karishma Sharma, Lun Du, and Yan Liu. 2024. Toward mitigating misinformation and social media manipulation in LLM era. In Companion Proceedings of the ACM on Web Conference 2024 (Singapore Singapore), Vol. 19. ACM, New York, NY, USA, 1302–1305. [212] Chenye Zhao and Cornelia Caragea. 2021. Knowledge distillation with BERT for image tag-based privacy prediction. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021). [213] Xinyi Zhou, Ashish Sharma, Amy X Zhang, and Tim Althoff. 2024. Correcting misinformation on social media with a large language model. arXiv preprint arXiv:2403.11169 (2024). [214] Xinyi Zhou and Reza Zafarani. 2020. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR) 53, 5 (2020), 1–40. [215] Liu Zhuang, Lin Wayne, Shi Ya, and Zhao Jun. 2021. A Robustly Optimized BERT Pre-training Approach with Post-training. In Proceedings of the 20th Chinese National Conference on Computational Linguistics, Sheng Li, Maosong Sun, Yang Liu, Hua Wu, Kang Liu, Wanxiang Che, Shizhu He, and Gaoqi Rao (Eds.). Chinese Information Processing Society of China, Huhhot, China, 1218–1227. https://aclanthology.org/2021.ccl-1.108/

, Vol. 1, No. 1, Article . Publication date: August 2026.

36

A A.1

Xiong et al.

Supplementary Material: Extended Landscape Analysis Extended Landscape Analysis Topic 1: Information Disorder

How LLMs Enhance Detection and Prevention. LLMs’ potential in misinformation detection has evolved from foundational approaches to sophisticated methods. As shown in Table 3, detection techniques can be broadly categorized based on their methodology, including early BERT-based models and more advanced fine-tuned, knowledge-based approaches. Early BERT-based methods demonstrated initial capabilities in misinformation detection tasks [3, 49, 82]. For example, Dulhanty et al. [49] explores automated methods through position detection, while Jiang et al. [82] develops BERT-based transfer-learning models. Strong capability as the LLMs have, their performance on unseen data or circumstances, named as zero-shot setting, can be further improved [30]. Recent advances have produced several specialized detection strategies. Fine-tuned models like FACT-GPT [33], which leverages GPT-4 generated datasets for automated fact-checking, and FactLLaMA [31], which employs instruct-tuning for automated verification, represent significant progress in detection capabilities. Knowledge-based approaches have further enhanced detection systems, with ChatGPT being used to construct knowledge-based semantic structures [198]. Multimodal approaches have emerged through the integration of LLaVA and CLIP for knowledge transfer [99], while LEMMA [196] combines Large Vision Language Models with Chain-of-Thought reasoning for comprehensive detection. Recent frameworks like MUSE [213] attempt to address this through automated fact-checking using online credible sources, generating queries, and providing explanations with credible source links. Advanced adaptive techniques have also been developed, such as MetaAdapt [205], which employs meta-learning for domain-adaptive few-shot detection, demonstrating superior performance over existing baselines. Zero-shot detection methods [30] have shown promise in identifying both human-written and LLM-generated misinformation, with performance varying based on content characteristics. Detection performance is influenced by multiple factors, with content length being a key determinant—longer texts often yield higher accuracy due to richer contextual cues [30]. To address domain shifts and imbalanced data distributions, adaptive methods such as MetaAdapt have been developed [205]. In parallel, researchers have explored advanced prompting strategies and multi-step verification workflows [99, 198], incorporating multimodal tools like CLIP and LLaVA to further enhance model robustness [99]. Despite these advances, significant challenges remain. Detection systems still struggle with subtle manipulations and short-form content, while accuracy varies based on text origin and model architecture. The rapid evolution of generative models necessitates continuous updates to detection mechanisms, as evidenced by systems like FACT-GPT [33] and FactLLaMA [31]. Social media platforms have fundamentally transformed fake news propagation, enabling rapid and extensive reach with minimal resources [90, 141, 187, 208]. This accelerated dissemination significantly erodes public trust in both social media platforms and institutions [83, 156, 204]. Realworld examples of fake news reveal how easily misinformation can be framed to appear credible, emphasizing the need for timely and accurate detection mechanisms. The challenge is particularly acute in low-resource settings and across linguistic barriers. For example, Rathinapriya et al. [146] reveal substantial accuracy drops when detection systems trained on high-resource languages were applied to regional Indian languages, emphasizing the critical gap in cross-lingual detection capabilities. The speed and scale of social media sharing make traditional fact-checking methods increasingly inadequate, as false information can reach massive audiences before corrections can be implemented [112]. The increasing prevalence of multimedia content presents significant technical challenges [5, 53]. Current detection frameworks face multiple limitations: they rely heavily on human annotation,

, Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 37

limiting scalability [66]; they struggle with cross-platform content verification; and they often fail to capture context-dependent nuances. Additionally, generative AI advancements enable the creation of increasingly sophisticated and credible false content [104], making detection more complex. These technical challenges are particularly evident in real-time detection scenarios, where the need for immediate response conflicts with the requirement for accurate verification. The societal impact of fake news manifests across multiple critical domains. In healthcare, misinformation during the COVID-19 pandemic led to public confusion, vaccine hesitancy, and inadequate crisis responses [13, 29, 65]. In politics, fake news serves as a powerful tool for manipulation and polarization [171, 192], undermining democratic processes and public discourse. In economics, it disrupts markets and consumer behavior through false financial information and manipulated market signals [72, 96]. These problems are particularly severe in regions with limited media literacy [112, 146, 149], where both active and passive consumers [128] can be significantly influenced by false narratives. Fake news exploits emotional and cultural elements to enhance its credibility and impact [53, 145, 192]. Its sophisticated adaptation to regional contexts and local sensitivities maximizes societal impact [112, 146], particularly through multimedia elements that increase persuasiveness [5, 53]. The emotional manipulation often targets existing societal divisions, amplifying conflicts and polarization. These challenges are further intensified by social media’s connectivity [90, 141, 187, 208], where echo chambers and algorithmic content promotion can rapidly spread misinformation within susceptible communities, significantly affecting public discourse and social cohesion. A.2

Extended Landscape Analysis Topic 2: Social Bot

LLM-enhanced bots demonstrate significant potential in maintaining and enhancing information integrity on social media platforms. These advanced bots serve as powerful tools for automated content verification, real-time fact-checking, and proactive misinformation prevention through their sophisticated pattern recognition and contextual analysis capabilities Panagiotou et al. [132], Xu et al. [194]. Their multilingual abilities enable cross-language content verification [61, 122], while their advanced natural language processing capabilities allow for more accurate identification of suspicious patterns in both user behavior and content distribution [62, 64]. The integration of LLMs has particularly enhanced bots’ ability to understand nuanced context, engage in interactive fact-checking, and provide immediate feedback with educational resources about potential misinformation [73, 157]. Emerging applications suggest a promising future where LLM-enhanced bots can participate in collaborative verification systems, implement adaptive learning mechanisms for improved detection accuracy, and deploy context-aware intervention systems for maintaining the health and reliability of social media information ecosystems [140, 194]. The evolution of LLM-enhanced bots and their impact on information integrity has been a growing concern since 2019. Early research by Gupta et al. [70] and Ghanem et al. [63] revealed how malicious bots were increasingly adopting transformer-based architectures to generate more convincing spam content on Twitter. While their detection models achieved accuracy rates above 98% for traditional bot behaviors, these studies also pointed out an emerging challenge: as bots incorporated more sophisticated LLM capabilities, they became increasingly difficult to distinguish from human users. Recent studies by Li et al. [100] and Yang et al. [199] further revealed how these bots evolved to form large-scale coordinated networks, manipulating information across various domains, from health misinformation to financial markets [176]. The COVID-19 pandemic marked a critical period in the evolution of malicious bot capabilities. Multiple research teams documented the spread of health-related misinformation. Sai et al. [150] and Kar et al. [86] demonstrated how LLM-enhanced bots effectively disseminated COVID-19 misinformation across multiple languages. This was further confirmed by Kim et al. [92] and Sharma et al. [162], who found these bots achieved high accuracy in mimicking legitimate health information sources. Pranto et al. [137] specifically , Vol. 1, No. 1, Article . Publication date: August 2026.

38

Xiong et al.

identified ten distinct topics in fake news, illustrating the sophisticated nature of cross-lingual misinformation campaigns. Recent advances in multimodal capabilities have further complicated the threat landscape. Lyu et al. [115] show how GPT-4V-powered bots can effectively manipulate both textual and visual content. This capability was further explored by Mehta et al. [119], who demonstrated how these bots form sophisticated user communities to amplify their impact. The integration of advanced language models has enabled these bots to generate highly personalized phishing content, as documented by Haynes et al. [73] and Atzori et al. [11], who showed how bots exploit vulnerabilities across multiple social networks. The cross-lingual capabilities of these bots present particular challenges. Studies by Shukla et al. [167] and Ahmed et al. [2] revealed how LLM-enhanced bots effectively adapt their content across different languages and cultural contexts. This adaptability was further demonstrated by Harrag et al. [71] in their analysis of Arabic language manipulation, achieving deception rates of up to 98%. The development of defense mechanisms against LLM-enhanced bots has evolved into a multilayered approach, incorporating various technological and methodological innovations. Advanced Detection Architectures have emerged as a primary defense strategy. Alshattnawi et al. [7] report improved spam detection performance using deep neural networks with contextualized word embeddings. In their experimental setting, gains on the order of 10–15% over selected traditional baselines are observed, though these results are tied to specific datasets and evaluation protocols. This work was complemented by Sangher et al. [152], who integrated LSTM and BERT-based transformers to identify sophisticated cybercrime patterns. The effectiveness of these approaches was further validated by Shafee et al. [157], who evaluated various LLM chatbots for cybersecurity threat awareness. Proactive prevention systems represent another crucial defensive layer. Huntsman et al. [80] proposed innovative approaches using LLMs and sheaf theory to detect textual inconsistencies, while Milner et al. [123] developed lightweight phishing detection algorithms specifically optimized for mobile devices. These systems have been enhanced by Puppala et al. [139], who introduced a Federated Learning-based GPT system that ensures privacy while maintaining bot detection effectiveness. Cross-platform and Multilingual Defense strategies have become increasingly important. Liu et al. [109] developed early identification methods using topic expansion and SBERT models, while Askari et al. [10] conducted field experiments to evaluate the effectiveness of bot-based interventions in promoting legitimate news consumption. These approaches demonstrate the importance of comprehensive, platform-agnostic defense mechanisms. To move beyond static veracity checks, we distinguish between three hierarchical detection tiers: (i) Account-level classification, which focuses on profile metadata and historical longevity; (ii) Content-level detection, leveraging LLMs to identify stylistic "fingerprints" and semantic inconsistencies; and (iii) Campaign-level discovery, which identifies coordinated inauthentic behavior (CIB) through network-wide signals. Standard bot detection benchmarks often suffer from inflated performance due to random sampling. We argue for more rigorous negative sampling strategies, such as selecting "active-benign" users who share high-frequency posting habits with bots but lack coordination signatures. Furthermore, to ensure robustness against covariate shift, models must be evaluated via temporal splits (e.g., training on historical election data and testing on current discourse) and cross-platform transfer (e.g., training on X/Twitter and testing on Mastodon). This reveals the "generalization gap" where detectors often fail as bot personas evolve. While LLMs excel at semantic analysis, they are often blind to non-textual behavioral cues. A robust detection pipeline requires ablations on the following features: Network Homophily: Quantifying the tendency of bots to form dense, interconnected k-cores that are statistically distinct from human "small-world" networks. Temporal Burstiness: Measuring synchronized activity peaks across multiple accounts, which indicates automated scheduling or coordinated human-trolling. Action-Type Entropy: Humans typically exhibit high-entropy behavioral sequences (e.g., liking, then browsing, , Vol. 1, No. 1, Article . Publication date: August 2026.

Large Language Models and Social Media Information Integrity: Opportunities, Challenges, and Research Directions 39

then replying), whereas bots often follow low-entropy, repetitive state transitions. The "asymmetric co-evolution" of bot tactics leads to rapid concept drift. We identify Test-Time Adaptation (TTA) and Continual Learning as primary countermeasures. However, these introduce pseudo-labeling risks, where a model may inadvertently "self-train" on its own false positives, leading to the suppression of legitimate minority voices. Future research must prioritize uncertainty-aware models that can flag high-drift scenarios for human-in-the-loop review rather than relying on brittle, automated thresholding. A.3

Extended Landscape Analysis Topic 3: Privacy

Privacy-preserving techniques for LLM-enhanced environments show promising potential in addressing current privacy challenges. Knowledge distillation methods proposed by Zhao and Caragea [212] demonstrate how label prediction can effectively abstract content while minimizing sensitive data exposure. In URL filtering contexts, P-BERT introduced by Supriya and Akki [175] shows significant potential in integrating deep feature extraction with BERT for malicious link detection while reducing leakage risks. These advancements directly address the limitations identified by Patsakis and Lykousas [135] and Mattern et al. [118], where traditional anonymization methods fail against LLMs’ ability to infer user attributes from residual linguistic signals. Advanced privacy protection mechanisms are emerging to tackle the issue of raw data exposure during processing. Differential privacy approaches show particular promise, with Coffey et al. [36] demonstrating how differentially private stochastic gradient descent during fine-tuning can effectively limit model memorization of individual data points. Privacy-preserving synthetic data generation, such as SynthPAI by Yukhymenko et al. [206], offers a solution to avoid direct handling of personal information. Additionally, language-specific approaches for PII protection, as demonstrated by Jang et al. [81] for Korean data, provide promising templates for extending privacy safeguards across different linguistic contexts. Emerging solutions for deep anonymization are addressing the limitations of conventional de-identification methods. Linguistic steganography techniques Coffey et al. [36] show potential in encoding sensitive content in forms only intelligible to intended recipients, while authorship obfuscation frameworks Bao and Carpuat [15] demonstrate how reinforcement learning can effectively conceal personal stylistic features without compromising meaning. These approaches directly address the challenges identified by Patsakis and Lykousas [135] and Mattern et al. [118] regarding LLMs’ ability to infer user traits through subtle linguistic cues. Real-time monitoring and accountability systems represent crucial advancements in privacy protection. For example, Kim et al. [93] and Asimopoulos et al. [9] demonstrate potential in proactively detecting and suppressing sensitive information in model outputs. Federated learning approaches by Yao et al. [202, 203] show promise in reducing centralized data storage vulnerabilities, while explainable privacy frameworks proposed by Mireshghallah et al. [124] offer solutions for transparency and accountability concerns. These developments directly address the need for context-aware privacy reasoning in diverse and dynamic social media interactions, providing comprehensive solutions for protecting user data while maintaining system functionality. In recent years, significant advances have been made in addressing LLM privacy challenges, with a focus on several prominent research directions. First, model inversion attacks [25, 58, 126, 189] exploit output probabilities to reconstruct hidden prompts or input data, posing serious threats in LLM-as-a-service settings where prompts may encode private instructions. In terms of data leakage risks, studies have revealed two major vulnerabilities: LLMs can retain and reproduce sensitive information even when not overfitted [23, 46, 85, 155], and through membership inference attacks, adversaries can determine whether specific user data was included in the training set using techniques like SPV-MIA with paraphrasing and self-calibrated reference models [32, 138, 207]. To address these challenges, researchers have developed various privacy-preserving approaches: , Vol. 1, No. 1, Article . Publication date: August 2026.

40

Xiong et al.

differential privacy techniques [78, 102, 191] implement DP-SGD to provide formal privacy guarantees, while federated learning solutions [173] explore decentralized model adaptation to prevent centralized data exposure, though both approaches must balance privacy protection with model utility. These privacy vulnerabilities and protective measures represent the current landscape of privacy challenges in LLM-enhanced social media environments. The integration of LLMs in social media applications presents critical privacy challenges that demand immediate attention. The primary challenge lies in preventing unauthorized information inference: Staab et al. [170] reveal how LLMs can breach privacy by deducing sensitive personal information from pseudonymized content through subtle linguistic markers and reconstructing detailed user profiles by connecting seemingly unrelated pieces of information. The cross-platform privacy challenge is particularly pressing, as Treves et al. [180] exposes how tools like RURLMAN can breach user privacy by automatically linking identities across multiple platforms through shared URLs, creating comprehensive digital footprints without user consent. This vulnerability is further amplified by the challenge of preventing synthetic identity abuse, where Ayoobi et al. [12] identifies how these models enable the creation of deceptive profiles on professional networks like LinkedIn for malicious purposes. As Dogan et al. [44] emphasizes, there is a critical need to develop robust safeguards against the systematic linking of online personas with real-world identities, fundamentally challenging traditional privacy protection approaches. Real-time deployment of LLMs introduces additional urgent privacy challenges that require innovative solutions. Brown et al. [22] demonstrates the pressing need to establish clear boundaries between private and public information in ChatGPT’s live platform interactions, while Dou et al. [45] underscores the challenge of preventing unauthorized behavior pattern analysis that could expose users’ personal lives, health, and relationship information. In sensitive topic analysis, Cai et al. [24] identifies the critical challenge of balancing analytical capabilities with privacy protection, particularly in mental health discussions where inadvertent exposure of personal insights poses significant risks. These challenges necessitate the immediate development of enhanced privacy frameworks, as Martin et al. [117] argues for stronger ethical guidelines in governing LLM deployment, and Dickson and McCauley [43] emphasizes the urgent need to resolve the fundamental tension between analytical utility and privacy preservation in these sensitive contexts.

, Vol. 1, No. 1, Article . Publication date: August 2026.

Related documents

• Knowledge Integrity in Large Language Models: A State-of-The-Art Review#1133654
• CertMark: Distortion-Free Multi-Bit Watermarking with Certified Decoding#1118044
• More Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMs#799664
• Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models#924624
• Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models#920392