AI Alignment for Digital Democracy: Fairer Voters Representation against LLM Biases

AI Alignment for Digital Democracy: Fairer Voters Representation against LLM Biases Yi Qi

Srijoni Majumdar

Evangelos Pournaras

University of Leeds Leeds, United Kingdom Y.Qi@leeds.ac.uk

University of Leeds Leeds, United Kingdom S.Majumdar@leeds.ac.uk

University of Leeds Leeds, United Kingdom E.Pournaras@leeds.ac.uk

arXiv:2610.32780v1 [cs.CY] 23 Oct 2026

Abstract AI representation is emerging in digital democracy, assisting in voter emulation and decision-making. Large language models (LLMs) behave differently depending on the model, the prompt context, and the voter information they receive, representing some groups more accurately than others and potentially making the emulation unfair to certain groups. We investigate demographic biases in LLMemulated electorates: how accurately LLMs represent individual votes, whether some voter groups are represented more accurately than others, and whether AI alignment can reduce these disparities without sacrificing overall accuracy. Using three U.S. presidential elections, we compare a closed-source and an open-source LLM across voters’ demographic features and test AI alignment strategies toward fairer voter representation. Removing gender from the voter profile reduces the largest group error of the the models by 22–42%, and removing education reduces it by 64%, whereas removing self-reported race increases the mean group error by over 40%, all without lowering accuracy. Common alignment strategies, such as repeated queries or fairness instructions, do not reduce these errors and can double them.

Keywords Large Language Model, AI representation, Generative AI, Digital democracy voting, Election

1

Figure 1: Voter misrepresentation in AI representation. High individual accuracy (85%) and a small overall error (3 points) can coexist with a large error for one group (Group C, 14 points), reversing its ordering relative to another group.

Introduction

Large language models (LLMs) are increasingly used to emulate human respondents by grounding predictions on demographic, ideological, and contextual profiles [3, 20, 27, 45, 47]. In electoral settings, an LLM can predict a voting choice for each real voter profile and aggregate these predictions into a synthetic electorate. We refer to this process as AI representation of voters. It is used to pilot and complement pre-election polling and forecasting [7, 28], to estimate the preferences of abstaining or hard-to-reach voters who are missing from surveys [11, 27], to build synthetic populations for policy and social-science research [3, 45], and to support deliberation and collective decision-making in digital democracy [27, 35]. In each of these uses, AI representation is valuable only if the resulting electorate reproduces aggregate vote shares and individual voters and demographic groups are represented faithfully and as intended. We distinguish individual fidelity, where an emulated vote matches the corresponding human respondent’s reported vote, from aggregate accuracy, where the simulated electorate reproduces empirical vote-share distributions. These attributes do not align: aggregate agreement can conceal errors that are systematically concentrated within particular voter groups. Prior work on LLM-based opinion

simulation shows that model responses to survey and political questions shift with the demographic profile the model is conditioned on, and often align better with some groups’ opinions than others [3, 5, 19, 36, 39]. Studies that emulate voters specifically report a second problem: the predicted vote totals can miss the real election result, and the predicted differences between groups can be larger than those observed in real voters [17, 31, 42, 44]. What remains open is whether the error in predicting individual votes is itself unevenly distributed across voter groups, and whether this unevenness distorts the group vote shares of the emulated electorate. RQ1. Is the vote-share error of LLM-emulated electorates equal across demographic groups? We refer to such unequal predictive performance as representation disparity: a model may attain reasonable overall accuracy while reconstructing the preferences of some demographic groups less faithfully than others. When errors within a group are directional rather than random, for example when the model predicts too many Democratic votes for one group, they shift 1

that group’s emulated vote share away from its reported share, even when overall accuracy appears acceptable (Figure 1). We call such a group misrepresented. This matters because many uses of emulated electorates rely on group-level estimates, such as a group’s vote share in a poll, and because group errors that do not cancel out shift the overall vote share and can change the emulated winner in close elections. A natural response is repeated sampling, combining multiple generations by majority voting or vote-share averaging [27, 44]. This reduces random variation but cannot correct errors the model makes consistently: a mistake repeated in every run survives the majority vote. Correcting such errors requires changing how the model represents each voter, which motivates our second question. RQ2. Does steering individual emulated voters before aggregation reduce group-level misrepresentation without lowering individual accuracy? To answer RQ2, we compare three levels of control: the base model, a prompt-based instruction, and a decomposed prompt that separates vote-relevant reasoning from group representation; for the open-source model, we also add activation-level control. We evaluate them on every eligible respondent of ANES 2012, 2016, and 2020. Our contributions map onto the two research questions:

human samples [3]. Their results establish demographic prompting as a basis for LLM-based social simulation, but focus primarily on correspondence between synthetic and human response distributions rather than disparities in predictive error across voter groups [23]. Subsequent work has identified limitations of this approach [38, 46]. Bisbee et al. condition ChatGPT on profiles derived from ANES 2016 and 2020 respondents and find that synthetic responses can reproduce average public-opinion patterns while exhibiting substantially reduced variation and altered relationships between respondent characteristics and outcomes. They further show subgroup differences by race and partisanship, illustrating that aggregate agreement can conceal distortions within the synthetic sample [6]. We build on this observation by measuring such distortions in vote choice, comparing each group’s emulated vote share with its reported vote share, and testing whether alignment interventions reduce them. More recent work focuses directly on electoral behaviour [4, 8, 26]. Von der Heyde et al. use individual-level voter information to predict voting behaviour in the 2024 European Parliament elections and find that prediction accuracy varies substantially across national and linguistic contexts [42]. Yu et al. develop a large-scale voter-simulation framework using demographic, ideological, and temporal information and evaluate multiple LLMs across U.S. election settings. Their analysis shows that simulated voting patterns can overstate demographic differences involving gender, ethnicity, age, and education, even when aggregate electoral accuracy improves [44]. Majumdar et al. show that fair collective choice can be resilient to LLM biases and inconsistencies: under several aggregation methods and across proprietary and open-source models, AI-generated choices agree with human choices more consistently at the collective level than at the individual level [27]. However, that work focuses on the robustness of collective-choice mechanisms rather than on whether demographic groups experience systematically different representation errors before aggregation [22, 30]. Our findings complement this result, which aggregation absorbs random individual inconsistencies, but errors that are systematic within a demographic group persist through aggregation and distort that group’s emulated vote share.

• Measuring misrepresentation (RQ1). We define misrepresentation as the gap between a group’s emulated and reported vote shares and show, across three elections and two models, that it reaches 10 points or more while individual accuracy stays near 85%. • Which voter information is fair to use (RQ1). We show that removing gender from the voter profile reduces the largest gender-group error of the voter choices emulated by Claude by 42% and of the voter choices emulated by Llama by 22%, and removing education reduces the largest educationgroup error of the voter choices emulated by Claude by 64% (p < 0.01), whereas removing self-reported race increases the mean group error by over 40% (p < 0.05), all without lowering accuracy. • Testing alignment controls (RQ2). We show that repeated queries, aggregation, fairness instructions, and activationlevel control do not reduce misrepresentation, and that a fairness instruction can double the error for a group.

2

2.2

Background

In this section, we reviews how large language models have been used to emulate survey respondents and voters, and methods for measuring and reducing unfairness in LLM-based decision making. We combine the two by measuring fairness within voter emulation and testing whether such methods reduce it.

2.1

Fairness Analysis in LLM Decision Making

A parallel literature examines fairness in LLM-supported decision making [12], and fairness in LLM-based simulation [1] and decision support. Hall et al. propose Fairness Reward Models, which improve the fairness of LLM decisions in high-stakes tasks such as recidivism prediction and hiring [12]. The LLM generates several chains of reasoning for the same case, a separate model rates each chain for bias, and the final decision gives more weight to the less biased chains. Their evaluation explicitly measures disparities using equalized opportunity and equalized odds gaps [14] across protected attributes including race and gender, showing that fairness-oriented inference-time selection can reduce group disparities while maintaining predictive accuracy. Related work has studied both the measurement of prompt-induced political bias in LLMs [10, 33] and prompt-level and representation-level interventions for controlling multiple model objectives [24, 25, 41]. These methods define fairness as similar treatment across groups, which

Generative AI Representation of Voters

Early work on LLM-based human emulation established the possibility of constructing synthetic samples of real survey respondents by prompting language models on real respondent profiles [2, 9, 16, 32]. Argyle et al. introduce “silicon samples” by prompting GPT-3 on socio-demographic backstories drawn from human survey respondents and use the term algorithmic fidelity to show the extent to which model-generated responses reproduce patterns observed in 2

3.1

suits decisions such as hiring but not voter emulation, where the reported vote shares differ between groups. The fair model must reproduce each group’s actual vote share, and an intervention that equalises predictions across groups can increase misrepresentation rather than reduce it [13, 21, 40]. In contrast, our setting combines these two strands, we evaluate fairness within voter emulation by comparing representation error across demographic groups, and then test whether interventions applied before aggregation and at representative process reduce these disparities in the resulting emulated electorate. Table 1: Comparison of closely related work across six dimensions of fairness evaluation in AI voter representation. Per-group comparison

Fairness metric

Race/ ethnicity

Multiple elections

Open & closed LLMs

Voter emulation

△ △

✓ ✓

✓

✗ ✗ ✗

✗ ✗ ✗ ✓ ✓

△

✓ ✓ ✓ ✓ ✓ ✗

✓

✓

Argyle [3] Bisbee [6] von der Heyde [42] Yu [44] Majumdar [27] Hall [12]

△ △

△ △

✓

✓

✓

✓

✗ ✓ ✗ ✓ ✓ –

Ours

✓

✓

✓

✓

△ △

𝑁

Accind =

Notes: ✓ = explicit coverage, △ = partial or indirect coverage, ✗ = not addressed, – = not applicable.

1 ∑︁ 1[𝑦ˆ 𝑗 = 𝑦 𝑗 ], 𝑁 𝑗=1

(1)

where 1[·] is the indicator function.

3.2

To position our work relative to these two strands of literature, Table 1 compares closely related studies along six dimensions relevant to our setting whether they: (i) compare predictive performance across groups, (ii) employ a formal fairness metric, (iii) evaluate race or ethnicity, (iv) across multiple election cycles, (v) compare closed-source model and open-source models, and (vi) study voter emulation directly. Prior work addresses several of these dimensions individually, but none of the studies considered here combines all six within a single evaluation framework. In this table, A partial mark (△) indicates that a study covers the dimension indirectly, for example by reporting subgroup differences without a formal fairness metric. A cross (✗) means the dimension applies to the study but is not addressed, whereas a dash (–) means it does not apply, such as Hall et al. [12] study fairness in general decision tasks rather than elections, so election cycles are not part of their setting. Prior work covers several of these dimensions individually, but none combines all six, which is needed to test whether demographic misrepresentation in voter emulation is consistent across models and elections and whether fairness interventions reduce it.

3

Voter Profiles and Voter Emulation

Let D = {(𝑥 𝑗 , 𝑎 𝑗 , 𝑦 𝑗 )}𝑁𝑗=1 denote a set of 𝑁 voter profiles, where 𝑥 𝑗 contains individual-level profile information such as ideology, age, education, and political interest, 𝑎 𝑗 ∈ R is the respondent’s group attribute (here, self-reported race or ethnicity or other grouping criteria can be used), and 𝑦 𝑗 ∈ Y is the respondent’s reported vote. Each vote takes one of two values, Y = {Democrat, Republican}; respondents who reported a third-party vote or did not vote are excluded. Given a voter profile (𝑥 𝑗 , 𝑎 𝑗 ), the LLM generates a voting choice 𝑦ˆ 𝑗 ∈ Y for that voter; we call this process AI choice generation. The information included in the profile, such as race, gender, or education, is the input that shapes each generated choice. Individual fidelity refers to agreement between this emulated vote and the vote reported by the same respondent; the prediction for respondent 𝑗 is correct when 𝑦ˆ 𝑗 = 𝑦 𝑗 . We measure individual fidelity by individual accuracy Accind , the share of respondents whose emulated vote matches their reported vote:

Demographic Grouping

For group-level evaluation, respondents are partitioned by the attribute 𝑎 𝑗 . Let R denote the set of groups and 𝑁𝑟 = |{ 𝑗 : 𝑎 𝑗 = 𝑟 }| the number of respondents in group 𝑟 ∈ R. This partition lets us test whether emulation errors are distributed unevenly across groups, because each prediction is checked against the respondent’s reported vote 𝑦 𝑗 , and each group’s emulated vote share is compared with its reported vote share. We group respondents by the race or ethnicity they self-identify with for three reasons: survey data show that vote choice differs between voters who self-identify with different racial and ethnic groups in U.S. elections [29], LLMs show well-documented racial stereotyping, and fair representation of voters who self-identify with a racial or ethnic group is a central concern in electoral law and democratic legitimacy. The framework itself applies to any grouping, for example, other demographic attributes such as gender, age, education, religion, or region can reveal stereotyping and under-representation of minority opinions and combinations of attributes can reveal intersectional bias.

3.3

Problem Formulation

LLM Repeated Vote Generation

LLMs generate text by sampling each word from a probability distribution rather than always choosing the most likely one. This stochastic decoding, controlled by parameters such as temperature [15], means that the same voter profile can receive different predicted votes in different runs. We therefore query the model 𝐾 times per profile, obtaining predictions 𝑦ˆ 𝑗,1, . . . , 𝑦ˆ 𝑗,𝐾 , and summarize them by the share of runs that predict a Democratic vote:

We emulate an electorate in four steps. (i) Each survey respondent is described by a voter profile and their reported vote. (ii) Respondents are grouped by a demographic attribute (e.g. race). (iii) The LLM predicts one vote for each profile. Because predictions vary between runs, the model can be queried 𝐾 times per voter, and all 𝐾 predictions are then combined into a single emulated vote by majority, which keeps one vote per voter because each voter casts one ballot. (iv) The emulated votes of all voters in a group are aggregated into a group vote results and compared with the group’s reported vote results.

𝐾

𝑝ˆ 𝑗 =

1 ∑︁ 1[𝑦ˆ 𝑗,𝑘 = Democrat]. 𝐾 𝑘=1

3

(2)

4

The emulated vote is the majority prediction, 𝑦ˆ 𝑗 = Democrat if 𝑝ˆ 𝑗 > 0.5 and Republican otherwise, following self-consistency decoding [43]; we use odd 𝐾 to avoid ties. We measure how consistent the model is for respondent 𝑗 by 𝑐 𝑗 = max(𝑝ˆ 𝑗 , 1 − 𝑝ˆ 𝑗 ), which equals 1 when all runs agree and 0.5 when they are evenly split. Combining runs is done within each respondent and is distinct from aggregation, which combines different respondents into group voting.

3.4

Group Misrepresentation Metrics 4.1

After obtaining one emulated vote 𝑦ˆ 𝑗 per respondent, we aggregate votes within each group 𝑟 ∈ R. Because votes are binary, a group’s vote is fully described by its Share of emulated Democratic choices minus reported share; the Republican share is its complement and has the same error. We compare the share of emulated Democratic choices 𝜃ˆ𝑟 with the reported share of Democratic votes 𝜃 𝑟 of the same respondents: 1 ∑︁ 𝜃ˆ𝑟 = 1[𝑦ˆ 𝑗 = Dem], 𝑁𝑟 𝑗:𝑎 =𝑟 𝑗

𝜃𝑟 =

𝜀𝑟 = |𝜃ˆ𝑟 − 𝜃 𝑟 |,

1 ∑︁ 1[𝑦 𝑗 = Dem]. (3) 𝑁𝑟 𝑗:𝑎 =𝑟 𝑗

4.2 𝜀 max = max 𝜀𝑟 .

We use the American National Election Studies (ANES) 2012, 2016, and 2020 and include every respondent who reported a two-party presidential vote and has complete profile data (𝑛 = 3,793, 1,978, and 5,141; Table 2). Profiles state race and ethnicity, gender, age, education, ideology, and political attention. We group voters by race, gender, age, and education, and exclude groups with fewer than 30 respondents.

𝑟 ∈R

Table 2: Datasets used in the evaluation Dataset

Election

Vote task

𝑁

Role

ANES 2012 ANES 2016 ANES 2020

Obama vs. Romney Clinton vs. Trump Biden vs. Trump

Presidential vote Presidential vote Presidential vote

3,793 1,978 5,141

Temporal validation Temporal validation Main evaluation

Note. ANES data: https://electionstudies.org/data-center/. N counts respondents who reported a two-party presidential vote and have complete profile data.

From Individual to Group Errors

Improving individual predictions need not improve group vote shares, because the emulated vote is a majority over runs. A change in a voter’s vote distribution 𝑝ˆ 𝑗 has no effect unless it crosses 0.5, and then changes the group share by a full 1/𝑁𝑟 . Aggregating Í probabilities instead, 𝜃˜𝑟 = 𝑁1𝑟 𝑗:𝑎 𝑗 =𝑟 𝑝ˆ 𝑗 , removes this discontinuity.

4.3

Models and Levels of Control

Model. Claude Sonnet 4.6 is available only through an API and can be controlled only through its prompt; Llama-3.1-8B-Instruct is open-weight, so its internal activations can also be modified [34]. Both sample at temperature 𝜏 = 0.7 [18, 37].1

Lemma 1. Let 𝜃˜𝑟(1) , . . . , 𝜃˜𝑟(𝑀 ) be group shares obtained by probability aggregation from 𝑀 predictors, each with |𝜃˜𝑟(𝑚) − 𝜃 𝑟 | ≤ 𝜀. Then Í any convex combination 𝑚 𝑤𝑚 𝜃˜𝑟(𝑚) also has error at most 𝜀.

𝜀.

Datasets and Voter Profiles

(4)

We use the absolute rather than relative difference because a gap of a few points has the same electoral weight regardless of a group’s baseline share. We also report the mean group error 𝜀¯ and compare both with the population-level error 𝜀 pop = |𝜃ˆ − 𝜃 | computed over all respondents. Because all errors are measured in vote-share points against the same respondents’ reported votes, they are comparable across groups within a dataset. Small groups yield noisy estimates, so we report 𝑁𝑟 and a 95% confidence interval for every group’s signed error 𝜃ˆ𝑟 − 𝜃 𝑟 , and call a group misrepresented when this interval excludes zero. Unlike individual accuracy (Eq. (1)), these metrics capture whether errors cancel out or accumulate within a group.

Proof. |

Experimental Design

We use one closed-source and one open-source model because the two allow different kinds of control. Claude Sonnet 4.6 is a state-ofthe-art model available only through an API, so it can be controlled only through its prompt. Llama-3.1-8B-Instruct is a widely used open-weight model whose internal activations can be read and modified, which allows control inside the model [34]; its size also allows us to run it locally. Using both lets us test whether our findings hold for prompt-based and internal control. Comparing levels of control with the number of runs fixed shows the effect of control alone, and comparing numbers of runs at a fixed level shows the effect of repeated runs alone; for each setting we report both individual accuracy and group misrepresentation error.

The misrepresentation error of group 𝑟 is the gap between its emulated and reported shares, and our primary fairness metric is the largest such gap across groups:

3.5

Experimental Setup

We emulate the electorates of three U.S. presidential elections with two LLMs, one closed-source and one open-source. We evaluate them at three increasing levels of control: (i) the base LLM, (ii) prompt-based control, and (iii) decomposed control, which separates vote-relevant reasoning from group representation; for the open-source model, we add activation-level control. The base level is run several times per voter, with votes combined by majority.

Levels of Control. We evaluate each model at three increasing levels of control (prompts in the supplementary material). (i) Base: the model receives the voter profile and is asked for a vote. (ii) Prompt-based control: a single instruction is added, asking the model to consider individual electoral factors such as ideology and political

˜ (𝑚) − 𝜃 𝑟 | = | Í𝑚 𝑤𝑚 (𝜃˜𝑟(𝑚) − 𝜃 𝑟 )| ≤ Í𝑚 𝑤𝑚 𝜀 = 𝑚 𝑤𝑚 𝜃 𝑟

Í

□ 1 Claude: at most 60 output tokens. Llama: top-𝑝 = 1, at most 40 output tokens. An

answer is read as a vote for the first candidate it names; answers naming no candidate (none for Claude, under 1% for Llama) are excluded. Prompts and further details are in the supplementary material.

No such guarantee holds for majority votes, since the mode is not linear in the predictors. 4

interest, and to “avoid systematically under-representing the voting preferences of the voter’s racial or ethnic community” (verbatim prompt). This instruction assumes that voters who self-identify with the same racial or ethnic group share a voting preference; whether such an instruction helps is part of what we test. (iii) Decomposed control: the same two objectives are placed in separate blocks, one on individual-level signals and one on community representation, which is to be treated as an independent consideration that does not override the profile, followed by the final prediction. Both controls target race, the attribute of RQ1. For Llama, we add (iv) activationlevel control by contrastive activation addition (CAA) [34]: we take the hidden state of the last prompt token at layer ℓ = 16 for 100 voters who self-identify as White and 100 voters who self-identify with another racial or ethnic group (sampled at random from ANES 2020), and use the dominant direction separating the two sets, and add it with coefficient 𝛼 = 1 during generation. We chose 𝛼 as the largest value that keeps the overall Democratic share within 10 points of the uncontrolled model; larger values collapse almost all predictions to one candidate. The base level is run once (𝐾 = 1) and five times (𝐾 = 5) per voter; the controlled levels are run once and compared with the first base run on the same voters.

4.4

Worst-group error (race)

0.3

ANES 2020

0.2 0.1

Individual accuracy

0.8 0.7

Base

Prompt Decomp. Base

Claude Sonnet 4.6, K=1 Claude Sonnet 4.6, K=5

Prompt Decomp. Base

Prompt Decomp.

Llama-3.1-8B, K=1 Llama-3.1-8B, K=5

Figure 2: Worst-group error over voters who self-identify with a racial or ethnic group (top) and individual accuracy (bottom) at each level of control, with one run per voter (solid) or five runs at the base level (hollow). Accuracy is stable across levels, while race-targeted control does not reduce the worst-group error: for Claude, prompt-based control raises it in every election.

Evaluation Metrics

Reported

Claude Sonnet 4.6

Llama-3.1-8B

White (n=4,045) Black (n=325) Hispanic (n=352) Asian/NHPI (n=168)

Group Misrepresentation (RQ1)

Native Am. (n=89)

This section answers RQ1 using the base level, where no control is applied, with one run (𝐾 = 1) and five runs per voter (𝐾 = 5). Since the model is not steered at either setting, any difference between them comes from repeated runs alone.

5.1

ANES 2016

0.0 0.9

For each model, dataset, level of control, and number of runs, we report individual accuracy Accind (Eq. (1)), to check whether fairness gains come at the cost of prediction quality; the group errors 𝜀𝑟 and the worst-group error 𝜀 max (Eq. (4)), our primary fairness metric, together with the mean group error 𝜀¯ and the population-level error 𝜀 pop . We call a group misrepresented when the 95% confidence interval of its signed error excludes zero.

5

ANES 2012

Other (n=162) 0.4

0.5

0.6 0.7 0.8 0.9 Democratic vote share

1.0

Misrepresentation Across Models Figure 3: Signed misrepresentation error (share of emulated Democratic choices minus reported share of Democratic votes) for the voter choices emulated by Claude and by Llama, by self-identified racial group, ANES 2020 (base level, K = 5; bars: 95% intervals). For both models, the share of emulated Democratic choices is furthest from the reported share for voters who self-identify as Native American (about 14 points higher) and within 1 point for voters who self-identify as Black.

In ANES 2020, both models reach high accuracy (Claude 0.848, Llama 0.858) and small population-level errors (3.1 and 1.4 points), yet misrepresent particular voters who self-identify with a racial or ethnic group (Fig. 1). For voters who self-identify as Native American, the voter choices emulated by Claude and the voter choices emulated by Llama place the Democratic share 13.5 and 13.6 points above the reported share, respectively (95% intervals excluding zero), while for voters who self-identify as Black, the error of both models stays within 1 point. The models otherwise differ: The voter choices emulated by Claude places the Democratic share of voters who self-identify as White 4.1 points below their reported share, whereas the voter choices emulated by Llama places the Democratic share of voters who self-identify as Asian/NHPI 7.8 points above their reported share. Figure 3 compares each group’s share of emulated Democratic choices with its reported share. The largest gap for both models is for voters who self-identify as Native American (n = 89): the voter choices emulated by Claude places their Democratic share 13.5

points above the reported share, and the voter choices emulated by Llama 13.6 points above; as the smallest group, this is also the least precise estimate. For voters who self-identify as Black or Hispanic, the groups most often examined, the share of emulated Democratic choices of both models stays within 5 points of the reported share. The two the models then diverge: the voter choices emulated by Claude places the Democratic share of voters who self-identify 5

election (paired bootstrap, 𝑝 ≥ 0.51). The errors are therefore systematic: the model makes the same mistake for the same voter on every run, and repeated sampling preserves it.

as White below their reported share, whereas the voter choices emulated by Llama places the Democratic share of voters who selfidentify as Asian/NHPI above their reported share. Because the largest error falls on a group that a comparison limited to voters who self-identify as Black or Hispanic would miss, we evaluate the worst-group error over all groups.

5.2

Other demographic attributes. Misrepresentation is not specific to race; it is largest and most consistent for gender and education. In all three elections, the share of Democratic choices emulated by Claude is lower than the reported share for voters who self-identify as male (by 8.5, 9.4, and 10.6 percentage points) and higher than the reported share for voters who self-identify as female (by 3.3, 8.6, and 3.5 points), with every 95% interval excluding zero. As a result, the emulated gender gap is about three times the reported one. For voters with at most a high school education, the share of emulated Democratic choices is likewise lower than the reported share in every election (by 4.7, 7.6, and 10.8 points), widening the reported education gap. Llama errs in the same direction: the share of Democratic choices emulated by Llama is lower than the reported share for voters who self-identify as male (by 12.0, 13.7, and 2.7 points) and for voters with at most a high school education (by 11.0, 16.2, and 6.3 points) in every election, and its emulated gender gap is 2.3–3 times the reported one. Errors for racial groups are less stable. For Claude, no racial group is significantly misrepresented in 2016; for Llama, the share of emulated Democratic choices is below the reported share for voters in most self-identified racial groups in 2012 and 2016, in line with its overall shift towards the Republican candidate in those years. The largest racial error, for voters who self-identify as Native American in 2020, appears in both models (share of emulated Democratic choices 13.5 and 13.6 points higher than the reported share) but rests on the smallest group (𝑛 = 89). The most consistent errors thus follow one pattern, which the models push gender and education groups further apart than they really are, exaggerating real differences between groups [44]. These errors remain hidden by aggregate metrics: Claude’s population-level error never exceeds 3.1 points, while its errors for large groups reach 10.8 points.

Robustness Claude Sonnet 4.6

ANES 2012

Llama-3.1-8B

filled = gap significant (95% CI excludes 0)

ANES 2016

ANES 2020

White

Race

Black Hispanic Asian/NHPI Native Am. n < 30

n < 30

Other

Gender

Man Woman

Age

18-34 35-49 50-64 65+

Education

HS or less Some college Bachelor Postgraduate −.30

−.15

0

+.15

+.30 −.30

−.15

0

+.15

+.30 −.30

−.15

0

+.15

+.30

Emulated minus reported Democratic share (← too Republican · too Democratic →)

Figure 4: Signed misrepresentation error (Share of emulated Democratic choices minus reported share) by group at the base level (𝐾 = 5), with 95% intervals; filled markers exclude zero. For voters who self-identify as male and voters with at most a high school education, the share of Democratic choices emulated by both models is below the reported share in every election, while the errors for racial groups vary across elections.

Finer subgroups. Splitting voters who self-identify with a racial or ethnic group by gender raises Claude’s worst-group error to 0.139, 0.175, and 0.143. Because smaller groups yield noisier estimates, part of this increase is expected even without any additional bias. To separate the two, we shuffle gender labels within each voters who self-identify with a racial or ethnic group, which preserves group sizes and race-level errors, and recompute the worst-group error 1,000 times. The observed error exceeds this baseline in 2012 (𝑝 = 0.010) and 2016 (𝑝 = 0.025) but not in 2020 (𝑝 = 1.00); for Llama, only in 2016 (𝑝 = 0.030; 𝑝 = 0.077 in 2012 and 0.72 in 2020).

We now test whether this is a stable property of the models or an artifact of three measurement choices: (1) a single run per voter, (2) race as the only grouping, and (3) a single election. We emulate every eligible respondent in ANES 2012, 2016, and 2020 (𝑛 = 3,793, 1,978, and 5,141) at the base level with 𝐾 = 5 runs per voter, and report each group’s signed error (emulated minus reported share of Democratic votes) with a 95% confidence interval, for all groups with at least 30 respondents (Fig. 4)2 .

5.3

Source of Misrepresentation

We use the profile the model sees for three attributes 𝐴 (race, gender, and education) on ANES 2020, using the base-level prompt: (i) attribute only, a profile stating only 𝐴, repeated 20 times per group with alternating candidate order; (ii) the full profile; and (iii) the full profile without 𝐴.3 We summarize each prompt by its gap amplification, the slope of a weighted regression of emulated on

Repeated runs. If misrepresentation were sampling noise, majority voting over repeated runs would reduce it. It does not. Claude gives the same vote in all five runs for 99.3–99.6% of respondents, and Llama for 84–90%. The worst-group error over voters who self-identify with a racial or ethnic group is statistically indistinguishable between one and five runs for both models in every 2 An answer is read as a vote for the first candidate it names, ignoring neutral pairings

3 Full and without-𝐴 prompts use the same respondents and candidate order. Llama

such as “between Biden and Trump”. Answers naming no candidate (none for Claude, under 1% for Llama) are excluded.

uses majority voting over 𝐾 = 5 runs; Claude uses a single run, since its runs agree for 99.6% of respondents.

6

attribute only

full profile

profile without attribute

Claude Sonnet 4.6

attributes, which do not capture the differences in reported votes between voters who self-identify with different racial or ethnic groups. Including gender increases the group error: without it, the largest group error falls by 42% for Claude and 22% for Llama (both 𝑝 < 0.001), and Claude’s emulated gender gap shrinks from 3.6 to 1.3 times the reported one. Including education increases the group error for Claude in the same way: without it, the largest group error falls by 64% (𝑝 = 0.002), and the share of emulated Democratic choices for voters without a college degree is no longer 0.4–10.9 points lower than the reported share; for Llama, education makes no significant difference. Accuracy never decreases significantly. Which personal information the model receives therefore decides how fairly groups are represented, at no cost to accuracy.

Llama-3.1-8B

Race Gender Education 0.5 1

2

5 10 20

0.5 1

2

5 10 20

Gap amplification (emulated / reported group differences, log)

Figure 5: Gap amplification on ANES 2020 (1 = real group differences reproduced; log scale; 95% intervals). Given only the attribute, both models exaggerate group differences severalfold; the full profile still exaggerates gender and education gaps.

Answering RQ1 The vote-share error of emulated electorates is not equal across voters who self-identify with a racial or ethnic group. In ANES 2020, the share of Democratic choices emulated by both models is about 14 points higher than the reported share for voters who self-identify as Native American, while for voters who self-identify as Black it stays within 1 point. Which voters who self-identify with a racial or ethnic group are misrepresented, however, varies across elections and models. The unequal errors also extend beyond race: in every election, the share of Democratic choices emulated by both models is lower than the reported share for voters who selfidentify as male and for voters with at most a high school education, exaggerating real differences between groups. They arise because the models weight some demographic labels too heavily: removing gender or education from the profile reduces the errors, whereas removing self-reported race increases them. Giving the model a voter’s gender makes the largest gender-group error 29% (Llama) to 73% (Claude) larger, and giving it education makes Claude’s largest education-group error 2.8 times larger (𝑝 ≤ 0.002), without improving accuracy.

Table 3: Group error when one attribute is left out of the profile (ANES 2020). Negative change = fairer. Worst-group error

Mean group error

Model

Left out

Full → without

Change

Full → without

Change

Acc. change

Claude

Race Gender Education

.124 → .185 .107 → .062 .109 → .039

+49% −42% −64%

.043 → .061 .071 → .054 .048 → .021

+42% −25% −57%

−0.5 −0.3 +0.2

Llama

Race Gender Education

.136 → .124 .049 → .038 .063 → .072

−9% −22% +15%

.049 → .069 .038 → .023 .035 → .034

+41% −39% −4%

0.0 +0.5 +1.2

reported share of Democratic votes across the groups of 𝐴 (weights: group size). A value of 1 means the emulation reproduces the real differences between groups, values above 1 mean it exaggerates them, and values below 1 mean it flattens them (Fig. 5). Given only the attribute, Claude returns the same answer in all 20 repetitions for every group, assigning all voters who selfidentify with a given group entirely to one candidate: with no other information, the model falls back on a deterministic group-to-party stereotype. The resulting gaps are 3.1 (race), 18.1 (gender), and 4.8 (education) times the reported ones. Llama follows the same direction less deterministically (2.6, 11.8, and 2.0), and declines to answer half of the gender-only prompts, mostly the prompt stating a male voter (14 of 20). The full profile dilutes the stereotype but does not remove it. With the full profile, amplification falls close to 1 for race (Claude: 1.10, 95% CI [1.00, 1.19]; Llama: 1.00 [0.90, 1.08]) but remains above 1 for gender (3.58 [2.55, 6.36]; 2.34 [1.75, 3.87]) and, for Claude, education (1.34 [1.21, 1.48]; Llama: 1.11 [0.99, 1.23]). Leaving one attribute out of the profile shows which information helps and which harms (Table 3; paired bootstrap tests in supplementary material). Removing self-reported race increases the group error: without it, the mean group error rises by about 40% for both Claude and Llama (p = 0.008 and 0.04), and for voters who selfidentify as Black, the share of Democratic choices emulated by Claude moves from within 1 point of the reported share to 18.5 points lower. Without race, the models lean on the remaining profile

6

Mitigating Misrepresentation (RQ2)

Misrepresentation is systematic, persists across repeated runs, and is strongest for gender and education. We test whether promptbased and decomposed control reduce it without lowering individual accuracy, whether their effect extends beyond race, which both target, and why gains for individual voters do not always carry over to the electorate.

6.1

Effect of Aggregation

Before controlling the representation of each voter, we test whether the aggregation step alone can reduce misrepresentation. We compare majority voting over 𝐾 = 5 runs (𝜃ˆ𝑟 ) with probability aggregation (𝜃˜𝑟 ), which averages each voter’s share of Democratic votes 𝑝ˆ 𝑗 , at the base level on ANES 2012, 2016, and 2020. By Lemma 1, probability aggregation is never worse than the worst single run; majority voting has no such guarantee. The guarantee holds in practice, but brings little benefit. For Claude, whose runs agree for over 99% of voters, the two rules differ by at most 1.5 points for any group. For Llama, whose runs disagree for 10–16% of voters, the worst-group error changes by at most 1.4 points across race, gender, age, and education, without 7

Table 4: Accuracy, population-level error, and worst-group error 𝜀 max by attribute at each level of control (𝐾 = 1, same voters). ∗ : differs from base (𝑝 < 0.05, paired bootstrap).

Base

Year

Level

Acc.

Pop.

Race

Gender

Educ.

Claude

2012

Base Prompt Decomposed Base Prompt Decomposed

.822 .825 .815∗ .832 .832 .832

.026 .043∗ .050∗ .001 .019 .002

.051 .140∗ .084 .071 .167 .095

.085 .083 .095∗ .094 .108 .084

.048 .091∗ .078∗ .076 .043 .049∗

Base Prompt Decomposed

.847 .850 .848

.031 .026∗ .047∗

.124 .258∗ .135

.107 .095∗ .112

.109 .086∗ .101

Base Prompt Decomposed Base Prompt Decomposed

.786 .799∗ .757∗ .818 .829∗ .824

.083 .054∗ .019∗ .047 .034∗ .004∗

.106 .100 .079 .142 .113 .085

.127 .094∗ .054∗ .140 .129 .081∗

.122 .100∗ .069∗ .171 .153 .117∗

Base Prompt Decomposed

.851 .857 .850

.014 .005 .016

.169 .114 .146

.052 .049 .057

.060 .057 .053

2016

2020

Llama

2012

2016

2020

−.1

Activation (CAA)

Llama-3.1-8B

0

+.1 +.2 +.3

−.1

0

+.1 +.2 +.3

Emulated minus reported Democratic share (ANES 2020)

Figure 6: Signed misrepresentation error by group on ANES 2020 at each level of control (𝐾 = 1, same voters). Promptbased control doubles Claude’s error for voters who selfidentify as Native American; activation-level control shifts every Llama group toward the Democratic candidate.

level has the same direction but a smaller effect (0.084 and 0.135; not significant). Neither level reduces the gender errors: for voters who self-identify as male, the share of emulated Democratic choices remains 8–11 points lower than the reported share at every level and in every election. In 2012 and 2016, where the overall share of Democratic choices emulated by Llama is below the reported share, the decomposed level reduces the population-level error from 8.3 to 1.9 and from 4.7 to 0.4 points, and with it the worst-group error for gender (0.127 to 0.054; 0.140 to 0.081) and education (0.122 to 0.069; 0.171 to 0.117). The improvement lifts all groups together rather than correcting specific ones: for voters who self-identify as male and voters with at most a high school education, the share of emulated Democratic choices remains 5–12 points lower than the reported share. In 2020, where the base emulation has no overall shift, neither level changes any worst-group error significantly.

a consistent direction: probability aggregation is slightly better in some cases and slightly worse in others (e.g., gender in 2020: 4.9 vs. 5.4 points). The exaggerated gaps also remain, Llama’s emulated gender gap is 2.6, 2.9, and 2.4 times the reported one under probability aggregation, against 2.6, 2.9, and 2.3 under majority voting. Majority voting exceeds the error of the worst single run for only 1 of 46 groups, and the error of the average run for 16 of 46. When a model misjudges a voter, it does so in most of its runs, so 𝑝ˆ 𝑗 is close to 0 or 1 and both rules return the same group shares. Reducing misrepresentation requires changing how each voter is represented before aggregation.

6.2

Decomposed

White Black Hispanic Asian/NHPI Native Am. Other Man Woman HS or less Some college Bachelor Postgraduate

𝜀 max Model

Prompt-based

Claude Sonnet 4.6

Effect of Control

We emulate every voter once more at the prompt-based and decomposed levels and compare each with the base level on the same voters, using one run per voter at every level (Table 4, Fig. 2) and Fig. 6 shows the resulting error of each group in 2020. Accuracy changes by at most 1.3 points at either level, except for Llama’s decomposed level in 2012 (−2.9). Any change in misrepresentation therefore comes at almost no cost to individual accuracy, and accuracy again fails to reveal it. For Claude, the prompt-based instruction moves the shares of emulated Democratic choices for voters in different self-identified racial groups further apart than the reported shares. Where the share of emulated Democratic choices was already higher than the reported share, the gap widens: for voters who self-identify as Native American, from 12.4 to 25.8 points higher in 2020, and for voters who self-identify as Asian/NHPI, from 4.0 to 14.0 points higher in 2012. The worst racial error roughly doubles (0.051 to 0.140 in 2012, 0.124 to 0.258 in 2020, both 𝑝 < 0.05). The decomposed

6.3

Gaps under Control

We next ask whether control reduces the exaggeration of real differences. For gender, control reduces amplification for Claude (2020: 3.58 to 3.33 prompt-based and 3.20 decomposed; 𝑝 < 0.05 in all three elections at the decomposed level), mainly by moving the emulated share for voters who self-identify as female closer to their reported share, but the emulated gender gap remains 2.6–3.6 times the reported one at every level and election. For Llama, gender amplification falls significantly only in 2016 (2.96 to 2.56) and remains at least 2.4 everywhere. For race, the prompt-based level increases Claude’s amplification in every election (2020: 1.10 to 1.27; 𝑝 < 0.05) and the decomposed level in 2016 and 2020 (2020: 1.22), so racial differences become more exaggerated than without 8

Acknowledgments

control. Education gaps change little at either level. No level of control brings the exaggerated gaps close to the reported ones.

6.4

This work is supported by a UKRI Future Leaders Fellowship (UKRI3495): “Smart City Commons of Trustworthy Distributed Intelligence - ARTIO 2.0”

Prompt vs. Activation Control

We compare prompt-level and activation-level control on the same model and voters (Llama, ANES 2020, 𝐾 = 1). The decomposed level and activation-level control agree on 91.3% of voters (𝜅 = 0.82). This agreement is lower than each method’s agreement with the base level (93.3% and 92.7%), so it reflects the voters that neither method changes rather than a shared effect. The two methods change the vote of 338 and 367 voters, respectively, but only 133 voters under both. The direction of the changes also differs. The decomposed level moves voters in both directions (175 toward the Democratic and 163 toward the Republican candidate), leaving the overall share unchanged. Activation control moves voters almost only toward the Democratic candidate (315 vs. 52), which raises the population-level error from 1.4 to 6.6 points and raises the share of emulated Democratic choices for every group (Fig. 6). It worsens the worst-group error for gender (0.052 to 0.101; 𝑝 < 0.001) and education (0.060 to 0.098; 𝑝 = 0.08) without reducing it for race (0.169 to 0.182), while accuracy is unchanged (−0.6 points). Its lower education amplification (1.10 to 0.85) reflects this uniform shift, not a correction of particular groups. Prompt-level and activation-level control are therefore distinct interventions, and results obtained with one do not transfer to the other.

Answering RQ2 Not with the interventions we tested. Neither aggregation nor control at the level of individual voters reliably reduces misrepresentation. Probability aggregation changes group errors by at most 1.5 points. Race-targeted prompts leave the largest errors, for gender and education, in place, and for Claude they exaggerate racial differences further. Where control helps, as for Llama in 2012 and 2016, it does so by removing an overall partisan shift rather than by correcting the groups that are misrepresented. Activation-level control shifts the whole electorate and worsens misrepresentation. Aligning emulated electorates therefore requires controls that adjust how strongly each demographic attribute shapes a voter’s representation, and evaluation at the level of groups, since individual accuracy does not reveal whether a control helps or harms.

7

Conclusion

AI representation of voters can be accurate for individuals yet unfair to groups. About 85% of choices are generated correctly, while some groups are misjudged by 10 points or more. Removing gender from the voter profile reduces the largest group error of the the models by 22–42%, and removing education reduces it by up to 64%, whereas removing self-reported race increases the mean group error by over 40%, all without lowering accuracy. Common alignment strategies do not reduce these errors and can double them. Future work will test combinations of voter information, and alignment interventions that adjust how strongly each attribute shapes AI choices, across other elections, countries, and multi-party systems. 9

References

[26] Bolei Ma, Berk Yoztyurk, Anna-Carolina Haensch, Xinpeng Wang, Markus Herklotz, Frauke Kreuter, Barbara Plank, and Matthias Assenmacher. 2025. Algorithmic fidelity of large language models in generating synthetic german public opinions: A case study. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1785–1809. [27] Srijoni Majumdar, Edith Elkind, and Evangelos Pournaras. 2026. Generative AI voting: Fair collective choice is resilient to LLM biases and inconsistencies. EPJ Data Science (2026). [28] Evangelia N Markaki, Virginia Tsoukatou, Dimitra Mari, and Konstantinos Mamais. 2026. Predictive tools and methods for forecasting and strategic planning: AI in political marketing campaigns. In Understanding Voter Behavior With Predictive Modeling. IGI Global Scientific Publishing, 269–302. [29] Gandalf Nicolas and Aylin Caliskan. 2024. A taxonomy of stereotype content in large language models. arXiv preprint arXiv:2408.00162 (2024). [30] Joon Sung Park, Carolyn Q Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, et al. 2024. LLM agents grounded in self-reports enable general-purpose simulation of individuals. arXiv preprint arXiv:2411.10109 (2024). [31] Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. 2024. Hidden persuaders: LLMs’ political leaning and their influence on voters. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 4244–4275. [32] Weihong Qi, Hanjia Lyu, and Jiebo Luo. 2025. Representation bias in political sample simulations with large language models. In Companion Proceedings of the ACM on Web Conference 2025. 1264–1267. [33] Luca Rettenberger, Markus Reischl, and Mark Schutera. 2025. Assessing political bias in large language models. Journal of Computational Social Science 8, 2 (2025), 42. [34] Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering LLAMA 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 15504–15522. [35] Yuval Rymon. 2026. Of the people, by the algorithm: how AI transforms the role of democratic representatives? AI & SOCIETY (2026), 1–16. [36] Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect?. In International conference on machine learning. PMLR, 29971–30004. [37] SungUk Shin and Youngjoon Kim. 2025. Enhancing graph of thought: Enhancing prompts with LLM rationales and dynamic temperature control. In International Conference on Learning Representations, Vol. 2025. 80634–80651. [38] Pujen Shrestha, Dario Krpan, Fatima Koaik, Robin Schnider, Dima Sayess, and May Saad Binbaz. 2024. Beyond WEIRD: Can synthetic survey participants substitute for humans in global policy research? Behavioral Science & Policy 10, 2 (2024), 26–45. [39] Seungjong Sun, Eungu Lee, Dongyan Nan, Xiangying Zhao, Wonbyung Lee, Bernard J Jansen, and Jang Hyun Kim. 2024. Random silicon sampling: Simulating human sub-population opinion using a large language model based on grouplevel demographic information. arXiv preprint arXiv:2402.18144 (2024). [40] Bryan Chen Zhengyu Tan and Roy Ka-Wei Lee. 2025. Unmasking implicit bias: Evaluating persona-prompted LLM responses in power-disparate social scenarios. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 1075–1108. [41] Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Ulisse Mini, and Monte MacDiarmid. 2024. Activation Addition: Steering Language Models Without Optimization. Technical Report. doi:10.48550/arXiv.2308.10248 [42] Leah Von der Heyde, Anna-Carolina Haensch, Alexander Wenz, and Bolei Ma. 2024. United in diversity? Contextual biases in llm-based predictions of the 2024 european parliament elections. 80th Annual AAPOR Conference (2024). [43] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022). [44] Chenxiao Yu, Jinyi Ye, Yuangang Li, Zheng Li, Emilio Ferrara, Xiyang Hu, and Yue Zhao. 2024. A large-scale simulation on large language models for decisionmaking in political science. arXiv preprint arXiv:2412.15291 (2024). [45] Xinnong Zhang, Jiayu Lin, Libo Sun, Weihong Qi, Yihang Yang, Yue Chen, Hanjia Lyu, Xinyi Mou, Siming Chen, Jiebo Luo, et al. 2024. Electionsim: Massive population election simulation powered by large language model driven agents. arXiv preprint arXiv:2410.20746 (2024). [46] Jianpeng Zhao, Chenyu Yuan, Weiming Luo, Haoling Xie, Guangwei Zhang, Steven Jige Quan, Zixuan Yuan, Pengyang Wang, and Denghui Zhang. 2025. Large language models as virtual survey respondents: Evaluating sociodemographic response generation. arXiv preprint arXiv:2509.06337 (2025). [47] Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2024. Can large language models transform computational social science? Computational linguistics 50, 1 (2024), 237–291.

[1] Andrés Abeliuk, Vanessa Gaete, and Naim Bro. 2026. Auditing socio-demographic and cross-societal fairness in LLM-simulated public opinion. EPJ Data Science (2026). [2] Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In International conference on machine learning. PMLR, 337–371. [3] Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis 31, 3 (2023), 337–351. [4] Ashwini Ashokkumar, Luke Hewitt, Isaias Ghezae, and Robb Willer. 2026. Large language models can predict the results of social science experiments. Nature (2026), 1–8. [5] Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. 2024. Measuring political bias in large language models: What is said and how it is said. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 11142–11159. [6] James Bisbee, Joshua D Clinton, Cassy Dorff, Brenton Kenkel, and Jennifer M Larson. 2024. Synthetic replacements for human survey data? The perils of large language models. Political Analysis 32, 4 (2024), 401–416. [7] Roberto Cerina and Élise Rouméas. 2025. The democratic ethics of artificially intelligent polling. AI & SOCIETY 40, 5 (2025), 3209–3223. [8] Yaoyu Chen, Yuheng Hu, and Yingda Lu. 2024. Simulating field experiments with large language models. SSRN (2024). [9] Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. 2023. Can AI language models replace human participants? Trends in cognitive sciences 27, 7 (2023), 597–600. [10] Mats Faulborn, Indira Sen, Max Pellert, Andreas Spitz, and David Garcia. 2025. Only a little to the left: A theory-grounded measure of political bias in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 31684–31704. [11] Mehdi Gholamnia, Sarbast Moslem, and Francesco Pilla. 2026. Enhancing citizen engagement in transportation decision-making planning through digital voting using artificial intelligence. In Reliable Decision-Making for Sustainable Transportation. Elsevier, 237–253. [12] Zara Hall, Melanie Subbiah, Thomas Zollo, Kathleen McKeown, and Richard Zemel. 2026. Guiding LLM decision-making with fairness reward models. Advances in Neural Information Processing Systems 38 (2026), 145422–145456. [13] Patrick Haller, Ansar Aynetdinov, and Alan Akbik. 2024. Opiniongpt: Modelling explicit biases in instruction-tuned LLMs. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: System Demonstrations). 78–86. [14] Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. Advances in neural information processing systems 29 (2016). [15] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 (2019). [16] John J Horton, Apostolos Filippas, and Benjamin S Manning. 2023. Large language models as simulated economic agents: What can we learn from homo silicus? Technical Report. National Bureau of Economic Research. [17] Maximilian Kreutner, Marlene Lutz, and Markus Strohmaier. 2026. Personadriven simulation of voting behavior in the european parliament with large language models. In Findings of the Association for Computational Linguistics: EACL 2026. 490–511. [18] Erik Larsen. 2025. The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior. arXiv preprint arXiv:2512.12066 (2025). [19] Sanguk Lee, Tai-Quan Peng, Matthew H Goldberg, Seth A Rosenthal, John E Kotcher, Edward W Maibach, and Anthony Leiserowitz. 2024. Can large language models estimate public opinion about global warming? An empirical assessment of algorithmic fidelity and bias. PLoS Climate 3, 8 (2024), e0000429. [20] Hao Li, Ruoyuan Gong, and Hao Jiang. 2025. Political actor agent: Simulating legislative system for roll call votes prediction with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 388–396. [21] Yichen Li, Zhiting Fan, Ruizhe Chen, Xiaotang Gai, Luqi Gong, Yan Zhang, and Zuozhu Liu. 2025. Fairsteer: Inference time debiasing for LLMs with dynamic activation steering. In Findings of the Association for Computational Linguistics: ACL 2025. 11293–11312. [22] Zhicheng Lin. 2024. Large language models as linguistic simulators and cognitive models in human research. Available at SSRN 4974107 (2024). [23] Mitchell Linegar, Rafal Kocielnik, and R Michael Alvarez. 2023. Large language models and political science. Frontiers in Political Science 5 (2023), 1257092. [24] Andy T Liu, Mona Diab, and Daniel Fried. 2024. Evaluating large language model biases in persona-steered generation. In Findings of the association for computational linguistics: ACL 2024. 9832–9850. [25] Dawn Lu and Nina Rimsky. 2024. Investigating bias representations in llama 2 chat via activation steering. arXiv preprint arXiv:2402.00402 (2024).

10

A Theory and Proofs A.1 Notation

Majority voting has no such guarantee. Majority voting applies the threshold ⊮[𝑝ˆ 𝑗 > 1/2] before averaging, which is not linear. Suppose every voter in group 𝑟 has 𝑝ˆ 𝑗 = 0.6 and 𝜃 𝑟 = 0.6. Each run, and probability aggregation, reproduces the group share in expectation (𝜃˜𝑟 = 0.6, 𝜀𝑟 = 0), whereas majority voting assigns every voter to the Democrat (𝜃ˆ𝑟 = 1, 𝜀𝑟 = 0.4). Majority voting pushes uncertain voters toward the locally more likely party and can therefore exaggerate differences between groups. Section 6.1 tests both rules empirically.

Table 5 summarizes the notation. Let D = {(𝑥 𝑗 , 𝑎 𝑗 , 𝑦 𝑗 )}𝑁𝑗=1 be the surveyed voters, where 𝑥 𝑗 is the voter’s profile, 𝑎 𝑗 ∈ G is the voter’s group under a demographic attribute (race, gender, age, or education), and 𝑦 𝑗 ∈ {0, 1} is the reported vote (1 = Democrat). For a group 𝑟 ∈ G, let 𝐽𝑟 = { 𝑗 : 𝑎 𝑗 = 𝑟 } and 𝑁𝑟 = |𝐽𝑟 |. Table 5: Notation.

A.3 Symbol

Meaning

G, 𝑟 𝐽𝑟 , 𝑁𝑟 𝑦𝑗 𝑦ˆ 𝑗,𝑘 𝑝ˆ 𝑗 𝑦ˆ 𝑗 𝜃𝑟 𝜃ˆ𝑟 𝜃˜𝑟 (𝑘 ) 𝜃ˆ𝑟 𝜀𝑟 𝜀 max , 𝜀¯

Set of groups under one attribute; a group Voters in group 𝑟 ; their number Reported vote of voter 𝑗 (1 = Democrat) Emulated vote of voter 𝑗 in run 𝑘 = 1, . . . , 𝐾 Í Share of runs voting Democrat, 𝐾1 𝑘 𝑦ˆ 𝑗,𝑘 Majority vote, ⊮[ 𝑝ˆ 𝑗 > 1/2] Í Reported Democratic share, 𝑁1𝑟 𝑗 ∈ 𝐽𝑟 𝑦 𝑗 Í Emulated share under majority voting, 𝑁1𝑟 𝑗 ∈ 𝐽𝑟 𝑦ˆ 𝑗 Í Emulated share under probability aggregation, 𝑁1𝑟 𝑗 ∈ 𝐽𝑟 𝑝ˆ 𝑗 Emulated share from run 𝑘 alone Misrepresentation error, |𝜃ˆ𝑟 − 𝜃 𝑟 | Worst-group and mean group error over G

A.2

A.4

Proposition 1. Let 𝐵 𝑗,𝑘 = ⊮[𝑦ˆ 𝑗,𝑘 ≠ 𝑦 𝑗 ] indicate an error for voter 𝑗 in run 𝑘. Suppose that, given 𝑥 𝑗 , the 𝐵 𝑗,1, . . . , 𝐵 𝑗,𝐾 are independent with Pr(𝐵 𝑗,𝑘 = 1 | 𝑥 𝑗 ) ≤ 𝛿 < 1/2. Then the majority vote is wrong with probability  Í  

2 𝐾 Pr 𝑦ˆ 𝑗 ≠ 𝑦 𝑗 | 𝑥 𝑗 ≤ Pr 𝐾1 𝑘=1 𝐵 𝑗,𝑘 ≥ 12 𝑥 𝑗 ≤ exp −2𝐾 12 −𝛿 .

We state the result for any convex combination of emulations, which covers both averaging repeated runs of one model and combining several models. Let 𝑓1, . . . , 𝑓𝑀 be emulations that assign each voter a Democratic probability 𝑓𝑚 ( 𝑗) ∈ [0, 1], with group shares 𝜃˜𝑟 (𝑓𝑚 ) = Í Í𝑀 1 Í 𝑗 ∈ 𝐽𝑟 𝑓𝑚 ( 𝑗), and let 𝐹 = 𝑚=1 𝜆𝑚 𝑓𝑚 with 𝜆𝑚 ≥ 0 and 𝑚 𝜆𝑚 = 𝑁𝑟 1.

Í𝐾 𝐵 𝑗,𝑘 , so E[𝑆𝐾 | 𝑥 𝑗 ] ≤ 𝐾𝛿. The majority Proof. Let 𝑆𝐾 = 𝑘=1 vote can be wrong only if at least half of the runs err, that is, if 𝑆𝐾 ≥ 𝐾/2, and 𝐾/2 − E[𝑆𝐾 | 𝑥 𝑗 ] ≥ 𝐾 (1/2 − 𝛿) > 0. Hoeffding’s inequality for independent variables in [0,

1] gives Pr 𝑆𝐾 − E[𝑆𝐾 | 𝑥 𝑗 ] ≥ 𝐾 (1/2 − 𝛿) ≤ exp −2𝐾 (1/2 − 𝛿) 2 . □

Lemma 2 (restated). For every group 𝑟 ∈ G, 𝑀 ∑︁

𝜆𝑚 𝜃˜𝑟 (𝑓𝑚 ) − 𝜃 𝑟 ≤ max 𝜃˜𝑟 (𝑓𝑚 ) − 𝜃 𝑟 .

Under independent noise, more runs would therefore drive errors toward zero. Our runs violate this assumption: Claude gives the same vote in all five runs for 99.3–99.6% of voters and Llama for 84–90%, and one and five runs yield statistically indistinguishable worst-group errors (Section 5.2). Errors are systematic rather than independent, so increasing 𝐾 cannot remove them.

𝑚 𝑚=1

In particular, if every 𝑓𝑚 has error at most 𝜀 for every group, so does 𝐹. Proof. The group share is linear in the emulation, so 𝜃˜𝑟 (𝐹 ) = Í ˜ 𝑚 𝜆𝑚 𝜃 𝑟 (𝑓𝑚 ). Since 𝑚 𝜆𝑚 = 1,

Í

𝜃˜𝑟 (𝐹 ) − 𝜃 𝑟 =

𝑀 ∑︁

Repeated Runs under Independent Noise

The following proposition describes the idealized case in which repeated runs make independent errors. It is a reference point rather than a description of our setting.

Probability Aggregation Preserves Error Bounds

𝜃˜𝑟 (𝐹 ) − 𝜃 𝑟 ≤

Interpretation

The lemma identifies the property an aggregation rule needs for gains at the level of individual voters to survive aggregation: if each emulation represents every group within 𝜀, so does their probability average. It does not imply that aggregation removes misrepresentation, nor that controlling individual representations is sufficient under majority voting. When runs share the same directional error, every term on the right-hand side is large, and the bound offers no improvement.

B Experiment Design B.1 Data

𝜆𝑚 𝜃˜𝑟 (𝑓𝑚 ) − 𝜃 𝑟 .

𝑚=1

Taking absolute values and applying the triangle inequality gives the first inequality; the second holds because a convex combination never exceeds its largest term. □

We use the American National Election Studies (ANES) for 2012, 2016, and 2020. A respondent is included if they reported voting for one of the two major-party presidential candidates and have complete data on race and ethnicity, gender, age, education, ideology, and political attention. This gives 𝑛 = 3,793 (2012), 1,978 (2016), and 5,141 (2020) voters. Group shares are unweighted, so they describe the surveyed respondents rather than the national electorate.

Repeated runs. Setting 𝑓𝑘 ( 𝑗) = 𝑦ˆ 𝑗,𝑘 and 𝜆𝑘 = 1/𝐾 gives 𝐹 ( 𝑗) = 𝑝ˆ 𝑗 , so the probability-aggregated share satisfies 𝐾

1 ∑︁ ˆ(𝑘 ) 𝜃˜𝑟 − 𝜃 𝑟 ≤ 𝜃 𝑟 − 𝜃 𝑟 ≤ max 𝜃ˆ𝑟(𝑘 ) − 𝜃 𝑟 : 𝐾 𝑘

Harmonization. Each attribute is mapped to the same categories in all three years (Table 6). The ANES race codes differ between

𝑘=1

averaging runs is never worse than the average run. 11

years (for example, code 3 denotes Asian voters in 2012 and 2016 but Hispanic voters in 2020) and are recoded accordingly. Groups with fewer than 30 respondents are excluded from all group-level analyses; this removes Native American voters in 2012 and 2016.

voters who self-identify as Black, Hispanic, or Asian/NHPI (fixed seed; the two smallest groups are excluded from the contrast), and record the hidden state of the last prompt token at layer 16 of 32 for their base prompts. The steering direction is the top right-singular vector of the activations of the second sample centred on the mean of the first, used because it explains 43.5% of their variance (above a pre-set threshold of 40%; otherwise the mean difference is used). Its sign is aligned with the mean difference (cosine 0.95), and it is normalized to unit length. During generation, 𝛼 times this vector is added to the hidden state of the last token at layer 16.

Table 6: Demographic groups used in the analysis. Attribute

Groups

Race Gender Age Education

White, Black, Hispanic, Asian/NHPI, Native American, Other Man, Woman 18–34, 35–49, 50–64, 65+ High school or less, Some college, Bachelor’s, Postgraduate

Choice of 𝛼. We set 𝛼 before the main run as the largest value that keeps the overall Democratic share within 10 points of the uncontrolled model, on the first 200 voters of ANES 2020 (Table 7). Larger values collapse almost all predictions to the Democratic candidate and degrade the fluency of the answers. We use 𝛼 = 1.

Profiles. Each voter is described by six fields in a fixed order, for example: “Race/ethnicity: White; Gender: woman; Age: 40; Education: a high school diploma or less; Ideology: liberal; Pays attention to politics: always.” Ideology uses the seven-point ANES scale and political attention its five-point scale, both in their verbal labels. The order in which the two candidates are named in the question is randomized per voter with a fixed seed and kept identical across all models and levels of control.

B.2

Table 7: Overall Democratic share under activation-level control on 200 ANES 2020 voters (uncontrolled Llama: 0.58; reported: 0.58).

Models

We use one model from each access regime. Claude Sonnet 4.6 (claude-sonnet-4-6) is queried through the Anthropic API with temperature 0.7 and a limit of 60 output tokens. Llama-3.1-8BInstruct runs locally in bfloat16 on a single NVIDIA A100 GPU with temperature 0.7, top-𝑝 = 1, and a limit of 40 output tokens, which lets its answers finish after the candidate’s name.

B.5

2

4

8

Democratic share

0.65

0.73

0.99

1.00

Experiments

Table 8: Experiments. All use the voters described in Appendix B.1.

Levels of Control

Every voter is emulated at three levels of control, using the prompts in Appendix D: • Base: the profile and the vote question only. • Prompt-based: adds one instruction to consider individual electoral factors and to avoid under-representing the voting preferences of the voter’s racial or ethnic community. • Decomposed: separates this into a reasoning block on individual signals, an independent block on community representation, and a final prediction. For Llama, we additionally apply activation-level control (Appendix B.4) to the base prompt. The number of runs per voter, 𝐾, is a separate dimension: base-level emulations use 𝐾 = 5 (run 1 alone gives the 𝐾 = 1 comparison), and controlled emulations use 𝐾 = 1, so that all levels are compared on the same footing.

B.4

1

Table 8 lists the experiments behind each part of the results.

Answer parsing. An answer is read as a vote for the first candidate it names (by full name, surname, or first name), after removing neutral pairings such as “between Biden and Trump”. On answers that name only one candidate, this rule agrees with exact matching in every case. Answers that name no candidate are excluded: none for Claude and fewer than 1% for Llama in every election.

B.3

𝛼

Section

Elections

Level of control

Runs

Sec.5.2 Robustness Sec.5.3 Source Sec.6.1 Aggregation Sec.6.2 Control Sec.6.4 Prompt vs. activation

2012, 2016, 2020 2020 2012, 2016, 2020 2012, 2016, 2020 2020

Base Base, three profile variants Base Base, prompt-based, decomposed Decomposed, activation (Llama)

𝐾 =5 𝐾 = 1 or 5 𝐾 =5 𝐾 =1 𝐾 =1

Source of misrepresentation. For each attribute 𝐴 (race, gender, education), we compare three profiles: 𝐴 alone (one prompt per group, repeated 20 times with alternating candidate order), the full profile, and the full profile without 𝐴. Llama uses 𝐾 = 5 for the latter two; Claude uses 𝐾 = 1, as its runs agree for 99.6% of voters.

B.6

Statistical Analysis

Group errors. For each group, the 95% confidence interval of the signed error 𝜃ˆ𝑟 − 𝜃 𝑟 is computed from the paired per-voter differences 𝑦ˆ 𝑗 − 𝑦 𝑗 as their mean ± 1.96 standard errors. An error is called significant when this interval excludes zero. Comparisons. Differences between conditions evaluated on the same voters (one vs. five runs; levels of control; aggregation rules; profile variants) are tested with a paired bootstrap that resamples voters 1,000 times; we report percentile intervals and two-sided 𝑝-values.

Activation-Level Control

We use Contrastive Activation Addition (CAA). From the ANES 2020 sample, we draw 100 voters who self-identify as White and 100 12

Finer subgroups. To test whether the larger worst-group error across race × gender subgroups exceeds what smaller samples alone would produce, we shuffle gender within each racial group, which keeps group sizes and race-level errors fixed, recompute the worstgroup error 1,000 times, and report the share of shuffles at least as large as the observed value.

five runs is not statistically significant in any election for either model (paired bootstrap, 𝑝 ≥ 0.51; Section 5.2), and the errors for gender and education, the largest and most consistent in Section 5.2, stay essentially unchanged. More runs make the emulation more stable, but they do not correct a directional error that every run shares.

Gap amplification. For an attribute with groups 𝑟 ∈ G, gap amplification is the slope of a weighted least-squares regression of emulated on reported group shares, Í ¯ 𝑁𝑟 (𝜃 𝑟 − 𝜃¯)(𝜃ˆ𝑟 − 𝜃ˆ) 𝛽= 𝑟Í , ¯2 𝑟 𝑁𝑟 (𝜃 𝑟 − 𝜃 ) ¯ with group-size-weighted means 𝜃¯ and 𝜃ˆ. 𝛽 = 1 means the emulation reproduces the reported differences between groups, 𝛽 > 1 that it exaggerates them, and 𝛽 < 1 that it flattens them. For two groups, 𝛽 is the ratio of the emulated to the reported gap.

D

C

This appendix reports the prompt templates used in our experiments. All templates use the same profile, for example: “Race/ethnicity: White; Gender: woman; Age: 40; Education: a high school diploma or less; Ideology: liberal; Pays attention to politics: always.” The order of {Candidate A} and {Candidate B} is randomized per voter and kept fixed across models and levels of control. The model is instructed to answer with one candidate name, and answers are parsed as described in Appendix B.2. Activation-level control uses the base template unchanged (Appendix B.4).

Number of Runs

E

We test whether more runs per voter reduce misrepresentation by applying majority voting to the first 𝐾 ∈ {1, 3, 5} of the five base-level runs for every voter (Table 9).

𝜀 max Year

𝐾

Acc.

Race

Gender

Educ.

Claude

2012

1 3 5 1 3 5 1 3 5

.822 .822 .823 .832 .832 .831 .847 .847 .848

.051 .051 .051 .071 .071 .071 .124 .135 .135

.085 .086 .085 .094 .094 .094 .107 .106 .106

.048 .048 .047 .076 .076 .076 .109 .108 .108

1 3 5 1 3 5 1 3 5

.786 .792 .795 .818 .828 .825 .851 .857 .858

.106 .098 .092 .142 .135 .127 .169 .159 .136

.127 .123 .120 .140 .133 .137 .052 .051 .049

.122 .112 .110 .171 .169 .162 .060 .064 .063

2016

2020

Llama

2012

2016

2020

Intersectional Results

Table 11 reports the worst-group error when racial groups are split by gender, age, or education, at the base level with 𝐾 = 5. Subgroups with fewer than 30 respondents are excluded, so the number of subgroups varies across elections. Per-subgroup estimates with 95% confidence intervals are provided in the supplementary material. Splitting racial groups always raises the worst-group error, from 0.05–0.14 for race alone to 0.10–0.23 for its intersections. Much of this increase reflects smaller subgroups: several worst subgroups have fewer than 50 respondents, and for race × gender, a shuffle test attributes the increase to sampling noise in some elections (Section 5.2). One pattern, however, recurs across elections and models and includes some of the larger subgroups (up to 266 voters): for voters who self-identify as Hispanic and male, and for voters who self-identify as Hispanic with at most a high school education, the share of emulated Democratic choices is 15–22 points lower than the reported share. This matches the attributes with the most consistent errors in Section 5.2, where the share of emulated Democratic choices for voters who self-identify as male and for voters without a college degree is below the reported share.

Table 9: Accuracy and worst-group error 𝜀 max by attribute under majority voting over 𝐾 runs (base level).

Model

Prompt Templates

F

Statistical Tests for the Profile Variants

Table 12 reports, for each attribute left out of the profile, the change in worst-group error, mean group error, and accuracy relative to the full profile on ANES 2020, with 95% confidence intervals and two-sided 𝑝-values from a paired bootstrap that resamples voters 1,000 times. Claude uses one run per voter and Llama majority voting over five runs. Leaving out gender reduces the worst-group error significantly for both models, and leaving out education does so for Claude. Leaving out race significantly increases the mean group error for both models. Accuracy never decreases significantly; for Llama, it increases slightly when education is left out.

For Claude, the number of runs makes almost no difference: accuracy and the worst-group error by gender and education change by at most 0.1 points, because its runs agree for 99.3–99.6% of voters. The one larger change, race in 2020 (0.124 to 0.135), comes from the small Native American group (𝑛 = 89), where a single voter shifts the group share by more than one point. For Llama, whose runs disagree more often, accuracy rises by at most 1.1 points and the worst-group error changes by at most 1.5 points, except for race in 2020 (0.169 to 0.136), which again concerns the Native American group. The change in the racial worst-group error between one and

G

Discussion and Limitations

What the audit supports. Misrepresentation is measurable, systematic, and hidden by aggregate metrics. Across three elections and two models, emulated electorates reach individual accuracy of 13

Table 10: Prompt templates for the three levels of control. Placeholders in braces are replaced with the voter’s profile and the two candidates. Method

Prompt Template

Unsteered LLM

You are simulating a survey respondent in a U.S. election study. Given the following voter profile, predict which candidate this respondent would vote for. Profile: {profile_text}. Question: Based on this profile, would this respondent vote for {Candidate A} or {Candidate B}? Answer with only one candidate name. You are simulating a survey respondent in a U.S. election study. Given the following voter profile, predict which candidate this respondent would vote for. Profile: {profile_text}. When making the prediction, consider both individual-level electoral factors such as ideology and political interest, and avoid systematically under-representing the voting preferences of the respondent’s racial or ethnic community. Question: Based on this profile, would this respondent vote for {Candidate A} or {Candidate B}? Answer with only one candidate name. You are simulating a survey respondent in a U.S. election study. Given the following voter profile, predict which candidate this respondent would vote for. Profile: {profile_text}. [Shared electoral reasoning block] Consider task-relevant individual-level signals, including ideology, political interest, age, education, and other non-sensitive profile attributes. [Attribute-aware representation block] Separately consider whether the prediction would systematically under-represent the voting pattern of the respondent’s racial or ethnic community. This block should be treated as an independent consideration and should not override the individual-level profile evidence. [Final prediction] Based on both blocks, would this respondent vote for {Candidate A} or {Candidate B}? Answer with only one candidate name.

Naive Prompt Steering

Decomposed Steering

Prompt

0.80–0.86 and small population-level errors, yet misjudge particular groups by 10 points or more. The most consistent errors exaggerate real differences between groups, by gender and education in particular, and repeated runs or aggregation do not remove them. Which racial groups are misrepresented, however, varies across elections, so an audit of one model and election offers limited assurance for another. Studies that use emulated electorates should therefore report group-level errors alongside accuracy, per group and with group sizes, rather than a single summary number.

voters who self-identify as Native American in 2020, rests on 89 voters, and several intersectional subgroups have fewer than 50. We therefore report intersectional results in the appendix and do not claim that they reveal additional misrepresentation beyond what smaller samples produce. Steering-vector construction. The activation-level direction contrasts voters who self-identify as White with voters who self-identify with another racial or ethnic group, and is added to every voter regardless of their group. It therefore encodes demographic identity rather than the accuracy of a voter’s representation, and in practice it shifts the whole electorate toward the Democratic candidate: even at the strength we use, the Democratic share rises by 7 points, and larger strengths collapse almost all predictions to one party (Appendix B.4). Per-group vectors applied only to matching voters, or contrasts between correctly and incorrectly emulated voters within a group, are natural alternatives that we leave to future work.

What the mitigation results do not support. The evidence does not support a claim that the controls we tested reduce misrepresentation. Race-targeted prompts leave the gender and education errors in place and, for Claude, exaggerate racial differences further; where control helps, it removes an overall partisan shift rather than correcting particular groups; and activation-level control worsens misrepresentation. Our controls target race, whereas the largest errors concern gender and education, and the source analysis shows that removing a demographic label trades one error for another. Reducing misrepresentation therefore requires controlling how much weight each label receives, not only whether it is present.

Prompt confounds. The prompt-based and decomposed levels differ from the base level in prompt length, structure, and instruction content at the same time. Any effect of the decomposed level can therefore not be attributed to its block structure alone; a fully controlled comparison would require a single-block prompt of the same length and content. We treat the decomposed level as one

Statistical power. All analyses use every eligible respondent, and group errors are reported with 95% confidence intervals. Small groups nevertheless limit precision: the largest racial error, for 14

Table 11: Worst-group error 𝜀 max for race and its intersections (base level, 𝐾 = 5). “Worst subgroup” gives the subgroup with the largest error, its signed error (positive: share of emulated Democratic choices above the reported share), and its size. “Sig.” counts subgroups whose 95% interval excludes zero, out of all subgroups with at least 30 respondents. Model

Year

Grouping

𝜀 max

Worst subgroup

Claude

Llama

2012

Race Race × gender Race × age Race × education 2016 Race Race × gender Race × age Race × education 2020 Race Race × gender Race × age Race × education

.051 .139 .118 .149 .071 .175 .100 .184 .135 .143 .154 .203

2012

.092 .154 .152 .216 .127 .212 .111 .212 .136 .170 .231 .164

Race Race × gender Race × age Race × education 2016 Race Race × gender Race × age Race × education 2020 Race Race × gender Race × age Race × education

Signed (𝑛)

Sig.

White White men Other, 35–49 Hispanic, Bachelor’s Asian/NHPI Hispanic men Hispanic, 18–34 Hispanic, HS or less Native American Native American women Other, 50–64 Hispanic, HS or less

−.051 (2437) −.139 (1205) −.118 (34) +.149 (87) +.071 (42) −.175 (80) −.100 (40) −.184 (38) +.135 (89) +.143 (42) +.154 (39) −.203 (74)

2/5 3/9 3/14 3/14 0/5 4/8 0/10 1/10 2/6 6/12 5/18 4/18

Hispanic Hispanic men Other, 35–49 Hispanic, HS or less Hispanic Hispanic men Black, 35–49 Black, HS or less Native American Native American men Other, 50–64 Hispanic, HS or less

−.092 (524) −.154 (266) −.152 (33) −.216 (208) −.127 (142) −.212 (80) −.111 (45) −.212 (33) +.136 (88) +.170 (47) +.231 (39) −.164 (73)

3/5 5/9 8/14 7/14 3/5 4/8 2/10 3/10 2/6 5/12 5/18 5/18

Table 12: Change when one attribute is left out of the voter profile (ANES 2020). Differences are “without” minus “full”; negative error differences mean fairer representation. Bold: 𝑝 < 0.05. Model

Left out

Metric

Full

Without

Difference [95% CI]

Change

𝑝

Claude

Race

Worst-group error Mean group error Accuracy Worst-group error Mean group error Accuracy Worst-group error Mean group error Accuracy

.124 .043 .847 .107 .071 .847 .109 .048 .847

.185 .061 .842 .062 .054 .844 .039 .021 .849

+.061 [−.050, +.153] +.018 [+.004, +.043] −.005 [−.011, .000] −.045 [−.054, −.036] −.018 [−.031, −.004] −.004 [−.011, +.003] −.070 [−.100, −.027] −.027 [−.036, −.012] +.002 [−.003, +.008]

+49% +42%

.294 .008 .086 < .001 .020 .328 .002 .002 .502

Worst-group error Mean group error Accuracy Worst-group error Mean group error Accuracy Worst-group error Mean group error Accuracy

.136 .049 .858 .049 .038 .858 .063 .035 .858

.124 .069 .858 .038 .023 .862 .072 .034 .869

−.013 [−.054, +.058] +.020 [.000, +.031] .000 [−.004, +.006] −.011 [−.016, −.005] −.015 [−.026, +.001] +.004 [.000, +.009] +.009 [−.050, +.067] −.001 [−.017, +.016] +.011 [+.006, +.017]

Gender

Education

Llama

Race

Gender

Education

−42% −25% −64% −57% −9% +41% −22% −39% +15% −4%

.980 .040 .818 < .001 .070 .070 .766 .998 < .001

Contamination. Aggregate vote choices by group for the 2012, 2016, and 2020 elections are widely published and may appear in the models’ training data. A model whose share of emulated Democratic

way to implement prompt-level control and scope our claims accordingly. Activation-level control does not share this confound, as it uses the base prompt unchanged. 15

choices for a group is close to that group’s published aggregate vote share may be recalling the published figure rather than conditioning on the profile it was given. The source analysis (Section 5.3) shows that both models respond to the full profile and not only to the demographic label, but it cannot separate conditioning on the profile from recall of published group statistics.

two-party presidential vote, and group shares are unweighted, so they describe surveyed voters rather than the national electorate. Profiles contain six demographic and political fields and exclude party identification, which would make the task considerably easier. Whether the findings extend to other countries, multi-party elections, other survey instruments, or richer profiles remains to be tested.

Scope. We evaluate two models, one per access regime, on one survey series. The samples include respondents who reported a

16

Related documents

• AI Should Facilitate Democratic Deliberation at Scale#1033838
• AI Alignment for Digital Democracy: Fairer Voters Representation against LLM Biases#1448296
• Large Language Models and Augmented Democracy#1200063
• Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study#1355131
• AI-Enhanced Deliberative Democracy and the Future of the Collective Will#238920