arXiv CS open access

Targeted Counterfactual Fingerprinting for Black-Box LLM Ownership Verification

2026 · arxiv_cs

arXiv CS · Papers · 2026

cryptography cybersecurity privacy security

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Targeted Counterfactual Fingerprinting for Black-Box LLM Ownership Verification Yutong Wu1,∗ , Xiaofan Bai2,∗ , Shixin Li1 , Pingyi Hu1 , Ziqi Zhou1 , Zilong Wang1 , Xiaojing Ma1 Songfeng Lu1 , Yuhong Li2 , Jin Xuan2 , Yi Wang3 , Dongmei Zhang3 , Bin Benjamin Zhu3

arXiv:2608.08195v1 [cs.CR] 8 Aug 2026

Huazhong University of Science and Technology 2 Alibaba Group 3 Microsoft Corporation wyt@hust.edu.cn, baixiaofan.bxf@alibaba-inc.com ∗ Equal contribution.

Abstract— Large language models (LLMs) are high-value assets that can be derived through redeployment, fine-tuning, quantization, or further alignment. Because deployed LLMs are commonly exposed only through query APIs, ownership verification must often rely on black-box text responses. This setting is difficult: generations are open-ended and can vary across repeated queries, while existing black-box fingerprints rely on signals that are fragile under a final-response interface, including full-text matching, soft behavioral features, or model-specific prompts designed not to transfer. The central challenge is to obtain auditable evidence that separates derivatives of a protected model from independently trained models using only black-box queries. We propose TCF (Targeted Counterfactual Fingerprinting), a black-box LLM fingerprinting framework that converts openended generation comparison into constrained-answer targeted counterfactual transfer. TCF restricts each verification query to a finite answer space, reducing the surface-form ambiguity that enters the verification score, and optimizes a prompt perturbation toward a counterfactual target different from the protected model’s clean answer on the original prompt. The resulting perturbed prompt is a fingerprint: verification reduces to checking whether the suspect model’s parsed final answer matches the recorded target. To construct discriminative fingerprints offline, we introduce the source-model counterfactual margin (SCM), a protected-modelonly quantity that certifies the target is unlikely before the perturbation and likely after it; SCM controls target selection, perturbation stopping, and fingerprint filtering. Under explicit derived-preservation and independent-transfer budgets motivated by local behavioral closeness, we derive a target-accuracy gap between derived and independent models. The bounds explain when targeted counterfactual transfer becomes ownership evidence, while empirical SCM diagnostics motivate an interior feasible operating region rather than arbitrarily large margins. Across four LLM families, TCF achieves an average AUC of 0.9861, improving over state-of-the-art black-box fingerprints TRAP, ProFLingo, and ZeroPrint by 0.07–0.19. Project available at: https://github.com/Underflow-1/TCF-LLM-Fingerprinting

I. I NTRODUCTION Large language models (LLMs) are high-value intellectual property assets, but verifying whether a deployed model derives from a protected source remains a critical unsolved challenge. Training state-of-the-art LLMs

demands massive data, exascale computation, and costly post-training alignment, yet third parties can reuse these models with minimal friction—wrapping checkpoints in APIs, fine-tuning for downstream tasks, quantizing for efficiency, or adapting via preference tuning. All such reuse is typically delivered as a black-box cloud service: auditors can only submit prompts and read text responses, with no access to weights, hidden states, logits, or decoding configurations [1]–[4]. This creates a fundamental challenge: enforcement of licenses and usage policies requires reliable, auditable evidence, but the only available signal is ambiguous, variable text output. Existing ownership verification methods fall into two categories: watermarking and fingerprinting. Watermarking embeds ownership signals by modifying training, finetuning, decoding, or model parameters [5]–[7]. While effective when owners control the full pipeline, it is invasive and cannot be retroactively applied to released, leaked, or third-party-modified models. Fingerprinting, by contrast, constructs verification queries from pre-trained models without modification, making it a natural noninvasive approach for auditing deployed derivatives [1]– [3]. The core challenge for LLM fingerprinting is openended generation. Unlike classifiers with fixed label spaces, LLMs produce variable, discrete responses under stochastic decoding: semantically identical answers differ in wording, decoding settings alter formats, and semantic similarity scoring is brittle and hard to justify as audit evidence. Existing methods lean on signals that weaken under strict final-text interface: ProFLingo relies on fulltext response matching [2], which breaks under paraphrasing or formatting changes. TRAP uses GCG-optimized suffixes for exact identity verification [1], [8], but its non-transferable design misses fine-tuned, quantized, or modified derivatives. ZeroPrint estimates soft behavioral features via perturbations [3], [9], requiring embeddings

factual transitions itself as ownership evidence. VerifiWe propose Targeted Counterfactual Fingerprinting cation is strictly black-box and hard-label: we compute (TCF), a framework that reduces generation ambiguity target accuracy (TA) across the private fingerprint set, at the scoring stage by reformulating verification as using no suspect-model logits, embeddings, gradients, or a constrained-answer problem. Each prompt restricts hidden states. Across four LLM families, TCF achieves outputs to a finite set A = {a1 , . . . , aK } (implemented as an average AUC of 0.9861 (perfect 1.0000 on three {A, B, C, D} multiple-choice labels), and a deterministic families), outperforming TRAP (0.7931), ProFLingo parser maps responses to valid labels or an invalid symbol. (0.8880), and ZeroPrint (0.9128) by 0.07–0.19. These This converts noisy text generation into reproducible results validate our central thesis: constrained-answer hard-label events, eliminating semantic scoring and restriction reduces generation ambiguity at the verification reducing sensitivity to surface-form variation, decoding interface, while SCM-guided counterfactual perturbation produces a highly discriminative signal for source-family randomness, and format differences. ownership verification. Constrained answers alone do not provide ownership Our key contributions are: evidence—unrelated models may share public knowledge and answer natural questions identically. TCF therefore • Problem reformulation. We recast black-box LLM ownership verification from fragile open-ended comconstructs fingerprints as targeted counterfactual transparison into constrained-answer targeted counterfactual fers: for each base question, we record the protected transfer, producing a robust, auditable hard-label signal. model’s clean answer, then use GCG optimization to • Unified source-only design. We introduce SCM, a synthesize a text perturbation that flips the model’s answer single quantity computed exclusively from the protected to a preselected unlikely target. The resulting prompt model that unifies all stages of fingerprint construction, encodes a source-specific transition: a target that was eliminating the need for surrogates or reference models. improbable under the clean prompt becomes probable • Formal theoretical foundation. Under explicit derivedafter perturbation. preservation and independent-transfer budgets motiAll stages of fingerprint construction are governed by vated by local behavioral closeness, we derive quantitathe source-model counterfactual margin (SCM), a single tive target-transfer bounds and a sufficient condition for quantity computed exclusively from the protected model. a positive target-accuracy gap; empirical diagnostics SCM quantifies both the unlikelihood of the target under identify a useful feasible SCM operating region. the clean prompt and its likelihood after perturbation, • State-of-the-art performance. Under strict hard-label unifying target selection, perturbation stopping, and constraints, TCF matches or outperforms the strongest fingerprint filtering. For the analysis, local behavioral evaluated baseline across the four source families and closeness motivates explicit budgets for how much a maintains strong performance across diverse derivative derived model may weaken the source target preference subgroups. and how much an independent model may acquire through II. R ELATED W ORK generic transfer. Conditioned on these budgets, SCM yields quantitative lower and upper bounds on target Model ownership verification divides into two transfer and a sufficient condition for a positive target- paradigms: invasive watermarking and non-invasive finaccuracy gap. Empirically, very strong perturbations can gerprinting. also increase generic transfer, motivating a feasible SCM operating region rather than maximal optimization. A. Watermarking and Invasive Ownership Signals and similarity scoring rather than parsed text responses.

Watermarking embeds verifiable ownership signals by intervening in the model lifecycle. Classical DNN watermarking uses backdoor triggers, parameter-level signatures, adversarial frontiers, or output modulations [10]–[14]; LLM watermarking extends this to generation statistics, fine-tuning data, or trigger-response pairs [5]– [7], [15]–[18]. Recent variants include instructional backdoors [19], cross-turn contextual backdoors [20], This framing fundamentally differs from prior and probabilistic provenance injection [21]. adversarial-prompt work. TRAP optimizes for exact These methods share two critical limitations for our model identity [1], while ownership verification requires threat model: (1) they cannot be retroactively applied detecting family membership—derivatives may differ to released, leaked, or third-party-modified models; (2) from the original but still be unauthorized copies. TCF they require altering the protected model or its training therefore treats preservation of source-specific counter- pipeline. Additionally, watermarking faces unresolved TCF at a glance

Construct offline: choose a clean-unlikely answer, optimize the source model toward that target, and retain only SCM-qualified prompts. Verify online: send each private prompt to the suspect API, parse one final A/B/C/D label, and aggregate target hits. Ownership signal: derivatives preserve more source-induced counterfactual transitions than independently trained models.

tradeoffs in utility preservation, stealthiness, and attack resistance [22]–[24]. Notably, text watermarking targets content provenance (identifying generated text) rather than model ownership (verifying a suspect service derives from a protected source), making it orthogonal to our problem.

time) and use only parsed hard labels for verification. This produces a simple, reproducible signal compatible with text-only APIs that permit ordinary prompt queries, while SCM-guided perturbation ensures discriminative power for family-level ownership verification. Table I summarizes the taxonomy of LLM ownership verification methods, highlighting our unique position as a non-invasive, strict black-box, hard-label approach.

B. Non-Invasive Fingerprinting Fingerprinting constructs verification queries from pre-trained models without modification, addressing watermarking’s core limitations. For DNNs, prior work leverages decision boundaries, conferrable adversarial examples, universal perturbations, adversarial trajectories, and robustness signatures [25]–[31]. These methods motivate our use of transferable behavioral signals, but they are designed for fixed-label classifiers with continuous inputs—LLMs present unique challenges due to discrete prompts, autoregressive generation, and surface-form variability from paraphrasing, formatting, and stochastic decoding. LLM fingerprinting further splits by access level: White-box/representation-level methods require internal model information: HuRef extracts human-readable fingerprints from invariant parameters [32]; REEF compares representation similarity across models [33]. While these reveal stable internal structures in derivatives, they are incompatible with strict black-box APIs that expose only final text outputs. Our method is complementary: we use source-side access exclusively for offline fingerprint construction and verify suspects using only parsed text labels.

TABLE I LLM OWNERSHIP - VERIFICATION METHODS . “M ODIFIED ?” INDICATES WHETHER THE PROTECTED MODEL OR ITS TRAINING / DECODING PROCESS IS MODIFIED . “S TRICT BLACK BOX ” MEANS THAT VERIFICATION CAN BE PERFORMED FROM THE TEXT RESPONSES ONLY. Method class Text watermarking Backdoor / active watermark White-box / representation fingerprint Black-box behavioral fingerprint TCF

Examples

Modified?

Suspect access

Strict black box

[5]–[7] [15], [17], [19], [20] [32], [33] [1]–[3] Ours

Yes Yes No No No

Text/API Text/API Weights/rep./circuits Text/API Text/API

Partial Often yes No/partial Yes Yes

III. T HREAT M ODEL AND P ROBLEM F ORMULATION A. Threat Model We consider an LLM owner who protects a source model M0 and audits a suspect service Ms . The goal is to determine whether Ms is either the same model as M0 or a derivative of M0 . The owner can access M0 during offline fingerprint construction, but can only query Ms through its public text interface during verification. Owner capability. The owner has full access to the protected source model M0 before deployment or auditing. Therefore, the owner may use source-model probabilities, gradients, and decoding behavior to construct fingerprints. This access is only assumed for M0 and is never assumed for the suspect model. Suspect-model access. The suspect model Ms is a black-box text-generation service. During verification, the owner submits prompts and observes only the final generated text. The owner cannot access the suspect model’s parameters, logits, token probabilities, hidden states, tokenizer internals, system prompt, decoding configuration, training data, or post-training procedure.

C. Black-Box Behavioral Fingerprinting Our work focuses on the most restrictive and practically relevant setting: non-invasive, hard-label black-box verification using only final text responses. ProFLingo [2] constructs queries that elicit target responses from the source model, then checks for exact reproduction. It relies on full-text matching, which is fragile to paraphrasing, decoding variation, and formatting changes. TRAP [1], [8] optimizes GCG adversarial suffixes to force the source model to output random strings. Designed for exact identity verification, its suffixes are intentionally nontransferable, failing to detect fine-tuned, quantized, or modified derivatives. ZeroPrint [3], [9] estimates local Jacobian-like fingerprints via semantic-preserving perturbations and output embeddings. While it captures richer behavioral features, it requires embedding comparisons and repeated perturbation queries, violating strict hardlabel constraints. TCF addresses these limitations by reformulating verification as constrained-answer targeted counterfactual transfer. Unlike prior work, we restrict outputs to a finite label space (reducing surface-form ambiguity at scoring

Strict online evidence contract

The suspect-side audit uses only ordinary text queries and the final parsed label. No suspect-model probability, embedding, hidden state, gradient, tokenizer access, or semanticsimilarity model enters the ownership score. Invalid or ambiguous responses are counted as failures rather than being repaired by a soft scorer.

The positive class contains M0 and models derived from M0 . Such derived models may be produced by supervised fine-tuning, instruction tuning, RLHF, DPOstyle preference tuning, LoRA adaptation, pruning, quantization, or lightweight post-processing. The negative class contains independently trained or unrelated LLMs. These

models may still share public data, similar architectures, comparable sizes, or similar benchmark performance with M0 . Therefore, the verification task is not to test general capability similarity, but to identify whether the suspect service preserves source-family behavior of M0 . Adversary. The suspect provider may deploy a modified version of the source model, adjust decoding parameters, alter system prompts, change response formatting, apply output post-processing, or use a finetuned, compressed, quantized, merged, or otherwise transformed model. The provider is not assumed to reveal any implementation details. We assume the private fingerprint set is not known to the suspect provider before verification. The suspect service is expected to answer ordinary user prompts, including the verifier’s prompts, through its text-generation interface.

explicit derived-preservation and independent transfer budgets, and the formal separation results below are conditioned on those budgets. Under fixed budgets, larger SCM strengthens the separation bounds. Empirically, however, stronger perturbations can also increase generic transfer to independent models, which motivates a feasible operating region and early stopping rather than maximal optimization. The proof details, modeling qualifications, and cross-model validation are provided in Appendix A-B and Appendix A-A. A. Finite Constrained-Answer Space Let M0 denote the protected source LLM and let M denote a suspect LLM. For a prompt p, the raw output of M is a text response, denoted by RM (p). Our prompt template restricts valid answers to a finite constrained answer space

B. Verification Objective

A = {a1 , a2 , . . . , aK },

The owner constructs a private fingerprint set F = {(pi , yi , ti )}ni=1 ,

(3)

In our implementation, A is instantiated by multiplechoice labels such as {A, B, C, D}. Let Parse(·) be a deterministic parser that maps a raw response to one valid label in A, or to ⊥ if the response is invalid, ambiguous, or unparsable. We define the constrained-answer distribution of model M on prompt p as

(1)

where pi is the verification prompt, yi ∈ A is the source model’s clean answer associated with the underlying question, and ti ∈ A \ {yi } is the target answer recorded by the owner. During verification, the owner queries the suspect model Ms with each pi and counts a hit only when the parsed response equals the recorded target ti . The verification score is the fingerprint target accuracy:

qM (a | p) = Pr [Parse(RM (p)) = a] ,

a ∈ A. (4)

Invalid responses are not assigned to any valid label, so P q a∈A M (a | p) may be smaller than one. For a target label t ∈ A and a prompt p, we define the target log-odds L on model M as a target preference signal

1X I [Parse(rs (pi )) = ti ] . TA(Ms ; F) = n i=1

K < ∞.

(2)

The owner declares Ms as a derived model from M0 if its target accuracy reaches a pre-chosen verification threshold ζ. Verification goal. The goal is not to prove byte-level equality between Ms and M0 . Instead, the goal is to decide whether Ms exhibits sufficiently strong sourcefamily fingerprint behavior under the strict black-box interface.

LM (p, t) = log

qM (t | p) . 1 − qM (t | p)

(5)

LM (p, t) > 0 means that the target answer has a probability larger than 1/2, while a larger value means a stronger target preference. Details on probability estimation and clipping are deferred to Appendix A-B1. B. Source-model Counterfactual Margin

IV. T HEORETICAL A NALYSIS

For a question x, let p0 (x) be the clean prompt without In this section, we develop an LLM-specific theory for any perturbation. Let u be a discrete input perturbation, our fingerprinting method. The analysis is deliberately constructed only from M0 , and let separated from the practical construction of Section V. We are aiming to distinguish two model classes—models pu (x) = Template(u, x) (6) derived from the protected source M0 and independently trained models—and we ask: under what structural be the perturbed fingerprint prompt. The clean answer of property does a perturbed verification input, constructed the source model is only from M0 , induce a target-answer accuracy gap y(x) = arg max qM0 (a | p0 (x)). (7) a∈A between these two classes? Our analysis centers on the source-model counterfactual margin (SCM), the source- A target answer t(x) is drawn from A \ {y(x)}, but side target preference created by the optimized text differing from the clean answer is not by itself enough: the perturbation. A large SCM certifies a strong source-side fingerprint should make this target answer unlikely before counterfactual transition. The locality premise motivates the perturbation and likely after it. The counterfactual

requirement is therefore certified by the quantity below rather than by the choice t(x) ̸= y(x) alone.

then the fingerprint satisfies Γ(x, u, t) ≥ Γmin .

(17)

Definition IV.1 (Source-model counterfactual margin). For a candidate fingerprint (x, u, t), its Source-model Counterfactual Margin (SCM) is Γ(x, u, t) = min LM0 (pu (x), t), −LM0 (p0 (x), t) . (8)

Lemma 1 gives an explicit stopping rule for the perturbation optimizer: after selecting a clean-unlikely target, the perturbation should be optimized until the source target loss falls below the threshold in Eq. 16. Equivalently, the optimization should make the source The two terms in Eq. 8 correspond to two sides of the target probability exceed σ(Γmin ) and hence satisfy same counterfactual transition. The first term requires the Eq. 17. To verify Lemma 1, Fig. 1 records how the source target answer to be likely after the perturbation is applied. target loss and SCM evolve during GCG optimization. The second term requires the same target answer to be Panel (a) plots the target loss over steps, panel (b) shows unlikely for the clean question. Therefore, a large SCM the mean SCM over the same trajectory, and panel (c) directly compares target loss with SCM. The result is certifies consistent across the three views: target loss decreases, qM0 (t | p0 (x)) is small, qM0 (t | pu (x)) is large. SCM increases, and the two show a strong negative (9) relation. This supports using target-loss minimization as a The SCM also provides an explicit probability interpre- practical source-side proxy for increasing SCM, while the tation. Let σ(z) = 1/(1 + e−z ) be the sigmoid function. early stopping rule avoids unnecessary over-optimization. Since Eq. 5 is the logit of qM (t | p), we have Additional details are in Appendix A-B5. (10)

(a)

(b)

Target loss vs GCG step

(c)

Mean SCM vs GCG step

and

−1

ρ = − 1.00

100

−2

−4 0

−2 0

qM0 (t | p (x)) ≤ σ(−Γmin ).

100

GCG step

(12)

150

200

SCM

(11)

Mean SCM

qM0 (t | pu (x)) ≥ σ(Γmin ),

Mean target loss

Thus, if Γ(x, u, t) ≥ Γmin , Eq. 8 and Eq. 10 imply

Γ=1 Γ=2 Γ=3

Loss--SCM relation

Ltarget 0

GCG step

qM (t | p) = σ (LM (p, t)) .

100

150

200

GCG step

Target loss

Here, Γmin denotes the minimum required SCM for accepting a fingerprint. A candidate is called SCMqualified if Γ(x, u, t) ≥ Γmin . (13)

Fig. 1. The relationship between target loss and SCM. Optimizing the source-side input perturbation reduces the source target loss, increases the mean SCM, and produces a strong negative relation.

Increasing Γmin strengthens individual fingerprints but can reduce the number and diversity of retained fingerprints. The detailed feasibility discussion is in Appendix A-B2.

Target-loss minimization is an effective operational proxy for increasing SCM.

Takeaway

D. Local Behavioral Closeness Lemma 1 focuses on the protected model M0 . To reason about how the target transition induced by text input perturbation u behaves on other models, the analysis needs one structural premise. Let zM (p) ∈ RK be the answer-logit vector that M assigns to the labels in A on prompt p, and define the target margin

C. How the Perturbation Controls SCM The SCM is useful because it can be controlled using only the protected source model. The clean term −LM0 (p0 (x), t) is controlled by selecting a target answer that is unlikely under the clean source prompt. The perturbed term LM0 (pu (x), t) is controlled by optimizing the input perturbation u against M0 . Let the source-model perturbation loss be Lsrc (u; x, t) = − log qM0 (t | pu (x)).

mM (p, t) = zM (p)[t] − max zM (p)[a]. a̸=t

A positive margin means t is the model’s parsed hard answer in A; a large margin means the hard answer is stable.

(14)

Lemma 1 (Loss threshold for SCM control). Suppose the target label t satisfies 0

−LM0 (p (x), t) ≥ Γmin .

(15)

Assumption IV.2 (Local behavioral closeness). There exist a closeness radius β ≥ 0 and a small slack η ∈ [0, 1) such that, for a derived model MD , a fraction at least 1 − η of fingerprint inputs p ∈ {p0 (x), pu (x)} obey

(16)

|mMD (p, t) − mM0 (p, t)| ≤ β.

If the perturbation u satisfies Lsrc (u; x, t) ≤ log 1 + e−Γmin ,

(18)

(19)

In particular, if Γ ≥ Γmin , then

No such uniform high-probability closeness to M0 holds for an independent model MI .

qMD (t | pu (x)) ≥ σ Γmin − ϵD (MD , x, u, t) .

Assumption IV.2 states that a derived model’s local decision behavior stays near the protected model’s at each evaluated input, including both the clean and the perturbed prompt. The premise is probabilistic: it need only hold for a large fraction of fingerprints, and the verifier aggregates many fingerprints so that a per-fingerprint edge concentrates into a reliable separation. This premise motivates the transfer diagnostics below but is not, by itself, a numerical bound on an unknown suspect; the formal results are conditioned on the explicit ϵD and ϵI budgets. Appendix A-B6 gives the detailed interpretation and targeted-transfer intuition.

Lemma 2 shows how SCM compensates for posttraining drift. To guarantee a derived-model target-hit probability at least 1−τD under budget ϵD , it is sufficient that 1 − τD Γ ≥ ϵD + log . (25) τD To verify Lemma 2, Fig. 2 empirically examines whether derived models preserve the source-induced target preference. We compute the weakening drift ∆+ D over derived model–fingerprint pairs, summarize its ECDF in panel (a), and plug representative budgets into the lower bound σ(Γ − ϵD ) in panel (b). The result shows that smaller budgets correspond to stronger preservation, and that increasing SCM gives a stronger lower bound on derivedmodel target-hit probability. Higher-coverage budgets are more conservative. Appendix A-B7 further summarizes these results, and the corresponding cross-model ECDF diagnostics are reported in Appendix A-A.

E. Targeted Transferability of Derived Models We now instantiate the derived side of Assumption IV.2 on the parsed-log-odds scale used by the verifier. For a derived model MD and a candidate fingerprint (x, u, t), define n u ∆+ D (MD , x, u, t) = max 0, LM0 (p (x), t) (20)

o − LMD (pu (x), t) .

Takeaway

Derived models preserve enough of the source-induced target preference to support conditional transfer.

This quantity is the loss of perturbed target log-odds when moving from the source model to the derivative. For any budget ϵD (MD , x, u, t) satisfying ϵD (MD , x, u, t) ≥ ∆+ D (MD , x, u, t),

F. Targeted Transferability of Independent Models To operationalize the independent-model side of Assumption IV.2, we allow an explicit generic-transfer slack rather than assuming a fixed numerical distance. An independent model need not stay uniformly close to M0 at the perturbed input. The perturbation u is optimized only against M0 , so it follows M0 ’s local gradient geometry; on a model whose local geometry is misaligned with M0 ’s, the same direction produces only a weak, generic effect. For an independent model MI and a candidate fingerprint (x, u, t), define the generic-transfer slack

(21)

we have the deterministic inequality LMD (pu (x), t) ≥ LM0 (pu (x), t) − ϵD (MD , x, u, t). (22) Smaller budgets mean that the considered derivative class preserves the source prefixed target preference more strongly. (a)

(b)

Required preservation budget ECDF

1.0

Lower bound

P(ΔD+ ≤ εD )

o − LM0 (p0 (x), t) .

0.8

0.6 0.4 0.2 0.0

n u ∆+ I (MI , x, u, t) = max 0, LMI (p (x), t)

Derived transfer lower bound

1.0

0.8

95% 90% 3.69 2.97

50% 0.38 0

ΔD+

0.4

0.0

(26)

For any budget ϵI (MI , x, u, t) satisfying

0.6

ϵI (MI , x, u, t) ≥ ∆+ I (MI , x, u, t),

50% εD = 0.38 90% εD = 2.97 95% εD = 3.69

0.2

(24)

SCM threshold Γ

(27)

we have the deterministic inequality 5

LMI (pu (x), t) ≤ LM0 (p0 (x), t) + ϵI (MI , x, u, t). (28) The slack explicitly permits accidental transfer and generic prompt effects.

Fig. 2. Validation of derived-model local closeness (Assumption IV.2). (a) The empirical weakening budget ϵD from Pr(∆+ D ≤ ϵD ) over derived model–fingerprint pairs estimates the covered fraction on which the premise holds. (b) The resulting conditional transfer lower bound σ(Γ − ϵD ) under the marked empirical budgets.

Lemma 3 (Conditional independent-model targeted-transfer upper bound). Let Γ = Γ(x, u, t). If ϵI (MI , x, u, t) ≥ ∆+ I (MI , x, u, t), then qMI (t | pu (x)) ≤ σ −Γ + ϵI (MI , x, u, t) . (29)

Lemma 2 (Conditional derived-model targeted-transfer lower bound). Let Γ = Γ(x, u, t). If ϵD (MD , x, u, t) ≥ ∆+ D (MD , x, u, t), then qMD (t | pu (x)) ≥ σ Γ − ϵD (MD , x, u, t) . (23)

In particular, if Γ ≥ Γmin , then qMI (t | pu (x)) ≤ σ −Γmin + ϵI (MI , x, u, t) .

(30)

(a) Distribution of non-transfer slack

(b)

2.5

0.4 0.2

ΔI+

0.0

95% 90% 4.79 3.98

50% 0.63 0

εI

0.4

50% εI 90% εI 95% εI

0.2

Theorem IV.3 (SCM yields a target-accuracy separation under explicit transfer budgets). Let Γi = Γ(xi , ui , ti ). For a derived model MD , choose valid budgets ϵD,i ≥ ∆+ D (MD , xi , ui , ti ). For an independent model MI , choose valid budgets ϵI,i ≥ ∆+ I (MI , xi , ui , ti ). Then

0.6

0.0

SCM threshold Γ

Fig. 3. Validation of independent-model targeted transfer. (a) Distribution of the required generic-transfer slack ∆+ I . (b) Empirical coverage Pr(∆+ ≤ ϵ ) over independent model–fingerprint pairs. (c) I I Conditional independent-model transfer upper bound σ(−Γ+ϵI ) under the marked empirical slack budgets.

E[TA(MI )] ≤

1X σ(−Γi + ϵI,i ). n i=1

(34)

(35)

Consequently, the expected separability gap satisfies G ≜ E[TA(MD )] − E[TA(MI )] n 1X [σ(Γi − ϵD,i ) − σ(−Γi + ϵI,i )] . ≥ n i=1

(36)

Lemma 4 (Fixed-budget separability). If every fingerprint satisfies Γi ≥ Γmin , ϵD,i ≤ ϵD , and ϵI,i ≤ ϵI , then E[TA(MD )] ≥ σ(Γmin − ϵD ), E[TA(MI )] ≤ σ(−Γmin + ϵI ).

(37)

The fixed-budget lower bound on the expected gap is positive whenever 2Γmin > ϵD + ϵI .

(38)

By Lemma 4, the derived flip probability is lower bounded while the independent flip probability is upper bounded, so the operating point can be chosen inside the feasible band to obtain a positive expected gap. Averaging n diverse fingerprints then concentrates TA around the two class-specific rates, converting a per-fingerprint edge into a thresholdable separation. To verify Lemma 4,

Takeaway

Independent-model transfer requires explicit generic-transfer slack, explaining residual false positives under overoptimization.

(a) Bound separation, conservative case

Target accuracy / gap

1.0

G. SCM-Induced Separability of Target Accuracy Before aggregating fingerprints, consider the fixedbudget single-fingerprint gap envelope

(b) 1.0

positive gap: Γ > 3.48

0.8

0.6

Derived LB Indep. UB Gap LB

0.4 0.2

Empirical separability Derived TA Independent TA Gap

0.6

(31)

SCM threshold Γ

0.4

0.2

0.1

0.0

0.0 Q4 SCM=0.66

Q1 Q2 Q3 SCM=-2.16 SCM=-1.15 SCM=-0.54

SCM quantile

(c)

Q4-matched bound

1.0

ΓQ4 = 0.66

0.5

0.3

0.0

0.8

Derived LB Indep. UB Gap LB

0.6 0.4 0.2 p = 53.7%, P = 0.54

0.0 0

SCM threshold Γ

Fig. 4. SCM-induced separability and the feasible margin band. (a) Conservative fixed-budget envelope using the 90% empirical budgets ϵD ≈ 2.97 and ϵI ≈ 3.98; the resulting positive-gap condition Γ > 3.48 is sufficient, not necessary. (b) Empirical SCM-quantile trend on Qwen3-1.7B Base: independent target accuracy may also rise in the highest-SCM buckets because a stronger perturbation manufactures generic transfer, so the derived–independent gap is maximized at an interior operating margin (the band sweet spot) rather than by pushing SCM arbitrarily high. (c) Q4-matched fixed-slack envelope.

For fixed ϵD and ϵI , larger SCM increases the fixedbudget gap envelope; Appendix A-B11 gives the detailed fixed-budget interpretation and its empirical qualification. For a fingerprint set F = {(xi , ui , yi , ti )}ni=1 ,

1X σ(Γi − ϵD,i ), n i=1

and

For fixed ϵI , increasing SCM tightens the independent upper bound. If ϵI itself grows with SCM because stronger prefixes have larger generic prompt effects, empirical independent accuracy may increase even though the fixed-slack envelope decreases. This distinction is important to understand SCM-bucket plots in Fig. 4. To empirically verify Lemma 3, Fig. 3 measures how much a source-optimized perturbation transfers to independent models. We compute the generic-transfer slack ∆+ I , show its distribution in panel (a), convert it to an ECDF in panel (b), and plug the marked budgets into the upper bound σ(−Γ + ϵI ) in panel (c). The result shows that larger slack budgets cover more independent model–fingerprint pairs, but also give looser non-transfer guarantees. For a fixed slack, increasing SCM tightens the independent-model upper envelope, although empirical independent accuracy can still rise when stronger prefixes create generic transfer. Appendix A-B9 summarizes the diagnostic, and the corresponding cross-model nontransfer results are reported in Appendix A-A.

g(Γ; ϵD , ϵI ) = σ(Γ − ϵD ) − σ(−Γ + ϵI ).

E[TA(MD )] ≥

Target accuracy / gap

0.6

Derived-independent gap

0.5

0.8

Upper bound

P(ΔI+ ≤ εI )

Density

1.0

This is exactly the statistic used by the online verifier.

1.0

0.8

1.5

0.0

Independent Eq.27 coverage

1.0

90% 95%

Target accuracy

50% 2.0

(32)

define the hard-label target-accuracy statistic of a suspect model M as n 1X TA(M ) = I[Parse(RM (pui (xi ))) = ti ] . (33) n i=1

Fig. 4 empirically validates whether the derived and independent bounds translate into an observable targetaccuracy gap. Panel (a) combines the derived lower bound and independent upper bound using the 90% empirical

budgets, panel (b) compares this with the empirical SCM-bucket trend, and panel (c) repeats the bound calculation with high-SCM matched budgets. The result supports the feasible-band interpretation: higher-SCM fingerprints generally increase the derived–independent gap, but the highest-SCM bucket can also increase independent target accuracy when the perturbation becomes too generic. Thus, SCM should be strong enough to survive derived-model weakening, but not pushed indefinitely. Appendix A-B14 gives the detailed empirical interpretation, and model-wise SCM separability results are reported in Appendix A-A.

the hard-label indicators in Eq. 33 are independent conditional on the fixed fingerprint set. If a derived model satisfies µMD ≥ αD and an independent model satisfies µMI ≤ αI with αD > αI , then under the midpoint threshold in Eq. 40, n(αD − αI )2 ⋆ Pr[TA(MD ) < ζ ] ≤ exp − , (42) 2 and n(αD − αI )2 Pr[TA(MI ) ≥ ζ ⋆ ] ≤ exp − . (43) 2 Both the verifier and the construction filter read a hard label, so the final quantity that matters is whether the sampled answer equals t. The target margin of Eq. 18 controls this directly in an idealized constrained decoder. Specifically, under temperature-T softmax sampling normalized over only the K legal labels in A (or conditional on generation producing a valid legal label), 1 Pr[sample = t | p] ≥ . (44) 1 + (K − 1) e−mM (p,t)/T Thus a large source margin mM0 (pu , t) strengthens targetlabel stability under this constrained-label idealization. Eq. 44 does not assert that an arbitrary raw-text API samples only from the legal labels; the actual APIfacing guarantee remains empirical and is checked by the repeated-query stability filter of Sec. V. Appendix A-B16 gives the detailed qualification and margin–sampling argument.

Takeaway

SCM is most useful inside a feasible margin band, supporting early stopping rather than maximal optimization.

H. Threshold Selection and Verification Reliability The previous subsection gives expected target-accuracy bounds. Define n 1X αD = σ(Γi − ϵD,i ), n i=1 (39) n 1X σ(−Γi + ϵI,i ). αI = n i=1 Here, αD is a lower bound on the expected target accuracy of a derived model under the chosen budgets, and αI is an upper bound for an independent model under its chosen slacks. If αD > αI , then any threshold in (αI , αD ) separates the two expectation bounds. A natural threshold is αD + αI ζ⋆ = . (40) 2 In practice, the true budgets of an unknown suspect model are unavailable, so Eq. 40 should be viewed as a theoretical existence statement. Operational thresholds should be pre-registered from source-side construction rules and, when available, calibrated on a held-out validation pool of non-derived models rather than tuned on the test suspect pool. Under the fixed-budget interpretation in Lemma 4, an operational sufficient condition for derived hit probability at least 1 − τD and independent hit probability at most τI is 1 − τD 1 − τI Γmin ≥ max ϵD + log , ϵI + log . τD τI (41) This formula exposes the feasibility trade-off: stronger reliability requirements or larger transfer budgets require larger SCM, which can reduce the retained fingerprint pool.

V. O UR M ETHOD As shown in Fig. 5, we propose Targeted Counterfactual Fingerprinting (TCF), a black-box LLM ownership verification method based on SCM. The owner constructs prompts that strongly change the protected source LLM’s answer from a clean answer to a selected target answer. During verification, the owner queries a suspect model through its generated text and parses the output into a finite constrained answer space. A model is considered likely derived if it preserves a high fraction of these source-specific target answer transitions. A. Target Selection As shown in Fig. 5(A1–A3), target selection starts from a finite-answer candidate question, obtains the source clean answer, and then chooses a clean-unlikely target. For each candidate question x, we first evaluate the clean source prompt p0 (x). The source clean answer is y(x) = arg max qM0 (a | p0 (x)). a∈A

(45)

The target answer should be different from y(x) and should be unlikely under the clean source prompt. We therefore select targets from T (x) = a ∈ A \ {y(x)} : −LM0 (p0 (x), a) ≥ Γmin . (46)

Lemma 5 (Finite-sample reliability under independent fingerprint indicators). Let µM = E[TA(M )]. Suppose

A. Offline Fingerprint Construction on the Protected Source Model

Step A1 Candidate question x with finite legal answer space A = {A, B, C, D}

Finite legal answer space

Which planet is known as the Red Planet? A. Earth B. Mars C. Jupiter D. Venus Answer with only one capital letter.

Step A2

Candidate fingerprint

Step B1 (

Step B2

Source-model counterfactual margin (SCM)

Step C2

u prefix

y clean answer

)

SCM Gate Γ(x, u, t) = min{ LM0(pu, t), −LM0(p0, t) } A

SCM-qualified if Γ(x, u, t) ≥ Γmin

Clean answer y = B

Question x

target answer t is unlikely

Step B3

A B C D target answer t is likely

Verification stability (optional) Repeated source queries

Use GCG to optimize a prefix u

GCG prefix u

Raw text responses

Deterministic parsing to the legal answer space

retain fingerprints with stable target labels sb0(x, u, t) ≥ smin

Target answer t = D

Step B4

Private fingerprint set F

F = {(pi, yi, ti)}i=1n Source-specific counterfactual transition on M₀ B

Target answer t =

Legend Source-side / Offline

Qualification / Private

Online / Black-box

Parsing & Evaluation

Private set F

• SCM-qualified fingerprint prompts • recorded clean answers and target answers • offline and private

D D B

deterministic parser

⊥

A/B/C/D/⊥

Fingerprint-wise verification indicator Zi(Ms) = I[Parse(rs(pi)) = ti]

ID 1 2 3 4

Step C4

Prompt pᵢ ... ... ... ...

Target tᵢ D D B C

Step C5

Parsed label D B B ⊥

Zᵢ(Mₛ) 1 0 1 0

Fingerprint target accuracy

TA(Ms; F) = (1 / n) Σi=1n Zi(Ms)

Protected source model M0

Clean answer y =

Suspect model black-box API

"I think the answer is D." "The correct option is D." "After analysis, it should be B." "I am not sure."

StepC3

Selected target answer t = D

Query the suspect model with fingerprint prompts

After prefix pᵘ(x)

Before prefix p⁰(x)

Select a clean-unlikely target answer t ≠ y

Target selection: choose t ∈ A \ {y} target answer is unlikely under the clean prompt

Step A4

Step C1

target answer

Protected source model

p0(x)

C. Online Black-Box Ownership Verification

Private set F

x question

Query the protected source model with the clean prompt p0(x)

Clean prompt

Step A3

B. SCM Qualification and Private Fingerprint Set

Example: 15 / 20 = 0.75

Ownership verification decision

Derived model from M₀ positive class

Yes

e.g., SFT, LoRA, RLHF, pruning, quantization, compression

TA(Ms; F) ≥ζ?

Key Quantities

TCF Summary

Black-box Assumption

Γ(x, u, t) : source-model counterfactual margin (SCM) TA(Ms; F) : fingerprint target accuracy Γmin : SCM threshold; ζ : verification threshold

TCF verifies whether a suspect model preserves source-specific counterfactual transitions under a strict black-box, hard-label interface.

Only final generated text is observed; no logits, probabilities, embeddings, gradients, or hidden states.

Independent model negative class Independently trained or unrelated LLMs

Fig. 5. Overview of Targeted Counterfactual Fingerprinting (TCF). Panel A illustrates fingerprint generation on the protected source model, Panel B illustrates SCM qualification and fingerprint-set construction, and Panel C illustrates black-box ownership verification.

If T (x) is nonempty, we choose the unlikely valid target, t(x) = arg min qM0 (a | p0 (x)). a∈T (x)

and then stop early. We set Γtarget = Γmin , where the source-side SCM threshold is fixed before verification and is chosen only from source-side feasibility. We use early stopping to avoid unnecessary over-optimization and to retain the first source-qualified perturbation rather than blindly using the final GCG step. This choice is empirically supported by Fig. 4: as target loss decreases, SCM increases, but the strongest empirical SCM regime can also increase generic transfer to independent models. We therefore do not treat “larger SCM” as an unbounded optimization objective. In practice, Eq. 49 is the sourceside stopping condition, and the retained candidate is accepted only after its parser-side SCM and source stability are recomputed.

(47)

If T (x) is empty, the question is discarded because no target answer is sufficiently counterfactual on the source model. This target-selection rule controls the clean side of SCM before the perturbation is optimized. After this step, the remaining task is only to control the perturbed side, namely to make LM0 (pu (x), t) reach the construction target Γtarget . B. Perturbation toward a Target Margin As shown in Fig. 5(A4), given a selected target t, we optimize a discrete input perturbation (prefix) u with GCG [8]. For exposition, we write the objective using the source constrained-answer probability qM0 ; the differentiable source-side surrogate used during search and the parser-side acceptance check are detailed in Appendix A-B3. The objective is the target negative loglikelihood Lsrc (u; x, t) = − log qM0 (t | pu (x)).

Prefix length. The prefix length ℓu controls the search space. A longer prefix usually improves the chance of satisfying Γ(x, u, t) ≥ Γtarget but may reduce naturalness and increase optimization cost. Optimization budget. Let Tmax be the maximum number of GCG iterations and B be the number of candidate token replacements evaluated per iteration. Larger values improve success rate but increase construction cost.

(48)

By Lemma 1, the construction target gives an explicit stopping criterion. Since LM0 (pu (x), t) ≥ Γtarget is equivalent to qM0 (t | pu (x)) ≥ σ(Γtarget ), it is sufficient to optimize until Lsrc (u; x, t) ≤ log 1 + e−Γtarget , (49)

Verification stability. Because LLM outputs can vary across repeated queries under stochastic decoding, we optionally query the source model M0 for R0 repeated trials with the selected prompt and compute

VI. E XPERIMENTS We organize the experimental evaluation by the following five questions. First, does target-answer transfer separate derived models from independent models? Second, how large is the gap between TCF and existing black-box LLM fingerprinting baselines? Third, is the signal stable under decoding changes and simple adaptive perturbations? Fourth, how robust is TCF to adaptive attack? Finally, what is the computational cost compared with the evaluated baselines?

R0 h i 1 X (r) I Parse(RM0 (pu (x))) = t . sb0 (x, u, t) = R0 r=1 (50) A candidate is retained only if

sb0 (x, u, t) ≥ smin .

(51)

We recommend smin ∈ [0.8, 0.95]. This practical filter is motivated by the margin–sampling analysis of Sec. IV-H: under the constrained-label idealization, a larger source A. Experimental Setup target margin favors target-label stability. The repeatedSource models. We evaluate TCF on four representative query filter then checks the actual counterfactual stability open-source LLMs as protected source models: Qwen3under the decoding setting used by the source interface. 1.7B Instruct [34], Qwen3-1.7B Base [34], Llama3-8B To reduce response-format variation at the suspect Base Direct [35], [36], and Mistral-7B-v0.3 [37]. The interface, we append a fixed answer instruction to the public checkpoint identifiers on Hugging Face are listed GCG-optimized prompt and restrict valid parsed outputs in Table II. to the legal answer space. Thus, the verification prompt pi in Eq. 1 is TABLE II pi = si ⊕ qi ⊕ τ.

P ROTECTED SOURCE MODELS AND SUSPECT- MODEL POOLS .

(52)

Here, si = u⋆i is the selected GCG prefix, qi is the underlying multiple-choice question, and τ denotes the fixed answer instruction template that asks Ms to output one capital letter from {A, B, C, D}:

Source

Checkpoint identifier

Total

Source

Derived

Independent

Qwen3-1.7B Instruct Qwen3-1.7B Base Llama3-8B Base Direct Mistral-7B-v0.3

Qwen/Qwen3-1.7B Qwen/Qwen3-1.7B-Base meta-llama/Meta-Llama-3-8B mistralai/Mistral-7B-v0.3

19 22 25 25

1 1 1 1

9 12 13 16

9 9 11 8

Suspect model pool. For each protected source, we evaluate the source checkpoint, direct model positives, and independent negatives. Direct model positives cover supervised/task fine-tuning (SFT), RL-style adaptation (RL), language/domain adaptation, pruning, and other direct derivatives. Independent checkpoints are used only as independent negatives. Table II lists the fixed source/derived/independent model counts used for the main AUC in Table III. Appendix Tab. XI provides the extended checkpoint catalog, and Appendix A-E reports per-suspect diagnostics for checkpoints with the corresponding baseline artifacts.

Question ... A. ... B. ... C. ... D... Answer with only one capital letter: A, B, C, or D. Answer:

Algorithm 1 in Appendix A-C gives the complete offline construction procedure. To maintain a practically effective fingerprint set, we additionally enforce diversity over questions, source clean answers, and target answers during selection. C. Online Black-Box Ownership Verification As shown in Fig. 5(C1–C5), after constructing the fingerprint set, the owner performs online black-box ownership verification for a suspect model Ms by using only black-box queries as described in Sec. III-B. Verification threshold selection. ζ is a threshold chosen to control verification errors. Each verification process requires n queries to Ms , followed by a single pass to compute TA. Since the fingerprint pool F is generated offline and can be reused, multiple independent verification instances can be performed by sampling different subsets Q, enabling repeated verification while preserving query efficiency. Number of fingerprints. The number of fingerprints n controls statistical reliability. A larger n reduces variance but increases query cost. In the main evaluation, we use 20 fingerprints per protected source, matching the fixed registry described in Sec. VI and Appendix A-D1.

Fingerprint Generation. The fingerprint pool is built from Massive Multitask Language Understanding (MMLU) multiple-choice questions [38]. Each source model uses 20 fingerprints from four different subject categories, with five questions per category. The fourdomain design prevents the fingerprint set from being tied to a single topic and improves fingerprint diversity. The details are in Appendix A-D. Baselines. We compare with three state-of-the-art blackbox LLM fingerprinting baselines: TRAP [1], ProFLingo [2], and ZeroPrint [3]. Appendix Tab. X specifies the verification protocol and score definition for each method. B. Main Verification Results Table III reports the main verification AUC results. TCF achieves perfect AUC on Qwen3-1.7B Instruct,

All family

Qwen3-Inst. TPR

1.0

Original

SFT/Task

RLHF/GRPO

Pretrain/Language

Pruning

Other family

0.8 0.6 0.4

Method AUC Ours 1.00 ZPrint 1.00 ProF 0.94 TRAP 0.95

0.2 0.0