When Static Reachability Says No: Checking Joern-Based Negative Verdicts for Python Against Execution

arXiv:2610.39504v1 [cs.SE] 29 Oct 2026

When Static Reachability Says No: Checking Joern-Based Negative Verdicts for Python Against Execution Artur Zi˛eba-Kozarzewski

Bartosz Grzegorzewski

KRYPTON Polska Sp. z o.o. a.zieba@krypton-polska.com ORCID: 0000-0003-4696-4983

KRYPTON Polska Sp. z o.o. (work done as an intern) bartoszgrzegorzewski@gmail.com ORCID: 0009-0002-7696-3308

Abstract—A negative reachability verdict removes a finding from human review. We check such verdicts against execution. Our pipeline, SourceLens, maps Semgrep findings to a Python program’s call graph, whose edges a query takes from the calls that Joern links in its code property graph. We ran two command-line programs from the PyCG macro-benchmark, in which the benchmark’s manual call graph and Joern disagree, under coverage and recorded the calls between their functions. On these edges, 123 functions of asciinema are unreachable from the program’s entry points. The project’s own integration script executes 86 of them, and all our runs 113. In Sublist3r, 19 of 24 are executed. The latest Joern release gives the same verdicts. The missing calls go through values, such as classes and functions passed around, or have no call expression, as in with blocks and property reads. In asciinema, four missing edges make 50 of the 113 executed functions reachable. The manual call graphs also lack observed calls, in asciinema many of them property reads and with blocks, which the benchmark may not count as calls. A rule that turns negative verdicts into unknown recovers 62 of the 113 and 15 of the 19, mostly through dunder and private method names. It recovers 0 of the 5 security findings with a negative verdict, all in executed functions. Its selectivity cannot be measured, as only 10 and 5 negative verdicts were not executed, and under library entry-point models it degrades every negative verdict. Index Terms—reachability analysis, call graphs, Python, Joern, dynamic analysis, vulnerability management

I. I NTRODUCTION Dependency scanners match component versions against vulnerability databases and report many vulnerabilities in code that a project never executes. Zhou et al. report a 92.0% falsepositive rate for SBOM-based scanning on manually audited repositories and show that checking whether the vulnerable functions are used, in their experiment through coverage, removes 61.9% of these false positives [1]. Reachability analysis is therefore attractive as a filter. Its output, however, is not neutral. A verdict that a finding is not reachable is an instruction to stop looking at it. A vulnerable library function is reachable only through a path from the application’s entry points to the call site that invokes it, and that path runs through the application’s own call graph. Static call graphs are incomplete in practice [2]–[4], and Python makes many calls through attributes, decorators and

values passed around. To learn how often a negative verdict is wrong, one needs a reference for the negative class. A second static call graph is a weak reference, because it can miss the same calls. Execution is a one-sided reference. A function that runs is reachable, so every executed function with a negative verdict is an error, and the count is a lower bound. We check negative verdicts on the application’s own call graph. We built SourceLens, a pipeline that maps Semgrep findings in the application’s code to a Joern call graph and returns a three-valued verdict: reachable, not_reachable or unknown. A degradation rule turns a negative verdict into unknown when evidence about the specific target suggests that the graph is incomplete around it. We ran two programs of the PyCG macro-benchmark [5], asciinema and Sublist3r, under coverage, recorded the calls between their functions, and ask three questions: RQ1. How many negative reachability verdicts does execution contradict? • RQ2. Which missing call edges cause the contradicted verdicts? • RQ3. How many contradicted verdicts does a degradation rule with target-local signals recover? •

In both programs, most negative verdicts are contradicted. The missing edges are calls whose callee is a value, and calls that have no call expression. Because one missing edge cuts off everything below it, a handful of call sites decide the verdicts of dozens of functions. The manually built call graphs of the benchmark also lack observed calls. In Sublist3r, these are calls through self to overriding methods; in asciinema, many are property reads and with blocks, which the benchmark may not count as calls. Used as a reference, they agree with many of the contradicted verdicts. The rule recovers about half of the contradicted verdicts in asciinema, mostly through dunder and private method names, none of the property reads and none of the security findings. Because few negative verdicts remain uncontradicted, we cannot measure whether the rule is selective. The paper contributes: •

A check of Joern-based negative reachability verdicts

for two Python programs against execution, with lower bounds on the errors, and the same negative verdicts under a second Joern release. • An attribution of the errors to missing call edges of a few syntactic kinds, measured on observed calls, with an ablation, the cascade of single edges, and minimal test cases on two Joern releases. • An evaluation of a target-local degradation rule against execution. Scenarios, coverage data, call traces, test cases and code are available as an artifact [6]. II. BACKGROUND AND R ELATED W ORK Reachability as a precision instrument. Functionlevel reachability or usage is used to prune dependencyvulnerability alerts [1], [7]. Similar pruning underlies ecosystem-scale studies in Maven [8], [9]. In Python, Drosos et al. use reachability over stitched PyCG call graphs to classify dependency code as bloated, and note that the static graph may contain false negatives [10]. Zhou et al. list reflection and dynamic code loading among the threats to their analysis [1], and Mir et al. argue that false negatives cost more than false positives in security analysis [9]. Measuring what reachability misses. Eclipse Steady combines static analysis with test-driven dynamic analysis. In 7.9% and later 11.7% of the cases with reachable vulnerable code, reachability was established only by the combination of techniques, with reflection and inversion of control named as causes [11], [12]. Imtiaz et al. found 11 alerts on OpenMRS whose vulnerable code was executed by unit tests, while static analysis deemed the dependency unused [13]. Chuang et al. show for a large industrial Java project that adding call edges for dynamic features turns part of the “unused” dependency verdicts into used ones, and argue against a binary reachable/unreachable verdict [14]. SōjiTantei reports a 0.476 false-negative rate against a manual baseline on npm clients [15]. ConVReach evaluates dependency reachability in C/C++ against manually labelled cases [16], and Patel et al. compare sound and unsound binary call graphs with a compiler-level one [7]. Test-generation approaches establish that a library vulnerability can be triggered from client code [17]. Like Eclipse Steady and Imtiaz et al., we use execution to audit negative verdicts. Execution also serves as a reference for call graphs. Sui et al. and Helm et al. build dynamic baselines from tests or fixed inputs [2], [18], and JARVIS builds its Python reference from test traces and manual inspection [19]. DyPyBench compares static and dynamic call graphs of executable Python projects [4]. We use such a reference for the negative verdicts of a reachability filter and for the call edges whose absence produces them. Call-graph soundness. Sui et al. measure a median recall of 0.884 on 31 Java programs against a dynamic oracle [2]. On Android, static tools miss 61% of the dynamically executed methods, and constructors are prominent among the misses [3]. In Java analysis frameworks, missing call-graph elements in the core analysis stage most often stem from mishandled

implicit invocations of special methods [20]. For Python, PyCG reports about 70% average recall against manually built call graphs [5]. Against dynamic call graphs of 39 executable projects, only 49% of the observed edges and 48.5% of the observed callees appear in PyCG’s graphs, mostly because calls on built-in types are not modelled [4]. JARVIS misses calls to functions stored in lists, tuples and dictionaries, and its authors note that PyCG does not handle with statements [19]. A points-to analysis for Python finds edges that PyCG misses, and its authors conclude that recall rather than precision is the more pressing issue [21]. A single unresolved dynamic construct can cut off large parts of a program. Unresolved reflection can leave a Java call graph empty [18], and dynamically computed module names sometimes drop entire JavaScript modules [22]. Without runtime heap information, static analysis often reaches less than 10% of some Java benchmarks [23]. Chakraborty et al. attribute missing edges in JavaScript to root causes from dynamic data flows, including inter-dependent calls, and find that the causes vary across programs [24]. A call through a function passed as a parameter can only be resolved once the calls that supply it are inferred, and one additional edge can change overall reachability markedly [25]. Recovering dynamic property accesses at the point of dispatch raises recall [22]. Our cascade results in Section V-B show an analogous dependence for reachability in Python. Entry-point models. Which code is reachable depends on where the analysis starts. Reif et al. show that a call graph for a library has to follow the purpose of the client analysis, and describe the common practice of treating all non-private methods as entry points [26]. Ali and Lhoták build call graphs of an application alone under a separate compilation assumption. The library can call back only into application methods whose names it knows, and reflective access to application internals is assumed to be rare [27]. Helm et al. name the main methods of applications and the public API of libraries as typical choices [18]. Huang et al. report a recall of 0.76 for PyCG when every function is an entry point. When accuracy is computed along call chains from the application’s entry functions, the recall is 0.61, and they attribute the drop mainly to this difference in how accuracy is computed [19]. We study programs from their own entry point and use library models only to show a structural limit of the rule. Soundiness. Livshits et al. argue that practical wholeprogram analyses are unsound by design, and ask evaluations to compare results with observed dynamic behaviour and to report the consequences of unsoundness for the analysed programs [28]. III. A PPROACH Pipeline. SourceLens runs Semgrep on a source tree, classifies each finding by scope (application, test, build, documentation) and maps it to its enclosing function. It builds one Joern code property graph [29] for the whole tree, with the default options of Joern’s Python frontend. A call edge links a method

TABLE I D EGRADATION SIGNALS . A NY SIGNAL TURNS N O T _ R E A C H A B L E INTO U N K N O W N ; R E A C H A B L E IS NEVER CHANGED . Signal

Condition

Implicit or private target

method name is a dunder (e.g. __init__) or starts with an underscore target is a class with no method reached, or its name contains a framework hint (e.g. authenticator, plugin, adapter) target has no resolved incoming or outgoing edge in the whole graph resolved edges per method in the target’s file below 0.5 edges per method in the whole graph below 0.05

Class-level or framework role Isolated target Sparse neighbourhood Global floor

to each callee of its call sites that Joern resolves to a method of the analysed code (m.call.callee, external methods removed). A graph search from a set of entry points decides whether the enclosing function is reachable and reconstructs the call chain. In the application model APP, the entry points are the module code of every file and the functions named main. This set contains the program’s real entry point and more, so it can only make negative verdicts rarer. The verdicts thus depend both on the graph Joern builds and on this edge definition (Section V-B separates the two); for brevity we call the verdicts computed on these edges Joern’s verdicts. Three-valued verdict. A not_reachable verdict silences a finding. An unknown verdict states that the graph does not support a conclusion and is never silenced. The degradation rule is the only source of unknown in this study. Degradation rule. The rule inspects each not_reachable verdict and turns it into unknown if any of five signals fires (Table I). Four signals concern the target. They fire on an implicitly invoked or private method and on a class-level target whose methods are all unreached or whose name suggests a framework role. They also fire on a target with no resolved edge anywhere in the graph and on a sparse neighbourhood in the target’s own file. The fifth is a floor for a graph that has almost no edges. Since four of five signals are target-local, we call the rule target-local. Positive verdicts are never degraded. The signals were derived from the two CVE-Bench misses described in Section V-B and from an earlier run with an empty graph. The rule carries to call graphs an idea from our earlier work on declared dependency graphs [30]. There, a missing path is reported as unknown rather than used as a veto, and a graph-completeness signal decides when a missing path cannot be trusted. IV. S TUDY D ESIGN Programs. The PyCG macro-benchmark [5] provides 5 Python projects with manually built call graphs. We use repository commit ce37eb5 and the project commits it pins. The APP model fits a project only if its program entry point is in the analysed code. asciinema and Sublist3r have a main. face_classification consists of scripts. Its 23 negative verdicts all agree with the manual graph, and we did not execute it,

because its scripts need trained models and datasets. fabric’s entry point is a console script declared in package metadata, and under APP the manual graph reaches 1 of its 81 functions. autojump’s entry script has no .py suffix and is absent from both graphs. APP thus fits three projects, and we executed two of them. We fixed these criteria in an earlier analysis of the same release run, after its first results. We study asciinema and Sublist3r. In the three other projects, the manual graph agrees with Joern on every negative verdict, reaches almost no code, or lacks the entry script. Our counts therefore describe programs in which the two graphs disagree, not a sample of Python programs. Verdicts. The verdicts come from the release run of SourceLens: one run of version 0.9 at code commit 3c1218c with Joern 4.0.569 and Semgrep 1.180.0 on all projects and on CVE-Bench, with every input pinned. The population consists of the source-level functions in files of application scope, without Joern’s synthetic methods: 134 in asciinema and 98 in Sublist3r. A function has a negative verdict if no path in the saved Joern graph leads to it from the APP entry points. Joern methods are matched to source functions by qualified name. Execution. We ran both programs in a Python 3.8 container without network access, with coverage.py 7.2.7 active in every process, including forked worker processes. A function counts as executed if at least one statement of its own body ran, excluding nested definitions, a leading docstring and the def line, which runs at import. Two scenario sets drive asciinema. int is the project’s integration script tests/integration.sh, unchanged: help and version, auth, play and cat on recordings in both formats, and rec with signals, stdin capture, raw output and appending. ext is our own set. It adds uploads to a local stand-in for the asciinema API, playback from local URLs and web pages, error paths, settings from the configuration file, and notifications triggered by the pause key during recording. We wrote ext after seeing the verdicts, and some scenarios target known findings. Sublist3r has no scripted runs. Our scenarios follow its README and cover the remaining command-line options. Without network, its search requests fail, and the code that processes successful responses does not run. We ran all scenarios 3 times. The sets of executed functions were identical, and we report the first run. Executed lines differed in one signal-handling branch. Observed calls. In the third run, a profiling hook recorded each call between two project functions. The caller is the innermost project frame that encloses the call. If library frames lie in between, as for a thread target or an argparse type converter, the call is marked as made through a library. When a generator resumes, the hook sees the consumer as the caller, not the function that created the generator. Kinds of missing edges. An observed call that has no edge in the Joern graph is a missing edge. A script classifies each by rules over the syntax tree of the caller and assigns the first matching kind in a fixed order. Implicit calls have no call expression for the callee: C(...) reaching C.__init__,

with reaching __enter__ and __exit__, and the read of a @property. In a value call, a value determines the callee. This covers a class or function passed around and called later, a method called on an object held in an attribute, variable or parameter, and self.m() reaching an override in a subclass. A static call names its callee through a plain name, self, super() or a module alias. Calls made through a library and generator resumptions form their own groups. Unlike the JARVIS reference, which filters out some interpreterinvoked calls [19], we keep implicit calls between project functions. The rules are syntactic and were not checked by an independent coder. Ablation and cascade. We add the missing edges of one or more groups to the Joern graph and count the executed functions with a negative verdict that become reachable. Because an edge helps only once its caller is reachable, the groups do not add up, and we report every combination. For the cascade, we pick missing edges one at a time, each time the edge that makes the most remaining executed functions reachable. Manual call graphs. The PyCG authors built the call graphs of the macro-benchmark by hand, spending on average 10 hours per project, to measure the precision and recall of call edges [5]. We compare the observed direct calls with them. We did not establish whether property reads and context-manager calls fall under the benchmark’s notion of a call. Minimal test cases. We wrote 27 small modules for call patterns seen in the missing edges, each with a function run that calls the expected callees when executed. We built their graphs with Joern 4.0.569 and with 4.0.661, the latest release on 20 October 2026, using the edge definition of the pipeline. We also extracted the method references of the graph, to test whether a query that follows them would link more calls. Reproducibility. Each analysis first reproduces all 51 verdicts and 29 degradation decisions of the release run on the projects and all 20 verdicts on CVE-Bench. We extracted the graphs of both programs again with Joern 4.0.569. The call edges were identical to the release run (406 and 516 distinct edges). Joern 4.0.661 produced different graphs (310 and 295 edges) with the same set of negative verdicts in both programs. Missing edges, ablation and cascade are computed on the 4.0.569 graphs; for 4.0.661 we compare the negative verdicts and the test cases. V. R ESULTS A. RQ1: How many negative verdicts does execution contradict? Table II compares the verdicts with execution. In asciinema, Joern reaches 11 of the 134 functions from the entry points. The integration script alone executes 86 of the 123 others, and all scenarios together 113. In Sublist3r, 19 of 24 negative verdicts are contradicted. These counts are lower bounds. A function that did not run in our scenarios may still be reachable. The manual call graphs. Of the functions that Joern does not reach in asciinema, the manual graph reaches 57. The

TABLE II N EGATIVE VERDICTS AGAINST EXECUTION UNDER APP. J OERN NEG : FUNCTIONS J OERN DOES NOT REACH ; MANUAL : VERDICTS OF THE MANUAL CALL GRAPH ON THE SAME FUNCTIONS . C ELLS : EXECUTED / TOTAL . F OR S UBLIST 3 R , WHICH HAS NO SCRIPTED RUNS , “ALL” CONSISTS OF OUR SCENARIOS ONLY. asciinema int

ext

Sublist3r all

all

functions executed 97/134 119/134 124/134 Joern neg 86/123 108/123 113/123 manual reaches 39/57 55/57 56/57 manual does not 47/66 53/66 57/66 manual neg 47/66 53/66 57/66

70/98 19/24 17/18 2/6 43/64

TABLE III V ERDICTS ON FINDINGS OF SECURITY RULES IN THE RELEASE RUN . T HE RELEASE RUN COVERED ALL 5 PROJECTS ; APP FITS THREE OF THEM , AND WE STUDY TWO ( ASCIINEMA AND S UBLIST 3 R ). R E A C H A B L E WITHOUT A CHAIN : THE FINDING LIES IN AN ENTRY POINT, SUCH AS MODULE CODE .

Findings reachable with derived call chain reachable, finding in an entry point not_reachable before degradation degraded to unknown not_reachable after degradation

studied

all 5

6 1 0 5 0 5

15 2 6 7 2 5

remaining 66 would count as correct verdicts against the manual graph, and 57 of these 66 are executed. In Sublist3r, Joern reaches 58 functions that the manual graph does not reach, and 41 of them are executed. The reason is visible in the observed calls. Of the distinct direct calls observed in asciinema between functions of the manual graph, 34 of 145 have no edge in it, not counting the 7 absent generator resumptions. They include 12 property reads, 10 calls from with blocks and 7 calls on objects held in attributes. Whether the benchmark counts property reads and context-manager calls as calls is open (Section IV), so only the remaining absent calls of asciinema are omissions under any reading. In Sublist3r, 21 of 87 observed direct calls are absent, all of them calls through self that reach overriding methods of the search-engine classes, which have a call expression. Security findings. Table III shows the findings of security rules. In the two programs, 5 of 6 keep a not_reachable verdict after degradation, all in asciinema. We traced them by hand in the source (Appendix A). All 5 are executed in our runs, 2 of them by the integration script alone. The other three need our targeted ext scenarios. The manual graph reaches 2 of them. B. RQ2: Which missing call edges cause the contradicted verdicts? Kinds. Table IV lists the missing edges by kind. In asciinema, the most frequent are calls on objects held in attributes or parameters, with blocks and property reads. In Sublist3r, most missing edges come from one construct. main builds a list of search-engine classes and instantiates each element, so

TABLE IV O BSERVED CALLS WITHOUT AN EDGE IN THE J OERN GRAPH , BY SYNTAX OF THE CALL SITE . Group

Kind

implicit value

static library other

asciinema

Sublist3r

C(...) to __init__ with property read class passed as a value method on an object function passed as a parameter self.m() to an override self.m(), same class through library frames generator resumption

7 16 13 7 18 1 0 0 4 7

2 0 0 11 0 0 1 1 2 0

total

73

17

the class is a value. Ablation. In asciinema, 113 functions with a negative verdict are observed as callees. Adding the value edges leaves 46 of them unreachable, adding the implicit edges leaves 103, and adding both leaves 12. The edges through library frames close the rest. The causes do not add up. 67 functions become reachable through value edges alone, 10 through implicit edges alone, 1 through an edge made through a library, and 35 only when edges of several groups are present. In Sublist3r, value edges leave 4 of 19 unreachable, implicit edges 16, and both 1. One missed call through self within a class closes the rest. Against the manual graph as the oracle, the same two groups are also sufficient: they explain all 57 and 18 disagreements. Cascade. A few edges carry a large share of the effect. In asciinema, the command-line entry point registers each subcommand with set_defaults(cmd=...), later creates the command with args.cmd(...) and calls its execute(). The first three edges of the cascade restore this dispatch for playback and recording and make 27 executed functions reachable. The fourth is recorder.record calling record(...), a parameter whose default value is pty.record. It adds 23. These four edges make 50 of the 113 functions reachable. Reaching every one of the 113 takes 50 edges in total, most of which add a single function. With the integration script alone, the same four edges make 36 of 86 functions reachable. In Sublist3r, the first edge, from main to one search-engine class, makes 11 of 19 reachable, and 7 edges reach all of them. How much a missing edge matters depends on the code below it, not on how often its kind occurs. Minimal test cases. Table V shows which expected callees Joern reaches in the test cases. Under the pipeline’s edge definition, both releases miss with, property reads, and classes and functions used as values. They also miss a parameter whose default is a function of another module (k16) and methods on attributes that hold an object of an imported class. With the default function in the same file (k17), the call is linked. Constructor calls are linked when the class name starts with an upper-case letter (k01, k02, k18–k22), also inside with. They are not linked when it starts with a lower-case letter and the class is called by name in its own module or through a module or package alias (k23–k26). In

both programs, 9 of the 11 observed constructor calls of lowercase classes have no edge, and all 7 of upper-case classes have one. Each missed call names the class in its own module or through a module name. The linked lower-case calls all construct raw in asciinema, which the calling modules import by name (from asciinema.term import raw). The test case for this form (k27) is linked on both releases, so the lower-case calls are missed in exactly the forms of k23–k26. In 17 of 27 cases with 4.0.569 and 18 with 4.0.661, at least one expected callee is not reached; the newer release also loses a method on an attribute in a single file (k06). Some calls on receivers of unknown type are linked to every method of the same name: in k15, N2.notify is reached although only N1 is passed. This is consistent with Joern reaching functions that the manual graph does not. k10 passes, yet one call through self within a class is missing in Sublist3r; the test cases do not isolate what differs there. The open issue #64131 of the Joern repository reports parameter default values missing from the graph, the pattern behind record=pty.record; issue #5357 reports shortcomings in handling Python imports. Edge definition. The pipeline links calls only (m.call.callee). The graph also holds method references. In the test cases, a with block on an upper-case class of the same module carries references to __enter__ and __exit__ (k03, k18, k20), while functions and classes used as values appear as plain identifiers. In the two programs, no with block carries such a reference. Following all method references inside method bodies makes 10 more of the 123 functions of asciinema reachable, 8 of them property getters, and 0 in Sublist3r, with the same result on both releases. The edge definition thus explains a small part of the errors. A second setting. CVE-Bench2 is analysed as a library, with every public function as an entry point (model LIB). In 13 of its 20 tasks the target is itself an entry point. Of the other 7, 2 are not_reachable. Both misses are implicit calls. In GHSA-r758-8hxw-4845 (justhtml), the target is JustHTML.__init__, called by JustHTML(...) from outside the package. In CVE-2026-33175 (oauthenticator), the target class defines only private methods decorated with @default, which the traitlets framework calls when an attribute is read before it is set. C. RQ3: How many contradicted verdicts does the rule recover? Application model. The rule turns 62 of the 113 executed functions in asciinema into unknown and keeps 51 as not_reachable. In Sublist3r it degrades 15 of 19 and keeps 4. Without the name signal it degrades 14 and 0. The rule thus recovers mostly methods with dunder or private names. Property reads are implicit calls too, but the 12 executed 1 https://github.com/joernio/joern/issues/6413 and https://github.com/ joernio/joern/issues/5357, accessed 20 October 2026. 2 https://github.com/GiovanniGatti/cve-bench, commit 45cb1bf. A collection of vulnerabilities in Python packages, each with a vulnerable and a fixed commit and a security test, published as a repository rather than a peerreviewed paper.

TABLE V M INIMAL TEST CASES : EXPECTED CALLEES REACHED FROM R U N IN THE J OERN GRAPH . Case

Pattern

k01 k02 k03 k04

C(), then c.m() mod.C() with C(): for cls in [A, B]: cls(1).go() set_defaults(cmd=C); args.cmd(a).execute() self.api = Api(); self.api.upload() a = Api(); a.hostname() parameter n.notify() @property read self.m(), same class table['h']() as k05, class imported as k08, class imported as k06, class imported parameter, two classes with notify; one passed def record(p, record=mod.record); record(p) as k16, default defined in the same file as k03, class defined after run as k01, class defined after run with C(f) inside with open(p) as f with cm.C(x), cm a module alias as k18, class inherits an imported class as k01, class name in lower case as k03, class name in lower case as k23, called as mod.c(x) as k23, class in a package __init__ as k24, class imported by name

k05 k06 k07 k08 k09 k10 k11 k12 k13 k14 k15 k16 k17 k18 k19 k20 k21 k22 k23 k24 k25 k26 k27

4.0.569

4.0.661

2/2 1/1 1/3 2/3

2/2 1/1 1/3 2/3

2/4

2/4

4/4

3/4

2/2 2/2 1/2 2/2 0/1 0/2 2/2 3/4 2/2

2/2 2/2 1/2 2/2 0/1 0/2 2/2 3/4 2/2

1/2

1/2

2/2 1/3 1/1 1/3

2/2 1/3 1/1 1/3

1/3 1/3 0/1 0/3 0/1 0/1 1/3

1/3 1/3 0/1 0/3 0/1 0/1 1/3

property getters of asciinema have public names, and the rule keeps 12 of them as not_reachable. The functions behind the command dispatch in asciinema have ordinary public names, resolved edges elsewhere and files that are not sparse. The missing edge lies several calls above them, where a signal computed from the target and its file cannot observe it. The global floor, the counterpart of the completeness signal for declared dependency graphs [30], fires on no negative verdict in either program. Its file-level variant, the sparseneighbourhood signal, fires on 15 of the 123 in asciinema and on 0 of the 24 in Sublist3r. The missing edges here sit at a few call sites (Section V-B). Since most negative verdicts are contradicted, we cannot measure whether the rule separates wrong verdicts from correct ones. Only 10 and 5 negative verdicts were not executed (the rule degrades 3 and 2 of them), and not running is no proof of unreachability. Overapproximating call candidates, for example by method name, would reach code cut off above the target at a cost in precision that we did not measure. Library models. Under LIB every public function is an entry point, and under LIB+C also the constructors of public classes. A function can then be unreachable only if its name starts with an underscore, so the name signal fires on every negative verdict. The rule degrades all 56 negative verdicts

of face_classification, asciinema, Sublist3r and fabric under LIB, all 25 under LIB+C, and 5 322 of 5 322 on the CVEBench graphs under LIB+C. autojump is left out of all models, because its entry script is absent from both graphs. In these models the rule cannot be selective by construction. VI. T HREATS TO VALIDITY Scope. We execute two small command-line programs, analysed by one frontend of one call-graph builder. The evaluations of Joern on Python that we found concern taint analysis [31], [32], not call edges. The counts describe these programs. The share of contradicted verdicts depends on where in a program the first missing edge lies, as the cascade shows. Scenarios. Execution proves reachability, possibly under a particular configuration or input, such as an installed notification tool or the pause key. We consider such functions reachable, because a negative verdict claims unreachability for every input. Our ext scenarios were written after seeing the verdicts and target some known findings. The int set is the project’s own and gives the more conservative count. Without network, Sublist3r runs its error paths, and its counts are lower than with network access. Call tracing and classification. The lower bounds rest on executed functions. Observed edges also depend on how the caller is attributed: it is the innermost project frame, and generator resumptions are attributed to the consumer. The kinds are syntactic. They describe how a call is written, not how hard it is to resolve: a method on an attribute of a known class is in the value group, as is a class chosen at run time. The classification rules were written by us and not checked by an independent coder. The cascade is greedy and depends on the observed calls. Verdicts depend on the query that defines call edges. Following method references as well makes 10 of 123 and 0 of 24 negative verdicts reachable (Section V-B); other queries could change more. Test cases. We wrote the test cases after observing the missing edges. They show that the patterns occur, not how common they are. The explanation of name-based linking is inferred from the behaviour of the graphs, not from Joern’s code. Selection of programs. We fixed the criteria for applications and the file exclusions in an earlier analysis of the same data, after its first results. face_classification was not executed. Rule design and own tool. The signals were derived from the two CVE-Bench misses that the rule recovers. Two earlier versions of the rule were corrected after re-running it on the pipeline results, which include these projects. Both corrections removed implementation errors that made a signal fire on every input. We keep the thresholds of the release run. In an earlier analysis against the manual graphs, we varied the sparse-neighbourhood ratio between 0.25 and 1.0. The difference between the degraded shares of disagreeing and agreeing verdicts in asciinema then moved between −16.4 and 5.7 percentage points, and Sublist3r did not change. Varying the global floor between 0.01 and 0.1 changed nothing in this range. The authors built and evaluate SourceLens. The

Semgrep rules of the public repository are licensed for internal use only, so the artifact references them by commit instead of including them. VII. C ONCLUSION We studied two Python programs in which the benchmark’s manual call graph and Joern disagree. In both, most functions that our query finds unreachable in Joern’s graph run when the program runs. The errors come from call edges whose callee is a value and from calls without a call expression, and a few such edges decide the verdicts of a large share of the functions. Following method references as well makes 10 of the 123 negative verdicts of asciinema and 0 of the 24 of Sublist3r reachable. The PyCG manual call graphs also lack observed calls and agree with many of these wrong verdicts. In asciinema, many of the calls they lack are property reads and with blocks, which the benchmark may not count as calls. A target-local rule recovers methods with dunder and private names, and misses property reads, the code cut off above the target and every security finding with a negative verdict. Whether it separates wrong from correct negative verdicts remains unmeasured, and under library entry-point models it degrades all of them. Completeness signals computed over the whole graph or a file do not flag these errors, because they sit at a few call sites. A negative verdict is therefore absence of evidence, and it is weakest for code behind a dispatch. Evaluations of such verdicts need a reference that does not share the analysed graph’s blind spots, and execution is one. Resolving value-passing idioms such as argparse subcommands and parameter defaults would change many verdicts at once. Constructor calls of lower-case classes are a narrower pattern, which our test cases isolate: they are missed unless the class is imported by name. A PPENDIX A C ALL CHAINS OF THE SURVIVING MISSES IN ASCIINEMA Source: asciinema at commit 350d5e9, the commit pinned by the PyCG macro-benchmark. Line numbers refer to that commit. All five functions are executed in our runs. • Dispatch (all chains). __main__.py:79 registers set_defaults(cmd=RecordCommand) (likewise for play, cat, upload and auth); __main__.py:106-107 assigns command = args.cmd(...) and calls command.execute(). • pty.record (CWE-78, finding at pty.py:135). commands/record.py:71 calls recorder.record(...); recorder.py:12 declares the default parameter record=pty.record; recorder.py:49 calls record(...). • URLLibHttpAdapter.post (CWE-939, finding at urllib_http_adapter.py:76). commands/upload.py:13 and commands/record.py:101 call self.api.upload_asciicast(...); api.py:21 assigns self.http_adapter =

...URLLibHttpAdapter(); api.py:35 calls self.http_adapter.post(...). • The two notify methods of Notifier and CustomCommandNotifier (CWE-78, findings at notifier.py:11 and :64). commands/record.py:81 passes the notifier to recorder.record; recorder.py:48 wraps it in async_notifier; pty.py:25 calls notifier.notify(...), which enqueues; the worker loop at async_worker.py:31 calls perform, and recorder.py:95 calls self.notifier.notify(...). • asciicast.open_url (CWE-939, finding at asciicast/__init__.py:48). commands/play.py:17 and commands/cat.py:15 enter with asciicast.open_from_url(...); asciicast/__init__.py:91 in __enter__ calls open_url(...). A RTIFACT The code at commit 3c1218c, the pinned inputs, all results of the release run, the scenarios, coverage data, call traces and test cases are deposited at https://doi.org/10.5281/zenodo. 23554742 [6]. D ECLARATION OF G ENERATIVE AI AND AI-A SSISTED T ECHNOLOGIES During the preparation of this work the authors used Claude (Anthropic), via the Claude Code CLI (2026), for four purposes: assisting with drafting and language of the manuscript, supporting the literature search, helping implement the analysis tooling described in Section IV (the execution and calltracing harness, the analysis scripts and the syntactic classifier of missing edges), and writing the minimal test cases. Literature identified with AI assistance was verified against primary sources before citation. Every number of this study is generated by a script from result files that are part of the artifact. The authors reviewed and edited all content and take full responsibility for the content of the published article. The generative-AI tool is not, and cannot be, an author. A complete record of the assisted workflow is preserved in the project’s version-control history. R EFERENCES [1] L. Zhou, M. Dacier, and C. Konstantinou, “A Reality Check on SBOMbased Vulnerability Management: An Empirical Study and A Path Forward,” in Proceedings of the Sixteenth ACM Conference on Data and Application Security and Privacy (CODASPY ’26), 2026, pp. 255– 268. [2] L. Sui, J. Dietrich, A. Tahir, and G. Fourtounis, “On the recall of static call graph construction in practice,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 1049–1060. [3] J. Samhi, R. Just, T. F. Bissyandé, M. D. Ernst, and J. Klein, “Call Graph Soundness in Android Static Analysis,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, 2024, pp. 945–957.

[4] I. Bouzenia, B. P. Krishan, and M. Pradel, “DyPyBench: A Benchmark of Executable Python Software,” Proceedings of the ACM on Software Engineering, vol. 1, no. FSE, pp. 338–358, 2024. [5] V. Salis, T. Sotiropoulos, P. Louridas, D. Spinellis, and D. Mitropoulos, “PyCG: Practical Call Graph Generation in Python,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021, pp. 1646–1657. [6] A. Zi˛eba-Kozarzewski and B. Grzegorzewski, “Artifact for ‘When Static Reachability Says No: Checking Joern-Based Negative Verdicts for Python Against Execution’,” Zenodo, 2026, dataset and code, doi: 10.5281/zenodo.23554742. [7] H. Patel, A. Snit, and M. Polychronakis, “Supply Chain Reaction: Enhancing the Precision of Vulnerability Triage using Code Reachability Information,” in 2025 IEEE Annual Computer Security Applications Conference (ACSAC), 2025, pp. 92–107. [8] Y. Wu, Z. Yu, M. Wen, Q. Li, D. Zou, and H. Jin, “Understanding the Threats of Upstream Vulnerabilities to Downstream Projects in the Maven Ecosystem,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2023, pp. 1046–1058. [9] A. M. Mir, M. Keshani, and S. Proksch, “On the Effect of Transitivity and Granularity on Vulnerability Propagation in the Maven Ecosystem,” in 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 2023, pp. 201–211. [10] G.-P. Drosos, T. Sotiropoulos, D. Spinellis, and D. Mitropoulos, “Bloat beneath Python’s Scales: A Fine-Grained Inter-Project Dependency Analysis,” Proceedings of the ACM on Software Engineering, vol. 1, no. FSE, pp. 2584–2607, 2024. [11] S. E. Ponta, H. Plate, and A. Sabetta, “Beyond Metadata: CodeCentric and Usage-Based Analysis of Known Vulnerabilities in OpenSource Software,” in 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2018, pp. 449–460. [12] ——, “Detection, assessment and mitigation of vulnerabilities in open source dependencies,” Empirical Software Engineering, vol. 25, no. 5, pp. 3175–3215, 2020. [13] N. Imtiaz, S. Thorn, and L. Williams, “A comparative study of vulnerability reporting by software composition analysis tools,” in Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2021, pp. 1–11. [14] C.-C. Chuang, L. Cruz, R. van Dalen, V. Mikovski, and A. van Deursen, “Removing dependencies from large software projects: are you really sure?” in 2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM), 2022, pp. 105–115. [15] B. Chinthanet, R. Gaikovina Kula, R. Eliza Zapata, T. Ishio, K. Matsumoto, and A. Ihara, “SōjiTantei: Function-Call Reachability Detection of Vulnerable Code for npm Packages,” IEICE Transactions on Information and Systems, vol. E105.D, no. 1, pp. 19–20, 2022. [16] W. Feng, X. Li, R. Feng, Y. Zhang, Y. Li, Z. Zhou, Y. Wang, and Y. Li, “Determining the Unreachable: Constraint-Guided Reachability Analysis for Dependency Vulnerabilities,” Proceedings of the ACM on Programming Languages, vol. 10, no. OOPSLA1, pp. 1514–1541, 2026. [17] Z. Chen, X. Hu, X. Xia, Y. Gao, T. Xu, D. Lo, and X. Yang, “Exploiting Library Vulnerability via Migration Based Automating Test Generation,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, 2024, pp. 1–12. [18] D. Helm, S. Keidel, A. Kampkötter, J. Düsing, T. Roth, B. Hermann, and M. Mezini, “Total Recall? How Good Are Static Call Graphs Really?” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, 2024, pp. 112–123. [19] K. Huang, Y. Yan, B. Chen, Y. Li, Z. Tao, and X. Peng, “JARVIS: Scalable and Precise Application-Centered Call Graph Construction for Python,” arXiv:2305.05949v5, 2024. [20] H. Zhang, Y. Pei, S. Liang, Z. Xing, and S. H. Tan, “Characterizing and Detecting Program Representation Faults of Static Analysis Frameworks,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, 2024, pp. 1772–1784. [21] I. Rak-amnouykit, A. Milanova, G. Baudart, M. Hirzel, and J. Dolby, “PoTo: A Hybrid Andersen’s Points-To Analysis for Python,” in 39th European Conference on Object-Oriented Programming (ECOOP 2025), ser. Leibniz International Proceedings in Informatics (LIPIcs), vol. 333, 2025, pp. 27:1–27:29. [22] M. R. Laursen, W. Xu, and A. Møller, “Reducing Static Analysis Unsoundness with Approximate Interpretation,” Proceedings of the ACM on Programming Languages, vol. 8, no. PLDI, pp. 1165–1188, 2024.

[23] N. Grech, G. Fourtounis, A. Francalanza, and Y. Smaragdakis, “Heaps don’t lie: countering unsoundness with heap snapshots,” Proceedings of the ACM on Programming Languages, vol. 1, no. OOPSLA, pp. 1–27, 2017. [24] M. Chakraborty, R. Olivares, M. Sridharan, and B. Hassanshahi, “Automatic Root Cause Quantification for Missing Edges in JavaScript Call Graphs,” in 36th European Conference on Object-Oriented Programming (ECOOP 2022), ser. LIPIcs, vol. 222, 2022, pp. 3:1–3:28. [25] M. Chakraborty, A. Gnanakumar, M. Sridharan, and A. Møller, “Indirection-Bounded Call Graph Analysis,” in 38th European Conference on Object-Oriented Programming (ECOOP 2024), ser. LIPIcs, vol. 313, 2024, pp. 10:1–10:22. [26] M. Reif, M. Eichberg, B. Hermann, J. Lerch, and M. Mezini, “Call graph construction for Java libraries,” in Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2016, pp. 474–486. [27] K. Ali and O. Lhoták, “Application-Only Call Graph Construction,” in ECOOP 2012 – Object-Oriented Programming, ser. Lecture Notes in Computer Science, 2012, pp. 688–712. [28] B. Livshits, M. Sridharan, Y. Smaragdakis, O. Lhoták, J. N. Amaral, B.Y. E. Chang, S. Z. Guyer, U. P. Khedker, A. Møller, and D. Vardoulakis, “In defense of soundiness,” Communications of the ACM, vol. 58, no. 2, pp. 44–46, 2015. [29] F. Yamaguchi, N. Golde, D. Arp, and K. Rieck, “Modeling and Discovering Vulnerabilities with Code Property Graphs,” in 2014 IEEE Symposium on Security and Privacy, 2014, pp. 590–604. [30] A. Zi˛eba-Kozarzewski, “No Edges, No Verdict: A Large-Scale Empirical Study of Declared Dependency Graphs in 78K SBOMs in the Wild,” arXiv:2607.22140, 2026. [31] S. D. Baker Effendi, X. Pinho, A. M. Dreyer, and F. Yamaguchi, “Scalable Language Agnostic Taint Tracking using Explicit Data Dependencies,” in Proceedings of the 14th ACM SIGPLAN International Workshop on the State Of the Art in Program Analysis (SOAP), 2025, pp. 36–42. [32] Y. Wang, S. Wang, J. Zhao, S. Shi, T. Li, Y. Cheng, L. Bian, K. Yu, Y. Zhao, and H. Wang, “YASA: Scalable Multi-Language Taint Analysis on the Unified AST at Ant Group,” in Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering (FSE Companion), 2026, pp. 524–535.

Related documents

• When Static Reachability Says No: Checking Joern-Based Negative Verdicts for Python Against Execution#1490899
• Beyond Static Guarantees: Measuring the Static-Pass Dynamic-Fail Gap in Security-Sensitive and LLM-Generated Python Code#997752
• Beyond Static Guarantees: Measuring the Static-Pass Dynamic-Fail Gap in Security-Sensitive and LLM-Generated Python Code#673433
• IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspective#1013701
• IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspective#919483