Typify: A Tool for Usage-driven Type Inference in Python

Typify: A Tool for Usage-driven Type Inference in Python Hassan Aman

Ali Aman

Muhammad Asaduzzaman

Shaowei Wang†

Jessica Chen

arXiv:2611.14879v1 [cs.SE] 11 Nov 2026

School of Computer Science, University of Windsor, Canada † Department of Computer Science, University of Manitoba, Canada {amanh, burkia, masaduzz, xjchen}@uwindsor.ca, shaowei.wang@umanitoba.ca

Abstract—Python’s dynamic type system, while expressive and flexible, creates significant challenges for static analysis tools in unannotated or partially annotated codebases. Existing approaches either require pre-existing type annotations, or depend on deep learning models that demand large training corpora and significant computational resources. We present T YPIFY, a lightweight, usage-driven static type inference engine that infers precise type information across entire Python projects without annotations or training data. T YPIFY integrates symbolic execution, fixpoint analysis, and context-matching retrieval to propagate types across modules, functions, and recursive call structures. The tool is available as both a command-line interface (CLI) for batch inference, evaluation and downstream integration with other tools, and a Visual Studio Code (VS Code) extension for interactive type assistance. Evaluation on the ManyTypes4Py and Typilus datasets shows that T YPIFY outperforms all static baselines and closely matches state-of-the-art hybrid deep learning approaches, while running approximately 10× faster. Screencast: https://youtu.be/ANkhv1dci3s Website: https://ali-aman.ca/typify Index Terms—type inference, static analysis, Python, developer tools, VS Code

I. I NTRODUCTION Modern software development increasingly relies on Python, yet Python’s dynamic type system means that most real-world code carries no type annotations. Empirical studies show that fewer than 10% of annotatable code elements are typed in practice [1]. Without type information, tools such as linters, static analyzers, and IDEs cannot provide reliable support, and bugs that could be caught early surface only at runtime. While Python’s gradual typing standards [2] allow developers to add annotations voluntarily, doing so manually in large or legacy codebases is tedious and error-prone. Several categories of tools attempt to address this gap. Static type checkers such as Mypy [3], Pyre [4], and Pyright [5] are effective when annotations are already present, but fall back to Any when they are not. Deep learning-based tools such as Type4Py [6] and Typilus [7] achieve broad coverage but are non-deterministic, require large training corpora, and perform poorly on project-specific or user-defined types. Hybrid approaches like HiTyper [8] combine static reasoning with neural predictions, but operate at the function level and inherit the reproducibility limitations of the models they rely on. In this paper, we present T YPIFY, a tool for usage-driven static type inference in Python. T YPIFY infers types by reasoning about how variables, functions, and data structures

are used across a project, rather than relying on annotations or training data. It builds a project-level dependency graph and propagates type information across modules using fixpoint analysis. For unresolved slots, a context-matching retrieval step fills the remaining gaps without sacrificing determinism. T YPIFY is available as both a command-line interface (CLI) and a Visual Studio Code (VS Code) extension. A complete description of the underlying technique is available in our technical paper [9]. II. A PPROACH T YPIFY operates in four stages. We briefly describe each stage below. Full algorithmic details, including formal definitions and threshold selection, can be found elsewhere [9]. A. Dependency Graph Construction Before inference begins, T YPIFY scans the project root and identifies all Python modules, including those nested within packages. Each module is assigned a fully qualified name and represented as a node in a dependency graph, where an edge Mi → Mj indicates that Mi imports Mj . The graph is built by traversing each module’s abstract syntax tree (AST) and extracting its import statements. This gives T YPIFY a precise map of inter-module relationships that governs the order and scope of all downstream inference. Real-world projects often contain circular imports. T YPIFY detects strongly connected components (SCCs) in the dependency graph and treats each SCC as a unified analysis region, iterating inference within it until all module snapshots stabilize. B. Scheduling and Fixpoint Resolution Once the dependency graph is built, T YPIFY schedules modules for analysis. For projects without circular imports, modules are processed in topological order, so that any types inferred in a dependency are available when its dependents are analyzed. For cyclic dependencies, T YPIFY iterates over each SCC, comparing each module’s type snapshot after every pass. Analysis terminates when no snapshot changes, guaranteeing convergence. In practice, most projects complete in a single forward pass. C. Usage-Driven Inference Usage-driven inference is the core of T YPIFY. Rather than relying on annotations or training data, T YPIFY infers types

directly from how variables, functions, and data structures are used. All inferred types are represented as TypeExpr instances, a uniform recursive structure capturing a base type and any nested type arguments. 1 class TypeExpr: 2 base: TypeExpr 3 args: list[TypeExpr]

For each module, T YPIFY constructs a symbol table and traverses the module body statement by statement. Assignments, function calls, and return statements all contribute evidence that refines the inferred type of each type slot. Type updates are monotonic: types only become more specific or remain unchanged, so analysis always converges. Figure 1 illustrates this process. 1 def collect_items(flag): 2 items = [] 3 if flag: 4 items.append(1) 5 else: 6 items.append("x") 7 return items 8 9 collect_items(True)

Codebase

Dependency Graph Builder

Dependency Graph

SCCs Finder

Typeslots Collector

Symbol Tables

Symbol table builder

Processing Sequences

Typeslots

Inference Engine

Typeslots

Stability Checker

Export Results

Typeslots

Retrieval Querier

Typeslots

Fig. 2. An overview of T YPIFY’s architecture.

III. A RCHITECTURE # Step 2

Figure 2 shows an overview of T YPIFY’s architecture. The pipeline begins with the Dependency Graph Builder, which scans the project and constructs a module-level import graph, # Step 4 establishing the order and scope of all downstream analysis. # Step 5 The SCCs Finder identifies strongly connected components # Step 1 in this graph to handle circular imports, grouping mutually dependent modules into unified analysis regions. The Symbol Table Builder then constructs a symbol table for each module, Fig. 1. Example of usage-driven and call-site-driven inference. mapping identifiers to their scopes and definitions. These are 1) The call site collect_items(True) provides con- passed to the Typeslots Collector, which extracts all type slots — crete evidence that flag is bool, which T YPIFY the specific locations in the code where a type is to be inferred. propagates to the parameter slot before entering the The Inference Engine performs usage-driven type inference over these slots, propagating types from call sites, assignments, and function body. 2) items = [] establishes a provisional type of return statements. The Stability Checker determines whether the fixpoint has been reached by comparing type snapshots across list[Any]. 3) items.append(1) refines the slot to list[int]. passes, feeding updated typeslots back into the Inference Engine 4) items.append("x") unifies the existing type if not. Once stable, the Retrieval Querier fills any remaining with list[str], producing list[Union[int, uninferred slots by querying a pre-built index of annotated code for contextually similar entries. Finally, the Export Results str]]. 5) The return statement binds the inferred type of items component writes the inferred types to JSON artifacts consumed to the function’s return slot, giving list[Union[int, by downstream tools or the VS Code extension. str]]. IV. I MPLEMENTATION D. Context-Matching Retrieval T YPIFY is available in two forms: a command-line interface # Step 3

After usage-driven inference, some type slots may remain (CLI) for batch inference and evaluation, and a Visual Studio unknown, for example in uncalled functions or sparsely used Code (VS Code) extension for interactive use. variables. For these cases, T YPIFY applies a context-matching The T YPIFY inference engine implements the four-stage retrieval step. It queries a pre-built index of (context, type) pipeline described in Section II. It is distributed as a standalone pairs extracted from an annotated codebase, matching each Python package with a command-line interface, and requires uninferred slot against indexed entries by the similarity of their Python 3.11 or later. Installation and invocation are covered surrounding source context. Retrieved candidates are treated in Section V. The available commands are summarized in as suggestions rather than definitive answers, and they never Table I. A config.json file can be passed to the relevant override a type already established by usage-driven inference. commands via the --config flag, and allows users to adjust T YPIFY assigns three confidence levels to its predictions: high parameters of the inference pipeline, including the number of for usage-driven results, mid for retrieval matches, and low for top-k results returned by context-matching retrieval, whether name-convention heuristics applied when retrieval yields no context-matching retrieval is enabled, whether the deep learning result. model integration is enabled, and other inference parameters.

TABLE I S UMMARY OF T YPIFY CLI C OMMANDS . M ORE DETAILS OF THESE COMMANDS CAN BE FOUND IN [10].

typify infer <project-path> <output-path> [--config <config-path>] Runs inference on a project and writes inferred types to the output file. typify gt <dataset-path> <output-path> Extracts ground-truth type annotations from an annotated dataset. typify dataset <dataset-path> <output-path> [--config <config-path>] Runs batch inference over a dataset and saves predictions for evaluation. typify eval <gt-path> <tool-path> Evaluates predictions against ground truth using type-matching metrics. typify build <dataset-path> <index-path> [--workers <n>] Builds a retrieval index from an annotated Python corpus.

Given a project directory and an output path, the engine runs the full inference pipeline and writes the inferred types for all type slots to the output directory as JSON artifacts, ready to be consumed by downstream tools. T YPIFY has the capability of integrating with a deep learning model such as Type4Py [6], which serves as a fallback for low-confidence slots. This combination outperforms HiTyper + Type4Py, setting a new state of the art. Context-matching retrieval is powered by Tantivy [11], a full-text search engine library. Type4Py queries are handled via the Requests [12] HTTP library. The VS Code extension communicates with the inference engine by invoking the CLI as a subprocess and consuming its JSON output. Inferred types are displayed on hover: hovering over any identifier shows its inferred type, and for function definitions, the hover panel also lists known call sites and their locations. Where a type annotation is missing, the hover panel includes an Annotate button that inserts the inferred type directly into the source file. A sidebar panel exposes the same configuration toggles as config.json; changes write back to the file and trigger re-analysis. When a file changes, the extension re-analyzes only the affected files instead of rerunning inference over the whole project, which keeps the feedback loop fast for continuous use during development.

T YPIFY runs the inference pipeline and writes the inferred types as JSON artifacts. Each entry records the inferred type, its confidence level, and the source location of the type slot. The developer can adjust inference behaviour by editing config.json, toggling context-matching retrieval and Type4Py integration, or adjusting the usage-driven inference parameters to balance accuracy and performance. B. Case II: Interactive Type Assistance via the VS Code Extension Consider a developer working on a legacy Python codebase. They encounter a function with no type annotations and want to understand its expected inputs and return type. The developer hovers over the function definition, and the T YPIFY overlay appears, showing inferred types for each parameter and the return value. The panel also lists the known call sites from which these types were derived, including the observed argument types and return value, allowing the developer to verify the evidence behind each inferred type. The developer clicks Annotate, and T YPIFY inserts type annotations for the function’s parameters and return value directly into the source file. The function is now typed, allowing type mismatches to be caught statically rather than at runtime. Figure 3 illustrates this workflow across all three stages. VI. E VALUATION

This section summarizes the quantitative evaluation of T YPIFY. Full details of the evaluation procedure and results can be found in the technical paper [9]. Setup. We evaluate T YPIFY on two datasets: ManyTypes4Py [13], a benchmark of Python projects with type annotations, and the Typilus dataset [7], consisting of annotations collected from 600 Python repositories. We report two metrics: exact match, where a prediction is correct only if it matches the ground truth in both base type and type arguments, and base match, where type arguments are ignored. We compare T YPIFY against three compared techniques: Pyre Infer [4], an industry static type checker developed by Meta; HiTyper [8], a hybrid approach that combines static inference with deep learning; and Type4Py [6], a deep learning-based tool. We also evaluate T YPIFY + Type4Py, where Type4Py serves as a fallback for low-confidence slots, and compare it against HiTyper + Type4Py. Results. T YPIFY outperforms all compared techniques by substantial margins on both datasets (Table II). On ManyV. U SAGE S CENARIOS Types4Py, it improves over HiTyper’s static component by 11.7– 16.5 points overall, with especially large gains on argument A. Case I: Batch Type Inference via the CLI Consider a developer who wants to run type inference over types (35–40 points), and by 45–56 points over Pyre Infer. a Python project to feed inferred types into a downstream tool, T YPIFY also beats Type4Py overall despite being fully static. such as a static analyzer or documentation generator. They in- Combined with Type4Py as a fallback, T YPIFY + Type4Py stall the inference engine via pip install typify-cli surpasses HiTyper + Type4Py on most tasks, e.g., 75.0% vs. and invoke the typify infer command (see Table I for 74.2% exact match for variable types, and a 14.7-point gain on argument prediction. The same trend holds on the Typilus the full list of available commands): dataset, confirming the hybrid combination sets a new state of 1 typify infer <project-path> <output-path> the art across both benchmarks.

A

B

C

Fig. 3. Using T YPIFY in VS Code: (a) unannotated function in the editor, (b) hover overlay showing inferred types, call sites, and the Annotate button, (c) the function after clicking Annotate with type annotations inserted.

TABLE II T OP -1 TYPE PREDICTION ACCURACY (%). EM = EXACT MATCH , BM = BASE MATCH .

ManyTypes4Py Task

Variable

Argument

Return

Typilus

Approach

EM

BM

EM

BM

T YPIFY T YPIFY + Type4Py HiTyper HiTyper + Type4Py Type4Py Pyre Infer

62.7 75.0 57.0 74.2 55.2 6.9

74.7 86.5 63.8 81.6 60.1 7.1

18.5 19.0 16.3 18.3 14.4 0.9

55.3 65.2 52.7 63.3 33.6 1.1

T YPIFY T YPIFY + Type4Py HiTyper HiTyper + Type4Py Type4Py Pyre Infer

43.0 60.0 7.8 45.3 46.0 9.4

47.7 65.5 8.1 47.7 47.9 9.4

49.2 65.4 7.0 52.1 50.3 8.8

52.2 69.4 7.1 54.0 51.7 8.8

T YPIFY T YPIFY + Type4Py HiTyper HiTyper + Type4Py Type4Py Pyre Infer

38.4 48.0 34.0 46.8 33.9 35.2

50.5 60.4 43.7 56.9 36.4 36.2

31.6 40.6 26.9 40.7 29.4 27.3

44.0 55.2 37.5 52.9 32.4 28.3

Inference Speed. T YPIFY requires an average of 4.7 ms per data point on ManyTypes4Py, compared to 48.2 ms for HiTyper and 36.5 ms for Type4Py, corresponding to a 90% reduction in latency relative to HiTyper. A similar trend holds on the Typilus dataset, where T YPIFY requires 3.9 ms per data point compared to 45.3 ms for HiTyper. Since T YPIFY is fully static, these figures exclude training time, which Type4Py and HiTyper both require. VII. U SER S TUDY We conducted a user study to assess the usefulness of T YPIFY and to identify any usability issues. All five participants are Computer Science students with at least three years of Python programming experience. As this is a small-scale study, our results should be read as preliminary evidence rather than a general claim of effectiveness. To avoid the confound of participants struggling with an unfamiliar codebase, we

developed a controlled Python project implementing a library management system, covering operations such as checking books in and out, managing loans, and tracking members; we view this toy project as a useful first step, and plan to evaluate T YPIFY on larger, real-world codebases with more participants in future work. Each participant completed a set of tasks with and without the T YPIFY VS Code extension enabled. Tasks included adding new functionality, adjusting existing functions, and identifying expected types in unfamiliar parts of the code. When working without T YPIFY, participants used Pylance (which uses Pyright), the standard Python type inference environment that ship with the VS Code Python extension. Participants completed tasks without T YPIFY first, then repeated equivalent tasks with the extension enabled, followed by an interview. All five participants agreed that T YPIFY helped them understand the codebase more quickly. Pylance and Pyright fell back to Any for unannotated functions, giving participants little actionable information. T YPIFY inferred concrete types and surfaced the call sites from which those types were derived, which participants found particularly useful for understanding not just what type was inferred, but why. The Annotate button also reduced the friction of adding annotations manually. Without T YPIFY, participants took noticeably longer to reason about types and in several cases introduced type mismatches that would only have surfaced at runtime. The study also surfaced several areas for improvement. Two participants noted that T YPIFY could not provide linter features such as auto-completion. One participant suggested the sidebar show the Type4Py API endpoint and allow changing it for custom self-hosted instances — we found this valuable and incorporated it. The remaining features are currently under active development. VIII. R ELATED W ORK A number of static type checkers are available for Python, including but not limited to Mypy [3], Pyre [4], Pyright [5], and Pytype [14]. However, these tools focus on type checking rather than inference, and often default to Any when annotations are

missing, limiting their usefulness for legacy or unannotated Visual Studio Code extension for interactive use and as a codebases. command-line interface for batch inference and integration with To address this, several tools infer types using neural downstream tools. T YPIFY not only outperforms traditional models trained on large code corpora. Typilus [7] uses graph static analysis and deep learning–based approaches in type neural networks, Type4Py [6] applies deep similarity learning, inference accuracy, but also performs inference significantly DeepTyper [15] uses sequence models, and TypeWriter [16] faster (e.g., T YPIFY is approximately 10× faster than HiTyper), uses search-based validation. More recent work has explored making it practical for real-time use in developer tools. The LLM-based annotation generation [17], [18], and HiTyper [8] source code and datasets are publicly available, enabling combines static analysis with deep learning over type depen- other researchers to replicate and extend our work [10]. The dency graphs. These LLM-based assistants generate annotations demonstration will showcase how T YPIFY can be used in a by predicting plausible tokens from patterns learned during variety of usage scenarios. training, rather than by verifying how a value is actually R EFERENCES constructed and used, so their output can vary across runs and [1] L. D. Grazia and M. Pradel, “The evolution of type annotations in Python: is not guaranteed to be consistent with the surrounding project. An empirical study,” in Proceedings of the 30th ACM Joint European T YPIFY instead derives each annotation deterministically by Software Engineering Conference and Symposium on the Foundations of tracing concrete usage through the project’s dependency graph, Software Engineering (ESEC/FSE), 2022, pp. 209–220. so a given codebase always yields the same, traceable result. [2] G. van Rossum, J. Lehtosalo, and Łukasz Langa, “PEP 484: Type hints,” https://peps.python.org/pep-0484/, 2014, retrieved Jan 29, 2026. However, existing approaches are either non-deterministic, [3] Mypy Contributors, “Mypy,” https://mypy-lang.org/, 2026, retrieved Jan limited to function-level inference, or struggle with project29, 2026. [4] Meta Platforms, “Pyre,” https://pyre-check.org/, 2026, retrieved Jan 29, specific types. T YPIFY requires no training data, produces 2026. deterministic results, and performs whole-project inference, [5] Microsoft, “Pyright,” https://microsoft.github.io/pyright/, 2026, retrieved with T YPIFY + Type4Py outperforming HiTyper + Type4Py Jan 29, 2026. [6] A. M. Mir, E. Latoškinas, S. Proksch, and G. Gousios, “Type4Py: on most tasks. Beyond Python, gradual typing systems such Practical deep similarity learning-based type inference for Python,” as TypeScript [19] and Flow [20] combine inference with in Proceedings of the 44th International Conference on Software constraint solving. Engineering (ICSE), 2022, pp. 2241–2252. IX. L IMITATIONS T YPIFY handles the majority of real-world Python code well, as reflected in its evaluation results across diverse open-source repositories. However, several language features present challenges for purely static analysis. Metaprogramming constructs and decorators that dynamically modify definitions at runtime are not fully supported, and dynamic imports cannot be resolved through the dependency graph. For example, consider the following code: 1 module_name = "os.path" 2 mod = importlib.import_module(module_name) 3 result = mod.join("usr", "bin")

Since the module name is determined at runtime, T YPIFY cannot resolve the type of mod or result statically, and falls back to Any for those slots. Higher-order functions are supported in common cases, but complex patterns involving mutual recursion or context-dependent signatures may not be fully resolved, causing T YPIFY to fall back to Any for the affected slots. Support for several of these features is currently under active development. In practice, these limitations rarely affect usability since the majority of Python codebases do not rely heavily on such constructs. X. C ONCLUSION We present T YPIFY, a tool for usage-driven static type inference in Python. T YPIFY infers types by reasoning about and tracing how variables, functions, and data structures are used throughout an entire project, without relying on existing type annotations or training data. It is available both as a

[7] M. Allamanis, E. T. Barr, S. Ducousso, and Z. Gao, “Typilus: Neural type hints,” in Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2020, pp. 91–105. [8] Y. Peng, C. Gao, Z. Li, B. Gao, D. Lo, Q. Zhang, and M. Lyu, “Static inference meets deep learning: A hybrid type inference approach for Python,” in Proceedings of the 44th International Conference on Software Engineering (ICSE), 2022, pp. 2019–2030. [9] A. Aman, M. Asaduzzaman, and S. Wang, “Typify: A lightweight usage-driven static analyzer for precise Python type inference,” in Proceedings of the 34th IEEE/ACM International Conference on Program Comprehension (ICPC ’26), 2026, to appear. [10] A. Aman, “Typify CLI repository,” https://github.com/ali-aman-burki/ typify-cli, 2026, retrieved Jan 29, 2026. [11] Tantivy Contributors, “Tantivy: A full-text search engine library,” https: //github.com/quickwit-oss/tantivy, 2024, retrieved Jan 29, 2026. [12] K. Reitz et al., “Requests: HTTP for humans,” https://requests. readthedocs.io, 2024, retrieved Jan 29, 2026. [13] A. M. Mir, E. Latoškinas, and G. Gousios, “ManyTypes4Py: A benchmark Python dataset for machine learning-based type inference,” in Proceedings of the IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), 2021, pp. 585–589. [14] Google, “Pytype,” https://google.github.io/pytype/, 2026, retrieved Jan 29, 2026. [15] V. J. Hellendoorn, C. Bird, E. T. Barr, and M. Allamanis, “Deep learning type inference,” in Proceedings of the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2018, pp. 152–162. [16] M. Pradel, G. Gousios, J. Liu, and S. Chandra, “TypeWriter: Neural type prediction with search-based validation,” in Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2020, pp. 209–220. [17] Y. Zhang et al., “Generating Python type annotations from type inference,” Communications of the ACM, 2024. [18] Y. Peng, C. Gao, Z. Li, D. Lo, and M. Lyu, “Generative type inference for Python,” arXiv, 2023. [19] Microsoft, “Typescript,” https://www.typescriptlang.org/, 2026, retrieved Jan 29, 2026. [20] Meta Platforms, “Flow,” https://flow.org/, 2026, retrieved Jan 29, 2026.

Related documents

• Typify: A Tool for Usage-driven Type Inference in Python#1648909
• Automating Just-In-Time Python Type Annotation Updating#361526
• Improving Large Language Models for Code through Runtime Program-State Reasoning#1119432
• Prompt Less, Infer More: Static Analysis-Guided LLM Type Annotation for Python#1534786
• TTSR: Test-Time Self-Evolving via Reflection#1014626