arXiv CSopen access

Project-wise Comparison of Software Birthmarks Using Weighted Partial Similarity

Project-wise Comparison of Software Birthmarks Using Weighted Partial Similarity Nikolay Fedorov ∗ fedorov [email protected],

Akito Monden †, [email protected],

Haruaki Tamada §, [email protected],

Hiroki Inayoshi ‡, [email protected],

Masateru Tsunoda ¶ [email protected],

arXiv:2606.25418v1 [cs.SE] 24 Jun 2026

June 25, 2026

1

Abstract Software birthmarks provide a robust approach to detecting code plagiarism even under substantial modifications, while distinguishing independently developed software. Existing similarity measures are typically applied at the module level (e.g., source or class files). However, in practice, software reuse often occurs at the project level, where only a subset of modules may be reused. This setting introduces two key challenges: (1) partial reuse, where reused modules constitute only a small fraction of the project, and (2) incidental similarity from small modules, which can lead to false positives. In this paper, we establish a framework for project-wise birthmark comparison based on a symmetric aggregation of module-level similarities. On top of this framework, we propose two complementary mechanisms to address the above challenges. First, we introduce a weighting scheme that assigns higher importance to larger modules, reducing the influence of noisy matches from small modules. Second, we propose a partial similarity method that focuses on the top fraction of highly similar module pairs, enabling robust detection of partial reuse. We evaluate the proposed approach on 35 open-source Java projects across ten categories, where different versions of the same project are treated as reuse cases. The dataset and experimental artifacts are made publicly available to support reproducibility. Performance is assessed using two complementary properties of software birthmarks, resilience and credibility, combined via their harmonic mean. The results show that the proposed method consistently outperforms existing approaches, achieving robust and stable detection of partial code reuse at the project level.

Introduction

Software plagiarism and unauthorized code reuse have become significant concerns in modern software development, particularly with the widespread availability of open-source software (OSS) and code-sharing platforms [1, 2]. These unethical practices involve copying or reusing code without proper attribution or permission. Recent studies have highlighted the prevalence of unauthorized code reuse across various software development platforms. Research on StackOverflow1 has examined how uploaded code snippets are reused without proper attribution [4, 5], while other work has focused on detecting code cloning in open-source projects [1,3]. Golubev et al. [6] analyzed Java projects in the Public Git Archive [33] and reported that approximately 29.6% of code fragments may have been incorporated without authorization, with about 9.4% potentially violating original licenses. Similarly, Lopes et al. [2] found that nearly 70% of code hosted on GitHub2 consists of clones of existing files. These findings underscore the urgent need for reliable techniques to detect code reuse in real-world software systems. Software birthmarks have been proposed as an effective technique for detecting software plagiarism [7], offering an advantage over software watermarking methods [20, 21] in that they do not require any modification to the original files. A birthmark captures intrinsic characteristics of a program, enabling similarity analysis even when the code has undergone transformations such as obfuscation [9, 10]. Various types of birthmarks have been proposed, including instructionbased [8, 10], structure-based [15, 19], and API call-based approaches [11–13,36,38,42]. Similarity functions such as cosine similarity and the Jaccard coefficient are commonly

∗ Corresponding author. N. Fedorov is with Graduate School of Environmental, Life, Natural Science and Technology, Okayama University, Okayama, Japan † A. Monden is with Faculty of Environmental, Life, Natural Science and Technology, Okayama University, Okayama, Japan ‡ H. Inayoshi is with Faculty of Environmental, Life, Natural Science and Technology, Okayama University, Okayama, Japan § H. Tamada is with Faculty of Information Science and Engineering, Kyoto Sangyo University, Kyoto, Japan ¶ M. Tsunoda is with Faculty of Informatics, Cyber Informatics Research Institute, Kindai University, Higashiosaka-shi, Japan 1 https://stackoverflow.com 2 https://github.com/

1

• We construct a benchmark dataset and evaluation methodology based on resilience and credibility.

used to quantify similarity between modules [23, 52]. Most existing studies focus on module-level comparison (e.g., source or class files), which is effective for small-scale scenarios such as plagiarism detection in programming assignments [46, 47]. However, this approach becomes insufficient in large-scale and real-world settings [40], where software systems are composed of numerous modules and reuse may occur only partially. Real-world cases further highlight this limitation. For example, Vizio’s SmartCast platform allegedly violated the GNU GPL by reusing code from BusyBox and coreutils [43,44], and Teraproc’s OpenLava project was found to incorporate proprietary IBM code [45]. Such cases demonstrate that plagiarism often occurs at the level of entire software systems rather than individual files. These observations motivate a two-stage detection process: (1) project-level retrieval, which identifies candidate projects that may contain reused code, and (2) module-level verification, which performs detailed analysis of reuse. Without an effective project-wise similarity measure, the first stage cannot be performed reliably, making large-scale plagiarism detection impractical. Despite its importance, project-wise birthmark comparison has received limited attention. One reason is the lack of systematic evaluation settings. Another challenge lies in aggregating module-level similarities into a reliable project-level measure. This setting introduces two key difficulties: (1) partial reuse, where reused code appears only in a subset of modules, and (2) incidental similarity, where small or generic modules produce misleading similarity across unrelated projects. In this paper, we address challenges by establishing a method for project-wise birthmark comparison. Our approach builds upon module matching and similarity aggregation, and introduces two key mechanisms. First, a weighting scheme assigns higher importance to larger modules, which are more likely to contain meaningful reuse. Second, a partial similarity strategy focuses on the most relevant subset of module pairs, enabling robust detection even when reuse is limited. These mechanisms jointly improve both resilience to modifications and credibility in distinguishing unrelated projects. To enable systematic evaluation, we construct a dataset of 35 open-source Java projects across 10 categories, where different versions of the same project are treated as reuse cases. In this study, we focus on static k-gram birthmarks to provide a controlled and consistent evaluation setting, while the proposed framework itself is general and applicable to other types of birthmarks. Our contributions are summarized as follows:

• We demonstrate through extensive experiments that the proposed method improves detection performance over existing approaches. The remainder of this paper is organized as follows. Section 2 introduces the concept of software birthmarks. Section 3 formulates the problem of project-wise similarity and discusses its inherent limitations. Section 4 presents the proposed framework for project-wise birthmark comparison. Section 5 describes the experimental design, including the dataset and preprocessing methods. Section 6 reports the experimental results and provides detailed analysis. Section 7 discusses threats to validity, and Section 8 concludes the paper.

2

Software Birthmarks

2.1

Definitions

A software birthmark is a representation of a program that captures its intrinsic characteristics and enables similarity-based comparison between programs [9–11,23]. Formally, let B(P ) denote a function that extracts a birthmark from a program P . If program Q is an exact copy of P (i.e., Q ≡ P ), then their birthmarks are identical: B(P ) = B(Q). (1) In practice, software birthmarks are compared using similarity functions to detect unauthorized code reuse or plagiarism. A detailed explanation on birthmark similarity calculation will be given in Section 2.4. Beyond plagiarism detection, they have also been applied to tasks such as malware analysis and bug detection [3, 11, 28, 30, 32, 50]. A desirable software birthmark should satisfy two key properties [11, 25, 31]: • Credibility : If programs P and Q are independently developed, their birthmarks should not be similar: B(P ) ≁ B(Q), (2) • Resilience: If program Q is derived from P , their birthmarks should remain similar: B(P ) ∼ B(Q).

(3)

Traditionally, resilience has been defined with respect to semantic-preserving transformations, such as code obfuscation or decompile–recompile processes [8, 26, 48, 49]. However, such transformations do not fully reflect realistic software reuse scenarios. In practice, reused code is often modified, extended, or partially rewritten to fit new contexts, introducing functional changes beyond purely semantic-preserving transformations [50]. In this work, we adopt a broader interpretation of resilience that includes such practical modifications. Specifically, we consider a program Q to be derived from P if Q is created by reusing and modifying parts of P , even

• We formulate the problem of project-wise birthmark comparison and position it as a key component in large-scale plagiarism detection. • We identify two fundamental challenges—partial reuse and incidental similarity—in project-level similarity measurement. • We propose a weighted and partial similarity framework to address these challenges. 2

when additional functionality is introduced. Under this interpretation, resilience captures the ability of a birthmark to preserve similarity when reused code is modified and integrated into a different software system. In practical scenarios, an adversary may reuse code from an existing project, apply modifications such as extensions, refactoring, or partial rewriting, and distribute it as a new product [50]. A resilient birthmark should still enable the detection of such derivation despite these changes. In our evaluation, we approximate this scenario by comparing birthmarks across different versions of the same software, where later versions include modifications and extensions over earlier ones. While resilience is essential for detecting reused code, credibility is equally important to avoid false positives [8, 26]. In module-level comparison, achieving high credibility is inherently challenging, as small or generic modules may exhibit incidental similarity even when independently developed [51]. By “module” we refer to a source file/compiled source file, as well as any other project component, from which the birthmark is extracted. These observations highlight a fundamental limitation of existing birthmark definitions, which are commonly centered on module-level comparison. When moving to project-level analysis, where a program consists of many modules, both resilience and credibility must be reconsidered in an aggregated context. In particular, project-level similarity should remain robust to partial reuse while suppressing incidental similarity arising from small modules.

dependence paths [40], and project logic paths [41]. They can represent higher-level program characteristics, but often incur higher computational cost and may be sensitive to compiler optimizations [11, 24]. • API-based birthmarks, which are derived from the usage patterns of standard libraries or system calls. For example, such as used classes (UC) and sequences of method calls (SMC) [10], static API call sequences [11], dynamic API call sequences and frequencies [13, 26, 37]. They are generally more robust to program modifications, as API usage patterns are harder to alter without affecting functionality [11, 24]. These categories reflect inherent trade-offs between robustness, granularity, and computational cost. Importantly, the challenges addressed in this paper—such as partial reuse and incidental similarity—are not specific to any particular type of birthmark, but arise from how birthmark similarity is aggregated at the project level. Therefore, the proposed project-wise comparison framework is designed to be applicable across different types of birthmarks, specifically, birthmarks that are calculated and compared per module or per specific part of the project.

2.3

k-gram Birthmark

In this study, we employ the static k-gram birthmark for experimental evaluation. A k-gram birthmark represents 2.2 Birthmark Types Overview a program as a set of contiguous instruction sequences Software birthmarks can be broadly characterized along of length k (i.e., k-grams), extracted statically from its two dimensions: (1) the analysis approach and (2) the code [8, 11]. Given a program P , its birthmark is defined program elements they capture. With respect to the anal- as:  B(P ) = k-grami | i = 1, . . . , K , (4) ysis approach, birthmarks are classified as either static or dynamic. Static birthmarks are extracted without where K denotes the number of unique k-grams. In this executing the program (e.g., from source code, byte- representation, the order and frequency of k-grams are code, or binaries), whereas dynamic birthmarks are ob- typically ignored, which reduces sensitivity to minor code tained by observing runtime behavior under specific in- transformations [8]. puts [11, 22, 23, 27, 35]. Dynamic approaches tend to be The k-gram birthmark is widely used in prior studmore resilient to program transformations, but their ef- ies due to its simplicity and scalability [8, 34, 39]. It can fectiveness depends on input coverage. In contrast, static be extracted from various program representations (e.g., approaches are easier to apply at scale and provide full source code, bytecode, or binaries) with minimal preprocode coverage, but may be more sensitive to modifica- cessing, making it suitable for large-scale analysis. Moretions. over, the parameter k controls the granularity of the repWith respect to the program elements they cap- resentation, enabling a trade-off between robustness and ture, birthmarks are commonly categorized into three discriminative power. groups [11]: While k-gram birthmarks are known to be sensitive to code modifications and may grow in size for large pro• Instruction-based birthmarks, which represent grams [11, 25], these limitations are acceptable for the low-level program elements such as instruction sepurpose of this study. Our goal is not to optimize the quences (e.g., static k-grams of Java opcodes [8] birthmark itself, but to evaluate how similarity should be and dynamic k-grams [34, 39]). They are simple computed at the project level. In this context, k-grams and scalable, but can be sensitive to code modificaprovide a simple and widely adopted baseline that allows tions [8, 11]. us to isolate the effect of the proposed similarity frame• Structure-based birthmarks, which capture work. In addition, for Java programs, publicly available structural relationships such as control flow birth- tools such as pochi 3 facilitate birthmark extraction, furmarks [22], whole path birthmarks [15], program ther lowering the barrier to use. 3 https://github.com/tamada/pochi

3

Importantly, the proposed project-wise similarity method is not specific to k-gram birthmarks and is equally applicable to other types of birthmarks, including those with stronger resilience to code transformations.

3

Project-wise Similarity: Problem and Limitations

3.1

Problem Formulation

In large-scale software ecosystems, a vast number of projects exist, each consisting of many modules (e.g., To determine whether code has been reused between two source files, classes, etc.). In this context, detecting code software modules, it is necessary to quantify the similarity reuse cannot rely solely on module-wise comparison. between their birthmarks. This is typically achieved by Let a project P be represented as a set of modules: a similarity function, which maps a pair of birthmarks to P = {p1 , p2 , . . . , pn }, (6) a numeric value in [0.0, 1.0], where higher values indicate greater similarity. where each module pi is associated with a birthmark Formally, for two modules p and q, let X = B(p) and B(pi ). Given two projects P and Q, module-wise simY = B(q) denote their birthmarks. A module-wise simi- ilarity can be computed for all pairs: larity function is defined as: {sim(pi , qj )}, ∀pi ∈ P, ∀qj ∈ Q. (7) sim(p, q) = f (X, Y ), (5) However, this formulation raises several fundamental where f is a similarity function. challenges. Application of various similarity functions have been First, module-wise comparison produces a large numproposed in prior work [8, 9, 11, 52], which can be broadly ber of similarity scores across module pairs, but does not categorized into three groups: provide a clear mechanism for interpreting these results • Vector-based similarity, such as cosine similar- at the project level. Even if all pairwise similarities are ity, where birthmarks are transformed into feature computed, it remains unclear how to determine whether vectors and compared based on their geometric sim- two projects are related as a whole. Second, in realistic scenarios, a developer or analyst is ilarity. typically given a target project P and must search for po• Set-based similarity, such as the Jaccard coeffi- tentially related projects Q among a vast collection of cancient, Dice index, and Simpson index, which mea- didates. Without a project-level similarity measure, there sure the overlap between sets of birthmark elements. is no principled way to rank or filter candidate projects • Sequence-based similarity, such as edit dis- for further inspection. Third, software reuse is often partial. A project may tance, which compares ordered sequences of birthreuse only a subset of modules from another project, while marks and captures structural differences between many other modules remain unrelated. This dilutes the them. signal of reuse when considering all module pairs uniHowever, the described functions would be applica- formly. ble only for instruction-based and API-based birthmarks, Finally, small or generic modules may exhibit inciwhile structure-based would require different similarity dental similarity, leading to false positives [51]. Such computation approach, due to their usual representation noise can dominate the similarity signal if all modules as graphs [11]. are treated equally. These similarity functions exhibit different characterThese challenges indicate that module-wise similarity istics. Vector-based methods capture global distribu- is insufficient as a standalone approach. Instead, it is tional patterns, set-based methods emphasize shared el- necessary to define a project-wise similarity function: ements, and sequence-based methods preserve ordering
Sim(P, Q) = F {sim(pi , qj )} , (8) information. The choice of similarity function therefore affects the sensitivity to different types of code modifica- which aggregates module-wise similarities into a sintions. gle project-level score, enabling direct comparison of In this study, we employ representative similarity projects. functions from each category to ensure a comprehensive The design of such a function must address the folevaluation (the list of employed functions will be provided lowing requirements: in Section 5.5). However, the focus of this paper is not • Robustness to partial reuse, where only a subset of on the choice of module-level similarity function itself, modules is reused; but on how such similarities should be aggregated at the project level. • Robustness to incidental similarity arising from Finally, it is important to note that module-wise simsmall or generic modules; ilarity alone is insufficient for large-scale plagiarism de• The ability to provide meaningful project-level tection. While it can identify reuse between individual scores that support ranking and filtering of candimodules, it does not directly provide a mechanism to comdate projects in large-scale software ecosystems. pare entire software projects. This limitation motivates the need for project-wise similarity measures, which will However, as we discuss next, existing approaches do be discussed in the next section. not fully satisfy these requirements.

2.4

Birthmark Similarity Calculation

4

3.2

Limitations of Existing Project-wise Summary of limitations Overall, existing projectwise similarity methods do not fully satisfy the requireSimilarity Methods

ments identified in Section 3.1. In particular, they do not Several approaches have been proposed to compute sim- adequately address: ilarity between software projects. In this section, we re• Partial reuse, where only a subset of modules are view representative methods and discuss their limitations relevant; with respect to the requirements identified in Section 3.1. • Incidental similarity caused by small or generic modules, which should not dominate the overall Baseline method (Lee et al.) Lee et al. [11] proposed similarity score. a project similarity measure based on procedure-wise (due These limitations motivate the design of a new to the work focusing on the API-based birthmark) simi- project-wise similarity framework, which we present in larities. We adapt this method by using modules in place the next section. of procedures. For each module in project P , the maximum similarity to modules in Q is computed, and the results are aggregated as follows: 4 Proposed Framework
Pn 2 × i=1 max {sim(pi , qj ) | j = 1, . . . , m} To address the limitations discussed in Section 3, we pro,pose a project-wise similarity framework based on two Simp (P, Q) = n+m (9) key ideas: (1) weighting module-wise similarities to rewhere n and m are the number of modules in P and Q duce the impact of incidental matches, and (2) focusing respectively. on the most relevant subset of module pairs to capture This method is simple but not symmetric, i.e., partial reuse. Simp (P, Q) ̸= Simp (Q, P ), which leads to inconsistent results depending on the order of the inputs. 4.1 Symmetric Aggregation As a preliminary step, we define a simple symmetric agAggregated similarity To address this limitation, our gregation method of module-wise similarities, referred to previous work [14] introduced aggregated similarity, which as Symmetric Aggregation (SA), which serves as the base computes a symmetric project-level score by considering of our framework. For a module x and a project Y , let high-similarity module pairs from both projects. Formally, let T opN (pi , Q) denote the set of the topT op1(x, Y ) = max sim(x, y) (12) y∈Y N highest similarity values between module pi and all denote the highest similarity between x and all modules modules in Q. Aggregated similarity is defined as: in Y . P U Then, the symmetric aggregation (SA) similarity is Simagg (P, Q) = , (10) defined as: |U | where U is the union of all selected similarity values:

Pn SimSA (P, Q) =

U = {T opN (pi , Q)} ∪ {T opN (qj , P )},

∀pi ∈ P, ∀qj ∈ Q (11) By incorporating the parameter N , this method considers multiple high-similarity matches per module, which, in theory, would help to reduce the impact of spurious matches, providing a more stable aggregation, compared to using only a single maximum, while also addressing cases where original module is plagiarized by several modules (i.e., parts of the original being split into many modules to avoid detection). However, the previous study was conducted only for N = 1, therefore, the effectiveness of higher values of N is yet to be tested [14]. On the other hand, despite these improvements, aggregated similarity still treats all selected module pairs uniformly. As a result, it does not explicitly account for differences in module importance or size, and remains sensitive to incidental similarity arising from small or generic modules. Furthermore, it does not explicitly address partial reuse, where only a subset of modules should dominate the similarity score.

i=1 T op1(pi , Q) +

Pm

n+m

j=1 T op1(qj , P )

,

(13) which can also be represented using baseline method equation 9: Simp (P, Q) + Simp (Q, P ) . (14) 2 This aggregation ensures that project-level similarity is independent of the order of inputs, and corresponds to the case where no weighting is applied and all module pairs are considered. SimSA (P, Q) =

4.2

Weight Assignment

Not all modules contribute equally to plagiarism detection. In particular, smaller modules are more likely to exhibit incidental similarity [51]. To mitigate this effect, we assign weights to modules based on the size of their birthmarks. Let K denote the number of k-grams in a module. A naive approach would assign weights proportional to K. However, in projects containing very large modules, 5

such a linear weighting would cause a small number of 5 Experiment Design modules to dominate the overall similarity score, potentially overshadowing the contributions of the remaining 5.1 Overview and Objectives modules. To mitigate this effect, we apply a logarithmic trans- The objective of this experiment is to evaluate the effectiveness of the proposed project-wise similarity frameformation: W (B(p)) = ln(K). (15) work from three perspectives: This transformation compresses the range of weights, reducing the dominance of extremely large modules while still preserving the relative importance of module size. We note that the choice of the weight assignment function is not the primary √ focus of this study. While alternative formulations (e.g., K or linear scaling) are possible, our goal is to demonstrate the effectiveness of incorporating size-based weighting, rather than to optimize the specific functional form. To normalize weights within a pair of projects P and Q, we define:

• Whether project-wise similarity can reliably distinguish reused and non-reused project pairs; • Whether size-based weighting reduces false positives caused by small or generic modules; • Whether partial similarity improves detection of partial code reuse.

To achieve this, we design an experiment consisting of the following steps: W (B(p))
. Wnorm (p, P, Q) = max {W (B(pi ))} ∪ {W (B(qj ))} • Dataset preparation and filtering; (16) Finally, the weighted similarity between two modules • Module-wise similarity computation; is defined as: simweighted (pi , qj , P, Q) =

• Project-wise similarity calculation using multiple methods;

sim(pi , qj ) × min Wnorm (pi , P, Q), Wnorm (qj , P, Q) . (17)

• Evaluation using threshold-based classification and harmonic mean (Hmean).

This formulation reduces the influence of smaller modules, which are more prone to incidental similarity.

4.3

Partial Similarity

5.2

Even with weighting, aggregating all module pairs may dilute the similarity result, especially when only a subset of modules is reused. To address this, we propose partial similarity, an extension of SA, which focuses only on the most relevant subsets of module pairs. We define T op1weighted (x, Y ) as modification of T op1(x, Y ) that employs simweighted (x, y, X, Y ) in place of sim(x, y). Let S(X, Y ) denote sets of the highest weighted similarity results for each module in project X:

Dataset

For this study, we constructed a custom dataset consisting of 35 Java projects across 10 categories (Appendix A (Table 5)). Projects are grouped into categories to ensure that non-reused pairs share similar functionality, making the credibility evaluation more challenging and realistic. The categorization was conducted manually based on project tags and descriptions provided in their GitHub repositories. Different versions of the same project are treated as reused pairs, simulating realistic scenarios in which reused S(X, Y ) = {T op1weighted (x, Y )|∀x ∈ X}, (18) code is modified, extended, or integrated into new conWe sort similarity results in S in descending order and texts. select the top α% of the highest values: The projects were selected from GitHub according to the
Sα (X, Y ) = Top-α% S(X, Y ) . (19) following criteria: The partial similarity is then defined as: P P • Primary language (Java). Java must be the Sα (P, Q) + Sα (Q, P ) dominant language in the project. Projects that Simpartial (P, Q, α) = , (20) |Sα (P, Q)| + |Sα (Q, P )| combine Java with other languages are accepted, but Java must be listed as the primary language on where α is the parameter that controls the comparison the repository page. scope. By focusing on high-similarity module pairs, this method emphasizes reused components while ignoring un• Minimum project size. Each project (and each related parts of the projects. selected version) must contain approximately 50 or In this study, we evaluate α ∈ {1, 5, 10, 25, 50, 75}. For more Java class files to ensure sufficient structural α = 100, the method is identical to SA. complexity. 6

• Project popularity. Only projects with at least 50 stars on GitHub were included, to focus on relatively well-maintained and non-trivial software systems.

adopt a modified version of the filtering method used in a past study [16]. In the original approach, modules with fewer than 30 lines of source code are removed. However, raw line counts may include empty lines, comments, and brackets, which do not reflect the actual complexity of • Version availability. Each project must provide the code. at least four released versions. These versions are To address this limitation, we employ logical line count used to evaluate resilience by treating newer ver- (LLC), which excludes such non-functional lines. Modsions as modified derivatives of earlier ones. The ules with LLC ≤ 30 are removed from the dataset. process of version selection is described in ApIt is worth noting that this filtering step may appear pendix A. redundant given the use of size-based weighting in our Compiled .jar files were obtained either from the project’s proposed method. However, LLC-based filtering remains official release page or from the Maven repository4 when necessary for two reasons. First, removing small modules is not an uncommon releases were not directly available. practice in software birthmark research [16]. Such modFor each project version, software birthmarks were ex- ules are typically generic and contribute little to distracted using the pochi tool. Subsequently, module-wise tinguishing program characteristics, while disproportionsimilarity was computed for all relevant module pairs us- ately increasing the number of incidental matches. Thereing the similarity functions which will be described in fore, filtering them out improves the signal-to-noise ratio Section 5.5. and reduces computational cost without sacrificing meanOverall, this dataset enables systematic evaluation of ingful information. Second, our experimental evaluation aims to provide a both resilience (via intra-project version comparisons) fair comparison between the proposed weighting method and credibility (via inter-project comparisons within the and conventional similarity methods. If LLC-based filtersame category), providing a realistic and controlled enviing were omitted, the conventional methods would be disronment for assessing project-wise similarity methods. proportionately affected by small-module noise because To facilitate reproducibility and future research, the they do not explicitly address the influence of small and 5 dataset used in this study was made publicly available . generic modules. As a result, the comparison could unfairly favor the proposed method simply due to differences 5.3 Dataset Filtering Methods in noise handling. Applying the same filtering condition Using .jar files as-is is inappropriate for similarity anal- across all methods therefore provides a more balanced ysis, as they often contain external libraries and small and reliable evaluation framework. generic modules that do not contribute meaningful information for plagiarism detection. Such modules may introduce noise and inflate similarity scores between unrelated projects. To address these issues, we apply two filtering strategies.

In Java, nested classes may be compiled into separate class files depending on compiler settings [17,18]. To keep the filtering process simple, nested class files are removed only if their corresponding parent class is also filtered out.

The effect of the filtering process is summarized in Table 1. Overall, approximately 82.5% of modules are reExternal module filtering. While some projects pro- moved. vide dependency-free .jar files, most include external libraries. We categorize modules into internal (compiled To further validate the filtering criterion, we analyze the from the project’s own source code) and external (third- relationship between LLC and bytecode size. Table 2 party libraries and unrelated files). Since external mod- shows that one logical line corresponds to approximately ules are not indicative of code reuse between the inspected 2–3 bytecode instructions on average, which were exprojects, including them would introduce irrelevant com- tracted using javap 9 . This suggests that LLC serves as a reasonable proxy for code size. parisons (e.g., internal–external or external–external). Modules are classified using build configuration files Although a large portion of modules is removed, this filsuch as pom.xml 6 (Maven), build.xml 7 (Ant), and tering process improves the overall reliability of similar8 build.gradle (Gradle). Based on this information, all exity computation by eliminating irrelevant and misleading ternal modules are removed prior to similarity computacomparisons. In cases where source code is unavailable, tion. filtering based on approximate instruction counts (e.g., Logical line count (LLC)-based filtering. To reduce corresponding to LLC thresholds) could be used as an false positives caused by small and generic modules, we alternative in future work. 4 https://repo1.maven.org/ 5 The dataset is released at: https://gitlab.com/ou-salab-fedorov/project-wise_comparison_of_software_birthmarks. 6 https://maven.apache.org/pom.html 7 https://ant.apache.org/manual/using.html 8 https://docs.gradle.org/current/userguide/build_file_basics.html 9 https://docs.oracle.com/javase/9/tools/javap.htm

7

Table 1: External module filtering results.

Class file count

Total (before filtering)

After external module filtering

After LLC (30) filtering

198906

57780

34855

Table 2: Average bytecode instruction counts per single LLC.

5.4

Subset

Average instruction count per 1 LLC

Total LLC ≤ 30 LLC > 30

2.707 2.042 3.33

Compared Methods

The proposed method, which integrates both sizebased weights and partial similarity. First, modulewise similarities are adjusted using weights derived from module size. Then, only the top percentage of similarity values (denoted as comparison scope) is selected from both projects, and their overall average is used as the project-wise similarity.

To evaluate the effectiveness of the proposed approach, we compare it with several baseline and variant methods. Each method is designed to highlight different aspects of project-wise similarity computation. • Baseline method [11]

In this study, we evaluate scope ∈ {1, 5, 10, 25, 50, 75} to analyze the effect of focusing on different subsets of high-similarity module pairs.

The method proposed by Lee et al. [11], which computes project similarity based on the average of maximum module-wise similarities. This method is inherently asymmetric, i.e., Simp (P, Q) ̸= Simp (Q, P ). To mitigate this issue in our experiments, we randomly select either Simp (P, Q) or Simp (Q, P ) and repeat the process 10 times, reporting the average result.

• Random similarity A baseline method in which project similarity scores are assigned randomly over the course of 100 iterations and then rounded to either 0 or 1. This serves as a lower bound for performance, representing the expected outcome without any meaningful similarity computation.

• Aggregated similarity [14]

The method proposed in our previous work [14], These methods collectively enable a systematic evalwhich aggregates the top-N module-wise similariuation of the contributions of symmetry, weighting, and ties from both projects. In this study, we evalupartial selection in project-wise similarity computation. ate two configurations: N = 1 and N = 2, to examine the effect of considering multiple candidate 5.5 Module-wise Similarity Functions matches. Project-wise similarity is computed by aggregating similarities between matched module pairs. Therefore, the choice of module-wise similarity function can directly afThe base aggregation method used in the proposed fect the final results. framework, defined as the average of the highest In this study, we consider the following representative module-wise similarity results between two projects, similarity functions: which also equates to the the average of Simp (P, Q) and Simp (Q, P ). This method addresses the asym• Cosine similarity (count vectorization); metry issue of the baseline method without intro• Cosine similarity (TF-IDF vectorization); ducing weighting or partial selection. It corresponds to the unweighted, full-scope version of our pro• Dice index; posed framework and serves as a reference point for evaluating the contributions of weights and partial • Jaccard similarity coefficient; similarity. • Simpson similarity index;

• Symmetric aggregation (SA)

• Weighted partial similarity

• Edit distance.

10 https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html 11 https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html

8

For cosine similarity, birthmarks are vectorized using two commonly used approaches (count and TF-IDF vectorization) implemented in scikit-learn: CountVectorizer10 and TfidfVectorizer11 . These similarity functions are treated as experimental factors, and their impact on project-wise similarity is analyzed in Section 6.5.

5.6

Threshold selection The performance of a similarity measure depends on the choice of threshold E. In this study, rather than fixing a predefined threshold, we determine the optimal threshold for each project category. Specifically, for each category, we perform an exhaustive search over E ∈ (0.0, 1.0) with a step size of 0.001, and select the value that maximizes the harmonic mean of resilience and credibility:

Evaluation Metrics

2 . (25) Hmeancategory = 1 1 + To evaluate the effectiveness of project-wise similarity resilience rate credibility rate measures, we extend the concepts of resilience and credThis procedure is intended to evaluate the relative efibility, originally defined for module-level birthmarks, to fectiveness of similarity measures under their best achievthe project level. able conditions, rather than to simulate a deployment scenario where the threshold must be fixed in advance. Due Project-wise similarity result evaluation Given a to the limited number of projects per category, splitting project-wise similarity score Sim(P, Q), we determine the dataset into separate training and evaluation subsets whether two projects are considered similar using a is not feasible without significantly reducing statistical threshold E: reliability. Therefore, the reported results should be interpreted as an upper-bound comparison of the methods, ( ≤ E not similar, focusing on their intrinsic capability to balance resilience (21) Sim(P, Q) = and credibility. > E similar. Overall evaluation Using the optimal threshold for each category, we compute the final performance across all categories using the macro-level harmonic mean:

Evaluation sets To evaluate different aspects of performance, we define two sets of project pairs: • Reused pairs (Preused ): Pairs of different versions of the same project. These simulate realistic reuse scenarios where code is modified and extended.

Hmean = P

k i=1

• Non-reused pairs (Pnot reused ): Pairs of different projects within the same category. These pairs share similar functionality but are independently developed, making the task of distinguishing them more challenging.

Pnot reused = T N + F P,

resilience rate =

(22)

We define two evaluation

TP , TP + FN

 , (26)

Rationale We adopt the harmonic mean because it penalizes imbalanced performance: a method that achieves high resilience but low credibility (or vice versa) will receive a low score. This is particularly important in software reuse detection, where both avoiding missed detections (FN) and preventing false alarms (FP) are critical. Compared to metrics such as F1-score, which focuses on a single classification setting, the proposed Hmean explicitly balances the two complementary objectives of reuse detection: robustness to modification and discrimination between independent projects.

where T P , F N , T N , and F P denote true positives, false negatives, true negatives, and false positives, respectively. Resilience and credibility metrics:

1 1 resilience ratei + credibility ratei

where k is the number of categories.

Each set is further decomposed as: Preused = T P + F N,

2k 

(23)

6

Experimental Results

6.1

Overall Comparison of Methods

Fig. 1 presents the overall comparison of project-wise similarity methods in terms of Hmean. The results are aggregated over all configurations of k-gram sizes and TN credibility rate = . (24) module-wise similarity functions to ensure a fair comTN + FP parison without favoring any specific parameter setting. The resilience rate measures the ability to correctly For the method incorporating partial similarity, the bestidentify reused projects despite modifications, while the performing scope (identified in Section 6.3) is used. credibility rate measures the ability to avoid false posiThe results show a clear ranking of methods. The proposed method (weighting + partial similarity, scope tives among independently developed projects. 9

1.0 0.9 0.8 0.7 0.6 0.5

om nd

se Ba

Ra

li

ne

2) ed Ag

gr

eg

at Ag

gr

eg

at

ed

(N

= (N

fu , ht ig we

o SA

(n

=

) ll

) 1% l, ia rt pa + ht ig Pr

op

os

ed

(S

A

+

we

1)

0.4

Figure 1: Overall comparison of project-wise similarity methods in terms of Hmean. For the proposed method, the best scope (1%) is used, while results are aggregated over all k-gram sizes and module-wise similarity functions to ensure a fair comparison. = 1%) achieves the highest performance, with a noticeable margin over all other methods. In addition to its superior median Hmean, it also exhibits relatively low variance, indicating stable performance across different configurations (Table 3). The symmetric aggregation method without weighting or partial similarity (SA (no weight)) performs consistently better than the existing approaches, including aggregated similarity and the baseline method. This result suggests that the symmetric formulation itself provides a strong foundation for project-wise similarity, even without additional enhancements. Although SA (no weight) is conceptually similar to aggregated similarity with N = 1, it is simpler in formulation while achieving comparable or better performance, making it a suitable baseline for further extensions. Comparing the aggregated similarity variants, N = 1 outperforms N = 2. This indicates that introducing the parameter N to consider cases where the original module is plagiarized by multiple modules and to lower the impact of false positives, does not necessarily improve overall performance in this setting. A possible explanation is that higher values of N might also increase false negatives when trying to reduce the false positives, resulting in lower overall Hmean. Since this method is not the focus of this study, we do not investigate this effect further. The baseline method shows lower performance and larger variance compared to both SA and aggregated similarity. This can be attributed to its asymmetric nature, which introduces instability even when averaging over multiple runs. As expected, the random similarity method performs significantly worse than all other methods, confirming that the evaluation framework is meaningful. We conduct Wilcoxon signed-rank test using the results from Proposed (weight + partial, 1%) and Aggregated (N = 1) methods. p = 3.0496 × 10−5 < 0.01, thus,

the difference in performance is statistically significant. Overall, these results demonstrate that the proposed method clearly outperforms existing approaches, and that the symmetric aggregation formulation provides a strong and effective basis for project-wise similarity.

6.2

Effect of Weighting

Fig. 2 shows the effect of weighting on the proposed method (partial similarity, scope = 1%) in terms of Hmean. 1.0 0.8 0.6 0.4 0.2 0.0

10

Proposed (with weight)

Proposed (without weight)

Figure 2: Effect of weighting on the proposed method (partial similarity, scope = 1%) in terms of Hmean. The results demonstrate that weights has a substantial impact on performance. When weights are applied, the method achieves significantly higher Hmean values with relatively low variance, indicating both strong accuracy and stability across configurations. In contrast, without weighting, the performance drops dramatically and exhibits very large variability. This improvement is statistically significant based on the Wilcoxon signed-rank test (p = 1.6766 × 10−7 < 0.01). Notably, the median performance without weight assignment is even lower than that of the random similarity method shown in Fig. 1. This result is particularly striking, given that the SA without weighting (Fig. 1) performed as the second-best method. This indicates that

Table 3: Average, median Hmean and variance per project-wise similarity method. Method

Avg. Hmean

Median Hmean

Variance

Proposed (weight + partial, 1%) SA (no weight) Aggregated (N = 1) Aggregated (N = 2) Baseline Random

0.9497 ± 0.0695 0.8968 ± 0.0929 0.8779 ± 0.0896 0.8527 ± 0.0837 0.8145 ± 0.1105 0.4707 ± 0.0288

0.9728 0.94 0.9157 0.8875 0.8645 0.4757

0.0048 0.0086 0.008 0.007 0.0122 0.0008

combining partial similarity (scope = 1%) with an unweighted formulation leads to a severe degradation in performance. A plausible explanation is that, under partial similarity, only the top fraction of module pairs is considered. Without weights, these top-ranked pairs are often dominated by small modules, which tend to exhibit incidental similarity even when no actual code reuse exists. As a result, false positives increase significantly, leading to unreliable project-wise similarity scores. In contrast, the weighting scheme suppresses the influence of small modules by assigning them lower importance. This prevents such modules from dominating the top-ranked pairs, thereby reducing false positives and stabilizing the results. These findings suggest that partial similarity alone is insufficient and may even be harmful without an appropriate mechanism for assignment of weights. Instead, weighting and partial similarity should be considered as complementary components that jointly enable robust project-wise similarity estimation.

0.9

0.8

0.7

0.6

SA (with weight)

effect becomes substantially more pronounced, as previously observed.

6.3

Effect of Partial Similarity

Fig. 4 illustrates the effect of the comparison scope on the performance of partial similarity with weighting. The comparison scope controls the proportion of top similarity values used in the aggregation. The results indicate that partial similarity improves performance compared to using all similarity values (i.e., scope = 100%). Smaller scope values generally lead to higher Hmean, suggesting that focusing on the most similar module pairs helps emphasize meaningful reuse signals. Small scope values (e.g., 1% and 5%) consistently achieve the highest performance, with 1% often yielding the best results. While larger scope values (10–75%) produce similar performance levels, their improvements over the full-scope setting are less pronounced. We also conduct a Wilcoxon signed-rank test comparing the best-performing scope value (i.e., scope = 1%) with the full-scope setting (scope = 100%). The results show statistically significant improvements (p = 0.0095 < 0.01), supporting the effectiveness of partial similarity. These results suggest that partial similarity serves as an effective complementary mechanism to weighting. Also, focusing on a very small subset of highly similar module pairs is effective for capturing reuse signals.

SA (without weight)

6.4

Figure 3: Effect of weighting on the SA in terms of Hmean.

Effect of k-gram Size

Fig. 5 shows the impact of different k-gram sizes on the To isolate the effect of weighting from that of partial performance of the proposed method. The results indicate that performance improves as k similarity, we additionally evaluate the proposed method increases from 1 to 4, after which the improvement satuunder the condition of full scope (i.e., scope = 100%), rates. This behavior suggests that smaller values of k do which is equal to SA. The results, shown in Fig. 3, indinot capture sufficient structural information, while larger cate that weight assignment consistently improves pervalues provide diminishing returns. formance even without partial similarity. Specifically, Based on these observations, values k ∈ {3, 4} appear the weighted variant achieves higher Hmean values and to be a practical choice for achieving a good balance beexhibits slightly lower variance compared to the nontween performance and computational cost. weighted version. The difference is also statistically significant according to the Wilcoxon signed-rank test (p = 0.0121 < 0.05): 6.5 Effect of Similarity Function These findings confirm that weighting alone contributes positively to performance by reducing the influ- Fig. 6 presents the comparison of module-wise similarity ence of incidental similarities from small modules. At functions for the proposed method (weighting + partial the same time, when combined with partial similarity, its similarity, scope = 1%), evaluated in terms of Hmean. 11

1.00 0.95 0.90 0.85 0.80 0.75 0.70 0.65

A)

) rt

Fu

ia

l

ll

(7

(S

5%

) Pa

Pa

rt

ia

l

(5

0%

) (2 rt

Pa

Pa

rt

ia

l ia

l

(1

5%

) 0%

%) (5 l ia rt Pa

Pa

rt

ia

l

(1

%)

0.60

Figure 4: Effect of comparison scope (partial similarity ratio) on the proposed method (partial similarity with weighting), including the full-scope case (100%) (SA), evaluated using Hmean. 1.00 0.95 0.90 0.85 0.80 0.75 0.70 0.65 0.60 1-gram

2-gram

3-gram

4-gram

5-gram

6-gram

Figure 5: Effect of k-gram size (k ∈ {1, . . . , 6}) on the proposed method (weighting + partial similarity, scope = 1%), evaluated in terms of Hmean. 1.00 0.95 0.90 0.85 0.80 0.75 0.70 0.65

x

t

de in n so mp Si

cc

ar

d

Ed

it

co

ef

di

fi

st

ci

an

en

ce

x de in ce Di

ne

Ja

si Co

Co

si

ne

(T

(c

F-

ou

ID

nt

F

ve

ve

ct

ct

.)

.)

0.60

Figure 6: Comparison of module-wise similarity functions for proposed method (weighting + partial similarity, scope = 1%), evaluated in terms of Hmean.

12

The results clearly show that edit distance achieves the highest performance among all evaluated similarity functions. In addition to its superior median Hmean, edit distance also exhibits remarkably low variance, indicating stable performance across different configurations. The Jaccard coefficient provides the second-best performance, with relatively high Hmean values and moderate variability. In contrast, the remaining similarity functions show both lower performance and substantially higher variance, suggesting sensitivity to parameter settings and reduced robustness. In particular, the Simpson index performs the worst among all evaluated functions. It not only yields lower Hmean values but also demonstrates very large variability, indicating unstable and unreliable behavior in this context. Overall, these results suggest that edit distance is the most suitable similarity function for module-wise comparison in the proposed framework, providing both high accuracy and strong robustness.

6.6

Ranking-Based Analysis Across Configurations

Overall, the results confirm that the proposed framework maintains both effectiveness and stability across heterogeneous software categories.

6.8

Summary of Findings

The experimental results can be summarized as follows. First, the proposed symmetric aggregation (SA) achieves better performance compared to the existing methods, even without additional enhancements (weight assignment and partial similarity), indicating that the symmetric formulation itself provides a strong baseline. Second, weight assignment plays a crucial role in improving performance by reducing the impact of incidental similarities, particularly those arising from small modules. While weighting alone already leads to measurable improvements, its effect becomes substantially more pronounced when combined with partial similarity. Third, partial similarity further enhances the results by focusing on the most relevant subset of module pairs. However, its effectiveness strongly depends on the presence of weighting; without it, partial selection may amplify noise and degrade performance. This indicates that weight assignment and partial similarity are complementary mechanisms that jointly contribute to robust similarity estimation. Finally, the proposed method demonstrates robustness across different parameter settings and project categories, consistently achieving high performance with low variance. These findings confirm the effectiveness and practical applicability of the proposed project-wise similarity framework.

Table 4 presents the top-ranked configurations based on Hmean across all parameter combinations. The ranking results reveal that the highest-performing configurations are consistently dominated by the proposed method (SA + weighting + partial similarity). In particular, configurations with small comparison scopes (e.g., 1% and 5%) frequently appear at the top of the ranking. Additionally, edit distance is the most commonly used similarity function among the top-ranked configurations. Notably, non-SA methods do not appear in the top 30 rankings, indicating that the superiority of the proposed 7 Threats to Validity framework is not limited to specific parameter choices. These results demonstrate that the proposed method This section discusses potential limitations of the study is robust across a wide range of parameter configurations. and their possible impact on the interpretation and generalization of the results.

6.7

Per-category Analysis

Fig. 7 presents the performance of the proposed method across different project categories. The analysis focuses on configurations that achieved strong performance in previous sections, namely k ∈ {4, 5, 6} and scope ∈ {1%, 5%}, using edit distance as the module-wise similarity function. The results show that the proposed method consistently achieves high Hmean values across all categories, indicating strong overall performance. While some variation can be observed depending on the category, the differences are relatively moderate and no category exhibits a significant degradation in performance. These observations suggest that the proposed method is robust to differences in application domains and does not rely on category-specific characteristics. In particular, the consistently high performance across diverse categories supports the general applicability of the method for project-wise similarity estimation.

7.1

13

Dataset-related Threats

• Assumption of reuse and non-reuse pairs: In this study, different versions of the same project are treated as reused pairs, while different projects within the same category are treated as nonreused pairs. This design provides a practical and reproducible approximation of project-level software reuse scenarios, enabling systematic evaluation across many configurations. However, the dataset does not contain confirmed real-world plagiarism cases or adversarially modified software. Therefore, the evaluated reuse patterns may not fully represent all forms of unauthorized code reuse observed in practice. Constructing and evaluating datasets based on real-world plagiarism cases remains an important direction for future work.

Table 4: Top-30 configurations ranked by Hmean across all combinations of methods and parameter settings, including k-gram size, comparison scope, and module-wise similarity function. Rank

Method

Birthmark

Sim. function

Hmean

1 2 3 3 3 4 4 4 4 4 5 6 7 8 9 10 11 12 13 13 14 15 16 16 17 18 19 20 20 20 21 22 22 23 24 25 26 27 28 29 30 30

Partial (5%) Partial (1%) Partial (5%) Partial (5%) Partial (5%) Partial (1%) Partial (1%) Partial (1%) Partial (1%) Partial (1%) Partial (1%) Partial (5%) Partial (25%) Partial (10%) Partial (1%) Partial (5%) Partial (25%) Partial (25%) Partial (25%) Partial (25%) Partial (10%) Partial (10%) Partial (25%) Partial (25%) Partial (1%) Partial (50%) Partial (10%) Partial (5%) Partial (1%) Partial (1%) Partial (5%) Partial (5%) Partial (25%) Partial (25%) Partial (50%) Partial (50%) Partial (1%) Partial (1%) Partial (50%) Partial (25%) Partial (50%) Partial (75%)

5-gram 1-gram 4-gram 6-gram 3-gram 5-gram 2-gram 3-gram 6-gram 4-gram 2-gram 1-gram 3-gram 1-gram 3-gram 2-gram 5-gram 1-gram 3-gram 2-gram 2-gram 2-gram 4-gram 4-gram 4-gram 3-gram 3-gram 3-gram 6-gram 5-gram 4-gram 2-gram 6-gram 6-gram 5-gram 2-gram 5-gram 4-gram 4-gram 6-gram 4-gram 4-gram

Edit distance Edit distance Edit distance Edit distance Edit distance Edit distance Edit distance Edit distance Edit distance Edit distance Jaccard coefficient Edit distance Edit distance Edit distance Jaccard coefficient Edit distance Edit distance Edit distance Jaccard coefficient Edit distance Jaccard coefficient Edit distance Edit distance Jaccard coefficient Jaccard coefficient Edit distance Edit distance Jaccard coefficient Jaccard coefficient Jaccard coefficient Jaccard coefficient Jaccard coefficient Jaccard coefficient Edit distance Edit distance Edit distance Cosine (TF-IDF vect.) Cosine (TF-IDF vect.) Jaccard coefficient Dice index Edit distance Jaccard coefficient

0.9823 0.982 0.982 0.982 0.982 0.9819 0.9819 0.9819 0.9819 0.9819 0.9819 0.9816 0.9802 0.9801 0.9799 0.9799 0.9794 0.9793 0.9787 0.9787 0.9786 0.9784 0.9773 0.9773 0.9773 0.9771 0.977 0.9769 0.9769 0.9769 0.9769 0.9766 0.9766 0.9765 0.9765 0.9761 0.976 0.976 0.976 0.9759 0.9758 0.9758

1.0 0.9 0.8 0.7 0.6

k am fr g in st

al

Un

it

te

in rm Te

or ew

it

io

ed

at

xt

pl ap

Te

ic

ne gi en e rs ve

Re

or

n

g er

ay

in

er

n pl a di Me

in M LL

Im

ag

e

pr

te

oc

gr

es

at

si

io

ng

or at ul

Em

ma ta Da

CL

I

fr

na

am

ge

ew

me

or

k

nt

0.5

Figure 7: Per-category performance of the proposed method (weighting + partial similarity) in terms of Hmean. Results are shown for configurations with k ∈ {4, 5, 6} and scope ∈ {1%, 5%} using edit distance similarity function.

14

• Dataset construction and selection: The dataset consists of Java open-source projects selected from GitHub repositories under several filtering conditions, including project size, availability of executable artifacts, and category alignment. Although these conditions were necessary to enable systematic comparison, they may introduce biases that do not fully reflect the diversity of real-world software ecosystems.

the findings may not directly generalize to other programming languages, software ecosystems, or alternative birthmark representations. • Scope of reuse scenarios: The study focuses on project-wise similarity under partial reuse scenarios, where reused modules are embedded within larger software systems. The experiments do not explicitly evaluate robustness against strong obfuscation or adversarial code transformations. Such scenarios are outside the scope of the current study and should be investigated separately in future work.

In addition, the dataset mainly contains established projects with over 4 or more official releases across the lifetime and a minimum of 50 stars on GitHub. The observed similarity characteristics may there- Despite these limitations, the study provides a comprefore differ from those of smaller, proprietary, or less hensive empirical evaluation across diverse datasets, paactively maintained systems. rameter configurations, and similarity formulations, offering useful insights into project-wise similarity measure• Temporal diversity of project versions: The ment for software birthmarks. analyzed project versions span a long time period, from 2002 to 2025. During this period, software development practices, frameworks, and tooling have 8 Conclusion significantly evolved. Such temporal differences may affect project structure and similarity charac- In this paper, we addressed the need for project-wise simteristics across versions. ilarity in software birthmark comparison, which serves

7.2

Evaluation-related Threats

• Threshold-based evaluation: The proposed methods are evaluated using threshold-based binary classification derived from project-wise similarity scores. The threshold for each project category is selected to maximize the harmonic mean of resilience and credibility. This evaluation strategy is intended to compare the relative effectiveness of similarity measures under optimized conditions, rather than to estimate directly deployable real-world detection accuracy. Because the dataset size is limited, the study does not perform separate threshold training and testing procedures. Therefore, the reported results should be interpreted primarily as comparative performance estimates between methods. In addition, alternative threshold-independent evaluation strategies, such as ranking-based metrics or ROC/AUC analysis, were not investigated in this study and remain part of future work. • Parameter sensitivity: The performance of project-wise similarity depends on several parameters, including the k-gram size, module filtering conditions, weighting scheme, and partial similarity scope. Although extensive experiments across many parameter configurations were conducted, different parameter ranges or alternative formulations may lead to different performance characteristics.

7.3

Generalization-related Threats

• Programming language and birthmark scope: The evaluation focuses exclusively on Java software projects and static k-gram birthmarks. Therefore, 15

as a retrieval step for identifying candidate projects that may contain reused or plagiarized code prior to detailed analysis. We proposed a project-wise similarity framework based on symmetric aggregation of module-level similarities, and introduced two complementary mechanisms: size-based weight assignment and partial similarity. While the symmetric aggregation provides a simple and stable foundation, our focus was on improving robustness against two key challenges in project-level comparison: incidental similarity from small modules and partial code reuse. Through extensive experiments, we obtained several important findings. First, symmetric aggregation alone already provides a strong and reliable baseline, performing competitively with or better than existing approaches. Second, weighting plays a critical role in improving performance by suppressing the influence of small, noiseprone modules. Third, partial similarity further enhances detection by focusing on the most relevant subset of module pairs; however, its effectiveness strongly depends on the presence of weights. Without weighting, partial selection may amplify noise and degrade performance, whereas their combination yields substantial improvements. These results indicate that weight assignment and partial similarity are complementary mechanisms that jointly enable robust project-wise similarity estimation. In addition, we observed that configurations with small scopes (e.g., 1%, 5%), higher k-gram sizes, and edit distance similarity function tend to provide stable and high performance across diverse project categories. Overall, the results demonstrate that, in the inspected context, in the absence of obfuscation (and other transformations) in the compared projects, effective project-wise similarity does not require complex matching strategies. Instead, a simple combination of symmetric aggregation,

size-aware weights, and selective focus on high-similarity [5] S. Romansky, C. Chen, B. Malhotra, and A. Hindle, module pairs provides a robust and practical solution. “Sourcerer’s Apprentice and the study of code snippet migration”. arXiv:1808.00106 [cs.SE], 2018. As future work, we plan to extend the evaluation to other programming languages and larger-scale datasets, investigate robustness against obfuscation and adversarial [6] Y. Golubev, M. Eliseeva, N. Povarov, and T. Bryksin, “A Study of Potential Code Borrowing and transformations, and explore alternative weighting strateLicense Violations in Java Projects on GitHub”. gies and similarity representations, as well as expanding arXiv:2002.05237 [cs.SE], 2020. to other types of birthmarks. [7] H. Tamada, M. Nakamura, A. Monden, and K. Matsumoto, “Detecting the Theft of Programs UsA Overview of the Used Opening Birthmarks”, Information Science Technical Report, NAIST-IS-TR2003014, ISSN 0919-9527, Gradsource Software Projects uate School of Information Science, Nara Institute of Science and Technology, Japan, 2003. Table 5 lists all projects used in this study, including their categories and selected versions. [8] G. Myles and C. Collberg, “k-gram based software For each project, four versions were selected to evalubirthmarks”, Proc. 2005 ACM symposium on Applied ate resilience under functional evolution: the earliest and Computing, pp.314—318, ACM, 2005. the latest available versions (as of June 18, 2025), along with two intermediate versions sampled at approximately [9] H. Tamada, M. Nakamura, A. Monden, and K. Matequal intervals from the version history. This design prosumoto, “Design and evaluation of birthmarks for devides a balanced coverage of both short-term and longtecting theft of Java programs”, in Proc. IASTED SE term code evolution. 2004, pp.569—575, Innsbruck, Austria, 2004. To ensure sufficient structural complexity for meaningful comparison, only versions containing approxi- [10] H. Tamada, M. Nakamura, A. Monden, and K. Matsumoto, “Java birthmarks - detecting the software mately 50 or more Java class files were considered. If theft”, IEICE Trans. Inf. Syst., E88-D(9): pp.2148— a selected version did not provide an available .jar file 2158, 2005. DOI: 10.1093/ietisy/e88-d.9.2148. (either on GitHub or the Maven repository), the closest alternative version was used instead. [11] D. Lee, D. Kang, Y. Choi, J. Kim, and The selection process was automated using a Python D. Won, “API-based software birthmarking script to ensure consistency. method using fuzzy hashing,” IEICE Trans. For projects containing multiple .jar files in a given Inf. Syst., E99.D(7): pp.1836-–1851, 2016. DOI: version, all available project components were included. 10.1587/transinf.2015EDP7379. Note that external dependencies are subsequently removed during the filtering stage described in Section 5.3, [12] S. Choi, H. Park, H. I. Lim, and T. Han, “A static API birthmark for windows binary executaensuring that only project-internal modules are analyzed. bles”, Journal of Systems and Software, 82(5):862— Each version is represented in the format <version 873, 2009. name> <release date in ISO 8601 [29] format>. [13] D. Schuler and V. Dallmeier, “Detecting Software Theft with API Call Sequence Sets”, in Proc. of References the 8th Workshop Software Reengineering (WSR’06), Germany, 2006. [1] D. Kim, S. Cho, S. Han, M. Park, and I. You, “Open Source Software Detection using Function-level Static [14] N. Fedorov, H. Tamada, H. Inayoshi, and A. MonSoftware Birthmark”, Journal of Internet Services den, “Comparison of Similarity Functions for n-gram and Information Security (JISIS), vol.4, no.4, pp. 25– Software Birthmarks,” in Proc. of the 2024 WSSE, 37, 2014. pp.169—176, 2024. DOI: 10.1145/3698062.3698087. [2] C. V. Lopes, P. Maj, P. Martins, V. Saini, D. Yang, [15] G. Myles and C. Collberg, “ Detecting software theft J. Zitny, H. Sajnani, and J. Vitek, “DéjàVu: A Map via whole program path birthmarks”, Information seof Code Duplicates on GitHub”, in Proc. ACM Procurity, pp.404—415, Springer, 2004. gram. Lang. 1, OOPSLA, Article 84, 28 pages, 2017. [16] T. Kakimoto, A. Monden, Y. Kamei, H. Tamada, M. DOI: 10.1145/3133908. Tsunoda, and K. Matumoto, “Using software birth[3] R. Duan, A. Bijlani, M. Xu, T. Kim, and W. Lee, marks to identify similar classes and major function“Identifying Open-Source License Violation and 1-day alities”, in Proc. of the 2006 MSR, pp.171–172, 2006. Security Risk at Large Scale”, CCS ’17, USA, 2017. DOI: 10.1145/1137983.113802. DOI: 10.1145/3133956.3134048. [17] IBM Corporation, “Effects of nested classes.” [On[4] L. An, o. Mlouki, F. Khomh, and G. Antoniol, line]. Available: www.ibm.com/docs/en/clearcase/ “Stack Overflow: A Code Laundering Platform?”. 11.0.0?topic=omake-effects-nested-classes, arXiv:1703.03897 [cs.SE], 2017. Accessed on: Mar. 10, 2025. 16

Table 5: Open-source projects. Category

Project name

Version 1

Version 2

Version 3

Version 4

CLI framework

Airline JCommander picocli Termd

0.4 2012-8-21 1.30 2012-10-27 0.9.0 2017-04-27 1.0.0 2015-10-06

0.5 2013-01-10 1.48 2015-04-10 3.0.1 2018-05-15 1.1.2 2016-08-17

0.7 2014-11-06 1.69 2017-04-17 4.0.1 2019-07-19 1.1.6 2018-10-06

0.9 2019-12-06 1.82 2022-01-10 4.7.7 2025-04-19 1.1.10 2024-11-20

Data management

JetCache Gobblin Data Transfer Project P6Spy

2.1.3 2017-04-17 0.12.0 2018-06-20 0.1.5 2019-02-06 2.0.0 2014-03-04

2.5.1 2018-05-15 0.13.0 2018-09-06 0.3.65 2022-06-13 2.3.0 2016-05-11

2.6.0.RC 2020-03-01 0.15.0 2020-11-30 0.4.3 2022-12-14 3.6.0 2017-11-12

2.7.8 2025-04-28 0.17.0 2023-06-13 1.1.13 2025-05-19 3.9.1 2020-07-26hb

Emulator

Coffee GB ZX-Poly unidbg

1.0.0 2017-12-22 2.0.0 2019-11-20 0.3.0 2020-06-05

1.0.2 2023-03-05 2.0.7 2020-08-16 0.8.0 2020-11-06

1.2.0 2024-02-26 2.2.3 2022-01-15 0.9.4 2021-07-29

1.2.1 2024-02-26 2.3.5-S 2024-12-08 0.9.8 2024-09-05

Image processing

Thumbnailator ImageJ ImgLib2 Pixelitor

0.4.1 2012-04-01 1.48c 2013-09-13 2.0.0 2014-10-18 4.0.0 2016-02-18

0.4.7 2013-12-24 1.51a 2016-05-13 3.3.0 2017-02-18 4.2.1 2019-09-26

0.4.13 2020-10-18 1.52s 2019-12-12 5.9.3 2020-07-19 4.2.4-beta2 2021-03-13

0.4.20 2023-06-28 1.54p 2025-02-18 7.1.5 2025-04-21 4.3.1 2023-09-06

LLM integration

LangChain Java LangChain4J Langtorch

0.1.5 2023-06-07 0.4.0 2023-06-20 0.0.4 2023-04-13

0.1.8 2023-07-06 0.18.0 2023-07-26 0.0.7 2023-05-12

0.1.11 2023-08-04 0.30.0 2024-01-19 0.0.12 2023-05-30

0.2.2 2023-12-21 1.0.1 2024-04-29 0.0.17 2023-06-22

Media player

DJ Native Swing MediaPlayer vlcj

0.9.8 2009-03-24 v30SepBeta 2015-09-30 2.1.0 2012-05-20

1.0.1 2011-03-18 v20151020 2015-10-20 3.10.1 2015-12-30

1.0.2 2013-11-03 v20180320 2018-03-20 4.5.1.1 2020-04-18

1.0.3 2023-03-06 v0.0.9.5 2019-05-20 4.11.0 2025-05-04

Reverse engineering

Recaf JByteMod-Beta JVM Explorer

0.2 2017-08-13 pre-1 2017-09-19 v0.5.0 2022-07-16

1.15.7 2019-07-11 1.4.1 2018-01-09 v0.5.4 2022-07-17

2.8.0 2020-08-17 1.6.1 2018-03-02 v0.8.0 2022-08-04

2.21.14 2024-04-27 1.8.2 2018-08-29 v1.0.1 2022-08-14

Terminal application

Jmxterm Muon (Snowflake) TN5250J

1.0.0 2017-07-28 v1.0 2019-10-04 0.6.2 2010-12-09

1.0.2 2020-09-04 v1.0.2 2019-10-25 0.7.6 2015-10-20

1.0.3 2022-08-06 v1.0.3 2019-11-14 0.8.0-beta1 2021-03-25

1.0.4 2022-10-27 v1.0.4 2020-02-07 0.8.0-beta2 2021-04-24

Text editor

RSyntaxTextArea neoeedit

1.4.1 2010-05-01 v353 2021-09-19

2.5.8 2015-09-09 v378 2022-06-12

3.1.5 2021-12-30 v383 2023-03-01

3.6.0 2025-03-18 v387 2024-05-22

Unit testing framework

JUnit 4 Mockito PowerMock Robolectric TestNG

3.8.1 2002-09-04 1.3 2008-04-10 1.4.6 2010-10-13 2.0-rc1 2013-05-08 5.0 2006-07-24

4.6 2009-04-13 2.0.97-beta 2016-08-04 1.5.5 2014-05-28 3.3.2 2017-03-28 6.4 2012-02-12

4.12-beta-2 2014-09-25 2.13.0 2017-12-06 1.7.1 2017-08-12 4.5.1 2021-01-31 6.13 2017-11-27

4.13.2 2021-02-13 5.18.0 2025-05-21 2.0.9 2020-11-01 4.14.1 2024-11-20 7.11.0 2025-02-13

[18] Oracle Corporation, “Nested Classes.” [Online]. Available: https://docs.oracle.com/javase/ tutorial/java/javaOO/nested.html, Accessed on: Mar. 10, 2025.

ware Birthmarks Based on API Calls”, Information Science Technical Report, NAIST-IS-TR2007011, ISSN 0919-9527, Graduate School of Information Science, Nara Institute of Science and Technology, Japan, 2007.

[19] D.-K. Chae, J. Ha, S.-W. Kim, B. Kang, and E. G. Im, “Software plagiarism detection: a graph- [25] B. Yuan, J. Wang, Z. Fang, and L. Qi, “A New Software Birthmark based on Weight Sequences of Dybased approach”, Proc. 22nd ACM Intern. Conf. on namic Control Flow Graph for Plagiarism Detection”, Inf. and Knowl. Manag., pp.1577–1580, 2013. DOI: The Computer Journal, vol.61, no.8, pp.1202—1215, 10.1145/2505515.2507848. 2018. DOI: 10.1093/comjnl/bxy055. [20] W. Zhu, C. Thomborson, and F.-Y. Wang, “A survey of software watermarking”, Intel. and Sec. Inf., [26] H. Tamada, K. Okamoto, M. Nakamura, and A. Monden, “Dynamic Software Birthmarks to Detect pp.454-–458, Springer, 2005. the Theft of Windows Applications”, International [21] A. Monden, H. Iida, K. Matsumoto, K. Inoue, and Symposium on Future Software Technology, vol. 20, K. Torii, “A practical method for watermarking Java no. 22, 2004. programs”, in Proc. 24th IEEE compsac2000, pp.191– [27] Z. Tian, Q. Zheng, T. Liu, M. Fan, E. Zhuang, 197, Taipei, Taiwan, 2000. and Z. Yang, “Software Plagiarism Detection [22] H.-I. Lim, H. Park, S. Choi, and T. Han, “A method with Birthmarks Based on Dynamic Key Instrucfor detecting the theft of java programs through analtion Sequences”, IEEE Trans. on Software Engiysis of the control flow information”, Information neering, vol.41, no.12, pp.1217–1235, 2015. DOI: and Software Technology, vol.51, no.9, pp.1338-–1350, 10.1109/TSE.2015.2454508. 2009. [28] S. Cesare and Y. Xiang, “Malware Variant Detec[23] S. Nazir, S. Shahzad, and N. Mukhtar, “Softtion Using Similarity Search over Sets of Control Flow ware birthmark design and estimation: A systemGraphs”, 2011IEEE 10th International Conference on atic literature review”, Arab. Journal for Science Trust, Security and Privacy in Computing and Comand Engineering, vol.44, pp.3905–3927, 2019. DOI: munications, Changsha, China, 2011, pp. 181–189. 10.1007/s13369-019-03718-9. DOI: 10.1109/TrustCom.2011.26. [24] H. Tamada, K. Okamoto, M. Nakamura, and A. Monden, “Design and Evaluation of Dynamic Soft-

[29] Markus Kuhn, “A summary of the international standard date and time notation”, [Online]. Avail17

able: https://www.cl.cam.ac.uk/~mgk25/isotime.html. Accessed on: Mar. 13, 2025.

[42] X. Wang, Y.-C. Jhi, S. Zhu, and P. Liu, “Detecting software theft via system call based birthmarks”, Computer Security Applications Conference [30] J. Jang, A. Agrawal, and D. Brumley, “ReDeBug: ACSAC’09., 2009. finding unpatched code clones in entire os distributions”, in Proc. of the 33rd IEEE Symposium on Se- [43] Software Freedom Conservancy, Inc., “Current Status of Vizio Case”, Software Freedom Conservancy, curity and Privacy (Oakland), USA, 2012. [Online]. Available: https://sfconservancy.org/ [31] C. Alejandro, H. Tamada, and Y. Kanzaki, “Towards copyleft-compliance/vizio.html. Accessed on: the auto extraction for the dynamic software birthMay 24, 2025. marks with the inputs from the plaintiff software”, [44] R. G. Sanders, S. V. Vakili, J. A. Schlaff, D. N. J-Global, vol.22, no.1, pp.165–168, 2023. Schultz, and S. P. Hoffman, “CASE NO.: 30-2021[32] P. Charland, B. C. Fung, and M. R. Farhadi, “Clone 01226723-CU-BC-CJC. COMPLAINT FOR: (1) search for malicious code correlation”, In ATO RTO BREACH OF CONTRACT; and (2) DECLARASymposium on Information Assurance and Cyber DeTORY RELIEF”, SUPERIOR COURT OF THE fense (IST-111), 2012. STATE OF CALIFORNIA COUNTY OF ORANGE - CENTRAL JUSTICE CENTER. [Online]. [33] V. Markovtsev, W. Long, “Public Git Archive: a Big Available: https://sfconservancy.org/static/ Code dataset for all”. arXiv:1803.10144 [cs.SE], 2018. docs/software-freedom-conservancy-v-vizioDOI: 10.48550/arXiv.1803.10144. complaint-2021-10-19.pdf. Accessed on: May 24, 2025. [34] B. Lu, F. Liu, X. Ge, B. Liu, and X. Luo, “A Software Birthmark Based on Dynamic Op- [45] District Court S.D. New York, “IBM Corcode n-gram”, International Conference on Semanporation v. Teraproc Inc. (7:16-cv-07989)”, tic Computing (ICSC 2007), pp.37–44, 2007. DOI: CourtListener. [Online]. Available: https: 10.1109/ICSC.2007.15 //www.courtlistener.com/docket/4524777/ibmcorporation-v-teraproc-inc/. Accessed on: May [35] Y.-C. Jhi, X. Jia, X. Wang, S. Zhu, P. Liu, and 24, 2025. D. Wu, “Program Characterization Using Runtime Values and Its Application to Software Plagiarism [46] M. Joy and M. Luck, “Plagiarism in Programming Detection”, in Proc. of the ACM/IEEE 33rd InterAssignments”, University of Warwick. Department of national Conference on Software Engineering (ICSE Computer Science. (Department of Computer Science 2011), Software Engineering in Practice Track, USA, Research Report). (Unpublished), 1998. 2011. [47] T. Le, A. Carbone, J. Sheard, M. Schuhmacher, M. [36] X. Wang, Y.-C. Jhi, S. Zhu, and P. Liu, “Behavd. Raadt, and C. Johnson, “Educating computer proior Based Software Theft Detection”, CCS’09, USA, gramming students about plagiarism through use of 2009. a code similarity detection tool”, 2013 Learning and Teaching in Computing and Engineering, 2013. DOI: [37] Z. Tian, Q. Zheng, T. Liu, and M. Fan, “DKISB: Dy10.1109/LaTiCE.2013.37 namic key instruction sequence birthmark for software plagiarism detection”, in Proc. IEEE Int. Conf. High [48] B. Yadegari, B. Johannesmeyer, B. Whitely, and S. Perform. Comput. Commun., pp. 619-–627, 2013. Debray, “A Generic Approach to Automatic Deobfuscation of Executable Code”. In 2015 IEEE Sympo[38] D. Schuler, V. Dallmeier, and C. Lindig, “A dynamic sium on Security and Privacy. IEEE, USA, 674–691, birthmark for Java,” in Proc. of the twenty-second 2015. DOI: 10.1109/SP.2015.47 IEEE/ACM international conference on Automated software engineering (ASE ’07), pp. 274-–283, 2007. [49] C. Collberg, C. Thomborson, and D. Low, “A Taxonomy of Obfuscating Transformations”. Department of [39] Y. Bai, X. Sun, G. Sun, X. Deng, and X. Zhou, “DyComputer Science, The University of Auckland. New namic k-gram based software birthmark”, in Proc. Zealand, 1997. URL: http://www.cs.auckland.ac. 19th Australian Softw. Eng. Conf., pp. 644-–649, nz/staff-cgi-bin/mjd/csTRcgi.pl?serial. 2008. [50] C. K. Roy and J. R. Cordy, “A survey on software [40] C. Liu, C. Chen, J. Han, P. S. Yu, “GPLAG: detecclone detection research,” Queen’s School of Computtion of software plagiarism by program dependence ing TR, vol. 541, no. 115, pp. 64–68, 2007. graph analysis”, KDD ’06, pp. 872–881, 2006. [51] H. -i. Lim, H. Park, S. Choi and T. Han, “A [41] F. Zhang, D. Wu, P. Liu, and S. Zhu, “Program Logic Static Java Birthmark Based on Control Flow Edges,” Based Software Plagiarism Detection”, 2014 IEEE 2009 33rd Annual IEEE International Computer Soft25th International Symposium on Software Reliability ware and Applications Conference, Seattle, WA, USA, Engineering, pp.66–77, 2014. 2009, pp. 413-420, DOI: 10.1109/COMPSAC.2009.62. 18

[52] Z. Tian, Q. Zheng, T. Liu, M. Fan, X. Zhang and Z. Yang, “Plagiarism detection for multithreaded software based on thread-aware software birthmarks, ” In Proceedings of the 22nd International Conference on Program Comprehension (ICPC 2014). Association for Computing Machinery, New York, USA, 2014, pp. 304–313, DOI: 10.1145/2597008.2597143.

B

Biography Section

Nikolay Fedorov received the B.E. degree in software engineering in 2021 and M.E. degree in computer science in 2023 from Dubna State University, Russia. He is currently a doctoral student of Graduate School of Environmental, Life, Natural Science and Technology, Okayama University, Japan. His research interests include software analytics and security, as well as internet of things (IoT) systems. Akito Monden received the B.E. degree in electrical engineering from Nagoya University, in 1994, and the M.E. and D.E. degrees in information science from Nara Institute of Science and Technology (NAIST), in 1996 and 1998, respectively. He is currently a Professor with the Faculty of Environmental, Life, Natural Science and Technology, Okayama University, Japan. His research interests include software measurement and analytics, and software security and protection. He is a member of IEEE, IEICE, IPSJ, and JSSST. Hiroki Inayoshi received the B.E., M.E., and D.E. degrees from Nagoya Institute of Technology, Japan, in 2018, 2021, and 2024 respectively. He is currently an Assistant Professor with the Faculty of Environmental, Life, Natural Science and Technology, Okayama University, Japan. His research interests are in dynamic program analysis, mobile security, and privacy. He is a member of ACM, IEEE, and IPSJ. Haruaki Tamada is a professor in the Faculty of Information Science and Engineering at Kyoto Sangyo University, having joined in 2008. His research interests include software security and programming education. He received a Doctor of Engineering in Information Science from Nara Institute of Science and Technology. He is a member of IEICE and JPSJ. Masateru Tsunoda is an associate professor in the Department of Informatics at Kindai University, Japan. His research interests include software measurement and human factors in software development. He received 19

a Doctor of Engineering in information science from Nara Institute of Science and Technology. He is a member of IEEE, IEICE, IPSJ, JSSST, and JSISE.