arXiv CSopen access

Improving Communication of Changes in Model-Based Engineering with Model-Independent Change Descriptions

I MPROVING C OMMUNICATION OF C HANGES IN M ODEL -BASED E NGINEERING WITH M ODEL -I NDEPENDENT C HANGE D ESCRIPTIONS A P REPRINT

arXiv:2607.21084v1 [cs.SE] 23 Jul 2026

Philip Ochs Institute of Information Security and Dependability Karlsruhe Institute of Technology Am Fasanengarten 5, Karlsruhe, 76131, Germany. [email protected] Tobias Pett Institute of Information Security and Dependability Karlsruhe Institute of Technology Am Fasanengarten 5, Karlsruhe, 76131, Germany. [email protected]

Lars Gesmann Institute of Product Engineering Karlsruhe Institute of Technology Kaiserstrasse 10, Karlsruhe, 76131, Germany. [email protected] Ina Schaefer Institute of Information Security and Dependability Karlsruhe Institute of Technology Am Fasanengarten 5, Karlsruhe, 76131, Germany. [email protected]

July 24, 2026

A BSTRACT In model-based engineering, inter-disciplinary teams collaborate through models, which change over time for purposes of system development, what makes the proper description of such changes crucial for engineers. However, any change made by the engineer of one discipline will be difficult to understand by the engineers of other disciplines. To overcome this limitation, model-independent change descriptions can be derived instead, which preserve semantics of the changes and do not require model-specific knowledge. The two opposing approaches here are to describe changes using either informal language or formal notions of change. While informal language lacks objectivity and standardisation, formal notions of change lack human interpretability, and thus offering no support for inter-disciplinary communication. In this paper, we propose functions to map formally specified changes, represented in the approach of delta modelling, to change descriptions in modelindependent language. In an exhaustive mixed-methods evaluation, we bridge the gap between the theoretical and the practical representation of changes. We quantitatively assess technical feasibility, with an implementation framework, and technical applicability, along a case study; and we qualitatively assess plausibility, practical applicability, and extensibility, in a user study. Our work shows a promising starting point for automated, model-independent description of changes in model-based engineering projects. Keywords MBSE, change description, change communication, delta modeling Disclaimer This preprint presents work that has been submitted to the International Journal on Software and Systems Modeling (SoSyM) for peer review. Please note that the manuscript may undergo further revisions before final publication. The current version is dated July 24, 2026. Copyright may belong to Springer. This preprint is shared under the terms of arXiv’s default license.

Improving Communication of Changes in Model-Based Engineering

1

A P REPRINT

Introduction

Model-based cyber-physical engineering [40] requires the pooling of expertise from multiple disciplinary backgrounds. Teams of engineers work in coordinated collaboration on models but also independently make changes to the models for purposes of system development. Hence, both the development and also the collaboration between developers depend crucially upon appropriately describing model changes over time. Inconsistencies between the models are hard to detect manually but lead to significant drawbacks and introduce high financial risks for the manufacturers [39]. To preserve consistency, a change to a model must be described to other engineers across the domains involved so that they are aware of the change and can initiate subsequent changes for consistency preservation to affected models. However, a concise description of a model change is highly domain-specific, and thus hard to understand by engineers outside the domain in which the change was initiated. This opens the challenge of describing highly domain-specific model changes in order to preserve consistency of the models involved throughout the engineering project. Informal languages for the description of changes, such as natural language used in code commit messages, are well interpretable by engineers but are usually subjective and lack standardisation [17]. Furthermore, concrete change descriptions in informal languages must be set up manually and so lack crucial parts for automated processing. Consequently, informal languages are infeasible in the sorts of large projects typical of cyber-physical engineering. On the other hand, model-specific change descriptions in formal languages can be processed computationally but are neither supportive for human interpretability [19] nor for interdisciplinary communication, as necessary in cyber-physical engineering processes. In this paper, we approach the challenge of describing highly domain-specific model changes in multi-disciplinary engineering, by combining formal, computation-based languages with informal, human-interpretable languages. For that, we propose mapping functions to translate formalised model changes into model-independent, humaninterpretable change descriptors. As a representation of formal languages of change, we use the concept of delta modelling [36], originating in the field of software product lines, for expressing changes over time on arbitrary models. As a representation of informal languages of change, we use the description model of System Generation Engineering (SGE) [1, 2], originating in the field of mechanical engineering, providing classification and metrics of change. In an extensive, threefold mixed-methods evaluation, we bridge the gap between formal and informal description of change. We quantitatively assess (1) technical feasibility and (2) technical applicability of our approach, by implementing a generic framework for change descriptors, and applying it to Component Architecture Models (CAMs) and State Machine Models (SMMs) of the real-world Body Comfort System (BCS) case study [21] as subject system. We qualitatively assess (3) plausibility, practical applicability, and extensibility of our proposed approach, by conducting a study with twelve experts, from the research field of the description model of SGE, containing a semi-structured interview. Our contributions advance the interdisciplinary communication and propagation of model changes within modelbased engineering projects, by enabling automated, instead of manual description and analysis of evolution. This again simplifies preservation of the project’s consistency, thus lowering development risk for manufacturers. The remainder of this paper is structured as follows: In Section 2, we introduce fundamental concepts to our work as well as a running example. In Section 3, we present the concept of our description approach in detail, which we exhaustively evaluate in Section 4. In Section 5, we discuss existing literature, related to the description of changes. Finally, in Section 6, we summarise our work and outlook to future work.

2

Background

This section covers concepts which serve as a foundation of our work. We first introduce delta modelling [36] as a representative of a formal, computation-based change description language, together with its context of software product lines. Then we introduce the description model of SGE [2] as a representative of an informal, human-interpretable language for change. 2.1

Software Product Lines & Delta Modelling

In software product lines, individual product variants share common properties, but differ in their functionality, introducing variability in space [8]. The definition of possible functionalities, called features, and the theoretical composition of them into individual configurations, for example for a specific customer, are given in the problem space of a product line. In turn, in the solution space, concrete realisation artefacts for a configuration are derived. The realisation artefacts comprehend, in context of this work, models, for example from the Systems Modeling Language (SysML) 2

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

standard [40]. Thus, within a single model, we refer to a model variant as a realisation artefact of a concrete configuration. Delta Modelling, proposed by Schaefer [36], is an approach to derive model variants in a software product line. From a core model variant, individual model variants are derived by applying a combination of predefined deltas to this core model variant. A delta itself consists of a set of delta operations, which represent atomic operations, such as additions, modifications, and deletions to a model. A delta model defines all applicable deltas to a model, together with their application conditions, based on selected and deselected features in a configuration. This means that, for a concrete configuration, it is known which deltas have to be applied to the core model variant in order to get the desired model variant. Thus, a delta model is a solution space representation of a product line’s variability. To compose deltas from delta operations, the delta dialect of a model defines a set of delta operations applicable in the model. Specifically, delta operations change either modelling elements, or relations over modelling elements, by adding, modifying, or deleting them. For example, the delta dialect of a SysML class model could contain delta operations for adding a class, or for modifying cardinalities in an relation between two classes. On the instance level in this example, the delta operations would add a specific instance of the class, or would modify cardinalities of a specific instance of the relation between two specific class instances. To tackle growing size and complexity, and ease analysis for delta models, refactoring techniques concern structural properties of deltas, such as monotonicity [9]. Monotonicity in context of delta modelling describes how a delta changes a model, to which the delta is applied, either increasingly (adding elements) or decreasingly (deleting elements). Specifically of interest in our work, as edge case consideration, are strictly-decreasing monotonic deltas, which only contain delta operations for deletion of modelling elements. This means that, after applying a strictlydecreasing monotonic delta, a model only contains modelling elements which were existing before the delta was applied, and which were not modified by the applied delta. 2.2

System Generation Engineering (SGE)

The description model of System Generation Engineering (SGE), proposed by Albers et al. [1, 2], originating in mechanical engineering, is an approach to describe and analyse the development of cyber-physical systems in interdisciplinary, model-based engineering. While it was tailored to mechanical systems initially [1], the authors later generalised their approach, to be applicable to cyber-physical systems in model-based engineering [2]. The description model of SGE is predicated on the hypothesis that a new system is always built upon subsystems, contained in a reference system, varied and integrated into the new system. There exist three types of variations to a subsystem: With (1) a Carryover Variation (CV), a subsystem is taken almost without change, only with adjusted interfaces, integrated into the new system. With (2) an Attribute Variation (AV), a subsystem is changed in the properties of its contained elements, but maintaining structural and functional relations. In (3) a Principle Variation (PV), a subsystem is changed in the relations of its contained elements, thus in its functional principle. In combination, a new system hence is described by the union of all variations applied to the subsystems of the reference system, classified by the three variation types: The set of subsystems varied by CVs, the set of subsystems varied by AVs, and the set of subsystems varied by PVs. These three sets of subsystems, for CVs, AVs, and PVs, are then used in product development analyses. Specifically, their cardinalities are set into relation to each other, thus the numerical shares of each variation type in the new system, so called variation shares, are calculated. Variation shares are indicators for assessing innovation potential and development risks. For example, a high share of CVs indicates reduced development risk, because mostly known technologies are used. In turn, a high share of AVs and PVs indicates the opposite: increased development risk because of more technical novelty.

3

Translating Model-Specific Changes into Model-Independent Change Descriptors

We formalise the specification of model changes in a model-based engineering process so as to enable an automated description of changes. We use delta modelling [36], because it can be applied on an arbitrary model, as long as a meta-model is defined for it. This versatility of delta modelling is crucial in multi-domain engineering projects with many different models involved. However, as delta modelling was initially designed in the context of variability in space, we adapt it to our context of variability in time (Section 3.1). We use formalised changes, from our adaption of delta modelling, and lift them up to model-independent change descriptions, understandable outside the domain of the underlying model, by defining two mapping functions: (1) atomic mappings specify the translation of delta operations into change descriptors, explained in Section 3.2, and 3

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

Figure 1: UML class model of the running example representing an electro mechanical brake system, together with its delta dialect (represented with coloured symbols). (2) composed mappings specify the translation of deltas into aggregated change descriptors, explained in Section 3.3. Throughout the sections, we give application examples of change descriptors and aggregated change descriptors within the description model of SGE as model-independent language, and illustrations along a running example. 3.1

Preliminaries: Delta Modelling for Variability in Time

Delta modelling [36] represents an approach to derive model variants in product lines. Thus, a delta initially is specified and applied in context of variability in space. In contrast, the context of our work is variability in time (evolution). However, we can adapt concepts from delta modelling to be used in our context, namely (1) deltas as unit of versioning of a model, (2) delta operations as atomic changes to a model, and (3) delta dialects as specification of possible atomic changes for a model. We elaborate on these three parts in the following, and illustrate along our running example. We use a delta to represent changes between two (consecutive) versions of a model. This means, starting from an initial version of a model, with a delta we formally describe an ordered sequence of change operations applied to a model, to then represent its next version. Delta operations contained in a delta are applied to modelling elements of a model, thus either add new, modify, or delete existing modelling elements. For example, delta operations directly represent atomic actions, done by a developer changing a model. For a specific model, we define possible delta operations, which are allowed to be applied to model instances, with an associated delta dialect. The delta dialect of a model does not need to be complete, i.e. not all modelling elements must necessarily be changeable by applying delta operations. For example, an identification attribute of a modelling element is typically designed to be immutable, thus no possibility to modify such an attribute is defined in the associated delta dialect. Running Example Our running example represents a part of the controller software of an electro-mechanical brake system, thus picking up aspects of multi-domain engineering; software engineering and mechanical engineering specifically. The controller software is modelled as a Unified Modeling Language (UML) class model, as shown in Figure 1 (without coloured symbols). Briefly, the controller is known to one brake type, either a drum brake or a disk brake; however, both these brake types share attributes (not modelled here explicitly) but each also has distinct, unshared attributes, such as the brake disk diameter of the disk brake. The delta dialect for the UML class model of our running example is marked with coloured symbols in Figure 1, and specifies the following delta operations: (1) the modification of the diameter of the disk brake, (2) the replacement of the actual brake type instance the controller is known to and (3) the addition and deletion of a brake type instance. 3.2

Describing Delta Operations: Atomic Mappings

To preserve semantic information of each (atomic) change operation while decoupling it from model-specific language, we propose atomic mappings. An atomic mapping is used to specify how a delta operation, defined in the delta dialect of a model, is translated into a change descriptor, defined in the model-independent language. With the description model of SGE as model-independent language, we use a variation type (AV, PV) as change descriptor. This means that, by applying an atomic mapping to an applied delta operation within a model, the delta operation is mapped to either an AV, which describes a change of one of the model’s properties, or to a PV, which describes a change of the model’s functioning principle. We omit using CVs, because, according to Albers et al. [2], they describe minor changes to the interface of a carried-over modelling element ("carryover"), which, again, we understand as a change of a modelling element’s properties. Running Example For the UML class model in our running example, we formalised a delta dialect with three delta operations, shown in Figure 1. We now specify the atomic mappings for the three delta operations as follows: (1) 4

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

The modification of the diameter of the disk brake maps to an AV, because this delta operation concerns a (visible) attribute of the brake disk, and does not address any functioning principle of the brake system’s UML class model. (2) The modification of the actual brake type instance, the controller is known to, maps to a PV, because this delta operation concerns the principle of how the brake system works physically. (3) The addition and deletion of a brake type instance are delta operations which are necessary to apply the delta operation for the replacement of the brake type instance (delta operation (2)); for example, first a new brake type instance is added, then the reference to the brake type of the controller is modified, and finally the former brake type instance is (optionally) deleted. We argue that the delta operations for adding and deleting a brake type instance are AVs to this UML class model, because we understand brake type instances as properties of the described brake system, and because the actual change of the brake system’s function is represented by the modification of the controller’s reference to a brake type instance. 3.3

Describing Deltas: Composed Mappings

To compose change descriptions from an atomic level, covered by atomic mappings, we propose composed mappings. A composed mapping specifies how the impact of an applied delta to an underlying model is described model-independently, with an aggregated change descriptor. As we understand a delta as the model-specific change description between two versions of an underlying model, an aggregated change descriptor thus represents the modelindependent change description between two versions of an underlying model. Technically, we require the applied delta itself, the results of the applied atomic mappings for the delta, and the initial version of the model, to which the delta is applied, as inputs when applying a composed mapping; and we expect a string as output, which represents the concrete aggregated change descriptor. This way, composed mappings are versatile in their specification, and are enabled to aggregate results of applied atomic mappings as well as to work directly on the applied delta itself. With the description model of SGE as model-independent language, we model an aggregated change descriptor as metric for development risk, taken by the manufacturer by applying a delta to an underlying model. This aligns to the understanding of [1] on the central purpose of the description model of SGE. We realise this metric for development risk by combining two individual metrics, (1) about the extent to which a model have been changed by applying a delta, and (2) about the proportion of variation types which describe the contained delta operations within the delta. We realise (1) with the share of changed modelling elements, by calculating the ratio between the number of changed modelling elements and the total number of modelling elements in a model, after applying the delta. We understand a modelling element to be changed iff (a) it is an entity instance and was either added or modified in one of its attributes or (b) it is an relation instance over entity instances and was either added or modified in one of its entity associations. We realise (2) with the proportion of variation types, by aggregating the results of the applied atomic mappings and counting the number of AVs and PVs. In the resulting aggregated change descriptor, we concatenate both individual metrics into a string of format "<x>% changed modelling context, thereof <y>% AVs and <z>% PVs", where <x> is a variable for the share of changed modelling elements, <y> is the number of AVs, and <z> is the number of PVs. Remarks on Strictly-Decreasing Monotonic Deltas Strictly-decreasing monotonic deltas [9] contain only delta operations for deletion of modelling elements. As the share of changed modelling elements is calculated on changed modelling elements after a delta is applied, none of the deleted modelling elements are counted in, because they are no longer in the model itself. This results in a share of changed modelling elements of 0%, which may sound counterintuitive. However, we argue that, from perspective of the successive model version implied by the application of a strictly-decreasing monotonic delta, all modelling elements existing are fully carried over (CVs) from the previous version of the model. This stays in line with the understanding of Albers et al. [1], where new versions are always developed only by reuse (CVs) and adaption (AVs and PVs), thus ignoring non-reused or non-adapted modelling elements. Running Example We illustrate an evolution scenario of our running example with three versions V0 , V1 , V2 . Figure 2 and Figure 3 show the evolution of UML object models, instantiated from the UML class model introduced in Figure 1, with deltas coloured in purple, and delta operations coloured in orange. We assume an initial version V0 of our running example with a drum brake. The delta from version V0 to version V1 specifies a replacement from a drum brake to a disk brake, thus consisting of (1) an addition of a disk brake instance, (2) a modification of the brake type reference (to the new disk brake) of the controller and (3) a deletion of the old drum brake instance. We apply our atomic mapping specifications from Section 3.2 to the applied delta operations and map (1) the addition of the disk brake instance to an AV, (2) the modification of the brake type reference to a PV, and (3) the deletion of the old drum brake instance, again, to an AV. Regarding the composed mapping specification, we directly calculate the proportion of variation types, from the results of the applied atomic mappings, which is 67% 5

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

AVs and 33% PVs. We then calculate the share of changed modelling elements: there are in total three modelling elements after applying the delta, namely the controller instance, the disk brake type instance and the relation between the controller and the brake type. Out of these three modelling elements, two have been changed: The disk brake type instance was added (delta operation (1)) and the reference from the controller instance to the brake type instance was modified (delta operation (2)). Therefore, we compute the share of changed modelling elements to be 67%. The resulting aggregated change descriptor for the delta from version V0 to version V1 is "67% changed modelling context, thereof 67% AVs and 33% PVs". In a further delta from version V1 to version V2 , a modification of the disk diameter of the disk brake instance is made. This applied delta operation is mapped to an AV, according to the atomic mapping. The resulting proportion of variation types thus is 100% AVs and 0% PVs. The disk brake instance is the only modelling element in version V2 which has been changed, thus the share of changed modelling elements is 33%. Thus, the resulting change descriptor for the delta from version V1 to version V2 is "33% changed modelling context, thereof 100% AVs and 0% PVs".

Figure 2: Evol. of the running example’s UML object model, version V0 to version V1 .

Figure 3: Evol. of the running example as UML object model, version V1 to version V2 .

4

Evaluation

We present a mixed-methods evaluation [16] of our approach for model-independent change descriptions. We quantitatively address technical feasibility and technical applicability, and qualitatively address plausibility, practical applicability and extensibility. While delta modelling, as conceptual side of our approach, was evaluated for conceptual soundness [36, 7], the description model of SGE, as practical side, was evaluated for usability in industry [1, 2, 3]. Hence, our evaluation bridges the gap between the conceptual and the practical side. Particularly, we aim to answer the following research questions (RQs): 6

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

RQ1 We quantitatively evaluate technical feasibility by answering the question: How can model-specific delta operations and deltas be automatically translated into change descriptors and aggregated change descriptors? We implement our approach in the Java programming language, to assess how definitions and functionalities of atomic and composed mappings can be abstracted from concrete applications, into a generalised framework. The implementation serves as a foundation for automated application of atomic and composed mappings. RQ2 We quantitatively evaluate technical applicability by answering the question: How can atomic mappings and composed mappings be applied to existing product evolution scenarios? We use our implementation for the application to existing product evolution scenarios of our subject system, adapted from the automotive realworld Body Comfort System (BCS) case study [21], consisting of deltas and delta operations. We assess how delta operations can be mapped to change descriptors by atomic mappings, and how deltas can be mapped to aggregated change descriptors by composed mappings, within the description model of SGE. RQ3 We conduct a qualitative user study with domain experts from the field of the description model of SGE, to assess three sub-RQs: RQ3.1 We evaluate plausibility by answering the question: How plausible are generated change descriptors, and the procedure to generate them, to participants? We perform our description approach together with the participants along an exemplary subject system. In a semi-structured interview, we then assess whether the participants think the resulting change descriptors are convincing and whether they approve the procedure to generate them. RQ3.2 We evaluate practical applicability by answering the question: To which extent do participants see practical application of our description approach? We assess how supportive change descriptors are in model-based systems engineering and whether the participants see potential of our description approach to be used in automated system development processes. RQ3.3 We evaluate extensibility by answering the question: Which possible extensions to our description approach are mentioned by participants? We explicitly extract how change descriptors, and the procedure to generate them, can be extended to advance their plausibility and practical applicability, to lay foundations of our future work. This section is organised as follows: We present the subject system we use for both our quantitative and qualitative evaluation in Section 4.1, before we present the actual evaluation of our three research questions; RQ1 in Section 4.2, RQ2 in Section 4.3, and RQ3 in Section 4.4. Lastly, we cover possible threats to validity in Section 4.5. 4.1

Subject System

As subject system for both research questions RQ2 and RQ3, we use an adaption of the BCS case study [21], a deltaoriented cyber-physical product line from the automotive industry. Expressing the user-experienceable functionality of a car, it specifies features of power windows, electric mirrors and an alarm system. From its modelled configurability with over 11000 different possible product variants, we use a representative subset of 18 product variants, pre-defined by Lity et al. [21] along pairwise feature interaction coverage [31]. As concrete modelling artefacts, the case study provides specifications for CAMs and SMMs; precisely a core CAM and 25 CAM deltas, and a core SMM and 42 SMM deltas. In the time-variable context of this paper, we apply the given deltas as version deltas so that, based on the core product variant P0 as common initial version V0 , 17 different product evolution scenarios are specified by individual sequences of applied deltas. For example, the CAM of product variant P14 then consists of ten applied deltas, representing its product evolution from version V0 to version V10 . Figure 4 shows the evolution scenarios of CAMs and SMMs of all 17 products. In general, this results in a total of 465 applied deltas, thereof 140 applied deltas for CAMs and 325 applied deltas for SMMs. The applied deltas contain a total of 16092 applied delta operations, thereof 11575 applied delta operations for CAMs and 4517 applied delta operations for SMMs. All artefacts used are provided in our Zenodo package [30]. 4.2

RQ1: Technical Feasibility

We first present the resulting implementation of our description approach, and then discuss design decisions we made in context of the technical feasibility of our approach. 7

Improving Communication of Changes in Model-Based Engineering

D P07 B D H K I H A E P10 F B G C E D G J B D F B C

I J L I E

P N O H J G

L U O V R W

P11

R P13 K T P01 I P V P05 J I P U O V

P06

P08

P17

D H K M Q S P12 F B E G J P L O B D I H K M S T Q S A B E G J I P L D J I P H

P17

F I D G E H F C E

K L D F J A E F

B

N O G M F B D C J F E J

Q R L L M C B P F

A J F P E J F C E J J F P

CAM

P02 P04

M Q S T K M Q S

P07 P10

U U L J C D D S D T T P D

V V W S J G G M G S S D G

Edge Legend A: DLEDFingerProtection B: DAS C: DHeata le D: DAutomaticPW E: DLEDEM F: DLEDManPW G: DCLSM H: DCLSA I: DASIM J: DLEDHeata le K: DRCKA L: DRCKM M: DALA N: DLEDAPW O: DLEDCLSM P: DLEDAS Q: DRCKSFA R: DRCKSFM S: DRCKCAP T: DCASA U: DALM V: DLEDASIM W: DCASM X: DLEDAPWCLS Y: DLEDCLSA

P03 P15 T P14 W

U O V R W P16 X V K Y Q S T

A P REPRINT

P09

SMM H H U W P S S W T W M G T

I X V V G M M U S V W T S

K I H H S Y L V Y L X S M

R Z X X W I Y H I Y L M Y

B' E' O N L' Q P08 R D' E' O K' O' P05 Y I Z R C' D' E' O K' O' J' M' P06 L I T M Y C' E' O K' O' R F' Z J' N' I' M' P16 V H X L U I T M Y A' K C' E' O K' O' R * P09 A' K F' Z G' H' I' M' N Q P15 I K Z F' R G' H' I' N Q P01 X Y I F' C' D' E' O K' J' N' I' P03 A' K G' H' I' M' N Q P12 I C' E' R J' N' I' M' P13 U Y F' Z C' K' O' R D' J' N' I' M' P11 Y I A' K F' Z G' H' I' M' N Q P14 A' K F' G' H' I' M' N Q P02 Z G' H' I' M' N Q P04

Edge Legend A: DAddRCKCtrl, B: DAddRCKSF, C: DAddAS, D: DRemFPTran,ManPWAddTran,Aut*PW, E: DAddStatu,LED F: DAddEMHeat, G: DRemManPWBSM, H: DAddStatu,LEDEM, I: DAddEMHeatBSM, J: DAddCLS, K: DRemManPW L: DAddStatu,LEDFP, M: DAddASBSM, N: DAddAut*PW, O: DAddStatu,LEDEMBSM, P: DAddRCKCAP, Q: DAddAut*PWBSM R: DAddStatu,LEDFPBSM, S: DAddRCKCtrlBSM, T: DAddRCKSFBSM, U: DAddStatu,LEDPW, V: DAddStatu,LEDEMheata le W: DAddStatu,LEDCLS, X: DAddStatu,LEDAS, Y: DAddCLSBSM, Z: DAddASBSMIM, A': DAddRCKCAPBSM B': DAddStatu,LEDAut*PW, C': DAddStatu,LEDCLSBSM, D': DAddStatu,LEDManPW, E': DAddStatu,LEDEMheata leBSM F': DAddASBSMCAS, G': DAddCLSBSMAut*PW, H': DAddCLSBSMAut*PWRCK, I': DAddCLSBSMRCK J': DAddCLSBSMManPW, K': DAddStatu,LEDASBSM, L': DAddStatu,LEDAut*PWBSM, M': DAddCLSBSMAL N': DAddCLSBSMManPWRCK, O': DAddStatu,LEDASBSMIM *: c*ntain, delta, DAddStatu,LEDAut*PW, DAddASBSMCAS, DAddASBSMIM, DAddCLSBSMAut*PW, DAddCLSBSMAut*PWRCK, DAddCLSBSMRCK, DAddAut*PW, DAddStatu,LEDAut*PWBSM, DAddAut*PWBSM

0

2

4

6

8

10

12

14

16

Version

18

20

22

24

26

28

Figure 4: Evolution scenarios for CAMs and SMMs of 17 products, adapted from the BCS case study [21].

8

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

Figure 5: Component architecture and purposes of our cps-communicator framework (in green) and its applications (in purple).

4.2.1

Result: The cps-communicator Framework

We implemented a generic framework called cps-communicator1 , which provides functionality to translate modelspecific changes to model-independent change descriptors. The framework can be extended and used for multiple models, such as CAMs and SMMs, and model-independent languages, such as the description model of SGE. The interaction is based on a command line interface, whose set of commands can be individually extended as more models and model-independent languages are added. For example, a developer can specify a command to trigger the translation of certain model changes to a certain model-independent language. The cps-communicator framework consists of three main components, as illustrated in Figure 5 in green: (1) The component cps-communicator-base defines the functionality of a command (interface Command, class AbstractCommand) and implements its discovery and execution, so that other components do not have to register individually. Further, the component implements basic utilities such as file handlers (class JsonIO) and object converters (class EObjectIO). (2) The component cps-communicator-mapping defines both atomic mappings (class AtomicMapping) and composed mappings (class ComposedMapping). Based on this, a developer can add concrete mappings between model-specific changes and model-specific languages, and with own commands to trigger them. We evaluate these steps in detail in our second research question on technical applicability, in Section 4.3. (3) The third component cps-communicator provides technical infrastructure for managing all contained components, defines the entry point of the command line interface, and glues the framework together. Technically, the cps-communicator framework is implemented in the Java programming language2 , so that it can be used on several execution platforms. Gradle3 is used as a build tool for managing the source code of the underlying components. A Gradle command shadowJar is specified which collects the components source code into a single Java archive (JAR-file), which itself can then be executed with Java in the command line. For source code version control, each component has its own Git4 repository, so that it can evolve independently from other components. The framework uses each component’s version specified in the file .gitmodules, which itself is tracked by the Git repository of the component cps-communicator, enabling fine-grained version control over included and excluded components. Regarding modelling elements, i.e entities of a model, the cps-communicator framework relies on the definition from the interface EObject within the Eclipse Modeling Framework (EMF)5 . It provides necessary properties, such as object containment references. Within the mapping implementations, an atomic mapping (class AtomicMapping) consists of a class reference representing a delta operation, and a reference to a generic change descriptor. In the application, a concrete instance of a delta operation is mapped to a concrete instance of a change descriptor. A composed mapping (class ComposedMapping) covers the application of atomic mappings to compute change descriptors with the method getChangeDescriptors, and the computation of an aggregated change descriptor with the method getAggregatedChangeDescriptor. For both of these methods, the composed mapping takes a set of delta operations, and, for the aggregated change descriptor additionally, the set of modelling elements of the updated model.

1

cps-communicator on GitHub: https://github.com/KIT-TVA/cps-communicator Java Version 17; https://www.java.com 3 Gradle Version 8.10; https://gradle.org 4 Git Version 2.42; https://git-scm.com 5 EMF Version 2.35; https://projects.eclipse.org/projects/modeling.emf.emf 2

9

Improving Communication of Changes in Model-Based Engineering

4.2.2

A P REPRINT

Discussion: Design Decisions in Context of Technical Feasibility

The aim behind our implementation, the cps-communicator framework, is to show general applicability of the underlying approach to arbitrary models and delta dialects, such as of CAMs and SMMs, and model-independent languages, such as the description model of SGE. Hence, we abstract from (1) models, (2) delta dialects, and (3) both atomic and composed mappings as far as possible, as discussed in the following parts. (1) We abstract from concrete models using a meta-model, because our concepts of composed mappings and atomic mappings are based on (changed) modelling elements within these models. For example, modelling elements of a CAM include components and connectors, while modelling elements of a SMM include states and transitions. We choose EMF as modelling element abstraction, because (a) with the interface EObject, it provides such an abstraction, with necessary identifiers and containment functionalities; and (b) because common modelling tool suites, such as Papyrus6 , also build upon EMF with their UML implementations. (2) We abstract from concrete delta dialects using a delta meta-model, because we require delta dialects to comply with general delta modelling concepts [36], such as delta operations derived from either additions, modifications or deletions. For example, modelling elements, such as components in a CAM or states in a SMM, can be added, modified and deleted. An abstract implementation of delta modelling is given in DeltaEcore7 [37], which itself is based on EMF, and used in our implementation for two reasons: (a) We can generate a concrete delta dialect automatically from a given meta-model within EMF, so by the ease of use we do not have to create it manually. (b) We can generate an according delta language for a concrete delta dialect, which is an implementation of the delta dialect in a programming language for programmatic use, e.g. instantiable objects, in our case in the Java programming language. (3) We generalised atomic and composed mappings using a generic representation, because concrete mapping specifications have structure and functionality in common: For each atomic mapping, the structure is given by the assignment to one specific delta operation of a delta dialect, and to one specific change descriptor of a model-independent language; and the functionality is given by the application to a concrete delta operation contained in a delta. For each composed mapping in turn, the functionality is given by the application to a set of concrete delta operations. Regarding the results of the applied mappings, atomic mappings are a canonical set of change descriptors, and composed mappings are aggregated change descriptors. The latter is more difficult to represent generically, because it can represent a metric, such as the share of changed modelling elements, or even further sets of elements, for example all changed modelling elements itself. This means, while atomic mappings work solely on delta operations, a composed mapping may also take the set of modelling elements of the updated model into account, for example to compute the share of changed modelling elements. Thus, a composed mapping takes a set of delta operations and a set of modelling elements of the updated model into account. In summary, we answer RQ1 as follows: We assume our approach as technically feasible; specifically, we achieved a full separation between abstract definitions and functionalities, given in the cps-communicator framework, and concrete implementations for application, further assessed in Section 4.3. 4.3

RQ2: Technical Applicability

We quantitatively assess how atomic and composed mapping specifications can be applied to existing product evolution scenarios from the BCS case study [21, 30]. For that, we use and extent our implementation from RQ1 so that we can translate model-specific changes into the description model of SGE. In this section, we first describe our setup and the application procedure itself, and then present the resulting change descriptors, and finally discuss overall technical applicability. 4.3.1

Setup

Along the BCS case study, we describe the setup of delta dialects, of a meta-model for the description model of SGE, and of specifications of atomic and composed mappings, in the following paragraphs. First, we make the underlying delta dialect for each model explicit, because it specifies all possible delta operations, and because our atomic mappings are specified upon these delta operations. However, instead of re-engineering each delta dialect from the BCS case study by hand, we use a more forward-faced approach by deriving the delta dialects from meta-models. More concretely, we first define EMF-based meta-models for (a) CAMs and (b) SMMs, such that we are able to express the instantiations given in the BCS case study. For (a) CAMs, illustrated in Figure 6a, we model entities for components, ports, connectors and signals, so that a component has input ports and output ports, and a connector connects one source port with one target port. A component can also contain subcomponents, and ports comply to a signal specification. For (b) SMMs, illustrated in Figure 6b, we model entities for regions, transitions, 6 7

https://eclipse.dev/papyrus https://github.com/chseidl/deltaecore

10

Improving Communication of Changes in Model-Based Engineering

(a)

A P REPRINT

(b)

Figure 6: Meta-model for CAMs (a) and SMMs (b).

states, triggers, effects and signals, so that regions can contain states, and states in turn can contain subregions. A state can be an initial state and/or a final state. A transition connects a source state with a target state. A transition can have triggers and effects, which comply to a signal specification. Next, we use DeltaEcore [37] to derive the delta dialect for (a) CAMs with 46 different delta operations, from which twelve are additions, 22 are modifications and twelve are deletions. For (b) SMMs we derive a delta dialect with 50 different delta operations, from which ten are additions, 27 are modifications and 13 are deletions. In a second step, we set up a meta-model of the model-independent description language to represent change descriptors for atomic mappings, and aggregated change descriptors for composed mappings. As we use the description model of SGE, we specifically model an entity for a variation which has a reference to a variation type, i.e. an AV, PV, or CV, for completeness. Third, we specify atomic mappings to translate delta operations into the description model of SGE. For that we take a reduced delta dialect as a basis, as shown in Table 1: (a) From the CAM delta dialect, only 14 out of 46 defined delta operations were practically used in the BCS case study: Five delta operations for addition, seven for modification, two for deletion of modelling elements. As the remaining 32 delta operations are never used in the BCS case study, we do not consider them for our atomic mappings. Analogously, (b) from the SMM delta dialect, only 14 out of 50 defined delta operations were practically used in the BCS case study: Eight delta operations for addition, five for modification, three for deletion of modelling elements. Generally, we aligned our atomic mapping specifications as described in our concept (Section 3.2): A change of either a modelling element or a relation between two modelling elements is considered to be a PV, while a change of an attribute of a modelling element is considered to be an AV. Thus, (a) for the delta operations of the reduced CAM delta dialect, we translate a delta operation into an AV, as long as it does not add, modify or delete either the source port or the target port of a connector. Else, we assume a change in the functional principle of the modelled system and hence translate the delta operation into a PV. Analogously, (b) for the delta operations of the reduced SMM delta dialect, we translate a delta operation into an AV, as long as it does not add, modify or delete either the source state or target state of a transition; or does not add, modify or delete the trigger or effect of a transition; or does not modify the properties of a state being initial or final. Else, we assume a change in the behaviour of the modelled system and hence translate the delta operation into a PV. Table 2a shows our atomic mapping specifications for the delta operations contained in the reduced delta dialect of CAMs. The 14 delta operations are mapped to ten AVs and four PVs. In a fourth step, we specify composed mappings, to generate aggregated change descriptors for each delta. Specifically, we take the idea of assessing the development risk incorporated in a delta, described in our concept (Section 3.3), by expressing (a) the extent to which a delta is changing a model, and (b) the proportion of variation types which describe the contained delta operations within the delta. To technically execute our setup, we use our implementation from RQ1, the cps-communicator framework. We extent the framework with the component cps-communicator-mapping-sge, as illustrated in purple in Figure 5. Within this new component, we implement the class SGEComposedMapping, which realises the computation of the share of changed modelling elements and the proportion of variation types. We further implement commands for the command 11

Improving Communication of Changes in Model-Based Engineering

Model

no. Delta Operations for add.

for mod.

for del.

A P REPRINT

∑

CAM

in Delta Dialect ,→ in reduced Delta Dialect

12 ,→ 5

22 ,→ 7

12 ,→ 2

46 ,→ 14

SMM

in Delta Dialect ,→ in reduced Delta Dialect

10 ,→ 8

27 ,→ 5

13 ,→ 3

50 ,→ 14

Table 1: Number of delta operations specified in delta dialect, and reduced delta dialect implied from the BCS case study [21], for CAMs and SMM. Variation Type AV PV

AVs

AddToComponentsInSystem AddToConnectorsInSystem AddToInputPortsInComponent AddToOutputPortsInComponent AddToSignalsInSystem

x

134

x x x

993 761 763

ModifyAttributeNameInComponent ModifyAttributeNameInConnector ModifyAttributeNameInPort ModifyAttributeNameInSignal SetSignalInPort SetSourcePortInConnector SetTargetPortInConnector

x x x x x

134 1005 1754 763 2010

RemoveFromComponentsInSystem RemoveFromConnectorsInSystem ∑

x

Delta Operation

no. Applications PVs

x

1005

x x

1005 1005 21

x 10 (a)

4

222 8338

3237 (b)

Table 2: 14 delta operations specified in the reduced delta dialect for CAMs with (a) their atomic mapping specifications and (b) their number of applications in the BCS case study.

line interface of the framework to trigger the application of specified atomic and composed mappings to the CAMs (ApplyDeltasComponentCommand) and SMMs (ApplyDeltasStatemachineCommand) of the BCS case study. 4.3.2

Procedure

For CAMs and SMMs of the BCS case study, we iterate over all 17 products per model, and over all applied deltas per product, inducing the product’s evolution. For each delta, we apply the composed mapping, to get the share of changed modelling elements and the proportion of variation types. For each delta operation contained in a delta, we apply the according atomic mapping, to get the variation type within the description model of SGE. 4.3.3

Results

Based on our procedure, the results, provided in our Zenodo package [30], are structured along CAMs and SMMs. Specifically, we present data of the applied deltas (140 for CAMs, 325 for SMMs) and their contained delta operations (11575 for CAMs, 4517 for SMMs) with regard to the share of changed modelling elements and the proportion of variation types. In Figure 7, we scattered applied deltas to the the number of modelling elements related to the share of changed modelling elements. The left scatter plot belongs to CAMs, the right to SMMs. The horizontal axes represent the number of modelling elements after a delta was applied, and have the same range for CAMs and SMMs, to make absolute model sizes comparable. The vertical axis represents the share of changed modelling elements in percent, caused by a delta application. The hue of a delta represents the version in which the delta is applied to a model, while greenish colours represent earlier versions and blueish colours represent later versions. Both scatter plots show a trend 12

Improving Communication of Changes in Model-Based Engineering

CAMs

SMMs

39

30 30

25

Version

Share of Changed ME [%]

35

A P REPRINT

20

20

15 10

10

5 0

200

300

400

total no. ME

500

200

300

400

total no. ME

500

2

Figure 7: Share of changed modelling elements in relation to total number of modelling elements for each applied delta in CAMs and SMMs.

towards an increased number of modelling elements in later versions. With an average number of modelling elements of 290 (red dotted line), CAMs have a higher number of modelling elements than SMMs, with an average number of modelling elements of 182 (beige dotted line). Furthermore, with an average share of changed modelling elements of 12 % (red dashed line), CAM deltas cause a higher share of changed modelling elements than SMM deltas, with an average share of changed modelling elements of 3 % (beige dashed line). In turn, with an average number of versions of 20 (beige solid line in colour bar), SMMs have a higher number of versions than CAMs, with an average number of versions of 9 (red solid line in colour bar). Overall, both scatter plots show a trend for a decreasing share of changed modelling elements for later versions and for higher numbers of modelling elements. The proportion of variation types across all applied delta operations was in average 72 % AVs to 28 % PVs for CAMs, and 62 % AVs to 38 % PVs for SMMs. For applied delta operations in CAMs in detail, Table 2b shows how often each delta operation of the reduced CAM delta dialect was applied and thus mapped to either an AV (8338 times, ≈ 72 %) or a PV (3237 times, ≈ 28 %). For product P14 as a concrete example of an evolution scenario, Table 3 shows collected data of its ten applied deltas from version V0 to version V10 . The ten applied deltas consist of a total of 730 contained delta operations. The number of contained delta operations per applied delta decreases almost monotonously across the versions, from 175 (delta DHeatable, V1 → V2 ) to 20 (delta DCASA, V10 → V11 ), with a two exceptions (delta DCLSA, V5 → V6 , and delta DRCKCAP, V9 → V10 ). With atomic mappings, the 730 delta operations are mapped to a total of 518 AVs and 212 PVs, resulting in a proportion of AVs to PVs of 71 % to 29 %. The total number of modelling elements is increasing monotonously, from 165 in version V1 (after application of delta DHeatable) to 342 in version V10 (after application of delta DCASA), while the number of changed modelling elements decreases, again almost monotonously, from 54 in version V1 (after application of delta DHeatable) to 8 in version V10 (after application of delta DCASA). This results in an also decreasing share of changed modelling elements from 33 % in version V1 to 2 % in version V10 . 4.3.4

Discussion

The technical application of our approach to the subject system led to the expected results, i.e. all 16092 delta operations were translated into change descriptors by atomic mappings, and all 465 deltas were translated into aggregated change descriptors by composed mappings. The share of changed modelling elements typically decreases over time, as shown by the scatter plots in Figure 7 and shown by the exemplary evolution of product P14 in Table 3. We account that trend to deltas containing mostly additive delta operations instead of subtractive, shown for CAMs in Table 2b. This means, the models get larger in terms of 13

Improving Communication of Changes in Model-Based Engineering

no. Variation Types Delta V0 → V1 DHeatable V1 → V2 DAS V2 → V3 DAuto.PW V3 → V4 DASIM V4 → V5 DCLSA V5 → V6 DRCKA V6 → V7 DALA V7 → V8 DRCKSFA V8 → V9 DRCKCAP V9 → V10 DCASA ∑

D o.

p*

O

n

AVs

PVs

175 158 99 36 70 50 48 24 50 20

115 (66%) 119 (75%) 64 (65%) 27 (75%) 52 (74%) 38 (76%) 36 (75%) 18 (75%) 35 (70%) 14 (70%)

60 (34%) 39 (25%) 35 (35%) 9 (25%) 18 (26%) 12 (24%) 12 (25%) 6 (25%) 15 (30%) 6 (30%)

730

518 (71%)

212 (29%)

A P REPRINT

no. Model. Elem.

ed l ang a t to ch SoCME** 165 218 245 257 280 297 313 321 336 342

54 53 32 12 23 17 16 8 16 8

≈ 33% ≈ 24% ≈ 13% ≈ 5% ≈ 8% ≈ 6% ≈ 5% ≈ 2% ≈ 5% ≈ 2%

* DOp: Delta Operations; ** SoCME: Share of Changed Modelling Elements

Table 3: CAM evolution scenario for product P14 with 10 applied deltas and 730 contained delta operations. their number of modelling elements, but the applied deltas do not change proportionally as many of the modelling elements as the models grow. Regarding the proportion of variation types, AVs typically occur more often, in average for 72 % in CAMs and for 62 % in SMMs. This is a consequence of our atomic mapping specifications, where we use PVs to describe changes in a system’s functionality (CAM) or behaviour (SMM). The resulting, lower proportion of PVs in the application then means, that roughly two thirds of the changes mainly concern the models attributes (AV), and not their function principles (PV). For example, regarding Table 2a and Table 2b, four out of 14 specified delta operations of the reduced CAM delta dialect are mapped to a PVs, and are applied 3237 out of 11575 times in total. In summary, we answer RQ2 as follows: Our approach is technically applicable; specifically, by extending our implementation, the cps-communicator framework, we enabled the shift, from manually translating 16092 applied delta operations of our subject system, into specifying 28 atomic mappings once, and then apply them 16092 times to the occurring delta operations. 4.4

RQ3: Plausibility, Practical Applicability & Extensibility

To assess RQ3, we conduct a qualitative user study, where participants apply our description approach to a product evolution scenario, and then are asked to answer questions in semi-structured interviews. Specifically, our subject system is a product evolution scenario of a CAM, taken from the BCS case study. The model-independent language used is the description model of SGE, thus our participants are directly from this research field. We first let the participants specify atomic mappings for a subset of the CAM delta dialect, into the description model of SGE, and then apply their mapping specifications to the product evolution scenario. In the interviews, we obtain the participants opinions, to extract findings, and to finally discuss our three sub-RQs RQ3.1, on plausibility, RQ3.2, on practical applicability, and RQ3.3, on extensibility. 4.4.1

Participants

We worked together with twelve participants directly from the research field of the description model of SGE, because we used the description model of SGE as model-independent language in this study. Furthermore, within this research field, we assumed that the participants have an elementary understanding of artefacts from model-based systems engineering, which the BCS case study formally represents. From each participant, we collected demographic information a small, unmoderated questionnaire upfront, to get a more fine-grained insight into our participants background, in relation to the description model of SGE and modelbased systems engineering. Specifically, we asked about each participant’s experience with the description model of SGE in years (intervals; less than 1 year, 1 to 3 years, 3 to 10 years, more than 10 years), and for selfassessed knowledge in the description model of SGE, in the Dreyfuss Skill Acquisition scale [11] (ordinal; Novice, Advanced Beginner, Competent, Proficient, Master). Second, we asked about each participant’s self-assessed knowledge in model-based systems engineering, again in in the Dreyfuss Skill Acquisition scale [11], to see how 14

A P REPRINT

Mas er Proficien Compe en

Knowledge in MBSE Mas er Proficien Compe en Adv. Beginner

Adv. Beginner Novice t.

l.

N = 12

s rs rs ear yea 0 year 0 yea 1y 3 1 1 o 3 o 1 m. .

Experience in Year in De cription Model of SGE

2 2 t1 her1 sor1 n er ager e e s c d n r e Stu esea Prof Engi Man R

8 7 6 5 4 3 2 1 0

no. Participant

Knowledge in De cription Model of SGE

Improving Communication of Changes in Model-Based Engineering

Working Relation hip (in 1Academia or 2Indu try)

Figure 8: Overview of demographic information about participants.

strong experience and knowledge in the description model of SGE correlates to knowledge in model-based systems engineering. Third, we asked about each participant’s working relationship, were we distinguished between academia (categorial; Student, (Post-) Doctoral Researcher, Professor) and industry (Engineer, Manager). Figure 8 shows the results of the demographic questionnaire. On the left, we scattered each participant’s experience with the description model of SGE (horizontal axis) to the participant’s knowledge in the description model of SGE (vertical axis). The hue represents the participant’s knowledge in model-based systems engineering. We observe that all three dimensions correlate positively, meaning the more experience participants have with the description model of SGE, the more knowledge in the description model of SGE (Pearson correlation 0.603) and the more knowledge in model-based systems engineering (Pearson correlation 0.554) are they assessing themselves. On the right in Figure 8, the number of participants in each working relationship category are shown. From academia participated zero students, eight (post-) doctoral researchers and two professors. From industry participated one engineer and one manager. 4.4.2

Subject System

As a subject system for our user study, we used a subset of the BCS case study; in particular, the evolution of the CAM of product P14 , from version V0 to version V2 . This is because, first, we assumed that three versions (V0 , V1 , V2 ) suffice for the purpose of showing our approach without mentally overloading the participants. Second, we assumed that CAMs are known to the participants, as mechanical/ system engineers, better than SMMs, so they should familiarise faster with the setting of the evaluation. Third, we used product P14 , because we assume the first two applied CAM deltas of product P14 , delta DHeatable and delta DAS (see Table 3), to be illustrative for an exemplary model evolution. Specifically, delta DHeatable exchanges a component for a non-heatable electric mirror with a component for a heatable electric mirror, and delta DAS add a component for an alarm system. As for the specification of atomic mappings, we further tried to reduce the mental and temporal load of the participants by only let them specify six out of 14 atomic mappings for the reduced CAM delta dialect. We included two delta operations for additions, namely (1) adding an input port to a component and (2) adding a connector; two delta operations for modifications, namely (3) modifying the name of a component and (4) modifying the source port of a connector; and two delta operations for deletions, namely (5) removing a connector and (6) removing a component. We took the remaining eight atomic mappings from our setup of RQ2, where we translate a delta operation into an AV, as long as it does not add, modify, or delete either the source port, or the target port of a connector. Else, we assume a change in the functional principle of the modelled system, and hence translate the delta operation into a PV. In summary, we prepared six delta operations to be mapped by the participant, and complement them with eight delta operations mapped by ourselves. With these, in total 14 atomic mapping specifications, the deltas for exchanging a 15

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

component for a non-heatable electrical mirror with a heatable one (delta DHeatable) and for adding an alarm system (delta DAS) can be described within the evolution of product P14 from version V0 to version V2 . We ...

The Participant ... Introduction Phase

Material: demographic questionnaire, visual explanation of research rationale

1 3

give an introduction on goals and course of the study. is asked to about demographic information. give an introduction on CAMs.

2

Specification Phase Material: illustration of delta operations, confidence questionnaire

4

show six selected delta operations. is asked to specify atomic mappings. is asked about confidence in specifying atomic mappings.

5 6

Application Phase Material: illustration of evolution scenario

8 9

is asked to take a short break. digitalise atomic mapping spec., generate change descriptors. show and explain the evolution scenario.

7

Interview Phase Material: none

10 11

ask interview questions / is asked to to answer interview questions. close the session.

10

Table 4: Session procedure and material of our user study.

4.4.3

Session Procedure & Material

We conducted the study individually with each participant, so that direct influences between participants are mitigated, as the participants perceived different knowledge and experience levels among them (see Figure 8 on the left). Furthermore, with individual sessions, the existing causality from the specified atomic mappings to the generated change descriptors is personalised to the participant, which we assumed to provide better insights into the participant’s understanding of the matter. To further investigate how the participants were specifying atomic mappings, we explicitly asked them how confident they were thereby, directly after, in a questionnaire. We prelimited options to (ordinal) never confident, sometimes confident, mostly confident, and always confident. The material of the sessions comprises of the demographic questionnaire, an optional visual explanation of the research rationale, an illustration of all delta operations to be translated into the description model of SGE with atomic mappings, a questionnaire about the participant’s confidence in specifying atomic mappings, and an illustration of the evolution scenario. We prepared the sessions to be done in person and online. For the sessions in person, we printed necessary material on paper, so that the participants could lay out and perceive the information as suitable. For the sessions online, we used Microsoft Teams8 for audio-visual communication, and Miro9 as virtual desk. Each session’s procedure was intended to take between 45 and 60 minutes, and was structured along an introduction, specification, application and interview phase, as shown in an overview in Table 4, and presented in detail in the following enumeration. Introduction Phase (short) 8 9

Microsoft Teams: https://www.microsoft.com/microsoft-teams Miro: https://miro.com

16

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

1. We give an introduction on the higher level goals and the course of the study, so that the participant can contextualise the work and understand the research rationale. 2. The participant is asked to fill out the questionnaire on demographic information. 3. We give an introduction on CAMs, so that the participant can get familiar with the purpose of CAMs and the elements within CAMs, because the following tasks in the study are mainly based on CAMs. Specification Phase (extensive) 4. We show each of the six selected delta operations, one after another, and ... 5. ... the participant is asked to specify the according atomic mapping, into a variation type (AV or PV) within the description model of SGE. 6. The participant is asked to fill out the questionnaire about confidence in specifying the atomic mappings. Application Phase (short) 7. We ask the participant to take a short break while ... 8. ... we digitalise the obtained atomic mapping specifications into our notebook, generate the change descriptors and aggregated change descriptors with the notebook, and wrote them back on paper for the evolution scenario. 9. We show the evolution scenario to the participant and explain it orally in detail, so that the participant can understand on a higher level which changes the evolution scenario represents. We include the generated change descriptors and aggregated change descriptors in our explanations already, so that the participant can understand the relation between the change descriptors and aggregated change descriptors and our oral explanation of the evolution scenario. Interview Phase (extensive) 10. We ask the participant to answer some questions about the passed study, i.e. give the interview, so that we can elicit desired information about the participant’s opinions on the change descriptors and aggregated change descriptors, and our approach itself. 11. We close the session. During the case study we answered questions advancing the participant’s understanding, and gave support where appropriate, as long as it would not have directly led to answers for which we asked the participant originally. 4.4.4

Interview Guide

In the interview phase of our user study (step 10), we pose three questions to each participant, aimed to answer RQ3. Specifically, question Q10 refers to RQ3.1 on plausibility, and is bound directly to the generated change descriptors of the treated evolution scenario, while Questions Q20 and Q21 refer to RQ3.2 on practical applicability, and address our description approach more generally. RQ3.3 on extensibility is covered implicitly by all interview questions. The interview questions are as follows: Q10 How plausible are the change descriptors, and the procedure to generate them, in an SGE context, to you? Q20 How supportive is the approach to describe changes within an model-based systems engineering context to you? Q21 To what extent do you think the approach could replace manual effort within an model-based systems engineering context? With Question Q10, we elicit whether the participants can relate the generated change descriptors to the concrete product evolution scenario, e.g. if a proportion of 25% PVs, to exchange a non-heatable electric mirror with a heatable one, makes sense to them. Additionally, we elicit whether the participants understand and approve the procedure to generate the change descriptors, e.g. if atomic mapping specifications for delta operations into variation types makes sense to them. With Question Q20, we elicit to which extent and for which purposes the participants see an information gain in the generated change descriptors, in context of model-based systems engineering; for example, a descriptive purpose for documenting changes. With Question Q21, we elicit whether the participants see potential of our description approach to be applied in automated processes, e.g. automatically generate documentation without experts additions. 17

Improving Communication of Changes in Model-Based Engineering

4.4.5

A P REPRINT

Data Analysis

We recorded audio from all sessions and transcribed (audio-to-text) with aTrain10 , a locally running, artificialintelligence-driven transcription tool from academia, to comply with privacy regulations. We then analysed the transcripts with MAXQDA11 tool support. Specifically, from the transcripts, we picked out segments in which participants gave answers to our interviews questions. We inductively extracted codes and code categories from these segments, so that code categories refer to our interview questions, and codes within a code category represent different answers to the associated interview question. We denote code categories enumerated and with reference to the numbered interview question, e.g. C10-0, C10-1 etc. refer to interview Question Q10. We denote codes enumerated and with reference to the belonging code category, e.g. C10-0A, C10-0B etc. refer to code category C10-0. For each code, we record the number of marked segments from transcripts absolute, and relative in relation to the whole code category the code belongs to (henceforth called evidence). We further give the number of participants for which the code in question occurred. This way we can identify majorities among answers to our interview questions, per code category. 4.4.6

Findings

We present the findings of our user study structured along the session procedure. Specifically, we cover the atomic mapping specifications of the specification phase, and the participants answers to the questions of the interview phase. In our Zenodo package [30], we provide the collected atomic mapping specifications and generated change descriptors of all sessions, as well as transcripts in the Hoffmann-Riem flavour [12], a code matrix of the interview answers and evidence for all findings presented in this section. Specification Phase In total, ten out of twelve participants performed the atomic mapping specifications. Two participants did not, for split reasons: Either they expected a contextualisation of all presented delta operations into an application context already, by arguing about a missing reference system, or they would have mapped some delta operations to CVs, which was not intended by our study design. In both cases, we used our own atomic mapping specifications from RQ2 for the application to the evolution scenario. In the application phase, both participants were then introduced to change descriptors based on our own atomic mapping specifications. This way, even if participants did not specify mappings before, we could show actual change descriptors for the presented evolution scenario to all participants, and then start the interview phase on the same foundation, for every participant. For the ten participants who did the atomic mapping specifications, Table 5a shows the resulting distribution and standard deviation12 among the participants, for each of the six delta operations mapped to either AV or PV. Delta operation (3), for modifying the name of a component, was mapped unanimously to an AV by all participants. Delta operations (4), for modifying the source port of a connector, and (5), for removing a connector, were mostly (standard deviation 0.4) mapped to PVs. Delta operations (2), for adding a connector, and (6), for removing a component, show increased scattering (standard deviation 0.458), mapped to AV and PV. Delta operation (1), for adding an input port to a component, was equally often mapped to an AV as well as a PV (standard deviation 0.5), showing the highest scattering among the participants. We see two causes for the high standard deviation in the atomic mapping of delta operation (1): First, delta operation (1) was the first we asked the participants to map. Thus, we naturally expect a lower confidence, compared to the remaining five atomic mappings. Second, four participants gave evidence (code C00-0A) for the description model of SGE to be a highly subjective matter, most generally because the mapping of a change into AVs and PVs depends, for example, on individual knowledge of the person itself. However, we also asked about each participant’s confidence in specifying these atomic mappings, and the majority of the participants felt mostly confident, as shown in Figure 9. Furthermore, compared to our own atomic mapping specifications, which are based on the understanding of Albers et al. [2], and shown in Table 5b, nearly all participants atomic mapping specifications match up. This indicates that, although subjectivity plays a role in the specification of atomic mappings, most of the participants have, whether knowingly or unknowingly, a similar understanding in the application of the description model of SGE. Finding F1: Even though participant subjectivity plays a role in specifying atomic mappings for our subject system, most of the participants have a similar understanding within the description model of SGE. 10

aTrain Version 1.2.4; https://business-analytics.uni-graz.at/en/research/atrain/ MAXQDA Version 24.11.1; https://www.maxqda.com/de/produkte/maxqda 12 standard deviation (σ) calculated by encoding AV as 0, and PV as 1 11

18

Improving Communication of Changes in Model-Based Engineering

Participants Delta Operation

Distribution* AV PV

Stdd. Dev. (σ)12

(1) Add Input Port to Component (2) Add Connector (3) Modify Name of Component (4) Modify Source Port of Connector (5) Remove Connector (6) Remove Component

5 3 10 2 2 7

0.5 0.458 0.0 0.4 0.4 0.458

5 7 0 8 8 3

A P REPRINT

Mapping Specif. from RQ2 AV PV AV PV PV AV

*among number of participants (in total 10) who mapped to either AV or PV (a)

(b)

no. Participants

Table 5: Distribution and standard deviation for six selected delta operations (a), mapped to either AV or PV (predominant one colourised), by in total ten participants, and (b) our own mapping from RQ2.

5 4 3 2 1 0

N = 10

never sometimes mostly confident

always

Figure 9: Distribution of confidence in specifying atomic mappings for six selected delta operations.

Interview Phase We aligned the interviews along our interview guide with three main questions Q10, Q20 and Q21, and inductively extracted four code categories. From the answers to Question Q10 (Plausibility, RQ3.1), we extracted two code categories: (1) Whether the participants think the concrete change descriptors for the presented evolution scenario are convincing (Result Convincingness, code category C100), and (2) whether the participants approve our procedure to generate change descriptors (Procedure Approval, code category C101). From the answers to Question Q20 (Practical Applicability, RQ3.2), we extracted one code category: Whether the participants think our description approach in general is supportive in model-based systems engineering (Approach Supportiveness, code category C20-0). From the answers to Question Q21 (Practical Applicability, RQ3.2), we extracted one code category: Whether the participants see potential of our description approach to be used in automated system development processes without manual expert intervention (Automation Potential, code category C21-0). Code categories, and codes together with their evidence and the number of participants who gave evidence, are presented in tables 6, 7, 8 and 9. Table 6 lists codes for code category C10-0, about whether participants think the generated change descriptors are convincing (Result Convincingness), referring to Question Q10. We found a strong evidence (79%) that most of the participants think that the generated change descriptors are actually convincing (code C10-0A). In contrast, one participant mentioned that the change descriptor for the first delta DHeatable was not convincing, while the change descriptor for the second delta DAS was convincing (code C10-0B). The participant specifically added, that he would have expected a higher share of PV, regarding that adding a heating function to an electric mirror is, in the participant’s intuition, a lot of PVs. However, in the atomic mapping specifications, the participant mapped five out of six delta operations to an AV, which resulted, for delta DHeatable, in a share of AVs of 82% and a share of PVs of 18%. Finally, for code category C10-0, we found evidence (14%) from a third of the participants that they had difficulties judging the generated change descriptors (code C10-0C). As a reason, these participants mostly said that they do not know the evolution scenario in such detail to judge about the concrete numbers in the change descriptor, e.g. if 28% AVs is actually correct. In conclusion, the majority of the participants are convinced about the generated change descriptors for the given evolution scenario.

19

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

Finding F2: The majority of participants state that the generated change descriptors for the given evolution scenario are convincing.

Result Convincingness

(C10-0)

evidence (rel.)

convincing partially convincing difficult to judge

(C100A) (C100B) (C100C)

33 3 6

(79%) (7%) (14%)

no. particip. 9 1 4

Table 6: Extracted codes with absolute and relative evidence, and number of participants who gave evidence, for code category C100 Result Convincingness (RQ3.1). Table 7 lists codes for code category C10-1, about whether participants approve our procedure to generate change descriptors (Procedure Approval), again referring to Question Q10. As described before, two participants (with 21% evidence) did not specify atomic mappings at all, thus gave reasons for not approving our procedure to generate change descriptors (code C10-1C). In contrast, we found evidence (46%) from two thirds of participants that they approve the procedure in general (code C10-1A). However, five participants (with 33% evidence) also mentioned possible conditions to the procedure (code C10-1B): In detail, three participants mentioned that the delta operations need to be less abstract, to be understandable by mechanical engineers, and to be usable for describing change in general. As another condition, two participants mentioned that the delta operations need to be related to a reference system, so that it is traceable if and how the change occurred before already, within the project or the company. In conclusion, most participants approve our current, or an extended procedure to generate change descriptors. Finding F3: Participants approve our procedure to generate change descriptors, but also mention conditions to the procedure to be applicable.

Procedure Approval

(C10-1)

evidence (rel.)

approved conditionally approved not approved

(C101A) (C101B) (C101C)

11 8 5

(46%) (33%) (21%)

no. particip. 8 5 2

Table 7: Extracted codes with absolute and relative evidence, and number of participants who gave evidence, for code category C101 Procedure Approval (RQ3.1). Table 8 lists codes for code category C20-0, about whether participants think our description approach is supportive in model-based systems engineering (Approach Supportiveness), referring to Question Q20. All twelve participants (with 31% evidence) stated that our description approach is supportive for indicating, documenting or archiving change (code C20-0A), meaning supportive for making change explicitly visible. Around a third of the participants mentioned two other support possibilities: first, for comparing different system development alternatives (code C20-0B, with 6% evidence), for example, which alternative has a higher or lower share of changed modelling elements, and hence a higher or lower development risk [1]; second, for synthesis of change (code C20-0C, with 4% evidence), for example, for developing changes below an upper boundary for the proportion of PVs, hence limiting development risk [1]. As extracted in code C20-0D, most of the participants (with 53% evidence) explained seven possible extensions, to make change descriptors more supportive: 1. Change descriptors could be contextualized, for example with an assessment, such as a traffic light system, or an follow-up action, such as a meeting after a change with high share of PVs. 2. Change descriptors could relate the described change to a reference system or, in general, a repository for already known changes. An analogue extension was mentioned for the mapping specifications already (code C10-2B), where some participants asked to relate single delta operations to reference systems. 3. Change descriptors could be aggregated so that they describe change on a higher level than on delta operations. For example, delta DHeatable of the subject system could be described with "[...] component added, function added, [and] PV or AV in tendency" (Participant 12). We argue, however, that the inclusion of modelspecific elements in a change descriptor, such as "component" from CAMs, reduces independence from the model itself, making the change descriptor less versatile to be used for interdisciplinary communication. 20

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

4. Five participants asked for explicit information on the share of CVs. In our use of the description model of SGE, this translates to the share of modelling elements which were not changed, and which is given only implicitly. 5. Change descriptors could indicate how distributed changes are in a model, for example whether they modify only a single element, or modify nearly all elements. 6. One participant mentioned that the whole description approach could be extended so that an engineer could query existing changes with dedicated in-depth questions on demand. 7. One participant pointed out that the "quality of the data", as given in our subject system, is "available only in the end of a project" (Participant 11), thus our description approach should also be able to cope with lower quality models. Independent of what "model quality" actually means, we argue that changes to a model, formalised with delta operations, are applicable as long as the model is syntactically correct, meaning a valid instance of its meta-model. So, even if the model still is in early development, we can generate change descriptors for every change, as long as the model is syntactically correct. As last code of code category C20-0, three participants (6% evidence) emphasized that our description approach serves as a foundation of technical support (C20-0E), for managing changes in model-based systems engineering per-se. Concluding code category C20-0, all participants see our description approach supportive for indicating, documenting or archiving change, and also give diverse possible extensions to further enhance supportiveness. Finding F4: Participants see our description approach mostly supportive for indicating, documenting or archiving change while stating diverse extensions to enhance its supportiveness.

Approach Supportiveness

(C20-0)

evidence (rel.)

no. particip.

supp. for indicating, documenting, archiving change supp. for comparing different system development alternatives supportive for synthesis of change conditionally supportive as foundation for technical support

(C20-0A)

26

(31%)

12

(C20-0B)

5

(6%)

4

(C20-0C) (C20-0D) (C20-0E)

3 45 5

(4%) (53%) (6%)

3 11 3

Table 8: Extracted codes with absolute and relative evidence, and number of participants who gave evidence, for code category C20-0 Approach Supportiveness (RQ3.2). Table 9 lists codes for code category C21-0, about whether participants see potential of our description approach to be used in automated system development processes without manual expert intervention (Automation Potential), referring to Question Q21. Two participants (with 35% evidence) mention use of our description approach for automation in development within constraints (code C21-0A). For example, if the proportion of PVs of two development alternatives differs strongly from each other, then the decision for one the two development alternatives may be automated. This relates highly to the supportiveness of our description approach for comparing development alternatives in general (see code C20-0B). Again two participants (with 18% evidence) see automation potential of our description approach if it would additionally give verbose reasoning behind its decisions (code C21-0B). In contrast to codes C21-0A and C21-0B, four participants (with 47% evidence) could not see any use of our description approach in automated system development processes (C21-0C), and individually gave four reasons: First, two participants generally assess the impact of automated decisions as too high. Second, one participant would make decisions not solely based on quantitative metrics, such as the share of changed modelling elements. Third, one participant misses clear project management responsibilities if decisions would be made automatically. Fourth, one participant sees human creativeness in development processes as not negligible to be replaced. In conclusion, the participants do not see clear potential of our description approach to be used in automated system development processes. Finding F5: Participants are mostly cautious with respect to an automated use of our description approach in system development processes.

21

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

Automation Potential

(C21-0)

evidence (rel.)

no. particip.

usable for automation within constraints conditionally usable for automation not usable for automation

(C21-0A)

6

(35%)

2

(C21-0B)

3

(18%)

2

(C21-0C)

8

(47%)

4

Table 9: Extracted codes with absolute and relative evidence, and number of participants who gave evidence, for code category C21-0 Automation Potential (RQ3.2).

4.4.7

Discussion

Regarding RQ3.1 on plausibility, we found evidence that the participants have a common understanding of applying the description model of SGE for specifying atomic mappings (Finding F1), that they are convinced of the generated change descriptors (Finding F2), and that they approve the procedure of our description approach (Finding F3). Thus, we conclude that generated change descriptors, and the procedure to generate them, are plausible. Regarding RQ3.2 on practical applicability, we found strong evidence that the participants think our description approach is supportive in model-based systems engineering (Finding F4), especially for indicating, documenting, and archiving change. In contrast to supportiveness, we found only few evidence for automation potential of our description approach (Finding F5), mostly argued by limits in its current status. Only few constructive suggestions for enhancing the automation potential were given. Thus, we conclude that our description approach is practically applicable mostly in descriptive manners, and only with uncertainties in automated system processes. Regarding RQ3.3 on extensibility, we found strong evidence that our description approach could generally be more contextualised. Specifically, delta operations could be more embedded into concrete models, to be less abstract and more understandable by engineers (Finding F3, code C10-1B), enhancing approval of our procedure to generate change descriptors. Change descriptors itself could be more related to reference systems (Finding F3, code C10-1B; Finding F4, code C20-0D), for example, to trace changes within the engineering project, and more related to risk assessments, such as traffic light systems (Finding F4, code C20-0D), enhancing supportiveness. Furthermore, we found evidence that the share of CVs could be described explicitly, and not only implicitly (Finding F4, code C20-0D). For several other extension possibilities mentioned, no clear tendency to one of the extension possibilities is obvious. Thus, we conclude that plausibility and practical applicability, mostly referring to supportiveness, of our description approach are open for concrete extensions, while the enhancement of automation potential could not be constructively detailed out in this user study. 4.5

Threats to Validity

We discuss possible threats to the validity of our mixed-method evaluation along the three RQs, because aspects of validity and quality differ between quantitative and qualitative evaluation methods. 4.5.1

RQ1: Technical Feasibility / Implementation (Quantitative)

For our quantitative evaluation on technical feasibility (RQ1), we discuss internal and external validity of our implementation, the cps-communicator framework. Internal Validity As a threat to internal validity, we identified possible errors in our generic implementation. The generic implementation reads in deltas and delta operations, processes them iteratively with provided mappings, and writes out results of mappings. However, it does not contain explicit specifications of atomic and composed mappings, which are part of the application, shown in RQ2 on technical applicability. Thus, we assume that code complexity may be introduced in the application earliest, not in the generic implementation. Furthermore, we applied best software engineering practices, to mitigate possible errors, in our implementation. External Validity We identified one threat to external validity. While mapping specifications and informal languages of change are concern of the application, our concept is fundamentally built upon delta modelling as formal language of change. We argue that this does not limit the generalisability of our implementation, because it is possible to create a delta dialect for all models that fundamentally consist of modelling elements and relations between modelling 22

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

elements. Even if tool support for delta modelling (e.g. DeltaEcore [37]) is not applicable, a delta dialect may be created manually. 4.5.2

RQ2: Technical Application (Quantitative)

For our quantitative evaluation on technical applicability (RQ2), we discuss internal validity, external validity and reproducibility of our application to the BCS case study. Internal Validity We identified two threats to internal validity, regarding our subject system, the BCS case study: (1) The BCS case study originally represents a system with variability in space (configurability), but we used it in the context of variability in time (evolution). Specifically, we took core model variants as initial model versions, and variant deltas as version deltas. However, according to Seidl [37], both dimensions of variability can be used interchangeably, if not both at the same time. Thus, we see no limitations in the understanding of variant deltas as version deltas. (2) We used only a subset of all possible product configurations from the BCS case study for product evolution scenarios, and only a reduced delta dialect for atomic mapping specifications. However, using all possible product configurations and the full delta dialect would not come with any further advantages for concluding the evaluation on technical applicability, as we could show technical applicability already with the subset of product configurations and the reduced delta dialect. External Validity We identified one threat to external validity, specifically to generalisability: In the motivation of our work, we claimed that our description approach is applicable in a model-based engineering context, and also brought a running example from the SysML/UML world. In the BCS case study, we then used our own EMF metamodels for CAMs and SMMs, and not SysML or UML derivatives. However, as this research question concerns technical applicability, we argue that working with real SysML or UML meta-models should technically be equivalent, as there exists tool support for these modelling languages based on EMF, for example Eclipse Papyrus [29]. In conclusion, we therefore see the existence of change descriptors for our own EMF-based models as direct evidence for technical applicability. Reproducibility For reproducibility of this evaluation, we provide our used subject system in a Zenodo package [30], and the code base of the cps-communicator framework and its application in RQ2 publicly on GitHub13 . Thus, the application results should be reproducible. 4.5.3

RQ3: User Study (Qualitative)

For our qualitative evaluation on plausibility, practical applicability and extensibility, we discuss relevant quality criteria for qualitative methods found in literature [33]; namely internal validity, construct validity, reflexivity, credibility, and transferability. Internal Validity We identified two threats to internal validity in our user study: (1) The selected evolution scenario, as subject system, may not be suitable for the selected participants. However, we selected this evolution scenario, as extensively described in the according section, because we assumed that participants from mechanical engineering are able understand CAMs as well as the changes represented in the evolution scenarios. We further gave them an introduction on CAMs and explained the evolution scenario to them in detail. (2) Data analysis, meaning extraction of relevant segments, codes and code categories from the transcribed answers of our participants, may be subjective. To mitigate subjective influence, we used multiple eye principle among the authors for data analysis. Construct Validity We argue that we actually evaluate plausibility of change descriptors and the procedure to generate them, because we explicitly asked the participants thereof (Question Q10), because we let them explain in detail why, and because we could cover both parts, the procedure to generate change descriptors (code category CC10-0) as well as the resulting change descriptors (code category CC10-1), in our findings. We argue that we actually evaluate practical applicability of our description approach, because, again, we explicitly asked the participants thereof (Question Q20, Question Q21), and because we assume our participants have the necessary background in their research field to be able to assess practical applicability in context of the description model of SGE. We argue that we actually evaluate extensibility, because we collected mentioned extension possibilities from the participants answers in an exploratory manner. 13

cps-communicator on GitHub: https://github.com/KIT-TVA/cps-communicator

23

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

Reflexivity / Interactions & Biases The sessions with the participants were done by one single author (interviewer) from the field of software engineering. As our participants are from the field of mechanical engineering, few to no prior interaction between interviewer and participants happened. Furthermore, the interviewer, from software engineering, was not biased in the participants research field, mechanical engineering. For data analysis, biases towards individual participants were mitigated by first transcribing all sessions, then anonymising transcriptions, then analysing transcriptions. Credibility of Participants We argue that our participants are credible, because we designed the procedure of our user study the way that each participant has the ability to understand the research rationale (introduction phase), the subject system (introduction phase), the description approach (specification phase and application phase), with its procedure and its results, before the interview. Thus, we expect that the participants gave credible answers in the interviews itself. Furthermore, we expect that participants gave honest answers in the interviews, because of the discussed low interaction and bias between interviewer and participant. Transferability of Findings We argue that our results are transferable to other participants from the research field of the description model of SGE. We further argue that our results are transferable to a study design where an other model-independent language than the description model of SGE is used, as long as the original intentions of modelindependent languages are comparable, meaning description and analysis of development of cyber-physical systems [2]. We see a mediocre transferability to other subject systems, as this depends highly on the prior knowledge of each participant, for example which other models, than CAMs, the participant knows and how well he could understand the presented evolution scenario.

5

Related Work

In this chapter, we present existing work from the literature, related to change description, and take both the theoretical and practical side of change description into account. Thus, we address contributions to (1) formal notions of change, to (2) semantic analysis of changes, and (3) to informal change description. Formal Notions of Change In related work on formal notions of change, change can be understood in various dimensions. Besides change in time, i.e. evolution, approaches also address change as unit of difference between variants in product lines [8]. This relates strongly to the original background of the notion of change we use in our description approach, Delta Modelling [36]; and more generally, to two-dimensional variability in space (variants) and time (evolution), i.e. product line evolution. Seidl et al. [38] proposed Hyper Feature Models, where features are versioned individually, together with a formal constraint language to analyse configurations for validity. Nieke et al. [27, 28] proposed Temporal Feature Models, where versions of feature models are related to each other for evolution traceability and planning. Michelon et al. [26] proposed a framework to specify and compose variants in evolving product lines, where features are versioned with revisions. Lity et al. [20] proposed Higher-Order Delta Modelling, where delta models are evolved by higher-order deltas, based on addition, modification, and deletion of variant-deltas. As a combined problem and solution space model, Ananieva [5] proposed the Unified Conceptual Model, where revisions of features and system in the problem space are mapped to implementation fragments in the solution space. However, all mentioned approaches consider modelling of changes for different purposes and modelling languages, but do not aim at describing those changes in a model-independent language. Semantic Analysis of Changes Differencing methods extract changes between two modelling artefacts, for example the versioning tool Git14 on line-based artefacts, on a syntactical level. But even if two modelling artefacts differ on syntactical level, they may describe the same system, thus do not differ semantically. Semantic change analyses propose differencing procedures based on semantic rather than syntactic change, and therefore have a similar aim as our description approach, to extract semantics behind changes. For instance, semantic differencing approaches have been proposed for UML class models with the Object Constraint Language (OCL) by Rumpe et al. [35] and Maoz et al. [23], for feature models by Drave et al. [10], and for code artefacts by Jackson and Ladd [13]. However, these approaches contribute to differencing of specific modelling artefacts, by analysing if changes concern semantics of a model itself, instead of semantically describing change, as our description approach does. For semantically describing change, approaches have been proposed for UML models, by Briand et al. [6], where a change to a model is enriched by attributes, such as concerned modelling elements, and a textual representation of the change itself. Kehrer et al. [18] proposed semantic change descriptions for EMF models, with Semantic Lifting, where patterns are dynamically detected for atomic sets of change operations to a model. Parnin and Goerg [32] proposed 14

Git: https://git-scm.com

24

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

semantic change descriptions for bytecode models, where changes to bytecode are enriched by attributes, such as location of occurrence and involved data types, and then analysed with bytecode analysis techniques. Ma et al. [22] proposed semantic change descriptions for physical models of mechanical systems, where a graph structure based on ontologies is built from physical parts and their attributes, and a change to physical parts is analysed along the graph structure. More generally for ontologies, Javed et al. [15] proposed a process where change patterns are detected in order to provide semantics behind changes. However, although these approaches describe changes semantically, they are mostly limited to description languages which can not be used interchangeably nor are explicitly modelindependent, which is the goal in our description approach. Informal Change Descriptions In related work on informal change descriptions, the term change is used in context of natural language, i.e. less as technical term, what poses difficulties finding approaches comparable to the informal language of change we use in our description approach, the description model of SGE [2]. While the field of Engineering Change Management essentially provides methods and frameworks for handling changes in system development [41, 14], it rather addresses business processes, which is not the scope of our work. The informal language of change we use in our description approach, the description model of SGE, uses a classification of reference systems, proposed by Albers et al. [4]. For a more fine-grained classification of reference systems, further dimensions of variations, besides the variation type (AV, PV, CV), have been proposed by Martin et al. [24]. The authors propose change complexity, change novelty, and prior knowledge of the reference system as further dimensions for variations. This classification is integrated into the authors risk assessment framework [25], so that a higher change complexity, higher change novelty and less prior knowledge of the reference system is understood to introduce more development risk. However, the mentioned approaches consider descriptive extensions or practical uses of the description model of SGE, but do not address automated generation of change descriptions based on specified mappings from formal notions of change. Summary In summary, existing approaches in the literature either focus on the modelling side (formal modelling languages), or on practical side (informal change descriptions) of describing changes, and do not connect both sides with the intention of advancing communication of changes between different engineering disciplines in a multi-domain engineering environment, as our description approach does. Furthermore, the listed work is not evaluated methodologically to connect the theoretical modelling side to the practical side of describing changes, to the extent in which we evaluated our work; usually, only one side is evaluated, for example conceptual soundness on the theoretical modelling side, and usability in industry on the practical side.

6

Conclusion & Future Work

In this paper, we presented an approach for describing model-specific changes in model-based engineering projects with model-independent languages. By combining a formal notion of change with informal descriptions of change, we bridged the gap between technical change representations and human-interpretable notions. We realised our approach by specifying mappings for translating deltas into change descriptors. In a mixed-methods evaluation, reflecting the translation from theory to practice, we have shown that our description approach is technically realisable and applicable, and that change descriptors are plausible and practically applicable. In the evaluation, we have shown that our description approach is technically feasible, by implementing a generic framework, called cps-communicator, which provides functionality to specify and apply mappings to deltas and delta operations. The cps-communicator framework enables a versatile use, as model-independent language and mappings can be specified and then applied to concrete models with loose restrictions. Furthermore, we have quantitatively evaluated technical applicability of our description approach, by applying the cps-communicator framework to two concrete model types, CAMs and SMMs, from the existing BCS case study. Essentially, we specified 28 mappings and applied them 16092 times overall, showing technical applicability even for larger models. We have qualitatively evaluated plausibility, practical applicability, and extensibility, by conducting a user study with twelve participants with varying degrees of experience. Our findings show that most participants understand and approve our description approach (plausibility), and see a supportive use, especially in indicating, documenting, and archiving changes (practical applicability). Based on these evaluation results, we are convinced that extensions to our description approach advance its practical applicability even more. We identified two major directions for future work from the findings of our user study. Our first idea is to better contextualise changes into the model and its environment, in which changes occur. Thus, we aim to embed change descriptors deeper in a project context, for example by taking change propagation across multiple models into account (e.g. with multi-model consistency [34, 39]), or by taking affected product variants of a product line into account (e.g. with delta-oriented product lines [36, 20]). Second, we aim to improve the automation potential of our description 25

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

approach, by extending the cps-communicator framework with a more user-friendly interface, and with a more accessible backend. The latter could provide technical interfaces used by automated agents, for example to automatically detect model changes (e.g. with differencing techniques [18, 38]) and describe them accordingly.

Acknowledgements This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) SFB 1608 501798263. Thanks to our textician. Thanks to Lennart Rak.

References [1]

[2]

[3]

[4]

[5] [6] [7] [8] [9]

[10]

[11]

[12] [13]

[14] [15]

Albert Albers, Nikola Bursac, and Eike Wintergerst. “Product generation developmentimportance and challenges from a design research perspective”. In: New developments in mechanics and mechanical engineering 13 (2015), pp. 16–21. Albert Albers and Simon Rapp. “Model of SGE: System Generation Engineering as Basis for Structured Planning and Management of Development”. In: Design Methodology for Future Products: Data Driven, Agile and Flexible. Ed. by Dieter Krause and Emil Heyden. Cham: Springer International Publishing, 2022, pp. 27–46. ISBN : 978-3-030-78368-6. DOI : 10.1007/978-3-030-78368-6_2. Albert Albers et al. “Entwicklung nachhaltiger Systeme und Produkte Kontinuierliche und zirkuläre Systemevolution mit dem Modell der SGE Systemgenerationsentwicklung”. In: Zukunftsgestalter Deutschlands: Pioniergeschichten aus Wirtschaft, Wissenschaft, Politik und Zivilgesellschaft. Ed. by Marion A. WeissenbergerEibl. Berlin, Heidelberg: Springer Berlin Heidelberg, 2025, pp. 97–119. ISBN: 978-3-662-70324-3. DOI: 10. 1007/978-3-662-70324-3_6. Albert Albers et al. “The Reference System in the Model of PGE: Proposing a Generalized Description of Reference Products and their Interrelations”. en. In: Proceedings of the Design Society: International Conference on Engineering Design 1.1 (July 2019), pp. 1693–1702. ISSN: 2220-4342. DOI: 10.1017/dsi.2019.175. Sofia Ananieva. “Consistent View-Based Management of Variability in Space and Time”. PhD Thesis. Karlsruhe Institute of Technology, Germany, 2022. Lionel Briand, Yvan Labiche, and L. O’Sullivan. “Impact analysis and change management of UML models”. In: Oct. 2003, pp. 256–265. ISBN: 0-7695-1905-9. DOI: 10.1109/ICSM.2003.1235428. Dave Clarke, Michiel Helvensteijn, and Ina Schaefer. “Abstract delta modeling”. In: SIGPLAN Not. 46.2 (Oct. 2010). Place: New York, NY, USA, pp. 13–22. ISSN: 0362-1340. DOI: 10.1145/1942788.1868298. Paul Clements and Linda Northrop. Software product lines: practices and patterns. The SEI series in software engineering. Boston: Addison-Wesley, 2002. ISBN: 978-0-201-70332-0. Ferruccio Damiani and Michael Lienhardt. “Refactoring Delta-Oriented Product Lines to achieve Monotonicity”. In: Electronic Proceedings in Theoretical Computer Science 206 (Mar. 2016), pp. 2–16. ISSN: 2075-2180. DOI : 10.4204/eptcs.206.2. Imke Drave et al. “Semantic Evolution Analysis of Feature Models”. In: Proceedings of the 23rd International Systems and Software Product Line Conference - Volume A. SPLC ’19. New York, NY, USA: Association for Computing Machinery, 2019, pp. 245–255. ISBN: 978-1-4503-7138-4. DOI: 10.1145/3336294.3336300. Stuart E. Dreyfus and Hubert L. Dreyfus. A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition: tech. rep. Fort Belvoir, VA: Defense Technical Information Center, Feb. 1980. DOI: 10.21236/ ADA084551. Christa Hoffmann-Riem. Das adoptierte Kind: Familienleben mit doppelter Elternschaft. ger. 4th ed. Übergänge 8. München: Fink, 1998. ISBN: 978-3-7705-2248-4. Daniel Jackson and David A. Ladd. “Semantic Diff: A Tool for Summarizing the Effects of Modifications”. In: Proceedings of the International Conference on Software Maintenance. ICSM ’94. USA: IEEE Computer Society, 1994, pp. 243–252. ISBN: 0-8186-6330-8. T. A. W. Jarratt et al. “Engineering change: an overview and perspective on the literature”. In: Research in Engineering Design 22.2 (Apr. 2011), pp. 103–124. ISSN: 1435-6066. DOI: 10.1007/s00163-010-0097-y. Muhammad Javed, Yalemisew M. Abgaz, and Claus Pahl. “Ontology Change Management and Identification of Change Patterns”. In: Journal on Data Semantics 2.2 (June 2013), pp. 119–143. ISSN: 1861-2040. DOI: 10.1007/s13740-013-0024-2. 26

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

[16] R. Johnson, Anthony Onwuegbuzie, and Lisa Turner. “Toward a Definition of Mixed Methods Research. Journal of Mixed Methods Research, 1, 112-133”. In: Journal of Mixed Methods Research 1 (Apr. 2007), pp. 112–133. DOI : 10.1177/1558689806298224. [17] Erik Kamsties et al. “Detecting ambiguities in requirements documents using inspections”. In: Proceedings of the first workshop on inspection in software engineering (WISE01). Vol. 13. 2001. [18] Timo Kehrer, Udo Kelter, and Gabriele Taentzer. “A rule-based approach to the semantic lifting of model differences in the context of model versioning”. In: 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). 2011, pp. 163–172. DOI: 10.1109/ASE.2011.6100050. [19] John C. Knight. “Challenges in the utilization of formal methods”. In: Formal Techniques in Real-Time and Fault-Tolerant Systems. Ed. by Anders P. Ravn and Hans Rischel. Berlin, Heidelberg: Springer Berlin Heidelberg, 1998, pp. 1–17. ISBN: 978-3-540-49792-9. [20] Sascha Lity, Matthias Kowal, and Ina Schaefer. “Higher-order delta modeling for software product line evolution”. In: Proceedings of the 7th International Workshop on Feature-Oriented Software Development. FOSD 2016. New York, NY, USA: Association for Computing Machinery, 2016, pp. 39–48. ISBN: 978-1-4503-4647-4. DOI : 10.1145/3001867.3001872. [21] Sasha Lity et al. Delta-oriented Software Product Line Test Models The Body Comfort System Case Study. Tech. rep. 2012-07. Technische Universität Braunschweig, 2012. [22] Songhua Ma, Zhaoliang Jiang, and Wenping Liu. “A design change analysis model as a change impact analysis basis for semantic design change management”. In: Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 231.13 (2017), pp. 2384–2397. DOI: 10 . 1177 / 0954406216633035. [23] Shahar Maoz, Jan Oliver Ringert, and Bernhard Rumpe. “CDDiff: Semantic Differencing for Class Diagrams”. In: ECOOP 2011 Object-Oriented Programming. Ed. by Mira Mezini. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 230–254. ISBN: 978-3-642-22655-7. [24] Alex Martin et al. “A Model-Based Approach to Analyze Change Propagation and Impact in ProductProduction-CoDesign”. In: Procedia CIRP 138 (2026), pp. 274–279. ISSN: 2212-8271. DOI: https://doi. org/10.1016/j.procir.2026.01.048. [25] Alex Martin et al. “Communication Approach for the Model-Based Engineering Change Management”. In: Procedia CIRP 136 (2025), pp. 734–739. ISSN: 2212-8271. DOI: https://doi.org/10.1016/j.procir. 2025.08.125. [26] Gabriela Karoline Michelon et al. “Evolving software system families in space and time with feature revisions”. In: Empirical Software Engineering 27.5 (May 2022), p. 112. ISSN: 1573-7616. DOI: 10.1007/s10664-02110108-z. [27] Michael Nieke. “Consistent Feature-Model Driven Software Product Line Evolution”. In: SoftwaretechnikTrends Band 42, Heft 2 (2022). ISSN: 0720-8928. [28] Michael Nieke et al. “Guiding the evolution of product-line configurations”. In: Software and Systems Modeling 21.1 (Feb. 2022), pp. 225–247. ISSN: 1619-1374. DOI: 10.1007/s10270-021-00906-w. [29] Eric Aristhide Nyamsi. “Eclipse Papyrus Framework”. In: Programming4Modeling: Codes in Modellen auf Basis von Java und UML. Wiesbaden: Springer Fachmedien Wiesbaden, 2025, pp. 331–432. ISBN: 978-3-65847333-4. DOI: 10.1007/978-3-658-47333-4_3. [30] Philip Ochs. inter-disciplinary-description-of-changes-in-model-based-engineering. May 2026. DOI: 10 . 5281/ZENODO.20444806. [31] Sebastian Oster et al. “Pairwise feature-interaction testing for SPLs: potentials and limitations”. In: Proceedings of the 15th International Software Product Line Conference, Volume 2. SPLC ’11. New York, NY, USA: Association for Computing Machinery, 2011. ISBN: 978-1-4503-0789-5. DOI: 10.1145/2019136.2019143. [32] Chris Parnin and Carsten Görg. “Improving change descriptions with change contexts”. In: Proceedings of the 2008 International Working Conference on Mining Software Repositories. MSR ’08. New York, NY, USA: Association for Computing Machinery, 2008, pp. 51–60. ISBN: 978-1-60558-024-1. DOI: 10.1145/1370750. 1370765. [33] Paul Ralph et al. Empirical Standards for Software Engineering Research. _eprint: 2010.03525. 2020. [34] Ralf Reussner et al. “Consistency in the View-Based Development of Cyber-Physical Systems (Convide)”. In: 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C). 2023, pp. 83–84. DOI: 10.1109/MODELS-C59198.2023.00026. 27

Improving Communication of Changes in Model-Based Engineering

A P REPRINT

[35] Bernhard Rumpe et al. “Semantic Difference Analysis with Invariant Tracing for Class Diagrams Extended by OCL”. In: Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems. MODELS Companion ’24. New York, NY, USA: Association for Computing Machinery, 2024, pp. 1066–1075. ISBN: 979-8-4007-0622-6. DOI: 10.1145/3652620.3687818. [36] Ina Schaefer. “Variability Modelling for Model-Driven Development of Software Product Lines.” In: VaMoS 10 (2010), pp. 85–92. [37] Christoph Seidl, Ina Schaefer, and Uwe ASSmann. “DeltaEcore-A Model-Based Delta Language Generation Framework”. In: Modellierung 2014. Bonn: Gesellschaft für Informatik e.V., 2014, pp. 81–96. ISBN: 978-388579-619-0. [38] Christoph Seidl, Ina Schaefer, and Uwe ASSmann. “Integrated management of variability in space and time in software families”. In: Proceedings of the 18th International Software Product Line Conference - Volume 1. SPLC ’14. New York, NY, USA: Association for Computing Machinery, 2014, pp. 22–31. ISBN: 978-1-45032740-4. DOI: 10.1145/2648511.2648514. [39] George Spanoudakis and Andrea Zisman. “Inconsistency Management in Software Engineering: Survey and Open Research Issues”. In: Dec. 2001, pp. 329–380. DOI: 10.1142/9789812389718_0015. [40] Technical Operations International Council on Systems Engineering (INCOSE). INCOSE Systems Engineering Vision 2020. INCOSE-TP-2004-004-02 2.03. 2007. [41] I. C. Wright. “A review of research into engineering change management: implications for product design”. In: Design Studies 18.1 (1997), pp. 33–42. ISSN: 0142-694X. DOI: https://doi.org/10.1016/S0142694X(96)00029-4.

28