arXiv:2604.24373v1 [cs.SE] 27 Apr 2026
Exploring Creativity in Human–Human-LLM Collaborative Software Design Victoria Jackson
Grischa Liebel
[email protected] University of Southampton Southampton, UK
[email protected] Reykjavik University Reykjavik, Iceland
Rafael Prikladnicki
André van der Hoek
[email protected] Pontifícia Universidade do Rio Grande do Sul Porto Alegre, RS, Brazil
[email protected] University of California, Irvine Irvine, USA
Abstract While the use of Large Language Models (LLMs) in programming has been extensively studied, there is limited understanding of how LLMs support collaborative work where creativity plays a central role. Software design, as a collaborative and creative activity, provides a valuable context for exploring the influence of LLMs on creativity. This study investigates how and where creativity naturally emerges when software designers collaborate with an LLM during a design task. In a laboratory setting simulating a workplace environment, 18 pairs of software professionals with design experience were asked to complete a design task. Each pair had 90 minutes to produce a software design based on a set of requirements, with optional access to a custom LLM interface. Pairs were not primed to be creative. We find that creativity was present in all pairs in design processes, with 13 producing design documents containing creativity. We primarily attribute creativity to the human designers, driven by traits such as prior experience, empathy, and the use of analogies. The LLM contributed by producing novel ideas and elaborating human ideas. However, in some cases, the LLM appeared to hinder creativity by suggesting complex solutions or adding to unproductive digressions. LLMs can support creativity in collaborative software design, but human insights remain central. To effectively augment human creativity, designers must be intentional in their engagement with LLMs.
CCS Concepts • Software and its engineering → Designing software; • Humancentered computing → Empirical studies in collaborative and social computing.
Keywords Software design, LLM, Generative AI, creativity
1
Introduction
Large Language Models (LLMs) have been adopted broadly by software engineers due to their ability to speed up and improve various aspects of software engineering (SE) activities. To date, efforts have mainly focused on using LLMs for software development and maintenance tasks, with the majority addressing generation and classification problems [17]. Results indicate clear benefits of LLMs, e.g., in developer productivity [41, 54].
In many SE-related tasks, such as writing code, testing, or designing, software engineers engage in creative problem solving activities [4]. That is, they do not only seek functional solutions, but additionally ideas or artifacts that are new, surprising, and valuable [2]. Thus, this focus on creative problem solving can have an important impact on contributing novel and valuable features to software products [34, 35]. As a result, substantial research has focused on creativity in various aspects of SE (e.g., [11, 13, 25, 40, 43]). While the potential of using LLMs for creative activities has been picked up in existing research (e.g., [12, 21, 49]), only a few recent studies focus on this topic within SE (e.g., [9, 20, 36, 53]). This motivates a need for further research on using LLMs for creativity in SE [24]. Representing an important part of SE practice, collaborative activities are commonplace [16]. Existing research has shown how LLMs can support collaborative work in other domains, including for creative activities [14, 15, 31]. However, integrating LLMs in collaborative, creative SE activities has not yet been studied. To address this gap, we investigate how and where creativity emerges naturally in collaborative software design sessions involving pairs of software professionals with access to an LLM. Our research question (RQ) is thus: How, and where, does creativity appear naturally when designing with an LLM? We conducted a laboratory study simulating a remote workplace environment with 18 pairs of software professionals. Pairs produced software designs based on an initial set of requirements and had access to an LLM interface that they could choose to use should they wish to. Creativity appeared in the process of all 18 pairs, which ultimately led to 13 of the pairs producing a design document that contained creative elements, showing that creativity emerges naturally, even when pairs are not primed to be creative, e.g., as in hackathons [9]. We attribute this creativity primarily to the software engineers, though the LLM contributed by producing some novel ideas and by elaborating on the input provided by the engineers. Notably, we observe that LLM use hindered creativity in several cases as well.
2 Background and Related Work 2.1 Creativity Definitions While there are many definitions of creativity [47], we use Boden’s definition [2], “Creativity is the ability to come up with ideas or artifacts that are new, surprising and valuable”. This definition
Jackson, Liebel, Prikladnicki, van der Hoek
reflects the creative process and the resulting product, recognizing that creative (new, surprising, valuable) products or features do not emerge without some sort of creative activities feeding it. Our research seeks to understand both, and particularly uses Boden’s perspective of new, surprising, and valuable as the lens to identify creativity in the product (software design). To identify creativity in the design process, our study uses Dorst and Cross’s concept of a “creative event” as an analytical lens. In studying industrial designers, they observed that designers would commence exploring the problem space, frame a small part of the problem space and, from this, make a leap to a small part of the solution space to extend it in some way before returning to the problem space and further considering the problem. Thus, the problem and solution spaces co-evolve [33]. When a leap is more than a routine back-and-forth, and leads to the emergence of a novel concept that is visible in the resulting design, the moment of insight when the problem and solution spaces are framed leading to the identification of the bridge is considered by Dorst and Cross as a “creative event” [5]. This concept of a creative event has been used to explore creativity in software design previously [25] and is the one we adopt in this study also (see Section 3). Creative expression and experiences vary across individuals. Recognizing that individual creativity exists on a continuum, Kaufman and Beghetto categorize creativity on four levels in their 4C model [26]. “mini-c” is personal creativity that is meaningful to the individual, “little-c” is everyday creativity often used in problem solving and perhaps only recognized by the individual, “pro-C” is professional creativity requiring domain expertise and recognized by others, and “Big-C” creativity is groundbreaking leading to major innovations in a field. We refer to the 4C model in the final discussion to contextualize some of our findings.
2.2
Creativity in Software Design
Although software design has long been considered creative [4], there is little research on creativity within software design. Two papers [38, 40] illustrate how framing requirements to give the impression of certainty inhibits the creativity of designers tasked with producing a User Interface design concept. The first [40] finds that prioritized requirements led to less original but more practical designs than those resulting from requirements framed as ideas. The second [38] finds that fixation and reduced critical thinking ensue when designers are presented with more formalized requirements. The combination of the findings of the two papers suggests that, if more innovative solutions are needed, requirements should be framed as ideas, with the resulting ambiguity forcing designers to explore both the problem and solution space. In a study examining the effect of physical distance between partners on creativity in the software design process, Jolak et al. [25] find that distance does not make a difference to creativity. Specifically, the authors use creative events (as described above) to analyze creativity in the design processes of pairs of software designers, and find no difference in the presence and number of creative events between co-located and remote pairs. While our study also uses creative events as an analytic lens into creativity in the design process, it differs in that it also examines creativity in the creative product (the software design).
Finally, in reflecting on three decades of research on high-performing software teams, it was noted that experienced designers use contrasts and specific design moves to spark creativity in software development [43]. An example of such a design move is to identify structural resonances between the problem in hand and similar problems or solutions. Moreover, experienced designers exhibit a design creativity mindset, such as practicing reflection. Our study complements these prior studies on software design by making an LLM available to designers, thus enabling an exploration of how creativity emerges with and without the help of the LLM.
2.3
Generative AI and Creativity
Prior studies on GenAI in various software development contexts have found that LLMs are not particularly used for creative tasks in practice. A study on creativity in start-ups found that LLMs had limited use for creative work [11], while another on the adoption of LLMs by software teams notes that GenAI was little used in tasks often considered as requiring creativity, such as design [42]. In contrast, several studies have shown how LLMs, and GenAI more broadly, could in theory assist with creativity in software engineering. One study explores the use of GenAI to trigger creativity within application designers [53] when designing new user interfaces. A second makes use of an approach that combines Natural Language Processing techniques and an LLM to identify novel candidate product features from existing project information [36], while a third argues that introducing “seams” into the user interfaces of AI-powered developer tools can foster creativity by exposing underlying complexity rather than hiding it (and thus forcing developers to consider the problem as much as the solution). We contend that additional approaches need to be explored before results may trickle to practice. Speculating that creativity will be more important than ever in a world dominated by GenAI fuels a broader research agenda for exploring the impact of GenAI on creativity within software engineering [24]. The research agenda identifies five themes for future research: creativity as it pertains to: (1) the individual, (2) team, (3) product, (4) unintended consequences, and (5) wider society. By exploring the influence of an LLM on a collaborative task (software design), our study thus contributes to the team and product themes. Beyond software engineering, researchers have studied how users can co-create with GenAI on various creative tasks, such as writing [29], music composition [32], and meme generation [55]. Within this body of research, one stream of inquiry examines whether humans ideating with GenAI generate more novel ideas compared to human ideation alone. For example, one study explored the use of a GenAI-powered design tool that provides inspirational images [28] to aid in a design task, finding that users of the tool produced more novel and varied ideas compared to those who used a version that provided random images. Another study explored the use of an LLM in group brain-writing tasks [49] and noted that the LLM helped support divergent thinking, although there was some evidence that the ideas were uncreative. A related focus area is exploring how groups can effectively incorporate GenAI-based tools into collaborative creative work involving groups of humans and GenAI (e.g, [14, 15, 18]). Benefits of such tools include that they can help ideas to be quickly tested and
Exploring Creativity in Human-Human-LLM Collaborative Software Design
discussed in the group [14], maintain the flow of ideas if the group experiences a lull in ideation [15], and validate human-generated ideas [18]. Some challenges of using such tools are that they can stifle human creativity [18], the potential for homogeneity in the generated ideas [15], and a lack of trust in the answers [14].
3
Research Method
To answer our research question of “How, and where, does creativity appear naturally when designing with an LLM?”, we decided upon a laboratory-based setting that purposefully simulates a remote working environment. Moreover, we decided to recruit software professionals with some experience in software design. This decision recognizes that software design is often carried out by people in varying roles rather than being the sole preserve of a full-time architect or software designer. Finally, participants had access to an LLM, but were not required to use it. This approach was taken to allow participants to work in a way natural to them, further simulating a workplace environment. The materials provided to participants, coding guidance, and coding examples are available [23].
3.1
Study Design
Our exploratory study simulates a remote working environment where professionals collaborate in pairs to complete a design task. The study was designed not to be prescriptive, allowing participants to complete the task as they would in a typical work environment. As we wished to see how creativity naturally arose, we did not ask the participants to “be creative”. Instead, they were told the design would be judged on completeness. The task asked pairs of participants to produce a design document that satisfied a set of product requirements provided in a Product Requirements Document (PRD). The PRD contained a list of features and high-fidelity mock-ups for a bicycle parking application aimed at university students seeking secure bicycle parking spots on or near campus. The features were: (1) find bike parks near their current location, (2) search for a destination to find nearby bike parks, (3) get directions to the bike park, (4) pin the bike park to indicate where they park their bicycle, (5) read reviews of a bike park, and (6) crowdsourced bike park reviews. A browser-based chatbot in the form of a custom LLM wrapper was developed that enabled interaction with an LLM (specifically OpenAI’s ChatGPT 3.5 Turbo). The LLM wrapper logged all prompts and responses entered by participants. The LLM wrapper was accessible to a single user and made available to all participants. Note, this paper complements another paper [22] that uses the same dataset to examine how professionals incorporate an LLM into their software design process.
3.2
Participants
Eighteen pairs of software professionals based in the U.S. participated in the study (𝑛 = 36 individuals). Recruitment was conducted through directly contacting potential participants via the researchers’ professional networks and a LinkedIn post. To further mimic a workplace setting where teammates collaborate on a task, each potential participant was asked to identify and recruit their own partner. In two cases where participants were unable to find partners, the researchers matched pairs. The demographics of the
participants varied (see supplementary data [23]). Most (𝑛 = 25) had used an LLM at work, while nine had used an LLM for personal use. 27 of the participants identified as male and nine as female. The average years of professional work experience was 11 years, with 13 participants having less than 5 years of experience, and 7 participants having 20 years or more of experience. All self-identified as having some experience designing software, with the majority (𝑛 = 25) working in technical roles (e.g., engineer, architect), and others working in roles such as product management. We thus use the term ’designers’ in this paper to acknowledge that all participants have some design experience. We refer to specific participants as 𝑃𝑥𝑎 and 𝑃𝑥𝑏 where 𝑥 refers to the pair, 𝑎 is one partner, and 𝑏 the other partner (e.g., 𝑃6𝑎 , 𝑃 6𝑏 ).
3.3
Procedure
All sessions were conducted via Zoom. Participants were given 90 minutes to complete the design task. Except for P8, who worked in the same physical space, all pairs worked remotely from separate locations. Participants were permitted to use any tools and create any artifacts they deemed necessary to articulate their design. Following the task completion, a short exit interview was conducted to understand their experiences. During the task, participants had access to the custom LLM wrapper. Participants were informed of the tool’s availability but were not required to use it. Two pairs (P3, P11) chose not to use it. Task completion times varied. Eight participants completed the design task before the 90-minute deadline. The shortest completion time was 59 minutes and the average was 82 minutes. The sessions were performed between October 2023 and March 2024.
3.4
Data Collection
For each pair, we collected the design session recording (including the interview), their design document, and the LLM wrapper logs. Participant demographic information was collected via an online survey. A professional transcription agency transcribed all recordings of the design task, while the interviews were auto-transcribed by Zoom with subsequent review and correction by a researcher.
3.5
Data Analysis
To answer the research question, we conducted a qualitative analysis of creativity in both the design process and the product (the design document). We decided to identify all instances of creativity in the process and the product, regardless of whether the LLM was involved. This approach facilitates a comparison between creativity arising with and without LLM use. Two researchers with expertise in software design undertook the data analysis. One researcher has significant experience building software applications in industry, while the second has prior experience analyzing design conversations for creativity. To aid consistency in coding, we prioritized negotiated agreement between the two researchers over the computation of Inter-Rater Reliability (IRR). Below, we describe our analysis with further details available (see [23]). 3.5.1 Creativity in the Design Process. We operationalized creativity in the design process as Jolak et al. [25] did, using Dorst and Cross’s definition of creative events (described above) to identify creativity in the software design process. As per Jolak et al. [25],
Jackson, Liebel, Prikladnicki, van der Hoek
a creative event was identified if a snippet of the designers’ conversation could be considered related to the identification of: (1) a connection that simplifies an issue resulting in part of a solution to the problem, (2) an issue that complicates the design problem, (3) an appealing simple solution to an issue, (4) a sudden realization (an “aha” moment) related to the problem or solution, or (5) an understanding of the problem. This definition served as the starting point for the codebook used to abductively code the design session transcripts. The two researchers independently coded the first three transcripts before discussion, leading to the identification of additional codes for the code book: “analogies”, “LLM aiding problem”, “LLM aiding solution”, and “idea of LLM use”. This revised codebook was re-applied to the first three transcripts. Subsequently, one researcher coded the odd-numbered transcripts, and the second coded the even-numbered transcripts. The researchers then reviewed each other’s coding and discussed any disagreements to reach consensus across all transcripts. No new codes were identified in this final round of coding. Upon completion, the lead researcher reviewed the entire set of creative events across all transcripts to assess their commonality. Several themes were identified, including “re-use of third-party libraries”, “deferring design”, “analogies facilitating insights”, and “missed opportunities”. Additionally, the creative events resulting from LLM use were reviewed closely to determine what influence the LLM had on creative events, leading to several themes, including “identify simple solution”, “unveil issue(s)”, “decision making”, and “hindrances”. These themes, along with examples of creative events, are presented in Section 4.2. 3.5.2 Creativity in the Product (Design). To understand how and where creativity arises in the product (the software design), we needed to: (1) identify designs that were creative and (2) the source of this creativity (the human or the LLM). We decided to use Boden’s definition of creativity “...artifacts that are new, surprising, and valuable” [2]. To operationalize this, we considered a design as creative if it contained at least one element that was new AND surprising AND valuable in part of any of the different kinds of artifacts that the design contained (e.g., data models, architectural components, API definitions), since creativity could arise in any of those. Because the PRD provided a concrete framing of the problem, it served as the baseline against which all designs were considered. Specifically, an element was considered new if it did not appear in the PRD and could not be reasonably inferred as an implicit requirement. Routine features (e.g., user login/registration, the reuse of OAuth (P1) or Google Services (P17)) were treated as conventional features, whereas bike-park occupancy tracking (e.g., P8) were considered new as the PRD made no mention of such capabilities. An element was considered surprising if it was not considered predictable from the requirements. For example, sourcing external police theft records (P3) and introducing a lewd image detector (P5) were considered surprising as these extend beyond the problem space. Minor variations (filtering results similar to Google Maps (P3)) were not considered surprising. An element was considered valuable if it extends the problem space in a way that aligns with the goals of the design problem, or contributes a helpful solution. Examples of valuable additions included non-functional architectural
#
Source
Description
1
Knowledge/Exp.
2
Empathy
3
Ambiguity
4
Analogy
5
Underlying goal
A designer’s prior knowledge or experience provides an idea A designer imagines using the application themselves (literally or “as someone else”) An ambiguity in the requirements led to varying interpretations Discussing or exploring analogical applications gave insight into a novel feature or solution Considering the underlying root problem of the design task or the goal rather than the requirements directly
6
LLM sparks
7
LLM elaborates
8
LLM provides
Unexpected insight triggered in a human who is reviewing LLM output Human-initiated idea is enriched by the LLM LLM-suggested idea not considered previously by the designers
Table 1: Eight sources of creativity in the product. #1-5 are human-initiated, #6-7 are the result of a human-LLM collaboration, #8 is suggested directly by the LLM.
concerns to improve the scalability or performance (e.g., P15, P18) and generalizations to improve adaptability of the system (P10). Although Boden’s definition does not explicitly mention usefulness, we excluded features that added undue complexity without a clear benefit. This aligns with the practical meaning of creativity in software engineering, where usefulness is considered integral [19]. However, none of the final designs contained complex elements, as designers either did not consider them or ruled them out themselves. For example, P18 ignored the LLMs suggested architecture that combined capabilities from multiple cloud providers, instead proposing an architecture from a single provider. Identifying creative elements. The two researchers reviewed one design together (P2) to identify creative elements (e.g., “metrics to track usage”) and discussed any issues encountered in using the definition detailed above. Next, both researchers independently identified creative elements in the remaining 17 design documents. They then cross-checked each other’s work and discussed any differences to reach full agreement on the creative elements across all design documents. Such an agreement mitigates the subjectivity inherent in judging for newness, surprise, and value. Source of creative elements. To identify the source (human or LLM), each creative element was triangulated with the corresponding transcript and the LLM wrapper logs. The lead researcher conducted the analysis, while the second reviewed the results; any disagreements were discussed. The triangulation identified the creative events, conversation snippet, and, if the use of the LLM was observed, the LLM prompts and responses leading up to the identified creative element. Pertinent information was captured in a
Exploring Creativity in Human-Human-LLM Collaborative Software Design
memo. On reviewing the memos, eight common sources for the creative elements (see Table 1) were identified: five human-initiated sources and three involving the LLM in different ways (LLM sparks, LLM elaborates, LLM provides). Finally, across all designs, the different sources were counted to assess their prevalence.
3.6
Ethics
The study adhered to the Human Research Protection protocols from the University of California, Irvine (IRB: #3652). Each participant was provided with a study information sheet ahead of the study and gave verbal consent at the start of the task.
4
Findings
This section presents our findings to the research question: “How, and where, does creativity appear naturally when designing with an LLM?” We include within our findings all observations related to the appearance of creativity, in both the process and product, not just where creativity arose due to the use of the LLM .
4.1
Summary of LLM Usage
To provide context for the findings, we note the varied use of the LLM by the 18 pairs. Two pairs (P3, P11) did not use the LLM wrapper to assist in their design. P3 preferred to solve the design problem themselves, and P11 did not trust LLMs more generally and consciously decided not to engage with it. Three pairs (P7, P9, P14) used the LLM to produce their entire design through ongoing prompting of the LLM wrapper. Five pairs (P4, P5, P10, P15, P17) used it to seek information helpful to their design, while creating the design themselves. For example, P5 used the LLM wrapper to explore the Google Maps API, which informed their design. Eight pairs (P1, P2, P6, P8, P12, P13, P16, P18) used it to seek information and generate parts of the design, which they subsequently reviewed and incorporated into their design. For example, P1 asked the LLM wrapper to create a data model that they then refined by hand.
4.2
Creativity in the Design Process
Across all 18 pairs, process-level creativity was evident. Pairs often achieved creative leaps by simplifying (assumptions, re-use, deferrals) and by analogical transfer from similar applications. LLMs rarely reshaped the problem space. Instead, their influence was on solution proposals. Sometimes the LLM was helpful, other times it anchored decisions or provoked unproductive digressions. Before discussing these in greater detail, we first present three vignettes of creative events from three different pairs to illustrate how creativity emerged in the design process. Note, words related to the qualitative coding of the creative events are shown in quotes (e.g., “complication”, “realization”) in the vignettes. 4.2.1 Creativity in Action. The first example illustrates the role of re-use in finding an appealing solution to a complication that had arisen. P1a raised a “complication” with the design problem in that the requirements do not mention user registration, despite the pair deciding earlier that user accounts were required to store bike park reviews. In parallel, P1b was amending the proposed database design. In so doing, P1b noticed the database design mentioned users, and this caused the “realization” of needing user registration
functionality, before suggesting a “simple solution” that re-used a third-party component “OAuth” (P1b) for user login. This “realization” and suggestion of a “simple solution” helped them make the creative leap from the problem to the solution. P13 similarly preferred a “simpler solution” at one point in the design process. Specifically, at an earlier point, they decided a user should have multiple pins to indicate where their bike was stored or a preferred bike park. When designing the data model for the user, they discussed this further, and simplified their solution by deciding to make an “assumption” about the requirements as they designed the data model, which led them to decide on one pin only as “That’s easier to model” (P13a). This “simplification” helped them make the creative leap from the design problem (the need to store location) to the solution (a pin). In our final example, inspecting a part of a solution generated by the LLM led a pair to identify a “complication” in the LLM’s provided design, which they resolved by exploring an “analogical” application that provided an “insight” into a simplified solution. Specifically, P2 used the LLM wrapper to define a database schema. In reviewing the tables in the LLM’s response, they noted it had a separate LocationSearch table that stored each search as a location. This led P2a to identify a “complication” about what should be stored in the user’s search history: the final location selected or the search term itself. To resolve the complication, they opened an analogous application (Google Maps) to see how it handled search history, before deciding to adopt its simple solution of storing the search term. The LLM provided design was subsequently amended to store the search term. The “insight” helped them make the creative leap.
4.2.2 Insights through Simplification. Often, a simplification from the pair themselves helped identify the creative leap needed to resolve a dissonance between the problem and the solution. Mechanisms used for simplification included making assumptions or straight-up decisions to go in one direction or another, re-using existing solutions, and deferring part of the design to later. Assumptions and decisions shaped the space of the problem or solution. This shaping made it easier to identify potential solutions for part of the problem and to take the creative leap necessary to bridge the problem and solution spaces. Sometimes, the creative leap was made by making an assumption about the problem leading to a reduced scope and simpler design, “Is this app going to be U.S. only? In which case, we’re just defining address in the U.S. format.”(P7a). They thus designed their database table to support U.S. addresses only. At other times, decisions about the solution helped with the creative leap. For example, P14 decided the application would be “agnostic to device, so iOS or Android”, settling on a technology that supported both operating systems (React-Native). Other pairs simplified their solution by using third party libraries for addressing part of the design problem. This re-use made it easier to take the creative leap. Such re-use has the dual benefit of both simplifying the solution, as they can rely on the third party to provide part of the solution, and reducing the amount of work for the designers. For example, P17, who, on determining that user accounts would be required for user reviews, decided to use Google authentication services, “So I think using Google or whatever, email to log in and not store the credentials is critical”.
Jackson, Liebel, Prikladnicki, van der Hoek
Another mechanism employed by the pairs to simplify the design was to defer part of the design until it was truly required to support the application’s user base. Such deferring simplified the solution space. As an example, some pairs discussed the initial size of the user base and decided components to support scalability and performance could be deferred, “Let’s focus on the small first. Cause it sounds like this is an initial product. So there’s no need to build something that’s like super scaled.” (P2). Shaping the problem and solution space through assumptions, decisions, clever reuse, and active deferral simplified the design task to help the designers take the creative leap necessary to progress their design. 4.2.3 Use of Analogies. There was widespread (14/18 pairs) use of analogies when discussing the design. This use of analogies helped to identify appealing solutions to problems. Mobile applications (e.g., Google Maps, Yelp, Uber) with similar features, such as searching for a place, getting directions, and reviews, were often mentioned. Indeed, 6/14 pairs mentioned Yelp in the first five minutes of their design session, sometimes influencing their future design. At other times, analogies were used to discuss a specific aspect of the design problem. For example, in discussing how users may want to use review criteria to help identify a bike park, P3b described how Google Maps lets one filter on certain criteria and suggested their app could do the same. Some pairs went beyond just mentioning an analogy, and turned to engaging with analogical applications in detail to see how they solved a problem similar to the one they faced. For example, when modeling their database, P1 were unsure whether photos should be linked to the review of a bike park, directly to the bike park, or both. Therefore, they opened up Google Maps to explore how it supported photos for a location. Doing so, they decided to limit the scope, thus shaping the problem space, “There’s no requirement for users to submit their own bike park locations.” (P1b) and simplifying their solution by deciding, “users can only submit photos as part of a review” (P1b). Analogies helped address issues or complications, thus facilitating creative leaps. This behavior is to be expected, given that designers are known for using analogies when solving problems [5]. 4.2.4 LLM Influence on Creative Events. In using the LLM to automate the more mundane aspects of design, such as outputting boilerplate API definitions, the LLM provided space for the pairs to think more deeply about the design problem. Thus, it indirectly influenced creativity, “It gives the foundational kind of like busy work out of the way. And so it provides more time for me and [P18a] to have discussions about like, does this make sense like, what, how do we refine this?” (P18b). Additionally, we observe that the LLM primarily assisted with finding a solution rather than reshaping the problem space. Indeed, we did not see any instance of the LLM for reshaping the problem space or the LLM itself suggesting considerations, including complications—these parts of a creative event always came from the designers themselves. As noted in Section 4.1, many pairs asked the LLM to generate part of their design. While not all generated parts were simple, sometimes the LLM offered a simple solution that the pair could immediately use. P14 asked the LLM to define the database tables
needed. On reviewing the solution proposed by the LLM containing “Locations”, “Reviews”, and “Users”, they decided those were “the three main things” and kept them in their design. Other times, the pair asked the LLM for multiple options and, after reviewing, decided which to select. For example, when discussing the backend architecture, P6 asked the LLM for “What are the difference choices for the backend architecture for this app”, and decided upon serverless. These two cases illustrate contrasting human behavior. In the first, they did not consider alternatives, unlike the second. This points to the risk of anchoring to the first proposed solution, which could hinder creativity by preventing exploration of alternatives. In reviewing LLM responses to prompts, the discussions among the pairs sometimes unveiled issues and complications with the problem or solution, requiring insight from the designers to resolve and make the creative leap. Consider P2, who asked the LLM to generate a set of server APIs. On reviewing the response detailing an API for finding bike parks near the current location, they noted the API represented the location as latitude and longitude. P2 noted this could be problematic as users would be searching by a location, not by latitude and longitude. To resolve the issue, they went back to the PRD and spent some time walking through the user flows before concluding that the provided APIs would still work. The LLM aided in the decision-making necessary to facilitate the creative leap from the problem to the solution space on several occasions. Sometimes the LLM implicitly made a decision when proposing a design, other times it was the designers who made the decision based on information from the LLM. For example, P7, who used the LLM to generate their entire data model, asked it to include the user’s prior searches. It did so as an array of strings, which P7 agreed with on review. This simple solution thus enabled the pair to take the creative leap from the problem (need to store searches) to a solution (updated database design). Note that such straightforward decision-making did not always occur. Indeed, in other cases, the designers had to significantly amend the proposed design after a review, because they were not happy with the design choice made by the LLM (e.g., P2 amended suggested APIs). Moreover, the LLM sometimes did not make a decision at all. When P6 asked the LLM for advice on the best backend architecture, for instance, it explicitly declined to make a choice. Instead, it provided considerations to aid the decision-making and left the decision to the pair. Sometimes, the LLM actively hindered the designers through over-complications and provoking digressions. For example, in one case, it produced an overly complex design. P18 asked it to help with the architecture design, but found the response unhelpful as it referenced components from two different cloud providers: AWS Cognito and Google Firebase. Another time, the response caused an unnecessary digression. P7 had already reached consensus on the use of user accounts, but upon receiving a response from the LLM referencing user accounts, the conversation started afresh, leading to an unncessary digression. 4.2.5 Missed Opportunities for Creativity in the Product. In discussing the problem and potential solutions, some designers mentioned novel ideas that would have led to creativity in the product. Yet, while some were elaborated and included in the final design (discussed below), others were not. Sometimes this was due to a conscious decision to declare an idea out of scope. For example,
Number creative elements in design
Exploring Creativity in Human-Human-LLM Collaborative Software Design
4 LLM Sparks LLM Elaborates
3
LLM Provides Human Initiated 2
# Creative Element
Pairs
P8, P14e , P15, P16, P18s 2 Track bike park capacity and occupancy P3, P8p , P9p , P12p 3 Scanner to detect thefts from police reports P3, P5 4 Data protection requirements P16e , P18p 5 Lewd image scanner on image upload P5 6 Profanity checker on review submission P5 7 Validate user input for spam and inappro- P9p priate content 8 Content moderation P12p 9 Upvote/downvote reviews P12p 10 Bike park accessibility information P12p 11 Push notifications to remind user to review P10 bike park 12 Shows location bike parked on home P11 screen 13 Tracks recently used bike parks P18 14 User configurable radius search distance P10 15 Metrics to track bike park usage P2e 16 Generalization for multiple universities P10 17 Generalization for many review attributes P10 18 Sources bike parks and reviews from ex- P4 ternal sources 19 Accessible application P18p 20 Cost optimization P14e 1 Scalability or performance solutions
1
0 P10 P12 P18 P5 P3 P8 P9 P14 P16 P2 P4 P11 P15 P1 P6 P7 P13 P17
Pair
Figure 1: Number of creative elements included in the design document of each pair, categorized into the source (human or LLM).
P11 raised the possibility of a “Report closed” button, inspired by a similar feature in Yelp. However, they dismissed it as “feature creep”, and simply noted it as an idea rather than elaborating on it further in their design. Other times, novel ideas were mentioned, yet not discussed further or revisited, even when the pair liked the idea. For example, P18 briefly discussed an appropriateness filter on review comments early in their design, which they agreed would “definitely” be needed. Yet, it was not mentioned again. Responses from the LLM that included novel suggestions were sometimes treated in the same way as well. P10a reacted favorably to the suggestion within their first LLM response that users could share their reviews and favorite bike parks, “Social sharing. Oh, that’s good. I like that.” (P10a), yet this feature was not included in their design.
4.3
Table 2: Creative elements presented in the designs by pair. If LLM assisted, superscripts are used to indicate how: e for elaborate, s for sparks, p for provides
Creativity in the Product
In contrast to the observation that creativity in the process was present for all, fewer pairs had creative elements in their design documents. We elaborate further on product creativity. 4.3.1 Presence of Creativity within the Design. Although the designers were not primed for creativity, many design documents (13/18) included at least one new, surprising, and valuable element (see Figure 1). The number of creative elements varied: three design documents contained four, while five had none. The most common creative element was scalability of performance concerns (five designs), followed by tracking bike park availability (four designs). Differences in the number of creative elements reflect different attitudes to scope expansion and the varying context provided to the LLM. Twenty different creative elements were identified across the 13 design documents containing creative elements (see Table 2). Four creative elements appeared in multiple design documents, indicating that they are not novel across the entire set of designs. For example, four designs incorporated a feature to track a bike park’s capacity and measure its occupancy. Other times, the features were creative elements unique to a single design, e.g., push notifications to remind a user to review a bike park. The most frequent creative element (five designs) provided a scalability or performance solution. The PRD made no mention of user numbers or projected growth. Yet, some pairs were concerned with ensuring the application could support a reasonable number
of users, so they designed the architecture to scale and perform well. For example, P15 included a Redis cache for photos to serve them more quickly to users. Scalability or performance solutions being the most common creative element is indicative of a factor that potentially explains why some pairs had more creative elements than others: some pairs limited the scope of the problem or solution by more or less sticking to the PRD, while others extrapolated potentially important additional features. Within Section 4.2.2, performance and scalability considerations were provided as an example of where some pairs decided that the initial number of users would be small, which is clearly opposite to the example just provided of P15. This differing behavior highlights the impact that scoping decisions and intentional limits have on the creativity of the end product. Five out of sixteen designs from the pairs that used the LLM did not contain creative elements. In contrast, designs from the two pairs that did not use the LLM had creative elements. Although this is a small sample, these figures indicate that using the LLM does not guarantee creativity in the product. Other factors for how creativity arises are at play. One potential explanation could be the amount of problem context provided to the LLM. Four out of five pairs that had no creative elements used the LLM to generate some or all of their designs. In doing so, they all provided much context
Jackson, Liebel, Prikladnicki, van der Hoek
about the problem by copying requirements from the PRD into the LLM prompts. In effect, they defined a fixed problem space to which the LLM obliged by providing a constrained potential solution that matched the PRD very closely, leading to outcomes similar to those noted in the study on fixed requirements hindering creativity [40]. 4.3.2 Source of the Creativity. Notably, the creative influence of the LLM was less than that of human-initiated sources, such as prior knowledge or empathy from the designers. Of the 29 creative elements in the designs, 15 are attributable to the five sources characterizing designers’ behaviors and skills (see Table 3), a further five are the outcome of the two sources typifying joint human-LLM creativity (LLM elaborates, LLM sparks), and nine are directly suggested by the LLM (LLM provides). Prior knowledge/experience. The knowledge and experience of the designers is the most frequent human-initiated source, with eight occurrences across five pairs. Examples include P5, who included a profanity checker and lewd image detector in their solution. In the exit interview, P5a mentioned these ideas came directly from their work experience, “where I work, we have something somewhat similar, just because we handle a lot of images every day.” Experience in design helped also, as designers often use design principles gained over the years to guide their work [44]. For example, P10a (an experienced professional) mentioned design principles such as the desire to “generalize” and the need to consider “scalability”. These design principles guided the pair to a generalized database design that supported multiple universities, arbitrary review attributes, and user-configurable features such as search radius. Empathy. Two pairs imagined using the application and, in doing so, gained insight into beneficial features. For example, P11 discussed what the application should display upon launch. P11a noted that they, “would want something where I’m just like to go to pin” to easily find their retrieved bike. This led to the idea of adding a link to the pinned bike park on the application’s home screen. Ambiguities. Two pairs found ambiguities in the PRD leading to the identification of creative elements. One ambiguity was the word Table 3: Sources of the creative elements within the design documents. Each creative element is counted against one source. The sources are shown with the total number of creative elements and the number of attributed participant creative elements in parentheses. Source
Total
Pairs
Knowledge/Exp.
8
Empathy Ambiguity Underlying goal Analogy
2 2 2 1
P5 (2), P8 (1), P10 (3), P15 (1), P16 (1) P5 (1), P11 (1) P4 (1), P18 (1) P3 (2) P10 (1)
LLM elaborates LLM sparks
4 1
P2 (1), P14 (2), P16 (1) P18 (1)
LLM provides
9
P8 (1), P9 (2), P12 (4), P18 (2)
“crowdsourcing” in the context of user reviews. While the intent of this was to indicate users of the application inputting reviews of a bike park, P4b was unsure what it meant, “Is it the ones that the user has saved on this app? Or is it going to source it from everywhere on the internet?” After a brief discussion, they decided to include both, and so their design included a “Review Sourcer” service to find external reviews from Google. Underlying goal of the design problem. One pair disregarded the PRD and decided to focus on three core considerations implied in the goal of the problem, “Do you want to break this up into safe, vacant, and close?” (P3a). This led to much discussion about the best way to determine how safe a bicycle park was, whether it was vacant or occupied, and how to decide which bike parks to show in the search results. Their conversation thus included aspects such as sourcing bicycle theft records from the police, tracking whether a bicycle has been securely retrieved from a bicycle park, and whether bike parks with high theft rates should be excluded from the search results. Their resulting database design contained a bike park table with the novel attributes total_bike_count, vacant_count, success_retrieval_pct, and thefts_per_month. Analogies. Although, as noted above, discussing analogical applications with similar features helped to solve the design problem, only one pair (P10) used analogies to unearth creative elements. P10 in particular discussed the challenge of getting a user to return to the application to leave a review or pin the bicycle’s location after the application launched a third-party mapping application to navigate to a selected bike park. They resolved the challenge by reusing the approach (push notifications to mobile devices) used by Lyft to remind riders to leave a review of their driver. LLM elaborates. The LLM helped four pairs elaborate on ideas generated by the designers themselves. Consider P2b, who raised the need to estimate the operational cost of running the application on a cloud-based infrastructure such as Amazon AWS, noting that at work, “We’re pressed all the time to think about cost.” This led to a discussion on how to monetize the application to pay for its operational costs, whereupon P2a had the idea that the city should pay for the application. P2a felt the app “probably want[s] to collect metrics” to provide to the city. Subsequently, they asked the LLM wrapper for metrics that they included in their design. LLM sparks. Human-AI collaboration was also evident when the LLM’s response sparked an idea in P18. They asked the LLM to produce a design document with instructions to consider cloud-native technologies and a 3-tier architecture. The response mentioned AWS S3 for photo storage, which sparked the idea to add a Content Distribution Network (CDN) to their design as “CDN provides performant access” (P18b). A CDN was not mentioned in the prior LLM responses or in the conversation, so presumably this decision was based on the designers’ prior experience. LLM produces. While elaborating and sparking can be considered an indirect influence of the LLM on creativity, the LLM also aided more directly by generating a response containing a novel idea not previously considered by the pair. Three pairs were assisted in this way, although their prompting approaches differed. P12’s first prompt to the LLM wrapper was broad and open-ended, with no
Exploring Creativity in Human-Human-LLM Collaborative Software Design
mention of the PRD requirements, “Help me design a mobile app that finds me a secure bike park to park my bike”. In subsequent exploration of the suggested features through the LLM wrapper, the pair identified novel elements, including content moderation and tracked bike park capacity and occupancy. In contrast, P8 and P18 entered prompts requesting specific design artifacts. P8 asked for a data model for a parking lot (an analogy), and the response mentioned capacity and the number of available spots, which they included in their design. P18’s prompt was more open, giving the LLM flexibility in its response by asking for cross-functional requirements and fitness functions. The LLM suggested accessibility and data protection features, which they incorporated. These varying prompting styles are another potential factor in why the LLM’s influence on product creativity varies across pairs.
5
Discussion
Software design is an integral part of building software applications and often considered a creative task [4]. However, the influence of an LLM on such a collaborative and creative task has been little examined in comparison to studies that consider the use of LLMs in coding [17]. By examining how, and where, creativity naturally emerges when using an LLM in software design, we provide novel insights into the impact of an LLM on creativity in software design. We firstly note that human creativity remains central to software design, even when using an LLM. The designers consistently demonstrated creativity in the design process by reshaping the problem, identifying simple solutions that included reuse, and drawing upon analogies to aid problem-solving, regardless of whether they used the LLM. Creativity was present in the design process for all pairs, reinforcing the view that creativity is key to software design [4]. Notably, this was the case without priming the study participants to be creative. Creativity in the design process resulted in creative elements in 13/18 pairs’ (72%) designs, with the LLM helping spark, elaborate, and produce creative elements. However, we observe that human traits, such as prior knowledge and empathy, contributed to identifying creative elements more often than the LLM, per Table 3, particularly also because both LLM spark and LLM elaborate strongly depend on interplay with humans. Our findings also suggest that the LLMs’ primary influence is on product-level creativity rather than on process creativity. We observed many creative events [6] described in Section 4.2, with the LLM only playing a role in solution finding rather than reshaping the problem space. Importantly, using the LLM did not guarantee creativity, with 5/16 pairs using the LLM having no creative elements while both non-LLM pairs did, underscoring that availability alone is insufficient without adequate integration into the design process. While using the LLM led to several creative elements in the sense of novelty, we also observed that creative elements attributed to the LLM were not entirely surprising and did not differ that much from those produced by humans alone (Table 2). That is, the LLM-produced ideas were new with respect to the PRD, but typically related to commodity functionality, such as adding content moderation. This could be due to our detailed PRD constraining exploration ([39]) by the designers, including in their interactions
with the LLM. Only one pair using the LLM was observed to have expansive, open-ended prompts devoid of PRD detail, leading to creative elements. This observation encourages a deeper investigation of how LLMs can be used for creative work beyond commodity, perhaps by exploring idea-framed requirements. More broadly, we note the creative elements, irrespective of source (human or LLM), are not at a “Big-C” level, as none in Table 3 are groundbreaking. Interestingly, accompanying the more direct influence of the LLM was an indirect effect. Using the LLM helped free cognitive space by automating the routine parts of design, such as writing boilerplate. Such space allowed the pairs to think more deeply about the design problem. This observation suggests that automating rote work through the use of an LLM not only benefits productivity [56], but can also play a role in aiding creativity. While freeing up of cognitive space through automation has been speculated upon [24], our study finds that it can indeed happen. Using an LLM was not always positive as, sometimes, the LLM introduced distractions, which could potentially hinder creativity. Additionally, these distractions stand in direct opposition to the previous point, i.e., freeing up cognitive space. One pair in particular spent time on a tangential discussion about a non-essential and well-solved problem (user accounts) as a result of a response from the LLM. This observation highlights the need for a more critical engagement with the LLM to prevent digressions. Continuing to utilize well-understood design practices [44] can likely enhance such engagement, as can the development of design tools shaped to augment human-led creativity (as discussed below). We note pairs exhibited differing creativity in their designs, perhaps due to varying responses to scope creep [8], irrespective of LLM use. Some pairs seemingly embraced newly-suggested features, while others rejected ideas due to concerns about scope creep. The tension between innovation arising from new ideas and the need to deliver has always been present in software teams. With the advent of LLMs and their ability to augment human creativity, teams may need to re-evaluate how they consider scope creep to ensure innovative ideas are not unintentionally squashed. Another reason some designs lacked creativity could be the varying levels of creativity among our heterogeneous sample. Some have decades of experience and potentially Pro-C creativity, while others are novices likely to be exhibiting mini-c or little-c creativity [26]. Moreover, prior research has noted the influence of personality traits on creativity [1]. Although creativity arose for all pairs in the design process, it did not always lead to novelty in the design product as perceived by external observers. This connects to the concept of everyday creativity [26], which exhibits itself as minic, little-c, and Pro-c in the creative work of designers and aids in problem-solving and gaining personal insights. Such creativity is important both for completing tasks and for personal reasons such as improved well-being [51], and should not be overlooked as AI is increasingly adopted for software development tasks. Finally, our empirical study on where, and how, creativity arises when designing with an LLM, contributes to broader conversations on human-AI co-creation (e.g., [3, 12, 21]). We have shown that creativity naturally emerges in a collaborative act such as software design, irrespective of whether AI is used. While AI can help in amplifying human creativity, it is not replacing human creativity.
Jackson, Liebel, Prikladnicki, van der Hoek
This perspective of amplification is shared by HCI scholars such as Shneiderman [50] and we touch upon it in future work below.
5.1
Implications for Practitioners
Our findings lead to several practical implications for practitioners. Firstly, when using an LLM, it is important to consider multiple alternatives before selecting a solution. This reduces the risk of anchoring on the first solution that we observed in this study. Secondly, designers should consider open-ended, expansive prompts with minimal context if they wish to elicit creative input from the LLM, rather than providing detailed requirements that constrain it. Interacting in a more playful manner [46] can also help to amplify human creativity. Thirdly, designers should perhaps time-box interactions with the LLM, including discussions of its responses, to prevent LLM-induced digressions. Finally, creativity thrives when designers have space to think deeply about a problem, as they can attain the flow state [45] known to aid creativity [7]. As noted in the findings, using an LLM to automate the mundane, time-consuming work of design (e.g., completing boilerplate artifacts) can create this space.
5.2
Future Research Directions
One direction is to explore the influence of an LLM on creativity in an industry setting, as real-world software design is more complex than the task used in this study. Such research would help unpack a recent observation that some developers believed ChatGPT aided creativity, while for others it hindered [37]. Research could also explore whether the “space” created by using LLMs to automate rote tasks, really does aid creativity by providing time for deep thinking, or whether it gets swept away as other tasks fill the gap. A second direction is to explore how novel LLM-based tools can amplify human creativity in software design. Such creativity support tools [10] could be designed to support practices known to aid creativity, for example by encouraging exploration of the problem space, identifying alternative solution designs, fostering periodic reflection, or mitigating some of the downsides of LLM use in creative work (e.g., design fixation [52]). Ensuring humans retain their agency in creative work should be a key design principle in designing new LLM-based creativity support tools.
6
Limitations and Threats to Validity
One limitation is that participants were not explicitly informed that the study would assess creativity. This was by design, as we wanted to see how and where creativity naturally arose in a simulated work environment. “Big-C” creativity (not observed in this study) may have arisen if the participants had been primed for creativity within the instructions or the requirements were framed as ideas, as noted by Mohanani et al [40]. A second is that our participants differed in their experience (e.g., years working, role, experience of LLM use). This variety will influence their design processes, their use of the LLM, and the resulting level of creativity. While this heterogeneity is indicative of industry teams, this variety could influence the level of creativity we observed. Finally, participants had only 90 minutes to complete the task, which could have led to undue time pressure, the impact of which is nuanced with mixed outcomes [27, 30].
The research team has significant experience in software design from research and industry. This experience facilitated rich discussions when coding and in meetings with the wider team. Moreover, our findings are strengthened by the triangulation of multiple datasets. As this is an exploratory study, we make no claims to data saturation [48]. Further studies may find alternate ways in which creativity arises in collaborative software design. Findings may not be fully transferable to industry or when designing for an existing application, as the study was laboratory-based. To increase the transparency of the findings, we provide a detailed study design in Section 3 and share supplementary materials [23]. The same two researchers performed the data analysis, thus reducing researcher bias. By coding individually, discussing, and reviewing each other’s work, we reached consensus on the coding and subsequent findings.
7
Conclusion
Software design is a collaborative and inherently creative activity that forms an essential part of software development. Through a laboratory-based study, where 18 pairs of software professionals were asked to complete a design task within 90-minutes and had access to an LLM, we found that human creativity remains a vital part of software design process. While the LLM occasionally sparked ideas or suggested novel features, it was the humans who undertook the core creative work of re-shaping the problem, considering scope, drawing analogies with existing systems, and the overall direction of exploration. We find the LLM’s influence was more pronounced—even though still relatively limited—in the design product, as a number of designs included novel, surprising, and valuable additions beyond the PRD. That said, the creative elements suggested by the LLM were relatively homogeneousWe recommend that practitioners hone their AI literacy skills in crafting exploratory prompts to better leverage LLMs for idea generation. Moreover, using an LLM to automate routine aspects of design may create space for more in-depth design thinking. Future research directions include exploring how to design LLM-backed creativity support tools to augment human creativity in tasks such as software design.
Acknowledgments Thanks to our participants for undertaking the design challenge. The authors appreciate Sadid Khan’s assistance in developing the chat tool. Prof. Marian Petre from the Open University, UK, provided valuable feedback on the study. Prikladnicki is partially funded by Fapergs and CNPq, Brazil; van der Hoek acknowledges support by the National Science Foundation under grants CCF-2210812 and 2326489.
References [1] Aamir Amin, Shuib Basri, Mobashar Rehman, Luiz Fernando Capretz, Rehan Akbar, Abdul Rehman Gilal, and Muhammad Farooq Shabbir. 2020. The impact of personality traits and knowledge collection behavior on programmer creativity. Information and Software Technology 128 (2020), 106405. doi:10.1016/j.infsof. 2020.106405 [2] Margaret A. Boden. 2004. The Creative Mind: Myths and Mechanisms. Vol. 2. Routledge, London. [3] Sebastian G Bouschery, Vera Blazevic, and Frank T Piller. 2023. Augmenting human innovation teams with artificial intelligence: Exploring transformerbased language models. Journal of Product Innovation Management 40, 2 (2023), 139–153.
Exploring Creativity in Human-Human-LLM Collaborative Software Design
[4] Frederick P. Brooks Jr. 2010. The Design of Design: Essays From a Computer Scientist. Pearson Education Inc. [5] Nigel Cross. 2025. Creative Cognition in Design I: The Creative Leap. In Designerly Ways of Knowing and Thinking. Springer London, London, 45–62. doi:10.1007/978-1-4471-7541-4_4 [6] Nigel Cross. 2025. Creative Cognition in Design II: Co-evolution of Problem and Solution. In Designerly Ways of Knowing and Thinking. Springer London, London, 63–73. doi:10.1007/978-1-4471-7541-4_5 [7] Mihaly Csikszentmihalyi. 2013. Creativity: Flow and the Psychology of Discovery and Invention. HarperCollins, NY, USA. [8] Bill Elliott. 2007. Anything is possible: Managing feature creep in an innovation rich environment. In 2007 IEEE International Engineering Management Conference. IEEE, IEEE, 304–307. [9] Jeanette Falk, Yiyi Chen, Janet Rafner, Mike Zhang, Johannes Bjerva, and Alexander Nolte. 2025. How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 198, 23 pages. doi:10.1145/3706598.3713447 [10] Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019. Mapping the landscape of creativity support tools in HCI. In Proceedings of the 2019 CHI conference on human factors in computing systems. Association for Computing Machinery, 1–18. [11] Kiev Gama, Cleidson R. B. de Souza, Alexander Nolte, and George Valença. 2025. A C-Level Perspective on the Role of Developers in the Creative Process of Software Startups: Fostering Developer Creativity and Participation in Innovation. IEEE Software 42, 3 (2025), 98–106. doi:10.1109/MS.2025.3541435 [12] Luca Grilli and Mattia Pedota. 2024. Creativity and artificial intelligence: A multilevel perspective. Creativity and Innovation Management 33, 2 (2024), 234– 247. [13] Wouter Groeneveld, Laurens Luyten, Joost Vennekens, and Kris Aerts. 2021. Exploring the Role of Creativity in Software Engineering. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS). IEEE, 1–9. doi:10.1109/ICSE-SEIS52602.2021.00009 [14] Yuanning Han, Ziyi Qiu, Jiale Cheng, and RAY LC. 2024. When Teams Embrace AI: Human Collaboration Strategies in Generative Prompting in a Creative Design Task. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 176, 14 pages. doi:10.1145/3613904.3642133 [15] Jessica He, Stephanie Houde, Gabriel E. Gonzalez, Darío Andrés Silva Moran, Steven I. Ross, Michael Muller, and Justin D. Weisz. 2024. AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual Canvas. In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work (Newcastle upon Tyne, United Kingdom) (CHIWORK ’24). Association for Computing Machinery, New York, NY, USA, Article 9, 14 pages. doi:10.1145/3663384.3663398 [16] James D Herbsleb. 2007. Global software engineering: The future of sociotechnical coordination. In future of software engineering (FOSE’07). IEEE, 188– 198. [17] Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large Language Models for Software Engineering: A Systematic Literature Review. ACM Trans. Softw. Eng. Methodol. 33, 8 (Dec. 2024). doi:10.1145/3695988 [18] Stephanie Houde, Kristina Brimijoin, Michael Muller, Steven I Ross, Dario Andres Silva Moran, Gabriel Enrique Gonzalez, Siya Kunde, Morgan A Foreman, and Justin D Weisz. 2025. Controlling AI Agent Participation in Group Conversations: A Human-Centered Approach. In Proceedings of the 30th International Conference on Intelligent User Interfaces. Association for Computing Machinery, 390–408. [19] Sarah Inman, Sarah D’Angelo, and Bogdan Vasilescu. 2024. Developer Productivity for Humans, Part 8: Creativity in Software Engineering. IEEE Software 41, 2 (March 2024), 11–16. doi:10.1109/MS.2023.3340831 [20] Sarah Inman, Ambar Murillo, Sarah D’Angelo, Adam Brown, and Collin Green. 2025. Seamful AI for Creative Software Engineering: Use in Software Development Workflows. IEEE Software 42, 3 (2025), 59–66. doi:10.1109/MS.2025.3534085 [21] Zorana Ivcevic and Mike Grandinetti. 2024. Artificial intelligence as a tool for creativity. Journal of Creativity 34, 2 (2024), 100079. [22] Victoria Jackson, Yoonha Cha, Rafael Prikladnick, and André van der Hoek. 2026. The Role of LLMs in Collaborative Software Design. arXiv preprint arXiv:2604.09120 (2026). [23] Victoria Jackson, Grischa Liebel, Rafael Prikladnicki, and André van der Hoek. 2026. Supplementary data. https://doi.org/10.5281/zenodo.19002870. [24] Victoria Jackson, Bogdan Vasilescu, Daniel Russo, Paul Ralph, Maliheh Izadi, Rafael Prikladnicki, Sarah D’angelo, Sarah Inman, Anielle Andrade, and André van der Hoek. 2024. The Impact of Generative AI on Creativity in Software Development: A Research Agenda. ACM Trans. Softw. Eng. Methodol. (Dec. 2024). doi:10.1145/3708523 [25] Rodi Jolak, Andreas Wortmann, Grischa Liebel, Eric Umuhoza, and Michel R. V. Chaudron. 2023. Design thinking and creativity of colocated versus globally distributed software developers. Journal of Software: Evolution and Process (2023).
doi:10.1002/smr.2377 [26] James C. Kaufman and Ronald A. Beghetto. 2009. Beyond Big and Little: The Four C Model of Creativity. Review of General Psychology 13, 1 (March 2009), 1–12. doi:10.1037/a0013688 [27] Anis Khedhaouria, Francesco Montani, and Roy Thurik. 2017. Time Pressure and Team Member Creativity within R&D Projects: The Role of Learning Orientation and Knowledge Sourcing. International Journal of Project Management 35, 6 (2017), 942–954. doi:10.1016/j.ijproman.2017.04.002 [28] Jingoog Kim and Mary Lou Maher. 2023. The effect of AI-based inspiration on human design ideation. International Journal of Design Creativity and Innovation 11, 2 (2023), 81–98. [29] Jeongyeon Kim, Sangho Suh, Lydia B Chilton, and Haijun Xia. 2023. Metaphorian: leveraging large language models to support extended metaphor creation for science writing. In Proceedings of the 2023 ACM Designing Interactive Systems Conference. ACM, 115–135. [30] Miikka Kuutila, Mika Mäntylä, Umar Farooq, and Maëlick Claes. 2020. Time pressure in software engineering: A systematic review. Information and Software Technology 121 (2020), 106257. doi:10.1016/j.infsof.2020.106257 [31] Huiting Liu, Xiyuan Zhang, Jiangping Zhou, Yuancong Shou, Yang Yin, and Chunlei Chai. 2024. Cognitive Styles and Design Performances in Conceptual Design Collaboration with GenAI. International Journal of Technology and Design Education (2024). doi:10.1007/s10798-024-09937-y [32] Ryan Louie, Andy Coenen, Cheng Zhi Huang, Michael Terry, and Carrie J Cai. 2020. Novice-AI music co-creation via AI-steering tools for deep generative models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, 1–13. [33] Mary Lou Maher, Josiah Poon, and Sylvie Boulanger. 1996. Formalising Design Exploration as Co-Evolution. In Advances in Formal Design Methods for CAD: Proceedings of the IFIP WG5.2 Workshop on Formal Design Methods for ComputerAided Design, June 1995, John S. Gero and Fay Sudweeks (Eds.). Springer US, Boston, MA, 3–30. doi:10.1007/978-0-387-34925-1_1 [34] Neil Maiden, Sara Jones, Kristine Karlsen, Roger Neill, Konstantinos Zachos, and Alastair Milne. 2010. Requirements engineering as creative problem solving: A research agenda for idea finding. In 2010 18th IEEE International Requirements Engineering Conference. IEEE, 57–66. [35] Neil Maiden, Cornelius Ncube, and Suzanne Robertson. 2007. Can requirements be creative? experiences with an enhanced air space management system. In 29th International Conference on Software Engineering (ICSE’07). IEEE, 632–641. [36] Neil Maiden, Konstantinos Zachos, Kostas Petrianakis, James Lockerbie, Chantakan Chanpalangsri, Holger Ernst, and Seyfettin Kara. 2025. Beyond just generative AI for discovering software opportunities . IEEE Software 42, 03 (Feb. 2025), 7–12. doi:10.1109/MS.2025.3540672 [37] Fairuz Nawer Meem, Justin Smith, and Brittany Johnson-Matthews. 2025. Why Do Software Practitioners Use ChatGPT for Software Development Tasks?. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering (Clarion Hotel Trondheim, Trondheim, Norway) (FSE Companion ’25). Association for Computing Machinery, New York, NY, USA, 1508–1514. doi:10.1145/3696630.3731667 [38] Rahul Mohanani, Paul Ralph, Burak Turhan, and Vladimir Mandić. 2022. How Templated Requirements Specifications Inhibit Creativity in Software Engineering. IEEE Transactions on Software Engineering 48, 10 (Oct. 2022), 4074–4086. doi:10.1109/TSE.2021.3112503 [39] Rahul Mohanani, Prabhat Ram, Ahmed Lasisi, Paul Ralph, and Burak Turhan. 2017. Perceptions of Creativity in Software Engineering Research and Practice. In 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE, 210–217. doi:10.1109/SEAA.2017.21 [40] Rahul Mohanani, Burak Turhan, and Paul Ralph. 2021. Requirements Framing Affects Design Creativity. IEEE Transactions on Software Engineering 47, 5 (May 2021), 936–947. doi:10.1109/TSE.2019.2909033 [41] Stoyan Nikolov, Daniele Codecasa, Anna Sjövall, Maxim Tabachnyk, Satish Chandra, Siddharth Taneja, and Celal Ziftci. 2025. How is Google using AI for internal code migrations?. In 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 481–492. [42] Guilherme Vaz Pereira, Victoria Jackson, Rafael Prikladnicki, André van der Hoek, Luciane Fortes, Carolina Araújo, André Coelho, Ligia Chelli, and Diego Ramos. 2025. Exploring GenAI in Software Development: Insights from a Case Study in a Large Brazilian Company. In 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 330–341. [43] Marian Petre and Mary Shaw. 2025. Contrasting to spark creativity in software development teams . IEEE Software 42, 03 (Feb. 2025), 67–74. doi:10.1109/MS. 2025.3538670 [44] Marian Petre and André van der Hoek. 2016. Software Design Decoded: 66 Ways Experts Think. MIT Press. [45] Saima Ritonummi, Valtteri Siitonen, Markus Salo, Henri Pirkkalainen, and Anu Sivunen. 2023. Flow Experience in Software Engineering. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (San Francisco, CA, USA) (ESEC/FSE 2023). Association for Computing Machinery, New York, NY, USA, 618–630.
Jackson, Liebel, Prikladnicki, van der Hoek
doi:10.1145/3611643.3616263 [46] Mohammad Ronagh Nikghalb and Jinghui Cheng. 2025. Interrogating AI: Characterizing Emergent Playful Interactions with ChatGPT. Proc. ACM Hum.-Comput. Interact. 9, 2, Article CSCW117 (May 2025), 23 pages. doi:10.1145/3711015 [47] Mark A. Runco and Garrett J. Jaeger. 2012. The Standard Definition of Creativity. Creativity Research Journal 24, 1 (Jan. 2012), 92–96. doi:10.1080/10400419.2012. 650092 [48] Benjamin Saunders, Julius Sim, Tom Kingstone, Shula Baker, Jackie Waterfield, Bernadette Bartlam, Heather Burroughs, and Clare Jinks. 2018. Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & Quantity 52, 4 (2018), 1893–1907. doi:10.1007/s11135-017-0574-8 [49] Orit Shaer, Angelora Cooper, Osnat Mokryn, Andrew L Kun, and Hagit Ben Shoshan. 2024. AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 1050, 17 pages. doi:10.1145/3613904.3642414 [50] Ben Shneiderman. 2022. Human-Centered AI (1 ed.). Oxford University Press. [51] Kaile Smith, Alan Pickering, and Joydeep Bhattacharya. 2022. The Creative Life: A Daily Diary Study of Creativity, Affect, and Well-Being in Creative Individuals. Creativity Research Journal 34, 4 (Oct. 2022), 460–479. doi:10.1080/10400419.2022. 2122371 Publisher: Routledge. [52] Samangi Wadinambiarachchi, Ryan M. Kelly, Saumya Pareek, Qiushi Zhou, and Eduardo Velloso. 2024. The Effects of Generative AI on Design Fixation
and Divergent Thinking. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (2024-05-11) (CHI ’24). Association for Computing Machinery, Hawaii, USA, 1–18. doi:10.1145/3613904.3642919 [53] Jialiang Wei, Anne-Lise Courbis, Thomas Lambolais, Gerard Dray, and Walid Maalej. 2025. On AI-Inspired UI-Design . IEEE Software 42, 03 (Feb. 2025), 50–58. doi:10.1109/MS.2025.3536838 [54] Justin D. Weisz, Shraddha Vijay Kumar, Michael Muller, Karen-Ellen Browne, Arielle Goldberg, Katrin Ellice Heintze, and Shagun Bajpai. 2025. Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25). Association for Computing Machinery, New York, NY, USA, Article 673. doi:10.1145/3706599. 3706670 [55] Zhikun Wu, Thomas Weber, and Florian Müller. 2025. One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor. In Proceedings of the 30th International Conference on Intelligent User Interfaces (IUI ’25). Association for Computing Machinery, New York, NY, USA, 1082–1092. doi:10.1145/3708359.3712094 [56] Albert Ziegler, Eirini Kalliamvakou, X Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian. 2024. Measuring GitHub Copilot’s Impact on Productivity. Commun. ACM 67, 3 (2024), 54–63.