Case Studies and Reflections on Agentic Software Engineering for Rapid Development of Digital Music Instruments Matthew John Yee-King
arXiv:2605.14016v1 [cs.SE] 13 May 2026
Computing, Goldsmiths London, UK
Figure 1: NIMEs programmed by Codex.
Abstract
1
The article explores the use of agentic software engineering (ASE) in the development of innovative audio software. It begins with a review of background work that lays out the challenges of longevity, interoperability and barriers to entry in digital music instrument creation, explaining recent developments in ASE and highlighting the possibility that ASE can lower barriers to entry and facilitate creation of interoperable software with greater longevity. Following that, we present case studies wherein we used ASE technology in three distinct ways to develop audio software in the C++ language with the JUCE framework. In case study 1, we re-implement Laurie Spiegel’s ‘Music Mouse’ software as a native plugin. In case study 2, we translate Pachet’s ‘Continuator’ system from Python into a native plugin. In case study 3, we develop a new 3D user interface for an existing ‘tracker’ sequencer using OpenGL. We describe the experiences of the human developer in the case studies via autoethnographic discussion of the prompt logs and snapshots of the software as it was developed. We identify effective practice for ASE use in this domain and suggest future steps for the work involving evaluation of the method with non-programmer musicians.
In this paper, I explore the possibilities of agentic software engineering for audio software development. Agentic software engineering (ASE) involves humans developing software in collaboration with tool-using large language models which through an ‘agentic’ layer, can develop and carry out plans. The human user inputs text, image and other media prompts and the agent enacts its plan by reading and analysing source code and documentation, creating and editing files, running build commands and so forth. The particular type of software I consider in the paper is ‘software digital music instruments’ which I will henceforth refer to as SDMIs for brevity. I focus on two areas of interest to the NIME community: the problem of longevity and interoperability and the challenge of lowering the barrier to entry for the potentially large community of non-programmers who wish to experiment with the creation of new music technology. I explore these questions through three distinct audio software development case studies: re-creating a well-known SDMI (Music Mouse) from the user manual and screenshots, translating an SDMI (Continuator) from Python to C++ and developing a new 3D OpenGL user interface for my own tracker sequencer. This article was inspired by my recent experiences working with ASE tools to develop audio software. My experience has been that the tools very significantly accelerate audio software development cycles and that with an appropriate starting template and documentation, it is possible to develop interoperable audio software plugins in C++ entirely through natural language prompts, with limited knowledge of the underlying implementation. Thus, there is a lot of potential here for increasing interoperability of existing SDMIs (e.g. through translation into plugins) and lowering the barrier to entry to a wider community of SDMI creators. Whilst there is potential, there is limited published work exploring the use of ASE in the audio software domain, which is well-known to be a challenging area for software development.
Keywords C++, Vibe-coding, Agentic Software Engineering, Audio software development, Plugins
This work is licensed under a Creative Commons Attribution 4.0 International License. Goldsmiths Computing, London, UK, © 2026 Copyright held by the owner/author(s).
Introduction
Goldsmiths Computing, London, UK, Trovato et al.
This is especially true when one considers the need to create interoperable, standardised software such as plugins with which musicians are comfortable and familiar because plugins are generally written in C++, a notoriously hard language to learn. To address this gap in the published work, I report on some initial experiments I have conducted to evaluate the capabilities of these emerging technologies. The contributions of the paper are as follows: (1) A description of a methodology for developing native audio software using ASE technology (2) The application of the methodology to three distinct case studies and reflections on problems encountered and how they were overcome (3) A discussion of the potential of ASE technology to address well known problems in research-oriented SDMI development, namely longevity, interoperability and the barrier to entry for non-programmers
2 Previous work 2.1 Making NIME and SDMI development accessible Developing more accessible and efficient means to get from idea to prototype is not a new activity for computer music researchers. There is a long history of specialised programming languages and environments from Music-N through CSound, Max/MSP, PureData, SuperCollider and Faust and onto Bela, Livecoding languages and so forth, many of which are discussed by Dannenberg[6] Whilst Dannenberg claims that “Computer music languages have enabled composers who are not software engineers to nevertheless use computers effectively”, anyone who has taught a class to composers about a textual language such as SuperCollider will be aware of the difficulty learners face in setting up the tools, then the brittleness and abstractness of the languages. For example, seasoned SuperCollider educator Collins discusses the challenges of transitioning learners from parameter editing to more complex program design[4]. A lot of work has been done to address these issues: concerning the problems of brittleness and abstractness, visual languages such as Max/MSP are less brittle and less abstract and educators report significant success with non-programmers[25]. Concerning tool setup, a widely used solution is the use of ‘zerosetup’, web-based environments such as the educationally focused MIMIC platform[13, 27], the web-based version of the Tidal Cycles livecoding system Strudel[7, 14, 22], and the web interface provided with the Bela hardware platform[8]. Another problem associated with computer music languages, and a popular subject in the literature, is that of longevity and sustainability of the software created using them. For example, in a review of 40 years of NIMEs, Sullivan and Wanderley stated that “a DMI may not be viable unless basic stability, reliability and compatibility standards have been met”[24]. An emerging trend which addresses compatibility is the development of plugins which interoperate with standard audio software used by musicians[26]. Examples are the DrumGAN and RAVE VSTs[2, 15]. The problem with plugins is that they are typically written in the C++ language which is notoriously difficult. To quote Bjarne Stroustrup, the creator of C++: “C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your
whole leg off”1 . Given this, it seems reasonable to assert that the barrier to entry for non-programmer musicians to plugin development is higher than that for, say, Max/MSP. Though we should note that plugins are not the only route to interoperability, for example, ‘Max for Live’ allows Max/MSP patches to run inside Ableton Live2 ; the Faust IDE allows users to develop in the Faust language and to use the web platform to directly compile their program into a binary VST plugin (as well as many other formats)[11]. But if musicians wish to develop software that integrates with their other music tools, C++, Max for Live and Faust all share the requirement to operate and express ideas in the unfamiliar medium of a programming language (even if it is a visual language like Max).
2.2
From code completion to agentic software engineering
Vibe-coding became the Collins English Dictionary word of the year in 2025, which defined it as “the use of artificial intelligence prompted by natural language to assist with the writing of computer code”[5]. LLM-supported programming did not start in 2025 - OpenAI described the first Codex model in 2021 as a “a GPT language model fine-tuned on publicly available code from GitHub”. In the HumanEval benchmark, this initial version of Codex was able to provide functionally correct Python code in response to docstrings (i.e. to write code from natural language descriptions) for 28.8% of the problem set. As of 2025, GPT-5 (version unspecified) scores 93.5%, with the Kimi-K2 open-weights model surpassing it at 94.5% 3 . Through the 2020s, the capabilities of LLM-powered coding tools rapidly expanded with the addition of features such as codebase indexing and command-calling with custom-designed IDEs such as Cursor4 . These extended capabilities of LLMs whereby they can ‘do things’ in a computer system are referred to as tooluse[20]. There are different ways to present a set of tools to an LLM; for example, Anthropic defined the Model Context Protocol (MCP) in 2024 and it allows the description of tools and their execution5 [16]. A further innovation is the reasoning LLM, which carries out an iterated process of reasoning prior to emitting a result[19]. Finally, ‘Agentic AI’ combines LLMs, tool-use and reasoning into a software agent which can adaptively develop and carry out plans on a computer system[1] For programming tasks, the current manifestation of all these capabilities is referred to as agentic software engineering[9]. In October 2025, OpenAI made their latest iteration of Codex ‘generally available’ (i.e. to lower-tier, paying customers), and it supports multi-modal agentic software development using build tools on your local machine6 . The Codex agent itself is open source7 but it calls out to dynamically selected, closed source, closed weights language models running in the cloud. Codex is the tool used in the work presented in this article, but there are other ASE systems available such as Anthropic’s Claude Code8 . Tt is even possible to run the agent and the models on your own
1 https://www.stroustrup.com/quotes.html
2 https://cycling74.com/products/maxforlive
3 https://llm-stats.com/benchmarks/humaneval, https://huggingface.co/moonshotai/Kimi-
K2-Instruct-0905
4 https://cursor.com/
5 https://www.anthropic.com/news/model-context-protocol 6 https://openai.com/index/introducing-gpt-5-2-codex/
7 https://github.com/openai/codex
8 https://github.com/anthropics/claude-code
Goldsmiths Computing, London, UK, Case Studies and Reflections on Agentic Software Engineering for Rapid Development of Digital Music Instruments
Table 1: Three case studies using agentic software engineering to develop audio software Case study Music Mouse Continuator MYK-Tracker
Process type From manual and images Translate New UI
Template code JUCE CMake project JUCE CMake/ HTML UI project JUCE CMake project
Documentation input Manual, UI screenshots Python code C++ code
machine with tools such as Roocode9 , LMStudio10 , and open weights models11 , though you will need an expensive system. Compared to the chat-style interaction and code completion available in the first generation of ‘vibe-coding’ systems, the full agentic software engineering workflow is remarkably powerful and, perhaps most importantly for the widening access theme in this paper, highly autonomous. The author’s experiments developing audio software in C++ with Codex in late 2025 inspired this paper.
2.3
The basic process I carried out in this research was to identify three interesting software projects or case studies, then to use ASE technology to interactively develop the software, whilst recording data about my experience. More details of the data capture are presented below. Each case study explores a distinct ‘challenge’, as described in table 1. The challenges are 1) create a plugin from scratch to a specification, 2) translate from a Python web application to a C++ plugin and 3) create a new 3D OpenGL interface for an existing audio program. In acknowledgement of the ‘longevity of NIMEs’ theme of this paper, I decided that two of the case studies would involve the re-implementation of well-known computer music systems from the past. This has been a theme in previous NIME papers such as Masu et al. who re-implemented NIMEs from past conferences in 2023[12], and Calegario et al. who evaluated documentation for a range of NIMEs to evaluate reproducibility[3].
Using language models to write audio software
There is an emerging body of work discussing ASE in general, but there is limited work discussing the use of LLM technology for audio software development. Zhane et al. benchmarked LLMs in their abilities to work in visual dataflow languages, generating programs in Max/MSP, MaxPy, JavaScript/ Web Audio API and their own visual language Wavir[28]. The work presented here differs in that we discuss the creation of large, complex programs in C++ through multi-stage agentic methods, instead of just working on a prompt-to-code basis. Smith discussed the use of LLMs to convert the filter design Python code in scipy into a C++ library with the objective of creating an audio plugin that allows you to design your own filters[10]. Similarly, one of the case studies reported below explores translation from Python to C++.
3
Method: three case studies in agentic audio software engineering Figure 3: Project file structure for WebView starter template (top left) and JUCE UI template (tio right). User interfaces for WebView (bottom left) and JUCE UI (bottom right). The JUCE git repository sits in the libs folder
3.1
Programming environment and project templates
9 https://github.com/RooCodeInc/Roo-Code
I used the Codex ASE system to develop the software described in the case studies. You can see a screenshot of the Codex extension running inside VSCode in figure 2. I worked in the C++ language on top of the JUCE framework12 . JUCE is a widely used, open-source, cross-platform framework which provides project build management and a C++ class library for audio and GUI development. JUCE allows the development of VST, AudioUnit and other plugin formats as well as standalone and command-line audio programs. From previous work[anon], I had two template projects which I used as starting points for development. The most basic template was written completely in C++ with the user interface built using the JUCE GUI classes. The other template used an HTML/CSS user interface which would be embedded to the native interface
11 https://huggingface.co/
12 https://juce.com/
Figure 2: Interacting with codex in the VSCode extension.
10 https://github.com/lmstudio-ai
Goldsmiths Computing, London, UK, Trovato et al.
via a WebView component. The web UI interacts with the audio code via REST API calls and JavaScript callbacks. This workflow is an emerging trend in plugin development which allows developers to develop their interfaces in HTML and CSS instead of the more limited and obscure native GUI libraries available. I expected Codex to be able to work very well in HTML and CSS, as it would have featured much more strongly in its training data than, for example, the JUCE UI library. Figure 3 shows the file structure of the two templates and the initial user interfaces they provided. The projects included a CMake build configuration that would allow them to be built on the command line. They also included a complete copy of the JUCE framework in a ‘libs’ sub-folder with examples, documentation and library code that Codex could explore. Prompting of Codex was done with text and images.
3.2
Data capture and analysis
I aimed to use a light form of auto-ethnography to analyse my experiences and thoughts when developing the software. To this end, I collected the following data whilst carrying out the development: (1) Screencast recordings of my computer screen (2) Audio recordings with transcripts from a mic that allowed me to comment on what I was doing and experiencing whilst working with the agent (3) Detailed, timestamped logs of all user and agent actions in JSON format, including the prompt sequence (4) The source code itself To analyse the data, I planned to watch the screencasts, to listen to the commentary, to read the generated source code and to use automated tools to analyse and visualise the log data.
3.3
Case study 1: re-creation of a classic SDMI from the user manual and screenshots (Music Mouse)
the system generates chord and single note events according to a range of parameter settings. Music Mouse was commercially available during the 80s and 90s for the Atari ST, Commodore Amiga and MacOS platforms. Whilst the original last release, V2.1.2 Classic was made available in 2005, I discovered during the preparation of this paper that Eventide planned to re-release Music Mouse in 202613 - mit dem Zeitgeist leben!
3.4
Case study 2: Python to C++ native translation of classic SDMI (Continuator)
Pachet’s Continuator is a Markov-based system designed for interactive musical sequence generation, originally created in the early 2000s[17]. Later developments involved advanced techniques for steering sequence generation using constraints[18]. Continuator was implemented in the Java language, with no source code available, but in 2025 Pachet re-implemented the system in Python with a gradio web interface and released the code on GitHub14 . In this case study, I converted the Python code into a C++ VST/AudioUnit plugin using the WebView template, since the Python version also had a web interface.
3.5
Case study 3: user interface conversion from JUCE to OpenGL ([anon]-tracker)
In the final case study, I investigated the capabilities of ASE techniques in converting the user interface for a fairly complex ‘tracker’ sequencer from JUCE UI components to an OpenGL UI. This would allow me to observe the agent working in a preexiting, moderately complex codebase. Trackers are a form of music sequencer often associated with the Chiptune scene, but they were also used extensively in 1990s UK electronic music by artists such as Black Dog[21]. I created the open-source tracker software some time ago with purely human C++ programming, aiming to run it on a small system such as a Raspberry Pi15 . In the original project, I started out with a command-line curses interface without JUCE and later adapted that to a JUCE plugin with JUCE UI components so I could more easily implement DSP functionality. In the case study, I planned to work with ASE to implement a completely new user interface.
4
Results and analysis
In figure 5, I present visualisations of the activities that took place in each case study. There you can see when I wrote prompts, when Codex used tools and the number of tokens processed over time. In the following subsections, I will describe my activity and observations during the three case studies.
4.1
Figure 4: Music Mouse running on an emulated Atari ST; Music Mouse was also available for Amiga and MacOS. The first case study involved re-implementing an SDMI from scratch using the basic JUCE UI plugin template. I selected an SDMI from the literature that was a) well documented, b) software only, not hardware, and c) had an interesting/ distinctive user interface. I selected Laurie Spiegel’s Music Mouse software for re-implementation as it meets the three criteria[23]. In Music Mouse, the user moves the mouse around in a note grid and
Case study 1: Music Mouse
The Music Mouse session is shown at the top of figure 5. The visualisation begins with the first prompt, but prior to that I spent around 20 minutes preparing the project repository. I located a copy of the Music Mouse manual in HTML format on the Wayback Machine which I downloaded and placed into a ‘docs’ folder in the project folder. Next, I searched the Internet for images of the original Music Mouse software, as there were no screenshots in the manual. I found an image of the MacOS version of the program and an image representing an Atari ST keyboard shortcut ‘cheat-sheet’. I placed these in the docs folder as well. Next, I prepared the example template, which was the basic JUCE 13 https://www.eventideaudio.com/software/music-mouse/ 14 https://github.com/fpachet/continuator 15 anon
Goldsmiths Computing, London, UK, Case Studies and Reflections on Agentic Software Engineering for Rapid Development of Digital Music Instruments music-mouse-log
0m
5m
10m
15m
20m
25m
30m
35m
40m
45m
User prompts Agent tool calls File edits/creates
Accumulated token count 2,500,000 2,000,000 1,500,000 1,000,000 500,000 0
0m
15m
continuator-log
30m
45m
4m
9m
14m
19m
4m
9m
14m
19m
User prompts Agent tool calls File edits/creates
4,000,000
Accumulated token count
3,000,000 2,000,000 1,000,000 0
tracker-real-gl-conversion 0m
5m
10m
15m
20m
25m
30m
35m
40m
45m
50m
55m
60m
65m
70m
75m
80m
85m
User prompts Agent tool calls File edits/creates
Accumulated token count 3,000,000 2,000,000 1,000,000 0
0m
15m
30m
45m
60m
75m
Figure 5: Timelines for the three case studies: Music Mouse (top), Continuator (middle), OpenGL tracker interface (bottom). Rows of circles show actions: first row prompts, second row tool calls, third row file edit actions. Graph shows accumulated tokens sent to the LLM. Codex was used to prepare this visualisation. / CMake plugin project consisting of a CMakeLists.txt file plus the cpp and header files for the user interface and audio modules. I cloned the JUCE repository into a ‘libs’ sub folder and ran a test build to verify everything worked. I was now ready to prepare the first prompt. In the prompt I explained the objective, to re-implement Music Mouse, and described the structure of the repository including the location of the user manual and the JUCE repository. I also highlighted the importance of the mouse interaction and the keyboard controls. I included the MacOS screenshot with the prompt. About eight minutes later Codex had designed and carried out its first plan, which involved parsing the manual using Python commands, implementing the mouse and keyboard interaction models, and building a MIDI event queueing system. In the next phase I tested the plugin by instantiating it in a plugin hosting
environment. I wired it into a MIDI logger plugin to show the MIDI events it was generating, and an emulated Casio CZ100 synthesizer (as mentioned in the Music Mouse documentation I had read) so I could hear the notes. I tried various keyboard shortcuts and mouse movements and was able to control transposition and chord generation. Then I developed several further prompts to improve firstly the note sending behaviour and then the way the user interface indicated the notes being played. I began to use an efficient workflow wherein I would be testing the plugin whilst Codex was working on the next build, then writing a new prompt ready to set off the next development cycle. Overall, the session lasted around 50 minutes and resulted in what seemed to be a reasonable implementation of the core keyboard and mouse interactions
Goldsmiths Computing, London, UK, Trovato et al.
Figure 6: Four iterations of the Music Mouse reimplementation, clockwise from top left. V1 approximates the user interface with a mostly working mouse to MIDI interaction and keyboard controls; V2 includes the piano display, V3 adds graphical indications of the note positions and V4 enhances the horizontal and vertical bars. for Music Mouse. Main phases in the session were: project preparation, major prompt, plugin testing then smaller prompting and testing iterations.
4.2
Case study 2: Continuator
Figure 7 illustrates the progression of the Continuator plugin user interface during the session, including the original Python web interface. As for Music Mouse, the first phase involved setting up the project folder, except now I used the WebView plugin template. I added a clone of the Python Continuator repository and an example of a command-line JUCE unit-testing program. The unit-testing example code was there because I planned to include a request for unit-tests of the Continuator algorithm in my prompts. Next, I developed and sent the first prompt which explained briefly what Continuator does and that I wanted a translation of the Python project into a plugin with a web interface. I also included a request to write unit tests to validate the algorithm. After 12 minutes of work, Codex returned; I ran the plugin and the core of the algorithm appeared to be in place in that you could load and model a MIDI file and then generate note events from the model in text format. I re-organised the files in the repository to my liking and asked Codex to fix the build with the new arrangements. I returned to the project some days later for the next phase wherein my aim was to get the plugin to not just generate its output in text format but also to emit correctly timed MIDI messages. One prompt and four minutes later, the plugin was emitting MIDI messages. I tested in the Reaper DAW, connecting the output to a Rhodes electric piano sound. The second stage of the session is not shown in figure 5 but it follows a similar pattern to the other timelines, with the first prompt taking a longer time to process followed by multiple, smaller prompt processing stages. At this point, I felt compelled to contact Continuator’s original creator Francois Pachet (with whom I occasionally correspond), so I sent him a video of the plugin in action. He was very impressed to find out that it wasn’t just a front-end for the Python code but a complete, native translation. I worked with him to get the CMake build running on his system which was very straightforward and he easily ran the AudioUnit version of the plugin in the Logic Pro DAW. He highlighted some interesting problems such as what would be the canonical version to which bug fixes
Figure 7: Original Python Continuator interface (left) then three iterations of the Codex written version from left to right. V1 can model MIDI files and generate constrained output, V2 can generate the output as correctly timed MIDI messages and V3 implements the full Continuator call and response interaction. were applied - the Python or C++ implementation? Also how do you know it is functioning correctly? We discussed next steps and he was keen to see the complete Continuator ‘call and response’ behaviour implemented so I worked with Codex to implement that which took two further phases and a few minutes. I ended the session there. Compared to Music Mouse, the Continuator case study took place over several sessions, but it followed a similar pattern of preparation, major prompt, prompt processing, testing then multiple smaller prompt-test iterations.
4.3
Case study 3: OpenGL tracker interface
This case study took place over a single, longer session of around 85 minutes. Unlike the other studies, I spent more iterations making finer adjustments to the user interface as I had quite specific ideas about how it should look and work. You can see in the timeline visualisation that there were 10 prompts in total. Figure 8 shows the original tracker interface then two versions from the development of the new, 3D OpenGL interface, as well as a later iteration running in a custom hardware setup. In the first prompt, which took Codex around 10 minutes to respond to, I included references to examples in the libs/JUCE/examples folder which the agent could examine to find out how to add OpenGL
Goldsmiths Computing, London, UK, Case Studies and Reflections on Agentic Software Engineering for Rapid Development of Digital Music Instruments
To summarise, this case study had similar patterns to the other two - starting with a solid foundation in the project folder, identifying examples that Codex could build on, the big prompt at the start followed by multiple smaller prompts. But I added some more techniques such as requesting explanation and manually editing code.
5
Figure 8: Four iterations of the tracker UI: original JUCE version (far left), V1 with text overlay problem (centre left), V2 with correctly overlaid text and mouse-controlled camera (centre right), then a much later iteration running in a custom physical enclosure (far right).
capability to a JUCE program. I also attached a screenshot of the current, 2D interface. I did not provide much detail on how the keyboard controls worked but referred it to a particular C++ class in the project which contained the implementation, so unlike the other case studies, I was using specialised knowledge of my codebase in my prompts. The result of the first prompt was a non-working build so I prompted Codex to run the cmake command and fix any errors, which it did in about 30 seconds, resulting in the top right image in figure 8. At this stage, the 3D interface was in place, with the playhead cursor and edit cursor moving correctly but the data for the steps in the sequence was overlaid in 2D instead of being overlaid on the surfaces of the blocks. I sent Codex a screenshot of the interface as it was along with a new prompt and it was able to correctly overlay the text on the surface of the blocks. The implementation was interesting in that it generated a cached texture map of the text and overlaid it on the blocks. In the next phase I instructed the agent to add mouse control over the camera so I could zoom in and out and change perspective. In 30 seconds, it had completed this task successfully, resulting in the interface shown on the bottom left of figure 8. I returned later to the project for a few more interactions and was able to work towards the interface shown running in its custom hardware in the bottom right of figure 8. To get to this version, I used a combination of prompting and manual code editing. In some of the prompts, I asked Codex to explain how its code worked so I could find where to make my edits. I also instructed it to refactor the code to a more modular style. So, this case study stands apart from the others in that I did make use of my C++ skillset in various ways to speed things up. For example, using a natural language prompt to adjust a parameter that controls the decay rate of the orange glow in the boxes seemed much less efficient than asking where that parameter was and editing it myself.
Discussion
I started the paper by identifying two themes which would be of interest to the NIME community: the problem of longevity and interoperability, and the challenge of lowering the barrier to entry for the potentially large community of non-programmers who wish to experiment with the creation of new music technology. My aim was to explore the potential of agentic software engineering to address these challenges. In the Continuator case study, I was able to convert an SDMI written in Python and which depended on package installs and virtual environments, into a native plugin that would load in any DAW. I was able to do so only using natural language. In the Music Mouse case study, I was able to re-create an SDMI using only a user manual and some screenshots, again with natural language prompts. I did need some music technology domain knowledge, for example, around MIDI events and timing and generally understanding what plugins were and how they operated. I did not need any deep C++ knowledge. Considering the theme of lowering the barrier for non-programmers, it is clear that it is possible to develop plugins from natural language, but you do need a well-designed starter program, and you do need to have a set of somewhat complex tools installed correctly beforehand. One problem for non-programmers is failing builds with obscure error messages, but Codex can deal with this, including explaining how it did so. More advanced, yet critical plugin concepts such as thread-safe data sharing between audio and GUI modules can be implemented through natural language, if beginners understood the basics of these ideas and how to request them. In the third case study, I showed how it is possible to work on a reasonably complex, existing audio program using ASE techniques. Here my familiarity with the codebase naturally led me to cut in and edit the code by hand sometimes, but the ASE tools assisted here as they could also explain to me how the code worked and indicate where I could make my edits. Using the tools to refactor also made the AI-generated code easier to understand. I was not very familiar with OpenGL programming, and I enjoyed learning some new techniques in OpenGL. So, for the intermediate or advanced programmer, or someone wishing to re-animate a dead SDMI’s codebase there is plenty of value here too.
6
Conclusion and future work
In this paper, I have presented three case studies wherein I used agentic software engineering techniques to re-create an existing SDMI from its user manual, translate an existing SDMI from Python to C++, and to develop a new user interface. I have considered how my findings relate to the problems of longevity and interoperability in SDMIs as well as the barrier to entry for nonprogrammers to SDMI creation. There are many areas to explore in ASE for audio software. I have chosen not to dive into ethical aspects here, but certainly others might find this a rich vein to explore. An interesting direction for my work will be developing
Goldsmiths Computing, London, UK, Trovato et al.
workshop materials to support non-programmers in developing their own plugins. Another area with potentially high impact is the creation of a set of plugins that re-implement well known SDMIs from the literature, which would otherwise be lost in obscurity.
7
Acknowledgements
I would like to acknowledge the fantastic work of Laurie Spiegel and Francois Pachet and team who developed the original Music Mouse and Continuator systems. Whilst Codex was used extensively to carry out the research, LLM technology was not used in any way to prepare the text in this paper, aside from assisting in generating the timeline visualisation.
References [1] Mohamad Abou Ali, Fadi Dornaika, and Jinan Charafeddine. 2025. Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directions. Artificial Intelligence Review 59, 1 (Nov. 2025), 11. https: //doi.org/10.1007/s10462-025-11422-4 [2] Antoine Caillon and Philippe Esling. 2022. Streamable Neural Audio Synthesis With Non-Causal Convolutions. https://doi.org/10.48550/arXiv.2204.07064 arXiv:2204.07064 [cs] [3] Filipe Calegario, João Tragtenberg, Christian Frisson, Eduardo Meneses, Joseph Malloch, Vincent Cusson, and Marcelo M. Wanderley. 2021. Documentation and Replicability in the NIME Community. In NIME 2021. PubPub. [4] Nick Collins. 2016. Live Coding and Teaching SuperCollider. Journal of Music, Technology & Education 9, 1 (May 2016), 5–16. https://doi.org/10.1386/jmte.9. 1.5_1 [5] Collins English Dictionary. 2025. VIBE CODING Definition and Meaning. [6] Roger B. Dannenberg. 2018. Languages for Computer Music. Frontiers in Digital Humanities 5 (Nov. 2018). https://doi.org/10.3389/fdigh.2018.00026 [7] Georgios Diapoulis. 2024. Teaching Strudel to Young Girls: Realizing Live Coding through Performance Practice. [8] Liam Donovan, S. M. Bin, Jack Armitage, and Andrew P. McPherson. 2017. Building an IDE for an Embedded System Using Web Technologies. (2017). [9] Ahmed E. Hassan, Hao Li, Dayi Lin, Bram Adams, Tse-Hsun Chen, Yutaro Kashiwa, and Dong Qiu. 2025. Agentic Software Engineering: Foundational Pillars and a Research Roadmap. https://doi.org/10.48550/arXiv.2509.06216 arXiv:2509.06216 [cs] [10] Julius O. Smith III. 2024. Using AI to Port Python’s SCIPY.SIGNAL FilterRelated Functions to C++ for Use in Real Time. [11] Stéphane Letz, Romain Michon, and Yann Orlarey. 2024. WHAT’S NEW IN THE FAUST ECOSYSTEM IN 2024?. In International Faust Conference. [12] Raul Masu, Fabio Morreale, and Alexander Refsum. 31–2023. The O in NIME: Reflecting on the Importance of Reusing and Repurposing Old Musical Instruments. In New Interfaces for Musical Expression 2023. Mexico City. [13] Louis McCallum and Mick S. Grierson. 2020. Supporting Interactive Machine Learning Approaches to Building Musical Instruments in the Browser. In Proceedings of the International Conference on New Interfaces for Musical Expression. 271–272. [14] Alex McLean. 2014. Making Programming Languages to Dance to: Live Coding with Tidal. In Proceedings of the 2nd ACM SIGPLAN International Workshop on Functional Art, Music, Modeling & Design. ACM, Gothenburg Sweden, 63–70. https://doi.org/10.1145/2633638.2633647 [15] Javier Nistal, Cyran Aouameur, Ithan Velarde, and Stefan Lattner. 2022. DrumGAN VST: A Plugin for Drum Sound Analysis/Synthesis With Autoencoding Generative Adversarial Networks. https://doi.org/10.48550/arXiv.2206.14723 arXiv:2206.14723 [cs, eess] [16] Grigoris Ntousakis, Julian James Stephen, Michael V. Le, Sai Sree Laya Chukkapalli, Teryl Taylor, Ian M. Molloy, and Frederico Araujo. 2025. Securing MCPbased Agent Workflows. In Proceedings of the 4th Workshop on Practical Adoption Challenges of ML for Systems (PACMI ’25). Association for Computing Machinery, New York, NY, USA, 50–55. https://doi.org/10.1145/3766882.3767177 [17] Francois Pachet. 2003. The Continuator: Musical Interaction With Style. Journal of New Music Research 32, 3 (Sept. 2003), 333–341. https://doi.org/10. 1076/jnmr.32.3.333.16861 [18] François Pachet and Pierre Roy. 2011. Markov Constraints: Steerable Generation of Markov Sequences. Constraints 16, 2 (April 2011), 148–172. https://doi.org/10.1007/s10601-010-9101-4 [19] Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki Van Stein, and Thomas Bäck. 2026. Multi-Step Reasoning with Large Language Models, a Survey. Comput. Surveys 58, 6 (April 2026), 1–35. https://doi.org/10.1145/ 3774896 [20] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. https://doi.org/10.48550/arXiv.2307.16789 arXiv:2307.16789 [cs]
[21] Markku Reunanen. [n. d.]. Trackers: The Rise, Bloom and Later Developments of a Paradigm. Technical Report. [22] Felix Roos and Alex McLean. 2023. Strudel: Live Coding Patterns on the Web. In Proceedings of the 7th International Conference on Live Coding. [23] Laurie Spiegel. 1986. Music Mouse™ - An Intelligent Instrument Version for Atari ST. [24] John Sullivan and Marcelo Wanderley. 2018. Stability, Reliability, Compatibility: Reviewing 40 Years of NIME Design. Ph. D. Dissertation. McGill University/Université McGill. [25] Daniel Walzer and Mariana Lopez. 2020. Audio Education: Theory, Culture, and Practice. Routledge. [26] Matthew Yee-King and Mark d’Inverno. 2024. Strategies for Building AIenhanced Audio Software with Impact. In AIMC 2024. Goldsmiths, University of London. [27] Michael Zbyszynski, Mick Grierson, Matthew Yee-King, and Leon Fedden. 2017. Write Once Run Anywhere Revisited: Machine Learning and Audio Tools in the Browser with C++ and Emscripten. In ICMC 2017. Goldsmiths, University of London. [28] William Zhang, Maria Leon, Ryan Xu, Adrian Cardenas, Amelia Wissink, Hanna Martin, Maya Srikanth, Kaya Dorogi, Christian Valadez, Pedro Perez, Citlalli Grijalva, Corey Zhang, and Mark Santolucito. 2024. Benchmarking LLM Code Generation for Audio Programming with Visual Dataflow Languages. https://doi.org/10.48550/arXiv.2409.00856 arXiv:2409.00856 [cs]