2026
Matthew Peachey
Investigating Continuous Interactions for generative Tools in the Context of Musical Creativity PhD Thesis
Dalhousie University, 2026.
Abstract | BibTeX | Tags: continuous interaction, continuous-interaction-and-machine-learning, control mapping, creativity support tools, digital musical instruments, machine learning, music, sound & music computing
@phdthesis{Peachey2026b,
title = {Investigating Continuous Interactions for generative Tools in the Context of Musical Creativity},
author = {Matthew Peachey},
year = {2026},
date = {2026-10-19},
urldate = {2026-10-19},
school = {Dalhousie University},
abstract = {This thesis investigates continuous interaction as a design principle for Artificial Intelligence (AI)-powered music creativity support tools. Discrete text prompting, the de facto standard for current generative AI systems, can limit musicians’ sense of artistic agency and creative ownership. Continuous interaction instead can enable fine-grained, real-time control over the generative process, helping musicians retain a sense of authorship over the generated output.
This thesis presents three studies that together examine how continuous interaction, integrated into real-time interfaces, can better support musicians’ creative workflows. First, I derived low-dimensional latent representations from a Variational Autoencoder with an intentionally small latent space, integrated them into a real-time GUI, and evaluated them in a mixed-methods study with both novice and experienced synthesizer users (n = 18). Second, I interviewed experienced musicians (n = 10) about their hesitations toward generative AI; their concerns centered on a lack of compelling tools and workflows, further motivating continuous interaction as a design preference. Finally, I developed a framework for Aging Digital Musical Instruments (AgingDMIs), a novel class of instruments that adapt to a user’s play style over time. I then implemented a hardware prototype instantiating one point in the AgingDMI design space, and evaluated it longitudinally with experienced musicians (n= 4) for twelve recurring sessions over several weeks.
The findings presented in this thesis suggest that integrating continuous interaction into generative AI-powered creativity support tools increases musicians’ artistic agency and creative ownership and fits better within their existing creative workflows. I further contribute a validated
design framework for AgingDMIs, a novel class of adaptive and personalized digital musical instruments, and a hardware prototype that together demonstrate the principles of that framework in practice.},
keywords = {continuous interaction, continuous-interaction-and-machine-learning, control mapping, creativity support tools, digital musical instruments, machine learning, music, sound & music computing},
pubstate = {published},
tppubtype = {phdthesis}
}
This thesis investigates continuous interaction as a design principle for Artificial Intelligence (AI)-powered music creativity support tools. Discrete text prompting, the de facto standard for current generative AI systems, can limit musicians’ sense of artistic agency and creative ownership. Continuous interaction instead can enable fine-grained, real-time control over the generative process, helping musicians retain a sense of authorship over the generated output.
This thesis presents three studies that together examine how continuous interaction, integrated into real-time interfaces, can better support musicians’ creative workflows. First, I derived low-dimensional latent representations from a Variational Autoencoder with an intentionally small latent space, integrated them into a real-time GUI, and evaluated them in a mixed-methods study with both novice and experienced synthesizer users (n = 18). Second, I interviewed experienced musicians (n = 10) about their hesitations toward generative AI; their concerns centered on a lack of compelling tools and workflows, further motivating continuous interaction as a design preference. Finally, I developed a framework for Aging Digital Musical Instruments (AgingDMIs), a novel class of instruments that adapt to a user’s play style over time. I then implemented a hardware prototype instantiating one point in the AgingDMI design space, and evaluated it longitudinally with experienced musicians (n= 4) for twelve recurring sessions over several weeks.
The findings presented in this thesis suggest that integrating continuous interaction into generative AI-powered creativity support tools increases musicians’ artistic agency and creative ownership and fits better within their existing creative workflows. I further contribute a validated
design framework for AgingDMIs, a novel class of adaptive and personalized digital musical instruments, and a hardware prototype that together demonstrate the principles of that framework in practice.
This thesis presents three studies that together examine how continuous interaction, integrated into real-time interfaces, can better support musicians’ creative workflows. First, I derived low-dimensional latent representations from a Variational Autoencoder with an intentionally small latent space, integrated them into a real-time GUI, and evaluated them in a mixed-methods study with both novice and experienced synthesizer users (n = 18). Second, I interviewed experienced musicians (n = 10) about their hesitations toward generative AI; their concerns centered on a lack of compelling tools and workflows, further motivating continuous interaction as a design preference. Finally, I developed a framework for Aging Digital Musical Instruments (AgingDMIs), a novel class of instruments that adapt to a user’s play style over time. I then implemented a hardware prototype instantiating one point in the AgingDMI design space, and evaluated it longitudinally with experienced musicians (n= 4) for twelve recurring sessions over several weeks.
The findings presented in this thesis suggest that integrating continuous interaction into generative AI-powered creativity support tools increases musicians’ artistic agency and creative ownership and fits better within their existing creative workflows. I further contribute a validated
design framework for AgingDMIs, a novel class of adaptive and personalized digital musical instruments, and a hardware prototype that together demonstrate the principles of that framework in practice.
Arsh Shah
RowSim: Designing and Evaluating Ambient Interaction in Mixed-Reality Rowing Masters Thesis
Dalhousie University, 2026.
Abstract | BibTeX | Tags: ambient display, control mapping, immersive visualization, IMU, projection mapping, rowing, sensor fusion, simulation
@mastersthesis{Shah2026,
title = {RowSim: Designing and Evaluating Ambient Interaction in Mixed-Reality Rowing},
author = {Arsh Shah},
year = {2026},
date = {2026-08-31},
urldate = {2026-08-31},
school = {Dalhousie University},
abstract = {A rowing ergometer reproduces the effort of rowing while removing its perceptual accom-
paniment, filling the gap with a console that must be read. Ambient-display research has
argued for three decades that information can occupy the periphery, but almost entirely for
people at rest.
RowSim augments an ergometer with a floor-projected fluid simulation driven by the
rower’s movement. Effort scales the wake’s brightness and how hard it pushes the fluid;
nothing is a number or recoverable as a value. Seventeen rowers, twelve casual and five
elite, rowed an open-ended ten-minute session with the console covered, then answered
questionnaires and an interview.
The groups differed by a factor of 3.7 in mean power, with all seven physical measures
separating them; no engagement measure did. Two participants independently reported that
the flow did not read as their own, because no action could stop it: restoring optic flow is a
coupling problem, not a rendering one.
The contribution is a working system and evidence that a peripheral, non-symbolic
display remains usable during exertion, together with a perceptual account of why the
missing accompaniment matters and of the one condition the display failed to meet.},
keywords = {ambient display, control mapping, immersive visualization, IMU, projection mapping, rowing, sensor fusion, simulation},
pubstate = {published},
tppubtype = {mastersthesis}
}
A rowing ergometer reproduces the effort of rowing while removing its perceptual accom-
paniment, filling the gap with a console that must be read. Ambient-display research has
argued for three decades that information can occupy the periphery, but almost entirely for
people at rest.
RowSim augments an ergometer with a floor-projected fluid simulation driven by the
rower’s movement. Effort scales the wake’s brightness and how hard it pushes the fluid;
nothing is a number or recoverable as a value. Seventeen rowers, twelve casual and five
elite, rowed an open-ended ten-minute session with the console covered, then answered
questionnaires and an interview.
The groups differed by a factor of 3.7 in mean power, with all seven physical measures
separating them; no engagement measure did. Two participants independently reported that
the flow did not read as their own, because no action could stop it: restoring optic flow is a
coupling problem, not a rendering one.
The contribution is a working system and evidence that a peripheral, non-symbolic
display remains usable during exertion, together with a perceptual account of why the
missing accompaniment matters and of the one condition the display failed to meet.
paniment, filling the gap with a console that must be read. Ambient-display research has
argued for three decades that information can occupy the periphery, but almost entirely for
people at rest.
RowSim augments an ergometer with a floor-projected fluid simulation driven by the
rower’s movement. Effort scales the wake’s brightness and how hard it pushes the fluid;
nothing is a number or recoverable as a value. Seventeen rowers, twelve casual and five
elite, rowed an open-ended ten-minute session with the console covered, then answered
questionnaires and an interview.
The groups differed by a factor of 3.7 in mean power, with all seven physical measures
separating them; no engagement measure did. Two participants independently reported that
the flow did not read as their own, because no action could stop it: restoring optic flow is a
coupling problem, not a rendering one.
The contribution is a working system and evidence that a peripheral, non-symbolic
display remains usable during exertion, together with a perceptual account of why the
missing accompaniment matters and of the one condition the display failed to meet.
2024
Matthew Peachey; Kyle Smith; Joseph Malloch
Extending Networked Mapping Research Middleware Into the Browser Sandbox Proceedings Article
In: Proceedings of the IEEE International Symposium on the Internet of Sounds, IEEE Communications Society Erlangen, Germany, 2024.
Links | BibTeX | Tags: control mapping, creativity support tools, interactive media systems, open source software
@inproceedings{Peachey2024_IS2,
title = {Extending Networked Mapping Research Middleware Into the Browser Sandbox},
author = {Matthew Peachey and Kyle Smith and Joseph Malloch},
url = {https://ieeexplore.ieee.org/document/10704162},
year = {2024},
date = {2024-09-30},
urldate = {2024-09-30},
booktitle = {Proceedings of the IEEE International Symposium on the Internet of Sounds},
address = {Erlangen, Germany},
organization = {IEEE Communications Society},
keywords = {control mapping, creativity support tools, interactive media systems, open source software},
pubstate = {published},
tppubtype = {inproceedings}
}
Matthew Peachey; Joseph Malloch
Maplet: Integrating Distributed Data Signal Mappings for Performative Interactions Within the Eurorack Modular Synthesizer Ecosystem Proceedings Article
In: Proceedings of the International Audio Mostly Conference, Milan, Italy, 2024.
Abstract | Links | BibTeX | Tags: control mapping, DMI mappings, interactive media systems, libmapper, music, open source software
@inproceedings{peachey2024maplet,
title = {Maplet: Integrating Distributed Data Signal Mappings for Performative Interactions Within the Eurorack Modular Synthesizer Ecosystem},
author = {Matthew Peachey and Joseph Malloch},
url = {https://dl.acm.org/doi/10.1145/3678299.3678333},
year = {2024},
date = {2024-09-20},
urldate = {2024-09-20},
booktitle = {Proceedings of the International Audio Mostly Conference},
address = {Milan, Italy},
abstract = {Eurorack is a popular format of modular synthesizers that allow users to customize their instruments by swapping in various modules and creating patches based on their specific needs. While these instruments are very effective at creating a wide range of timbres, the way they in which they are typically performed is not always engaging to an audience or even to the performer. This paper present an initial prototype of Maplet, a module for integrating the libmapper project into the world of Eurorack. libmapper is an open-source project built to support the creation of mappings between components of Digital Musical Instruments. Decades worth of literature has shown how effective mappings are the backbone of the way both performers as well as their audience engage with the output of a digital instrument. With this in mind, we hypothesize that by introducing this mapping-first approach to performing with Eurorack synthesizers that musicians will be empowered to experiment with novel interfaces and techniques for performing with their instrument.},
keywords = {control mapping, DMI mappings, interactive media systems, libmapper, music, open source software},
pubstate = {published},
tppubtype = {inproceedings}
}
Eurorack is a popular format of modular synthesizers that allow users to customize their instruments by swapping in various modules and creating patches based on their specific needs. While these instruments are very effective at creating a wide range of timbres, the way they in which they are typically performed is not always engaging to an audience or even to the performer. This paper present an initial prototype of Maplet, a module for integrating the libmapper project into the world of Eurorack. libmapper is an open-source project built to support the creation of mappings between components of Digital Musical Instruments. Decades worth of literature has shown how effective mappings are the backbone of the way both performers as well as their audience engage with the output of a digital instrument. With this in mind, we hypothesize that by introducing this mapping-first approach to performing with Eurorack synthesizers that musicians will be empowered to experiment with novel interfaces and techniques for performing with their instrument.
2023
Matthew Peachey; Joseph Malloch
FAUSTMapper: Facilitating Complex Mappings for Smart Musical Instruments Best Paper Proceedings Article
In: Proceedings of the 4th International Symposium on the Internet of Sounds (IS2), Pisa, Italy, 2023.
Links | BibTeX | Tags: control mapping, digital musical instruments, FAUST, libmapper, music
@inproceedings{Peachey2023b,
title = {FAUSTMapper: Facilitating Complex Mappings for Smart Musical Instruments},
author = {Matthew Peachey and Joseph Malloch},
url = {https://ieeexplore.ieee.org/document/10335404},
year = {2023},
date = {2023-10-26},
urldate = {2023-10-26},
booktitle = {Proceedings of the 4th International Symposium on the Internet of Sounds (IS2)},
address = {Pisa, Italy},
keywords = {control mapping, digital musical instruments, FAUST, libmapper, music},
pubstate = {published},
tppubtype = {inproceedings}
}
Brady Boettcher; Eduardo Meneses; Christian Frisson; Marcelo M. Wanderley; Joseph Malloch
Addressing Barriers for Entry and Operation of a Distributed Signal Mapping Framework Proceedings Article
In: Proceedings of the International Conference on New Interfaces for Musical Expression (NIME), Mexico City, Mexico, 2023.
BibTeX | Tags: collaborative systems, control mapping, music
@inproceedings{Boettcher2023,
title = {Addressing Barriers for Entry and Operation of a Distributed Signal Mapping Framework},
author = {Brady Boettcher and Eduardo Meneses and Christian Frisson and Marcelo M. Wanderley and Joseph Malloch},
year = {2023},
date = {2023-05-31},
urldate = {2023-05-31},
booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression (NIME)},
address = {Mexico City, Mexico},
keywords = {collaborative systems, control mapping, music},
pubstate = {published},
tppubtype = {inproceedings}
}
2022
Brady Boettcher; Joseph Malloch; Johnty Wang; Marcelo M. Wanderley
Mapper4Live: Using Control Structures to Embed Complex Mapping Tools into Music Production Environments Proceedings Article
In: Proceedings of the International Conference on New Interfaces for Musical Expression, 2022.
BibTeX | Tags: Ableton Live, control mapping, creativity support tools, libmapper, max, music
@inproceedings{Boettcher2022,
title = {Mapper4Live: Using Control Structures to Embed Complex Mapping Tools into Music Production Environments},
author = {Brady Boettcher and Joseph Malloch and Johnty Wang and Marcelo M. Wanderley},
year = {2022},
date = {2022-06-28},
urldate = {2022-06-28},
booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
keywords = {Ableton Live, control mapping, creativity support tools, libmapper, max, music},
pubstate = {published},
tppubtype = {inproceedings}
}
2021
Christian Frisson; Mathias Bredholt; Joseph Malloch; Marcelo M. Wanderley
MapLooper: Live-looping of Distributed Gesture-to-Sound Mappings Proceedings Article
In: Proceedings of the International Conference on New Interfaces for Musical Expression, Shanghai, China, 2021.
BibTeX | Tags: control mapping, digital musical instruments, gesture, live looping
@inproceedings{Frisson2021,
title = {MapLooper: Live-looping of Distributed Gesture-to-Sound Mappings},
author = {Christian Frisson and Mathias Bredholt and Joseph Malloch and Marcelo M. Wanderley},
year = {2021},
date = {2021-06-15},
booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
address = {Shanghai, China},
keywords = {control mapping, digital musical instruments, gesture, live looping},
pubstate = {published},
tppubtype = {inproceedings}
}
2018
Joseph Malloch; Stephen Sinclair; Marcelo M. Wanderley
Generalized Multi-Instance Control Mapping for Interactive Media Systems Journal Article
In: IEEE Multimedia, vol. 25, no. 1, pp. 39–50, 2018.
Abstract | Links | BibTeX | Tags: connected devices, control mapping, graph theory, information propagation, interactive media systems, libmapper, multimedia communication, node connectivity, open source software, sensor arrays, temporal objects
@article{Malloch2018_Multimedia,
title = {Generalized Multi-Instance Control Mapping for Interactive Media Systems},
author = {Joseph Malloch and Stephen Sinclair and Marcelo M. Wanderley},
url = {https://ieeexplore.ieee.org/document/8259406/},
doi = {10.1109/MMUL.2018.112140028},
year = {2018},
date = {2018-01-01},
journal = {IEEE Multimedia},
volume = {25},
number = {1},
pages = {39–50},
abstract = {We articulate a need for the representation of temporal objects reflecting dynamic, short-lived mapping connections instantiated from a template, in tools for designing and using interactive media systems. A list of requirements is compiled from an examination of existing tools, practical use cases, and abstract considerations of node connectivity and information propagation within a graph of connected devices. We validate the concept through implementation in the open source software libmapper, and explore its application by integration with existing controller/synthesizer software and hardware.},
keywords = {connected devices, control mapping, graph theory, information propagation, interactive media systems, libmapper, multimedia communication, node connectivity, open source software, sensor arrays, temporal objects},
pubstate = {published},
tppubtype = {article}
}
We articulate a need for the representation of temporal objects reflecting dynamic, short-lived mapping connections instantiated from a template, in tools for designing and using interactive media systems. A list of requirements is compiled from an examination of existing tools, practical use cases, and abstract considerations of node connectivity and information propagation within a graph of connected devices. We validate the concept through implementation in the open source software libmapper, and explore its application by integration with existing controller/synthesizer software and hardware.