30-second summary
We asked a narrow design question: if the same music drives two different degrees of visual motion, do people respond differently? The system translated amplitude and roughness into a real-time three-dimensional form. The published study then compared a soft condition with a more intense condition using self-report measures and interviews.
I built the TouchDesigner visualization system and audio-to-visual mappings, contributed to the study work, and co-authored the paper. I do not present the study as clinical validation: it measured short-term responses from 30 young adults in one controlled setting.
Context
- Type: research project and peer-reviewed publication
- Team: six co-authors across design, computing, and research institutions
- My role: visualization implementation, feature mapping, study contribution, and co-authorship
- Publication: Pervasive Health 2025 proceedings, Springer LNICST 693, 2026
The system
The prototype used a mono audio signal sampled at 48 kHz. A short-time Fourier transform extracted the spectrum; threshold clipping, peak extraction, and a first-order smoothing filter reduced frame-to-frame noise before the signal drove the visual parameters.
The visual scene combined an inner sphere with surrounding cube instances. The smoothed audio energy controlled vertex displacement, twist, scale, color, and movement. TouchDesigner rendered the response at 60 frames per second.
Two configuration ranges produced the experimental conditions:
| Condition | Mapping range | Intended visual behavior |
|---|---|---|
| Soft | 0.4 - 2.5 | smoother motion and less deformation |
| Intense | 0.4 - 4.7 | larger movement and more pronounced shape change |
The important decision was not simply to make the visual “reactive.” Both conditions used the same music source, so the comparison isolated the degree of roughness and amplitude mapping more clearly.
Study design
The controlled experiment recruited 30 participants aged 18 to 22. Fifteen viewed the soft condition and fifteen viewed the intense condition. The evaluation combined:
- PSS-10 for perceived stress;
- PANAS for positive and negative affect;
- an aesthetic-pleasure measure;
- semi-structured interviews about comfort, immersion, color, rhythm, and memory.
This mixed-method design mattered because one score could show a change while an interview explained why the same dynamic felt relaxing to one participant and tense to another.
What the paper reports
The paper reports that both conditions reduced negative affect in the short term, with the softer condition producing a stronger reduction. Aesthetic pleasure was associated with higher positive affect and greater stress relief. Interview themes also pointed to dynamic amplitude, personalized color response, and music-visual synchronization as important design dimensions.
The qualitative evidence complicated the simple “soft is better” story. Some participants wanted more visual variation; others reacted differently to colors or remembered personal experiences. The intense condition could create tension, but its stronger rhythm also captured attention.
Scope and limits
- The sample was small and limited to young adults in one setting.
- The study measured immediate response, not long-term therapeutic effect.
- Participants watched a fixed recording rather than a personalized, real-time intervention.
- Color, privacy, duration, and viewing environment were not fully controlled as adaptive variables.
Those limitations led naturally to the next research question: how could a system adapt cautiously to individual response without treating noisy physiological or affective signals as ground truth?
Publication
Hua Ma, Qin Zhou, Yixin Chen, Qixuan Chu, Dujiao Yang, and Sun Xu. “An Empirical Study on the Design of Dynamic Music Visualization for Therapy Based on Synesthesia Feature Mapping.” Pervasive Computing Technologies for Healthcare, LNICST 693, pp. 512-527. Springer, 2026. DOI: 10.1007/978-3-032-27582-0_30.