Playful exercise
A virtual companion turns repeated arm movement into a care task.
04 / Project
Mixed-reality rehabilitation concept
HatchWell
Explores how playful MR interaction, sensor feedback, and a companion app could support engagement during at-home rotator cuff recovery.
- Team / duration
- 7-person team · 48-hour SCAD hackathon
- My Role
- Data Research & Interaction Prototyping Contributor
- Team Outcome
- Functional MR proof of concept
My contribution
Reviewed research, corrected pitch-deck data, and prototyped praise and encouragement audio for movement outcomes.
Tools & workflow
My prototype: Unity collision events → feedback selection → audio playback. Team workflow: Unity / Quest MR, Arduino sensor input, and a Figma / Lovable companion app.
My audio prototype is documented in Unity scripts and scene configuration; integration into the recorded final team demo is unconfirmed.
Team prototype evidence
The mixed-reality exercise running on Quest.
This footage documents the team outcome. My individual contribution is separated below so the shared system and personal prototype work remain clear.
48-hour challenge
Reduce a broad healthcare idea to one testable interaction.
Home rehabilitation is repetitive, difficult to track, and easy to abandon. The team narrowed multiple surgery types, AI coaching, social accountability, and remote therapy into one repeatable upper-limb movement focused on motivation and adherence.
One movement. One companion. One working proof of concept.
Team system
Three prototype layers, built together.
Physical input
An Arduino-based concept explores sending movement-related input into Unity.
Recovery support
Goals, schedules, progress, and therapist communication extend beyond the headset.
Team outcome
What the group delivered
- Working Quest APK and mixed-reality exercise demo
- Sensor-to-Unity prototype
- High-fidelity companion app
- Pitch deck and GitHub system documentation
My contribution
Where I directly contributed
- Reviewed research and information used in the final presentation
- Corrected inaccurate pitch-deck data
- Prototyped supportive audio feedback and its interaction logic
- Joined brainstorming, technical discussion, and limited development
My prototype focus
Audio feedback for successful and unsuccessful exercise actions.
I prototyped two feedback paths for movement outcomes: praise after a successful target action and reassurance after a non-target collision. The source code selects a clip from the assigned bank, applies a configurable playback volume, and blocks playback inside a minimum time interval.
Movement eventTarget state?
PlaybackRandomize · volume · debounce
View code evidence
The following are two separate, verbatim excerpts from the Unity prototype.
HandTargetMaterial.cs · trigger branch
if (other.CompareTag("injuredHand"))
{
if (highlightMaterial != null)
{
rend.material = highlightMaterial;
Log("Material switched to highlightMaterial.");
}
// Success hit: play praise
if (voice) voice.PlayPraise();
}
else
{
// Non-target collision: play encouragement/correction
if (voice) voice.PlayEncourageNonTarget();
}VoiceFeedbackPlayer.cs · complete PlayFromBank method
private void PlayFromBank(AudioClip[] bank, string label)
{
if (bank == null || bank.Length == 0)
{
Log($"{label} skipped: no clips assigned.");
return;
}
if (!PassDebounce(label)) return;
int idx = Random.Range(0, bank.Length);
var clip = bank[idx];
speaker.PlayOneShot(clip, voiceVolume);
lastVoiceTime = Time.time;
Log($"{label} played: {clip.name}");
}Team companion app
Support continues outside the headset.
View team companion-app screens
These screens show the team outcome, not my individual design ownership.
Outcome & reflection
The useful lesson was how much the team removed.
My strongest learning was how reducing the idea to one user need and one interaction made the 48-hour prototype achievable. The audio prototype explored encouragement and reassurance for repetitive movement.