Connected health / August 2026
Smart
Medication Monitor.
Making the moments
around medication visible.
I connected physical sensing, embedded intelligence and an iPhone app to understand what happens between a reminder and a medication event.
01 / From reminder to evidence
A small device.
A complete system.
A reminder acknowledgement leaves a gap. The prototype connects medication removal, visible actions and a timestamped record, preserving what each sensor can actually tell us.
What left the platform?
Weight sampled at approximately 10 Hz.
~3 s baseline · 5 s event · 1.5 s settling.
On-board classification, sent to the app over BLE.
What happened next?
ESP32 streams MJPEG over local Wi-Fi.
24 sampled frames become 2,576 temporal features.
Six visible actions, processed on the phone.
One event, with both signals preserved.
SwiftUI brings together the expected dose, weight result and visual action. Raw camera frames are processed in memory and are not saved to history. The normal flow runs without a desktop computer.
The system records supporting behavioral evidence. Weight removal and an ingestion-like action cannot confirm swallowing.
02 / Physical rules meet machine learning
Let physics handle the count.
Let the model read the motion.
One or two removed pills produce a useful net change. Near zero, the final weight alone cannot distinguish no removal, a pill returned or a disturbance. I routed these events to a model of the full trajectory.
Explore the physical gate
A little uncertainty
is part of the design.
Move the weight ratio to see which branch receives the event.
The net change lies in the one-pill band. The physical rule returns ONE directly.
Interactive explanation of the documented thresholds, using a 0.848 g mock pill. This does not run the trained forest. Near-zero events need 24 trajectory features to classify NONE, RETURN or DISTURBANCE.
The same ending can hide different events.
The dynamic branch excludes final net weight and looks at duration, dips, recovery, variation and area. A return and a disturbance can look similar; that ambiguity explains the remaining holdout errors.
- 24 dynamic features
- 500 trees on ESP32
- 3,174 exported tree nodes
03 / An interaction became a measurement problem
The best button
was off the device.
Pressing the physical button changed the load path through the scale. I compared five button-triggered trials with five BLE-triggered trials, then moved the trigger into the app.
5 + 5 trial engineering experiment / Report, pp. 22–23
lower mean absolute error with BLE
Interaction design became part of sensing quality. Adding the OV2640 later also occupied GPIO 4, reinforcing the decision to remove the button.
In a physical interface, the way a user starts a measurement can change the measurement itself.
04 / Bringing the system into everyday use
Evidence, translated
into an experience.
The bilingual SwiftUI app connects planning, device interaction and event review. Each result keeps the weight and camera evidence readable, including when the signals disagree.
A reminder starts a conversation.
Daily prescriptions, BLE device status and 15-minute follow-ups bring the next action into one place.
Two signals. One understandable event.
Weight and camera results stay visible separately. Matching the expected count plus a TAKE action supports the event.
A record with its context intact.
Local history and a patient ID support an on-device Doctor View. Remote clinician access is future work.
English screenshots captured from the working MedBox app in the iOS Simulator. Home and result screens use development Mock events. These images illustrate the interface, not measured hardware performance.
Weight classification on the microcontroller; vision on the iPhone.
Expected count + TAKE supports an event. Conflicting signals remain visible.
On-device history and Doctor View; no cloud or remote patient lookup.
05 / What the experiments establish
A working prototype.
Evidence at every layer.
The clearest model comparison uses the same cross-session holdout: hierarchical classification reached 82%, up eight percentage points from the flat Random Forest. The other tests answer different questions.
Cross-session weight accuracy
Train on 50 Round 1 events; test on 50 Round 2 events. The flat Random Forest baseline reached 74%.
Camera internal 5-fold CV
120 clips across six actions, from one participant in one session. Cross-user performance remains untested.
Python / C++ prediction agreement
Recorded and synthetic events match the frozen Python pipeline. This checks implementation parity, not accuracy.
Weight internal 5-fold CV
100 real events pooled across two rounds. This describes internal separability, not an independent field result.
Where weight recognition breaks
All nine holdout errors came from DISTURBANCE. Seven were classified as RETURN and two as NONE, pointing towards richer disturbance data and more varied mechanical conditions.
Where visual recognition overlaps
Four internal-validation mistakes occurred between ADJUST and TOUCH_FACE. The next useful test separates participants, sessions, lighting and camera placement.
The final report records iPhone hardware flow verification as passed by 21 August 2026, alongside firmware compilation and software checks. Structured latency, power and long-duration stability measurements remain future work.
Reflection / The next iteration
Design the evidence.
Then earn the confidence.
This project brought hardware, signal processing, model deployment and interface design into one loop. The most useful decisions came from understanding how those layers affect each other.