GardenViewer
About the Project
Year 3 IoT Mini Project 2, done solo. A XIAO ESP32S3 Sense runs an image-classification model on-device, publishes its inference results over MQTT, and a browser dashboard renders the live camera frame, the per-class confidences, and the rest of the garden's sensors.
GitHub Repo: Here
Demo Video: Here
What It Detects
The model classifies five things, trained on Kaggle image datasets in Edge Impulse: bee, bird, butterfly, man, and woman. The three animal classes tell me what's visiting the garden, and the two person classes drive the intruder alert.
Training the Model
My first attempt trained on the full set of 2275 images at 96×96, with a learning rate of 0.001, 20 training cycles, a batch size of 128, and a 20% validation split. That run scored 83.9% accuracy.
I then cut the dataset down to 500 images (100 per class, 80/20 train/test), raised the learning rate to 0.005, and raised the input size to 160×160, keeping the same 20 cycles and batch size of 128. Validation accuracy came out to 88.6%, a gain of 4.7 points, and training was faster too. My source photos are high resolution, so the larger input size paid for itself.
After that, I tested the retrained model properly and it scored 96.97% accuracy. It only struggled telling the two genders apart — it had no trouble with the three animal classes.
Public Edge Impulse Project: Here
The Dashboard
The dashboard, dashboardv7.html, is a single self-contained file with no build step, connecting over WebSockets to my MQTT broker at mqtt.chiambucket.com. It subscribes to sadiq_iotproject1 for the DHT22, MQ135, PIR, LDR, rain and soil-moisture readings, and to sscma/v0/gardenviewer/tx for the AI inference results.
Nine sensor cards with sparklines and status pills sit alongside a tenth AI card holding the camera feed and its confidence readout.
The trickiest part was reading the AI payload. SSCMA-Micro sends classification results as [score, target_index] pairs, like "classes":[[100,1]], not the named-key form the SenseCraft website shows. The target indices map alphabetically: 0=bee, 1=bird, 2=butterfly, 3=man, 4=woman. The camera frame itself arrives base64-encoded and gets painted onto a canvas with drawImage().
Intruder Alert and Night Vision
While Intruder Detection is switched on, any man or woman reading above 80% confidence turns the AI card red and displays "INTRUDER ALERT".
Night Vision is a prototype rather than real infrared: it filters the canvas to 120% brightness, 120% contrast, and 200% saturation, and switches on the garden LED strip at the same time.
What Didn't Work
Moving the model from my laptop's webcam onto the XIAO ESP32S3 Sense made it noticeably slower. On-device it also defaults to detecting "bird" when nothing is in frame, and — oddly — detecting bee and butterfly became slower than detecting man or woman.
The model also still confuses the two genders on certain poses, on both laptop and device. I tried adjusting the confidence threshold on the XIAO board, but it did not fix either issue.