This commit is contained in:
Joshua King
2026-10-02 12:23:58 -04:00
parent f0af1dbbcb
commit ad1e7e6d01
219 changed files with 488314 additions and 0 deletions
@@ -0,0 +1,3 @@
cmake_minimum_required(VERSION 3.16)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(freddy_camera_person)
+21
View File
@@ -0,0 +1,21 @@
# On-device person-position experiment
This ESP-IDF project combines the DFR1154 camera pin map with Espressif's [pedestrian detection model](https://github.com/espressif/esp-dl/blob/master/models/pedestrian_detect/README.md). It prints timestamped person centers and bounding boxes over USB and broadcasts [gaze frames](../../protocol/gaze.md) from GPIO43 to the two eye boards. Physical delivery and synchronized eye motion are bench-verified.
The project contains local Wi-Fi OTA support. Its `partitions.csv` has two 7 MB app slots and its bootloader enables rollback. The one-time USB migration and first OTA update succeeded on 2026-09-30. A later USB flash on 2026-10-01 installed the tracking filter in slot 0; see the [tracking check](../../hardware/tracking-filter-2026-10-01.md) and [OTA procedure](../ota.md). OTA cannot change the partition table itself.
## Tracking filter — 2026-10-01
After a clearly empty 20-second view yielded 15 apparent person boxes, the camera now rejects top-edge boxes at least 95% of image height. All 15 archived empty-scene boxes fit this geometry; none of 203 matched held-person detections in earlier center/left/right logs did. Once three consistent detections confirm a target, `tracking::Gate` sends the last confirmed position for up to one second through brief model misses or a new unconfirmed candidate. A persistent miss returns `valid=0`. The camera still sends the same version-1 UART frame to both eyes; each eye uses `valid=1` for red/track and `valid=0` for blue/idle. The change compiled, passed its native state-transition test, and was MAC-checked and USB-flashed with verified hashes. The current application SHA-256 is `76d16c8817db9e9568eaf2485a486a0debaa2687fc5a8447954d0cd0d80d468a`.
In the same empty view after flashing, the camera sent **0 valid targets in 80 frames** and recorded 15 rejected full-height candidates. Both eyes accepted the same invalid states for all 78 overlapping logged sequences. A close real person might also fill the image height, so test this filter with a visitor at the intended near/middle/far approach positions before accepting it for the porch. [Raw captures and interpretation](../../hardware/tracking-filter-2026-10-01.md).
On 2026-09-27, the gaze build was flashed to the camera after its MAC `E8:F6:0A:87:0B:4C` was checked. Following a temporary malformed-frame diagnostic build, the normal camera application (SHA-256 `0acc763624b20a09afbf512e20afad51ffc9de4276102ffa639a0345d39d70a5`) was restored and hash-verified after flashing. Both eyes accepted matching camera coordinates; Joshua confirmed synchronized visible tracking. The [bench log](../../hardware/gaze-link-bench.md) records rejection of malformed/stale commands and recovery. A live USB sample showed 231–232 ms capture-to-result time and `gaze_valid` transitions. The current three-consecutive-frame gate rejects isolated detections, and the camera rejects detections whose boxes span at least 90% of both image dimensions. Other persistent false detections can still pass; this is not finished target selection.
The project builds with ESP-IDF v5.5.2 and was hash-verified after flashing to the camera identified by MAC `E8:F6:0A:87:0B:4C`. Its first 20-second live serial sample at the model's default 0.70 score threshold reported `NO_PERSON` at 320 × 240. A subsequent run at threshold 0.35, with better lighting and a person in view, produced stable timestamped bounding boxes at left, center, and right held positions. The corresponding logs and JPEGs are in `../../hardware/camera-captures/`, summarized in `../../hardware/bringup.md`. Typical capture-to-result latency was 233 ms (about four results per second). Espressif reports about 118 ms model time for a 224 × 224 input on an ESP32-S3; the measured figure here includes capture and JPEG decode. Camera pins match DFRobot's [CameraWebServer example](https://github.com/DFRobot/DFR1154_Examples/blob/master/5.3%20CameraWebServer/source%20code/5_3/5_3.ino). Dependencies are pinned to compatible release ranges in `main/idf_component.yml`; `dependencies.lock` records the resolved versions. Diagnostic JPEG output is disabled by default; enable it with `FREDDY_CAMERA_DIAGNOSTIC_JPEG=1` for captures.
The current build also drives GPIO47 high at boot to enable the board's infrared illumination, as identified in [DFRobot's DFR1154 guide](https://wiki.dfrobot.com/dfr1154/). The camera must be aimed clear of nearby surfaces: an IR-lit frame captured close to a surface washed out into a purple blur, while a re-aimed frame showed the room. This has not yet established better detection in darkness, and the raw model reports some false positives in an empty room. Diagnostic JPEG output is disabled by default in the gaze build and can be re-enabled with `FREDDY_CAMERA_DIAGNOSTIC_JPEG=1` for captures.
When diagnostic JPEG output is enabled, use `capture_diagnostic.py` with PySerial to record synchronized logs and JPEGs. Test continuous walking, dim and brighter lighting, a close person, a distant person, two people, and a person crossing the camera edge before relying on the coordinates for gaze. Improve target persistence and false-positive rejection beyond the preliminary three-frame gate. The DFRobot bundled `Person_Detection_inferencing.zip` is only a 96 × 96 binary classifier (`non_person` versus `person`), so it cannot provide the required position directly.
On 2026-09-27 Joshua reported intermittent person lock. A temporary diagnostic image showed the camera aimed mostly at the ceiling/window, with only part of him at the far left edge; the model alternated `PERSON` and `NO_PERSON` and reported left-edge boxes. See [the diagnosis and paired frames](../../hardware/tracking-drop-diagnosis.md). The normal build above was restored and verified; first reframe the camera, then retest before tuning tracking hysteresis.
@@ -0,0 +1,29 @@
Executing action: all (aliases: build)
Running ninja in directory /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build
Executing "ninja all"...
[1/9] Performing build step for 'bootloader'
[1/1] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x0 /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/bootloader.bin
Bootloader binary size 0x5240 bytes. 0x2dc0 bytes (36%) free.
[2/9] No install step for 'bootloader'
[3/9] Completed 'bootloader'
[4/9] Building CXX object esp-idf/main/CMakeFiles/__idf_main.dir/main.cpp.obj
[5/9] Linking C static library esp-idf/main/libmain.a
[6/9] Generating esp-idf/esp_system/ld/sections.ld
[7/9] Linking CXX executable freddy_camera_person.elf
[8/9] Generating binary image from built executable
esptool.py v4.12.dev1
Creating esp32s3 image...
Merged 2 ELF sections
Successfully created esp32s3 image.
Generated /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/freddy_camera_person.bin
[9/9] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/partition_table/partition-table.bin /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/freddy_camera_person.bin
freddy_camera_person.bin binary size 0x288bd0 bytes. Smallest app partition is 0xa00000 bytes. 0x777430 bytes (75%) free.
Project build complete. To flash, run:
idf.py flash
or
idf.py -p PORT flash
or
python -m esptool --chip esp32s3 -b 460800 --before default_reset --after hard_reset write_flash --flash_mode dio --flash_size 16MB --flash_freq 80m 0x0 firmware/camera_person_idf/build/bootloader/bootloader.bin 0x8000 firmware/camera_person_idf/build/partition_table/partition-table.bin 0x10000 firmware/camera_person_idf/build/freddy_camera_person.bin
or from the "/Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build" directory
python -m esptool --chip esp32s3 -b 460800 --before default_reset --after hard_reset write_flash "@flash_args"
@@ -0,0 +1,29 @@
Executing action: all (aliases: build)
Running ninja in directory /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build
Executing "ninja all"...
[1/9] Performing build step for 'bootloader'
[1/1] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x0 /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/bootloader.bin
Bootloader binary size 0x5240 bytes. 0x2dc0 bytes (36%) free.
[2/9] No install step for 'bootloader'
[3/9] Completed 'bootloader'
[4/9] Building CXX object esp-idf/main/CMakeFiles/__idf_main.dir/main.cpp.obj
[5/9] Linking C static library esp-idf/main/libmain.a
[6/9] Generating esp-idf/esp_system/ld/sections.ld
[7/9] Linking CXX executable freddy_camera_person.elf
[8/9] Generating binary image from built executable
esptool.py v4.12.dev1
Creating esp32s3 image...
Merged 2 ELF sections
Successfully created esp32s3 image.
Generated /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/freddy_camera_person.bin
[9/9] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/partition_table/partition-table.bin /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/freddy_camera_person.bin
freddy_camera_person.bin binary size 0x288e00 bytes. Smallest app partition is 0xa00000 bytes. 0x777200 bytes (75%) free.
Project build complete. To flash, run:
idf.py flash
or
idf.py -p PORT flash
or
python -m esptool --chip esp32s3 -b 460800 --before default_reset --after hard_reset write_flash --flash_mode dio --flash_size 16MB --flash_freq 80m 0x0 firmware/camera_person_idf/build/bootloader/bootloader.bin 0x8000 firmware/camera_person_idf/build/partition_table/partition-table.bin 0x10000 firmware/camera_person_idf/build/freddy_camera_person.bin
or from the "/Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build" directory
python -m esptool --chip esp32s3 -b 460800 --before default_reset --after hard_reset write_flash "@flash_args"
@@ -0,0 +1,8 @@
I (120984) freddy_person: PERSON t_ms=120020 score=0.800 cx=207 I (364804) freddy_person: NO_PERSON t_ms=363838 frame=320,240 latency_ms=230
I (366054) freddy_person: PERSON t_ms=365088 score=0.370 cx=165 cy=173 box=136,134,194,212 frame=320,240 latency_ms=231
I (366304) freddy_person: PERSON t_ms=365338 score=0.455 cx=174 cy=174 box=154,138,195,211 frame=320,240 latency_ms=231
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-empty-frame-1441.jpg (4499 bytes)
I (370644) freddy_person: NO_PERSON t_ms=369678 frame=320,240 latency_ms=230
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-empty-frame-1461.jpg (4550 bytes)
I (373734) freddy_person: PERSON t_ms=372773 score=0.375 cx=153 cy=175 box=135,137,171,213 frame=320,240 latency_ms=326
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-empty.log; 4 person detections
@@ -0,0 +1,23 @@
I (123) esp_image: segment 1: paddr=00093494 vaddr=3fc98900 sizeI (42934) freddy_person: PERSON t_ms=41969 score=0.500 cx=293 cy=209 box=267,180,319,239 frame=320,240 latency_ms=232
I (43184) freddy_person: PERSON t_ms=42219 score=0.515 cx=293 cy=211 box=267,183,319,239 frame=320,240 latency_ms=231
I (43434) freddy_person: PERSON t_ms=42468 score=0.364 cx=294 cy=209 box=269,180,319,239 frame=320,240 latency_ms=230
I (43684) freddy_person: PERSON t_ms=42718 score=0.375 cx=294 cy=210 box=269,183,319,238 frame=320,240 latency_ms=231
I (44184) freddy_person: PERSON t_ms=43218 score=0.364 cx=293 cy=210 box=267,181,319,239 frame=320,240 latency_ms=231
I (44434) freddy_person: PERSON t_ms=43468 score=0.354 cx=305 cy=212 box=292,186,319,239 frame=320,240 latency_ms=231
I (44684) freddy_person: PERSON t_ms=43719 score=0.395 cx=159 cy=119 box=0,0,319,239 frame=320,240 latency_ms=231
I (44934) freddy_person: PERSON t_ms=43968 score=0.415 cx=293 cy=209 box=268,180,319,239 frame=320,240 latency_ms=231
I (45184) freddy_person: PERSON t_ms=44219 score=0.500 cx=293 cy=209 box=268,181,319,238 frame=320,240 latency_ms=231
I (45684) freddy_person: PERSON t_ms=44718 score=0.364 cx=160 cy=119 box=1,0,319,239 frame=320,240 latency_ms=231
I (46434) freddy_person: PERSON t_ms=45469 score=0.459 cx=159 cy=119 box=0,0,319,239 frame=320,240 latency_ms=231
I (46684) freddy_person: PERSON t_ms=45720 score=0.580 cx=159 cy=119 box=0,0,319,239 frame=320,240 latency_ms=233
I (47424) freddy_person: PERSON t_ms=46457 score=0.395 cx=215 cy=119 box=112,0,319,239 frame=320,240 latency_ms=230
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-ir-empty-frame-181.jpg (2941 bytes)
I (47734) freddy_person: PERSON t_ms=46769 score=0.385 cx=215 cy=119 box=111,0,319,239 frame=320,240 latency_ms=291
I (47984) freddy_person: PERSON t_ms=47018 score=0.405 cx=213 cy=119 box=108,0,319,239 frame=320,240 latency_ms=230
I (48234) freddy_person: PERSON t_ms=47268 score=0.484 cx=217 cy=119 box=115,0,319,239 frame=320,240 latency_ms=230
I (48984) freddy_person: PERSON t_ms=48018 score=0.468 cx=214 cy=119 box=110,0,319,239 frame=320,240 latency_ms=230
I (49224) freddy_person: PERSON t_ms=48267 score=0.364 cx=211 cy=119 box=104,0,319,239 frame=320,240 latency_ms=229
I (49984) freddy_person: PERSON t_ms=49018 score=0.459 cx=213 cy=119 box=107,0,319,239 frame=320,240 latency_ms=230
I (50234) freddy_person: PERSON t_ms=49267 score=0.415 cx=213 cy=119 box=108,0,319,239 frame=320,240 latency_ms=230
I (50484) freddy_person: PERSON t_ms=49517 score=0.405 cx=213 cy=119 box=107,0,319,239 frame=320,240 latency_ms=230
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-ir-empty.log; 21 person detections
@@ -0,0 +1,28 @@
I (179914) freddy_person: PERSON t_ms=178950 score=0.661 cx=34 cy=99 box=0,0,68,199 frame=320,240 latency_ms=232
I (180164) freddy_person: PERSON t_ms=179200 score=0.530 cx=32 cy=98 box=0,0,64,196 frame=320,240 latency_ms=232
I (180414) freddy_person: PERSON t_ms=179449 score=0.459 cx=30 cy=99 box=0,0,60,198 frame=320,240 latency_ms=232
I (180664) freddy_person: PERSON t_ms=179700 score=0.573 cx=28 cy=99 box=0,1,56,197 frame=320,240 latency_ms=232
I (180914) freddy_person: PERSON t_ms=179949 score=0.530 cx=26 cy=98 box=0,1,52,196 frame=320,240 latency_ms=232
I (181414) freddy_person: PERSON t_ms=180450 score=0.559 cx=29 cy=97 box=0,0,59,195 frame=320,240 latency_ms=232
I (181664) freddy_person: PERSON t_ms=180700 score=0.625 cx=29 cy=97 box=0,0,58,194 frame=320,240 latency_ms=232
I (182164) freddy_person: PERSON t_ms=181199 score=0.459 cx=28 cy=97 box=0,0,56,194 frame=320,240 latency_ms=232
I (182414) freddy_person: PERSON t_ms=181449 score=0.385 cx=99 cy=133 box=58,85,140,182 frame=320,240 latency_ms=231
I (182664) freddy_person: PERSON t_ms=181700 score=0.492 cx=27 cy=93 box=0,0,55,187 frame=320,240 latency_ms=232
I (182914) freddy_person: PERSON t_ms=181950 score=0.612 cx=29 cy=97 box=0,1,59,194 frame=320,240 latency_ms=232
I (183164) freddy_person: PERSON t_ms=182200 score=0.580 cx=34 cy=96 box=0,0,68,193 frame=320,240 latency_ms=232
I (183414) freddy_person: PERSON t_ms=182449 score=0.395 cx=40 cy=119 box=0,1,81,238 frame=320,240 latency_ms=232
I (183914) freddy_person: PERSON t_ms=182949 score=0.385 cx=44 cy=91 box=0,0,88,182 frame=320,240 latency_ms=232
I (184164) freddy_person: PERSON t_ms=183199 score=0.375 cx=49 cy=119 box=0,0,99,238 frame=320,240 latency_ms=232
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-ir-reaimed-frame-721.jpg (5539 bytes)
I (185024) freddy_person: NO_PERSON t_ms=184059 frame=320,240 latency_ms=231
I (185274) freddy_person: PERSON t_ms=184309 score=0.395 cx=48 cy=119 box=0,0,96,239 frame=320,240 latency_ms=232
I (185774) freddy_person: PERSON t_ms=184810 score=0.612 cx=38 cy=101 box=0,0,76,202 frame=320,240 latency_ms=232
I (186024) freddy_person: PERSON t_ms=185060 score=0.650 cx=34 cy=101 box=0,0,68,203 frame=320,240 latency_ms=232
I (186274) freddy_person: PERSON t_ms=185310 score=0.538 cx=26 cy=106 box=0,1,53,211 frame=320,240 latency_ms=232
I (186524) freddy_person: PERSON t_ms=185560 score=0.545 cx=25 cy=134 box=0,51,50,217 frame=320,240 latency_ms=233
I (186774) freddy_person: PERSON t_ms=185810 score=0.395 cx=26 cy=104 box=0,3,52,206 frame=320,240 latency_ms=232
I (187024) freddy_person: PERSON t_ms=186060 score=0.459 cx=24 cy=132 box=0,47,49,217 frame=320,240 latency_ms=232
I (187274) freddy_person: PERSON t_ms=186310 score=0.459 cx=23 cy=138 box=0,55,46,221 frame=320,240 latency_ms=232
I (187524) freddy_person: PERSON t_ms=186560 score=0.707 cx=25 cy=134 box=0,46,50,222 frame=320,240 latency_ms=232
I (187774) freddy_person: PERSON t_ms=186810 score=0.538 cx=23 cy=132 box=0,54,46,210 frame=320,240 latency_ms=232
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-ir-reaimed.log; 25 person detections
@@ -0,0 +1,13 @@
I (50974) freddy_person: PERSON t_ms=50017 score=0.375 cx=215 cyI (73984) freddy_person: NO_PERSON t_ms=73027 frame=320,240 latency_ms=229
I (74234) freddy_person: PERSON t_ms=73277 score=0.364 cx=212 cy=119 box=105,0,319,239 frame=320,240 latency_ms=229
I (74494) freddy_person: PERSON t_ms=73527 score=0.385 cx=216 cy=119 box=113,0,319,239 frame=320,240 latency_ms=230
I (75984) freddy_person: PERSON t_ms=75027 score=0.375 cx=216 cy=117 box=113,0,319,235 frame=320,240 latency_ms=229
I (76494) freddy_person: PERSON t_ms=75527 score=0.424 cx=214 cy=119 box=110,0,319,239 frame=320,240 latency_ms=230
I (76734) freddy_person: PERSON t_ms=75777 score=0.385 cx=214 cy=119 box=110,0,319,239 frame=320,240 latency_ms=229
I (77244) freddy_person: PERSON t_ms=76277 score=0.424 cx=214 cy=119 box=110,0,319,239 frame=320,240 latency_ms=230
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-ir-settled-frame-301.jpg (2942 bytes)
I (79794) freddy_person: PERSON t_ms=78837 score=0.385 cx=213 cy=119 box=107,0,319,239 frame=320,240 latency_ms=229
I (80554) freddy_person: PERSON t_ms=79588 score=0.433 cx=214 cy=119 box=109,0,319,239 frame=320,240 latency_ms=230
I (80804) freddy_person: PERSON t_ms=79838 score=0.405 cx=214 cy=119 box=110,0,319,239 frame=320,240 latency_ms=230
I (81044) freddy_person: PERSON t_ms=80087 score=0.375 cx=215 cy=119 box=111,0,319,239 frame=320,240 latency_ms=229
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-ir-settled.log; 11 person detections
@@ -0,0 +1,33 @@
I (41804) freddy_person: PERSON t_ms=40840 score=0.586 cx=166 cyI (98354) freddy_person: PERSON t_ms=97390 score=0.515 cx=215 cy=119 box=111,0,319,239 frame=320,240 latency_ms=233
I (98854) freddy_person: PERSON t_ms=97890 score=0.484 cx=217 cy=120 box=122,4,313,237 frame=320,240 latency_ms=233
I (99104) freddy_person: PERSON t_ms=98140 score=0.593 cx=220 cy=123 box=121,8,319,239 frame=320,240 latency_ms=232
I (99354) freddy_person: PERSON t_ms=98390 score=0.599 cx=237 cy=119 box=156,0,319,239 frame=320,240 latency_ms=232
I (99604) freddy_person: PERSON t_ms=98640 score=0.515 cx=234 cy=120 box=149,2,319,239 frame=320,240 latency_ms=232
I (99854) freddy_person: PERSON t_ms=98889 score=0.468 cx=250 cy=131 box=181,23,319,239 frame=320,240 latency_ms=232
I (100104) freddy_person: PERSON t_ms=99139 score=0.433 cx=230 cy=139 box=142,39,319,239 frame=320,240 latency_ms=232
I (100354) freddy_person: PERSON t_ms=99390 score=0.484 cx=229 cy=139 box=139,40,319,239 frame=320,240 latency_ms=232
I (100604) freddy_person: PERSON t_ms=99640 score=0.530 cx=260 cy=141 box=201,44,319,239 frame=320,240 latency_ms=232
I (100854) freddy_person: PERSON t_ms=99890 score=0.476 cx=257 cy=142 box=196,46,319,238 frame=320,240 latency_ms=233
I (101104) freddy_person: PERSON t_ms=100139 score=0.545 cx=218 cy=142 box=120,46,316,239 frame=320,240 latency_ms=232
I (101354) freddy_person: PERSON t_ms=100390 score=0.484 cx=234 cy=142 box=150,45,319,239 frame=320,240 latency_ms=232
I (101604) freddy_person: PERSON t_ms=100640 score=0.468 cx=258 cy=143 box=197,48,319,239 frame=320,240 latency_ms=232
I (101854) freddy_person: PERSON t_ms=100890 score=0.637 cx=230 cy=143 box=141,48,319,239 frame=320,240 latency_ms=232
I (102104) freddy_person: PERSON t_ms=101140 score=0.566 cx=241 cy=143 box=163,48,319,239 frame=320,240 latency_ms=232
I (102354) freddy_person: PERSON t_ms=101390 score=0.723 cx=236 cy=144 box=154,50,319,239 frame=320,240 latency_ms=232
I (102604) freddy_person: PERSON t_ms=101640 score=0.776 cx=228 cy=146 box=138,53,319,239 frame=320,240 latency_ms=232
I (102854) freddy_person: PERSON t_ms=101890 score=0.690 cx=243 cy=145 box=167,52,319,239 frame=320,240 latency_ms=232
I (103104) freddy_person: PERSON t_ms=102140 score=0.637 cx=238 cy=145 box=158,52,319,239 frame=320,240 latency_ms=232
I (103354) freddy_person: PERSON t_ms=102390 score=0.612 cx=232 cy=143 box=147,48,317,239 frame=320,240 latency_ms=232
I (103604) freddy_person: PERSON t_ms=102640 score=0.702 cx=224 cy=144 box=129,49,319,239 frame=320,240 latency_ms=232
I (103854) freddy_person: PERSON t_ms=102890 score=0.791 cx=230 cy=144 box=141,49,319,239 frame=320,240 latency_ms=232
I (104104) freddy_person: PERSON t_ms=103140 score=0.612 cx=225 cy=145 box=132,51,319,239 frame=320,240 latency_ms=232
I (104354) freddy_person: PERSON t_ms=103390 score=0.786 cx=222 cy=144 box=126,50,319,239 frame=320,240 latency_ms=232
I (104604) freddy_person: PERSON t_ms=103640 score=0.834 cx=227 cy=144 box=136,50,319,239 frame=320,240 latency_ms=232
I (104854) freddy_person: PERSON t_ms=103890 score=0.734 cx=222 cy=143 box=126,48,319,239 frame=320,240 latency_ms=232
I (105104) freddy_person: PERSON t_ms=104140 score=0.723 cx=231 cy=143 box=145,48,318,239 frame=320,240 latency_ms=233
I (105354) freddy_person: PERSON t_ms=104390 score=0.745 cx=219 cy=143 box=120,48,319,239 frame=320,240 latency_ms=232
I (105604) freddy_person: PERSON t_ms=104639 score=0.586 cx=219 cy=145 box=121,51,317,239 frame=320,240 latency_ms=232
I (105854) freddy_person: PERSON t_ms=104889 score=0.740 cx=221 cy=145 box=123,52,319,239 frame=320,240 latency_ms=232
I (106104) freddy_person: PERSON t_ms=105140 score=0.702 cx=251 cy=146 box=183,54,319,239 frame=320,240 latency_ms=232
I (106354) freddy_person: PERSON t_ms=105390 score=0.593 cx=250 cy=147 box=185,55,316,239 frame=320,240 latency_ms=232
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-left.log; 32 person detections
@@ -0,0 +1,52 @@
I (50984) freddy_person: PERSON t_ms=50020 score=0.848 cx=61 cy=I (108564) freddy_person: PERSON t_ms=107600 score=0.723 cx=214 cy=138 box=123,37,306,239 frame=320,240 latency_ms=233
I (108814) freddy_person: PERSON t_ms=107851 score=0.810 cx=216 cy=137 box=123,35,310,239 frame=320,240 latency_ms=233
I (109064) freddy_person: PERSON t_ms=108101 score=0.901 cx=219 cy=138 box=128,40,311,237 frame=320,240 latency_ms=233
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-left-image-frame-421.jpg (6708 bytes)
I (109454) freddy_person: PERSON t_ms=108490 score=0.707 cx=210 cy=139 box=119,39,301,239 frame=320,240 latency_ms=373
I (109704) freddy_person: PERSON t_ms=108740 score=0.888 cx=218 cy=139 box=120,39,316,239 frame=320,240 latency_ms=233
I (109954) freddy_person: PERSON t_ms=108990 score=0.734 cx=214 cy=131 box=125,41,304,221 frame=320,240 latency_ms=233
I (110204) freddy_person: PERSON t_ms=109241 score=0.839 cx=209 cy=139 box=123,41,296,238 frame=320,240 latency_ms=233
I (110454) freddy_person: PERSON t_ms=109490 score=0.834 cx=214 cy=137 box=128,40,301,235 frame=320,240 latency_ms=233
I (110704) freddy_person: PERSON t_ms=109740 score=0.824 cx=217 cy=139 box=123,40,312,239 frame=320,240 latency_ms=233
I (110954) freddy_person: PERSON t_ms=109990 score=0.713 cx=219 cy=139 box=132,40,307,239 frame=320,240 latency_ms=233
I (111204) freddy_person: PERSON t_ms=110240 score=0.834 cx=214 cy=139 box=127,39,302,239 frame=320,240 latency_ms=233
I (111454) freddy_person: PERSON t_ms=110490 score=0.755 cx=220 cy=138 box=130,39,310,238 frame=320,240 latency_ms=233
I (111704) freddy_person: PERSON t_ms=110740 score=0.839 cx=218 cy=140 box=126,41,310,239 frame=320,240 latency_ms=233
I (111954) freddy_person: PERSON t_ms=110991 score=0.599 cx=215 cy=138 box=118,37,312,239 frame=320,240 latency_ms=233
I (112204) freddy_person: PERSON t_ms=111241 score=0.888 cx=218 cy=139 box=128,39,308,239 frame=320,240 latency_ms=233
I (112454) freddy_person: PERSON t_ms=111490 score=0.786 cx=221 cy=138 box=128,40,314,237 frame=320,240 latency_ms=233
I (112704) freddy_person: PERSON t_ms=111740 score=0.702 cx=217 cy=139 box=127,40,308,239 frame=320,240 latency_ms=233
I (112954) freddy_person: PERSON t_ms=111990 score=0.829 cx=222 cy=139 box=128,40,317,238 frame=320,240 latency_ms=233
I (113204) freddy_person: PERSON t_ms=112240 score=0.884 cx=220 cy=138 box=131,38,309,239 frame=320,240 latency_ms=233
I (113454) freddy_person: PERSON t_ms=112491 score=0.631 cx=212 cy=138 box=122,41,302,235 frame=320,240 latency_ms=233
I (113704) freddy_person: PERSON t_ms=112740 score=0.897 cx=219 cy=139 box=126,39,312,239 frame=320,240 latency_ms=233
I (113954) freddy_person: PERSON t_ms=112990 score=0.786 cx=221 cy=137 box=132,39,310,236 frame=320,240 latency_ms=233
I (114204) freddy_person: PERSON t_ms=113240 score=0.815 cx=208 cy=136 box=127,39,290,234 frame=320,240 latency_ms=233
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-left-image-frame-441.jpg (6659 bytes)
I (114594) freddy_person: PERSON t_ms=113629 score=0.566 cx=217 cy=137 box=130,38,304,237 frame=320,240 latency_ms=372
I (114844) freddy_person: PERSON t_ms=113881 score=0.530 cx=204 cy=138 box=102,40,307,237 frame=320,240 latency_ms=233
I (115094) freddy_person: PERSON t_ms=114130 score=0.740 cx=206 cy=135 box=127,41,286,230 frame=320,240 latency_ms=233
I (115344) freddy_person: PERSON t_ms=114381 score=0.690 cx=212 cy=136 box=129,39,295,233 frame=320,240 latency_ms=233
I (115594) freddy_person: PERSON t_ms=114630 score=0.791 cx=202 cy=136 box=125,38,279,235 frame=320,240 latency_ms=233
I (115844) freddy_person: PERSON t_ms=114880 score=0.723 cx=208 cy=136 box=127,38,290,235 frame=320,240 latency_ms=233
I (116094) freddy_person: PERSON t_ms=115130 score=0.820 cx=213 cy=138 box=125,38,302,239 frame=320,240 latency_ms=233
I (116344) freddy_person: PERSON t_ms=115380 score=0.573 cx=206 cy=136 box=130,39,282,233 frame=320,240 latency_ms=233
I (116594) freddy_person: PERSON t_ms=115630 score=0.745 cx=208 cy=134 box=126,39,291,230 frame=320,240 latency_ms=233
I (116844) freddy_person: PERSON t_ms=115881 score=0.786 cx=203 cy=138 box=122,39,285,238 frame=320,240 latency_ms=233
I (117094) freddy_person: PERSON t_ms=116130 score=0.875 cx=218 cy=138 box=128,37,308,239 frame=320,240 latency_ms=233
I (117344) freddy_person: PERSON t_ms=116380 score=0.879 cx=220 cy=138 box=129,37,312,239 frame=320,240 latency_ms=233
I (117594) freddy_person: PERSON t_ms=116630 score=0.820 cx=220 cy=138 box=124,38,316,239 frame=320,240 latency_ms=233
I (117844) freddy_person: PERSON t_ms=116880 score=0.810 cx=205 cy=136 box=121,38,290,235 frame=320,240 latency_ms=232
I (118094) freddy_person: PERSON t_ms=117131 score=0.755 cx=211 cy=138 box=119,38,304,238 frame=320,240 latency_ms=233
I (118344) freddy_person: PERSON t_ms=117380 score=0.702 cx=215 cy=137 box=124,36,306,239 frame=320,240 latency_ms=233
I (118594) freddy_person: PERSON t_ms=117630 score=0.679 cx=214 cy=138 box=125,37,304,239 frame=320,240 latency_ms=233
I (118844) freddy_person: PERSON t_ms=117880 score=0.824 cx=214 cy=138 box=128,38,300,239 frame=320,240 latency_ms=233
I (119094) freddy_person: PERSON t_ms=118130 score=0.631 cx=192 cy=134 box=80,37,305,232 frame=320,240 latency_ms=232
I (119344) freddy_person: PERSON t_ms=118381 score=0.637 cx=218 cy=138 box=119,38,318,239 frame=320,240 latency_ms=233
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-left-image-frame-461.jpg (6661 bytes)
I (119734) freddy_person: PERSON t_ms=118769 score=0.810 cx=223 cy=139 box=128,39,319,239 frame=320,240 latency_ms=371
I (119984) freddy_person: PERSON t_ms=119021 score=0.530 cx=219 cy=135 box=129,37,310,233 frame=320,240 latency_ms=233
I (120234) freddy_person: PERSON t_ms=119271 score=0.857 cx=217 cy=138 box=122,38,313,239 frame=320,240 latency_ms=233
I (120484) freddy_person: PERSON t_ms=119520 score=0.786 cx=199 cy=137 box=122,38,276,237 frame=320,240 latency_ms=233
I (120734) freddy_person: PERSON t_ms=119770 score=0.707 cx=204 cy=132 box=124,39,285,225 frame=320,240 latency_ms=233
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-left-image.log; 48 person detections
@@ -0,0 +1,34 @@
I (106604) freddy_person: PERSON t_ms=105639 score=0.805 cx=218 I (133904) freddy_person: PERSON t_ms=132940 score=0.476 cx=35 cy=140 box=0,45,70,236 frame=320,240 latency_ms=232
I (134154) freddy_person: PERSON t_ms=133190 score=0.552 cx=240 cy=119 box=162,0,318,239 frame=320,240 latency_ms=233
I (134404) freddy_person: PERSON t_ms=133440 score=0.530 cx=242 cy=119 box=167,0,318,239 frame=320,240 latency_ms=233
I (134654) freddy_person: PERSON t_ms=133690 score=0.566 cx=38 cy=140 box=0,43,77,237 frame=320,240 latency_ms=233
I (134904) freddy_person: PERSON t_ms=133941 score=0.538 cx=235 cy=119 box=152,0,319,239 frame=320,240 latency_ms=233
I (135154) freddy_person: PERSON t_ms=134190 score=0.667 cx=240 cy=119 box=162,0,319,239 frame=320,240 latency_ms=233
I (135404) freddy_person: PERSON t_ms=134440 score=0.580 cx=42 cy=140 box=1,41,83,239 frame=320,240 latency_ms=233
I (135654) freddy_person: PERSON t_ms=134691 score=0.552 cx=40 cy=139 box=0,39,81,239 frame=320,240 latency_ms=233
I (135904) freddy_person: PERSON t_ms=134941 score=0.723 cx=235 cy=119 box=151,0,319,239 frame=320,240 latency_ms=233
I (136154) freddy_person: PERSON t_ms=135190 score=0.545 cx=234 cy=119 box=150,0,319,239 frame=320,240 latency_ms=233
I (136404) freddy_person: PERSON t_ms=135440 score=0.679 cx=47 cy=139 box=0,40,94,239 frame=320,240 latency_ms=233
I (136654) freddy_person: PERSON t_ms=135690 score=0.468 cx=43 cy=139 box=0,39,87,239 frame=320,240 latency_ms=233
I (136904) freddy_person: PERSON t_ms=135941 score=0.515 cx=42 cy=137 box=0,35,85,239 frame=320,240 latency_ms=233
I (137154) freddy_person: PERSON t_ms=136190 score=0.523 cx=52 cy=138 box=0,38,105,239 frame=320,240 latency_ms=233
I (137404) freddy_person: PERSON t_ms=136441 score=0.586 cx=238 cy=119 box=157,0,319,239 frame=320,240 latency_ms=234
I (137654) freddy_person: PERSON t_ms=136691 score=0.656 cx=47 cy=138 box=0,37,95,239 frame=320,240 latency_ms=233
I (137904) freddy_person: PERSON t_ms=136940 score=0.650 cx=235 cy=119 box=152,0,319,239 frame=320,240 latency_ms=233
I (138154) freddy_person: PERSON t_ms=137191 score=0.661 cx=236 cy=119 box=154,0,318,239 frame=320,240 latency_ms=233
I (138404) freddy_person: PERSON t_ms=137441 score=0.559 cx=49 cy=135 box=1,32,97,239 frame=320,240 latency_ms=233
I (138654) freddy_person: PERSON t_ms=137691 score=0.740 cx=235 cy=119 box=152,0,318,239 frame=320,240 latency_ms=233
I (138904) freddy_person: PERSON t_ms=137941 score=0.643 cx=237 cy=119 box=155,0,319,239 frame=320,240 latency_ms=233
I (139154) freddy_person: PERSON t_ms=138191 score=0.523 cx=241 cy=119 box=164,0,319,239 frame=320,240 latency_ms=233
I (139404) freddy_person: PERSON t_ms=138440 score=0.538 cx=237 cy=119 box=156,0,319,239 frame=320,240 latency_ms=233
I (139654) freddy_person: PERSON t_ms=138690 score=0.552 cx=240 cy=119 box=162,0,318,239 frame=320,240 latency_ms=233
I (139904) freddy_person: PERSON t_ms=138941 score=0.667 cx=236 cy=119 box=153,0,319,239 frame=320,240 latency_ms=233
I (140154) freddy_person: PERSON t_ms=139191 score=0.661 cx=235 cy=119 box=152,0,319,239 frame=320,240 latency_ms=233
I (140404) freddy_person: PERSON t_ms=139440 score=0.643 cx=240 cy=119 box=161,0,319,239 frame=320,240 latency_ms=232
I (140654) freddy_person: PERSON t_ms=139691 score=0.508 cx=243 cy=119 box=167,0,319,239 frame=320,240 latency_ms=233
I (140904) freddy_person: PERSON t_ms=139940 score=0.734 cx=239 cy=119 box=160,0,319,239 frame=320,240 latency_ms=232
I (141154) freddy_person: PERSON t_ms=140191 score=0.612 cx=241 cy=119 box=163,0,319,239 frame=320,240 latency_ms=233
I (141404) freddy_person: PERSON t_ms=140440 score=0.707 cx=237 cy=119 box=156,0,318,239 frame=320,240 latency_ms=233
I (141654) freddy_person: PERSON t_ms=140690 score=0.696 cx=240 cy=119 box=161,0,319,239 frame=320,240 latency_ms=233
I (141904) freddy_person: PERSON t_ms=140940 score=0.734 cx=239 cy=119 box=161,0,318,239 frame=320,240 latency_ms=232
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-right.log; 33 person detections
@@ -0,0 +1,51 @@
I (123) esp_image: segment 1: paddr=000933e4 vaddr=3fc98900 sizeI (38744) freddy_person: PERSON t_ms=37780 score=0.862 cx=68 cy=145 box=0,51,136,239 frame=320,240 latency_ms=233
I (38994) freddy_person: PERSON t_ms=38031 score=0.810 cx=61 cy=145 box=0,51,123,239 frame=320,240 latency_ms=233
I (39244) freddy_person: PERSON t_ms=38281 score=0.824 cx=63 cy=143 box=0,48,127,239 frame=320,240 latency_ms=233
I (39494) freddy_person: PERSON t_ms=38530 score=0.650 cx=62 cy=144 box=0,49,124,239 frame=320,240 latency_ms=233
I (39744) freddy_person: PERSON t_ms=38781 score=0.661 cx=63 cy=144 box=0,50,127,239 frame=320,240 latency_ms=233
I (39994) freddy_person: PERSON t_ms=39030 score=0.795 cx=65 cy=143 box=0,48,131,239 frame=320,240 latency_ms=233
I (40244) freddy_person: PERSON t_ms=39280 score=0.707 cx=63 cy=142 box=0,46,127,239 frame=320,240 latency_ms=233
I (40494) freddy_person: PERSON t_ms=39531 score=0.834 cx=66 cy=143 box=0,48,133,239 frame=320,240 latency_ms=233
I (40744) freddy_person: PERSON t_ms=39781 score=0.843 cx=61 cy=143 box=1,48,122,239 frame=320,240 latency_ms=233
I (40994) freddy_person: PERSON t_ms=40030 score=0.745 cx=63 cy=143 box=0,48,126,239 frame=320,240 latency_ms=233
I (41244) freddy_person: PERSON t_ms=40280 score=0.829 cx=64 cy=144 box=0,50,129,239 frame=320,240 latency_ms=232
I (41494) freddy_person: PERSON t_ms=40530 score=0.755 cx=60 cy=143 box=0,47,120,239 frame=320,240 latency_ms=232
I (41744) freddy_person: PERSON t_ms=40780 score=0.765 cx=62 cy=142 box=0,46,124,239 frame=320,240 latency_ms=233
I (41994) freddy_person: PERSON t_ms=41030 score=0.839 cx=63 cy=143 box=0,47,127,239 frame=320,240 latency_ms=232
I (42244) freddy_person: PERSON t_ms=41279 score=0.897 cx=61 cy=140 box=0,41,123,239 frame=320,240 latency_ms=232
I (42494) freddy_person: PERSON t_ms=41530 score=0.919 cx=61 cy=139 box=0,39,123,239 frame=320,240 latency_ms=233
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-right-image-frame-161.jpg (5650 bytes)
I (42864) freddy_person: PERSON t_ms=41899 score=0.862 cx=61 cy=140 box=0,42,123,239 frame=320,240 latency_ms=351
I (43114) freddy_person: PERSON t_ms=42150 score=0.884 cx=63 cy=141 box=0,44,126,239 frame=320,240 latency_ms=233
I (43364) freddy_person: PERSON t_ms=42400 score=0.888 cx=62 cy=141 box=0,44,124,239 frame=320,240 latency_ms=233
I (43614) freddy_person: PERSON t_ms=42649 score=0.829 cx=64 cy=140 box=0,42,128,239 frame=320,240 latency_ms=232
I (43864) freddy_person: PERSON t_ms=42900 score=0.820 cx=64 cy=141 box=0,43,128,239 frame=320,240 latency_ms=232
I (44114) freddy_person: PERSON t_ms=43150 score=0.910 cx=63 cy=141 box=0,43,126,239 frame=320,240 latency_ms=232
I (44364) freddy_person: PERSON t_ms=43400 score=0.884 cx=63 cy=141 box=0,43,126,239 frame=320,240 latency_ms=232
I (44614) freddy_person: PERSON t_ms=43650 score=0.910 cx=62 cy=140 box=0,42,124,239 frame=320,240 latency_ms=232
I (44864) freddy_person: PERSON t_ms=43900 score=0.901 cx=62 cy=141 box=0,44,124,239 frame=320,240 latency_ms=233
I (45114) freddy_person: PERSON t_ms=44150 score=0.888 cx=61 cy=140 box=0,42,123,239 frame=320,240 latency_ms=232
I (45364) freddy_person: PERSON t_ms=44400 score=0.871 cx=62 cy=141 box=0,44,124,239 frame=320,240 latency_ms=232
I (45614) freddy_person: PERSON t_ms=44650 score=0.781 cx=62 cy=141 box=0,44,124,239 frame=320,240 latency_ms=232
I (45864) freddy_person: PERSON t_ms=44900 score=0.852 cx=62 cy=141 box=1,43,124,239 frame=320,240 latency_ms=232
I (46114) freddy_person: PERSON t_ms=45151 score=0.879 cx=63 cy=141 box=0,44,127,239 frame=320,240 latency_ms=233
I (46364) freddy_person: PERSON t_ms=45401 score=0.914 cx=64 cy=141 box=0,44,128,239 frame=320,240 latency_ms=233
I (46614) freddy_person: PERSON t_ms=45651 score=0.906 cx=62 cy=142 box=0,46,125,239 frame=320,240 latency_ms=233
I (46864) freddy_person: PERSON t_ms=45900 score=0.884 cx=62 cy=142 box=0,46,125,239 frame=320,240 latency_ms=232
I (47114) freddy_person: PERSON t_ms=46150 score=0.871 cx=65 cy=143 box=0,47,131,239 frame=320,240 latency_ms=232
I (47364) freddy_person: PERSON t_ms=46400 score=0.848 cx=62 cy=142 box=0,45,124,239 frame=320,240 latency_ms=233
I (47614) freddy_person: PERSON t_ms=46650 score=0.734 cx=62 cy=141 box=0,44,124,239 frame=320,240 latency_ms=232
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-right-image-frame-181.jpg (5545 bytes)
I (47974) freddy_person: PERSON t_ms=47017 score=0.834 cx=63 cy=142 box=0,46,127,239 frame=320,240 latency_ms=349
I (48234) freddy_person: PERSON t_ms=47271 score=0.857 cx=61 cy=142 box=0,46,122,239 frame=320,240 latency_ms=233
I (48484) freddy_person: PERSON t_ms=47520 score=0.857 cx=64 cy=142 box=0,45,128,239 frame=320,240 latency_ms=232
I (48734) freddy_person: PERSON t_ms=47770 score=0.750 cx=63 cy=143 box=1,47,125,239 frame=320,240 latency_ms=232
I (48984) freddy_person: PERSON t_ms=48020 score=0.871 cx=62 cy=142 box=0,46,125,239 frame=320,240 latency_ms=233
I (49234) freddy_person: PERSON t_ms=48270 score=0.871 cx=61 cy=142 box=0,46,122,239 frame=320,240 latency_ms=233
I (49484) freddy_person: PERSON t_ms=48519 score=0.857 cx=62 cy=143 box=0,47,124,239 frame=320,240 latency_ms=232
I (49734) freddy_person: PERSON t_ms=48770 score=0.805 cx=62 cy=142 box=0,46,124,239 frame=320,240 latency_ms=233
I (49984) freddy_person: PERSON t_ms=49020 score=0.786 cx=62 cy=142 box=0,45,125,239 frame=320,240 latency_ms=233
I (50234) freddy_person: PERSON t_ms=49270 score=0.857 cx=62 cy=142 box=1,46,123,239 frame=320,240 latency_ms=233
I (50484) freddy_person: PERSON t_ms=49520 score=0.866 cx=61 cy=142 box=0,45,123,239 frame=320,240 latency_ms=233
I (50734) freddy_person: PERSON t_ms=49770 score=0.820 cx=61 cy=142 box=0,45,123,239 frame=320,240 latency_ms=232
Saved /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/hardware/camera-captures/person-detector-right-image.log; 48 person detections
@@ -0,0 +1,64 @@
"""Record serial detections and reconstruct diagnostic JPEGs from the camera."""
import argparse
import pathlib
import time
import serial
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--port", default="/dev/cu.usbmodem1101")
parser.add_argument("--seconds", type=float, default=30)
parser.add_argument("--name", default="person-detector-diagnostic")
args = parser.parse_args()
output = pathlib.Path(__file__).resolve().parents[2] / "hardware/camera-captures"
output.mkdir(parents=True, exist_ok=True)
log_path = output / f"{args.name}.log"
frame_number = None
expected_length = 0
hex_lines: list[str] = []
detections = 0
lines: list[str] = []
with serial.Serial(args.port, 115200, timeout=0.5) as port:
deadline = time.monotonic() + args.seconds
while time.monotonic() < deadline:
raw = port.readline()
if not raw:
continue
line = raw.decode("utf-8", "replace").strip()
lines.append(line)
if "freddy_person: PERSON " in line:
detections += 1
print(line, flush=True)
elif "freddy_person: NO_PERSON " in line and len(lines) % 20 == 0:
print(line, flush=True)
if line.startswith("FRAME_BEGIN "):
_, number, length = line.split()
frame_number = int(number)
expected_length = int(length)
hex_lines = []
elif line.startswith("FRAME_HEX ") and frame_number is not None:
hex_lines.append(line.removeprefix("FRAME_HEX "))
elif line.startswith("FRAME_END ") and frame_number is not None:
number = int(line.split()[1])
if number == frame_number:
data = bytes.fromhex("".join(hex_lines))
if len(data) == expected_length and data.startswith(b"\xff\xd8") and data.endswith(b"\xff\xd9"):
photo = output / f"{args.name}-frame-{number}.jpg"
photo.write_bytes(data)
print(f"Saved {photo} ({len(data)} bytes)", flush=True)
else:
print(f"Incomplete frame {number}: {len(data)}/{expected_length} bytes", flush=True)
frame_number = None
log_path.write_text("\n".join(lines) + "\n")
print(f"Saved {log_path}; {detections} person detections", flush=True)
if __name__ == "__main__":
main()
@@ -0,0 +1,103 @@
dependencies:
espressif/dl_fft:
component_hash: 3e8f7d34e04bfa6333f289a6d126745bebe0cee9311b81a6a5d106047f63ef65
dependencies:
- name: idf
require: private
version: '>=5.0'
source:
registry_url: https://components.espressif.com
type: service
version: 0.7.0
espressif/esp-dl:
component_hash: 233ee8d37dfedeb637fc74713b87bd861ac530cfbcc4c63c57c2e51064cdda31
dependencies:
- name: espressif/dl_fft
registry_url: https://components.espressif.com
require: private
version: '>=0.6.0'
- name: espressif/esp_new_jpeg
registry_url: https://components.espressif.com
require: private
version: ^1
- name: idf
require: private
version: '*'
source:
registry_url: https://components.espressif.com
type: service
targets:
- esp32
- esp32c2
- esp32c3
- esp32c5
- esp32c6
- esp32p4
- esp32s2
- esp32s3
- esp32s31
version: 3.3.11
espressif/esp32-camera:
component_hash: bc9c8a6b51df777a014fa295825b3de5069bc0300c317acff20c97cf4a10ac7d
dependencies:
- name: espressif/esp_jpeg
registry_url: https://components.espressif.com
require: public
version: ^1.3.1
- name: idf
require: private
version: '>=5.1'
source:
registry_url: https://components.espressif.com/
type: service
version: 2.1.7
espressif/esp_jpeg:
component_hash: defb83669293cbf86d0fa86b475ba5517aceed04ed70db435388c151ab37b5d7
dependencies:
- name: idf
require: private
version: '>=5.0'
source:
registry_url: https://components.espressif.com
type: service
version: 1.3.1
espressif/esp_new_jpeg:
component_hash: e1e7e6991193c41ed6c9fc386702aa7821433319e7aa9c3147cd3236310cd69c
dependencies: []
source:
registry_url: https://components.espressif.com
type: service
targets:
- esp32
- esp32s2
- esp32s3
- esp32s31
- esp32p4
- esp32c2
- esp32c3
- esp32c5
- esp32c6
- esp32c61
- esp32h4
version: 1.0.2
espressif/pedestrian_detect:
component_hash: 882a6986cb5e7e9bb8e87861fb42379a7eeb2f4a4a78104c6a78f42a6cb35fb4
dependencies:
- name: espressif/esp-dl
registry_url: https://components.espressif.com
require: private
version: ~3.3.0
source:
registry_url: https://components.espressif.com/
type: service
version: 0.3.2
idf:
source:
type: idf
version: 5.5.2
direct_dependencies:
- espressif/esp32-camera
- espressif/pedestrian_detect
manifest_hash: da1a3cbbaa4755170f4ffdc5b90495e507c88e07fa97fb46cd1af3e5f728a198
target: esp32s3
version: 2.0.0
@@ -0,0 +1,114 @@
Executing action: flash
Running ninja in directory /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build
Executing "ninja flash"...
[1/5] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/partition_table/partition-table.bin /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/freddy_camera_person.bin
freddy_camera_person.bin binary size 0x288bd0 bytes. Smallest app partition is 0xa00000 bytes. 0x777430 bytes (75%) free.
[2/5] Performing build step for 'bootloader'
[1/1] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x0 /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/bootloader.bin
Bootloader binary size 0x5240 bytes. 0x2dc0 bytes (36%) free.
[3/5] No install step for 'bootloader'
[4/5] Completed 'bootloader'
[4/5] cd /Users/joshua/esp/esp-idf/components/esptool_py && /opt/homebrew/bin/cmake -D IDF_PATH=/Users/joshua/esp/esp-idf -D "SERIAL_TOOL=/Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python;;/Users/joshua/esp/esp-idf/components/esptool_py/esptool/esptool.py;--chip;esp32s3" -D "SERIAL_TOOL_ARGS=--before=default_reset;--after=hard_reset;write_flash;@flash_args" -D WORKING_DIRECTORY=/Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build -P /Users/joshua/esp/esp-idf/components/esptool_py/run_serial_tool.cmake
esptool.py --chip esp32s3 -p /dev/cu.usbmodem1101 -b 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x0 bootloader/bootloader.bin 0x10000 freddy_camera_person.bin 0x8000 partition_table/partition-table.bin
esptool.py v4.12.dev1
Serial port /dev/cu.usbmodem1101
Connecting...
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
USB mode: USB-Serial/JTAG
MAC: e8:f6:0a:87:0b:4c
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00010000 to 0x00298fff...
Flash will be erased from 0x00008000 to 0x00008fff...
SHA digest in image updated
Compressed 21056 bytes to 13390...
Writing at 0x00000000... (100 %)
Wrote 21056 bytes (13390 compressed) at 0x00000000 in 0.4 seconds (effective 405.0 kbit/s)...
Hash of data verified.
Compressed 2657232 bytes to 1115837...
Writing at 0x00010000... (1 %)
Writing at 0x0001e8c5... (2 %)
Writing at 0x00026ccf... (4 %)
Writing at 0x0002cb86... (5 %)
Writing at 0x0003235f... (7 %)
Writing at 0x00038125... (8 %)
Writing at 0x0003de36... (10 %)
Writing at 0x00043c8c... (11 %)
Writing at 0x000496a3... (13 %)
Writing at 0x0004fa83... (14 %)
Writing at 0x000557c4... (15 %)
Writing at 0x0005ae4b... (17 %)
Writing at 0x000605c4... (18 %)
Writing at 0x00066665... (20 %)
Writing at 0x0006bbdb... (21 %)
Writing at 0x0007236c... (23 %)
Writing at 0x0007744c... (24 %)
Writing at 0x0007d49f... (26 %)
Writing at 0x00082cef... (27 %)
Writing at 0x0009577a... (28 %)
Writing at 0x0009c3a6... (30 %)
Writing at 0x000a3a13... (31 %)
Writing at 0x000aa016... (33 %)
Writing at 0x000b15d1... (34 %)
Writing at 0x000b896d... (36 %)
Writing at 0x000bfd09... (37 %)
Writing at 0x000c8751... (39 %)
Writing at 0x000ce815... (40 %)
Writing at 0x000d4c5a... (42 %)
Writing at 0x000dbeea... (43 %)
Writing at 0x000e55f3... (44 %)
Writing at 0x000edb25... (46 %)
Writing at 0x000f3d22... (47 %)
Writing at 0x000fad22... (49 %)
Writing at 0x00102d30... (50 %)
Writing at 0x0010cd0e... (52 %)
Writing at 0x001156a3... (53 %)
Writing at 0x0011c149... (55 %)
Writing at 0x00123b6e... (56 %)
Writing at 0x0012d6a3... (57 %)
Writing at 0x00135e65... (59 %)
Writing at 0x0013e021... (60 %)
Writing at 0x00148b3b... (62 %)
Writing at 0x001566b5... (63 %)
Writing at 0x0015fa75... (65 %)
Writing at 0x00168e33... (66 %)
Writing at 0x00172f27... (68 %)
Writing at 0x0017c88d... (69 %)
Writing at 0x00186d9a... (71 %)
Writing at 0x0019200d... (72 %)
Writing at 0x0019e78a... (73 %)
Writing at 0x001a8ad4... (75 %)
Writing at 0x001b3649... (76 %)
Writing at 0x001b98b0... (78 %)
Writing at 0x001c74e4... (79 %)
Writing at 0x001d3605... (81 %)
Writing at 0x001dcaa2... (82 %)
Writing at 0x001e800e... (84 %)
Writing at 0x001f21ae... (85 %)
Writing at 0x00211ab2... (86 %)
Writing at 0x0021a5bf... (88 %)
Writing at 0x00222864... (89 %)
Writing at 0x0022c094... (91 %)
Writing at 0x0023283c... (92 %)
Writing at 0x0023dd77... (94 %)
Writing at 0x0027ca68... (95 %)
Writing at 0x0028a884... (97 %)
Writing at 0x00291532... (98 %)
Writing at 0x00297f10... (100 %)
Wrote 2657232 bytes (1115837 compressed) at 0x00010000 in 19.5 seconds (effective 1088.0 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 105...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (105 compressed) at 0x00008000 in 0.1 seconds (effective 315.7 kbit/s)...
Hash of data verified.
Leaving...
Hard resetting via RTS pin...
Done
@@ -0,0 +1,114 @@
Executing action: flash
Running ninja in directory /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build
Executing "ninja flash"...
[1/5] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/partition_table/partition-table.bin /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/freddy_camera_person.bin
freddy_camera_person.bin binary size 0x288e00 bytes. Smallest app partition is 0xa00000 bytes. 0x777200 bytes (75%) free.
[2/5] Performing build step for 'bootloader'
[1/1] cd /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/esp-idf/esptool_py && /Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python /Users/joshua/esp/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x0 /Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build/bootloader/bootloader.bin
Bootloader binary size 0x5240 bytes. 0x2dc0 bytes (36%) free.
[3/5] No install step for 'bootloader'
[4/5] Completed 'bootloader'
[4/5] cd /Users/joshua/esp/esp-idf/components/esptool_py && /opt/homebrew/bin/cmake -D IDF_PATH=/Users/joshua/esp/esp-idf -D "SERIAL_TOOL=/Users/joshua/.espressif/python_env/idf5.5_py3.13_env/bin/python;;/Users/joshua/esp/esp-idf/components/esptool_py/esptool/esptool.py;--chip;esp32s3" -D "SERIAL_TOOL_ARGS=--before=default_reset;--after=hard_reset;write_flash;@flash_args" -D WORKING_DIRECTORY=/Users/joshua/Code/3d-creations/freddy_fazbear_digital_eyes/firmware/camera_person_idf/build -P /Users/joshua/esp/esp-idf/components/esptool_py/run_serial_tool.cmake
esptool.py --chip esp32s3 -p /dev/cu.usbmodem1101 -b 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x0 bootloader/bootloader.bin 0x10000 freddy_camera_person.bin 0x8000 partition_table/partition-table.bin
esptool.py v4.12.dev1
Serial port /dev/cu.usbmodem1101
Connecting...
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
USB mode: USB-Serial/JTAG
MAC: e8:f6:0a:87:0b:4c
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00010000 to 0x00298fff...
Flash will be erased from 0x00008000 to 0x00008fff...
SHA digest in image updated
Compressed 21056 bytes to 13390...
Writing at 0x00000000... (100 %)
Wrote 21056 bytes (13390 compressed) at 0x00000000 in 0.4 seconds (effective 404.8 kbit/s)...
Hash of data verified.
Compressed 2657792 bytes to 1116115...
Writing at 0x00010000... (1 %)
Writing at 0x0001e8b0... (2 %)
Writing at 0x00026d2d... (4 %)
Writing at 0x0002cbde... (5 %)
Writing at 0x000323bd... (7 %)
Writing at 0x0003817d... (8 %)
Writing at 0x0003de91... (10 %)
Writing at 0x00043ce2... (11 %)
Writing at 0x000496ef... (13 %)
Writing at 0x0004fad8... (14 %)
Writing at 0x0005582a... (15 %)
Writing at 0x0005aeab... (17 %)
Writing at 0x0006062a... (18 %)
Writing at 0x000666c3... (20 %)
Writing at 0x0006bc3f... (21 %)
Writing at 0x000723ba... (23 %)
Writing at 0x000774ab... (24 %)
Writing at 0x0007d509... (26 %)
Writing at 0x00082d47... (27 %)
Writing at 0x00095771... (28 %)
Writing at 0x0009c3f7... (30 %)
Writing at 0x000a39a3... (31 %)
Writing at 0x000a9f90... (33 %)
Writing at 0x000b150c... (34 %)
Writing at 0x000b88cb... (36 %)
Writing at 0x000bfc9f... (37 %)
Writing at 0x000c86ac... (39 %)
Writing at 0x000ce799... (40 %)
Writing at 0x000d4bff... (42 %)
Writing at 0x000dbd35... (43 %)
Writing at 0x000e55ae... (44 %)
Writing at 0x000edaec... (46 %)
Writing at 0x000f3cc1... (47 %)
Writing at 0x000facde... (49 %)
Writing at 0x00102cc4... (50 %)
Writing at 0x0010ccc5... (52 %)
Writing at 0x00115672... (53 %)
Writing at 0x0011c0eb... (55 %)
Writing at 0x00123ad4... (56 %)
Writing at 0x0012d46e... (57 %)
Writing at 0x00135daf... (59 %)
Writing at 0x0013df8a... (60 %)
Writing at 0x00148995... (62 %)
Writing at 0x001565f9... (63 %)
Writing at 0x0015f9a4... (65 %)
Writing at 0x00168bea... (66 %)
Writing at 0x00172e2d... (68 %)
Writing at 0x0017c7f4... (69 %)
Writing at 0x00186d0a... (71 %)
Writing at 0x00191e8e... (72 %)
Writing at 0x0019e70d... (73 %)
Writing at 0x001a8a1b... (75 %)
Writing at 0x001b354d... (76 %)
Writing at 0x001b9818... (78 %)
Writing at 0x001c73a9... (79 %)
Writing at 0x001d3588... (81 %)
Writing at 0x001dc97d... (82 %)
Writing at 0x001e7f86... (84 %)
Writing at 0x001f2161... (85 %)
Writing at 0x00211397... (86 %)
Writing at 0x0021a4e4... (88 %)
Writing at 0x00222832... (89 %)
Writing at 0x0022bf16... (91 %)
Writing at 0x0023281e... (92 %)
Writing at 0x0023d6d4... (94 %)
Writing at 0x00278406... (95 %)
Writing at 0x0028a879... (97 %)
Writing at 0x0029159d... (98 %)
Writing at 0x00297f4b... (100 %)
Wrote 2657792 bytes (1116115 compressed) at 0x00010000 in 19.5 seconds (effective 1092.8 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 105...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (105 compressed) at 0x00008000 in 0.1 seconds (effective 352.1 kbit/s)...
Hash of data verified.
Leaving...
Hard resetting via RTS pin...
Done
@@ -0,0 +1,3 @@
idf_component_register(SRCS "main.cpp" "ota_service.cpp" INCLUDE_DIRS "."
REQUIRES app_update esp_http_server esp_wifi esp_netif
nvs_flash mbedtls)
@@ -0,0 +1,3 @@
dependencies:
espressif/esp32-camera: "^2.1.7"
espressif/pedestrian_detect: "^0.3.2"
+251
View File
@@ -0,0 +1,251 @@
// DFR1154 V1.1 person-position experiment using Espressif's pedestrian model.
// Camera pin assignments match DFRobot's DFR1154 CameraWebServer example.
#include <algorithm>
#include <cstdint>
#include <cstdio>
#include "dl_image_jpeg.hpp"
#include "driver/gpio.h"
#include "esp_camera.h"
#include "esp_heap_caps.h"
#include "esp_log.h"
#include "esp_timer.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "pedestrian_detect.hpp"
#include "driver/uart.h"
#include "../../shared/GazeProtocol.h"
#include "../../shared/TrackingGate.h"
#include "ota_service.h"
namespace {
constexpr char kTag[] = "freddy_person";
constexpr float kDiagnosticScoreThreshold = 0.35f;
constexpr gpio_num_t kIrLedPin = GPIO_NUM_47;
constexpr uart_port_t kGazeUart = UART_NUM_1;
constexpr int kGazeTxPin = 43;
constexpr int kGazeBaud = 115200;
void init_gaze_uart() {
const uart_config_t config = {
.baud_rate = kGazeBaud,
.data_bits = UART_DATA_8_BITS,
.parity = UART_PARITY_DISABLE,
.stop_bits = UART_STOP_BITS_1,
.flow_ctrl = UART_HW_FLOWCTRL_DISABLE,
.rx_flow_ctrl_thresh = 0,
.source_clk = UART_SCLK_DEFAULT,
.flags = {},
};
ESP_ERROR_CHECK(uart_driver_install(kGazeUart, 1024, 0, 0, nullptr, 0));
ESP_ERROR_CHECK(uart_param_config(kGazeUart, &config));
ESP_ERROR_CHECK(uart_set_pin(kGazeUart, kGazeTxPin, UART_PIN_NO_CHANGE,
UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE));
ESP_LOGI(kTag, "GAZE_UART_READY tx_gpio=%d baud=%d", kGazeTxPin, kGazeBaud);
}
void send_gaze(uint16_t sequence, int x, int y, bool valid) {
gaze::Frame command = {sequence, static_cast<uint16_t>(x),
static_cast<uint16_t>(y), valid};
uint8_t bytes[gaze::kFrameSize];
gaze::encode(command, bytes);
if (uart_write_bytes(kGazeUart, bytes, sizeof(bytes)) != sizeof(bytes)) {
ESP_LOGW(kTag, "GAZE_UART_WRITE_ERROR seq=%u", sequence);
}
}
#ifndef FREDDY_GAZE_LINK_SELF_TEST
#define FREDDY_GAZE_LINK_SELF_TEST 0
#endif
#if FREDDY_GAZE_LINK_SELF_TEST
// Bench-only UART fault injection. Keep disabled in the normal camera build.
void run_gaze_link_self_test() {
uint8_t bytes[gaze::kFrameSize];
gaze::encode({0xF000, 500, 500, false}, bytes);
uart_write_bytes(kGazeUart, bytes, sizeof(bytes));
vTaskDelay(pdMS_TO_TICKS(150));
uart_write_bytes(kGazeUart, bytes, sizeof(bytes)); // Duplicate sequence.
vTaskDelay(pdMS_TO_TICKS(150));
gaze::encode({0xEFFF, 500, 500, false}, bytes);
uart_write_bytes(kGazeUart, bytes, sizeof(bytes)); // Old sequence.
vTaskDelay(pdMS_TO_TICKS(150));
gaze::encode({0xF001, 500, 500, false}, bytes);
bytes[2] = 2; // Wrong version.
uart_write_bytes(kGazeUart, bytes, sizeof(bytes));
vTaskDelay(pdMS_TO_TICKS(150));
gaze::encode({0xF001, 500, 500, false}, bytes);
bytes[10] ^= 1; // Damaged CRC.
uart_write_bytes(kGazeUart, bytes, sizeof(bytes));
vTaskDelay(pdMS_TO_TICKS(150));
send_gaze(0xF002, 500, 500, false); // Recovery.
ESP_LOGI(kTag, "GAZE_LINK_SELF_TEST_SENT");
}
#endif
#ifndef FREDDY_CAMERA_DIAGNOSTIC_JPEG
#define FREDDY_CAMERA_DIAGNOSTIC_JPEG 0
#endif
#if FREDDY_CAMERA_DIAGNOSTIC_JPEG
// Optional bench diagnostic: one JPEG roughly every five seconds.
void print_jpeg_for_diagnostics(const camera_fb_t *frame, unsigned frame_number) {
printf("FRAME_BEGIN %u %u\n", frame_number, static_cast<unsigned>(frame->len));
for (size_t offset = 0; offset < frame->len; offset += 48) {
printf("FRAME_HEX ");
for (size_t i = offset; i < frame->len && i < offset + 48; ++i) {
printf("%02x", frame->buf[i]);
}
printf("\n");
}
printf("FRAME_END %u\n", frame_number);
fflush(stdout);
}
#endif
camera_config_t camera_config() {
camera_config_t config = {};
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = 16;
config.pin_d1 = 18;
config.pin_d2 = 21;
config.pin_d3 = 17;
config.pin_d4 = 14;
config.pin_d5 = 7;
config.pin_d6 = 6;
config.pin_d7 = 4;
config.pin_xclk = 5;
config.pin_pclk = 15;
config.pin_vsync = 1;
config.pin_href = 2;
config.pin_sccb_sda = 8;
config.pin_sccb_scl = 9;
config.pin_pwdn = -1;
config.pin_reset = -1;
config.xclk_freq_hz = 20000000;
config.pixel_format = PIXFORMAT_JPEG;
config.frame_size = FRAMESIZE_QVGA;
config.jpeg_quality = 12;
config.fb_count = 1;
config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
config.fb_location = CAMERA_FB_IN_PSRAM;
return config;
}
} // namespace
extern "C" void app_main(void) {
init_gaze_uart();
#if FREDDY_GAZE_LINK_SELF_TEST
run_gaze_link_self_test();
#endif
ESP_ERROR_CHECK(gpio_reset_pin(kIrLedPin));
ESP_ERROR_CHECK(gpio_set_direction(kIrLedPin, GPIO_MODE_OUTPUT));
ESP_ERROR_CHECK(gpio_set_level(kIrLedPin, 1));
ESP_LOGI(kTag, "IR_GPIO47_HIGH");
camera_config_t config = camera_config();
const esp_err_t camera_error = esp_camera_init(&config);
if (camera_error != ESP_OK) {
ESP_LOGE(kTag, "CAMERA_INIT_ERROR 0x%X", camera_error);
return;
}
sensor_t *sensor = esp_camera_sensor_get();
if (sensor) {
ESP_LOGI(kTag, "SENSOR_PID 0x%04X", sensor->id.PID);
if (sensor->id.PID == OV3660_PID) sensor->set_vflip(sensor, 1);
}
auto *detector = new PedestrianDetect();
detector->set_score_thr(kDiagnosticScoreThreshold);
ESP_LOGI(kTag, "PERSON_DETECTOR_READY threshold=%.2f", kDiagnosticScoreThreshold);
ota_service_start();
ota_service_confirm_healthy();
unsigned frame_number = 0;
uint16_t gaze_sequence = 0;
tracking::Gate target_gate;
for (;;) {
const int64_t capture_started = esp_timer_get_time();
camera_fb_t *frame = esp_camera_fb_get();
if (!frame) {
ESP_LOGW(kTag, "CAPTURE_ERROR");
vTaskDelay(pdMS_TO_TICKS(100));
continue;
}
++frame_number;
#if FREDDY_CAMERA_DIAGNOSTIC_JPEG
if (frame_number % 20 == 1) {
print_jpeg_for_diagnostics(frame, frame_number);
}
#endif
dl::image::jpeg_img_t jpeg = {
.data = frame->buf,
.data_len = frame->len,
};
auto image = dl::image::sw_decode_jpeg(jpeg, dl::image::DL_IMAGE_PIX_TYPE_RGB888);
esp_camera_fb_return(frame);
if (!image.data) {
ESP_LOGW(kTag, "DECODE_ERROR");
vTaskDelay(pdMS_TO_TICKS(100));
continue;
}
auto &results = detector->run(image);
const int64_t completed = esp_timer_get_time();
const dl::detect::result_t *best = nullptr;
int rejected_full_frame = 0;
int rejected_full_height = 0;
for (const auto &result : results) {
const int box_width = result.box[2] - result.box[0];
const int box_height = result.box[3] - result.box[1];
// The empty-scene diagnostic repeatedly labels almost the entire image
// as a person. Keep this shape from commanding either eye.
if (box_width >= image.width * 9 / 10 &&
box_height >= image.height * 9 / 10) {
++rejected_full_frame;
continue;
}
// The confirmed empty view repeatedly produces boxes running nearly
// from the top edge to the bottom edge. Earlier real-person bench boxes
// did not have this geometry; validate again at the installed distance.
if (box_height >= image.height * 95 / 100 && result.box[1] <= 2) {
++rejected_full_height;
continue;
}
if (!best || result.score > best->score) best = &result;
}
const int center_x = best ? (best->box[0] + best->box[2]) / 2 : 0;
const int center_y = best ? (best->box[1] + best->box[3]) / 2 : 0;
const tracking::Decision target = target_gate.update(
best != nullptr, center_x, center_y,
static_cast<uint32_t>(completed / 1000));
const int gaze_x = (!best && !target.valid)
? 500
: std::clamp(target.x, 0, image.width - 1) * 1000 /
(image.width - 1);
const int gaze_y = (!best && !target.valid)
? 500
: std::clamp(target.y, 0, image.height - 1) * 1000 /
(image.height - 1);
send_gaze(gaze_sequence++, gaze_x, gaze_y, target.valid);
if (best) {
ESP_LOGI(kTag,
"PERSON t_ms=%lld score=%.3f cx=%d cy=%d box=%d,%d,%d,%d frame=%d,%d latency_ms=%lld gaze_valid=%d target_hold=%d rejected_full_height=%d",
static_cast<long long>(completed / 1000), best->score, center_x,
center_y, best->box[0], best->box[1], best->box[2], best->box[3],
image.width, image.height,
static_cast<long long>((completed - capture_started) / 1000),
target.valid, target.held, rejected_full_height);
} else {
ESP_LOGI(kTag, "NO_PERSON t_ms=%lld frame=%d,%d latency_ms=%lld rejected_full_frame=%d rejected_full_height=%d gaze_valid=%d target_hold=%d",
static_cast<long long>(completed / 1000), image.width, image.height,
static_cast<long long>((completed - capture_started) / 1000),
rejected_full_frame, rejected_full_height, target.valid,
target.held);
}
heap_caps_free(image.data);
vTaskDelay(pdMS_TO_TICKS(20));
}
}
@@ -0,0 +1,195 @@
#include "ota_service.h"
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include "esp_event.h"
#include "esp_http_server.h"
#include "esp_log.h"
#include "esp_netif.h"
#include "esp_ota_ops.h"
#include "esp_random.h"
#include "esp_timer.h"
#include "esp_wifi.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "mbedtls/sha256.h"
#include "nvs_flash.h"
#include "../../shared/OtaAuth.h"
#include "../../shared/OtaSecrets.h"
namespace {
constexpr char kTag[] = "freddy_ota";
constexpr char kDevice[] = "camera";
constexpr char kImageTarget[] = "FREDDY_OTA_TARGET:camera";
constexpr int kPort = 3232;
char nonce[33] = {};
int64_t nonce_time_us = 0;
httpd_handle_t server = nullptr;
void new_nonce() {
uint8_t random_bytes[16];
esp_fill_random(random_bytes, sizeof(random_bytes));
freddy_ota::to_hex(random_bytes, sizeof(random_bytes), nonce);
nonce_time_us = esp_timer_get_time();
}
esp_err_t info_handler(httpd_req_t *req) {
new_nonce();
const esp_partition_t *slot = esp_ota_get_next_update_partition(nullptr);
const esp_partition_t *running = esp_ota_get_running_partition();
char body[180];
snprintf(body, sizeof(body),
"{\"device\":\"%s\",\"nonce\":\"%s\",\"slot_bytes\":%u,\"running_slot\":%d}",
kDevice, nonce, slot ? static_cast<unsigned>(slot->size) : 0,
running ? static_cast<int>(running->subtype) - ESP_PARTITION_SUBTYPE_APP_OTA_0 : -1);
httpd_resp_set_type(req, "application/json");
return httpd_resp_sendstr(req, body);
}
bool read_header(httpd_req_t *req, const char *name, char *out, size_t capacity) {
const size_t length = httpd_req_get_hdr_value_len(req, name);
return length > 0 && length < capacity &&
httpd_req_get_hdr_value_str(req, name, out, capacity) == ESP_OK;
}
void restart_task(void *) {
vTaskDelay(pdMS_TO_TICKS(1000));
esp_restart();
}
esp_err_t upload_handler(httpd_req_t *req) {
char request_nonce[33], request_sha[65], signature[65];
const esp_partition_t *slot = esp_ota_get_next_update_partition(nullptr);
if (!slot || req->content_len <= 0 ||
static_cast<size_t>(req->content_len) > slot->size ||
!read_header(req, "X-Freddy-Nonce", request_nonce, sizeof(request_nonce)) ||
!read_header(req, "X-Freddy-SHA256", request_sha, sizeof(request_sha)) ||
!read_header(req, "X-Freddy-Signature", signature, sizeof(signature)) ||
esp_timer_get_time() - nonce_time_us > 60000000 ||
strcmp(request_nonce, nonce) != 0 ||
!freddy_ota::verify(FREDDY_OTA_KEY, kDevice, nonce, request_sha,
req->content_len, signature)) {
httpd_resp_send_err(req, HTTPD_403_FORBIDDEN, "OTA authorization failed");
return ESP_FAIL;
}
nonce[0] = 0; // Prevent replay even after an interrupted upload.
esp_ota_handle_t handle;
esp_err_t result = esp_ota_begin(slot, req->content_len, &handle);
if (result != ESP_OK) {
httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "OTA begin failed");
return ESP_FAIL;
}
mbedtls_sha256_context sha_context;
mbedtls_sha256_init(&sha_context);
result = mbedtls_sha256_starts(&sha_context, 0) == 0 ? ESP_OK : ESP_FAIL;
uint8_t buffer[2048];
int remaining = req->content_len;
unsigned timeouts = 0;
while (result == ESP_OK && remaining > 0) {
const int received = httpd_req_recv(
req, reinterpret_cast<char *>(buffer),
remaining < static_cast<int>(sizeof(buffer)) ? remaining : sizeof(buffer));
if (received == HTTPD_SOCK_ERR_TIMEOUT && ++timeouts < 5) continue;
if (received <= 0) {
result = ESP_FAIL;
break;
}
timeouts = 0;
result = esp_ota_write(handle, buffer, received);
if (result == ESP_OK && mbedtls_sha256_update(&sha_context, buffer, received) != 0)
result = ESP_FAIL;
remaining -= received;
}
uint8_t digest[32] = {};
if (result == ESP_OK &&
mbedtls_sha256_finish(&sha_context, digest) != 0) result = ESP_FAIL;
mbedtls_sha256_free(&sha_context);
char actual_sha[65];
freddy_ota::to_hex(digest, sizeof(digest), actual_sha);
if (result == ESP_OK &&
!freddy_ota::constant_time_equal(actual_sha, request_sha, 64))
result = ESP_FAIL;
if (result == ESP_OK) result = esp_ota_end(handle);
else esp_ota_abort(handle);
if (result == ESP_OK) result = esp_ota_set_boot_partition(slot);
if (result != ESP_OK) {
ESP_LOGE(kTag, "OTA upload failed: %s", esp_err_to_name(result));
httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "OTA image rejected");
return ESP_FAIL;
}
httpd_resp_set_type(req, "application/json");
httpd_resp_sendstr(req, "{\"ok\":true}");
xTaskCreate(restart_task, "ota_restart", 2048, nullptr, 5, nullptr);
return ESP_OK;
}
void wifi_event(void *, esp_event_base_t base, int32_t id, void *data) {
if (base == WIFI_EVENT && id == WIFI_EVENT_STA_DISCONNECTED) {
esp_wifi_connect();
} else if (base == IP_EVENT && id == IP_EVENT_STA_GOT_IP) {
const auto *event = static_cast<ip_event_got_ip_t *>(data);
ESP_LOGI(kTag, "OTA_READY device=%s image=%s ip=" IPSTR " port=%d",
kDevice, kImageTarget, IP2STR(&event->ip_info.ip), kPort);
}
}
} // namespace
void ota_service_start() {
if (!FREDDY_WIFI_SSID[0] || strlen(FREDDY_OTA_KEY) < 32) {
ESP_LOGW(kTag, "OTA_DISABLED: configure firmware/private/ first");
return;
}
esp_err_t result = nvs_flash_init();
if (result == ESP_ERR_NVS_NO_FREE_PAGES ||
result == ESP_ERR_NVS_NEW_VERSION_FOUND) {
ESP_ERROR_CHECK(nvs_flash_erase());
result = nvs_flash_init();
}
ESP_ERROR_CHECK(result);
ESP_ERROR_CHECK(esp_netif_init());
ESP_ERROR_CHECK(esp_event_loop_create_default());
esp_netif_create_default_wifi_sta();
wifi_init_config_t init = WIFI_INIT_CONFIG_DEFAULT();
ESP_ERROR_CHECK(esp_wifi_init(&init));
ESP_ERROR_CHECK(esp_event_handler_register(
WIFI_EVENT, ESP_EVENT_ANY_ID, wifi_event, nullptr));
ESP_ERROR_CHECK(esp_event_handler_register(
IP_EVENT, IP_EVENT_STA_GOT_IP, wifi_event, nullptr));
wifi_config_t config = {};
if (strlen(FREDDY_WIFI_SSID) >= sizeof(config.sta.ssid) ||
strlen(FREDDY_WIFI_PASSWORD) >= sizeof(config.sta.password)) {
ESP_LOGE(kTag, "Wi-Fi credentials exceed ESP32 limits");
return;
}
strncpy(reinterpret_cast<char *>(config.sta.ssid), FREDDY_WIFI_SSID,
sizeof(config.sta.ssid) - 1);
strncpy(reinterpret_cast<char *>(config.sta.password), FREDDY_WIFI_PASSWORD,
sizeof(config.sta.password) - 1);
ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA));
ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &config));
ESP_ERROR_CHECK(esp_wifi_start());
ESP_ERROR_CHECK(esp_wifi_connect());
httpd_config_t http_config = HTTPD_DEFAULT_CONFIG();
http_config.server_port = kPort;
http_config.stack_size = 8192;
http_config.lru_purge_enable = true;
ESP_ERROR_CHECK(httpd_start(&server, &http_config));
const httpd_uri_t info = {
.uri = "/ota/info", .method = HTTP_GET, .handler = info_handler,
.user_ctx = nullptr,
};
const httpd_uri_t upload = {
.uri = "/ota", .method = HTTP_POST, .handler = upload_handler,
.user_ctx = nullptr,
};
ESP_ERROR_CHECK(httpd_register_uri_handler(server, &info));
ESP_ERROR_CHECK(httpd_register_uri_handler(server, &upload));
}
void ota_service_confirm_healthy() {
const esp_err_t result = esp_ota_mark_app_valid_cancel_rollback();
if (result == ESP_OK) ESP_LOGI(kTag, "OTA_IMAGE_CONFIRMED");
}
@@ -0,0 +1,4 @@
#pragma once
void ota_service_start();
void ota_service_confirm_healthy();
@@ -0,0 +1,6 @@
# Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x4000
otadata, data, ota, 0xd000, 0x2000
phy_init, data, phy, 0xf000, 0x1000
ota_0, app, ota_0, 0x10000, 0x700000
ota_1, app, ota_1, 0x710000, 0x700000
1 # Name Type SubType Offset Size
2 nvs data nvs 0x9000 0x4000
3 otadata data ota 0xd000 0x2000
4 phy_init data phy 0xf000 0x1000
5 ota_0 app ota_0 0x10000 0x700000
6 ota_1 app ota_1 0x710000 0x700000
@@ -0,0 +1,11 @@
CONFIG_IDF_TARGET="esp32s3"
CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y
CONFIG_ESPTOOLPY_FLASHMODE_DIO=y
CONFIG_ESPTOOLPY_FLASHFREQ_80M=y
CONFIG_SPIRAM=y
CONFIG_SPIRAM_MODE_OCT=y
CONFIG_SPIRAM_SPEED_80M=y
CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y