Freddy-Fazbear-Digital-Eyes/hardware/bringup.md

Hardware bring-up log

Eye board 1 — 2026-09-21

• Connected to laptop through its USB port. The laptop enumerated /dev/cu.usbmodem1101 as an Espressif USB JTAG/serial device; this path may change on reconnect.
• A read-only esptool v5.2.0 flash-id query identified an ESP32-S3 revision v0.2, 8 MB embedded PSRAM, and 16 MB flash. Connection mode: USB-Serial/JTAG.
• At this initial identification step, firmware was not erased or flashed. The query reset the controller at completion; the later custom flash is recorded below.
• User reports the LCD displays an “Onboard parameters” screen and responds to touch. This matches the type of parameter display described for Waveshare's 01_LVGL_Arduino example, so the panel and touch input pass an initial visual check. The exact installed firmware is still unverified.
• Photos received: powered display and board back. The screen photo directly shows the factory-style “Onboard parameter” page, including 16 MB flash. The back silkscreen identifies ESP32-S3-Touch-LCD-2.1, Rev2.0. The connected left USB-C port is labeled USB; the separate right USB-C port is labeled UART. The 4-pin UART header and its RXD/TXD/3V3/GND labels are visible. Waveshare's catalog maps the printed non-B model to flat-panel SKU 28169; the numeric SKU is not printed on the board.
• The laptop has Arduino ESP32 core 3.3.5 installed, satisfying Waveshare's Arduino core requirement of 3.0.0 or newer. Waveshare's published 01_LVGL_Arduino example specifies LVGL 8.3.10.
• Update: Waveshare's LVGL_Arduino example compiled successfully with ESP32 core 3.3.5, bundled LVGL 8.3.10, 16 MB flash, and OPI PSRAM settings. The compile-only result was 1,431,075 bytes of program storage (45% of a 3 MB app partition) and 104,656 bytes of static RAM (31%). No firmware was uploaded. Build command and source provenance are recorded in vendor/README.md.

Eye board 2

• Connected to laptop in place of eye board 1 on 2026-09-21. It enumerated on /dev/cu.usbmodem1101, which is the port path reused after swapping devices.
• A read-only esptool v5.2.0 flash-id query identified an ESP32-S3 revision v0.2, 8 MB embedded PSRAM, and 16 MB flash. Connection mode: USB-Serial/JTAG. Its hardware address ends in D2:10, distinct from eye board 1 (CF:F8).
• At this initial identification step, firmware was not erased or flashed. The query reset the controller at completion; the later custom flash is recorded below.
• User reports the same “Onboard parameters” screen as eye board 1, and touch responds. Its LCD and touch input pass the initial visual check. Powered display and board back photos confirm the “Onboard parameter” page and the ESP32-S3-Touch-LCD-2.1, Rev2.0 silkscreen. The connected left USB-C port is labeled USB; the separate right one is UART. Waveshare's catalog maps this printed non-B model to flat-panel SKU 28169; the numeric SKU is not printed on the board.
• The same official LVGL_Arduino example and 16 MB flash/OPI PSRAM build settings apply. The example was compiled locally, not uploaded to either board; the observed screens are their preinstalled factory-style demo.

Next checks

1. Capture a photo or short bench recording and measure achieved frame rate for the renderer acceptance check.
2. Check the physical eye-socket clearance against the approximately 75 mm board assembly before designing the mount.

First custom flash — Board 1, 2026-09-21

• Source: eye test firmware with Waveshare display drivers and LVGL 8.3.10. It draws a blue moving iris, blinks every 4.6 seconds, and labels the screen BOARD 1 EYE TEST.
• Compile-only check passed: 1,123,899 bytes program storage and 96,404 bytes static RAM. Details and binary SHA-256 are in firmware/README.md.
• Before upload, /dev/cu.usbmodem1101 identified the board by hardware-address suffix CF:F8, matching Board 1. Upload at 460800 baud wrote all four standard images and verified each hash successfully; board reset afterward.
• Direct full-device backup failed with a serial stream error. The official Waveshare 16 MB stock image is saved under firmware/backups/ for recovery; it is not guaranteed identical to the image originally installed on this board.
• On-screen result: Joshua confirms the eyeball visibly looks around, blinks, and fits cleanly within the round display. Achieved frame rate and a post-flash photo or recording remain to be captured.

Second custom flash — Board 2, 2026-09-21

• Before upload, /dev/cu.usbmodem1101 again identified hardware-address suffix D2:10, matching Board 2.
• The same eye test was compiled with EYE_BOARD_NUMBER=2; the application binary contains BOARD 2 EYE TEST (SHA-256 27ab300069d7f48c82aacd50e667b9b4472260d2c197c382dcdad944e8fd80c7). Compile passed at 1,152,363 bytes program storage and 96,396 bytes static RAM.
• Upload at 460800 baud wrote the bootloader, partition table, boot-app selector, and eye application; esptool verified the hash of each written image and reset the board. Joshua confirmed the BOARD 2 EYE TEST label, eye movement, and clean blinking on screen.

Freddy iris test — Board 2, 2026-09-21

• Board 2 was confirmed again by hardware-address suffix D2:10 and flashed with the new build containing reference_images/official_freddy_eye_texture.webp as a cropped, transparent iris. All four written image hashes verified.
• Joshua confirmed the new iris texture looks good on the screen and continues to move and blink correctly. Board 1 has a compiled textured build but has not yet been reflashed.

Eyelid color update — Board 2, 2026-09-21

• The upper and lower blink lids were changed to #743824 at Joshua's request; the background remains #39271D.
• Board 2 (D2:10) was hash-verified after flashing the revised build. Joshua confirmed the new lid color looks good.
• Joshua then asked that the thin ring around the cream eye match the eyelids. The 444-pixel ring was changed from #27292A to #743824; Board 2 was reidentified, reflashed, and hash-verified. Joshua confirmed the ring matches the lids on screen. Board 1 has a compiled build with the same appearance, awaiting connection for flashing.

Matching appearance — Board 1, 2026-09-21

• Both boards were connected at once: Board 2 (D2:10) remained on /dev/cu.usbmodem1101; Board 1 (CF:F8) appeared on /dev/cu.usbmodem2101. Each identity was read before choosing the upload port.
• Board 1 was flashed with the textured iris and matching #743824 lids/ring. All four image hashes verified and the board reset. Joshua confirmed that both displays show the textured blue iris and matching brown lids/ring while moving and blinking.

Glowing-red preview — both boards, 2026-09-21

• The project now includes a red-eye state modeled on reference_images/freddy_glowing_red_pupils.png. It darkens the sclera and iris, pulses a small bright red center, and preserves gaze and blinking. This bench preview is toggled by tapping each screen; it is not yet controlled by person distance.
• Board 1 (CF:F8) and Board 2 (D2:10) were individually identified, flashed with their labeled variants, and all image hashes verified. Joshua confirmed both switch to the red state by touch while moving and blinking, then return to blue on a second tap.

Reference: Waveshare ESP32-S3-Touch-LCD-2.1 documentation.

DFRobot camera V1.1 — 2026-09-21

• Joshua connected the DFRobot DFR1154 and read the V1.1 board marking. /dev/cu.usbmodem1101 identified a distinct ESP32-S3, MAC E8:F6:0A:87:0B:4C, with 8 MB PSRAM and 16 MB flash; this port path may change on reconnect. DFRobot's board reference identifies the camera as an OV3660 and notes that the V1.1 Gravity + pin is a 3.3 V output.
• A five-second serial listen at 115200 baud produced no bytes from the installed application. The full 16 MB read-only backup failed after about 86 KB at 460800 baud. A retry at 115200 successfully saved the first 64 KB (firmware/backups/dfr1154-v11-flash-prefix-2026-09-21.bin) and the 1.25 MB app0 partition (firmware/backups/dfr1154-v11-app0-2026-09-21.bin). These are partial backups, not a full-device image. The app0 image passes esptool's checksum and validation-hash checks. Both files are local and ignored by Git.
• Camera bring-up firmware was compiled with Arduino ESP32 core 3.3.5: 387,343 bytes program storage and 32,428 bytes static RAM. Its camera pin map matches DFRobot's example. After verifying the MAC, the sketch was uploaded through /dev/cu.usbmodem1101; esptool verified hashes for the bootloader, partition table, boot-app selector, and application, then reset the board.
• The camera returned valid 320 × 240 JPEGs over USB serial. Initial frame showed the ceiling. After two physical aiming adjustments, the forward frame showed the intended room and walking area, with no visible corruption. The serial capture log records JPEG 6154 320 240 and DONE plus the image SHA-256. The room is dim, so person detection performance in this lighting remains untested.
• This proves camera initialization and frame capture. It does not yet prove person localization, tracking latency, or operation with the LiDAR attached.

DFRobot live person-position experiment — 2026-09-21

• ESP-IDF person-detector firmware built with ESP-IDF 5.5.2 and Espressif's pedestrian_detect 0.3.2 component. Its 2.66 MB application fits the custom 10 MB app partition. After confirming camera MAC E8:F6:0A:87:0B:4C, the firmware was uploaded and all flashed image hashes verified.
• An initial 20-second sample at the model's default 0.70 score threshold reported NO_PERSON with about 231–232 ms capture-to-result latency. See initial serial log. A person was not known to be in view for that sample.
• In a subsequent center-view test, the score threshold was 0.35, the room was brighter, and Joshua was visibly seated in the camera view. The matching JPEG shows him in front of the camera. The serial log contains 107 PERSON results, with median reported center x=139 in a 320-pixel-wide image (range 98–161), median score 0.606, and median capture-to-result latency 233 ms. This proves on-device person localization for this setup. Threshold, lighting, and target placement changed together, so the failed initial sample does not isolate which factor mattered.
• Held left/center/right tests now have paired serial logs and JPEGs. From Joshua's perspective, the median image x-coordinate was 214.5 at his left (log, image), 139 at center, and 62 at his right (log, image). The corresponding images visually confirm those locations; all three samples contain at least 48 detections. Left/right is mirrored because Joshua faced the camera. This satisfies the first live person-position check, but does not yet establish smooth tracking while walking.
• Multiple people, loss/reacquisition, and false-positive behavior are still unverified. The current firmware emits a diagnostic JPEG about every five seconds; remove that bench-only behavior before final use. A 3.3 V wired connection to the eye boards has not yet been tested.

Clear-room and infrared check — 2026-09-21

• With IR off and the camera aimed into an empty room, a 10-second sample had a few short false detections on furniture. The paired empty-room image confirms no person in that frame. A raw positive from the 0.35-threshold model is not yet safe to use directly as an eye target.
• DFRobot documents the onboard infrared illumination on GPIO47. The current camera firmware sets that pin high at boot; its flash was hash-verified. With the camera pointed very close to a surface, the IR-lit frame became a purple blur. After Joshua re-aimed it, a new IR-lit frame again showed the room. This visually establishes that the GPIO setting changes the image, but does not yet measure how much it improves person detection in darkness.
• The IR-lit empty-room log contains repeated apparent detections near the left image edge (often boxes approximately x=0–60), while the paired frame has no clear person there. A target-persistence and false-positive filter is required before sending gaze coordinates to the displays. The lights currently remain on in the camera firmware.