3.5 KiB
Pi-to-eye gaze link
gaze_link.py prepares the Raspberry Pi Zero 2 W to replace the DFRobot camera as the one gaze-message sender. It reproduces the ESP32 eye boards' existing v1 packet, including CRC and sequence wrap; the eye firmware does not need reflashing. The Pi now boots Raspberry Pi OS Lite and detects an IMX708 camera, but the person detector and camera adapter remain to be implemented and benchmarked. This module has been tested on the laptop only; it has not sent a command from the Pi to the physical eyes.
On the Pi, enable the hardware UART and disable its serial login console. After the 2026-10-01 reboot, /dev/serial0 exists, points to /dev/ttyS0, and the kernel command line no longer assigns a serial console. open_pi_uart() uses Python's standard library, so no python3-serial install is needed for the GPIO link. It sends normalized x, y, and valid values. Use GazeLink with an in-memory stream for development without driving the real eyes. The Pi GPIO14 TX/pin 8 goes to both eye GPIO44 RX pins and all three boards share ground. Disconnect the DFRobot camera TX first, and keep the Pi and ESP32 power rails separate. The eye UART USB-C connectors are a viable alternate path through a Pi USB hub, but they disable the eye UART headers while connected. See Pi trial plan.
Run the portable protocol check from this directory with python3 -m unittest test_gaze_link.py. It compares one encoded packet against a fixture generated by the existing C++ ESP32 protocol implementation. No camera or serial device is required for this check.
With the Pi UART enabled, the DFRobot TX wire disconnected, and both eye RX pins connected to Pi GPIO14, run python3 smoke_test_link.py. It sends left, center, and right gaze positions with the red tracking state for two seconds each, then returns both eyes to blue idle. On 2026-10-01 the script ran twice on the Pi, and Joshua visually confirmed both eyes matched throughout the second run.
For an initial full-body detector benchmark, install the Pi OS packages python3-picamera2 and python3-opencv, then run python3 benchmark_hog.py --duration 30 --rotation 180 with the current upside-down IMX708. It logs every frame's detection result and inference time under the project's diagnostics/ directory. This OpenCV HOG detector is a baseline to measure; it requires most of a person's body in view and has not been accepted as the final person tracker. It does not send commands to the eyes. Judge its speed and false positives before connecting a detector to the gaze link. See the Pi trial plan.
The first integrated trial is python3 run_hog_eye.py --duration 30 --rotation 180. It logs what it would send while leaving the eyes untouched. Add --drive-eyes for a timed hardware trial. It confirms a person after two detections and holds the last gaze through short misses. The first moving-person run on 2026-10-01 had a longest raw-detection gap of 3.34 seconds and Joshua saw the eyes briefly return to blue. Replaying that log showed a 3.5-second hold bridges all three observed red-to-blue drops, so the next trial uses 3.5 seconds. This also delays the return to blue after a person leaves and does not move the gaze during a detection gap. At the end it sends blue idle. This is a one-person proof of concept; continuous target tracking, multiple-target continuity, day/night testing, and false-positive evaluation remain.