Show HN: Eyesite - experimental website combining computer vision and web design

Summary

I wanted Apple Vision Pros, but I don’t have $3,500 in my back pocket. So I made Apple Vision Pros at home. I was interested in making a project that combined computer vision with web design—a website that users could physically interact with. This inspired me to make Eyesite, because who needs a mouse when you have your eyes? Eye tracking Luckily, there is already a Javascript library for eye tracking called WebGazer.js. We can achieve decent eye tracking through calibration: Make the user look at a point and click. This maps the current gaze to a point on the screen. Feed the gaze/coordinate mapping into WebGazer to calibrate. Repeat 9x times on the corners, sides, and center to get good mapping data. I found that it was best to get 5 mappings per point for better eye tracking accuracy. Calibration in debug mode. The top right shows how WebGazer tracks your eyes and face. The red dot is where it thinks I’m looking. Website Interaction Now that we have eye tracking, we can make some cool things with it! I decided to use the user’s gaze as a mouse and have them click with spacebar—kind of like how Apple Vision Pros have you look and pinch. Although I had the main functionality, it was far from finished. There were many considerations with making the experience as smooth and immersive as possible. The “Invisible” Mouse Initially, the user could see “where” they were looking at through a red dot.Main page in debug mode. This created some problems. First, the red dot was distracting, and users would unconsciously look at it instead of my buttons. Second, the red dot revealed how inaccurate the eye tracking was, which ruined the immersion. Ultimately, I decided to remove the “eye cursor” and also make the user’s mouse invisible. It made you really feel like you were controlling the website with your eyes rather than moving a mouse around. You can turn on debug mode to see your eye cursor and mouse. User feedback Since we don’t have a mouse, we need some way for the user...