Matt Moss, a 19-year-old computer science major at the University of California, Santa Barbara, has enjoyed working on games and social apps since high school. But he says that none of those projects has been as impactful as his latest, Hawkeye Access, an iOS app that enables users to control their iPhones and iPads with their eyes.
"It's good to bring people happiness from making a fun game, but it's not going to fundamentally change their lives," Moss said. "But I think accessibility through eye tracking has the ability to fundamentally improve someone's life. If you can't communicate easily or control your device easily, then being able to do so is really empowering."
Moss began work on the project back in June after he was awarded a student scholarship to attend the Worldwide Developers Conference (WWDC), Apple's annual meetup for software developers to learn about and work on the company's latest tech. Learning of Apple's ARKit 2's advanced face-tracking features would spark a question that he just had to answer: "My thought was, 'Could you use some of that data to find out where the user is looking on the screen?'"
Within three to four days, he had made it work and demoed it in a video that he put together and posted on Twitter (iOS , Android). Interest came quickly, and he decided to pursue it further. Maybe there'd be some interesting applications to be made with this kind of technology, Moss wondered. Perhaps increased accessibility for quadriplegics or anyone without the use of their limbs would be a good approach.
"Accessibility is one of the coolest ways of using eye tracking," Moss said. "For someone who can't necessarily touch the screen, being able to just look at different parts of the screen to control their device is really empowering, and I really liked the idea of working on a project like that. It's not all projects that get to affect people as much as something like that."
Officially launched a month ago for the new iPhone X models and more recently for the new iPad Pro, Hawkeye Access relies on these devices' TrueDepth front-facing camera systems -- which can capture a 3D model of the user's face and head -- and underlying ARKit 2 tech, to see where they're looking on the screen, so there's no need to direct movement with clicks and scrolls. In short, users can now browse any website, hands-free, all through eye movements.
Hawkeye Access is one of the easiest to use gaze control systems around, and it requires no additional hardware -- such as a costly eye-tracking rig -- which makes this technology accessible to even more people, which the accessibility community told Moss was a benefit.
"Twitter has been really great for connecting with people from the accessibility community, so I've heard how impacting it is to have this kind of technology available on the iPad, in particular, with its bigger screen," he said. "The expensive rig really wasn't that accessible for a lot of people, and if all you had to do is get an iPad, and it magically works, that's really powerful."
FOLLOW Download.com on Twitter for all the latest app news.
- Matt Moss, a 19-year-old computer science major at the University of California, Santa Barbara, developed Hawkeye Access, an iOS app that enables users to control their iPhones and iPads with their eyes.
- Hawkeye Access relies on the latest iPhones and iPads' TrueDepth front-facing camera systems -- which can capture a 3D model of the user's face and head -- and underlying ARKit 2 tech, to see and track where they're looking on the screen.
Check out the Hawkeye Access app in action below:
Instagram now helps visually impaired people use the app with the help of AI
Google brings Lens to mobile web to help you get more info straight from photos
HeadGaze app lets users with disabilities navigate with simple head movements
Disability community on a mission to build better maps (CNET)
Microsoft using AI to empower people living with disabilities (ZDNet)
Despite its promise, modern technology often fails to help disabled users (TechRepublic)