Apple unveils new iOS 18 accessibility features including Eye Tracking and Live Captions for Vision Pro
Apple has today announced several new accessibility features coming to iPhone, iPad, and Apple Vision Pro later this year.
The company has unveiled new Eye Tracking, Music Haptics, Vocal Shortcuts, Vehicle Motion Cues, and more. Apple says the features are coming “later this year” which means we can expect them to debut as part of iOS 18, which is set to be unveiled in just a few weeks at WWDC 2024.
Apple’s new Eye Tracking is a feature for iPhone and iPad powered by AI. Apple says it will give users a built-in option for navigating either device using just their eyes, monitored by the device’s front-facing camera with no additional hardware or accessories. It will even feature support for swipes and gestures.
More accessibility options come to iPhone and beyond
Apple will also deliver a new Music Haptics feature that plays taps, textures, and “refined vibrations” to help deaf or hard-of-hearing users better experience music on their iPhone. The feature will work “across millions of songs” on Apple Music.
The company has also announced new Vocal Shortcuts, which use Siri to launch shortcuts and complete complex tasks. Another new accessibility feature can use AI to listen for Atypical Speech, which will improve speech recognition on iPhone even for users with conditions that can impact speech such as ALS or cerebral palsy.
New Vehicle Motion Cues will reduce motion sickness for passengers in moving vehicles by using animated dots on the edges of your screen that “represent changes in vehicle motion to help reduce sensory conflict without interfering with the main content.” The dots move in sync with the vehicle as it makes turns.
Other vehicle-based improvements are CarPlay Voice Control, Color Filters, and Sound Recognition. Users will be able to navigate CarPlay with just their voice, and Sound Recognition will notify deaf or hard-of-hearing drivers of car horns or sirens, too.
Another significant update comes to Apple Vision Pro with systemwide Live Captions to help users follow along with spoken dialog in live conversations and from the audio in apps. Apple also announced a slew of other accessibility updates as follows:
- For users who are blind or have low vision, VoiceOver will include new voices, a flexible Voice Rotor, custom volume control, and the ability to customize VoiceOver keyboard shortcuts on Mac.
- Magnifier will offer a new Reader Mode and the option to easily launch Detection Mode with the Action button.
- Braille users will get a new way to start and stay in Braille Screen Input for faster control and text editing; Japanese language availability for Braille Screen Input; support for multi-line braille with Dot Pad; and the option to choose different input and output tables.
- For users with low vision, Hover Typing shows larger text when typing in a text field, and in a user’s preferred font and color.
- For users at risk of losing their ability to speak, Personal Voice will be available in Mandarin Chinese. Users who have difficulty pronouncing or reading full sentences will be able to create a Personal Voice using shortened phrases.
- For users who are nonspeaking, Live Speech will include categories and simultaneous compatibility with Live Captions.
- For users with physical disabilities, Virtual Trackpad for AssistiveTouch allows users to control their device using a small region of the screen as a resizable trackpad.
- Switch Control will include the option to use the cameras in iPhone and iPad to recognize finger-tap gestures as switches.
- Voice Control will offer support for custom vocabularies and complex words.
More from iMore
