AI is coming to your iPhone, iPad, and Mac this year, and there’s an awful lot to cover.
Apple Intelligence differentiates itself from the competition by trying to keep as much on-device as possible while focusing on privacy when data does go to the cloud. And, as leaked a while back, OpenAI has partnered with Apple, too.
Here’s everything we just learned about Apple Intelligence on iOS 18, iPadOS 18, and macOS Sequoia.
ChatGPT integration
Arriving late in the presentation, Apple confirmed it’s partnering with OpenAI for ChatGPT 4.0 functionality.
Apple says you’ll be notified when data needs to be sent off-device, and that ChatGPT 4.0 will be able to process images you share with it, too, letting you ask questions about documents. You’ll also be able to have it write to a brief for you.
Apple says “You control when ChatGPT is used and will be asked before any of your information is shared.”
“Anyone can access ChatGPT for free, without creating an account. ChatGPT subscribers can connect accounts to access paid features within these experiences.”
Siri
Perhaps the biggest beneficiary of the AI features rolled out as part of WWDC 2024 is Siri, which will now be much smarter – and sports a lovely new icon that you can see above.
Siri will now let users move between text and voice requests, with much better awareness of the current context. It’ll also be able to remember things you’ve requested, so that when you chain requests together you won’t need to start over from scratch.
Siri’s functionality has also changed, so there’s no more screen overlay. Instead, it’ll highlight your screen with a pinkish glow when activated, and if it can return a request without opening an app, it’ll do so with a smaller window that slides in from the top of the screen.
Siri will also let you change a request, so you no longer have to cancel a timer to reset it because you got the increment wrong.
Genmoji
We’re seemingly past Animoji, now, as Apple revealed Generative AI emoji it calls Genmoji.
Ensuring you always have an image that you can send for any given situation, these can appear in Sketched, Illustrated, or Animated versions, and you can use the new Image Playground app to create to your heart’s content.
They can even use data from Photos as inspiration, but Apple insists this processing is all done on-device. They can be generated right from the keyboard, too, and developers will be able to work with a new Image Playground API.
Photos
Not only has the Photos app been redesigned, but it’s also getting natural language support for search – so you can ask for “cat laying on the sofa” and it’ll find it for you.
The app will also allow you to edit images with a Siri request like “make this image ‘pop’ more”, while you can now finally use Clean Up to remove an unwanted bystander from your family photo.
You can also create new “Memory Movies” by entering a description. Apple’s example is “all the desserts I ate during the summer last year, set to a pop song”, which highlighted a whole host of sweet treats and ties in with Apple Music.
Notes
The Notes app is getting AI features for both text and images. For one, you’ll be able to solve mathematic equations with on the fly, while also being able to dump text from another app, or generated by Siri, right in there.
For more visual thinkers, they’ll be able to convert rough sketches into AI generated art that can pull context from the space around it. You can also generate one from scratch if you’re not artistically gifted.
Apple Intelligence will rewrite your emails to fit a different style once you prompt it, letting you amend your text to sound more professional, or convert it to a poem, if that’s what you’re looking for.
It’ll also summarize emails to help you catch up on what’s in your inbox more quickly and move priority emails to the top of your inbox automatically. The menu looks a fair bit like Notion’s options, which is interesting.
Smart Reply will let you quickly respond to emails via prompts, such as confirming attendance to an event using a simple “Yes” or “No” and having the email written by Apple Intelligence.
Notifications
A new Reduce Interruptions Focus Mode will ensure all of your devices only show priority notifications, such as time-sensitive ones or those from selected contacts.
They appear at the top of your notifications stack, but they’re also summarized so you can quickly glance at them to know what’s going on.
Phone
You’ll be able to capture audio recordings within the Phone app, and these will automatically transcribe and have a summary generated.
And don’t worry – everyone on the call will get an alert to say it’s being recorded.
Privacy
The “elephant in the room” with AI is Apple’s steadfast commitment to privacy, and the company is promising that user data and requests aren’t stored. Your data is only used for your requests, and as many tasks as possible are handled on-device.
For anything that requires processing by the cloud, Apple uses Private Cloud Compute, which are server-based Apple Silicon machines to power through complex requests.
What devices are getting new AI functionality?
As was rumored before the event, many of these features will require the latest iPhone 15 Pro’s A17 Pro.
For Mac and iPad, you’ll need an M-Series chip inside, but it does go right back to M1. You can reasonably expect newer models to handle these complex tasks more efficiently, though.
But wait, what about Machine Learning?
For years, Apple has used the term Machine Learning to describe its own AI technology framework. It’s been working on it for quite some time; Machine Learning is what highlights the Siri Suggestions when you look to use an app or learns the way you type on your iPhone, among others.
Why the company has changed to more plainly refer to it as AI is unclear, but there’s every chance the shift in semantics is down to just how pervasive the term “AI” has become over the last 18 months or so.
As if to underline its seriousness, Apple acquired DarwinAI in 2024, with the startup’s staff joining Apple’s AI effort. DarwinAI is a “visual quality inspection” company based out of Waterloo, Ontario. It has a patented Explainable AI platform that has been adopted by “numerous” Fortune 500 companies. “DarwinAI has developed AI technology for visually inspecting components during the manufacturing process and serves customers in a range of industries,” Mark Gurman reported.
“One of its core technologies is making artificial intelligence systems smaller and faster. That work could be helpful to Apple, which is focused on running AI on devices rather than entirely in the cloud.” According to Betakit, DarwinAI’s platform was built “under the guidance of University of Waterloo professor and Canada AI research chair Alexander Wong,” who is now employed by Apple as Director of Machine Learning research. The company was founded in 2017 and has been helmed by CEO Sheldon Fernandez since January 2018.
What’s going on with Apple and Google?
While Apple is making its own acquisitions to fuel its AI ambitions (and reportedly moving employees over from the canceled Apple Car project) it’s also reportedly been in discussions with Google.
According to Bloomberg, Apple and Google have been working on a “blockbuster agreement that would shake up the AI industry”, with the same reports suggesting Apple is in “active negotiations” to license Google’s Gemini AI tools.
This has been corroborated since by the New York Times, which seems to have all but confirmed the two tech giants working together.
“Apple is in discussions with Google about using the search giant’s generative artificial intelligence model called Gemini for its next iPhone, as the company races to embrace a technology that has upended the tech industry,” the New York Times wrote, noting the “exact scope of a potential deal hasn’t been defined”.
At WWDC, Apple confirmed its deal with ChatGPT isn’t exclusive, and mentioned that Google Gemini could also feature on iPhone at a later date.
A handy AI glossary
Below you’ll find a few key terms that you’re likely to see a lot in the coming weeks and months, as well as an explainer of each.
- Chatbot – Used in everything from AI technology to customer service, a chatbot is software that mimics human conversation.
- Generative AI – Technology that can create content, be that images, audio, video, or anything else, by receiving prompts from a user.
- LLM (Large Language Model) – In layman’s terms, a computer analyzing human speech and text to respond as a human.
- Machine Learning – Apple’s chosen branding of AI for many years, Machine Learning finds patterns to recommend your next app, or searches your images for pictures of a dog.
