WWDC 2024 sets correct tone for generative AI in an under-confident way 🍏

Following are key highlights from WWDC 2024 keynote & my view around Apple Intelligence at the end.

🥽 VisionOS 2

• AI based Spatial Photos
• Travel Mode support for Vision
• New Frameworks/APIs:
  • Volumetric APIs
  • TableTopKits – Cardboard kinds of experiences
  • Enterprise APIs – For Healthcare & Manufacturing industrial applications
  • Spatial Video – Phone to Vision!
  • Vimeo app for Vision OS
  • Apple Immersive Videos – BlackMagicDesign for Creators, first immersive movie?
• Rolling out Apple Vision to more countries. India NOT yet in the waiting list.

📱iOS 18

• Dark mode for app icons
• Tint colour for icons in dark mode
• Control Center customisation, extended use cases.
• Control Center API
• New controls can be used from buttons and Lock Screen actions
• More Privacy by locking apps and even hiding apps.
• Messages:
  • Tapbacks: Tapback with any Emoji
  • Schedule messages
  • Text formatting
  • Text animation effects
  • Messages via Satellite – END TO END ENCRYPTED
• Mail:
  • Auto-Categorisation for emails.
• Maps:
  • More in map details
• Wallet:
  • Tap to cash – hold devices close and tap to pay
  • New design
• Gaming
  • Game mode coming to iPhones
• Photos:
  • More organisation – PHOTO GRID
  • Options to filter and access easily – Time, People, Trip/Location.
  • Photo Carousel

🔊 Audio & Home

• Siri Interactions to communicate better with help of AirPods.
• Voice Isolation to AirPods Pro, to listen clearly.
• Personalised Spatial Audio API, available for gaming as well.
• InSight of Songs on iPhone and AppleTV for realtime info about songs being played.
• 21:9 format for projectors
• Screensaver animations for AppleTV

⌚️ WatchOS 11

• Training Load
• Age, Weight and Effort Rating will be used to calculate Training load.
• Customise Summary
• Change Rings based on Weekday
• New App for Health: Vitals app checks all health metrics and provides health alerts along with an explanation.
• Cycle Tracking also respects pregnancy considerations.
• Smart stack widgets now will be auto suggested. AI to the work.
• DoubleTap API to track from within app.
• Photo recommendation using AI and styles for your Watch face.

✍️ IPadOS 18

• New sizes
• New accessories
• Control Center & Icon Customisations like iOS
• Floating Tab-bar, consistent UI for same across all apps. API for same.
• SharePlay:
  • Tap and draw on screen to assist.
  • Remote control for iPad
• Freeform
  • Section selection
• Calculator now in iPad, History, Unit Conversions, using Pencil for Math Notes. Scientific Calculator that works realtime from your pencil notes. This one is by far the best feature and demo of the day.
• Math Notes work in Notes too.
• Smart Scripts: Notes refine your handwriting appearance becomes beautiful.
• Spell checks work inline, right in your handwriting.
• Scratch to erase.
• For me this is a real clean use of AI, without letting users know its AI and without interfering in real Human Creativity.

💻 MacOS Sequoia

• New Name – MacOS Sequoia
• Messages – send later, emoji tapback.
• Maps – plan hikes.
• Math notes in notes app as well.
• More new features.
• Continuity – Now with iPhone mirroring + remote controlling your phone from Mac.
• iPhone notifications to Mac. Click Notification on Mac -> Back to Phone Mirroring. Supports Audio too.
• Phone stays locked during mirroring. F! That’s a bang on experience.
• File Drag-Drop during Mirroring session.
• Tile position recommendation for multi-window productivity.
• Keychain advancements:
  • Passwords app: Easy management experience – Clean UI
  • Across all devices
• Safari:
  • World’s fastest browser on Sequoia
  • Private browsing, actually hide your activities from Websites.
  • AI for highlighting and summarising website data on fly.
  • Viewer experience: Video is front centred when a video plays in focus.
• Gaming:
  • Game porting toolkit 2, enables advanced games to developers.
  • Ubisoft: Assassin’s Creed next chapter and Prince of Persia coming to Mac. Full power of Silicon chip + Metal. Assassin’s Creed will also come to iPad? More power to Silicon.

⏳ Timelines

• BETAs available today.
• Public API will be available next month.
• New OS versions will be available from this fall.

🪄 Apple Intelligence

Apple prefers Personal Intelligence over Artificial Intelligence. Clean proposition – Apple Intelligence!

• Personal intelligence AI models embedded right on your device. Protects privacy at every step.
• Capabilities:
  • iPhone, iPad mad iMac will build Languages, Images and Actions as per your personal context.
  • Available across all devices, apps and experiences to build personalised experience.
  • And here drops the dumb IMAGE generations from AI. No matter what brand, no one understand the importance of hand drawn or camera clicked real life pictures. It’s clear today to me. AI images stand ZERO value to me. Makes internet boring.
  • Ok, now I am getting the same instinct of accessing all PERSONAL DATA in name of personalisation. Device in a wrong hand, will exposes all personal context?
  • Good thing loading: Privacy – On-device processing, data goes nowhere and remains private to your device. Silicon the to power.
  • Private Cloud Compute – For more complex compute demands made by Apple and Silicon chips. Apple OWNED AI POWERHOUSE it is.
  • Verifiable Privacy on Private Cloud Compute. Indeed, Groundbreaking. Someone please verify the verifiable though.

Siri

• More natural, relevant and personal.
• Continued conversation context.
• Type to SIRI for privacy.
• Apple Documentation now is a Siri experience.
• On screen commands to take actions.
• App intents are improved for same. App Intents API improved.
• Personal Context, now help you get done more. For example: Find photo, extract and fill a form using one voice command.
• These changes will be available in Mac, iPhone and iPad.

Writing tools

• Rewrite to improve your written work. TONE, FORMAT and COPY STYLE
• Proof Reading
• Summarise
• Smart Reply
• Available everywhere you write in the ecosystem.

Inbox

• Based on the contents in emails, inbox shows a summarised view for saving time. No need to open emails everytime.

Notifications

• Summarise & show you priority notifications.
• Reduce interruptions mode silents your notifications based on content.

Other areas

• Third party apps can also benefit from Apple Intelligence.
• Genmoji, create a emoji using Apple Intelligence. It’s a good use of AI image generation.
• Image Playground, Nah! Boring OR am I the only who hates AI images.
• Image Playground is available as API, an independent app & in most social interaction experience of the eco-system.
• Image Wand in notes app, improves images sketched on iPad to convert to images. Again not a big fan of AI images.
• Photo editing, provides cleanup tool – Again not a big fan. Promotes TOXIC PERFECTNESS in the world!
• Finding a particular moment in a video, can be real time saving.
• Recording and Call Summaries for Video calls and Phone Calls.
• Apple Intelligence is coming for FREE iPadOS 18, iPhoneOS 18 & MacOS Sequoia. That a +10. FREE and Secured!
• Siri can be allowed to use ChatGPT for answering user queries. Though it’s FREE and NO account is needed, I am not a fan of this feature considering what Apple is bringing in the plate with Apple Intelligence. The proposition is strong and clean.

Conclusion

Apple still has that Steve-Vision to look through the eyes of customer experiences. Free Apple Intelligence, keeping it On-Device, Silicon first, for personalisation is the expected direction. But, then the AI Image Generation, ChatGPT integration shows some under-confidence in this vision from the Apple team side. I would focus less on those toys.

Watch full keynote here: https://www.youtube.com/watch?v=RXeOiIDNNek