Apple's iOS 27 brings major Visual Intelligence upgrades with Siri integration

At a glance:

• Visual Intelligence moves to Camera app with dedicated Siri Mode for real-time object analysis
• New features include nutritional insights, bill splitting (US only), and cross-platform support for iPad, Mac, and Vision Pro
• Requires iPhone 15 Pro or later, with Apple silicon chips needed for Mac and compatible iPads

Enhanced Visual Intelligence in iOS 27

Apple's Visual Intelligence is receiving a substantial overhaul in iOS 27, making the feature significantly more accessible and powerful than previous iterations. The company has relocated Visual Intelligence to the Camera app itself, integrating it through a new Siri Mode that sits alongside traditional Photo and Video capture options. Users can now simply swipe over to Siri Mode and take a picture, allowing the system to immediately analyze what they're viewing and provide relevant information.

The enhanced integration goes beyond basic identification, enabling users to ask follow-up questions about captured objects or scenes. This represents a fundamental shift from earlier versions where complex queries were redirected to ChatGPT. Now, Siri's improved AI capabilities can handle sophisticated requests by pulling information directly from the web, creating a more seamless user experience without external dependencies.

New capabilities and practical applications

Among the most notable additions are several everyday utility features that demonstrate Apple's focus on practical AI applications. Nutritional Insights allows users to photograph meals or food items and receive general nutritional assessments, though the system notably avoids providing specific calorie counts. For US users specifically, Bill Splitting functionality enables photographing restaurant bills to calculate individual shares and facilitate payments through Apple Cash.

The update introduces several productivity-focused features including Import Multiple Events for batch calendar additions, Import Contacts from business card photos, and Wallet Passes creation from barcode images. These capabilities extend Visual Intelligence beyond simple identification into actionable assistance, streamlining common tasks that previously required manual data entry or separate applications.

Beyond these new features, Visual Intelligence maintains its comprehensive object recognition capabilities. The system can identify plants, animals, insects, landmarks, art, sculptures, and books while offering practical utilities like reading laundry labels, deciphering parking signs, scaling recipes, checking ingredient lists for allergens, and transcribing handwritten notes. It can also search the web for product reviews, identify obscure devices or cables, and solve mathematical problems from worksheets.

Cross-platform expansion and device requirements

Apple has significantly expanded Visual Intelligence availability across its ecosystem, bringing the feature to iPadOS 27, macOS Golden Gate, and visionOS 27. On Mac systems, users can access Visual Intelligence via Command-Shift-Space keyboard shortcut followed by display area selection. iPad users have multiple access methods including screenshot capture or Apple Pencil gestures from the lower left corner.

Vision Pro users benefit from the most intuitive implementation, simply looking at objects to trigger Visual Intelligence analysis of both digital and physical items. However, hardware requirements remain restrictive: iPhone 15 Pro or later for iOS, Apple silicon chips for Mac compatibility, and either Apple silicon or A17 Pro chip in iPad mini for tablet support.

Siri app integration and conversation management

All Visual Intelligence interactions through Siri are now logged within a dedicated Siri app, providing users with conversation history and the ability to revisit previous queries. The app offers flexible retention settings, allowing conversations to be preserved for one month, one year, or indefinitely based on user preferences. This integration reinforces Apple's emphasis on privacy-conscious AI implementation while maintaining utility.

Strategic implications and market positioning

These enhancements reflect Apple's broader strategy of embedding AI capabilities deeply into core operating system functions rather than treating them as separate applications. By moving Visual Intelligence into the Camera app and eliminating the need for ChatGPT handoffs, Apple strengthens its ecosystem lock-in while positioning Siri as a more capable standalone assistant. The feature expansion across multiple platforms suggests Apple views Visual Intelligence as a cornerstone capability for its future software development.

The practical focus of new features—nutrition tracking, bill splitting, contact management—demonstrates Apple's approach to AI development centered on solving real-world problems rather than showcasing technical capabilities. This consumer-first strategy may help drive adoption among mainstream users who might otherwise find AI features abstract or unnecessary.

Looking ahead

With these Visual Intelligence improvements, Apple continues building toward its vision of ambient computing where devices proactively assist users based on context and visual input. The cross-platform rollout suggests similar capabilities will become standard across all Apple devices, potentially influencing how competitors approach visual AI integration in their own ecosystems.

The requirement for newer hardware ensures that these advanced features will primarily be available to users who have recently invested in Apple's latest devices, aligning feature availability with revenue generation cycles.