On-Device or Cloud AI in iOS: Foundation Models vs the OpenAI API

Ask yourself where the intelligence in your app actually lives. That question matters more than "should we add AI?", and it's the one that changes how you design an iOS feature. Swapping a cloud call for Apple's on-device Foundation Models framework in a side project makes the point fast: you were never really choosing between two models, you were choosing between two places to run one.

Location beats benchmarks

Most comparisons argue over model scores. For an iPhone app, the more useful question is whether inference happens on the user's device or on somebody else's server.

That single decision leaks into everything else you ship: privacy promises, latency, running costs, offline behavior, even what App Store review will ask you about. Pick the location first and a lot of other choices stop being debates.

On-device with Foundation Models

Apple's framework gives you a language model that runs locally on Apple Silicon, using the Neural Engine. There's no network request, no API key to protect and no per-token invoice at the end of the month.

A minimal session looks like this:

import FoundationModels

// Create a session with the on-device model
let session = LanguageModelSession()

// Ask a simple question — runs entirely on-device
let response = try await session.respond(
    to: "Summarize this note in one sentence: \(userNote)"
)

print(response.content)

What you get in return for the smaller model:

• No network round trip, so it keeps working on a plane or in a subway tunnel.
• Nothing scales with your user count, because you aren't paying a provider per call.
• User text stays on the phone, which is a big deal for health, journaling and finance apps.
• It plugs straight into async/await and SwiftUI, so it feels like any other system framework.

The trade-off is size. This model is smaller and less capable than the GPT-4 class of cloud models. It's very good at summarizing, classifying and light reasoning. It's poor at long multi-step generation and anything that needs agent-style behavior, so don't build those features on it.

Cloud with the OpenAI API

The other side of the choice is frontier-level reasoning, paid for with a network dependency. Your feature now needs connectivity, a key, and a bill.

Here's the same kind of call made over HTTP:

// Basic async/await network call to OpenAI's API
func fetchCompletion(prompt: String) async throws -> String {
    var request = URLRequest(url: URL(string: "https://api.openai.com/v1/chat/completions")!)
    request.httpMethod = "POST"
    request.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")
    request.setValue("application/json", forHTTPHeaderField: "Content-Type")

    let body: [String: Any] = [
        "model": "gpt-4o",
        "messages": [["role": "user", "content": prompt]]
    ]
    request.httpBody = try JSONSerialization.data(withJSONObject: body)

    let (data, _) = try await URLSession.shared.data(for: request)
    // Parse response into your own model here
    return String(data: data, encoding: .utf8) ?? ""
}

I'd reach for this when output quality is the product. Say you're building a research assistant: the reasoning you get from a hosted model simply isn't something a small local model can match, so the network cost is worth it.

Use both

In production you rarely have to pick one. Split the work by what each task needs, and keep that decision out of your views, for example by putting it behind your MVVM layer so a view model asks for a result and never cares where it came from.

• Run on-device: autocomplete, tagging, quick summaries, and an offline fallback.
• Send to the cloud: deep, long-form generation and anything that needs a lot of context.

That arrangement keeps the app snappy and cheap for roughly 80% of interactions, and you only pay for a cloud call when the task justifies it. It pairs nicely with SwiftData too: cache results locally and repeat requests never hit the network at all.

Quick way to decide

• Offline support or strict data privacy is a requirement: use Foundation Models.
• You need top-tier reasoning or a huge context window: use the OpenAI API.
• You want to scale without wrecking the budget: go hybrid and make on-device the default path.
• You're prepping for an iOS interview question on AI architecture: explain both and why you'd pick each, since that's the answer interviewers are after.

Neither option wins outright. Foundation Models is now the sensible default for fast, private, cheap features, and heavy lifting still goes through a hosted API. By 2026 the better iOS architecture isn't about choosing a side. It's about knowing exactly which work deserves a network round trip and which doesn't.