{"id":130,"date":"2026-06-26T19:21:20","date_gmt":"2026-06-26T19:21:20","guid":{"rendered":"https:\/\/oliverng.com\/ai\/?p=130"},"modified":"2026-07-14T19:02:58","modified_gmt":"2026-07-14T19:02:58","slug":"on-device-llms-in-ios-27","status":"publish","type":"post","link":"https:\/\/oliverng.com\/ai\/2026\/06\/26\/on-device-llms-in-ios-27\/","title":{"rendered":"On-device LLMs in iOS 27"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Apple posted a nice WWDC demo of how local LLMs will work on iOS27.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"WWDC26: Integrate on-device AI models into your app using Core AI | Apple\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/gl5lD2gEhb0?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">I&#8217;ve been building a lot of iOS apps, replacing all the main apps with ones that are super personalized. Apps I have full control over. One of them is a news aggregator super-app. I was growing tired of opening each news site&#8217;s app, and while it&#8217;s nice to get AI generated morning summaries of news, it was inflexible compared to what a native app can do &#8211; think video, audio, article refinements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I wanted my app to surface articles and topics that resonated with me and I knew that required a model to run calculations across topics of interest. While I landed on using <a href=\"https:\/\/developer.apple.com\/documentation\/naturallanguage\/nlembedding\">NLEmbeddings<\/a> to link up my interactions with my interests, embeddings are simple and vector based. I kept thinking how much more I could do with the benefits of a local LLM.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As I was searching on new possibilities, this demo video from WWDC26 came up on YouTube.  It is a language learning app. Point your camera at an object, get a Mandarin vocab card generated entirely on device. On device, open-source models are doing the LLM work. SAM 3 for image segmentation, Qwen 0.6B for language generation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The video answered a lot of my curiosity for how local models would work in iOS27.  I am pleasantly surprised:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>No server, no API costs, no data leaving the device.<\/strong>\u00a0Private by default, which is the straight up benefit of being local.  But it&#8217;s  cool that I can download an entire edge model on the device.<\/li>\n\n\n\n<li><strong>Ahead-of-time compilation.<\/strong>\u00a0Pre-compile on your dev machine so that users are not waiting for model loads.  If you&#8217;ve run a local model on Mac, you know the pain of waiting for a model to load into memory.<\/li>\n\n\n\n<li><strong>Background asset delivery.<\/strong>\u00a0Models download only if the user opts into the feature.  This is obvious given the space that models take up, but it is good knowing the SDK acknowledges it.<\/li>\n\n\n\n<li><strong>Same code runs on Mac with a larger model.<\/strong>\u00a0Swap Qwen 0.6B for 8B and you get better reasoning, longer context, etc.  Another nice touch, in doing a swap based on system capability.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">I&#8217;m excited to see the SDK for iOS27 and building for it in the near term. And maybe it&#8217;s just me, but I&#8217;m blown away seeing Qwen mentioned in a live demo, on an iPhone, from Apple.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Apple posted a nice WWDC demo of how local LLMs will work on iOS27. I&#8217;ve been building a lot of iOS apps, replacing all the main apps with ones that are super personalized. Apps I have full control over. One of them is a news aggregator super-app. I was growing tired of opening each news [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-130","post","type-post","status-publish","format-standard","hentry","category-experimenting"],"_links":{"self":[{"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/posts\/130","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/comments?post=130"}],"version-history":[{"count":6,"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/posts\/130\/revisions"}],"predecessor-version":[{"id":163,"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/posts\/130\/revisions\/163"}],"wp:attachment":[{"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/media?parent=130"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/categories?post=130"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/oliverng.com\/ai\/wp-json\/wp\/v2\/tags?post=130"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}