Category: Models

The past month of Claude Opus 5 has been a disaster. It was a problematic release because I straight up could not understand it. If you’ve been using Opus 5 you know what I’m talking about. I felt like I spent more time understanding its replies than doing actual work. It was slow, made mistakes more often than I noticed, and seemed to get into these doom loops when reasoning to the point where I had to set effort to medium/low. I had never changed effort levels prior to Opus 5.

It was so bad, that I switched to OpenAI’s Sol for a month to see how well things were going on the other side of the fence. Sol and Astra are good. It’s quick. It’s concise. And the app does this thing, where all the reasoning gets collapsed after its turn to provide a short summary you can quickly read to understand what was accomplished. I hadn’t used ChatGPT for over 4 months, not having tested Astra or Sol lately.

I was impressed by the changes OpenAI has made to the harness, the desktop/mobile apps, and the models. It is very competitive with Claude. I’m still iffy on the security model for Codex, which leans way more permissive than I’m used to, but every harness is leaning towards autonomy over security of late. I really like how Sol talks and does single-threaded work. So just as I was about to buy a month of ChatGPT, Claude released Opus 5.5.

Opus 5.5 addresses most of this feedback. It is noticeably quicker. I’m not holding my breath every time I prompt the model, scared it will doom loop. The way Claude communicates is improved. I would say it is just back to what it was before, and not whatever 5 was doing. Honestly, it was so bad that I think most people will be happy to be back to that baseline. I still think Astra and Sol 6 are better at clear, concise communication. I’ve also noticed an improvement with code quality for Opus 5.5. I’m not sure if I’m just imagining things but even running effort medium, I’m not hitting as many fixes during my /code-review sessions.

I’m going to switch over to Astra for a bit. Fable is not accessible on base plans and Astra 6 is accessible with ChatGPT Plus. I’ve been moving between Claude Pro/Max depending on my work but on Pro I can’t use Fable without API credits. I wonder if I will pull my hair out moving my scheduled tasks and skills over to ChatGPT.

I’m playing around with Local LLMs again! Using Gemma 4 12b on my underpowered Mac M4 and it is good. Much better than when I first played with Gemma 4 E4B. The tool calling is so much better and the general day to day feels crisp. I’m not doing any benchmarks, just running my life.

A lot of r/LocalLLM chatter is because it is the first model to run on laptop class 16GB RAM, the likes of MacBook Pros with unified memory. And it does! Barely. Running in oMLX cost me 11GB. Was like living in 640K memory in the DOS days. Closing everything in activity monitor. A more realistic scenario is using the GGUF 8GB which is survivable. Anyway, point is, if we were not in 2026, living memory poor, one can easily see a future where laptops shipped with local LLMs doing real work, real tool calls, all privately….with more RAM.

Exploring this further I wanted to see what was new in the 12B architecture and went down a small rabbit hole about why Google released 12B and its architecture in their developer post.

Gemma 4 12B introduces several milestones for local AI:

1. A multimodal encoder-free architecture: Bypassing heavy multi-stage vision and audio encoders entirely, multimodal data is fed straight into the LLM backbone, reducing multimodal latency.
2. Our first medium-sized model with audio input: In the Gemma family, audio inputs were restricted to small, lightweight edge architectures (e.g. E4B). Gemma 4 12B is the first medium-sized model capable of natively ingesting audio.

Gemma4 being encoder free was a new one to me. On their developer post they had this image which only confused me more.

Thankfully, they also included a link to Maarten Grootendorst’s post A Visual Guide to Gemma 4 12B which explains it eloquently. Every few month there are new optimizations and ideas on how to fit more into less.

With Nvidia announcing the RTX Spark to challenge Apple’s MX reign on local LLMs, it is appropriate that Google dumped some toys on us the next day.

Gemma4 12B is out today, in a sweet spot between their prior 8B and 26B I’m super interested to see how it stands up. 8B was a great chatbot, but a terrible tool bot. 26B seems much better but I could never run it on my M4 Mac mini 16GB. Does Google see 12B is “about right” for most spec’d out systems today – it runs on 16GB! Downloading now….

Unexpectedly, Google added to LiteRT-LM a local LLM server / CLI!

The LiteRT-LM CLI provides a lightweight, zero-code tool for running language models locally. We are now expanding the tool with the serve command, letting the CLI act as a drop-in local LLM server. Use this functionality with Gemma 4 12B to point any standard tool, SDK, or framework (such as OpenClaw, Hermes, OpenCode, Pi, or popular extensions like Continue and Aider) directly to your local endpoint.

Aside from the fact that the internets are saying RTX spark is going to cost $5000+ it does not seem far away where local models, even on edge devices (looking at you iOS27),may start to be competitive with cloud for basic stuff. Not coding, not tool use, not yet. But everyday things, like voice typing, opening apps, regular analysis, asking offline questions, could be done in the very near future all on local LLM.

New Opus 4.8 came out. I saw the video announcement on YouTube but what wasn’t captured was this gem in the release notes.

One of the most prominent improvements in Opus 4.8 is its honesty. We train all our models to be honest—for instance, to avoid making claims that they can’t support. But a general problem with AI models is that they sometimes jump to conclusions, confidently claiming to have made progress in their work despite the evidence being thin. Early testers report that Opus 4.8 is more likely to flag uncertainties about its work and less likely to make unsupported claims. This is borne out in our evaluations, which show that Opus 4.8 is around four times less likely than its predecessor to allow flaws in code it has written to pass unremarked.

It’s nice to see a feature update focused on improving LLM alignments.