I’m playing around with Local LLMs again! Using Gemma 4 12b on my underpowered Mac M4 and it is good. Much better than when I first played with Gemma 4 E4B. The tool calling is so much better and the general day to day feels crisp. I’m not doing any benchmarks, just running my life.
A lot of r/LocalLLM chatter is because it is the first model to run on laptop class 16GB RAM, the likes of MacBook Pros with unified memory. And it does! Barely. Running in oMLX cost me 11GB. Was like living in 640K memory in the DOS days. Closing everything in activity monitor. A more realistic scenario is using the GGUF 8GB which is survivable. Anyway, point is, if we were not in 2026, living memory poor, one can easily see a future where laptops shipped with local LLMs doing real work, real tool calls, all privately….with more RAM.
Exploring this further I wanted to see what was new in the 12B architecture and went down a small rabbit hole about why Google released 12B and its architecture in their developer post.
Gemma 4 12B introduces several milestones for local AI:
- A multimodal encoder-free architecture: Bypassing heavy multi-stage vision and audio encoders entirely, multimodal data is fed straight into the LLM backbone, reducing multimodal latency.
- Our first medium-sized model with audio input: In the Gemma family, audio inputs were restricted to small, lightweight edge architectures (e.g. E4B). Gemma 4 12B is the first medium-sized model capable of natively ingesting audio.
Gemma4 being encoder free was a new one to me. On their developer post they had this image which only confused me more.
Thankfully, they also included a link to Maarten Grootendorst’s post A Visual Guide to Gemma 4 12B which explains it eloquently. Every few month there are new optimizations and ideas on how to fit more into less.
Leave a Reply