Category: Lessons

It is summer. I’ve slowed down with LLMs. More than anything I’m getting tired. Between running overnight agents, reviewing PRs and one shot-ing /goals …. it is time to take a break.

Since checking in my first git repo in early 2026, I haven’t stopped working on my hobby projects. Things have come a long way since early 2025.

So what have I learned.

1. LLMs close the prototyping gap. I can’t count the number of times I’ve sat on my hands, thinking, wow this is a great idea… If only I could find someone to help me get started. Now, my ideas get done with a prompt. Hours on hours of dev work, debugging, testing done in a matter of hours. Look at the git chart above and you can see how each week one of my ideas came to reality. And this is only the stuff checked-in. My day to day projects for my household go back even further.
2. LLMs don’t replace humans. (but maybe some coders). Models love to code. It used to be an art to review the beauty in someone’s code. There are many roads to a solution and elegance in the approach. I grew up with BASIC, VB and Java. But do I still care? I don’t care. I just want to put an idea to paper. And often, even crappy code, is good enough. For me, the narrative against AI slop is not about whether this code is good enough. For me, it is code vs no code at all.
3. Bad product manager = bad product. LLM coding is amazing, but a bad feature is still a bad feature. If you’re not coding, then most work is developing features, dog fooding the feature, iterating and verification tests. For now, critical thinking is better in the human brain. The number of times I’ve had to ask the LLM to re-design the app UX after adding a small detail is frustrating. It seems the current issue is that tokens are so expensive. If I wanted a model to code and to design UX, I would burn so many tokens and responses would be dreadfully slow. So the trade-off, is how much time and money do I want to give up each prompt for a multi-model, multi-context, coding model? It’s a familiar situation where we have a designer arguing against a coder for how an app should look and perform. “Why did you add a field that introduces line breaks and messes up the design?” Can I create an agent-team that argues this on every prompt? Sure. But I don’t want to burn all my tokens each turn either, so I must be more intentional. I am guessing this is the next step on frontier models, where it is less about the work itself, but the work “around” the work where things often go from good to “great”.
4. The harnesses are too technical. Coding is hard to explain to regular folks. “It creates and runs tests on its own!” is meaningless. But to people who code, we know how much time debugging takes. I would argue that having code that immediately runs without even a syntax error is a dream. So an argument for application prototyping is an argument assuming someone gets npm, node.js, git,PRs, branching, worktrees etc. Concepts that are foreign to many. For example, in Cowork, editing documents without a git repo can be immediately destructive. And people do not know how to revert, backup, or run change control on their documents. How would they know how to solve for that.
5. Differentiation is disappearing. I’ve shared my frustration with having to pre-select a model. It is unintuitive because the description of “your hardest work” is itself ambiguous. I likely do not have very hard work. Not only that, but it is not like I will re-run my prompt with another model to do a comparison so I end up assuming Fable’s work is just better than Sonnet work. For every model that I’ve chosen in the past 6 months, the job done is good enough. And while I’ve advocated for the Claude harness over others for months, Copilot, GPT, and Gemini all now have the exact same features. With open weights catching up, it is to the point where I question the added value of model point releases. I would advocate to those who are asking me what LLM to use, to use whatever gives you the highest model at the cheapest cost.
6. Verify, verify, verify. Current thinking models do not prioritize tests as much as they should. My guess is that writing tests sometimes is more context than writing code so the trade off of more tests vs longer runs is not worthwhile user experience. For many of the features I’m building that are outside of design and UX, I ask specifically for tests, code review on tests, /goals on tests. Setting these guardrails early as you vibe-code will save you hassle down the road because syntax perfection does not imply app perfection. I build a lot of apps that require calculations and financial models and my heart skips a beat every time I want to introduce a new component that touches my engine.

Enterprise isn’t sexy. But enterprise is hard. Always felt Microsoft knows enterprises best. I think about how they destroyed Slack’s growth when creating Teams. I remember the day Slack got cocky, took out a full page NYT ad, and the Startup world laughed. 10 years later Slack, now owned by Salesforce, is an enterprise afterthought. Microsoft has made some big moves to keep relevant and I admire the choices. I remember the day they moved Office to the cloud, made everything multi-platform (not Windows!), and took a bet on Azure. These were Satya Nadella’s decisions.

When I hear Satya Nadella, I take out a pen and paper to write down how he says everything because the manner in which he speaks and creates an argument, ties it back into his role as Microsoft CEO, and makes a case for it, puts me at awe. He speaks very differently than a startup founder. I admire how he takes a hard question, goes far away with an anecdote, raises a relatable point, and draws that point back to the question.

Casey Newton: “But so much has been happening at Microsoft with AI since then. So just catch us up. For people who may not have tuned in a little while, what have you guys been up to in AI?”

Satya Nadella: “Yeah, I mean, look, the fundamental thing that I feel we’re about to move from is not talking about AI as one thing, to sort of having even a mental picture of what is an ecosystem that is sort of driven by AI, right? So today, if you think about it, even since when you first used Sydney to now, it has been about frontier model. You sort of talked about Fable, what have you.

But if we are ever going to transition to an economy that is driven by AI, it can’t be about one model, it can’t be about three firms, it has to be something that’s broadly felt, where the economy is at the frontier, not a firm or a model is at the frontier. So Microsoft is a platform company, to me that’s what we are up to. So to me, we had our developer conference last week.

It was all about, hey, can we build a platform and the tools where every enterprise in every country can operate at the frontier. To me, that’s the question. To be saying, hey, my model does this, but the economy is growing at 2 percent, means this is not going to end well, unless we really get to a place where the economy is inflecting in terms of its economic growth and its broad spread because the frontier benefits are.”

He was the latest guest on the Hard Fork podcast by Kevin Roose and Casey Newton. And this week’s topic was talking about Microsoft’s vision for AI.