If there’s one thing that the last several decades have taught us, it’s that tech companies do not exist for the benefit of everyday people. Despite what the heartwarming Superbowl ads may have you believe, tech companies exist for the pleasure and benefit of venture capitalist investors. There are indeed companies that are not beholden to the interests of VCs, but they’re typically not successful enough for you to have heard of them. Any tech company that’s a household name is in the pocket of at least one glassy-eyed investor off of Sand Hill Road. It’s a historical constant that success and VC investment inevitably lead to enshittification. And it’s not a bug, it’s a feature of the VC investment model: The line must go up forever, and eventually the product (and users) will suffer in the long run to make that happen. It’s an unhappy ending that has yet to be avoided by anybody who has tried.

AI is the future. You can disagree with this, fight against it, or ignore it. But there is no scenario (short of a literal apocalypse) in which our future involves less AI than it does today. It is simply too useful to not stick around and improve. Even if AI progress stops with LLMs (it won’t), people will find more uses for it (both good and bad) and build more products around it. It’s not all hype; this technology has practical applications. I’m not necessarily endorsing the AI future, but I accept that it is reality. And it’s only when you accept reality that you can thrive within it.

As of December 2024, OpenAI is the only mainstream household name in AI. Anthropic is the company that is used by AI power users. In other words, OpenAI is for normies and casuals and Anthropic is for the hardcores. This will probably change pretty soon because there’s no stability in this space. It’s too early to tell who will be the winners and losers. But to be honest, I don’t really care. None of these AI companies are doing anything particularly interesting.

“But wait,” I hear you say. “In one breath you’re saying that AI is the future, and in another you’re saying that the leading AI companies aren’t doing anything interesting. That doesn’t make sense!”

My friend, you need to expand your notion of what AI is and can be. While these companies are indeed developing the most advanced AI technology on earth, their actual product is API access to their models. And therein lies the problem. As per the (inexplicably) standard Silicon Valley playbook, both OpenAI and Anthropic operate at a loss. I would bet on them eventually becoming profitable, but not before at least setting the stage for full-scale enshittification, if not fully embracing it. And by then it will be too late. The moats will be established and the API dependencies will be established. We’ve already seen this with cloud providers like AWS and there is, realistically, no way back. Big tech will win again because it always does.

But there is space for another option! Smaller, open source LLMs are flourishing right now and will only get better with time. Unlike the frontier models like Open AI’s GPT4 and Anthropic’s Claude that are trained on many hundreds of billions (maybe even trillions) of parameters, there are many smaller, open source LLMs ranging from less than one billion parameters on the low end to around 400 billion on the high end. These models are free to download and run on your own hardware as you see fit. And because these models run locally, you don’t have to worry about token costs, rate limiting, or privacy concerns. There’s no catch and it’s as good as it sounds!

To directly address the elephant in the room: Many of these open source models require significant hardware resources to run. You’d probably need a RAM and CPU-maxed MacBook Pro or an array of NVIDIA 4090’s to run the larger models well. But to focus on this is to miss the forest for the trees: There’s a respectable set of small but capable LLMs that run well on commodity-grade hardware and probably even the one you’re reading this blog post on. You don’t necessarily need high-end GPUs or specced-out Macs to run models such as Llama 3.2 3B or Qwen 7B at usable speeds. To offer one data point: My iPhone SE 3, which is the lowest-end iPhone Apple makes, can run Llama 3.2 3B locally via PocketPal. While Llama 3.2 3B objectively has a tiny fraction of the training data available to generate responses that something like GPT 3.5+ has, it has fully replaced ChatGPT for me. It’s in no way the best LLM, but it is easily good enough for general purpose LLM tasks. For more specific tasks such as coding, I use more specialized models such as Qwen.

While we can easily deduce that OpenAI and Anthropic will enshittify, it’s unclear when it will happen and what that will look like. It could be as simple as jacking up API rates and bleeding customers dry, or it could take on a more insidious form like monetizing user data to make ends meet for the investors à la Google. We simply don’t know. Personally, I’m not sticking around to find out. And I don’t need to, because I have amazingly capable “small” LLMs that I can easily run locally on my laptop and phone. I have zero dependence on the cloud for my AI needs and have no API bills to pay. This is a delightful reality that everyone is free to enjoy right now and in the future. If you’re curious to explore this exciting new world, I’d recommend installing Jan and Llama 3.2 3B via its model hub. If you’re not using a Mac, you’ll want to look into enabling “Vulkan Support” and “GPU Acceleration” in the settings for better performance.

I think you’ll be surprised at what you can do with small, local LLMs. I for one am continuously amazed at what I’m able to do with just my daily driver laptop and Llama 3.2 3B and Qwen. You just need to have a realistic sense of what they can and can’t do well and how to make the most of them. I’ve been deep-diving this rabbit hole a lot lately, and it’s clear that the difference between the “best” and “good enough” LLM models is insignificant for many tasks. And at the end of the day, it seems like an obviously worthwhile tradeoff to own your computing experience and digital future.

Don’t be another victim of enshittification and start owning your AI today!