One accusation that tends to get thrown my way often is being anti-innovation. If you’ve followed me for any amount of time, you’ve probably seen me dunking on everything from Cryptocurrency to Large Language Models. This often leads to people asking the question “why is someone so technical so anti-technology?”.
To a lot of people on social media, I come of as just some unhinged weirdo who posts their every half-baked thought out onto the internet. This is indeed correct. But when it comes to my research, work, and career, I’m extremely calculated and deliberate in the decisions I make.
The reason I’m not diving head first into everything AI isn’t because I fear it or don’t understand it, it’s because I’ve already long since come to my conclusion about the technology. I’m neither of the opinion that it’s completely useless or revolutionary, simply that the game being played is one I neither currently need nor want to be a part of.
One thing that’s certain is that Generative AI is in a bubble. That’s not to say AI as a technology will pop, or that there isn’t genuine room for a lot more growth; simply, the level of hype far outweighs the current value of the tech.
Most (reasonable) people I speak to are of one of three opinions:
These technologies are fundamentally unsustainable and the hype will be short-lived.
There will be some future breakthrough that will bring the technology in line with the hype, but in the meantime everyone is essentially just relying on creative marketing to keep the money flowing.
The tech has a narrow use case for which they are exceedingly valuable, but almost everything else is just hype.
Whenever I’m critical of anything GenAI, without fail I get asked the same question. “do you think every major CEO could be wrong?”
The answer to that is: yes. History is littered with examples of industry titans going nuts, losing more money than the GDP of an entire country, saying “lol, my bad”, then finding something else to do.
I grew up during the fallout from the great financial crisis. I watched first hand as the biggest most prestigious financial institution crashed the entire global economy. Turns out, in the short term playing hot potato with debt derivatives backed by imaginary money and fraud is a great business model. In the long term, not so much.
It’s not even necessarily that corporate executives are being stupid. Sometimes they are, which can result in things like sinking more money that it cost the US government to put the sun in a bomb into the worst VR game ever. But usually it’s just greed and shortsightedness.
AI right now feels much the same. An industry fueled by the gluttony of myopic visionaries. An industry grasping at every straw to find a use case for their technology. An industry built on the premise of hyperbole and empty promises. But in this specific case, I actually don’t think big tech companies are making the wrong decision, at least, considering the choices they have available.
The biggest threat to tech companies right now is AGI (Artificial General Intelligence). AGI is a theoretical AI model which surpasses humans on their ability to learn, reason, think, and adapt. The risk is that if one company were to figure out AGI, they’d have an extreme competitive advantage over then rest in almost every space.
Sinking 100 billion dollars into AI research isn’t going to kill any big tech company. If things don’t pan out, it won’t even matter. Every other company will have lost similar amounts of money chasing ghosts. But if AGI does happen, the market will shift so fast and so significantly that any major player without AGI will be left in the dust. Quite simply, the possibility of AGI is an existential threat to big tech.
So from an executive perspective, lighting comically large piles of money on fire trying to teach graphics cards how to read is, surprisingly, the logical play. The rest, well, that’s all just creative marketing. It’s very difficult to show up to a quarterly shareholder meeting and tell your investors you just vaporized another $10 billion for absolutely no return-on-investment. At least, that is, without them questioning if you’ve completely lost your mind. Which is where leaning into the hype plays into it.
Tesla was able to keep investment flowing for over a decade by claiming each year that full self-driving was coming the very next year. Big Tech can easily do the same. “90% of our code is now AI.”, “we’re seeing spark of AGI.”, “we’re so worried that our super smart word processor will become sentient and kill everyone”. It’s all just vapid bluster designed to keep investors on board long enough for them to hopefully figure out a path to AGI. Fake it till you make it, but at an entire industry-wide scale.
Now, obviously, without sitting down individually with every big tech CEO, I can’t tell you how many actually genuinely believe they are close to AGI. Externally, it’s difficult to tell true innovation from grifting & hubris, especially in the tech space. But based on the outright insanity of many claim being made by major tech CEOs, I’d guess it’s mostly the latter.
Which brings me on to the downstream effects.
When you have all these big tech companies releasing audacious claims about AI on a near daily basis, you start getting asked the question “well, what are WE doing to prepare for AI?” The correct answer is, of course, nothing. Unless you have billions upon billions of dollars to build and train your own AI model, you’re basically just a future customer of some big tech company’s expensive subscription service.
The hugely prohibitive cost to both researching and building Large Language Models makes it the perfect walled garden for any big tech company. Should they succeed at AGI, they can sell the subscriptions for slightly less than the equivalent human labor, cornering much of the market. Should they fail, they can just look for some other way to lock everyone into yet another overpriced subscription model.
But in the meantime, why not slap “powered by AI” on your toilet brush and win over a few extra customers from your competitors. This is essentially what you’re seeing now with every single company just ham fisting some half-baked LLM chatbot into their workflow.
Then, of course, there’s the outright grifters. The people who jump on the latest hype train, try to position themselves as visionaries in that space, then take their bag of cash and leave right before it all comes crashing down. It’s no surprise that half of the AI influencer were previously hawking NFTs, or trying to sell you blockchain toasters.
What I find most interesting though, is Apple. I’ve always been a fan of Apple, just not in the typical sense. I don’t care much for iPhones or MacBooks, but I’ve always been fascinated by the way the company operates. They tend to steer clear of grasping at straws to find new business models, massively over-hiring and over-firing, or immediately hopping on every new trend. They tend to sit back, calculate, then make very careful and deliberate moves.
Apple initially began exploring the LLM space, performing internal research, and was even in talks to invest in OpenAI. They later released two studies on LLM reasoning (one in October 2024, and a second in June 2025). Both studies argued that LLMs do not reason, they simply perform statistical pattern matching, which they pass off as reasoning. Without true reasoning ability, LLMs will never become AGI, which appears to be Apple’s current public stance.
While Apple is still a major player in the AI space, they typically lean towards a preference for traditional AI. They’re almost certainly still doing LLM research in the background, but they’ve decided against dropping everything to go full throttle on the hype train. Ironically, this has started to make their investors, who have bought into the hype, quite upset. People are starting to question “why isn’t Apple doing things with AI”.
It may very well be the case that Apple too finds themselves pressured into going all out on LLM mania. Despite being both calculated and cautious enough to avoid over-committing to AI, while still mitigating any risks posed by potential future AGI, Apple is still subject to the whims of its investors, who really love buzzwords.
Of course, language translation is not AI, Siri is not AI, image identification & classification is not AI, the only acceptable form of AI is shoving some half-baked LLM chatbot into somewhere it doesn’t belong. After all, how can we possibly be sure that Apple is “doing AI” if they aren’t boiling the ocean by shoving something like Copilot into every god damn app.
Apple’s conclusion is the same conclusion I came to very early on in my experimentation with LLMs. I don’t say this as some attempt to flex “I was here first”, because I’m not a 3 trillion dollar tech company with a lot to lose. The more important point is this whole time I have been operating from the position that LLMs are not, and never will be AGI. My position has remained completely unchanged. As such, if it sounds like I might’ve received psychic damage from having to listen to 5 years of non-stop AI drivel on LinkedIn, it’s because that is very much the case.
Reasoning has long been a very contentious topic among LLM proponents. This is heavily fueled by the fact these models are somewhat of a black box. We can’t just take a peek inside and draw conclusions about how or why it produced the output it did. Even the people who build LLMs for a living readily admit this.
LLMs (and Neural Networks in general) are very similar to the human brain in that we can explain what individual neurons do, but put enough of them together and nobody has any clue what caused it to do the thing it did. Even the best neuroscientists have very little understand of how the brain works as a whole.
This is where things like the ‘Stochastic Parrot’ or ‘Chinese room’ arguments comes in. True reasoning is only one of many theories as to how LLMs produce the output they do; it’s also the one which requires the most assumptions (see: Occam’s Razor). All current LLM capabilities can be explained by much more simplistic phenomena, which fall far short of thinking or reasoning.
If a machine is consuming and transforming incalculable amounts of training data produced by humans, discussed by humans, and explained by humans. Why would the output not look identical to human reasoning? If I were to photocopy this article, nobody would argue that my photocopier wrote it and therefore can think. But add enough convolutedness to the process, and it looks a lot like maybe it did and can.
For me, one of the clear distinctions between true reasoning and pattern matching, is what happens when you remove access to new information. Many argue that LLMs are not plagiarism machines, they learn like humans do. They consume knowledge from teachers and books, developing an understanding along the way.
As a professional researcher, I too learned most of what I know by reading the works of far greater researchers. But there became a point where I knew enough to perform my own original research in uncharted waters. I now can and regularly do research topics where there is no other source to check my work against. This is not something LLMs are good at, or debatably, can even do at all.
Giving an LLM access to the entire internet and all of recorded human knowledge, then testing them with a quiz designed for humans is obviously just a cheap parlor trick. I, too, could score 100% on a multiple-choice exam if you let me Google all the answers. The true measure of LLM intelligence and reasoning should not be refactoring existing information, but the ability to produce truly novel works.
Sure, an LLM could probably create a brand-new pop song, because it has plenty of existing songs to analyze, allowing it to produce something seemingly new, but really just based on existing patterns. Yet, every time I tried to get LLMs to perform novel research, they fail because they don’t have access to existing literature on the topic. Whereas, humans, on the other hand, discovered everything humanity knows.
Where people get tied up is with the argument, “well most humans don’t produce novel work either”. But this is not because they’re fundamentally incapable of it. The average person is simply just sandbagged with an unfulfilling job. Ideally, without the perverse incentives of shareholder value, LLMs would automate all the busy work, allowing humans to focus on more meaningful pursuits.
I’ve made this argument many times before. But if humans could come up with all the groundbreaking discoveries they have, reading only as many books or research papers as they realistically could. Where are all the major LLM discoveries? An individual human may be limited in their ability to make novel discoveries as a result of competing with 8 billion other humans. But LLMs have access to the knowledge of all.
You’d think that a machine with access to every book ever written, every paper ever published, every speech ever recorded, and every study ever produced could do a lot better than some person who can read maybe 1 book per day. Yet, nothing. There’s the odd “maybe the LLM did something novel, but we don’t know yet” posts here and there, but if they could actually think, with as much information as they have, there’d be groundbreaking discoveries literally falling from the sky.
In reality, all we’ve created is a bot which is almost perfect at mimicking human-like natural language use, and the rest is people just projecting other human qualities on to it. Quite simply, “LLMs are doing reasoning” is the “look, my dog is smiling” of technology. In exactly the same way that dogs don’t convey their emotions via human-like facial expressions, there’s no reason to believe that even if computer could think, it’d perfectly mirror what looks like human reasoning.
One of the main challenges with testing LLM reasoning is it usually relies on giving it a novel problem. But as soon as new problems are published, answers are published too, at which point the LLM can just regurgitate an existing answer from its training data.
A logical problem I previously used to tests early LLMs was one called “The Wolf, The Goat, And The Cabbage”. The problem is simple. You’re walking with a wolf, a goat, and a cabbage. You come to a river which you need to cross. There is a small boat which only has enough space for you and one other item. If left unattended, the wolf will eat the goat, and the goat will eat the cabbage. How do you get all 3 safely across?
The correct answer is you take the goat across, leaving behind the wolf and the cabbage. You then return and fetch the cabbage, leaving the goat alone on the other side. Because the goat and cabbage cannot be left alone together, you take the goat back, leaving just the cabbage. Now, you can take the wolf across, leaving the wolf and the cabbage alone on the other side, finally returning to fetch the goat.
Any LLM could effortlessly answer this problem, because it has thousands of instances of the problem and the correct solution in its training data. But it was found that by simply swapping out one item but keeping the same constraints, the LLM would no longer be able to answer. Replacing the wolf with a lion, would result in the LLM going off the rails and just spewing a bunch of nonsense.
This made it clear the LLM was not actually thinking or reasoning through the problem, simply just regurgitating answers and explanations from its training data. Any human, knowing the answer to the original problem, could easily handle the wolf being swapped for a lion, or the cabbage for a lettuce. But LLMs, lacking reasoning, treated this as an entirely new problem.
Over time this issue was fixed. It could be that the LLM developers wrote algorithms to identify variants of the problem. It’s also possible that people posting different variants of the problem allowed the LLM to detect the core pattern, which all variants follow, allowing it to substitute words where needed.
This is when someone found you could just break the problem, and the LLM’s pattern matching along with it. Either by making it so none of the objects could be left unattended, or all of them could. In some variants there was no reason to cross the river, the boat doesn’t fit anyone, was actually a car, or has enough space to carry all the items at once. Humans, having actual logic and reasoning abilities could easily identify the broken versions of the problems and answer accordingly, but the LLMs would just output incoherent gibberish.
But of course, as more and more ways to disprove LLM reasoning were found, the developers just found ways to fix them. I strongly suspect these issues are not being fixed by any introduction of actual logic or reasoning, but by sub-models built to address specific problems. If this is the case, I’d argue we’re moving away from AGI and back towards building problem specific ML models, which is how “AI” has worked for decades.
I’m personally leaning towards the opinion that LLMs as a technology will soon, or have already, capped out. They fast hit a ceiling where giving them more data, more parameters, and more token stopped leading to any noticeable improvement.
More recent technological developments, just feel more like hacks.
CoT essentially just has the LLM break the problem down into smaller parts. To give an oversimplified answer, if I asked an LLM 1
1
1 to get
For validity, it could then just answer the problem normally, or via different approaches, comparing the results.
This addresses hallucinations to a certain degree (which are an inherent feature of LLMs, not a bug that can be fixed). It also in some cases enabled LLMs to solve problems that they can’t simply one-shot based on their training data. But it’s still reliant on the LLM being able to break down the problem in the first place, not hallucinate any of the stages, and also takes a ton more time & compute.
Since the appeal of LLMs, for most users, is getting a semi-decent answer as quickly as possible, few people want to wait the several minutes it takes for the LLM to boil the ocean turning 1 prompt into
It’s also not really an improvement upon the LLM technology, it’s just solving the fundamental flaws of LLMs by adding more LLMs.
As a researcher and writer, this is by far my least favorite LLM feature.
RAG was designed to at least partially address the issue of the extreme time and computational cost of training LLMs. LLMs aren’t re-trained super frequently, which means the datasets quickly becomes stale. You’ll probably recall how ChatGPT used to respond with “as of my knowledge cutoff of , " before giving the completely wrong answer.
RAG basically allows the LLM to search the internet for fresh data relevant to the user’s query, enabling it to fetch the most up-to-date information. The LLM can then use its training data to summarize the information retrieved via RAG. Essentially, this combines LLMs and search engines into a single product.
So, why do I hate this? Well, it’s basically glorified plagiarism. While I’d argue LLMs in general are just Plagiarism-as-a-Service, RAG is a lot closer to actual plagiarism that typical LLM behavior. As I’ve already argued, LLMs don’t think or reason. Thus, all RAG is really doing is using the LLMs’ natural language abilities to summarize or re-word some news article, blog post, or research paper. This deprives the original author of revenue & website traffic, while not transforming their work in any meaningful way.
Since RAG is still just a wrapper that sits on top of the core LLM technology, it’s still vulnerable to hallucinations. The technology also struggles when there’s too few sources, or when “understanding” the information would require additional context which doesn’t exist in the LLMs training data. The other major flaw, is the LLM not knowing when to use RAG, or failing to identify stale information, since not all search results will have publish dates.
I ran into this issue very recently during a joke research project in which I gave an LLM several thousand dollars, a brokerage account with option trading enabled, and complete autonomy to place trades. I’ll publish the full breakdown once the project has run its course, but one problem it ran into a lot was the LLM not using RAG to fetch current data on stocks. The LLM would just quote stock prices based on whatever they were at the time it was last trained, which in my case was over a year ago. This lead to the bot making trades based upon completely inaccurate price information.
One thing that has become very clear to me, is that similar to Blockchain, BigData, Cloud, and NFTs, a lot of the activity in the LLM space is motivated by fear.
The fear itself, is extremely justified. I’ve not seen a job market this brutal in my entire career. Previously, layoffs in the double-digit percentage were something typically reserved for major economic crises or bankruptcies. Now, it’s just something massively profitable tech companies do on a whim for seemingly no reason.
I’m regularly getting desperate DMs asking for help finding a job. I’ve had extremely technically talented friends forced to work non-tech jobs to ride out the current market. The recruiters and hiring managers I talk to regularly tell me about having to pull down all their job postings after less than a day because they’ve already gotten thousands of applications.
Much of this fear is exacerbated by people attributing the mass layoffs and lack of available jobs to AI replacement. This is very much not the case, but the economics of it all is extremely complex and would require its own article. But what matter is that people believe it to be true, and tech companies are more than happy to lean into those narratives to hype up their AI products.
But fear makes people behave irrationally, and that’s currently a main driving force behind AI adoption.
Probably one of the most common fallacies I see in tech is that it pays to be early. The “first mover advantage”. This is not something that has ever seemed consistent with reality. When a new technology comes around, it’s riddled with flaws that need to be ironed out. The use cases aren’t clear, the viability of the technology isn’t clear, It’s a whole lot of trial-and-error.
But people still believe there is a benefit to being early. I suspect it’s due to the fact you only hear about the few companies who adopted a new technology early and became successful, not the thousands that failed along the way. I’d argue that far more companies succeed being late than early. They can analyze where others went wrong. They can look for gaps in the market. They build on what’s already been done. Running blindfolded into a minefield is simply not a good business model in my books. But many seem to think it is, at least when you call it “AI”.
The first web search engine, “Archie” was launched in
Yahoo launched in 1995 along with AltaVista. Dogpile and AskJeeves came about in 1996, and AOL in
Google, the current industry titan, only entered the race in
The same is true for Apple. When they announced the iPhone, most of the incumbents balked at it. They were certain it wasn’t a viable product. It went against the current conventional wisdom. In 2022, Apple became the first company on earth to reach a 3 trillion dollar valuation.
Tesla was over a century late to the electric car game, building a product long concluded to be of no interest to consumers. It’s now worth more than basically every car company combined. Objectively it’s not actually worth that, most of the investors are just drunk, but Tesla did successfully create a market for EVs.
I could go on for days and days. But simply put, first mover advantage is for board games and patent applications, not adopting new technologies.
Right now, LLMs are an extremely immature technology. I personally believe they’re not going to get much better than this, but a breakthrough innovation could change that. Either way, it doesn’t matter to me. If the technology is a fad and completely implodes, I couldn’t care less. If it’s not a bubble and LLMs actually turn out to be the new best thing, I can easily adopt them into my own business model.
I’m professionally late to every party. I learned Assembly language in
Malware reversing engineering in
Vulnerability Research in
This blog, mostly documenting manual malware analysis, something that has been ML automated since before I was born, is what made my career.
So when I see people jumping on the latest hype, telling me I’m going to get left behind, I can only chuckle. If LLMs as a technology are viable, they’ll still be around when and if I decide they’re useful for me. If not, I’ll have missed out on losing my life’s savings in Beanie Babies, The DotCom bubble, or NFTs.
Part of me wishes I could just claim that I’m not motivated by fear, because I’m just built different. But to be perfectly honest, spending your early twenties trapped in a foreign country while the FBI tries to put you in jail, sort of just…completely fries your nervous system. My entire fight or flight system is basically now just a single hamster on a wheel. One benefit, though, is its easy to know you’re not making decisions out of fear when you can’t experience any.
So, I ended up making the decision to learn new skills and expand my existing subject-matter expertise. I still regularly use LLMs, and keep up with new innovations in the space, but I have no intention of pivoting into “AI” right now. The field of doing $stuff with the latest $thing is extremely over-saturated, and I’m quite happy working on the cutting edge of existing technologies.
But what I believe to be the biggest harm is not fear, but the downstream effects of the rush to adopt LLMs.
One of my other main arguments for why I believe LLMs have peaked, is source cannibalization. Since LLMs do not think nor reason, they are heavily reliant on large corpuses of human-produced data for training and RAG.
But when LLMs deprive data publishers of revenue either indirectly via training on their works without permission, or directly as a result of straight plagiarism via RAG, this forces publishers behind paywalls. The paywalls not only limits the LLMs’ access to future training data, and it’s ability to use RAG, but also negatively impacts regular humans who are not using LLMs at all.
Right now big tech companies operate in a temporary utopia where they’ve been able to capitalize on mass-scale copyright infringement as a result of the free and open internet, but have not yet started to suffer the consequences of the damage they’re causing to the information ecosystem.
LLMs act as a sort of magic funnel where users only see the output, not the incalculable amounts of high-quality human-produced data which had to be input. As such, it’s likely people significantly overestimate how much their work (prompting) contributed to the output they received, and grossly underestimate how much of other peoples’ work was required to make it possible. It’s classic egocentric bias.
This kind of bias leads to people ignoring the threat LLMs pose to their own data sources. The problem is further amplified by AI slop (low-quality AI generated content), which floods the internet, degrading the average quality of information. This slop not only makes it harder for humans to find high-quality sources, but harder to train LLMs, since allowing the slop to enter LLMs datasets risks creating a feedback loop which could cause the LLM to undergo model collapse.
LLMs inherently hijacking the human brains’ reward system. By allowing people to quickly summarize & manipulate the work of others, LLMs use gives the same feeling of achievement one would get from doing the work themselves, but without any of the heavy lifting.
The brain is naturally very fragile to instant gratification, which is also part of the mechanism behind drug addiction. When feelings of accomplishment are tied to completion of a task, the less time taken to accomplish the task, the more frequent the dopamine hits. Simulation games often exploit this by reproducing real world tasks, but in a way where they can be completed with much less effort.
The most extreme example of short-circuiting the brain’s reward system, is of course drug use. By consuming chemicals which force the brain to release neurotransmitters associated with feelings of accomplishment, users can generate the same feelings of success, without necessarily needing accomplishing any tasks at all.
A while back I encountered several studies researching the effects of Adderall on neurotypicals. Both people with and without ADHD tend to report a significant increase in productivity resulting from taking Adderall. It’s well establish that Adderall boosts productivity in people with ADHD, likely by correcting counteracting their brain’s natural deficit of dopamine and norepinephrine.
With neurotypicals on the other hand, the results were very different. One study showed that neurotypicals felt more productive when taking Adderall vs a placebo. But their objective productivity remained unchanged or even declined while under its effects. Another study showed that Adderall use led to a noticeable decline in objective productivity.
Since most people without ADHD don’t have deficits in dopamine or norepinephrine, the Adderall increases neurotransmitter levels above normal, producing a high. Since dopamine and norepinephrine play a significant role in feelings of confidence, satisfaction, and gratification; it’s not really unexpected that a surplus would skew judgement.
Having ADHD myself, and having on many occasions accidentally double dosed my Adderall, I’ve personally experienced both sides of this. The genuine productivity boost from using a much-needed medication, and the overstimulated overconfident word soup, which I look back on at a later date with dismay.
Whenever I come across yet another fart-huffing self-aggrandizing take on LinkedIn about how programming is dead or LLMs are replacing cows and disrupting big milk, I think about the Adderall studies. These are not the words of a rational person objectively evaluating a new technology, but someone high out of their mind as the result of an LLM-induced dopamine overload. It evokes the exact same feeling of talking to someone who is high on cocaine.
What’s interesting is studies attempting to measure productivity increase due to use of LLMs are actually finding the opposite. Everyone feels more productive, but the data is showing a notable decrease in objective productivity among LLM users. My very un-scientific hypothesis is that many LLM users are simply just completely cracked out on dopamine. The euphoria resulting from their perceived now limitless abilities is clouding their judgement.
Which makes me wonder: what if we replicated the Adderall study with LLMs? Would we find similar results that LLM use does boost productivity for people with ADHD by increasing dopamine and reducing heavy lifting? Or is any increase in productivity negated by the fact that low-quality output is inherent to the LLM, and not purely tied the mental state of the user?
Either way, the current research perfectly lines up with what I’ve been observing. A whole lot of hyperbolic claims from people who just made their first totally not going fail B2B SaaS purely with vibe coding, but not a whole lot of substance. I think people are simply overestimating their productivity and abilities as the result of a dopamine high produced by their instant gratification machine.
The popular wisdom that’s seen as somewhat of a middle ground between “LLMs are useless plagiarism machines” and “LLMs are going to replace everything ever” is the hypothetical AI-accelerated employee. While not necessarily terrible advice, I feel like the mainstream interpretation is the opposite of what the advice should be.
The fact of the matter is, “prompt engineering” or whatever they’re calling it these days, has a skill cap that’s in the floor. Prompting LLMs simply just isn’t a skill, no matter what influencers who definitely weren’t previously claiming Bored Apes are the new Mona Lisa say. If you look at the prompts even the LLM developers themselves are using its things like “please don’t make stuff up” or “think extra hard before you answer”.
In fact, I’d make a strong argument that what you shouldn’t be doing is ‘learning’ to do everything with AI. What you should be doing is learning regular skills. Being a domain expert prompting an LLM badly is going to give you infinitely better results than a layperson with a ‘World’s Best Prompt Engineer’ mug.
The advice is completely backwards, and leads people towards over-reliance on AI, which brings me to the final and most serious issue.
LLMs are somewhat like lossy compression of the entire internet. They boil nuanced topics down into a form that isn’t quite layperson level, but loses a lot of nuance while still evoking a feeling of complete understanding. The part that’s missing is, you don’t know what you don’t know. If you lack an intricate understanding of the task you’re using an LLM for, you have no idea what nuance was lost, or worse, what facts it made up.
But I’d actually go a step further and bet $1,000 that in the next 5 years we’ll start to see an abundance of studies showing overuse of LLMs actually results in significant cognitive decline. The brain is much like a muscle in the sense that neural pathways have to be continuously re-enforced through mental exercises. I suspect many of us have had the experience of not using a skill long enough to completely un-learn it. LLMs enable this, but with every skill.
Maintaining skills isn’t just a case of accessing knowledge regularly, but interacting with it in different ways. There’s a reason why language learning tools have you say words, translate them, read them, write them, and use them in sentences. Every distinct means by which you apply knowledge or a skill further reinforces it and deepens your overall understanding.
When people use LLMs, they aren’t just being presented with flimsy surface level understandings of topics. They’re often outsourcing many of their means of reinforcing knowledge to the AI too. And in some cases, their logic and reasoning itself. The more people lean on LLMs, the more likely they are limit their knowledge expansion, undergo skill regression, and weaken their logic and reasoning ability.
So, in fear of being replaced by the hypothetical ‘AI-accelerated employee’, people are forgoing acquiring essential skills and deep knowledge, instead choosing to focus on “prompt engineering”. It’s somewhat ironic, because if AGI happens there will be no need for ‘prompt-engineers’. And if it doesn’t, the people with only surface level knowledge who cannot perform tasks without the help of AI will be extremely abundant, and thus extremely replaceable.
You can learn how to prompt LLMs at any time, but you can’t learn a decade of specialized skills in an afternoon.
So yes, while I may come off as a massive LLM hater, I feel like I have my reasons. With that said, I am still actively researching and experimenting with LLM regularly, and I’m always open to being proven wrong. But currently, I’m simply not seeing it. I’m not seeing heaps of successful LLM products, businesses, or use cases. What I’m seeing is a lot of shovel selling, and a huge black hole for VC money.
Maybe some day I’ll write a post about the viability of LLMs for something I’m building. But it won’t be today, this year, or likely anytime soon. In fact, I’m currently still getting job offers for manual reverse engineering jobs. It’s extremely common for security companies that use machine learning to hire manual analysts. ML models need constant tweaking and updating, which means a huge market for experts who can be a part of that process.
I’d expect this is where LLMs will go should the tech take off. Not job replacement, but a shift towards professionals using their experience to fine tune LLMs, instead of doing the work directly. In fact, I’ve started getting the same offers to consult for LLM companies that I am for traditional ML ones.
Times change, but technology progresses slowly.
Business, governance, and adoption-focused material. Real-world implementations, case studies, and industry impact.
