AI is a sandpit

Let me open by saying that I am not against AI. Used responsibly, AI represents a very useful tool not only to help automate repetitive tasks that require more than just regex (or maybe they could be done with regex, but it's probably a matter of feasibility over insanity).

When I say "AI", I'm referring to LLM's, whether that's an LLM that answers questions or one that produces 'creative works' - just an important clarification because technology moves fast and the meaning of 'AI' may be different a year from now.

I do have a problem with the absolute mess we're in. Like many technological advancements, there is always a transitionary period where adoption and exploration bring us many great things, but also so much wasted potential.

That's the sandpit but it's very quickly turning into a sinkhole, like that scene from The Simpsons Movie where Homer says 'Fix Sinkhole', then drops Maggie into the sinkhole to plug it, before exclaiming 'Check!' and ticking the item off his list.

That scene is reminiscent of what AI was not too long ago. It was funnny, quirky, and problems? What problems? It's so rudimentary that it can't possibly be dangerous.

Later in that movie, the sinkhole swallows the entire house, garden, etc. The Simpson family escaped through that sinkhole, and were spat out on the outside of the bubble that was towering over the town. That's what AI was recently. Drawing in everything it could get its claws on and mixing it up.

It seems like everyone's shared their opinions on it, and it seems like mine is just another voice echoing in the void. So, why write a blog post about it?

You are a risk to your users

I'm a developer, and I've always been amazed by what we can make computers do. Making text appear in a web browser was amazing to me when I was younger. Now we have LLM's that can build entire websites and apps.

I'm intrigued by it, and I'll even willingly admit I've tried to make it work. I've never intended to vibe-code an entire project, but guiding it to build out screens for apps seems like a reasonable ask.

I've used AI to help me write RLS policies and functions for Postgres, but only when I've either forgotten the syntax (the Postgres docs are OK, but horrible to actually read), or my brain is having a short-circuit and can't quite connect both parts of the solution together. AI can help with that, but I always validate its ideas and suggestions to verify that I'm not inadvertently exposing user data. I never blindly trust what it gives me.

I also never use commercial AI tools. I've been big on self-hosting for a long time, and AI is no exception. I've discussed this before, but to recap, I run Ollama + Open WebUI on an Unraid box with an RTX3090, using various freely available models.

But I see so many people throwing caution to the wind, and letting AI shit out a product. And I'm tired. I'm so fucking tired. I'm not tired because I'm jealous, I'm tired because I'm angry that this is what the industry is turning into.

I keep seeing posts in various subreddit's of people saying "I built this product in 3 weeks and I don't even know the syntax - please rate my app!". I see a few people boasting about their vibe-coded turd, and then only days later they're crying that they got breached and people were using their AWS keys without permission, and ran up bills in the thousands, and that none of this is their fault because they don't know how to code and... takes breath

Again, I do not disapprove of responsible AI usage, but I loathe irresponsible usage. People who prompt an AI to spit out a product, and are busy raising a toast to a 'successful launch', when someone with even a modicum of knowledge runs in shouting "STOP! THE PRODUCT HAS NO SECURITY!".

These same people who don't have the willingness to either learn the trade or hire people to do it properly have the gall to either charge a subscription fee to access their 'service', or expect the communities to review their entire slop-stack.

Good intentions

Please believe me when I say that I understand good intentions and the wish to make the world a better place. I understand you want to do something to validate your existence as being worthwhile.

The problem is when you don't have the foresight to understand why development is such a deep subject and why developers spend many years of their lives figuring it out. That's when you put people or their data at risk. Sometimes, that risk is technical and can be solved with technical solutions, but other times it's psychological.

Your good intentions become malicious incompetence when you start trying to vibe-code 'life coaches', 'dating advice', or anything financial. You pose an extremely high risk to other people when you begin pushing into areas dealing with the human psyche or things that are considered fundamental to exist within the systems we have built as a society.

Let's say you're building a psychological support app for people with 'common generalised disorders', such as anxiety, or depression. Let's say you're going to charge them a subscription fee of $5 per month to use the service.

Firstly, are you trained to support this individual for their needs? If you are, there's a very strong likelyhood that you also aren't well-versed enough to understand the technical aspects of running such a project. I know there are some exceptional individuals out there who have the experience and knowledge to do both psychology and coding to a high level and with responsibility, but those individuals are very rare.

Secondly, do you understand your legal obligations? Do you understand that by taking payments, you must ensure that you comply with various laws surrounding the protection and processing of payments, and various other laws surrounding the processing and usage of personally-identifiable information (PII)?

Sure, you're probably gonna offload your payments to a regulated provider like Stripe, but do you understand the technical requirements of actually linking your user data with their payment information? If you don't know how to do this, do you know how to find the information you need to do it, without using AI?

If the answer is no, then you do not possess knowledge of the fundamentals required to run the project and should stop thinking AI will make it workable. Learn, or hire - those are your options.

The problem with AI isn't AI

The problem with AI is people. AI is just a tool. It's very versatile, but every AI is a hammer and every problem is a nail. Sometimes the nails don't even exist, but it still tries to beat them into an imaginary plank.

People are the problem with AI. They treat it as the only solution rather than as a tool. The people don't want to spend the time to understand whether what they are doing is safe (from a technical aspect), but they damn well don't even stop to think whether what they're doing is ethical.

This too shall pass

Like many advancements, this phase we're in will pass eventually. Despite my doomspeaking, I still hold onto hope that AI will become an integral, but responsible, part of every day life. Not used for companies to spy on their users. Not used to produce slop that infringes copyright and destroys industries. I want AI to grow into a responsible, effective tool that offers responsible assistance and improvements to our quality of life.