AI-generated code – vibe coding – is an exciting prospect: it turns anyone into a computer programmer. But that is precisely what is wrong with it…
The problem is not that vibe coding introduces an excessive number of vulnerabilities. Comparative analysis shows AI vulnerabilities are at a similar density per line of code to those introduced by humans. Code quality is not the problem. It’s just there’s too much of it, too fast, and it lacks good judgment.
OX Research, who undertook an analysis, finds two issues. Firstly, where vulnerabilities do exist, they “reach production at unprecedented speed” – too fast for accepted code review processes to find all vulnerabilities. Breaches have already occurred through vibe-produced code that has been missed in review.
Secondly, AI may have read and understood the Beginner’s Guide to Programming, but it has not learned the good practices that only come with years of doing the job. Good code does not a good program make. The OX researchers discovered ten common anti-patterns that are introduced by AI. An anti-pattern in code is defined as something that is not necessarily wrong, but ineffective, counterproductive, and ultimately a bad practice. The problem with anti-patterns and AI coding is that if they occur once, they are likely to be repeated by the same AI coding system in many other outputs.
The first and most common anti-pattern is excessive commenting. While human coders add comments to help other coders (or future maintainers) to navigate complex areas, AI tends to strew them everywhere. While this could be considered as the AI being helpful, the researchers think it is inherent to the AI itself. Just as a chatbot retains context in a conversation with a user, AI provides its own internal context by commenting on almost everything: “They are a testament to the internal workings of GenAI, a clever workaround for their current limitations in long-term, scalable memory.”
A second common anti-pattern is the missing human urge for perfection. To AI, if it works, it’s good enough. A human developer might pause to think, wait, I could do that more eloquently, or fine tune it here, or make it more scalable, or maintain long term viability. The AI is simply responding to the user’s prompts, and if the user is an untutored newbie who programs simply because he can with AI-assistance (and is cheaper to employ than a long-term professional), then this anti-pattern is storing up problems for the future.
Other problems include over-specification generating single-use solutions rather than re-usable components; re-implements everything from scratch rather than using established libraries; lack of deployment awareness creating code that only runs on the local machine; and five more.
The two issues highlighted by the OX researchers have different solutions. The anti-pattern solution requires improved AI systems, and better prompting by the code instigator. The former is out of the users’ control. The latter can be minimized by developing best practices regarding the use of AI coding tools within their environments – and ensuring everybody (including non-professional programmers) abide by them. The coder must transition to an architect.
The exclusion of adequate development review caused by the sheer scale of code development demands a rethink of current process lest buggy software slip through. “Embed security guidelines directly into AI workflows rather than hoping to catch issues later,” suggests OX Research.
Meanwhile… Vibe-coding? Well, with apologies to Joe South from the ’70s, “You know all the words, and you hum all the tunes, but you never quite sing the song.” It would be wrong to assume that vibe-coding won’t improve over time and be able to sing a better song than it does today – because it most certainly will. But what the OX Research analysis shows very clearly is that if you use the current AI offerings, do so carefully, keep your eyes open, and take whatever precautionary options are available.
Related: Vibe Coding: When Everyone’s a Developer, Who Secures the Code?
Related: Flaw in Vibe Coding Platform Base44 Exposed Private Enterprise Applications
Related: Ox Security Launches AI Agent That Auto-Generates Code to Fix Vulnerabilities
Related: Ox Security Bags $60M Series B to Tackle Appsec Alert Fatigue
