Fawzooz
ع

Cognitive XOps

Book

Request the PDF

Enter your email and we will send you a code; your request is then recorded at once, and a link to download your copy reaches your email.

By continuing, your email and progress are kept in your account. Privacy

* The file is for your own reading; sharing follows the terms of use, and commercial use is not permitted.

Reading progress
0 of 13 sections read
4 / 13

Section 2

2The Problem

2 min read4 of 13Read it in the book · page 10

For decades, software behaved predictably. The same input and the same code gave the same output, and a unit test could prove it. AI changes that. A model can answer the same prompt differently twice, and the flaws it introduces are often subtle enough to slip past tests that were written for predictable code.

Three pressures AI puts on software delivery, and the modules that answer each.Three pressures AI puts on software delivery, and the modules that answer each.
Figure 1. Three pressures AI puts on software delivery, and the modules that answer each.
Text in this figure

01 · Vibe coding and shadow AI · Code accepted because it looks right, and tools nobody approved quietly moving data out. · Answered by · M01 · M06 · M09 · 02 · The tired reviewer · More code to review, more context switches, and open loops that follow people home. · Answered by · M02 · M10 · CLI · 03 · New attack surfaces · Hallucinated packages, runaway agent bills and cryptography that quantum computers will break. · Answered by · M03 · M04 · M05 · M07 · ≈ 1 in 5 · packages suggested by the models tested were hallucinated . · (Spracklen et al., 2025) · Figure 1. Three pressures AI puts on software delivery, and the modules that answer each.

2.1 Vibe coding and shadow AI

Under deadline pressure, many developers accept AI-generated code because it looks right and seems to run, without reading it line by line. This is “vibe coding” in its risky form. Alongside it grows shadow AI: browser extensions, personal API keys and local model wrappers that nobody approved. Each one is a door through which source code, customer data or credentials can quietly leave the organisation.

2.2 The tired reviewer

AI was supposed to lighten the load. For many developers it has done the opposite. Reviewing code you did not write is demanding work, and AI multiplies the amount of code waiting for review. Developers jump between prompts, diffs and agents all day, and the tasks they leave unfinished follow them home. Psychologists call this the Zeigarnik effect; replication studies disagree on how strongly it affects memory, but most engineers will recognise the feeling. A tired reviewer approves the happy path, and that is where many production incidents begin.

2.3 New attack surfaces

AI also opens doors that did not exist before. Models sometimes recommend software packages that do not exist: a large study presented at USENIX Security 2025 found that about one in five packages suggested by the models it tested were hallucinated (Spracklen et al., 2025). Attackers now register those invented names to plant malicious code, a practice known as slopsquatting.

Agents that call models in loops can also run up enormous bills. The OWASP Top 10 for LLM Applications (2025) lists this as Unbounded Consumption, and practitioners call its financial form Denial of Wallet. And the public-key cryptography that protects pipelines and model artefacts today, such as RSA and elliptic-curve cryptography, will one day be breakable by quantum computers, so pipelines need to be ready to switch algorithms without a rebuild.

Tip: use ← → to move between sections.