LLM Engineering

An engineering approach to requirements, planning loops, human control, and practical LLM use in software development.

The web development and digital product industry is changing quickly. Tools built around LLM models no longer feel like a weekend experiment: they are becoming part of the engineering process.

From what I see, there are several noticeable positions around LLM tools today.

Some people are highly excited about LLM agents and see them almost as a silver bullet. Others, often after a bad experience or without fully exploring the limitations, reject these tools completely. I feel closer to a third path: continuous experimentation and finding a balance between strict engineering discipline and the practical use of nondeterministic systems.

My strategy is adaptation: try things, compare results, keep the approaches that work, and discard what does not survive real use. Sometimes this is supported by metrics. Sometimes it comes from engineering taste, pattern recognition, and experience with complex tasks.

I do not believe in a “magical agent that does everything by itself.” But I do believe in processes where models make developers stronger: clarifying requirements, criticizing decisions, finding blind spots, checking plans, and reducing the risk of expensive mistakes before implementation starts.

Right now, my working toolset includes Claude, Codex, and Cursor. Subscriptions to these tools are still often more cost-effective than paying directly for API tokens. But one approach has had a particularly visible impact on the quality of my work with complex tasks: collaborative planning using strong models from different providers.

At first, this was a fully manual process.

I would take a task, ask one model to create a plan, ask another to critique it, and ask a third one to improve the result based on that critique. Then I would repeat the loop several times until the plan became clear, stable, and verifiable enough.

The value of this approach became visible quickly. Different models notice different risks, boundaries, and weak points. One may be better at decomposing architecture, another may be stricter at finding contradictions, and a third may help turn the result into an executable plan.

But the manual process was inconvenient and fragile. I had to move context between chats, track plan versions, preserve decisions, keep design separate from implementation, and manually check each time that the final document was actually useful for work.

After several of these runs, the conclusion became obvious: if the approach works reliably by hand, it should be automated.

That is why I built agent-quorum.

agent-quorum is an open-source tool for an iterative loop:

plan -> critique -> update -> validation

It orchestrates Codex, Claude Code, and Cursor Agent through their command-line interfaces to turn a raw request or a rough draft into a more mature, reviewed implementation plan.

What the tool does:

  • runs several models in different roles: one creates a plan, another critiques it, a third helps improve the result;
  • keeps intermediate versions, feedback, and the final plan;
  • watches the process so it does not hang, and allows it to continue after interruption;
  • lets a human step in and steer the work at the right moment;
  • keeps agents in a safer mode: they help plan and review, but do not change the code themselves.

For me, this is not an attempt to replace the engineer. Quite the opposite: it is a way to give the engineer more control over the chaotic power of LLM models.

I want agents to stop “doing magic in the repository” and instead help with thinking: propose a plan, receive critique, improve weak points, leave verifiable intermediate results, and let a human intervene when needed.

The more I work with these tools, the clearer it becomes to me: high-quality AI-assisted development does not start with planning or code generation.

It starts with requirements.

In my workflow, I separate a requirements stage before planning: a raw idea or problem first becomes a human-approved requirements document. At this stage, the focus is not on “how to build it” but on “what must become true”: context, goals, functional and non-functional requirements, acceptance criteria, constraints, risks, assumptions, and decisions around open forks.

The key principle: a human is not just a formal reviewer of a finished text. A human participates in the process. If there is a fork around scope, behavior, priority, compatibility, rollout, or a quality criterion, it should be discussed explicitly with a human, not silently guessed by an agent.

Only after that does it make sense to run a planning loop like agent-quorum.

For me, this is an important maturity boundary: an LLM model can be a powerful thinking tool, but responsibility for the product, requirements, trade-offs, and final decisions stays with the human.

I do not see the future as agents replacing engineers. I see it as engineers learning to design reliable processes around agents: with requirements, verifiable results, human participation at every important stage, access boundaries, and clear quality gates.

This is exactly where I want to keep growing: at the intersection of product thinking, engineering discipline, and the practical use of LLM models in real software development.

#softwareengineering #webdevelopment #aiagents #llm #developerexperience #opensource #typescript #productengineering #requirements #codex #claude #cursor