The setup

After less than two weeks, my Kanban project was running smoothly. I could write a GitLab issue with clear acceptance criteria and context, and Claude would handle the coding and open a merge request. The agent would either merge the request itself or let me know if it needed my review. I was able to add bug fixes or new features several times a day, and most merge requests were handled automatically.

With this setup, the project changed several times a day. I wrote the issues and gave direction, while the agents wrote the code. Here are a few examples of what we shipped in those first weeks:

• A “dossier” for every card. OpenBrain became my main tool, and every agent I used read from it for context. The dossier widget showed what OpenBrain already knew about a card whenever I opened it. This made it easy to find the information I needed for each task without switching tools.
• Stale-card management. If a card isn’t updated for two weeks, it gets flagged as stale. After three weeks without changes, it’s automatically archived. My workspaces started to manage themselves.
• A cross-workspace overview. I had a single view across all workspaces, showing each customer’s status and what was stuck. I no longer had to click through different boards to see where I was needed.
• A “what now?” recommendation engine. This backend service scored open cards by priority, age, staleness, and dossier info, then suggested the next thing I should work on.

All of this felt like real progress. I was more efficient and focused on what mattered most. Plus, I had a tool I actually used every day.

The drift

Even though I enjoyed the constant updates, something still bothered me, but I couldn’t quite figure out what it was.

By the end of April, I had built a solution that fit my needs and how I worked with customers. Even though adding new features and fixing bugs were fun and satisfying, I realized I had just made another Kanban board. The irony hit me while I was working on another feature. I paused and typed something like, “…hey, I think we’re missing the mark here.”

The one feature that could have made my Kanban solution better was still sitting untouched in GitLab.

The dispatcher’s goal was simple: I wanted to hand off basic tasks or questions to an agent instead of doing all the grunt work myself.

Anyone who’s worked through a backlog knows the feeling: it’s easy to finish the simple tasks, but the hardest ones keep getting pushed aside. That’s what happened with the dispatcher. I kept checking off small features and fixing bugs, while ignoring the one feature that could really change how I used the board.

My focus, and the speed the agent gave me, kept going toward the wrong things.

The pivot

Around the time I had my ‘ah ha’ moment, I found the /grill-with-docs skill by Matt Pocock. I decided to try it and see if it could help me and my agents focus on what mattered most.

The skill ended up asking me about eleven questions and helped me work out the details of the dispatcher feature. It turned our conversation into Architectural Decision Records (ADRs), which are simple Markdown files that capture decisions that are hard to change later in a project.

Here are a few decisions the skill helped me make, which could have caused problems later if left unaddressed:

• Project-scoped GitLab tokens. Since I wanted agents to do work for me and GitLab is the main source of truth, I needed to give them access to the right projects. I chose project-scoped tokens instead of group-scoped ones to limit the impact if anything went wrong.
• Concurrency. I chose to use a single agent for the whole system, running jobs in a FIFO SQS queue. This also helped control the impact of any issues. I wanted to avoid having too many agents working at once.
• Dog food as MVP demo. I found the easiest way to test the dispatcher was to give it the simple questions or tickets I get from customers daily. This let me check everything on low-risk items that were easy to verify after the agent finished. I call these ‘research’ cards.
• Risk classifier. Even though I was building an agent-based system, I decided that for version one the risk classifier should be rule-based rather than LLM-driven. The classifier decides if an agent’s merge request can be self-merged. Simple changes, like docs, can go through, but infrastructure changes still need a human to review.

Agents make things move fast, which is great. The problem is that speed lets you generate 2,000 lines of code in one session, even if it’s not the right direction. By pausing to write a memo before shipping code, I made it more likely that the agent’s code was correct.

The memo took about an hour to write. The full dispatcher MVP shipped end-to-end in about 10 merge requests in the same session. By bedtime, the agent had opened its first real merge request against the kanban. Writing a memo before coding is now my default for agentic development.

What changed

This small change has affected every part of how I work with agents now.

Before starting new work, I use the skill to make sure I think everything through first. Here’s my new workflow:

• Grill me. Step one is always a grill session, with one or more ADRs written to the project’s GitLab repository.
• Work Items. After the ADRs are created, the agent reviews them and makes clear work items in GitLab that any agent can use.
• Go. I use a skill called go that gives the agent instructions for the work I want done. It reviews work items in GitLab, creates worktrees, and gets started.

Why is this the agentic-AI failure mode

What I almost missed isn’t just about my Kanban board or me. Agentic coding makes it almost free to ship a feature, but it doesn’t lower the cost of shipping the wrong one. Those costs used to go together. When code was expensive, you could feel when you were heading in the wrong direction, and that friction slowed you down. Without that friction, the wrong work ships just as quickly and smoothly as the right work.

Velocity stops showing you’re on track and starts becoming noise that only feels like progress.

That’s how I almost shipped the wrong thing. Six features in two weeks made the project look like it was going well, because each part really was. The boards became more efficient, but the overall direction worsened, and my velocity didn’t warn me.