Anne-bot

Problem

PRD Reviews Looked Rigorous—Alignment Quietly Lagged

PRD reviews were thoughtful, Socratic, and widely respected. Standards were high, and documents were reviewed carefully by experienced leaders.

Despite this rigor, review cycles frequently stretched across multiple rounds—not because PMs were unprepared, but because foundational alignment was unresolved when reviews began. Much of the discussion centered on clarifying the problem itself. Definitions drifted. Assumptions surfaced late.

Because reviews ran on a fixed cadence, even a single additional revision round translated into real delay. The process appeared slow, but the underlying issue was that alignment entered the room too late.

Insight

Review Speed Wasn’t the Constraint—Shared Criteria Were

The bottleneck was not review capacity, tooling, or director availability. It was the absence of shared, explicit criteria for what constituted a strong problem statement—causing PMs and leadership to arrive at reviews solving different epistemic tasks.

Until the problem definition stabilized, reviewing anything else was effectively performative. Scope, metrics, and prioritization could not be meaningfully evaluated without first aligning on what problem was actually being addressed.

Structural Solution

Coaching Was Constrained to Problem Statements Before Broader Review

Anne-bot was designed as a constrained mentorship system focused narrowly on strengthening problem statements through Socratic questioning.

Rather than issuing evaluations or edits, it surfaced assumptions, gaps, and reasoning errors that typically emerged late in reviews.

Just as important were the deliberate non-solutions:

• No auto-generated PRDs
• No scoring or approval signals
• No replacement of director judgment

Human-in-the-loop transparency made context gaps explicit instead of masking them. Where alignment was missing, automation was intentionally deferred.

WHY THIS MATTERED BEFORE SCALE: AI coaching tools fail when they erode trust faster than they save time. If Anne-bot had confidently delivered feedback without acknowledging what it didn't know, PMs would have dismissed it after the first irrelevant suggestion—and the tool would have been abandoned before surfacing anything useful.

The real adoption gate wasn't AI capability. It was organizational readiness: whether shared criteria existed for the AI to codify. Without that foundation, automation would scale confusion, not clarity.

By treating the prototype as a diagnostic instrument rather than a production investment, the work avoided the most common failure mode in AI tooling: building confidently on ambiguous ground.

Synthesis: The decision to pause—not ship—was the output that mattered most.

Key Strategic Decisions

The focus was on doing things in the right order, and using AI to sharpen thinking rather than replace judgment.

Impact At a Glance

The POC revealed that context, not model capability, was the constraint on AI-assisted coaching. Two weeks of discovery and prototyping across 10+ PRDs with 6 PMs validated an architectural approach: when organizational knowledge is unavailable, human-in-the-loop interaction becomes the interface for context-gathering, not a limitation to overcome.

Quantitative Signals

These signals indicate the bottleneck was structural misalignment, not review capacity or tooling speed. The shift to problem-statement focus with explicit context-gathering checkpoints proved feasible and relevant—validating the architectural approach.

What Did Not Happen:

• No approval cycles accelerated solely due to AI speed (the constraint was alignment, not velocity)
• Comprehensive PRD review feedback did not improve adoption (confirmed scope-reduction was necessary)
• Directorial judgment was not replaced (preparation was sufficient)

Why This Impact Was Durable

The validation was structural, not circumstantial. Three independent methods—observation, artifact analysis, PM interviews—converged on the same findings. The human-in-the-loop approach is replicable because it treats context-gathering as a design requirement, not a workaround.

How This Sharpened My Judgment

Lessons from Real-World Use

• Context is the primary constraint on AI relevance in judgment-heavy domains. I designed around missing organizational knowledge by making context-gathering interactive—human-in-the-loop became the architecture.
• Scope reduction can be more impactful than comprehensive solutions. Full PRD feedback created noise; problem-statement focus created signal with fewer context dependencies.
• Acknowledging uncertainty produces more trusted guidance. Iteration 2 succeeded because the assistant admitted what it didn't know instead of guessing.
• The leverage point is often upstream of where you're building. The real constraint was shared criteria, not feedback speed—a foundational problem, not a tooling problem.
• Testing can reveal what not to build, not just what to build. The prototype's failures taught me that comprehensive coverage was strategically wrong.

A Transferable Pattern