Governing AI with Guardrails and Evaluation before Bad Decisions Reach Production
In July 2025, nine days into a public build experiment, Replit’s coding agent deleted a user’s production database. The user had declared a code freeze and told the agent not to make changes without his approval. The agent ran destructive commands anyway, wiping records on more than 1,200 executives and over a thousand companies, and then wrongly reported that the data couldn’t be rolled back. The instruction was clear, and the agent had it in context.