A/B test
cleanup bot.
After an A/B test ends, engineers still need to trace its history and remove the unused code. I built a bot that gathers that context and opens a cleanup pull request for review.
Identified the problem and built the bot.
Product, engineering, and QA.
Proposed cleanup pull requests with human review.
Follow the work, step by step.
Start with the
completed ticket.
The bot starts with the concluded experiment ticket. It uses the result and related work to understand what needs to be cleaned up.
Illustrative workflow. Company code, tickets, and internal data are omitted.
The decisions behind the workflow.
Cleanup was a coordination problem as well as a coding task. I focused on the preparation and handoffs around the work.
Understand the handoffs.
PMs had to work out which areas were affected and coordinate frontend and backend cleanup. Engineers also had to reconstruct experiments that could have run for months. That preparation could push the work into another planning cycle.
Start with the completed test.
I made the experiment ticket the starting point. The bot gathers its history and related changes before proposing cleanup, giving the work a clearer starting point for review.
Keep judgment with people.
Engineers and QA review the proposed changes before merge. A person also approves or declines lessons from PR feedback, because reviewers can have different coding preferences.
I built a workflow that connects experiment context to repository changes, then carries approved feedback into future runs.
Connect the history.
The bot uses MCP to read issue context and consults repository commits to trace the experiment and later changes. Team rules help it recognize frontend work.
Prepare a reviewable change.
With repository access, the bot finds the relevant code, creates a branch, and opens a cleanup pull request. Engineering and QA review that proposal before it is merged.
Make feedback reusable.
I built a step that turns PR comments into proposed lessons. Approved lessons become part of the context for future cleanups, giving users control over which guidance carries forward.
A cleanup proposal built from experiment history, ready for the team to review, plus an approval step for feedback used in future runs.
A review comment becomes a proposed lesson.