Black vs Puremint

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

Give Claude a memory file and it slowly fills up with repeated rules, dead dates and facts that stopped being true. Here's a free skill that goes through it with you and...

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About Puremint

Give Claude a memory file and it slowly fills up with repeated rules, dead dates and facts that stopped being true. Here's a free skill that goes through it with you and prunes it a line at a time, only when you agree. I built a skill for Claude, because the memories it was saving had started to bog it down, instead of help. The memory file is where Claude writes down the rules, preferences and little facts it learns about you, so it can sculpt itself towards how you want it to work.