Black vs LastShelf

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

Prepare documents, passwords, contacts, and critical bills now, so your family has step by step instructions when they need it most.

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About LastShelf

Prepare documents, passwords, contacts, and critical bills now, so your family has step by step instructions when they need it most. Make the hard day easier for the people you love, LastShelf Skip to content . LastShelf provides step-by-step instructions so your family does not have to hunt through drawers, post-it notes, and safe deposit boxes to track down documents, passwords and important bills. Eight quick questions document the contacts, insurance, home details, accounts, documents, and debts your family would need.