Black vs Testflight

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

Using TestFlight is a great way to help developers test beta versions of their apps.

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About Testflight

Using TestFlight is a great way to help developers test beta versions of their apps. Mic Captions turns spoken audio into easy-to-read captions in real time. Use your iPhone or iPad microphone to follow conversations, lectures, meetings, tv shows, and other spoken content. Mic Captions can also translate captions, save sessions for later review, replay recorded audio alongside the transcript, and help you navigate saved content with Topics and Words. Note: Developers can choose to set criteria for their beta, such as a preferred device or OS.