Black vs Genosyn

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

Open-source, self-hosted software for running a company with AI Employees. They hold real roles, work on their own schedule through the night, and stop for a Member only...

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About Genosyn

Open-source, self-hosted software for running a company with AI Employees. They hold real roles, work on their own schedule through the night, and stop for a Member only when a job genuinely needs a person. Every line is a Routine on a schedule: what it did, which product it did it in, and the one hour it stopped and put a question in front of a person. Each account is scored against the ideal-customer profile written into Robin's Soul.