Black vs Crux

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

Crux is an AI personal assistant that works across your desktop, files, and apps while keeping your chats and data local to you. Download for Mac and Windows.

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About Crux

Crux is an AI personal assistant that works across your desktop, files, and apps while keeping your chats and data local to you. Download for Mac and Windows. Build me a spending report and fix my budget tracker. The full breakdown is in spending_report.md, and your tracker now flags any category running 10% hot. Nimbus gives you hyperlocal forecasts, radar, alerts and more. It works everywhere and it is really good, we promise. Hyperlocal forecasts, live radar, and alerts that arrive before the rain does.