Black vs VocalCode

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

VocalCode, also written Vocal Code, is push-to-talk voice input for AI coding on Windows and Apple-silicon Mac. On-device speech recognition, $4.99 once, no subscription.

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About VocalCode

VocalCode, also written Vocal Code, is push-to-talk voice input for AI coding on Windows and Apple-silicon Mac. On-device speech recognition, $4.99 once, no subscription. What it looks like A capsule appears, you speak, the words are already in the file. Responsiveness varies with hardware, audio length, language and model; we do not publish an unsupported latency promise. Initial trial setup, activation, periodic licence refresh, model downloads and update checks use the internet. Parakeet detects European languages automatically; Chinese and English offer measured CPU choices; SenseVoice handles Korean and Japanese and is the balanced default for Chinese and English; Hindi uses the tested Qwen3-ASR route.