GhostQAGhostQA

Example Workflow

• Persona-based testing -- Define AI users with backgrounds, goals, frustrations, and tech comfort levels. They don't just follow scripts; they react to what they see.
• Vision-powered -- No selectors, no DOM queries. The AI interprets screenshots like a human would. Catches visual/layout issues that selector-based tests miss entirely.
• YAML-configured -- Products, personas, and journeys are all YAML files. PMs can read them. No code to maintain.
• Budget enforcement -- Per-run, per-day, and per-month cost caps. The engine hard-stops if you hit the limit. No surprise bills.
• JUnit XML output -- Drop --junit-xml results.xml and plug it into any CI system.
• Tiered model routing -- Haiku for cheap navigation, Sonnet for complex reasoning, optional local Ollama for zero-cost simple actions.
• Multi-platform -- Web apps (via Playwright), macOS native apps (via Accessibility API + pyobjc), iOS Simulator (via simctl). Same YAML format, different runners.
• Evidence collection -- Every run produces screenshots, a findings report, cost breakdown, and a structured JSON result. Everything is saved to an evidence directory.
• Stuck detection -- If the AI repeats the same action or the UI stops changing, the engine escalates to a stronger model, then aborts if nothing works. No infinite loops.
• Template variables -- Use {{persona.credentials.email}} in your journey steps. Variables resolve from persona configs at runtime.
• Precondition checks -- Verify services are up before running tests. Fail fast with clear errors instead of wasting API calls.

Limitations

• Requires an Anthropic API key. No API key, no testing. There's no free tier built into SpecterQA itself.
• Costs money. Every run makes API calls. A typical 3-step journey costs $0.30-0.60. Budget enforcement prevents surprises, but the meter is always running.
• Vision models aren't perfect. The AI sometimes misreads small text, clicks the wrong element, or gets confused by complex layouts. It's good, not infallible. You'll occasionally see false positives and false negatives.
• Not a replacement for unit tests. SpecterQA tests behavioral UX flows. It doesn't test your business logic, data integrity, or edge case handling. Use it alongside your existing test suite, not instead of it.
• macOS native testing requires pyobjc. The specterqa[native] extra pulls in pyobjc packages (~200MB). Only needed for native macOS and iOS Simulator testing.
• Alpha software. Version 0.4.0. APIs may change. File structure may change. Expect rough edges.
• Single-persona per journey (for now). Multi-persona concurrent testing (e.g., simulating a chat between two users) is on the roadmap but not yet supported.
• Deterministic reproduction is hard. Because the AI makes decisions at runtime, the exact sequence of actions varies between runs. Same journey, same persona, slightly different clicks. This is by design (it catches more issues) but makes exact reproduction tricky.

Requirements