Objective
The Objective tab is the first tab on every test suite. It captures the suite's type and its overarching goal, and both drive how the suite runs and how its results are judged.
Suite type
Every suite is either:
- API Workflow — steps are HTTP requests, evaluated against the response.
- UI Workflow — steps are browser actions described in natural language, evaluated against what a user would see.
Suite type is chosen at creation and can't be changed afterward — Flow contents differ fundamentally between the two. If you need the other type, create a new suite.
Fields
| Field | Required | What it's for |
|---|---|---|
| Suite name | Yes | Identifies the suite |
| Suite type | Yes | API Workflow or UI Workflow — locked after creation |
| Description | No | A short summary of what the suite covers |
| Intent | No | The goal a real user should be able to accomplish |
| Pass threshold | No (default 100%) | The percentage of Flow runs whose Intent Evaluation must pass for the suite to be considered passing |
The first time you focus the Intent field, it pre-fills from Description if Description is set and Intent is still empty. After that first focus, the two fields are independent — editing one doesn't touch the other.
An example Intent: "A first-time visitor should be able to understand what the product does and request early access without confusion, missing information, or dead ends."
How Intent Evaluation works
The Intent is threaded into evaluation in two places:
- Per-step qualitative checks — as each check runs during a Flow, the Intent is included as context, so the evaluator judges the response with the suite's overall goal in view rather than in isolation.
- End-of-run Intent Evaluation — after all steps in a Flow complete, an evaluator reviews the Intent alongside every step's results, extracted values, and check outcomes, and makes a holistic pass/fail judgment with reasoning.
This second step is what determines a Flow run's pass/fail — not a raw percentage of checks passing. A failed check doesn't automatically fail the run, and all checks passing doesn't automatically pass it: the Intent Evaluation can conclude the overall goal was still achieved despite an individual failure, or that it wasn't, using the Intent's own language to capture that nuance.
If a suite has no Intent set, its runs show as Pending in Results — there's nothing to evaluate the run against — until one is added.
A step that couldn't run at all (marked Blocked — an execution error, not a failed check) doesn't automatically stop Intent Evaluation from running. As long as at least one step in the Flow actually produced a result — passed or failed — the evaluator still weighs everything, blocked steps included, and reaches a verdict. A run only shows no verdict when nothing in it produced any usable signal at all.
On Failure settings on individual checks (continue / stop_request / stop_flow) are unaffected by any of this — they still control whether execution stops or continues past a failed check. That's a separate concern from what the Intent Evaluation concludes about the result.
Suite-level pass threshold
The suite's Pass threshold is the percentage of Flow runs — across all Test Case × Scenario combinations in the suite — whose Intent Evaluation passed. This is the same threshold Collections check when rolling up suite results.
See Assertions for how individual checks work, and Collections for how suite-level pass/fail rolls up further.