Assertions

Assertions validate that your API response meets expected criteria. A test passes only when all assertions pass.

Overview

Each test can have multiple assertions. Assertions are evaluated in order, and you can configure whether to stop on the first failure or continue checking all assertions.

Assertion Types

Status Code

Validate the HTTP response status code.

Pattern	Matches
200	Exactly 200
2xx	Any 200-299
200,201	200 or 201
2xx,3xx	Any 200-399

Examples:

• 2xx - Any successful response
• 200,201,204 - Common success codes
• 200 - Exactly 200 OK

Response Time

Ensure the response completes within a threshold.

• Threshold: Maximum time in milliseconds

Example: 500 fails if response takes longer than 500ms.

Use this to catch performance degradation before it impacts users.

JSON Path

Extract a value from JSON responses and compare it.

• Path: JSONPath expression (e.g., $.status, $.data[0].id)
• Expected: The value to match

Examples:

Path	JSON	Result
$.status	{"status": "ok"}	ok
$.count	{"count": 5}	5
$.items[0].name	{"items":[{"name":"first"}]}	first
$.data.enabled	{"data":{"enabled":true}}	true

Contains

Check if the response body contains a specific string.

• Text: The string to search for

Example: "healthy" passes if the response contains the word "healthy" anywhere.

This is case-sensitive. Use for simple text matching when JSONPath isn't needed.

Regex Pattern

Match the response body against a regular expression.

• Pattern: A valid regex pattern

Examples:

Pattern	Matches
"status":\s*"ok"	"status": "ok" or "status":"ok"
version.*[0-9]+\.[0-9]+	version: 1.0, version 2.3
(?i)success	Success, SUCCESS, success

Header Exists

Verify a specific header is present in the response.

• Header Name: The header to check (case-insensitive)

Examples:

• Content-Type - Verify content type is set
• X-Request-Id - Verify request tracking header
• Cache-Control - Verify caching headers

Header Value

Verify a header has a specific value.

• Header Name: The header to check
• Expected Value: The expected value

Stop on Failure

When enabled, the test stops evaluating remaining assertions after a failure. This is useful when:

• Later assertions depend on earlier ones
• You want to avoid unnecessary processing
• The first failure is the most important

When disabled, all assertions are evaluated and all failures are reported.

MCP Server Assertions

These assertions are available for MCP Server tests and evaluate the results of MCP protocol discovery.

Tool Count

Verify the server advertises at least a minimum number of tools.

• Expected: Minimum tool count (default: 1)

Example: 3 passes if the server has 3 or more tools.

Tool Exists

Verify a specific tool is available on the server.

• Expected: Tool name (case-insensitive)

Example: calculator passes if the server advertises a tool named "calculator".

Resource Exists

Verify a specific resource URI is available on the server.

• Expected: Resource URI (case-insensitive)

Example: file://config.json passes if the server exposes that resource.

Capability Check

Verify the server advertises a specific capability.

• Expected: Capability name (e.g., "tools", "resources", "prompts")

Example: tools passes if the server's initialize response includes the "tools" capability.

Prompt Exists

Verify a specific prompt template is available on the server.

• Expected: Prompt name (case-insensitive)

Example: summarize passes if the server advertises a prompt named "summarize".

A2A Agent Assertions

These assertions are available for A2A Agent tests and evaluate the agent card and probe task results.

Skill Exists

Verify the agent card includes a specific skill.

• Expected: Skill ID (case-insensitive)

Example: text-summarization passes if the agent card lists that skill.

Task Completion

Verify a probe task completes successfully.

Requires the "Send Probe Task" option to be enabled in test configuration. Passes if the probe task reaches the "completed" state.

Agent Card Valid

Verify the agent card contains all required fields (name).

No configuration needed. Passes if the agent card is valid JSON with a name field.

Capability Check

Verify the agent supports a specific capability.

• Expected: Capability name (e.g., "streaming", "pushNotifications")

Example: streaming passes if the agent card advertises streaming support.

GraphQL Assertions

These assertions are available for GraphQL tests and evaluate the results of schema introspection and canary queries.

Type Exists

Verify a schema type exists.

• Expected: Type name (e.g., "User"). Case-insensitive.

Example: User passes if the introspection result includes a type named "User".

Field Exists

Verify a type.field exists.

• Expected: "TypeName.fieldName" (e.g., "User.email"). Case-insensitive.

Example: User.email passes if the "User" type has an "email" field.

Query Success

Verify canary query returns data without errors.

Requires the canary query to be configured in test settings. Passes if the query response contains a data field and no errors field.

Max Deprecated Fields

Verify deprecated field count is within threshold.

• Expected: Maximum count (e.g., "5")

Example: 5 passes if the schema contains 5 or fewer deprecated fields.

Schema Unchanged

Verify schema hash matches expected value.

• Expected: Schema hash string. Detects schema drift.

Example: a1b2c3d4 passes if the computed schema hash matches. Useful for detecting unexpected schema changes between deployments.

gRPC Assertions

These assertions are available for gRPC tests and evaluate the results of health checks and service discovery.

Health Serving

Verify gRPC health check returns SERVING status.

No configuration needed. Passes if the health check response status is SERVING.

Service Exists

Verify a service is registered.

• Expected: Fully qualified service name (e.g., "myapp.UserService")

Example: myapp.UserService passes if the service is registered and responding to health checks.

Method Exists

Verify a method exists in a service.

• Expected: Method name (e.g., "GetUser")

Example: GetUser passes if the method is found via server reflection.

Service Count

Verify minimum number of registered services.

• Expected: Minimum count (e.g., "3")

Example: 3 passes if the server has 3 or more registered services.

Best Practices

Order Matters

Put the most important assertions first:

1. Status code (is the request successful?)
2. Response time (is it fast enough?)
3. Content validation (is the data correct?)

Start Simple

Begin with basic assertions:

Status Code: 2xx

Add more specific assertions as needed:

Status Code: 200
Response Time: < 500ms
JSON Path: $.status = "healthy"

Avoid Over-Asserting

Don't assert on volatile data:

• Timestamps
• Random IDs
• Counts that change frequently

Focus on stable indicators of health.