Minimum test matrix

Accept external capabilities one by one

File output

In WebChat, click the attachment or Markdown download link returned by the Agent, open the file, and verify its contents. A bare /oasn/... string, a file-exists log, or a screenshot of an internal path does not pass.

Plugin

openclaw plugins inspect <plugin-id> --runtime --json

The runtime surface must contain the expected Tool/hook. Then send a WebChat business request that is guaranteed to trigger it and inspect the result. Both checks are required.

MCP and A2A

Invoke every MCP Tool or openagent_agent_run operation actually used by the Program at least once, and check in order:

1. whether transport completed;
2. whether top-level isError reports an error;
3. whether structuredContent.status, or an equivalent business status, succeeded; and
4. whether the final text, file, or page artifact is genuinely usable.

“The Tool did not throw” is not a substitute for checking business status and artifacts. Retries of the same logical A2A request must reuse the original idempotency_key.

WebUI (only URLs verified by CLI 0.7.3)

Run oasn-sa-dev version webui-ports get --version-id <agent_version_id>. Use the returned current webui_urls only when both sync_status=ready and proxy_status=ready. Mapping readiness proves neither that the business process is listening nor that its page works. Open every URL and test the real page, business APIs, static assets, nested routes, and proxy subpath; for WebSocket applications, verify message round trips and reconnection.

HTTP 410 means the business proxy session is no longer valid; it does not mean the Owner OAuth session has expired. Query current port state to distinguish expired from revoked, then use an explicitly authorized workspace resume or webui-ports refresh only when its corresponding lifetime must be renewed. HTTP 502 normally means the mapped business service is not listening or cannot be reached: check that it is running on 0.0.0.0:<declared-port> before changing mappings.

Finally, make main return a real lowercase oasn-webui fenced block whose JSON is an object with exactly type, title, url, and defaultOpen. Verify that the client receives a projected view and preserves the path, query, and fragment. Invalid JSON, missing or extra fields, a wrong field type, or a non-exact fence remains ordinary text and is a failed WebUI test. A structurally valid block with an undeclared port, unsafe/private URL, or sensitive URL state fails closed with OPENCLAW_FINAL_REPLY_INVALID; later result-layer validation can report RUN_RESULT_INVALID or MCP_INVOCATION_RESULT_INVALID.

Test record format

Keep at least the following information for each test case. A single screenshot that merely looks successful is not sufficient:

Case ID:
Test time:
Workspace ID:              # May be recorded; never record a Token/private key
Workspace revision:
Input:
Attachments:
Expected:
Actual final response:
File/Tool/business-status evidence:
Conclusion: PASS | FAIL | NOT TESTED
Outstanding issues:

OpenAgent plugin testing: currently unsupported

The current development flow does not provide an entry point for finding and invoking the current draft version from an OpenAgent plugin. Do not search for, install, or invoke an unavailable test plugin, and do not present simulated output as plugin acceptance evidence.

• During development, use only WebChat for business validation.
• After publishing and going online, use the user_agent_url from the publish result, or find the Agent by its exact name in the website's public Agents list. On the public page, install it through Use in ChatGPT or Claude/WorkBuddy in the menu. Then find/select the current Agent by exact name in the actual client, verify its description and developer identity, and perform final acceptance.
• If the public page, installation dialog, or target client is unavailable, record BLOCKED/NOT TESTED. A development WebChat result cannot be promoted to a formal-invocation PASS.
• When a plugin test entry point is officially available, add separate search, selection, invocation, and result checks.

Go / No-Go before publishing