prowler-test-mcp - Claude MCP Skill

## Critical Rules

- ALWAYS drive tools through an in-memory client: `async with Client(mcp_root_server)`.
  Tool parameters use pydantic `Field(default=...)`, and only FastMCP's wrapper
  resolves those defaults. Calling a tool method directly with an argument omitted
  leaves it as a raw `FieldInfo` — which is truthy, so `if email:` silently builds
  a filter out of the `FieldInfo` repr. Direct calls MUST pass every argument.
- NEVER open a `fastmcp.Client` inside a fixture. FastMCP warns this causes
  hard-to-diagnose event-loop issues; open it inline in the test.
- ALWAYS use the `mock_api_client` fixture; NEVER construct a `ProwlerAPIClient`.
  Tool instances captured the singleton by reference at import time, so only an
  in-place patch of `.client` reaches them.
- NEVER clear `SingletonMeta._instances`. It orphans every registered tool on an
  instance holding a real `httpx.AsyncClient`. Use `isolated_api_client` if you
  genuinely need a fresh instance.
- NEVER strip `PROWLER_API_KEY`. Tools are built at import time and a construction
  failure is swallowed, so the whole `prowler_*` namespace silently drops to zero
  tools. It is pinned in `[tool.pytest_env]`.
- For `ProwlerAppAuth`, pass `mode=` / `base_url=` explicitly. Those are resolved in
  default arguments, evaluated once at module import, so `monkeypatch.setenv` has
  no effect on them.
- NEVER assert an exact tool count — every future branch would have to bump it.
- Assert on `result.data` (structured output), not `result.content[0].text`.
- Tests are `test_*.py` (prefix), like the API — not the SDK's `*_test.py` suffix.
- `__init__.py` IS required in every `tests/` subdirectory here (unlike the SDK's
  repo-root `tests/`), or same-named modules collide under pytest's import mode.
- Async tests need no marker (`asyncio_mode = "auto"`). Do not use `@pytest.mark.anyio`.
- Use only obviously-fake credentials from `tests.helpers.tokens` (TruffleHog).
- One behaviour per test; keep tests self-contained and order-independent.

---

## 1. Layout

Mirror the source tree *below the package root* — drop the `prowler_mcp_server/`
level, exactly as the SDK maps `prowler/providers/...` to `tests/providers/...`.
So `prowler_mcp_server/prowler_app/tools/` is tested in `tests/prowler_app/tools/`.

```text
mcp_server/tests/
├── conftest.py                  # all shared fixtures
├── helpers/                     # jsonapi.py, http.py, assertions.py, tokens.py
├── test_server.py               # mounted-server contract
├── test_health.py
├── prowler_app/{models,tools,utils}/
├── prowler_hub/
└── prowler_documentation/
```

---

## 2. Fixtures

| Fixture | Autouse | What it gives you |
|---------|---------|-------------------|
| `_pinned_environment` | yes | Deterministic env; blocks a developer's `.env` from leaking |
| `_no_real_network` | yes | Any real socket connect raises `RuntimeError` |
| `_singleton_registry_guard` | yes | Snapshots/restores `SingletonMeta._instances` |
| `mock_router` | no | Route registry + request recorder |
| `api_client` | no | The live `ProwlerAPIClient` singleton |
| `mock_api_client` | no | **The workhorse** — singleton with a mocked transport |
| `isolated_api_client` | no | Evicts the singleton, for construction/identity tests |
| `mcp_root_server` | no | The mounted root server (session-scoped) |
| `health_client` | no | Starlette `TestClient` for `/health` |
| `http_request_headers` | no | Injects headers for HTTP-mode auth |
| `hub_router` | no | Mocks the Hub sub-server's two sync clients |
| `docs_router` | no | Mocks the docs search engine's two sync clients |

### `MockRouter`

```python
mock_router.add("GET", "/api/v1/users", json=jsonapi_collection([...]))
mock_router.add("GET", "/api/v1/tasks/t1", json=task_document("t1", "completed"))

mock_router.request_for("GET", "/api/v1/users")     # last request, for header asserts
mock_router.query_params("GET", "/api/v1/users")    # decoded query string
mock_router.paths()                                 # everything requested so far
```

Register a route more than once to return a sequence — the last response repeats.
That is how you drive `poll_task_until_complete` (`executing`, `executing`,
`completed`). An unregistered request raises, listing what *was* registered.

---

## 3. Patterns

**Tool test** — see `assets/mcp_tool_test.py`. Register routes, call through the
in-memory client, assert on `result.data` *and* on the recorded request. When a
tool chooses between endpoints, assert `mock_router.paths()` — a wrong choice is
invisible in the response body.

**Model test** — see `assets/mcp_model_test.py`. Build the document with the
`jsonapi` helpers, run `from_api_response()`, assert on both the model and
`model_dump()`. `MinimalSerializerMixin` makes those differ, and an absent
relationship (`None`) must never be conflated with an empty one (`[]`).

**Contract test** — see `assets/mcp_contract_test.py`. Namespacing and
description coverage across every registered tool.

The worked example in the repo is `findings`, covered across both layers in
`tests/prowler_app/{models,tools}/test_findings.py`. Read those first — they
exercise every foundation capability in one feature.

### Reading coverage

Coverage has a meaningless high floor. Model modules are almost entirely class-body
`Field(...)` declarations that execute at import, and `prowler_app/server.py` imports
every model module at import time. **Importing the package with zero tests already
reports 36% overall**, and individual model modules 54–84%.

So a model module at ~68% with no tests has none of its logic covered — the missing
ranges are the `from_api_response()` bodies, which is the only part worth testing.
Compare against the import-only floor, never against zero, and do not set a Codecov
target from the raw total.

### Where fixture data lives

`tests/helpers/` is feature-agnostic and must stay that way: it holds the JSON:API
*shape*, not any feature's data. Per-feature attribute dictionaries
(`FINDING_ATTRIBUTES`, `CHECK_METADATA`, …) belong as module-level constants in
the test module that uses them. Do not add feature fixtures to `helpers/`.

---

## 4. Commands

From `mcp_server/`:

```bash
cd mcp_server

uv run pytest                              # whole suite
uv run pytest tests/prowler_app/models     # one area
uv run pytest --cov=./prowler_mcp_server   # with coverage
```

From the repository root:

```bash
make test-mcp   # runs the MCP suite exactly as CI does
```

---

## 5. Reference

- Fixtures and the reasoning behind them: `mcp_server/tests/conftest.py`
- Testing section of `docs/developer-guide/mcp-server.mdx`
- Official FastMCP testing guide: <https://gofastmcp.com/development/tests>