Adversarial API testing

Every request in this suite goes to the application under test. Nothing here probes a host service, scans a network, or reaches anything ModelForge does not own.

What is sent

malformed bodies      empty, truncated, non-JSON, bare scalars, NaN, Infinity, null bytes,
                      200-deep nesting
wrong types           integers, nulls, arrays, objects and booleans where a string is required
oversized input       100 KB identifiers, 5 MB bodies, 5 KB enum values
mass assignment       state, restore_eligible, verified_at, encryption_key, id
hostile strings       traversal, encoded traversal, absolute and UNC paths, file:// and
                      link-local URLs, SQL, template and JNDI expressions, shell metacharacters,
                      script and image XSS payloads, bidi overrides, astral-plane and emoji runs
identifiers           malformed UUIDs, SQL appended to a path segment, 500-character values
query bounds          zero, negative, out-of-range, non-numeric and injected limits
headers               CR/LF and null-byte injection into a correlation id
credentials           none, empty, wrong scheme, wrong case, oversized, null-byte, injected

What must happen

A rejected value and a stored-but-inert value are both acceptable. A 5xx is not: it means input reached somewhere it should not have.

400 / 422   malformed or wrongly typed, with a typed error envelope
403         authenticated but out of scope
404         well-formed identifier that does not exist
413 / 422   oversized
never 5xx   for any input in the matrix

Every error response carries code, message and correlation_id, and nothing else: no traceback, no SQL, no driver name, no DSN, no file path, no credential. This is asserted directly, including with the database stopped — a dependency failure is exactly when a framework is most likely to leak its internals.

Scope isolation

The capability a credential names is matched exactly. Case changes, appended whitespace, a second capability in the same string and zero-width characters do not widen it. A contract version is part of the identity: a client scoped to rag.embedding@1 cannot reach rag.embedding@2.

A client bound to a project may only serve that project's binding, even when its capability list would otherwise allow more, and a deprecated binding stops serving immediately.

No enumeration oracle

An unknown credential, a revoked one, a near-miss differing in the last character and an empty one all produce the same status and the same code. A caller cannot learn whether a guess was close.

Mass assignment

Request models are extra="forbid", so a caller cannot write a field the contract never offered. Attempting to set state, restore_eligible, verified_at or an id is a validation error rather than a silently ignored field — silently ignoring is how these become exploitable later.

Path handling

Backup identities become directory names, so the identity pattern refuses traversal outright rather than relying on later sanitisation. Restore destinations are parsed and identifier-validated: non-PostgreSQL schemes, SQL appended to a database name, an empty database and a null byte are all refused. The M15 allowlisted-root resolution remains the second layer.

A defect this suite found

Python's JSON parser accepts NaN and Infinity; its serialiser rejects them. The validation handler echoed the rejected value straight back into the error response, so a body containing one made the error response itself fail to serialise — turning a 422 into a server error, on input that needs no credential to send. Rejected values are now rendered safely and truncated, since rejected input is unbounded by definition.

3.7 KiB Raw Blame History