3.7 KiB
Adversarial API testing
Every request in this suite goes to the application under test. Nothing here probes a host service, scans a network, or reaches anything ModelForge does not own.
What is sent
malformed bodies empty, truncated, non-JSON, bare scalars, NaN, Infinity, null bytes,
200-deep nesting
wrong types integers, nulls, arrays, objects and booleans where a string is required
oversized input 100 KB identifiers, 5 MB bodies, 5 KB enum values
mass assignment state, restore_eligible, verified_at, encryption_key, id
hostile strings traversal, encoded traversal, absolute and UNC paths, file:// and
link-local URLs, SQL, template and JNDI expressions, shell metacharacters,
script and image XSS payloads, bidi overrides, astral-plane and emoji runs
identifiers malformed UUIDs, SQL appended to a path segment, 500-character values
query bounds zero, negative, out-of-range, non-numeric and injected limits
headers CR/LF and null-byte injection into a correlation id
credentials none, empty, wrong scheme, wrong case, oversized, null-byte, injected
What must happen
A rejected value and a stored-but-inert value are both acceptable. A 5xx is not: it means input reached somewhere it should not have.
400 / 422 malformed or wrongly typed, with a typed error envelope
403 authenticated but out of scope
404 well-formed identifier that does not exist
413 / 422 oversized
never 5xx for any input in the matrix
Every error response carries code, message and correlation_id, and nothing else: no
traceback, no SQL, no driver name, no DSN, no file path, no credential. This is asserted directly,
including with the database stopped — a dependency failure is exactly when a framework is most
likely to leak its internals.
Scope isolation
The capability a credential names is matched exactly. Case changes, appended whitespace, a second
capability in the same string and zero-width characters do not widen it. A contract version is part
of the identity: a client scoped to rag.embedding@1 cannot reach rag.embedding@2.
A client bound to a project may only serve that project's binding, even when its capability list would otherwise allow more, and a deprecated binding stops serving immediately.
No enumeration oracle
An unknown credential, a revoked one, a near-miss differing in the last character and an empty one all produce the same status and the same code. A caller cannot learn whether a guess was close.
Mass assignment
Request models are extra="forbid", so a caller cannot write a field the contract never offered.
Attempting to set state, restore_eligible, verified_at or an id is a validation error rather
than a silently ignored field — silently ignoring is how these become exploitable later.
Path handling
Backup identities become directory names, so the identity pattern refuses traversal outright rather than relying on later sanitisation. Restore destinations are parsed and identifier-validated: non-PostgreSQL schemes, SQL appended to a database name, an empty database and a null byte are all refused. The M15 allowlisted-root resolution remains the second layer.
A defect this suite found
Python's JSON parser accepts NaN and Infinity; its serialiser rejects them. The validation
handler echoed the rejected value straight back into the error response, so a body containing one
made the error response itself fail to serialise — turning a 422 into a server error, on input that
needs no credential to send. Rejected values are now rendered safely and truncated, since rejected
input is unbounded by definition.