The component, defined before it is built.

§ 1

Component identity

The adapter is registered as an infrastructure agent — slug mamba-nemotron-agw-adapter, version 0.1.0, owned by yawningmonsoon. It maps to the LLM Gateway slot of the Agent OS framework as a model-runtime upstream and registers itself in the Model Registry for the Nemotron variants it serves.

Distribution is specified as a cosign-signed OCI image and a Helm chart published over OCI. Neither is published today. Licence is BSL 1.1 with automatic conversion to Apache 2.0 on 2029-05-06. The component is pre-release: the specification is complete, the implementation is not started.

§ 2

Problem & wedge

Every gateway that proxies LLM calls today routes well to OpenAI, Bedrock, Azure OpenAI and Vertex. None ships a first-class governed adapter for the NVIDIA Nemotron family.

Institutions that bought on-premises GPU capacity are forced into a binary: spend hyperscaler tokens despite paid-for local inference, or wire bespoke Triton glue and lose every governance guarantee the framework otherwise gives them. This specification describes the component that would close that gap.

§ 3

Architecture

The request path is short: caller → gateway → adapter → Triton. The adapter is stateless across requests; rate-limit state lives in Redis, quota state in DynamoDB.

The emit path runs asynchronously off the request: an audit event to an Object Lock store, an OpenLineage RunEvent to Marquez, and OpenTelemetry traces, metrics and logs to a collector. The adapter buffers emits in-process with a hard byte cap and returns 503 rather than dropping events on overflow — auditability is the hard guarantee, latency and availability give before it does.

§ 4

Birthing manifest

The manifest is a Kubernetes CRD of kind Agent that the Birthing Engine admission webhook validates. It references the certificate, the capability CRD, the model registry entries for each Nemotron variant, the guardrail bundle and the evaluation pipeline.

The webhook rejects the manifest unless every reference resolves. Registration is a pull request — no out-of-band creation paths exist.

§ 5

Capability declaration

Every external call surface is declared up front: Triton over gRPC; storage to Redis, DynamoDB and two object-store buckets in write-only mode with object lock required; ingress only from the gateway namespace.

The OPA sidecar enforces this at runtime. Anything not on the list returns 403 to the caller and emits a capability_violations metric plus an audit event. There are no exceptions.

§ 6

Compliance mapping

The table below is the control-by-control mapping the headline promises an auditor can check. Each row names the artifact that satisfies the control — an audit event, a lineage record, a certificate field, an OPA policy or a signed evaluation result.

Mapped, not asserted. A mapping states where evidence would be found; it does not claim an audit has been performed. Nothing here has been assessed by a third party.

• EU AI Act

  Articles 12 (logging), 13 (transparency), 15 (accuracy & robustness)

• NIST AI RMF 1.0

  GOVERN-1.1, MEASURE-2.7, MANAGE-4.1

• ISO/IEC 42001:2023

  Clauses 8.3 (operations) and 9.1 (monitoring)

• SOC 2

  CC6.1 (mTLS, KMS) and CC7.2 (immutable audit) — scope not yet confirmed

• NAIC Model Bulletin

  §4.2 governance — owner, COE approval, evaluation evidence

• OpenLineage

  Standard RunEvent shape; ingests in DataHub, Atlan, Collibra, Purview

§ 7

Guardrail configuration

Two layers, both required. Layer A is a Bedrock-Guardrails-compatible pre and post filter — denied topics, content categories, PII detection, regex packs from mamba-types. Layer B is an OPA Rego bundle for runtime enforcement: certificate-bound model admission, per-line-of-business token budgets, SPIFFE identity gating.

Boundary-case testing is not optional. A 412-prompt suite covers jailbreaks, prompt injection, PII smuggling, capability escalation and system-prompt extraction. CI fails on any single bypass.

§ 8

8-Dimension evaluation

Eight dimensions, eight pass criteria, run on every release candidate and on a 30-day rolling cadence in production. Any single FAIL halts release. This is the asset the site leads with: the criteria are published before the implementation exists, so they cannot be tuned to the result.

1. 1Accuracy & Quality — LLM-as-judge on a 1,200-pair benchmark; ≥ 0.85 mean with no regression greater than 0.03
2. 2Security — Garak red-team plus the 412-prompt boundary suite; zero successful jailbreaks, PII leaks or system-prompt extractions
3. 3Infrastructure — OPA, Checkov and CDK Nag against pod spec and IAM; cosign-verified images; no privileged containers
4. 4Regulatory — mapped controls evaluate compliant against the Compliance Standards Library
5. 5Data Governance — 100% of test calls produce a complete OpenLineage record
6. 6Guardrail Adherence — boundary suite, zero bypasses
7. 7Capability Governance — static analysis of declared versus eBPF-traced call surface; exact match required
8. 8Auditability — 100% of calls produce a queryable, signed audit record within five seconds

§ 9

Certificate JSON

The certificate is the immutable record that proves the component passed every gate. It references the manifest, training-data version, models on the registry, guardrail config, capability list, evaluation scores and KMS-signed evaluation artifacts.

Revocation is append-only — original certificates are never deleted from the Object Lock store. The gateway re-validates the certificate on a five-minute cadence and refuses to route to a revoked adapter.

§ 10

Runtime deployment

The chart is specified to publish a Deployment (three replicas, anti-affinity, PDB), a Service, a ServiceMonitor, a NetworkPolicy, ConfigMaps for guardrail config and OPA bundle, a Secret for Triton mTLS and an OpenTelemetry collector sidecar. No chart is published today.

Resource budget per replica is small — 500m CPU and 1 GiB memory requested, 2 CPU and 4 GiB limit. The adapter is intentionally cheap to scale horizontally; the cost of inference lives in the GPU pool.

§ 11

Observability

OpenTelemetry-first and vendor-neutral. Prometheus scrapes /metrics; Grafana dashboards ship with the chart for overview, per-line-of-business cost attribution, guardrail health and Triton saturation.

Metrics are labelled by agent_cert_id, line of business, model and status, so every dollar of inference is attributable to a calling agent.

§ 12

Immutable audit trail

One event per request, emitted asynchronously to an object store in WORM compliance mode, retention five years, KMS-signed. Bucket layout partitions by component, version, line of business and date for fast queries.

This is a specification, not a deployment. No such store is wired to anything today, and the live site previously implied otherwise.

§ 13

Data lineage

OpenLineage RunEvents emitted to Marquez on every successful call. Standard event shape with no custom extensions, so DataHub, Atlan, Collibra or Purview ingest the events without translation.

§ 14

Memory & state

The adapter is stateless across requests. The only state it owns is quota counters in DynamoDB and rate-limit counters in Redis. Calling agents own their own session and persistent memory; the adapter never sees raw user data outside the prompt window.

§ 15

Capability enforcement

OPA sidecar via Envoy ext_authz on every egress. Blocked destinations return 403, increment capability_violations_total and emit an audit event with reason=undeclared_egress. NetworkPolicy provides a second enforcement layer at L3/L4.

§ 16

IP protection

All artifacts encrypted at rest with a per-component KMS key. Image signed with cosign keyless via Fulcio and Rekor; the admission webhook verifies the signature. BSL clauses restrict competitive hosted offerings during the licence window.

§ 17

Failure modes & SLOs

99.9% availability over a 30-day rolling window. P50 under 15 ms and P99 under 80 ms for the adapter itself, excluding the Triton call. These are targets in this document. They have never been measured, because there is no adapter to measure, and they must not be quoted as results.

Five named failure modes are documented with explicit responses, including audit-buffer overflow (return 503 rather than drop), guardrail-config staleness (fail closed after one hour) and certificate revocation in flight (gateway sheds within five minutes).

§ 18

Adaptive feedback loop

Human corrections from the upstream human-in-the-loop service flow into a corrections queue. The adapter forwards events whose model is one of its registered Nemotron variants to the training queue with a re-evaluation request.

The adapter never auto-fine-tunes. It proposes training inputs; a centre-of-excellence review gate decides re-evaluation versus re-certification.

§ 19

Off-sell packaging

Three tiers are specified — open source under BSL, enterprise with named SLAs and a signed compliance pack, and custom for vendors bundling the adapter into their own runtime distribution. None is on sale, because there is nothing to sell yet.

Design-partner access covers the evaluation harness rather than the adapter: the benchmark, the boundary suite, the OPA bundle and the certificate schema.

§ 20

Acceptance criteria

Ten gates close the v0.1.0 release: clean manifest admission, clean OPA admission, all eight evaluation thresholds met, certificate minted and resolvable, an end-to-end smoke test through the gateway against a real Triton endpoint, audit and lineage events queryable within five seconds, audit scope confirmed, and a signed chart and image published.

Zero of the ten are closed today. Anything short of all ten ships as a pre-release tag, not GA.