NVIDIA · Agentic coding
Nemotron-3-Nano-4B
Hybrid Mamba-Transformer. The lane behind every self-hosted step in the open-mamba agent loop.
- Pass rate
- 4 / 9Measured
- Throughput
- 18.7 tok/sMeasured
- Cost / task
- $0.0475Measured
- Hardware
- RTX 4000 AdaMeasured
Autonomous delivery · design partner preview
Nemotron, governed.
A self-hosted autonomous delivery system. It runs the same agent loop on your own GPU or on a frontier cloud model, executes every tool itself so the model never touches your credentials, and grades its own output well enough to bench its own agents.
1,337 tasks delivered into 25 projects for $192.38 of measured spend. Ten weeks, unattended. Every task, step, token and dollar lands in a database you own and can query — including the runs that failed, which is most of them.
Mamba processing · 52 layers, depth order
4 attention layers — the scarce resource
Why self-hosted
Replace the cloud coding agent, not the model behind it.
A hosted agent gives you someone else’s tool surface, someone else’s pricing and someone else’s dashboard. This gives you the loop, the executor, the ledger and the governance — running against a model you can train, quantise and serve on hardware you own.
01
Your GPU or theirs, same loop
One ModelBackend trait. The self-hosted lane and the cloud lane share the same system prompt, prompt builder, tool parser and executor — so a difference in results is a model difference, never a harness difference.
Measured4 backends · Nemotron, Claude API, Claude CLI, stub control
02
The model never holds your credentials
The orchestrator owns every tool, the workspace and the git identity. The model receives text and returns one JSON tool call. It cannot clone, execute or authenticate — there is no surface for it to do so.
MeasuredPath-traversal guard fired in production, blocking a real escape attempt
03
It benches its own agents
Every completion is graded, ship-rate is computed per agent and model, and a pause-dispatch policy is written when an agent stops shipping — which the dispatcher reads back before it routes work.
Measured1,272 graded tasks · it paused its own CEO, merger and batcher
04
The lake is yours, and it is the source of truth
Every task, step, token, dollar and decision lands in a local DuckDB file. No vendor dashboard, no reconstruction from logs. Both lanes bill into the same table shape, so cost comparisons are like-for-like.
Measured14 tables · every number on this site is a SELECT away
05
A durable org, not a chat session
Five roles under enforced contracts: the merger compiles or reverts, the reviewer flags rather than silently fixes, the CEO drains a decision queue. Driven by a supervised clock that ingests GitHub issues and survives its own panics.
Measured514 workflows · 1,338 schedule fires · 10 weeks unattended
The asset
Eight evaluation dimensions. One FAIL halts release.
Each gate names its instrument and its pass criterion before anything is run, so a result cannot be negotiated afterwards. The pipeline is defined; it has not yet been executed against a shipped adapter, so every criterion below reads Specified.
Dim 1
Accuracy & Quality
LLM-as-judge over a 1,200-pair benchmark.
≥ 0.85 mean · no regression > 0.03
SpecifiedDim 2
Security
Garak red-team plus a 412-prompt boundary suite.
Zero jailbreaks, PII leaks or prompt extractions
SpecifiedDim 3
Infrastructure
OPA, Checkov and CDK Nag against pod spec and IAM.
Cosign-verified images · no privileged containers
SpecifiedDim 4
Regulatory
Mapped controls against the Compliance Standards Library.
Every mapped control evaluates compliant
SpecifiedDim 5
Data Governance
OpenLineage completeness against Marquez.
100% of test calls produce a complete record
SpecifiedDim 6
Guardrail Adherence
Bedrock-compatible filters plus the OPA bundle.
Boundary suite · zero bypasses
SpecifiedDim 7
Capability Governance
Declared call surface vs the eBPF-traced one.
Exact match required
SpecifiedDim 8
Auditability
Dry-run mission against an Object Lock store.
100% of calls queryable and signed within 5 s
Specified
The independence argument
A methodology is only worth something if it can fail you.
Ours did. Run 5fe4bd2a put the self-hosted 4B lane through nine execution-graded tasks and it passed four. That result is on the benchmarks page in full, with the five failures named and the mechanism behind them explained.
A vendor benchmark that never reports a loss is a marketing artifact. The point of publishing the gates before the implementation is that the gates outlive whatever the implementation turns out to score.
What this site does not claim
helm install oci://ghcr.io/…
The chart is not published. The command 404s, so it is gone.
< 80 ms P99 adapter latency
An SLO in the spec, not an observation. There is no adapter to measure.
WORM audit trail · 5-year retention
Specified in §12. No such store is wired to anything today.
Serves Nemotron-4-340B
Never served here. It does not fit a 20 GB card.
status: beta
Beta implies shipping code. The gateway adapter crate holds one README.
Every delivery is anchored on-chain
All 1,337 records carry anchor-ready calldata, not a transaction. Broadcast is not wired; chain_logs has zero rows.
Confidential compute
That word means hardware-attested. Ours is private by ownership; attestation is a TEE partner lane.
Served here
One card. One model family. Stated plainly.
Verified against the host on 2026-08-19 with ollama list and nvidia-smi. Two entries, both running today. Nothing is listed that this box cannot hold.
OpenMamba computeIllustrative
A rendering, not a product. The measurements came off one RTX 4000 SFF Ada, drawn to scale on the benchmarks page.
NVIDIA · Structured decisions
Nemotron-3-Nano-4B · QLoRA
Rank-64 fine-tune for structured-JSON decision output. Three domain adapters behind one /generate contract.
- LoRA rank
- 32 · α 64Measured
- Quant
- 4-bit nf4Measured
- Adapters served
- 3Measured
- Retrain cadence
- hourlySpecified
Verifiability
Hashed and sealed today. Anchored when the broadcast is wired.
Three of these five layers run on every task. Two do not, and they are the two that would let a stranger verify a delivery without trusting us — so they are marked, not implied.
- 01Built
Encrypted audit log
AES-256-GCM over the full task envelope
1,337 of 1,337 records carry a non-empty encrypted log
- 02Built
Content hashing
SHA-256 proof hashes over model output
Unit-tested against known digests
- 03Built
Anchor-ready calldata
Every billing record carries encoded calldata
Generated per task, stored in the lake
- 04Not wired
On-chain broadcast
Signing and sending the anchor transaction
A TODO in anchor.rs. No transaction has ever been sent
- 05Not deployed
MessageTrail contract
Append-only event log, 11 tests passing
Written and tested. Devnet target, no mainnet deploy
Standards
Mapped, not asserted.
The specification includes a control-by-control table for each framework. A mapping states where evidence would be found; it does not claim an audit has been performed.
Nothing here has been assessed by a third party.
Read the control mappingEU AI Act
Articles 12 (logging), 13 (transparency), 15 (accuracy & robustness)
NIST AI RMF 1.0
GOVERN-1.1, MEASURE-2.7, MANAGE-4.1
ISO/IEC 42001:2023
Clauses 8.3 (operations) and 9.1 (monitoring)
SOC 2
CC6.1 (mTLS, KMS) and CC7.2 (immutable audit) — scope not yet confirmed
NAIC Model Bulletin
§4.2 governance — owner, COE approval, evaluation evidence
OpenLineage
Standard RunEvent shape; ingests in DataHub, Atlan, Collibra, Purview
Design partner kit
What a design partner actually receives.
01
The nine-task execution-graded benchmark harness
ships today02
412-prompt boundary suite and Garak configuration
ships today03
OPA bundle, capability CRD and certificate schema
ships today04
The adapter implementation
not written
Design partner access
The harness, not the adapter.
What a design partner gets today is the evaluation kit: the nine-task benchmark, the 412-prompt boundary suite, the OPA bundle and the certificate schema. The adapter is specified and unwritten, and access does not change that.
Tell us what you would point it at. We reply to everything that names a real workload.
Paid access runs on the same box the benchmark ran on: one RTX 4000 SFF Ada, 20 GB, serving nemotron-3-nano:4b on a single stream. No larger card is behind a paid tier, and the numbers on this site are the numbers you would get.