Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Contract Reference

microgpt-v1.yaml

The canonical contract for microGPT. All equations, proof obligations, and falsification tests are defined in contracts/microgpt-v1.yaml.

Equations

IDFormulaTestedProved
one_hotone_hot(i,C)[i][j] = 1 if j==i else 0L2L4
rms_normx / sqrt(mean(x^2) + eps)L2-
causal_mask0 if j<=i else -1e9L2L4
tokenize_decodedecode(tokenize(s)) == sL2L4
weighted_sampleP(return i) = probs[i]L2-
adam_stepKingma & Ba (2015) update ruleL2L4
forward_passLMHead(MLP(Attn(Embed(tokens))))L2L4

Proof obligations

IDDescriptionLevel
PARAM-COUNT-001Model has exactly 4,192 parametersL4
ONEHOT-ROW-001Each one-hot row sums to 1.0L2
MASK-CAUSAL-001Lower-triangular structureL2
TOKENIZE-ROUNDTRIP-001Encode/decode roundtrip identityL2
ADAM-MONOTONIC-001Step counter advances by 1L4
ADAM-V-NONNEG-001Second moment >= 0L4
FORWARD-SHAPE-001Output shape [n, 27]L2

Bindings

8 bindings in contracts/binding.yaml, all status: implemented.

EquationFunctionSignature
one_hotone_hotfn(indices, num_classes) -> Tensor
rms_normrms_normfn(x: &Tensor) -> Tensor
causal_maskcausal_maskfn(size) -> Tensor
tokenize_decodetokenizefn(name) -> Vec<usize>
tokenize_decodedecodefn(tokens) -> String
weighted_sampleweighted_sample_with_rfn(probs, r) -> usize
adam_stepAdam::stepfn(&mut self, params, lr)
forward_passMicroGPT::forwardfn(&self, tokens) -> Tensor