What is in it

The corpus, the tokenizer, the pretraining run and the post-training are all homespun's. That is the point: a fine-tune of somebody else's weights has a layer underneath it where the honest answer is "we do not know, ask them."

And the layers people actually worry about — what was in the corpus, whether the evaluation leaked into it — are exactly the ones underneath. So the pipeline trains the model rather than adopting one, and the attestation binds each layer by a hash.

The second and third rows are a shortcut, and they are labelled as one wherever they appear. They exist because a useful model this year beats a perfect model in three, and because the post-training and evaluation stages need exercising against real weights long before there are any of our own. A model made that way inherits a pretraining layer homespun did not make, so the attestation records it in a separate block rather than folding it in with the corpus we actually read.

One consequence worth naming: the contamination check builds its n-gram sketch from the data homespun materialises. On a shortcut path that is the post-training mixture only, so the check is structurally blind to whatever was in the base's pretraining corpus. On the from-scratch path it sees everything, because we read everything.

homespun trains on openly-licensed corpora, and the largest of them is Ai2's Dolma. Using a published corpus is what published corpora are for: it is an input we process and can account for document by document. Taking somebody's finished weights is a different act — a layer you would have to accept on their say-so.

Dolma is the largest source because it records a licence per document, which is the precondition for a data card that means anything. The Stack v2 would have been the obvious code source and is unusable here: it ships identifiers rather than file contents, and the bytes need cloud credentials and a signed agreement. Every source and every rejection is listed in the repo's data notes.

Because the interesting claim collapses otherwise. If the bottom of the stack is somebody else's checkpoint, "you can check how this was made" is true of the top two layers and an act of faith below them.

The cost is real and stated rather than implied: roughly 460 H100-hours for a 1B, 44,000 for a 7B, 404,000 for a 32B, at six floating-point operations per parameter per token and 40% utilisation. The 1B is proven. Nothing above it has been trained.