Tessera 1B
Our first public release: a ~1B-parameter model trained from scratch on a hand-curated 24.5B-token corpus — for about $315 on a single rented H100. We are releasing the weights, the data, and the hand-written alignment set. Open model, open dataset.
Tessera 1B is an honest base model. It writes fluent English (and some Japanese), but it has not been post-trained — so its reasoning and factual reliability are limited, and it is not a chat assistant. That is the point of releasing it: it is a clean, well-built starting block. It fine-tunes cleanly and makes an excellent foundation for a specialty model — one tuned to answer specific questions about a specific domain. If you want a base you can actually understand end to end, this is one.
The release
Specs, verified
| Parameters | 1.01B (32 layers · d1536 · 16 heads · 4096 context · 65,536 vocab · bf16) |
| Tokenizer | Tessera tokenizer (byte-level BPE), English + Japanese — open, with training recipe → |
| Training data | 24.5B tokens (~1 epoch), ≈24× Chinchilla |
| Compute | 1× H100 SXM 80GB (vast.ai, Japan) · 145.7h (~6 days) · ~$315 |
| Final eval loss | ~3.20 nats (perplexity ≈ 24.5) |
| Base license | Apache-2.0 (trained from scratch — no upstream model license) |
Said plainly
We measured what we measured and nothing more. The only rigorous metric is language-model loss; on a small TruthfulQA probe the base scored near chance. We would rather tell you it is a fluent base with weak cognition than publish benchmark numbers we did not run. Its base corpus is web, books, and academic text — no model-conversation transcripts, no synthetic reasoning traces, consistent with our data policy.
This is what a small, independent lab can do end to end — data, tokenizer, pretrain, alignment, release — and show its work. It is a beginning, released honestly.