Language-Model Research
Version2Labs is developing a mathematics-first, computationally grounded understanding of modern language models. The work follows the complete path from text and tokenization through embeddings, attention, residual representations, logits, and decoding. Rather than beginning with a chatbot or a large training run, the project starts with small, reproducible experiments designed to make each part of the system observable and testable.
The first experiment uses Pythia-160M to reconstruct causal attention directly from model weights and intermediate tensors, then compare that calculation with the framework implementation under fixed numerical tolerances. It also examines how tokenization, training checkpoints, and decoding rules affect model behavior.
The broader purpose is to establish a rigorous foundation for studying questions about model architecture, training dynamics, data quality, efficiency, uncertainty, and small-model capability. This remains an exploratory research initiative, not an announcement of a Version2Labs language model.