Running on CPU Upgrade
285
The Synthetic Data Playbook: Generating Trillions of the Finest Tokens
📝
Explore synthetic data experiments on a bookshelf visualization
AI & ML interests
We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)
Recent Activity
View all activity
Papers
How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Running
README
👀
Running
Featured
78
FinePDFs: Liberating 3T of the finest tokens from PDFs
📄
Explore the FinePDFs PDF token dataset
Running
Agents
15
FineWiki Viewer
🌐
Viewer to explore the finewiki dataset
Running
Featured
1.46k
FineWeb: decanting the web for the finest text data at scale
🍷
Explore the FineWeb dataset and its creation process
Running
96
Scaling FineWeb to 1000+ languages: Step 1: finding signal in 100s of evaluation tasks
📝
Evaluate multilingual models using FineTasks
Build error
Agents
1
Tasks Explorer
🏢
Explore and analyze experiment results
Runtime error
Agents
4
Datasets Metrics Explorer
📊
Launch an interactive demo interface