Poindexter-Labs/Copernicus-57B
Text Classification • Updated
AI & ML interests
Evaluation benchmarks and training datasets
Recent Activity
View all activity
Organization Card
An AI capability and efficiency lab for teams building and improving frontier systems.
We produce expert-generated data and research-grade evaluation for frontier model
development — and we publish our benchmarks openly rather than keeping them as sales material.
📊 PDL-Bench
🛠 PDL-SWE-Bench
🌍 Translation & Localisation
📄 About us (PDF)
🌐 poindexterlabs.ai
✉️ Contact us
𝕏 @Poindexter_Labs
What we do
Frontier model training
Expert-generated data pipelines across pre-training, fine-tuning and reinforcement learning, built to each model's specification.
Agent evaluation & optimisation
Structured evaluation and human-in-the-loop refinement, taking autonomous agents from prototype to production-grade reliability.
Red teaming and safety
Adversarial testing and safety evaluation by trained specialists, finding vulnerabilities before users do.
Vertical and STEM expertise
Domain-expert data and evaluation for the fields where generic training data falls short.
Localisation and translation
Professional tasks and their grading rubrics localised into other languages, and benchmarks that measure how models perform in them. Our first release covers German and Japanese.
Open benchmarks
PDL-Bench
128 tasks across seven mini-benchmarks: specialist problems written by our experts to beat frontier models, plus two adapted and attributed sets (localised GDPval tasks and retrograde chess compositions). Every task ships with what its grading needs — a final answer, a rubric, a worked solution, or an executable test suite — and every mini-benchmark has a downloadable Docker evaluation package, so results reproduce locally with your own API keys.
| Mini-benchmark | Focus | Tasks |
|---|---|---|
| STEM | Research-level scientific and mathematical reasoning | 44 |
| Business Operations | Operational analysis and professional business deliverables | 10 |
| Life Administration | Everyday planning, research and personal logistics | 8 |
| Software Delivery | Repository-scale engineering in executable environments | 20 |
| Cyber Operations | Security-critical protocol and trust-system implementation | 10 |
| GDPval Localisation | Professional tasks localised into German (14) and Japanese (10), with corrected rubrics | 24 |
| Retrograde Chess | Legal game reconstruction and analytical retrograde proofs | 12 |
All tasks (overall config) |
128 |
GDPval Localisation is our localisation and translation benchmark. Starting from OpenAI's GDPval, we localised 24 professional task variants into German and Japanese and corrected their evaluation rubrics. The tasks cover occupations in finance, government, health care, law, manufacturing, real estate, retail and wholesale trade, and are graded on the deliverable a model produces. It is a selected challenge subset, not the original GDPval evaluation.
PDL-SWE-Bench
18 repository-scale software engineering tasks in SWE-bench schema, with hidden test suites and gold patches.
Both datasets are CC BY 4.0; third-party source material in the GDPval and retrograde chess subsets keeps its own terms. Where a suite shows a model is weak, that is exactly the training data we build.
The people who do the work
Poindexters are STEM domain experts, selected through subject-knowledge interviews for the depth and judgment that frontier AI work demands.
46%Olympiad medallists
54%PhD level
34%Oxbridge, Ivy League or MIT
18%Professor level