AnankeEmbed-S (v1)
AnankeEmbed-S is a compact, tokenizer-free UTF-8 byte-level bidirectional embedding model built by AMEFORGE for AnankeSearch. It maps queries and documents into a shared 256-dimensional semantic space with Matryoshka Representation Learning (MRL) support (64, 128, and 256 dimensions).
Model Architecture
| Parameter | Value |
|---|---|
| Input Representation | Raw UTF-8 bytes (VOCAB = 259: 0..255 bytes, PAD=256, QUERY=257, DOC=258) |
| Tokenizer | None (native UTF-8 bytes β handles typos, accents, emojis, code, any language) |
| Local Mixing & Patching | Local byte convolution + 4x4 patching (max_bytes=512 β 128 positions) |
| Backbone | Bidirectional Transformer (d=384, layers=6, heads=6, RMSNorm, RoPE, SwiGLU) |
| Output Dimensions | 256 with Matryoshka truncation at [64, 128, 256] |
| Precision / Format | ONNX (ananke-embed.onnx, ~46.4 MB) + PyTorch (best.pt, ~46.3 MB) |
Evaluation Results
Trained on NVIDIA A100-SXM4-80GB (6.71 h, best checkpoint at step 38,000):
Retrieval Benchmarks (Matryoshka Dimensions)
| Dimension | MRR | Recall@1 | Recall@10 |
|---|---|---|---|
| 64 | 0.7974 | 0.7350 | 0.9155 |
| 128 | 0.7960 | 0.7320 | 0.9150 |
| 256 | 0.7966 | 0.7335 | 0.9165 |
Ananke Search Exam
examen_r1:0.70(70% Top-1)examen_mrr:0.791
Matryoshka Efficiency: Truncating embeddings to 64 dimensions achieves
MRR = 0.7974(on par with 256d) while reducing vector storage and similarity compute by 4x.
Repository Files
ananke-embed.onnxβ Exported ONNX graph for fast CPU/GPU inferenceananke-embed.jsonβ Model configuration (d,layers,heads,patch,max_bytes,out_dim,matryoshka)best.ptβ Best PyTorch checkpoint (step38,000)resultats.jsonβ Detailed evaluation metrics and training curve
Deployment & Usage
1. One-Command Server Install (AnankeSearch / Kaia)
HF_READ_TOKEN=hf_xxx MODEL_REPO=kaia-cod/anankesearch-embed-s bash scripts/ananke_embed_install.sh
2. HTTP Embedding Endpoint (serve.py)
curl -X POST http://127.0.0.1:8090/embed \
-H "Authorization: Bearer $ANANKE_EMBED_KEY" \
-H "Content-Type: application/json" \
-d '{"texts": ["fastapi background tasks"], "role": "query", "dim": 64}'
Intended Use & Limitations
- Designed for semantic retrieval, reranking, and private tenant indexing inside AnankeSearch and Kaia (SAM-G-Agent).
- Trained primarily on web/technical retrieval pairs; increasing French/multilingual training pairs is recommended for future versions (
v2+).