AnankeEmbed-S (v1)

AnankeEmbed-S is a compact, tokenizer-free UTF-8 byte-level bidirectional embedding model built by AMEFORGE for AnankeSearch. It maps queries and documents into a shared 256-dimensional semantic space with Matryoshka Representation Learning (MRL) support (64, 128, and 256 dimensions).

Model Architecture

Parameter Value
Input Representation Raw UTF-8 bytes (VOCAB = 259: 0..255 bytes, PAD=256, QUERY=257, DOC=258)
Tokenizer None (native UTF-8 bytes β€” handles typos, accents, emojis, code, any language)
Local Mixing & Patching Local byte convolution + 4x4 patching (max_bytes=512 β†’ 128 positions)
Backbone Bidirectional Transformer (d=384, layers=6, heads=6, RMSNorm, RoPE, SwiGLU)
Output Dimensions 256 with Matryoshka truncation at [64, 128, 256]
Precision / Format ONNX (ananke-embed.onnx, ~46.4 MB) + PyTorch (best.pt, ~46.3 MB)

Evaluation Results

Trained on NVIDIA A100-SXM4-80GB (6.71 h, best checkpoint at step 38,000):

Retrieval Benchmarks (Matryoshka Dimensions)

Dimension MRR Recall@1 Recall@10
64 0.7974 0.7350 0.9155
128 0.7960 0.7320 0.9150
256 0.7966 0.7335 0.9165

Ananke Search Exam

  • examen_r1: 0.70 (70% Top-1)
  • examen_mrr: 0.791

Matryoshka Efficiency: Truncating embeddings to 64 dimensions achieves MRR = 0.7974 (on par with 256d) while reducing vector storage and similarity compute by 4x.

Repository Files

  • ananke-embed.onnx β€” Exported ONNX graph for fast CPU/GPU inference
  • ananke-embed.json β€” Model configuration (d, layers, heads, patch, max_bytes, out_dim, matryoshka)
  • best.pt β€” Best PyTorch checkpoint (step 38,000)
  • resultats.json β€” Detailed evaluation metrics and training curve

Deployment & Usage

1. One-Command Server Install (AnankeSearch / Kaia)

HF_READ_TOKEN=hf_xxx MODEL_REPO=kaia-cod/anankesearch-embed-s bash scripts/ananke_embed_install.sh

2. HTTP Embedding Endpoint (serve.py)

curl -X POST http://127.0.0.1:8090/embed \
  -H "Authorization: Bearer $ANANKE_EMBED_KEY" \
  -H "Content-Type: application/json" \
  -d '{"texts": ["fastapi background tasks"], "role": "query", "dim": 64}'

Intended Use & Limitations

  • Designed for semantic retrieval, reranking, and private tenant indexing inside AnankeSearch and Kaia (SAM-G-Agent).
  • Trained primarily on web/technical retrieval pairs; increasing French/multilingual training pairs is recommended for future versions (v2+).
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support