README.md

---
language:
  - ig
  - ha
  - yo
  - en
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
  - embeddings
  - sentence-transformers
  - cross-lingual
  - multilingual
  - igbo
  - hausa
  - yoruba
  - information-retrieval
  - semantic-search
base_model: sentence-transformers/LaBSE
---

<div align="center">
  <img src="images/modular_logo.jpg" width="38%" alt="Modular" />
</div>
<hr>
<div align="center" style="line-height: 1;">
  <a href="https://huggingface.co/Modularcomputing" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Modular-ffc107?color=ffc107&logoColor=white"/></a>
  <a href="https://huggingface.co/Modularcomputing/Native-Bird2" target="_blank"><img alt="Native-Bird2" src="https://img.shields.io/badge/Multimodal-Native--Bird2-536af5?color=536af5&logoColor=white"/></a>
  <a href="https://github.com/Dox45" target="_blank"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-Dox45-181717?logo=github&logoColor=white"/></a>
  <br>
  <img alt="Languages" src="https://img.shields.io/badge/Languages-Igbo%20%7C%20Hausa%20%7C%20Yoruba%20%7C%20English-1f6feb"/>
  <img alt="Embedding dim" src="https://img.shields.io/badge/Embedding-768--d-2ea44f"/>
  <img alt="Base" src="https://img.shields.io/badge/Base-LaBSE-6f42c1"/>
</div>

<p align="center">
  <img src="images/native_bird_diagram.jpg" width="100%" alt="Native-Bird: an Igbo query and English documents in one shared semantic vector space" />
</p>

## Table of Contents

1. [Introduction](#1-introduction)
2. [Model Summary](#2-model-summary)
3. [Model Downloads](#3-model-downloads)
4. [Evaluation Results](#4-evaluation-results)
5. [Quick Start](#5-quick-start)
6. [Searching Real Document Collections](#6-searching-real-document-collections)
7. [Limitations](#7-limitations)
8. [Citation](#8-citation)
9. [Contact](#9-contact)

## 1. Introduction

**Native-Bird** is a 768-dimensional multilingual sentence-embedding model fine-tuned from **LaBSE** for cross-lingual retrieval across **Igbo, Hausa, Yoruba and English**.

Its primary use case is **native-language query → English document retrieval**: a user describes what they are looking for in Igbo, Hausa or Yoruba, the indexed documents are written in English, and Native-Bird maps both into the same embedding space for semantic search.

> **Igbo query:** “Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn.”
>
> **English document:** “The nucleus consists of two particles - neutrons and protons.”

Native-Bird is the text backbone of [**Native-Bird2**](https://huggingface.co/Modularcomputing/Native-Bird2), which extends the same space to images.

## 2. Model Summary

**Architecture**

| | |
|---|---|
| Architecture | SentenceTransformer (LaBSE encoder → pooling → dense → normalize) |
| Base model | [`sentence-transformers/LaBSE`](https://huggingface.co/sentence-transformers/LaBSE) |
| Embedding dimension | 768 |
| Max sequence length (training) | 128 tokens |
| Similarity | Cosine |
| Languages | Igbo (`ig`), Hausa (`ha`), Yoruba (`yo`), English (`en`) |

**Training**

| | |
|---|---|
| Objective | Cached Multiple Negatives Ranking Loss (CachedMNRL) |
| Training pairs | 76,150 native-language ↔ English pairs |
| Epochs | 4 |
| Per-device batch size | 256 (CachedMNRL mini-batch 128) |
| Learning rate / schedule | 2e-5, cosine, 10% warmup |
| Negatives | In-batch, NO_DUPLICATES sampling |
| Precision | BF16 |

## 3. Model Downloads

<div align="center">

| **Model** | **Modality** | **Embedding dim** | **Download** |
| :---: | :---: | :---: | :---: |
| Native-Bird | Text (ig / ha / yo / en) | 768 | [🤗 Hugging Face](https://huggingface.co/Modularcomputing/Native-Bird) |
| Native-Bird2 | Text + Image | 512 | [🤗 Hugging Face](https://huggingface.co/Modularcomputing/Native-Bird2) |

</div>

## 4. Evaluation Results

Cross-lingual retrieval against an English corpus of **1,004 sentences**, with **204 queries per language**. Each query has one known matching English sentence. Two query variants are tested: **clean** (original text) and **ASR-style noise** (diacritics removed, lowercased, punctuation stripped, whitespace normalized).

<div align="center">

| **Language** | **Query** | **LaBSE R@1** | **Native-Bird R@1** | **LaBSE MRR** | **Native-Bird MRR** |
| :--- | :--- | :---: | :---: | :---: | :---: |
| Igbo | Clean | 1.0000 | **1.0000** | 1.0000 | **1.0000** |
| Igbo | ASR-style noise | 0.9853 | **1.0000** | 0.9888 | **1.0000** |
| Hausa | Clean | 1.0000 | **1.0000** | 1.0000 | **1.0000** |
| Hausa | ASR-style noise | 0.9951 | **1.0000** | 0.9975 | **1.0000** |
| Yoruba | Clean | 0.9804 | **0.9804** | 0.9871 | **0.9902** |
| Yoruba | ASR-style noise | 0.9559 | **0.9804** | 0.9683 | **0.9867** |

</div>

Native-Bird reaches R@10 = 1.0000 in every setting (LaBSE: 0.9951 on noisy Igbo and Yoruba). Fine-tuning mainly improves **robustness to transcription-style noise**, which matters for voice and keyboard input without diacritics.

**Direct Igbo → English retrieval examples**

| Igbo query | Top English result | Cosine |
| :--- | :--- | :---: |
| Mgbanwe n'ọdịdị na-agbakwunye ọdịdị kejenetiki ọhụrụ, ma nhọpụta na-ewepụ ya n'ogwu ọdịdị nke a kọwapụtara. | Mutation adds new genetic variation, and selection removes it from the pool of expressed variation. | 0.7079 |
| Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn. | The nucleus consists of two particles - neutrons and protons. | 0.7049 |
| Enwere ọtụtụ ihe mere ha ji kara web prọgzi mma: ha na-agbanwe ụzọ trafik ịntanetị niile, ọ bụghị naanị http. | They are superior to web proxies for several reasons: They re-route all Internet traffic, not only http. | 0.6774 |

## 5. Quick Start

```bash
pip install -U sentence-transformers torch
```

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Modularcomputing/Native-Bird")

query = "Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn."
documents = [
    "The nucleus consists of two particles - neutrons and protons.",
    "The liver is an organ responsible for many metabolic functions.",
    "Photosynthesis converts light energy into chemical energy.",
]

q = model.encode(query, normalize_embeddings=True)
d = model.encode(documents, normalize_embeddings=True)
scores = d @ q

for i in scores.argsort()[::-1]:
    print(f"{scores[i]:.4f}  {documents[i]}")
```

The matching English sentence should rank first.

## 6. Searching Real Document Collections

Do **not** embed a multi-page document as one vector. Split each document into passages of roughly 100–300 words with overlap, and store each passage embedding with its parent document ID.

```text
Igbo / Hausa / Yoruba query
        ↓  Native-Bird encoder
768-d query vector
        ↓  cosine similarity (FAISS / vector DB)
English passage index
        ↓
Top-k passages → grouped by parent document
```

**Intended applications:** native-language search over English knowledge bases, cross-lingual RAG retrieval, multilingual educational search, and search over Nigerian institutional documents.

## 7. Limitations

- **Embedding model, not a translator or generator.** It represents text as vectors; it does not produce English answers.
- **Sentence-level evaluation.** Trained at 128 tokens and evaluated on sentences, so long documents need chunking. Scores do not guarantee the same accuracy on arbitrary long documents.
- **Domain shift.** Government, legal, medical and technical text can differ substantially from the benchmark; evaluate on your target domain before production use.
- **Status: research / early release.** The next validation step is a human-annotated benchmark of real Nigerian documents and real Igbo, Hausa and Yoruba information needs.

## 8. Citation

```bibtex
@misc{nativebird2026,
  title  = {Native-Bird: Cross-lingual Native-Language Embeddings for English Document Retrieval},
  author = {Akuegbo, Chima Emmanuel},
  year   = {2026},
  publisher = {Modular},
  howpublished = {\url{https://huggingface.co/Modularcomputing/Native-Bird}}
}
```

## 9. Contact

Questions, issues or collaboration: open a discussion on the [Community tab](https://huggingface.co/Modularcomputing/Native-Bird/discussions).