Pl@ntNet SimDINOv2 ViT-L/16 — PN22M + ImageNet-1K

This is a ViT-L/16 self-supervised vision backbone trained from scratch with SimDINOv2 on 22 million Pl@ntNet images and ImageNet-1K. It extracts global and patch-level representations, with a particular focus on plant and biodiversity imagery. The checkpoint uses four register tokens and is published in the native Hugging Face Transformers format.

This is a feature extractor, not a plant-species classifier. It does not contain a classification head or a species label vocabulary.

Model details

Property	Value
Method	SimDINOv2
Architecture	ViT-L/16 with four register tokens
Parameters	303,355,904
Transformer blocks	24
Hidden size	1,024
Attention heads	16
Patch size	16 × 16
Reference input size	224 × 224 RGB
Output at 224 × 224	1 CLS + 4 registers + 196 patch tokens
Training paradigm	Self-supervised learning
Initialization	From scratch
Training data	22M Pl@ntNet images + ImageNet-1K

Environment and dependencies

The reference checkpoint was trained with the sslplant/SimDINOv2 implementation using Python 3.11, PyTorch 2.5.1, CUDA 11.8-compatible PyTorch wheels, and xFormers 0.0.29. These versions describe the training environment; they are not all runtime requirements for this converted checkpoint.

For inference, the recommended environment is:

Python 3.11 or newer;
PyTorch 2.4 or newer;
Transformers 4.48 or newer;
Pillow and safetensors.

Quick start

pip install "torch>=2.4" "transformers>=4.48,<6" pillow requests safetensors

import torch
from PIL import Image
import requests
from transformers import AutoImageProcessor, AutoModel

model_id = "PlantNet/simdinov2-vit-large-pn22m-in1k"
image_url = "https://raw.githubusercontent.com/plantnet/PlantNet-300K/main/images/1.jpg"

response = requests.get(image_url, stream=True, timeout=30)
response.raise_for_status()
image = Image.open(response.raw).convert("RGB")

# Transformers 5.x: explicitly select the PIL preprocessing backend.
processor = AutoImageProcessor.from_pretrained(model_id, backend="pil")
model = AutoModel.from_pretrained(model_id).eval()

inputs = processor(images=image, return_tensors="pt")
with torch.inference_mode():
    outputs = model(**inputs)

# Global image descriptor: [batch, 1024]
global_embedding = outputs.pooler_output

# Token order: [CLS], four register tokens, then patch tokens.
register_embeddings = outputs.last_hidden_state[:, 1:5]  # [batch, 4, 1024]
patch_embeddings = outputs.last_hidden_state[:, 5:]      # [batch, 196, 1024]

The example image comes from the public PlantNet-300K repository. Replace image_url with your own image URL or use Image.open("plant.jpg").convert("RGB") for a local file.

For GPU inference:

model = model.to("cuda")
inputs = inputs.to("cuda")

Preprocessing

The bundled AutoImageProcessor reproduces the checkpoint's reference preprocessing:

Convert to RGB.
Resize directly to 224 × 224 using bilinear resampling (no center crop).
Rescale integer pixels by 1 / 255.
Normalize with Pl@ntNet image statistics:

mean = (0.442, 0.469, 0.326)
std  = (0.235, 0.221, 0.232)

Use the bundled processor with backend="pil" for strict reproducibility. With Transformers 4.x, use use_fast=False instead.

Intended uses

Image-level embeddings for retrieval, clustering, or similarity search.
Patch-level features for dense prediction or localization systems.
Initialization for plant-focused classification, detection, or segmentation.
Research on visual representations of plants and biodiversity imagery.

Limitations

The model has no classifier and therefore does not return species names.
Performance depends on the downstream data, head, and evaluation protocol.
The 224 × 224 direct resize changes the aspect ratio of non-square images.

Training and evaluation

This checkpoint was trained from scratch with SimDINOv2 using 22 million Pl@ntNet images together with ImageNet-1K.

The training pipeline follows the plant-domain SimDINOv2 methodology described in Self-Supervised Learning of Plant Image Representations (Moummad et al., ICPR CVBMC 2026). The training implementation is available in the sslplant repository.

Citation

If you use this checkpoint, please cite our plant SSL work:

@misc{sslplant,
  title={Self-Supervised Learning of Plant Image Representations},
  author={Ilyass Moummad and Kawtar Zaher and Herv\'e Go\"eau and Jean-Christophe Lombardo and Pierre Bonnet and Alexis Joly},
  year={2026},
  eprint={2604.27538},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2604.27538}
}