TerraIncognita Logo

A Dynamic Benchmark for Species Discovery Using Frontier Models

Shivani Chiranjeevi, Hossein Zaremehrjerdi, Zi K. Deng, Talukder Z. Jubery, Ari Grele,
Arti Singh, Asheesh K Singh, Soumik Sarkar, Nirav Merchant, Harold F. Greeney,
Baskar Ganapathysubramanian, Chinmay Hegde
Iowa State University, University of Arizona, New York University, Yanayacu Biological Station
2025 Datasets and Benchmarks Track
Paper Dataset Code arXiv
Figure 1

Figure 1: Overview of the TerraIncognita data collection pipeline. Insect specimens are first captured using light traps and examined by an entomologist or field expert. Images are then filtered using commercial-grade image classification tools to detect known taxonomic matches. If a specimen does not match existing species-level entries, it is flagged as potentially novel.

Abstract

The rapid global loss of biodiversity, particularly among insects, represents an urgent ecological crisis. Current methods for insect species discovery are manual, slow, and severely constrained by taxonomic expertise, hindering timely conservation actions. We introduce TerraIncognita, a dynamic benchmark designed to evaluate state-of-the-art multimodal models for the challenging problem of identifying unknown, potentially undescribed insect species from image data. Our benchmark dataset combines a mix of expertly annotated images of insect species likely known to frontier AI models, and images of rare and poorly known species, for which few or no publicly available images exist. These images were collected from underexplored biodiversity hotspots, realistically mimicking open-world discovery scenarios faced by ecologists. The benchmark assesses models' proficiency in hierarchical taxonomic classification, their capability to detect and abstain from out-of-distribution (OOD) samples representing novel species, and their ability to generate explanations aligned with expert taxonomic knowledge. Notably, top-performing models achieve over 90% F1 at the Order level on known species, but drop below 2% at the Species level—highlighting the sharp difficulty gradient from coarse to fine taxonomic prediction (Order → Family → Genus → Species). Discovery accuracy on novel species varies widely, from 55% to 88%, revealing inconsistencies in abstention and overcommitment strategies across model families. Our evaluation of contemporary frontier multimodal models highlights critical strengths and exposes limitations in their current open-world recognition capabilities. TerraIncognita will be updated regularly, and by committing to quarterly dataset expansions (of both known and novel species), will provide an evolving platform for longitudinal benchmarking of frontier AI methods. All TerraIncognita data, results, and future updates are available here.

Dataset Features

Order Count
Family Count

Figure 2: Distribution of species by order and family for Novel species. The dataset is skewed toward the order Lepidoptera and family Crambidae.

VLM Evaluation on Discovery Accuracy and Hierarchical Classification Accuracy

Discovery Accuracy

Table 1: Discovery Accuracy comparison between Known and Novel species across models.

BibTeX

@inproceedings{chiranjeevi2025terraincognita,
  title={TerraIncognita: A Benchmark for Insect Biodiversity Discovery},
  author={Chiranjeevi, Shivani and Zaremehrjerdi, Hossein and Deng, Zi K. and Jubery, Talukder Z. and Grele, Ari and Singh, Arti and Singh, Asheesh K. and Sarkar, Soumik and Merchant, Nirav and Greeney, Harold F. and Ganapathysubramanian, Baskar and Hegde, Chinmay},
  booktitle={2025 Datasets and Benchmarks Track},
  year={2025}
}