# Overlooked

> Every species documented from India, across sources. For each species, group and state: does it have an open record, do observations of it wait for a name, and what is its IUCN Red List status. Updated each week. Not an official checklist.

Overlooked builds a master list of species documented from India from the Zoological Survey of India (Fauna of India checklists), the Botanical Survey of India (national plant checklists), Royal Botanic Gardens, Kew (WCVP), the lichen checklist of India (Sinha, Nayaka and Mishra 2024, CC BY-NC 4.0: we use only names and the endemic mark), GBIF, iNaturalist, Plazi and the IUCN Red List. Names follow the Catalogue of Life. Each species is tagged with the sources that list it, and tracks two things: an open record (a record from India anyone can check online: any GBIF record, such as a photo, sighting, specimen, sound, DNA sample or a record cited from a paper, or an iNaturalist research-grade observation; the rule of GBIF, which takes only research grade from iNaturalist), and IUCN status (CR, EN, VU, NT, LC, DD, or not assessed). iNaturalist observations that wait for a name (needs ID) are not open records; a species that only they name is not on the list (status 2) until someone confirms the name.

## MCP server (no key)

https://overlooked-india.pages.dev/mcp: a Model Context Protocol server (Streamable HTTP, JSON responses, stateless, read only) on the current week of the master list. Tools: find_species (Latin or English name to key), get_species (one species in full: sources, open record, states, WII regions, IUCN category with year, DOI and trend, endemic, described, evidence, names in Indian languages, photo with credit, observations waiting for a name), count_species and list_species (filters that combine: group, state, WII biogeographic region or province, IUCN categories or "threatened", endemic, order, suborder (snakes, lizards), family, open record, recorded since, described since), new_species (described from India since 2007). Resources: overlooked://about, overlooked://groups, overlooked://states, overlooked://regions. Every answer carries the week, licence, attribution and disclaimer. No tool takes or gives a place finer than a state or biogeographic province, to protect species at risk from collection and trade (see /method#sensitive). For a place or a point, use the bharatlas MCP server's locate tool with layers ["lgd_states", "biogeographic_zones"] (the province layer is not in its default list) for the state and WII province (prov_code), then pass them as state and region; say "recorded in" the state or region, not "found here". Example client config: {"mcpServers": {"overlooked": {"type": "http", "url": "https://overlooked-india.pages.dev/mcp"}}}

## Open data (no key, CORS)

All data is static JSON and Parquet under https://overlooked-india.pages.dev/data/ . Weekly files never change after they are published; a correction adds new files.

- /data/layers/master/latest.json — names the current week of the master list (`base`)
- {base}summary.json — totals by group, for India (documented, with an open record, species per IUCN category, threatened with an open record, open record placed in a state, of them recorded in a state since `recent_from`) and per state (open record, threatened, recorded there since `recent_from`); `unlisted` (names not on the list, per group); `described_recent` (species on the list described since `recent_from`, the Linnean gap) and `dna` (species with a DNA sequence or sample from India on GBIF, the Darwinian gap), per group; `interactions` and `interactions_india` (species with an interaction recorded in GloBI, anywhere and in India: the Eltonian gap), per group; `trend_known` (species with a known population trend, increasing, stable or decreasing, in the latest IUCN Red List assessment, or for birds in State of India's Birds 2023: the Prestonian gap; null for groups neither covers), per group; `niche` (species with GBIF records placed to 1 km or better at 10 or more places in India: the Hutchinsonian gap; null until the weekly GBIF request has the count), per group; `traits` (species with a record placed in India that gives a life stage, a sex, or flowers or fruits: the Raunkiæran gap; null until the weekly GBIF request has the count), per group; `types` and `photos` (species with a GBIF record from India that has a type status, or a photo), `barcodes` (species with a DNA barcode on BOLD, anywhere: BOLD's barcodes on GBIF, or a BOLD record from India) and `barcodes_india` (of them, a BOLD record from India), and `local_names` (species with a name in an Indian language on Wikidata), per group, null when not measured; `gaps_known` (per group, the species where each of the seven gaps is known, in the order Linnean, Wallacean, Prestonian, Darwinian, Raunkiæran, Hutchinsonian, Eltonian; null where not measured); `papers` (birds with a paper in the Bibliography of South Asian Ornithology, compiled by Aasheesh Pittie; null for other groups); `redlist` (per group: assessed by IUCN, not assessed with open records in 5+ states, assessed before `recent_from`, Data Deficient with newer records, threatened with no recent record, endemic and not assessed; from IUCN's categories and assessment years on GBIF, CC BY 4.0); sources, labels, IUCN categories, states; `orders` (every order on the list, A to Z), `suborders`, `provinces` ([code, province, zone] for the 26 biogeographic provinces and 10 zones of India of the Wildlife Institute of India, such as ["5B", "Western Ghats: Mountains", "Western Ghats"]; there is no Eastern Ghats unit: its nearest is 6C, Eastern Highlands); `described` (range of years described, species with no year); licence, attribution, disclaimer
- {base}g/{group}.json — every species in one group: key, name, sources, status (0 no open record yet, 1 an open record), IUCN, labels, states with an open record (bit masks, in the order of summary.json), year described (null when no source gives it), latest year of a record placed in a state (null when none), order (index into summary.json `orders`; null when not known), suborder (index into `suborders`: Serpentes, snakes, or Sauria, lizards; order Squamata only, else null), WII biogeographic provinces with a GBIF record (bit mask in the order of `provinces`; null for lists built before 4 Nov 2026), English name (iNaturalist, else Wikidata; null when none; not in lists built before 9 Nov 2026)
- {base}w/{group}.json — species with iNaturalist observations that wait for a name (needs ID): the species' row as in g/{group}.json (status 2: not on the list, named only in these observations), then iNaturalist taxon id, observations waiting in India, [[state index, observations], ...]. summary.json `waiting` has the counts per group (species, observations, species with no open record in that area) and the iNaturalist place ids. Identify page: https://www.inaturalist.org/observations/identify?quality_grade=needs_id&taxon_id={taxon}&place_id={place}
- {base}wg/{group}.json — genera of iNaturalist observations named only to genus: genus, iNaturalist taxon id, observations in India, [[state index, observations], ...], species of the genus on the list, of them with no open record in India, threatened, endemic. summary.json `waiting.genus` has the counts per group and `waiting.coarse` the observations named only above genus, per iNaturalist iconic group (Identify page: add rank=genus, or iconic_taxa={group}&lrank=subtribe)
- {base}sp/{shard}.json — one object per species key: name, group, sources, status, IUCN, labels, states with an open record, iNaturalist taxon and observations waiting, year described, authority, for a species described since 2007 its paper [sources, journal, link, records after the description, latest year], then the year and DOI of its IUCN assessment, GloBI partner species (null: none recorded), an interaction recorded in India, its 2 km cells with an open record placed in India (area of occupancy; null: not counted yet), records from India with a type status, records from India with a photo, a DNA barcode (2: from India, BOLD; 1: elsewhere only; 0: none) its names in Indian languages [[language code, name]], a known population trend (State of India's Birds; null: not covered), a record placed in India with a trait, places with an exact record (null: not counted), papers in the Bibliography of South Asian Ornithology with the year of the latest (birds), and the population trend in its IUCN assessment (Increasing, Stable, Decreasing, Unknown; null: not assessed; IUCN 2026, version 2026-1), its family, its order, its suborder (Serpentes or Sauria; null for other orders), and the codes of the WII biogeographic provinces with a GBIF record placed there (null: not measured); shard = FNV-1a 32-bit hash of the key, modulo 512
- {base}fam/{shard}.json — per family (shard = the same hash of the family name): each species with a page [key, name, year described, latest record placed in a state, states with an open record, a known population trend, DNA from India, barcode, a trait record, places with an exact record, GloBI partners, an interaction in India, its picture URL or null], for the relatives on the species page
- {base}pic/{shard}.json — per species key with a picture or an English name: [image URL, its page, photographer, licence, source, English common name]; photos from India (research-grade iNaturalist observations), else iNaturalist taxon photos, else Wikimedia Commons; Creative Commons only, each photo keeps its own licence and credit; we link to the images and do not host them
- {base}web/{shard}.json — the species web (drawn as a tree): per species key {n: {kind: partner species}, p: {kind: [[partner name, its key on the list or null, records, records from India, [GloBI interaction types]]]}}, the 30 partners with most records per kind of link (eats, eaten, flowers, lives, hosts, other), from GloBI; same shards
- {base}history.json — the totals of every list so far, oldest first: measures [documented, unlisted (names not on the list, waiting for a name), observed (with an open record), in_state (open record placed in a state), recent (recorded in a state in the last 10 years), threatened (IUCN CR, EN or VU), species and observations on iNaturalist named to species but not yet confirmed (needs ID), described_recent, dna]; per list `india` and `groups` (same group order as summary.json)
- {base}changes.json — species that moved since the previous week: rows [kind, key, name, group, new states, new sources]; kinds iucn (IUCN Red List category changed; the row ends with [before, now]), observed (first open record in India), state (first open record in a state), documented (added), dropped (renamed or merged); counts per kind and per group
- {base}sitemaps/species-{0-4}.xml: the species pages for search engines (also at /sitemaps/)
- {base}mcp/rows.bin, mcp/text.txt, mcp/meta.json: the index the MCP server scans: one 32-byte little-endian row per species on the list (layout and families in meta.json), and a text line per row: key, Latin name, English name
- {base}search/{two letters}.json — search index by the first two letters of the name; {base}search/places.json — states and districts
- /data/latest.json — the current GBIF snapshot (district ledger, species history, records without a species name, records that cannot go in a district); its manifest lists every file and the GBIF download DOIs
- {snapshot}unlocated.json — GBIF records from India that cannot go in a district (no coordinates, coordinates uncertain by more than 25 km, or coordinates GBIF marks as possibly wrong): records and species per group for India and per state, records per problem, the datasets that hold most of them, and the species known from India only in such records. {snapshot}unresolved.json — records not named to species. The front page shows both as "Records to fix", for data publishers
- /data/layers/shortlist/latest.json: shortlists (experimental). run date, model, calibration version, and the groups on the page, each with its band (Quick to confirm or Genus first) and its test figures (`t1`: test observations, first name right %, right name in the list %). /data/layers/shortlist/{run}/g/{group}.json: observations on iNaturalist that wait for a name, most useful first: id, photo URL with its attribution and licence, date, place, LGD state code, the name on iNaturalist now, band, and the names to check (up to 3 species, or up to 5 genera). A machine shortlist, not an identification: a person checks every name on iNaturalist
- /data/layers/inat/latest.json — iNaturalist species lists per state (species, genus, above genus); /data/layers/newspecies/latest.json — species described from India since 2007

## Rights

Data and text: CC BY 4.0, Sathya Sankaran. Each source keeps its own licence: cite the GBIF download DOIs; credit the Botanical Survey of India and the Zoological Survey of India (Ministry of Environment, Forest and Climate Change, Government of India); biogeographic zones: Wildlife Institute of India (GODL-India), via bharatlas; IUCN Red List data is for non-commercial use with attribution. Every summary.json carries `licence`, `attribution` and `disclaimer`: pass them on with any answer built from this data.

For the status of a species, people should consult BSI, ZSI, the IUCN Red List and the state forest department. Overlooked is a monitor of public data, not an authority.

## Pages

- / — map of India, groups, search with filters by year described, source, status, IUCN category and label (the filters drive the map and the figures; `?described=2007-` gives the species described since 2007); in the Waiting view, the observations that wait for a name, with links to iNaturalist's Identify page
- /species?key={key} (the page carries the species' title, description, schema.org Taxon and a plain summary without JavaScript; all species are in /sitemap.xml) — one species: sources, status, IUCN, where it has an open record, observations that wait for a name, GBIF records by decade and district
- /changes — what changed since last week
- /fix — records to fix, for data publishers: GBIF records from India with no exact location or no species name, by dataset, state and group (`?state={LGD code}&group={family}`)
- /shortlist: (experimental) observations on iNaturalist that wait for a name, each with a short list of names from a model for a person to check; we post nothing to iNaturalist
- /method — sources, terms, definitions, downloads
- /mcp: for people, how to add the MCP server to an AI assistant, with examples; for agents, the MCP server itself
- Code (MIT): https://github.com/urbanmorph/overlooked