19 tasks · one specimen parser
Every question you can ask a herbarium sheet with a camera.
All of it rests on one primitive: segment the sheet into its regions — organs, label block, determination slips, packet, barcode, colour bar — then run task-specific heads on the right region. One parser, many heads. Contributors bring heads. The parser and the evaluation stay shared.
10 of the 19 heads below have essentially no prior work. That is not an oversight in the field. It is what happens when a discipline’s machine-learning attention goes to the two tasks that publish well.
Type specimens — the sheets that anchor a botanical name
Running now
The first task on this bench works today, on any specimen you point it at
Region detection needs no training data, which is why it already runs on museums this project has never seen. Everything else on this list is built on top of it.
Run it →Five scales, one pipeline
Seed identification and sheet phenology look like different projects only because nobody draws the ladder between them. A collection is a nested hierarchy of imaged objects, and every vision task in botany attaches to one rung. They share every hard part: find the object in the frame, score a trait, record whose claim it is, never overwrite a human determination.
The hinge is the fragment packet. It sits on the sheet, so the sheet parser finds it — and it holds loose seed and fruit, which is propagule-scale material. Detect the packet and a sheet corpus becomes a seed corpus. That one region is the bridge between a phenology atlas and a seed-identification tool.
- 00live
Collection
the cabinet
Aggregate structure: what is held, where the gaps are, which taxa and islands are represented and which are not. Nothing is measured here — this is the scale at which you learn what you cannot yet answer.
No imaging. Metadata, coverage, sample size.
- 01live
Sheet
one mounted specimen
The whole herbarium sheet as photographed: plant, label, apparatus, ageing mount. Region detection happens here, and everything below depends on it — you cannot crop an organ you have not located.
Standard flatbed or copy-stand capture. Roughly 40 million exist.
- 02building
Organ
a leaf, a fruit, an inflorescence
Crops from the sheet, at the resolution where traits are actually measurable. Leaf area, margin type, herbivory, fruit maturity. This is the scale the sheet-level models squint at and miss.
Derived crops at full sheet resolution, or targeted re-imaging.
- 03open
Propagule
a seed, a fruit, a spore
Loose reproductive material, imaged in its own right. Z-stacked seed photography at multiple orientations produces reference sets precise enough to identify an intercepted seed at a port of entry. The fragment packet on a herbarium sheet is where this material already lives, uncatalogued.
Z-stacked macro, multiple orientations, scale-calibrated.
- 04open
Micro
a stoma, a trichome, a pollen grain
Features below the resolution of ordinary specimen photography. Stomatal density tracks atmospheric carbon; trichome and pollen morphology carry taxonomic signal nothing else does. Herbaria are an accidental instrumental record at this scale.
Microscopy on sampled material. Destructive or semi-destructive.
Every task below runs over sheets like these · browse the collection →
The plant
Pressed biological material. Traits, organs, damage, reproductive state.
Reproductive state
buildingIs this specimen flowering, fruiting, budding, or sterile?
A century of sheets is a century of phenological observation. Shifts in flowering time are one of the clearest biological signals of a warming climate, and the record is sitting in cabinets, unscored.
Contested — several groups working
Fruit and seed presence
buildingDoes this sheet indicate collectable seed at that date and place?
Seed banks need to know when a population fruits before they send a team. The question is not academic phenology — it is whether a collecting trip in April will come home with anything.
Empty — nobody is doing this
Leaf morphometrics
openWhat are the leaf area, aspect ratio, margin type and venation?
The proportion of entire-margined species in a flora tracks mean annual temperature. Leaf shape is a palaeoclimate instrument, and every sheet is a reading.
Thin — some prior work
Herbivory damage
openWhat fraction of leaf area was removed by insects?
Insect pressure over a century, measurable against warming. The studies that have done this counted damage by eye on a few thousand sheets. There are millions.
Empty — nobody is doing this
Pathogen and gall signature
openDoes this specimen carry rust, mildew, leaf spot or galls?
A historical plant-disease record nobody has read. Pathogen range shifts have the same climate signal as phenology and none of the attention.
Empty — nobody is doing this
Bycatch organisms
openWhat else got pressed with the plant?
Insects, spiders, lichens, bryophytes and epiphylls were preserved accidentally and recorded nowhere. Herbaria are unintentional entomological collections. Nobody has looked systematically.
Empty — nobody is doing this
The label
Typed or handwritten text. Who, where, when, and what they thought it was.
Historical handwriting
openWhat does this nineteenth-century label say?
Printed label OCR is solved. Cursive is not, and it is the single largest blocker on mass digitisation. Millions of sheets stay dark because nobody can read them at scale.
Thin — some prior work
Collector attribution
openWhose hand wrote this unsigned label?
Many old sheets carry no collector name. Handwriting style matched against known exemplars can restore the attribution, and rolls up into a network graph of who collected with whom.
Empty — nobody is doing this
Phenology in free text
liveDid the collector already write down what the model is trying to infer, and can you trust the tag?
Collectors routinely wrote in fruit, fl., sterile — an expert label generated in the field, sitting in pixels. But a keyword scan that fires on flower-bearing organs tags a sterile sheet as flowering. Running OCR on the parser's label region and mining only assertive phrases separates the real claim from the morphology note. On this harvest, most keyword tags checked this way turned out to be the descriptive trap, and OCR recovered real assertions the metadata never carried.
Thin — some prior work
Type status detection
openIs this a type specimen the database does not know about?
Type specimens anchor every name in botany. Holotype is often written on the sheet and absent from the record. Each hit is nomenclaturally load-bearing.
Thin — some prior work
Locality to coordinates
openWhere is Nanumea, and how wrong might that be?
A quarter of the harvest has no coordinates. Locality text does. Georeferencing is the difference between a specimen and a data point — and the uncertainty radius matters as much as the centre.
Contested — several groups working
The apparatus
Barcode, colour bar, stamps, mounting tape, fragment packet, determination slips.
The determination stack
buildingWho has claimed what about this specimen, and when?
The slips glued down one over another are an append-only ledger of taxonomic opinion — signed, dated, and never erasing what came before. Reading the stack recovers a two-century argument that no database records.
Empty — nobody is doing this
Cross-institutional duplicates
buildingWhich sheets in different countries came off the same plant?
One collection event was split and mailed to a dozen herbaria. Reuniting them builds a global collection-event graph — and for colonial-era collecting, a map of where a place's plants actually ended up.
Thin — some prior work
Misidentification triage
buildingWhich sheets do not look like what they are filed as?
Curators do not want an oracle that names species. They want a worklist. Ranking the cabinet by disagreement between the filed name and the image is a far easier problem with far higher adoption.
Thin — some prior work
Undescribed species detection
openIs there something in this cabinet nobody has named?
New species routinely sit in herbaria for decades between collection and description. An outlier detector over morphospace is a discovery engine pointed at material already collected.
Thin — some prior work
The artifact
The sheet ageing. Pest damage, mould, adhesive failure, chemical residue.
Condition and pest damage
openWhich sheets are being eaten, moulded, or falling off their mounts?
Unglamorous and fundable. Pest outbreaks in a herbarium are a budget line and an insurance question, and collections monitor them by walking the aisles.
Empty — nobody is doing this
Mercuric chloride residue
openShould the next person to open this folder be wearing gloves?
Historic sheets were treated with mercury salts against pests. Residue shows as characteristic staining and is a handling hazard. A model that flags it is an occupational-safety tool, and it opens every door in the field.
Empty — nobody is doing this
Imaging intake QC
openIs this scan skewed, soft, mis-cropped, or missing its colour bar?
Every mass-digitisation project needs this and most do it by eyeball. It is the least interesting model on this bench and probably the most immediately useful.
Empty — nobody is doing this
Loss reconstruction
openWhat was on the part of the leaf that broke off?
You cannot measure leaf area on a torn leaf, and torn leaves are everywhere. Reconstructing missing lamina unblocks morphometrics at scale — and the fill propagates into a measurement, into a climate inference. It has to travel as a claim, never as a repair.
Empty — nobody is doing this