---
name: cane-format
description: >
  Read, write and parse CANE, a natural-language output format for LLM-based named entity recognition. Use when writing Python code that uses the cane_format package.
license: MIT
compatibility: Requires Python >=3.10.
---

# cane-format

Read, write and parse CANE, a natural-language output format for LLM-based named entity recognition.

## Installation

```bash
pip install cane-format
```

## API overview

### Parsing and formatting CANE

Turn model output into character spans, and documents into CANE.

- `loads`: Parse CANE model output and ground every record in ``text``
- `dumps`: Write a document's entities as CANE text, e.g. for a few-shot example
- `ParseOptions`: How model output is parsed. Every default is strict. Recorded on each ``ParseResult``
- `ParseResult`: The grounded document, everything that could not be grounded, and the settings used
- `ParseIssue`: One problem found while parsing model output
- `IssueKind`: Closed vocabulary of problems found while parsing model output
- `Variant`: The four CANE variants: one grammar, two switches (several parts, anchors)
- `Dialect`: Every literal token of the CANE surface form. ``ENGLISH`` is the default
- `ENGLISH`: Every literal token of the CANE surface form. ``ENGLISH`` is the default

### Prompts and evaluation

Few-shot prompts in the CANE paper's layout, and spans for nervaluate-disco.

- `build_prompt`: A few-shot prompt asking a model to annotate ``text`` in CANE
- `guideline`: The format instructions for one CANE variant, worded with the dialect's tokens
- `to_nervaluate`: Documents in nervaluate-disco's span format, one dict per document

### Document model

Text with entities, relations and segments; immutable and validated.

- `Document`: A text with its annotations
- `Entity`: A labelled annotation over one or more spans (more than one = discontinuous)
- `Span`: A contiguous character range ``text[start:end]``. Requires ``0 <= start < end``
- `Relation`: A relation, event or equivalence set over entities and relations
- `Member`: A participant in a relation: the ID of an entity or relation, and its role
- `Concept`: A link from an entity to a concept in a vocabulary
- `Segment`: A typed region of the text, such as a passage, sentence or token

### brat standoff

- `read_brat`: Read a brat directory, or one document given its ``.txt`` or ``.ann`` path
- `iter_brat`: Like ``read_brat``, yielding documents one at a time (sorted by file name)
- `parse_brat`: Build a document from the contents of a ``.txt`` and an ``.ann`` file
- `write_brat`: Write each document as ``<id>.txt`` and ``<id>.ann`` into ``directory``
- `format_brat`: The ``.ann`` content for a document. The text is written unchanged as the ``.txt``

### BioC JSON

- `read_bioc_json`: Read a BioC JSON file. See ``parse_bioc_json``
- `iter_bioc_json`: Like ``read_bioc_json``, yielding documents one at a time
- `parse_bioc_json`: Build documents from BioC JSON: a collection, a list of collections, or a PubTator3 export
- `write_bioc_json`: Write documents as one BioC JSON collection. Options as for ``format_bioc_json``
- `format_bioc_json`: Documents as one BioC JSON collection

### CoNLL / IOB

- `read_conll`: Read a CoNLL file. See ``parse_conll``; ``encoding`` is the file's text encoding
- `iter_conll`: Like ``read_conll``, yielding documents one at a time
- `parse_conll`: Build documents from CoNLL content
- `write_conll`: Write documents to one CoNLL file or stream. Options as for ``format_conll``
- `format_conll`: One document as CoNLL lines (ending with a blank line)
- `from_tokens`: One sentence given as tokens and tags, e.g. a Hugging Face ``tokens``/``ner_tags`` row
- `align`: Move a CoNLL document's annotations onto its true raw text

### Inline XML

- `read_xml`: Read an XML file. Options as for ``parse_xml``
- `iter_xml`: Like ``read_xml``, yielding documents one at a time
- `parse_xml`: Build documents from inline XML
- `write_xml`: Write ``<documents><document id=…>…</document>…</documents>`` to a path or stream
- `format_xml`: The document text with its entities as inline elements

### Offset JSON

- `read_json`: Read a JSON Lines file, a single JSON document, or a JSON array of documents
- `iter_json`: Like ``read_json``, yielding documents one at a time
- `parse_json`: Build a document from one JSON object
- `write_json`: Write documents as JSON Lines, one per line, to a path or an open text stream
- `format_json`: One document as a JSON object (on one line unless ``indent`` is given)

### Errors and warnings

- `CaneFormatError`: Base class of every exception raised by cane-format
- `ParseError`: Raised by ``ParseResult.raise_for_errors()``; carries every issue of the parse
- `IntegrityError`: A file's offsets do not reproduce the surface text it stores next to them
- `UnrepresentableError`: A writer or the CANE formatter was asked to write something its target cannot hold
- `LossyConversionWarning`: Information was dropped because the target format cannot hold it

## Resources

- [Full documentation](https://coree.github.io/cane-format/)
- [llms.txt](llms.txt) — Indexed API reference for LLMs
- [llms-full.txt](llms-full.txt) — Comprehensive documentation for LLMs
- [Source code](https://github.com/coree/cane-format)