# cane_format

> Read, write and parse CANE, a natural-language output format for LLM-based named entity recognition.

## Docs

### API Reference

#### Parsing and formatting CANE
> Turn model output into character spans, and documents into CANE.

- [loads](https://coree.github.io/cane-format/reference/loads.html): Parse CANE model output and ground every record in ``text``
- [dumps](https://coree.github.io/cane-format/reference/dumps.html): Write a document's entities as CANE text, e.g. for a few-shot example
- [ParseOptions](https://coree.github.io/cane-format/reference/ParseOptions.html): How model output is parsed. Every default is strict. Recorded on each ``ParseResult``
- [ParseResult](https://coree.github.io/cane-format/reference/ParseResult.html): The grounded document, everything that could not be grounded, and the settings used
- [ParseIssue](https://coree.github.io/cane-format/reference/ParseIssue.html): One problem found while parsing model output
- [IssueKind](https://coree.github.io/cane-format/reference/IssueKind.html): Closed vocabulary of problems found while parsing model output
- [Variant](https://coree.github.io/cane-format/reference/Variant.html): The four CANE variants: one grammar, two switches (several parts, anchors)
- [Dialect](https://coree.github.io/cane-format/reference/Dialect.html): Every literal token of the CANE surface form. ``ENGLISH`` is the default
- [ENGLISH](https://coree.github.io/cane-format/reference/ENGLISH.html): Every literal token of the CANE surface form. ``ENGLISH`` is the default

#### Prompts and evaluation
> Few-shot prompts in the CANE paper's layout, and spans for nervaluate-disco.

- [build_prompt](https://coree.github.io/cane-format/reference/build_prompt.html): A few-shot prompt asking a model to annotate ``text`` in CANE
- [guideline](https://coree.github.io/cane-format/reference/guideline.html): The format instructions for one CANE variant, worded with the dialect's tokens
- [to_nervaluate](https://coree.github.io/cane-format/reference/to_nervaluate.html): Documents in nervaluate-disco's span format, one dict per document

#### Document model
> Text with entities, relations and segments; immutable and validated.

- [Document](https://coree.github.io/cane-format/reference/Document.html): A text with its annotations
- [Entity](https://coree.github.io/cane-format/reference/Entity.html): A labelled annotation over one or more spans (more than one = discontinuous)
- [Span](https://coree.github.io/cane-format/reference/Span.html): A contiguous character range ``text[start:end]``. Requires ``0 <= start < end``
- [Relation](https://coree.github.io/cane-format/reference/Relation.html): A relation, event or equivalence set over entities and relations
- [Member](https://coree.github.io/cane-format/reference/Member.html): A participant in a relation: the ID of an entity or relation, and its role
- [Concept](https://coree.github.io/cane-format/reference/Concept.html): A link from an entity to a concept in a vocabulary
- [Segment](https://coree.github.io/cane-format/reference/Segment.html): A typed region of the text, such as a passage, sentence or token

#### brat standoff

- [read_brat](https://coree.github.io/cane-format/reference/read_brat.html): Read a brat directory, or one document given its ``.txt`` or ``.ann`` path
- [iter_brat](https://coree.github.io/cane-format/reference/iter_brat.html): Like ``read_brat``, yielding documents one at a time (sorted by file name)
- [parse_brat](https://coree.github.io/cane-format/reference/parse_brat.html): Build a document from the contents of a ``.txt`` and an ``.ann`` file
- [write_brat](https://coree.github.io/cane-format/reference/write_brat.html): Write each document as ``<id>.txt`` and ``<id>.ann`` into ``directory``
- [format_brat](https://coree.github.io/cane-format/reference/format_brat.html): The ``.ann`` content for a document. The text is written unchanged as the ``.txt``

#### BioC JSON

- [read_bioc_json](https://coree.github.io/cane-format/reference/read_bioc_json.html): Read a BioC JSON file. See ``parse_bioc_json``
- [iter_bioc_json](https://coree.github.io/cane-format/reference/iter_bioc_json.html): Like ``read_bioc_json``, yielding documents one at a time
- [parse_bioc_json](https://coree.github.io/cane-format/reference/parse_bioc_json.html): Build documents from BioC JSON: a collection, a list of collections, or a PubTator3 export
- [write_bioc_json](https://coree.github.io/cane-format/reference/write_bioc_json.html): Write documents as one BioC JSON collection. Options as for ``format_bioc_json``
- [format_bioc_json](https://coree.github.io/cane-format/reference/format_bioc_json.html): Documents as one BioC JSON collection

#### CoNLL / IOB

- [read_conll](https://coree.github.io/cane-format/reference/read_conll.html): Read a CoNLL file. See ``parse_conll``; ``encoding`` is the file's text encoding
- [iter_conll](https://coree.github.io/cane-format/reference/iter_conll.html): Like ``read_conll``, yielding documents one at a time
- [parse_conll](https://coree.github.io/cane-format/reference/parse_conll.html): Build documents from CoNLL content
- [write_conll](https://coree.github.io/cane-format/reference/write_conll.html): Write documents to one CoNLL file or stream. Options as for ``format_conll``
- [format_conll](https://coree.github.io/cane-format/reference/format_conll.html): One document as CoNLL lines (ending with a blank line)
- [from_tokens](https://coree.github.io/cane-format/reference/from_tokens.html): One sentence given as tokens and tags, e.g. a Hugging Face ``tokens``/``ner_tags`` row
- [align](https://coree.github.io/cane-format/reference/align.html): Move a CoNLL document's annotations onto its true raw text

#### Inline XML

- [read_xml](https://coree.github.io/cane-format/reference/read_xml.html): Read an XML file. Options as for ``parse_xml``
- [iter_xml](https://coree.github.io/cane-format/reference/iter_xml.html): Like ``read_xml``, yielding documents one at a time
- [parse_xml](https://coree.github.io/cane-format/reference/parse_xml.html): Build documents from inline XML
- [write_xml](https://coree.github.io/cane-format/reference/write_xml.html): Write ``<documents><document id=…>…</document>…</documents>`` to a path or stream
- [format_xml](https://coree.github.io/cane-format/reference/format_xml.html): The document text with its entities as inline elements

#### Offset JSON

- [read_json](https://coree.github.io/cane-format/reference/read_json.html): Read a JSON Lines file, a single JSON document, or a JSON array of documents
- [iter_json](https://coree.github.io/cane-format/reference/iter_json.html): Like ``read_json``, yielding documents one at a time
- [parse_json](https://coree.github.io/cane-format/reference/parse_json.html): Build a document from one JSON object
- [write_json](https://coree.github.io/cane-format/reference/write_json.html): Write documents as JSON Lines, one per line, to a path or an open text stream
- [format_json](https://coree.github.io/cane-format/reference/format_json.html): One document as a JSON object (on one line unless ``indent`` is given)

#### Errors and warnings

- [CaneFormatError](https://coree.github.io/cane-format/reference/CaneFormatError.html): Base class of every exception raised by cane-format
- [ParseError](https://coree.github.io/cane-format/reference/ParseError.html): Raised by ``ParseResult.raise_for_errors()``; carries every issue of the parse
- [IntegrityError](https://coree.github.io/cane-format/reference/IntegrityError.html): A file's offsets do not reproduce the surface text it stores next to them
- [UnrepresentableError](https://coree.github.io/cane-format/reference/UnrepresentableError.html): A writer or the CANE formatter was asked to write something its target cannot hold
- [LossyConversionWarning](https://coree.github.io/cane-format/reference/LossyConversionWarning.html): Information was dropped because the target format cannot hold it