Pangur docsTEI texts and transcription exports
This page is the reference for the formats in which Pangur exports a transcription, with or without page images, and for the TEI exchange with ESTC, in which you import a book's TEI text, correct it against page images and send a revision back.
This page is a reference for two related things. The first is exporting a transcription made on the transcription desk as TEI, PAGE XML, ALTO or plain text. The second is the TEI exchange for ESTC books, in which you import a TEI edition of a book, correct it against its page images, and export a new revision for ESTC. The two keep separate records. A transcription's named versions and a TEI text's revisions are different things, and changing one does not change the other.
Export formats
Choose Export in the transcription desk's bar, then a format:
| Menu entry | What you get |
|---|---|
| TEI P5 › Both readings | One TEI file with the diplomatic and expanded readings of each line, and each line's place on the page |
| TEI P5 › Diplomatic reading | One TEI file with the diplomatic reading only |
| TEI P5 › Expanded reading | One TEI file with the expanded reading only |
| PAGE XML | A ZIP of one PAGE XML file per page, for eScriptorium or Transkribus |
| ALTO | A ZIP of one ALTO file per page, for eScriptorium |
| With page images › PAGE XML and images | PAGE XML with the page images, as a folder Transkribus imports as it is |
| With page images › ALTO and images | ALTO with the page images |
| With page images › TEI and images | TEI with the page images |
| Plain text | The main text only, in the diplomatic or the expanded reading |
The formats differ in what they carry. TEI P5 records the page images as facsimile surfaces and zones, so that each line points at its region of the image. The two readings of a line become a <choice> of <orig> and <reg type="expanded">, and scholarly markup becomes <add>, <del>, <unclear>, <gap> and <supplied>. PAGE XML follows the 2019 PAGE schema and ALTO follows ALTO version 4.4, and both name text regions with the SegmOnto vocabulary, which eScriptorium and other tools use for kinds of region such as main text, margins and numbering. Plain text contains the main text only, without running titles, numbering or other page furniture.
If there is no transcription yet, Pangur tells you there is nothing to export.
Exports with page images
An export with page images is split into ZIP files of at most 60 pages each, and the web client downloads each part in turn. If Pangur could not fetch some images, because a repository did not respond for instance, it tells you which ones were left out.
Exporting part of a transcription, or a version
From the command line, --pages limits an export to some pages, written as a range such as 1-40 or a list such as 3,5,9-12, with up to 200 ranges. --version exports a named version instead of the current draft. For example:
pangur manuscript export ITEM --format page --images --pages 1-40This writes a ZIP of PAGE XML files and images for the first 40 pages, in several parts if needed. --format takes tei, page, alto or txt, and --reading takes diplomatic or expanded. Without --reading, TEI and ALTO keep both readings. The options are listed in the command-line reference.
An assistant connected over MCP can export with export_transcription, in TEI P5, PAGE XML, ALTO or plain text. That tool cannot fetch page images, but the assistant can get a short-lived link to a page image with temporary_page_link instead.
In an organisation's project, a member with the viewer role can export transcriptions.
TEI texts for ESTC books
A book with an ESTC number (from the English Short Title Catalogue) can hold an imported TEI edition of its text. Pangur keeps the edition exactly as you imported it, and every correction you save becomes a new revision, so earlier revisions are never changed. While an item has an imported edition, that edition is the item's text and takes precedence over any transcription, which means Ask and the reading room read it.
The TEI exchange lives in the item's Transcribe tab, under ESTC transcription, and only appears on books with an ESTC number.
Importing a text bundle
ESTC supplies a book's text as a text bundle, which is a .json file. Choose Import a text bundle and pick the file, which must be smaller than 10 MB. Pangur checks the bundle's checksum, refuses external XML declarations, and imports the TEI, which can be up to 8,000,000 characters long. Importing the same bundle twice does nothing the second time. If the same revision was already imported with different text, Pangur refuses the import and says so.
Correcting the text against the page images
Choose Open the TEI desk to work on the text beside the images.
- Under Revision, choose the latest revision. You can look at earlier revisions, but only the latest can be corrected.
- Under Image copy, choose the copy of the book whose images you want to use, and under TEI page the page you are correcting. Pages are listed by label and page break.
- Detect and read the lines of that page's image, if you have not already. Matching needs them.
- Choose Match recognised lines to this page. Pangur compares the words read from the image with the words of the TEI page and proposes which part of the TEI each line corresponds to. It shows each line as matched or unmatched with a percentage, and reports how many words remain unmatched. Matching changes no text.
- Choose Link this image to the TEI page to record that this image shows this page. Linking replaces any earlier link for the page and does not make a new revision.
- Correct the TEI source and choose Save new revision.
Matching compares words exactly, ignoring case and treating the long s as s. It proposes a match only where at least two words agree, in order, with at least 80% confidence. A page region of more than 20,000 words is too large to match and is refused.
If someone saved a revision after you opened the text, Pangur refuses your save and asks you to reload first.
Giving ESTC a page image
ESTC's editor can take a page image by address. Choose Get a temporary image link for ESTC and paste the link into ESTC's page-image URL field. The link is for that page only, works for 30 minutes, and checks your permission each time it is used.
Sending the corrected text back
Choose Export revision for ESTC to download the revision as a bundle named pangur-text- followed by the text's identifier and revision number, and import that bundle into ESTC.
TEI texts from an assistant
An assistant connected over MCP can list, read, import, revise and export TEI texts with list_tei_texts, read_tei_text, import_tei_text, revise_tei_text and export_tei_text, and propose matches with align_tei_page. A revision from an assistant is appended like your own, and the original and every earlier revision are kept (MCP tools).
To import a whole ESTC collection as a library collection and an analysis corpus, see Work with corpora.
Reading pages with a model
Transcribing a page image is two jobs, finding the lines and reading them, and Pangur does them with different software. This page explains which parts use a vision model, what the daily limits are and whose they are, and how a connected assistant can do the reading itself.
How Ask answers
This page explains what Pangur does between your question and its answer, what an answer can and cannot rest on, and which limits and allowances a question uses.