library add

Invocation#

lem-cli library add <source> [--pdf <path>] [--text <path>]

Inputs and options#

• The first argument is an arXiv id or arxiv.org URL, a DOI or doi.org URL, or the http(s) URL of the page that published the work.
• --pdf supplies a local PDF. --text supplies local text used when text cannot be extracted from the PDF. A ~/ prefix is expanded.

Output and state#

Metadata comes from the source's own publisher. An arXiv id is read from the arXiv Atom API: the entry whose id names it, which must carry a title, an abstract, its abstract page and its PDF; an answer without that entry refuses naming the ids it holds. A DOI is read from the DOI system's registered CSL record rather than from the page it redirects to, because a registrant may serve a storefront whose site description would otherwise be stored as the paper's abstract; its date is CSL's issued date, else the published, print or online date the agency registered, and its venue the container title, else the publisher. When no agency answers, the page the DOI resolves to is read; when that fails too, both causes are named. A page is read through its Highwire citation_* tags as Google Scholar's inclusion guidelines name them (citation_title, else the document title; citation_author; citation_publication_date or citation_date as YYYY/MM/DD or YYYY; the journal, conference or book title; citation_doi, citation_abstract, citation_keywords and citation_pdf_url), a meta tag's content being read under every key it names (name, property or itemprop); a page that carries a DOI is keyed on that DOI. A page with neither a citation_title nor a document title refuses.

Upserts one record by that identifier, keeps <id>.pdf and <id>.txt under the manifest's asset directory when available, and prints the saved record as JSON. A page with no PDF is kept as page text with MathML and SVG dropped and TeX annotations preserved.

Re-adding a work refreshes the record: it keeps the original added_at, the saved analysis, and every field the new reading did not supply, because the DOI agency holds the authors and the venue while the article page holds the PDF. What a source did not publish is recorded in acquisition_notes.

Acquisition notes tell a publisher's answer from this machine's own failure: “the source answered with a page instead of a PDF” when the downloaded bytes are not a PDF (the file is removed; a removal that fails is named in the same note), “<path> was downloaded but could not be read back: <cause>” when the file is there but unreadable (it is kept for inspection), and “the stored text <path> could not be read: <cause>” when a previously stored text can no longer be read; a record never loses its text in silence.

Refusals#

• A missing source refuses with “library add needs one arXiv id, DOI, or URL”. An unknown option refuses with “library add does not recognize <value>”; --pdf or --text without a path refuses with “<option> needs a path”.
• A source that is neither an arXiv id, a DOI, nor an address with a host refuses with “'<value>' is not an arXiv id, a DOI, or an http(s) URL”; an address whose scheme cannot be fetched is refused by the fetch, naming it. A page that publishes no title refuses with “could not read source metadata: <url>: the page published no citation_title, og:title, or document title”. A DOI resolver answer that is not CSL JSON, or a CSL field in a shape CSL does not define (a date part that is not a number, a link without an address), refuses with “could not read source metadata: <doi>: the DOI resolver's answer is not a CSL record: <the decoder's reason>”; a record without a title refuses with “<doi>: the CSL record carried no title”.
• A --pdf or --text file that cannot be read refuses with “could not read local literature file at <path>: <cause>”, the cause being the file system's own. A local file without a PDF header, a metadata failure, or a manifest write failure is printed as a lem-cli refusal and exits nonzero.