Perception
Jobs
client.jobs reads, lists, waits for and cancels jobs. job_id is a str or a uuid.UUID; anything that isn't a job id raises NaceError before anything is sent.
| Method | Parameters | Returns | Description |
|---|---|---|---|
get(job_id) | job_id | DocumentJob | One job and, once it has succeeded, its result until retention drops it. Classify may include a partial result while still running. Another account's job is a 404. |
list(**options) | limit: int (1 to 50, default 20), operation: str, status: str, cursor: str | DocumentJobList | One page of jobs, newest first, without results. Pass next_cursor as cursor for the next page. |
iter(**options) | operation: str, status: str | Iterator[DocumentJob] | Every job across every page, 50 a page. |
wait(job_id, *, timeout=600, raise_on_failure=True) | job_id, timeout: float (seconds), raise_on_failure: bool | DocumentJob | Polls until the job is terminal. See Wait for a job. |
delete(job_id) | job_id | None | Cancels a running job and drops it from the list. A finished job is only dropped. It stays readable by id. A cancelled job is not charged unless results were already delivered before the cancel; a finished one keeps its charge. |
events(job_id, *, last_event_id=None) | job_id, last_event_id: str | Iterator[JobEvent] | Streams progress as Server-Sent Events. See Progress events. |
list with status filters on the status Drex last recorded and leaves off a job that has moved on since, so a page can hold fewer jobs than limit, even none, while next_cursor says more follow. Keep paging until next_cursor is None, or use iter. A limit outside 1 to 50 raises NaceError.
client.jobs.list(limit=20, operation="parse", status="succeeded")
for job in client.jobs.iter(operation="extract"):
print(job.job_id, job.status, job.credits)
client.jobs.delete(job_id)See Jobs and results.
Wait for a job
wait polls get until status is succeeded, failed or cancelled: once straight away, then after 1 second, doubling to every 10 seconds, for up to timeout seconds in all, 600 by default. Each poll is a get, with the client's retries.
- Failed or cancelled: raises
NaceJobFailedError, with the job aserror.job. Withraise_on_failure=False, it returns the job instead. - Out of time: raises
NaceJobTimeoutError, witherror.jobanderror.timeout. The job keeps running; wait again or read it later.
from nace_sdk import NaceJobFailedError, NaceJobTimeoutError
try:
job = client.jobs.wait(job_id, timeout=1200)
except NaceJobFailedError as error:
print(error.job.error_code, error.job.error_message)
except NaceJobTimeoutError as error:
print("still", error.job.status)Progress events
events yields a JobEvent whenever the job's status or progress changes. The first event is the job's current state, and the stream ends when the job is terminal or after five minutes. It is a latency convenience: read the outcome with get or wait. The stream is never retried and counts against your per-minute limit. After a drop, pass the last event's sequence to continue its numbering:
last = None
for event in client.jobs.events(job_id):
last = event
print(event.sequence, event.status, event.progress)
if last is not None and last.status not in ("succeeded", "failed", "cancelled"):
for event in client.jobs.events(job_id, last_event_id=str(last.sequence)):
print(event.sequence, event.status)Job files
A job file's path is a link from the result, such as document.content_url, a sheet's rows_url or a figure block's image_url, or the part after the job id, such as parse-images/<ref>. Its query and fragment are ignored. A path that is empty or holds . or .. raises NaceError, and so does a full link from another job's result, before any request: pass that job's id instead.
| Method | Parameters | Returns | Description |
|---|---|---|---|
download(job_id, path, destination, *, wait_timeout=600) | job_id, path, destination: str | Path, wait_timeout: float (seconds) | Path | Saves any file the job produced. See Download a file. |
file_link(job_id, path) | job_id, path | FileLink | A five-minute signed link to a stored file. See Signed links. |
get_request(job_id) | job_id | JobRequest | The request the job ran under. See Job request. |
rows(job_id, path, *, start_row=None, limit=None) | job_id, path, start_row: int, limit: int | RowsPage | One page of a parsed sheet's rows. See Spreadsheet rows. |
| Kind | Path starts with | Read it with |
|---|---|---|
| Split segment files, figure crops, parse transcripts, ground crops, ground transcripts, converted PDFs, cell maps | split-artifacts/, parse-images/, parse-transcript/, ground-crops/, ground-transcript/, converted-pdf/, spreadsheet-cell-maps/ | download, or file_link for a signed link |
| Full Markdown of a document or sheet | document-content/, spreadsheet-content/ | download |
| A sheet's rows | spreadsheet-rows/ | rows, or download for one page as JSON |
| The request echo | request | get_request |
| The event stream | events | events |
Downloads of job files don't count against your per-minute limit; the request echo and the event stream do. Results and their files expire on the document service's retention schedule. After that, job files and the request echo raise NaceNotFoundError (404, or 410 with result_expired), while get and wait still return the job, with result set to None and job.result_state saying why. See Jobs and results.
Download a file
download(job_id, path, destination, *, wait_timeout=600) saves any job file to destination, overwriting it, and returns its Path. A stored file comes from its signed link, without your API key; everything else streams from Drex.
Markdown the document service is still preparing answers 202: download waits as long as its retry-after header or retry_after_seconds says (else 3 seconds) and asks again, for up to wait_timeout seconds, then raises NaceError; the content keeps being prepared, so download it again later. Preparing content that fails raises NaceConflictError. Opening each request is retried under the client's policy; a transfer cut off midway is not.
from nace_sdk import NaceClient
with NaceClient() as client:
job = client.jobs.wait(client.documents.parse("https://example.com/ledger.parquet").job_id)
document = job.result["document"]
if document.get("content_url"): # markdown is only a preview
client.jobs.download(job.job_id, document["content_url"], "ledger.md")Signed links
file_link(job_id, path) returns a FileLink: url, which works for five minutes without your API key, to hand to a browser or another service, and expires_at, in Unix seconds. Only stored files have one: split-artifacts, ground-crops, ground-transcript, parse-images, parse-transcript, converted-pdf and spreadsheet-cell-maps. Any other path raises NaceError pointing at download before a request goes out; so does a stored file where the document service can't sign links, after the request. A stored-file path this job doesn't have raises NaceNotFoundError (404). download works in every case.
client.jobs.download(job_id, "parse-images/<ref>", "figure-1.png")
link = client.jobs.file_link(job_id, "parse-images/<ref>")
print(link.url, link.expires_at)Job request
get_request(job_id) returns a JobRequest: request_type (the operation), and what the job was asked for, such as its source, options, schema, classes or targets, in model_extra. A PDF password is never returned. It expires with the job's result.
print(client.jobs.get_request(job_id).model_extra)Spreadsheet rows
rows(job_id, path, *, start_row=None, limit=None) reads one page of a parsed sheet's rows, in file order: path is the sheet's rows_url, which only Parquet sources carry, start_row is zero-based (default 0), and limit is 1 to 100 (MAX_ROWS_PAGE_SIZE, default 50). A negative start_row or a limit out of range raises NaceError. It returns a RowsPage.
for sheet in (job.result["document"].get("spreadsheet") or {}).get("sheets", []):
if not sheet.get("rows_url"):
continue
start = 0
while True:
page = client.jobs.rows(job.job_id, sheet["rows_url"], start_row=start, limit=100)
print([column.name for column in page.columns], page.rows[:2])
start += len(page.rows)
if not page.rows or start >= page.total_rows:
break