ast dataset

Inspect and curate the evaluation datasets in the active account. A dataset holds the traces you have curated from a deployed agent, with their evaluator values.

ast dataset <subcommand>

Every subcommand except list takes a dataset name. Run dataset list to see the names. The list shows datasets for currently deployed agents.

dataset list

ast dataset list [--limit <n>] [--offset <n>] [--json]

Lists the account’s datasets, newest first, with the blueprint and deployment each belongs to. The default --limit is 50 and the maximum is 100. When more datasets remain, the output prints the --offset for the next page.

dataset get

ast dataset get <dataset-name> [--json]

Shows the dataset’s item count and, for each evaluator, how many items have each value.

dataset items

ast dataset items <dataset-name> [--limit <n>] [--offset <n>] [--json]

Lists a page of items with their input, expected output, and evaluator values. The default --limit is 50 and the maximum is 100. --offset must be a multiple of --limit. Items whose values come from an earlier evaluation set are marked (outdated). Use --json for the full, untruncated values.

dataset add

ast dataset add <dataset-name> --trace-id <trace-id> [--json]

Adds the trace to the dataset as an item, with its current evaluator values. The trace must belong to the dataset’s deployment and have an input. Adding a trace that is already in the dataset fails.

dataset edit

ast dataset edit <dataset-name> --trace-id <trace-id> \
  --set <key>=<value> [--set <key>=<value> ...] \
  [--set-string <key>=<value> ...] [--json]

Replaces the item’s evaluator values. Evaluators you leave out are cleared. Get keys and accepted values from ast eval get.

• --set reads true and false as booleans, numbers as numbers, and anything else as text.
• --set-string always reads the value as text.

An item added under an earlier evaluation set cannot be edited. Remove it and add it again.

dataset remove

ast dataset remove <dataset-name> --trace-id <trace-id> [--confirm <trace-id>] [--json]

Removes the trace’s item and its evaluator values from the dataset. The trace and its evaluations are kept. The command asks for confirmation. Pass --confirm with the trace ID to skip the prompt.

dataset download

ast dataset download <dataset-name> [--output <path>]

Saves the dataset as a zip file containing one JSONL file of its items. The command writes <dataset-name>.zip to the current directory. --output (-o) sets a file or an existing directory, and --output - writes the zip to standard output.

Next steps

• ast eval: evaluate traces and save the reviews that fill a dataset
• ast agent: find the trace to add