/eval/

Add a known question

Answer type is inferred as you type.
More options

Golden answers 0

The shared library of known questions and answers. Pipelines pick which of these to run.
#QuestionGolden answerTypeRunsMean score

Settings

Assistants

The AI assistants to evaluate. Add an API key to include one in runs. Keys are stored on the server and never shown again.

Connectors

Connect data sources so the assistants can look answers up themselves. Only read-only tools are switched on.

Account

Everything here (assistant keys, connectors, golden answers, pipelines and runs) belongs to this account and is shared by its members.

Members

Everyone in the account can do everything. Invite people by the email address they sign in with; they join when they next sign in.
NameEmailStatus

API tokens

For the meltano-eval CLI and CI: set EVAL_TOKEN or pass --token. A token acts as you in this account. It's shown once, so copy it when you create it.
NameTokenCreatedLast used