promptquality.utils package

Submodules

promptquality.utils.api_client module

pydantic model ApiClient

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Fields:
field api_url: Annotated[Url] [Required]
Constraints:

  • max_length = 2083

  • allowed_schemes = [‘http’, ‘https’]

field token: SecretStr [Required]
static api_key_login(base_url, api_key)
Return type:

Dict[str, str]

create_job(job_request)
Return type:

Dict[str, str]

create_project(project_request)
Return type:

Dict[str, str]

create_run(run_request)
Return type:

Dict[str, str]

create_template(template_request)
Return type:

Dict[str, str]

create_template_version(template_version_request)
Return type:

Dict[str, str]

get_current_user()
Return type:

Dict[str, str]

get_estimated_cost(cost_request)
Return type:

Dict[str, float]

get_job_status(job_id)
Return type:

Dict[str, Any]

get_metrics(metrics_request)
Return type:

List[Dict[str, Any]]

get_project(project_id)
Return type:

Dict[str, str]

get_project_by_name(project_name)
Return type:

List[Dict[str, str]]

get_rows(request)
Return type:

List[Dict[str, Any]]

get_run_by_name(project_id, name)
Return type:

List[Dict[str, str]]

get_run_scorer_jobs(project_id, run_id)
Return type:

List[Dict[str, Any]]

get_template(project_id, template_id)
Return type:

Dict[str, str]

get_templates(project_id)
Return type:

List[Dict[str, str]]

static healthcheck(base_url)
Return type:

bool

ingest_chain_rows(ingest_request, project_id, run_id)
Return type:

Dict[str, Any]

list_registered_scorers()
Return type:

List[Dict[str, Any]]

put_integration(integration_request)
Return type:

Dict[str, Any]

put_template_version_selection(select_version)
Return type:

Dict[str, Any]

put_user_metrics(scores, project_id, run_id)
Return type:

Dict[str, Any]

register_scorer(scorer_name, scorer_file)
Return type:

Dict[str, Any]

upload_dataset(dataset_request)
Return type:

Dict[str, str]

static username_login(base_url, username, password)
Return type:

Dict[str, str]

property auth_header: Dict[str, str]
property base_url: str

promptquality.utils.config module

get_config_location()
Return type:

Path

promptquality.utils.dataset module

transpose_dict_to_rows(dataset)
Return type:

List[Dict[str, str]]

parse_dataset(dataset)

Set the dataset path and format.

If the dataset is a dictionary or list, it will be stored as a a temporary file and the path to the temporary file will be returned. If the dataset is a path or string, the path will be returned. If the dataset is not a dictionary, list, path, or string, a ValueError will be raised.

If the dataset is a dictionary or list and PyArrow is available, the dataset will be stored as a Feather file. Otherwise, the dataset will be stored as a CSV file.

Parameters:

dataset (DatasetType) – Dataset as provided by the user.

Returns:

Path to the dataset file and the dataset format.

Return type:

Tuple[Path, DatasetFormat]

Raises:

  • ValueError – If the dataset is not a dictionary, list, path, or string.

  • ValueError – If the dataset file is not a CSV or Feather file.

  • FileNotFoundError – If the dataset file does not exist.

promptquality.utils.dependencies module

is_dependency_available(name)
Return type:

bool

promptquality.utils.logger module

promptquality.utils.name module

ts_name(prefix)
Return type:

str

check_scorer_name(name)

Check if name contains only letters, numbers, space, - and _.

Return type:

str

promptquality.utils.request module

make_request(request_method, base_url, endpoint, body=None, data=None, files=None, params=None, headers=None, timeout=None)
Return type:

Any

class HttpHeaders(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

accept = 'accept'
content_type = 'Content-Type'
application_json = 'application/json'
static accept_json()
Return type:

Dict[str, str]

static content_type_json()
Return type:

Dict[str, str]

static json()
Return type:

Dict[str, str]

promptquality.utils.scorer module

bifurcate_scorers(scorers=None)
Return type:

Tuple[List[Scorers], List[CustomScorer], List[RegisteredScorer]]

Module contents