Aquarium Learning Python Client¶

class aquariumlearning.ClassMapEntry(*, name, class_id, color, train_name=None, train_id=None, category=None, category_id=None, has_instances=True, ignore_in_eval=False, train_color=None)¶

A description of how to interpret a classification.

In the common case, only three values are needed: - The string representation of the class - The int [0,255] representation of the class - What color to render it as.

In more complex cases, some classes may be ignored in evaluation, or collapsed together into a different class at inference time, or tracked as part of a larger category.

Parameters

name (str) – The string representation of the class
class_id (int) – The int representation of the class
color (Union[Tuple[int, int, int], List[int]]) – A length 3 list/tuple of RGB int values
train_name (Optional[int], optional) – The string representation of the class that this label will be inferred as. Defaults to None.
train_id (Optional[str], optional) – The int representation of the class that this label will be infererred as. Defaults to None.
category (Optional[str], optional) – The string representation of the parent category. Defaults to None.
category_id (Optional[int], optional) – The int representation of the parent category. Defaults to None.
has_instances (bool, optional) – Whether each label of this class is a separate instance. Defaults to True.
ignore_in_eval (bool, optional) – Whether to ignore this class while evaluating metrics. Defaults to False.
train_color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.

to_dict()¶

Returns a dictified form of this data structure.

Returns: dict represenation of Class Map Entry
Return type: Dict[str, Any]

class aquariumlearning.ClassMapUpdateEntry(*, name=None, class_id=None, color=None, train_color=None)¶

A description of a change to be made to a classification. Classification can be referred to by name or id

Parameters

name (Optional[str], optional) – The string representation of the class
class_id (Optional[int], optional) – The int representation of the class
color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values
train_color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.

to_dict()¶

Returns a dictified form of this data structure.

Returns: dict represenation of Class Map Update Entry
Return type: Dict[str, Any]

class aquariumlearning.Client(*, api_endpoint='https://illume.aquariumlearning.com/api/v1', **kwargs)¶

Client class that interacts with the Aquarium REST API.

Parameters: api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.

add_to_streaming_dataset(project_id, dataset_id, dataset=None, embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, update_type='SNAPSHOT', is_anon_mode=False, upload_version=None, validate_frames=False)¶

Append or update a streaming dataset with the provided data urls. Dataset must be have already been created using STREAMING pipeline_mode.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
upload_version (str, optional) – A name for a distinct set of frames that are new or updated within the dataset, for comparison down the line. Defaults to last-created upload version.
validate_frames (bool, optional) – Flag to tell the client to pre-check that all queued frames conform to a project’s primary_task params. Defaults to False
update_type (str) –

Return type

None

create_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)¶

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_dataset(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
pipeline_mode (str, optional) – BATCH or STREAMING
is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –

Return type

None

create_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH')¶

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_inferences(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING

Return type

None

create_or_update_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)¶

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
pipeline_mode (str, optional) – BATCH or STREAMING
is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –
update_type (str) –

Return type

None

create_or_update_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT')¶

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING
update_type (str) –

Return type

None

create_project(project_id, label_class_map, primary_task=None, secondary_labels=None, frame_links=None, label_links=None, default_camera_target=None, default_camera_position=None, custom_metrics=None, max_shown_categories=None, stratified_metrics=None, include_no_gt=None, metrics_confidence_threshold=None, metrics_iou_threshold=None, external_metadata=None)¶

Create a new project via the REST API.

Parameters

project_id (str) – project_id
label_class_map (LabelClassMap) – The label class map used to interpret classifications.
primary_task (Optional[str], optional) – Any specific primary task for a non-object detection or classification task. Can be ‘2D_SEMSEG’ or ‘CLASSIFICATION’ or ‘CLASSIFICATION_WITH_GEOMETRY’ or None.
secondary_labels ([type], optional) – List of secondary labels in classification tasks
frame_links (Optional[List[str]], optional) – List of string keys for links between frames
label_links (Optional[List[str]], optional) – List of string keys for links between labels
default_camera_target (Optional[List[float]], optional) – For 3D scenes, the default camera target
default_camera_position (Optional[List[float]], optional) – For 3D scenes, the default camera position
custom_metrics (Optional[ Union[CustomMetricsDefinition, List[CustomMetricsDefinition]] ], optional) – Defines which custom metrics exist for this project, defaults to None.
max_shown_categories (Optional[int], optional) – For categorical visualizations, set the maximum shown simultaneously. Max 100.
stratified_metrics (Optional[List[StratifiedMetricsDefinition]], optional) – Defines what object-level attributes to stratify metrics over.
metrics_confidence_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with IOU) to match your ground truth (GT) labels with your inference labels. Defaults to 0.1 if not specified.
metrics_iou_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with confidence) to match your ground truth (GT) labels with your inference labels. Defaults to 0.5 if not specified.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the project itself
include_no_gt (Optional[bool]) –

Return type

None

current_abstract_dataset_process_step_status(project_id, dataset_id)¶

Returns the process steps statuses for a given dataset or inferenceset

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

A set of process step statuses that exist for a given abstract dataset

Return type

Dict[str, Any]

current_dataset_process_state(project_id, dataset_id)¶

Current processing state of a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

current_inferences_process_state(project_id, dataset_id, inferences_id)¶

current processing state of inferences.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

dataset_exists(project_id, dataset_id)¶

Check if a dataset exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

Whether the dataset already exists.

Return type

bool

delete_dataset(project_id, dataset_id)¶

Mark a dataset for deletion

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Return type

None

delete_project(project_id)¶

Mark a project for deletion

Parameters: project_id (str) – project_id
Return type: None

get_dataset(project_id, dataset_id)¶

Get existing dataset for a project.

Parameters

project_id (str) – The project id.
dataset_id (str) – dataset_id

Returns

The dataset info.

Return type

dict

get_dataset_ingest_error_logs(project_id, dataset_id)¶

Get ingest error log entries for a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

List of error entries

Return type

list[dict]

get_datasets(project_id, include_archived)¶

Get existing datasets for a project.

Parameters

project_id (str) – The project id.
include_archived (Optional[bool]) –

Returns

A list of dataset info for the project.

Return type

list

get_frame(project_id, dataset_id, frame_id)¶

Get frame info (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_id (str) – The frame id.

Return type

Dict[str, Any]

get_frame_ids(project_id, dataset_id)¶

Get frame ids in a dataset (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.

Return type

List[str]

get_inferences_ingest_error_logs(project_id, dataset_id, inferences_id)¶

Get ingest error log entries for an inference set.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

List of error entries

Return type

list[dict]

get_issue_manager(project_id)¶

Get an issue manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The issue manager object.
Return type: IssueManager

get_metrics_manager(project_id)¶

Get a metrics manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The metrics manager object.
Return type: MetricsManager

get_project(project_id)¶

Get detailed info about a specific project

Returns: detailed info about a project
Return type: Dict[str, Any]
Parameters: project_id (str) –

get_projects()¶

Get info about existing projects

Returns: Project Info
Return type: list of dict

inferences_exists(project_id, dataset_id, inferences_id)¶

Check if a set of inferences exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

Whether the inferences id already exists.

Return type

bool

is_dataset_processed(project_id, dataset_id)¶

Check if a dataset is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

If the dataset is done processing.

Return type

bool

is_inferences_processed(project_id, dataset_id, inferences_id)¶

Check if a set of inferences is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

If the inferences set is done processing.

Return type

bool

preview_frame(project_id, labeled_frame, inference_frame=None)¶

prints out a URL that lets you preview a provided frame in the web browser Useful for debugging data and image url issues.

Parameters

project_id (str) – Name of project to be associated for this frame preview (for label class association)
labeled_frame (LabeledFrame) – Labeled Frame desired for preview in web-app
inference_frame (Optional[InferencesFrame], optional) – Labeled Inference Desired for preview in web-app. Defaults to None.

Return type

None

project_exists(project_id)¶

Checks whether a project exists.

Parameters: project_id (str) – project_id
Returns: Does project exist
Return type: bool

set_credentials(*, token=None, app_id=None, app_key=None, api_key=None)¶

Set credentials for the client.

Parameters

api_key (str, optional) – A string for a long lived API key. Defaults to None.
token (str, optional) – A JWT providing auth credentials. Defaults to None.
app_id (str, optional) – Application ID string. Defaults to None.
app_key (str, optional) – Application secret key. Defaults to None.

Raises

Exception – Invalid credential combination provided.

Return type

None

update_dataset_metadata(project_id, dataset_id, external_metadata)¶

Update dataset metadata

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_inferences_metadata(project_id, inferences_id, external_metadata)¶

Update dataset metadata

Parameters

project_id (str) – The project id.
inferences_id (str) – The inferences id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_label_class_map_colors(project_id, changed_label_classes)¶

Updates label class colors of a specific project

Parameters

project_id (str) – The project id.
changed_label_classes (List[ClassMapUpdateEntry]) – The list of label classes with changed colors. Must be a subset of the project’s overall label class map

Return type

None

update_project_metadata(project_id, external_metadata)¶

Update project metadata

Parameters

project_id (str) – The project id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

upload_asset_from_filepath(project_id, dataset_id, filepath)¶

Upload an asset from a local file path. This is useful in cases where you have data on your local machine that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
filepath (str) – The filepath to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

upload_asset_from_url(project_id, dataset_id, source_url)¶

Upload an asset from a private url. This is useful in cases where you have data easily accessible on your network that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
source_url (str) – The source url to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

class aquariumlearning.CollectionClient(*args, **kwargs)¶

Client class that interacts with the Aquarium REST API. Also handles extra work around collecting samples for collection campigns

Parameters: api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.

add_to_streaming_dataset(project_id, dataset_id, dataset=None, embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, update_type='SNAPSHOT', is_anon_mode=False, upload_version=None, validate_frames=False)¶

Append or update a streaming dataset with the provided data urls. Dataset must be have already been created using STREAMING pipeline_mode.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
upload_version (str, optional) – A name for a distinct set of frames that are new or updated within the dataset, for comparison down the line. Defaults to last-created upload version.
validate_frames (bool, optional) – Flag to tell the client to pre-check that all queued frames conform to a project’s primary_task params. Defaults to False
update_type (str) –

Return type

None

create_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)¶

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_dataset(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
pipeline_mode (str, optional) – BATCH or STREAMING
is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –

Return type

None

create_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH')¶

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_inferences(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING

Return type

None

create_or_update_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)¶

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
pipeline_mode (str, optional) – BATCH or STREAMING
is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –
update_type (str) –

Return type

None

create_or_update_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT')¶

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING
update_type (str) –

Return type

None

create_project(project_id, label_class_map, primary_task=None, secondary_labels=None, frame_links=None, label_links=None, default_camera_target=None, default_camera_position=None, custom_metrics=None, max_shown_categories=None, stratified_metrics=None, include_no_gt=None, metrics_confidence_threshold=None, metrics_iou_threshold=None, external_metadata=None)¶

Create a new project via the REST API.

Parameters

project_id (str) – project_id
label_class_map (LabelClassMap) – The label class map used to interpret classifications.
primary_task (Optional[str], optional) – Any specific primary task for a non-object detection or classification task. Can be ‘2D_SEMSEG’ or ‘CLASSIFICATION’ or ‘CLASSIFICATION_WITH_GEOMETRY’ or None.
secondary_labels ([type], optional) – List of secondary labels in classification tasks
frame_links (Optional[List[str]], optional) – List of string keys for links between frames
label_links (Optional[List[str]], optional) – List of string keys for links between labels
default_camera_target (Optional[List[float]], optional) – For 3D scenes, the default camera target
default_camera_position (Optional[List[float]], optional) – For 3D scenes, the default camera position
custom_metrics (Optional[ Union[CustomMetricsDefinition, List[CustomMetricsDefinition]] ], optional) – Defines which custom metrics exist for this project, defaults to None.
max_shown_categories (Optional[int], optional) – For categorical visualizations, set the maximum shown simultaneously. Max 100.
stratified_metrics (Optional[List[StratifiedMetricsDefinition]], optional) – Defines what object-level attributes to stratify metrics over.
metrics_confidence_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with IOU) to match your ground truth (GT) labels with your inference labels. Defaults to 0.1 if not specified.
metrics_iou_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with confidence) to match your ground truth (GT) labels with your inference labels. Defaults to 0.5 if not specified.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the project itself
include_no_gt (Optional[bool]) –

Return type

None

current_abstract_dataset_process_step_status(project_id, dataset_id)¶

Returns the process steps statuses for a given dataset or inferenceset

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

A set of process step statuses that exist for a given abstract dataset

Return type

Dict[str, Any]

current_dataset_process_state(project_id, dataset_id)¶

Current processing state of a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

current_inferences_process_state(project_id, dataset_id, inferences_id)¶

current processing state of inferences.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

dataset_exists(project_id, dataset_id)¶

Check if a dataset exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

Whether the dataset already exists.

Return type

bool

delete_dataset(project_id, dataset_id)¶

Mark a dataset for deletion

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Return type

None

delete_project(project_id)¶

Mark a project for deletion

Parameters: project_id (str) – project_id
Return type: None

get_dataset(project_id, dataset_id)¶

Get existing dataset for a project.

Parameters

project_id (str) – The project id.
dataset_id (str) – dataset_id

Returns

The dataset info.

Return type

dict

get_dataset_ingest_error_logs(project_id, dataset_id)¶

Get ingest error log entries for a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

List of error entries

Return type

list[dict]

get_datasets(project_id, include_archived)¶

Get existing datasets for a project.

Parameters

project_id (str) – The project id.
include_archived (Optional[bool]) –

Returns

A list of dataset info for the project.

Return type

list

get_frame(project_id, dataset_id, frame_id)¶

Get frame info (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_id (str) – The frame id.

Return type

Dict[str, Any]

get_frame_ids(project_id, dataset_id)¶

Get frame ids in a dataset (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.

Return type

List[str]

get_inferences_ingest_error_logs(project_id, dataset_id, inferences_id)¶

Get ingest error log entries for an inference set.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

List of error entries

Return type

list[dict]

get_issue_manager(project_id)¶

Get an issue manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The issue manager object.
Return type: IssueManager

get_metrics_manager(project_id)¶

Get a metrics manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The metrics manager object.
Return type: MetricsManager

get_project(project_id)¶

Get detailed info about a specific project

Returns: detailed info about a project
Return type: Dict[str, Any]
Parameters: project_id (str) –

get_projects()¶

Get info about existing projects

Returns: Project Info
Return type: list of dict

inferences_exists(project_id, dataset_id, inferences_id)¶

Check if a set of inferences exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

Whether the inferences id already exists.

Return type

bool

is_dataset_processed(project_id, dataset_id)¶

Check if a dataset is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

If the dataset is done processing.

Return type

bool

is_inferences_processed(project_id, dataset_id, inferences_id)¶

Check if a set of inferences is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

If the inferences set is done processing.

Return type

bool

preview_frame(project_id, labeled_frame, inference_frame=None)¶

prints out a URL that lets you preview a provided frame in the web browser Useful for debugging data and image url issues.

Parameters

project_id (str) – Name of project to be associated for this frame preview (for label class association)
labeled_frame (LabeledFrame) – Labeled Frame desired for preview in web-app
inference_frame (Optional[InferencesFrame], optional) – Labeled Inference Desired for preview in web-app. Defaults to None.

Return type

None

project_exists(project_id)¶

Checks whether a project exists.

Parameters: project_id (str) – project_id
Returns: Does project exist
Return type: bool

sample_probabilities(frames)¶

Takes a list of Labeled Frames and stores scores for each based on each synced collection campaigns

Parameters: frames (List[LabeledFrame]) – a List of Labeled frames to score based on synced Collection Campaigns
Return type: Any

save_for_collection(override_sampling_threshold=None, target_sample_count=None, dry_run=False)¶

Based on the sampling threshold, take all sampled frames and upload those that score above the sampling threshold for each Collection Campaign.

Parameters

override_sampling_threshold (float, optional) – Override sampling threshold for all campaigns to save to server. If this value is set it will not use the sampling threshold set in UI for each Collection Campaign.
target_sample_count (int, optional) – Instead of collecting all samples above a specified similarity threshold, you can specify this option to save the N most similar samples.
dry_run (bool) –

Return type

None

set_credentials(*, token=None, app_id=None, app_key=None, api_key=None)¶

Set credentials for the client.

Parameters

api_key (str, optional) – A string for a long lived API key. Defaults to None.
token (str, optional) – A JWT providing auth credentials. Defaults to None.
app_id (str, optional) – Application ID string. Defaults to None.
app_key (str, optional) – Application secret key. Defaults to None.

Raises

Exception – Invalid credential combination provided.

Return type

None

sync_state(target_project_names=[], target_issue_uuids=[])¶

Downloads all collection campaigns and preps sampler with sample frames

Parameters

target_project_names (Optional[List[str]], optional) – List of project names whose collection campaigns should be sampled for
target_issue_uuids (Optional[List[str]], optional) – List of issue uuids whose collection campaigns should be sampled for. This takes precedence over ‘target_project_names’ and is mutually exclusive.

Return type

None

update_dataset_metadata(project_id, dataset_id, external_metadata)¶

Update dataset metadata

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_inferences_metadata(project_id, inferences_id, external_metadata)¶

Update dataset metadata

Parameters

project_id (str) – The project id.
inferences_id (str) – The inferences id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_label_class_map_colors(project_id, changed_label_classes)¶

Updates label class colors of a specific project

Parameters

project_id (str) – The project id.
changed_label_classes (List[ClassMapUpdateEntry]) – The list of label classes with changed colors. Must be a subset of the project’s overall label class map

Return type

None

update_project_metadata(project_id, external_metadata)¶

Update project metadata

Parameters

project_id (str) – The project id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

upload_asset_from_filepath(project_id, dataset_id, filepath)¶

Upload an asset from a local file path. This is useful in cases where you have data on your local machine that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
filepath (str) – The filepath to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

upload_asset_from_url(project_id, dataset_id, source_url)¶

Upload an asset from a private url. This is useful in cases where you have data easily accessible on your network that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
source_url (str) – The source url to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

class aquariumlearning.CustomMetricsDefinition(name, metrics_type)¶

Definitions for custom user provided metrics.

Parameters

name (str) – The name of this metric.
metrics_type (str) – The metrics type, either ‘objective’ or ‘confusion_matrix’.

class aquariumlearning.Inferences¶

A container used to construct a set of inferences.

Typical usage is to create an Inferences object, add multiple InferencesFrames to it, then serialize the frames to be submitted.

add_frame(frame)¶

Add an InferencesFrame to this dataset.

Parameters: frame (InferencesFrame) – An InferencesFrame in this dataset.
Return type: None

write_embeddings_to_file(filelike)¶

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_label_assets_to_file(filelike)¶

Write the frame’s label assets to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_to_file(filelike)¶

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

class aquariumlearning.InferencesFrame(*, frame_id)¶

add_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, confidence=None, area=None, iscrowd=None, user_attrs=None, links=None)¶

Add a 2D bounding box.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
area (float, optional) – The area of the image.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_classification(*, sensor_id, label_id, classification, confidence=None, user_attrs=None, secondary_labels=None)¶

Add an inference for 2D classification.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
secondary_labels (Optional[Dict[str, Any]]) –

Return type

None

add_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, confidence=None, user_attrs=None)¶

Add an inference for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
keypoints (list of dicts) – The keypoints of this detection
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_line_segment(*, sensor_id, label_id, classification, x1, y1, x2, y2, confidence=None, iscrowd=None, user_attrs=None, links=None)¶

Add a 2D line segment.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_polygon_list(*, sensor_id, label_id, classification, polygons, confidence=None, center=None, user_attrs=None)¶

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_semseg(*, sensor_id, label_id, mask_url=None, mask_data=None)¶

Add an inference for 2D semseg. These should provide either a mask_url or a mask_data.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]

Return type

None

add_3d_classification(*, label_id, classification, confidence=None, coord_frame_id=None, user_attrs=None)¶

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
coord_frame_id (optional, str) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)¶

Add an inference for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float) – confidence of prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')¶

Add a per inference crop embedding

Parameters

label_id (str) – [description]
embedding (List[float]) – A vector of floats between length 0 and 12,000.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_custom_metric(name, value)¶

Add a custom metric for a given inference frame.

Parameters

name (str) – The name of the custom metric being added. Must match one of the custom_metrics already defined by the corresponding Project.
value (Union[float, List[Union[int, float]]]) – The value of the custom metric (either a float or 2d list of floats/integers).

Return type

None

add_frame_embedding(*, embedding, model_id='')¶

Add an embedding to this frame

Parameters

embedding (List[float]) – A vector of floats between length 0 and 12,000.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_inference_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, confidence, area=None, iscrowd=None, user_attrs=None)¶

Add an inference for a 2D bounding box.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
area (float, optional) – The area of the image.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_classification(*, sensor_id, label_id, classification, confidence, user_attrs=None)¶

Add an inference for 2D classification.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, confidence, user_attrs=None)¶

Add an inference for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
keypoints (list of dicts) – The keypoints of this detection
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_line_segment(*, sensor_id, label_id, classification, x1, y1, x2, y2, confidence, iscrowd=None, user_attrs=None, links=None)¶

Add an inference for a 2D line segment.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
confidence (float) –
links (Optional[Dict[str, Any]]) –

Return type

None

add_inference_2d_polygon_list(*, sensor_id, label_id, classification, polygons, confidence, center=None, user_attrs=None)¶

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_semseg(*, sensor_id, label_id, mask_url=None, mask_data=None)¶

Add an inference for 2D semseg.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]

Return type

None

add_inference_3d_classification(*, label_id, classification, confidence, coord_frame_id=None, user_attrs=None)¶

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
coord_frame_id (optional, str) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)¶

Add an inference for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float) – confidence of prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_inference_text_token(*, sensor_id, label_id, index, token, classification, visible, confidence, user_attrs=None)¶

Add a label for a text token.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float) – confidence of prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_text_token(*, sensor_id, label_id, index, token, classification, visible, confidence=None, user_attrs=None)¶

Add a label for a text token.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float) – confidence of prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

to_dict()¶

Convert this label set into a dictionary representation.

Returns: dictified frame
Return type: dict

class aquariumlearning.Issue(name, dataset, elements, element_type, created_at=None, updated_at=None, reporter=None, assignee=None, state=None, issue_id=None, inference_set=None)¶

Definition for issue.

Parameters

name (str) – The issue name.
dataset (Optional[str]) – The dataset for this issue.
elements (List[IssueElement]) – The elements of the issue.
element_type (str) – The element type of the issue (“frame”, “crop”).
created_at (str) – The time of issue creation.
updated_at (str) – The time of last issue update.
reporter (str) – Email of issue creator.
assignee (Optional[str], optional) – Email of the person assigned the issue. Defaults to None.
state (str) – Current state of issue (“triage”, “inProgress”, “inReview”, “resolved”, “cancelled”). Defaults to “triage”.
issue_id (str) – The issue id.
inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.

class aquariumlearning.IssueElement(element_id, frame_id, element_type, dataset, status=None, inference_set=None, frame_data=None, crop_data=None, label_metadata=None)¶

Definition for issue element.

Parameters

element_id (str) – The element id.
frame_id (str) – The frame id of the element.
element_type (str) – The element type of the issue element (“frame” or “crop”).
dataset (str) – The base dataset an element is from. (Can be formatted as either “project_name.dataset_name” or just “dataset_name”)
inference_set (str) – The inference set an element is from (if any). (Can be formatted as either “project_name.inference_set_name” or just “inference_set_name”)
status (str) – (For read purposes, not element modification). The status of the element.
frame_data – (For read purposes, not element modification). JSON object that is based on either a LabeledFrame or InferencesFrame
crop_data – (For read purposes, not element modification). JSON object for the specific “frame_data” crop that a “crop”-type element is based on.
label_metadata – (For read purposes, not element modification). JSON object with confidence and IOU info if the element was created from a ground truth/inference comparison.

class aquariumlearning.IssueManager(client, project_id)¶

An issue manager for interacting with issues within a given project.

Parameters

client (Client) – An Aquarium Learning Python Client object.
project_id (str) – The project id associated with this manager.

add_elements_to_issue(issue_id, elements)¶

Add elements to an issue.

Parameters

issue_id (str) – The issue id.
elements (List[IssueElement]) – The elements to add to the issue.

Return type

None

create_issue(name, dataset, elements, element_type, inference_set=None)¶

Create an issue.

Parameters

name (str) – The issue name.
dataset (str) – The dataset for this issue.
elements (List[IssueElement]) – The initial elements of the issue.
element_type (str) – The element type of the issue (“frame” or “crop”).
inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.

Returns

The created issue id.

Return type

str

delete_issue(issue_id)¶

Delete an issue.

Parameters: issue_id (str) – The issue id.
Return type: None

get_issue(issue_id, exclude_frame_data=False)¶

Get a specific issue in the associated project. This will also include all associated frame metadata associated with each element.

Parameters

issue_id (str) – The issue id.
exclude_frame_data (bool) – Set to True to exclude full frame data from the issue element (e.g. to cut down on download size).

Returns

The issue data (including frame_data, crop_data, and label_metadata).

Return type

Issue

list_issues()¶

List issues in the associated project.

NOTE: this does NOT include the frame_data or crop_data information for the issue elements. (Use get_issue instead to see that info).

Returns: List of all issues data.
Return type: List[Issue]

remove_elements_from_issue(issue_id, elements)¶

Remove elements from an issue.

Parameters

issue_id (str) – The issue id.
elements (List[IssueElement]) – The elements to remove from the issue.

Return type

None

update_elements_status(issue_id, element_ids, new_status)¶

Update issue elements status.

Parameters

issue_id (str) – The issue id.
new_status (str) – The new status. (“unstarted”, “done”)
element_ids (List[str]) –

Return type

None

update_issue_name(issue_id, issue_name)¶

Update issue name.

Parameters

issue_id (str) – The issue id.
issue_name (str) – The new issue name.

Return type

None

update_issue_state(issue_id, issue_state)¶

Update issue state.

Parameters

issue_id (str) – The issue id.
issue_state (str) – The new issue state. (“triage”, “inProgress”, “inReview”, “resolved”, “cancelled”)

Return type

None

class aquariumlearning.LabelClassMap(*, entries=None)¶

A collection of ClassMapEntries that defines how to interpret classifications.

Parameters: entries (list of ClassMapEntry, optional) – List of classification entries. Defaults to None.

add_entry(entry)¶

Add a ClassMapEntry.

Parameters: entry (ClassMapEntry) – entry
Return type: None

static from_classnames(classnames)¶

A helper utility to generate a set of distinct colors given a list of classnames based on the number of classes

Parameters: classnames (List[str]) – List of classifications as strings
Returns: A LabelClassMap containing categorical colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_max10(classnames)¶

A helper utility to generate a set of distinct colors given a list of classnames

Parameters: classnames (list of str) – List of up to ten classifications as strings
Returns: A LabelClassMap containing categorical colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_max20(classnames)¶

A helper utility to generate a set of distinct colors given a list of classnames

Parameters: classnames (List[str]) – List of up to twenty classifications as strings
Returns: A LabelClassMap containing categorical colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_turbo(classnames)¶

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the turbo color scheme, and will be assigned in order from dark to light.

Parameters: classnames (list of str) – List of classifications as strings.
Returns: A LabelClassMap containing colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_viridis(classnames)¶

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the viridis color scheme, and will be assigned in order from dark to light.

Parameters: classnames (list of str) – List of classifications as strings.
Returns: A LabelClassMap containing colors for the provided classnames.
Return type: LabelClassMap

class aquariumlearning.LabeledDataset(*, pipeline_mode='BATCH')¶

A container used to construct a labeled dataset.

Typical usage is to create a LabeledDataset, add multiple LabeledFrames to it, then serialize the frames to be submitted.

add_frame(frame)¶

Add a LabeledFrame to this dataset.

Parameters: frame (LabeledFrame) – A LabeledFrame in this dataset.
Return type: None

write_crop_embeddings_to_file_streaming(filelike)¶

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_embeddings_to_file(filelike)¶

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_frame_embeddings_to_file_streaming(filelike)¶

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_frames_to_file_streaming(filelike)¶

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_label_assets_to_file(filelike)¶

Write the frame’s label assets to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_labels_to_file_streaming(filelike)¶

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_to_file(filelike)¶

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

class aquariumlearning.LabeledFrame(*, frame_id, date_captured=None, device_id=None)¶

A labeled frame for a dataset.

Parameters

frame_id (str) – A unique id for this frame.
date_captured (str, optional) – ISO formatted datetime string. Defaults to None.
device_id (str, optional) – The device that generated this frame. Defaults to None.

add_audio(*, sensor_id, audio_url, date_captured=None)¶

Add an audio “sensor” data to this frame.

Parameters

sensor_id (str) – The id of this sensor.
audio_url (str) – The URL to load this audio data (mp3, etc.).
date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_coordinate_frame_3d(*, coord_frame_id, position=None, orientation=None, parent_frame_id=None)¶

Add a 3D Coordinate Frame.

Parameters

coord_frame_id (str) – String identifier for this coordinate frame
position (Optional[Dict[POSITION, Union[int, float]]], optional) – Dict of the form {x, y, z}. Defaults to None.
orientation (Optional[Dict[ORIENTATION, Union[int, float]]], optional) – Quaternion rotation dict of the form {w, x, y, z}. Defaults to None.
parent_frame_id (Optional[str], optional) – String id of the parent coordinate frame. Defaults to None.

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')¶

Add a per label crop embedding

Parameters

label_id (str) – [description]
embedding (List[float]) – A vector of floats between length 0 and 12,000.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_embedding(*, embedding, crop_embeddings=None, model_id='')¶

DEPRECATED! PLEASE USE add_frame_embedding and add_crop_embedding Add an embedding to this frame, and optionally to crops/labels within it.

If provided, “crop_embeddings” is a list of dicts of the form:: ‘uuid’: the label id for the crop/label ‘embedding’: a vector of floats between length 0 and 12,000.

Parameters

embedding (list of floats) – A vector of floats between length 0 and 12,000.
crop_embeddings (list of dicts, optional) – A list of dictionaries representing crop embeddings. Defaults to None.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_frame_embedding(*, embedding, model_id='')¶

Add an embedding to this frame

Parameters

embedding (List[float]) – A vector of floats between length 0 and 12,000.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_geo_latlong_data(lat, lon)¶

Add a user provided EPSG:4326 WGS84 lat long pair to each frame

We expect these values to be floats

Parameters

lat (float) – lattitude of Geo Location
lon (float) – longitude of Geo Location

Return type

None

add_image(*, sensor_id, image_url, preview_url=None, date_captured=None, width=None, height=None)¶

Add an image “sensor” data to this frame.

Parameters

sensor_id (str) – The id of this sensor.
image_url (str) – The URL to load this image data.
preview_url (Optional[str], optional) – A URL to a compressed version of the image. It must be the same pixel dimensions as the original image. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
width (Optional[int], optional) – The width of the image in pixels. Defaults to None.
height (Optional[int], optional) – The height of the image in pixels. Defaults to None.

Return type

None

add_image_overlay(*, sensor_id, overlay_id, image_url, date_captured=None, width=None, height=None)¶

Add an image overlay for the given “sensor” to this frame.

Parameters

sensor_id (str) – The id of this sensor.
overlay_id (str) – The id of this overlay.
image_url (str) – The URL to load this image data.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
width (Optional[int], optional) – The width of the image in pixels. Defaults to None.
height (Optional[int], optional) – The height of the image in pixels. Defaults to None.

Return type

None

add_label_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, iscrowd=None, user_attrs=None, links=None)¶

Add a label for a 2D bounding box.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.

Return type

None

add_label_2d_classification(*, sensor_id, label_id, classification, secondary_labels=None, user_attrs=None)¶

Add a label for 2D classification.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
secondary_labels (dict, optional) – dictionary of secondary labels
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, user_attrs=None)¶

Add a label for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
keypoints (list of dicts) – The keypoints of this detection
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_line_segment(*, sensor_id, label_id, classification, x1, y1, x2, y2, iscrowd=None, user_attrs=None, links=None)¶

Add a label for a 2D line segment.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_label_2d_polygon_list(*, sensor_id, label_id, classification, polygons, center=None, user_attrs=None)¶

Add a label for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_semseg(*, sensor_id, label_id, mask_url=None, mask_data=None)¶

Add a label for 2D semseg.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]

Return type

None

add_label_3d_classification(*, label_id, classification, coord_frame_id=None, user_attrs=None)¶

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
coord_frame_id (str, optional) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_3d_cuboid(*, label_id, classification, position, dimensions, rotation, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)¶

Add a label for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_label_text_token(*, sensor_id, label_id, index, token, classification, visible, user_attrs=None)¶

Add a label for a text token.

Parameters

sensor_id (str) – sensor_id
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_obj(*, sensor_id, obj_url, coord_frame_id=None, date_captured=None)¶

Add a .obj file to the frame for text based geometry

Parameters

sensor_id (str) – sensor id
obj_url (str) – URL to where the object is located
coord_frame_id (Optional[str], optional) – ID for the coordinate frame. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_point_cloud_bins(*, sensor_id, pointcloud_url=None, kitti_format_url=None, intensity_url=None, range_url=None, coord_frame_id=None, date_captured=None)¶

Add a point cloud sensor data point to this frame, contained in dense binary files of little-endian values, similar to the raw format of KITTI lidar data. You can provide a combination of the following values, as long as at least either kitti_format_url or pointcloud_url are provided.

Parameters

sensor_id (str) – Sensor id
pointcloud_url (Optional[str]) – URL for the pointcloud: float32 [x1, y1, z1, x2, y2, z2, …]
kitti_format_url (Optional[str]) – URL for the pointcloud + intensity: float32 [x1, y1, z1, i1, x2, y2, z2, i2, …]
intensity_url (Optional[str]) – URL for the Intensity Pointcloud: unsigned int32 [i1, i2, …]
range_url (Optional[str]) – URL for the Range Pointcloud: float32 [r1, r2, …]
coord_frame_id (Optional[str], optional) – Id for the Coordinate Frame. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_point_cloud_pcd(*, sensor_id, pcd_url, coord_frame_id=None, date_captured=None)¶

Add a point cloud sensor data point to this frame, contained in PCD format. ascii, binary, and binary_compressed formats are supported. Numeric values for the following column names are expected: x, y, z, intensity (optional), range (optional)

Parameters

sensor_id (str) – sensor id
pcd_url (str) – URL to PCD formated data
coord_frame_id (Optional[str], optional) – The coordinate frame id. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_text(*, sensor_id, text, date_captured=None)¶

Add a text “sensor” data to this frame.

Parameters

sensor_id (str) – The id of this sensor.
text (str) – The text body.
date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_user_metadata(key, val, val_type=None)¶

Add a user provided metadata field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

key (str) – The key for your metadata field
val (Union[str, int, float, bool]) – value
val_type (Literal["str", "int", "float", "bool"], optional) – type of val as string. Defaults to None.

Return type

None

add_user_metadata_list(key, val, list_elt_type=None)¶

Add a user provided metadata list field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

key (str) – The key for your metadata field
val (Union[List[int], List[str], List[bool], List[float], Tuple[int], Tuple[str], Tuple[bool], Tuple[float]]) – value
list_elt_type (Literal["str", "int", "float", "bool"], optional) – type of list elements as string. Applies to all members of val. Defaults to None.

Return type

None

add_video(*, sensor_id, video_url, date_captured=None)¶

Add a video “sensor” data to this frame.

Parameters

sensor_id (str) – The id of this sensor.
video_url (str) – The URL to load this video data (mp4, webm, etc.).
date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

to_dict()¶

Convert this frame into a dictionary representation.

Returns: dictified frame
Return type: dict

validate_and_backfill_crop_embeddings()¶

Determines if there are frame embeddings but not Crop embeddings Backfills crop embeddings if Frame embeddings exist Returns True if backfill was necessary

Returns: returns if backfill on frame was necessary
Return type: bool

class aquariumlearning.MetricsManager(client, project_id)¶

A manager for interacting with metrics within a given project.

Contains the following constants:

MetricsManager.ORDER_CONF_DESC
MetricsManager.ORDER_CONF_ASC
MetricsManager.ORDER_IOU_DESC
MetricsManager.ORDER_IOU_ASC
MetricsManager.ORDER_BOX_SIZE_DESC
MetricsManager.ORDER_BOX_SIZE_ASC

Parameters

client (Client) – An Aquarium Learning Python Client object.
project_id (str) – The project id associated with this manager.

fetch_confusions(dataset_id, inferences_id, opts)¶

Fetch confusions for a given dataset + inference set pair.

The options define the parameters of the query. The options are a dictionary of the form:

{
    'confidence_threshold': Union[float, int],
    'iou_threshold': Union[float, int],
    'queries': List[QueryEntry],
    'ordering': QueryOrdering,
    'max_results': int (optional, defaults to 10,000),
    'cur_offset': int (optional, defaults to 0),
}

Confidence and iou thresholds can be any multiple of 0.1 between 0.0 and 1.0. Queries are a list of queries produced from helper methods such as

metrics_manager.make_confusions_query()

Ordering is defined by constants on the class object, such as:

metrics_manager.ORDER_CONF_DESC

Parameters

dataset_id (str) – The dataset id.
inferences_id (str) – The inference set id.
opts (ConfusionsOpts) – Options for the query.

Returns

All of your query results, up to max_results (default = 10k)

Return type

ConfusionsResultDict

make_cell_query(gt, inf)¶

Make a query entry for a specific confusion of gt as as inf.

Parameters

gt (str) – The classname of the ground truth class.
inf (str) – The classname of the inference class.

Returns

Result query entry.

Return type

QueryEntry

make_confused_as_query(name)¶

Make a query entry for all confusions as name. (inf == name, gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns: Result query entry.
Return type: QueryEntry
Parameters: name (str) –

make_confused_from_query(name)¶

Make a query entry for all confusions from name. (gt == name, gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns: Result query entry.
Return type: QueryEntry
Parameters: name (str) –

make_confusions_query()¶

Make a query entry for all confusions (gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns: Result query entry.
Return type: QueryEntry

make_correct_query()¶

Make a query entry for all correct detections/classifications (gt == inf)

Returns: Result query entry.
Return type: QueryEntry

make_false_negatives_query()¶

Make a query entry for all false negative detections.

These are cases without corresponding inference detections for a ground truth detection.

Returns: Result query entry.
Return type: QueryEntry

make_false_positives_query()¶

Make a query entry for all false positive detections.

These are cases without corresponding ground truth detections for an inference detection.

Returns: Result query entry.
Return type: QueryEntry

class aquariumlearning.StratifiedMetricsDefinition(name, ordered_values)¶

Definitions for stratified metrics given object-level attributes

Parameters

name (str) – The name of this attribute, which should match the attribute on object labels.
ordered_values (List[str]) – The ordered list of valid values to group by.