Aquarium Learning Python Client

class aquariumlearning.ClassMapEntry(*, name, class_id, color, train_name=None, train_id=None, category=None, category_id=None, has_instances=True, ignore_in_eval=False, train_color=None)

A description of how to interpret a classification.

In the common case, only three values are needed: - The string representation of the class - The int [0,255] representation of the class - What color to render it as.

In more complex cases, some classes may be ignored in evaluation, or collapsed together into a different class at inference time, or tracked as part of a larger category.

Parameters
  • name (str) – The string representation of the class

  • class_id (int) – The int representation of the class

  • color (Union[Tuple[int, int, int], List[int]]) – A length 3 list/tuple of RGB int values

  • train_name (Optional[int], optional) – The string representation of the class that this label will be inferred as. Defaults to None.

  • train_id (Optional[str], optional) – The int representation of the class that this label will be infererred as. Defaults to None.

  • category (Optional[str], optional) – The string representation of the parent category. Defaults to None.

  • category_id (Optional[int], optional) – The int representation of the parent category. Defaults to None.

  • has_instances (bool, optional) – Whether each label of this class is a separate instance. Defaults to True.

  • ignore_in_eval (bool, optional) – Whether to ignore this class while evaluating metrics. Defaults to False.

  • train_color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.

to_dict()

Returns a dictified form of this data structure.

Returns

dict represenation of Class Map Entry

Return type

Dict[str, Any]

class aquariumlearning.ClassMapUpdateEntry(*, name=None, class_id=None, color=None, train_color=None)

A description of a change to be made to a classification. Classification can be referred to by name or id

Parameters
  • name (Optional[str], optional) – The string representation of the class

  • class_id (Optional[int], optional) – The int representation of the class

  • color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values

  • train_color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.

to_dict()

Returns a dictified form of this data structure.

Returns

dict represenation of Class Map Update Entry

Return type

Dict[str, Any]

class aquariumlearning.Client(*, api_endpoint='https://illume.aquariumlearning.com/api/v1', **kwargs)

Client class that interacts with the Aquarium REST API.

Parameters

api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.

add_to_streaming_dataset(project_id, dataset_id, dataset=None, embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, update_type='SNAPSHOT', is_anon_mode=False, upload_version=None, validate_frames=False)

Append or update a streaming dataset with the provided data urls. Dataset must be have already been created using STREAMING pipeline_mode.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself

  • is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.

  • upload_version (str, optional) – A name for a distinct set of frames that are new or updated within the dataset, for comparison down the line. Defaults to last-created upload version.

  • validate_frames (bool, optional) – Flag to tell the client to pre-check that all queued frames conform to a project’s primary_task params. Defaults to False

  • update_type (str) –

Return type

None

create_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_dataset(…))

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • data_url (Optional[str], optional) – A URL to the serialized dataset entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.

  • dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

  • is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.

  • seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.

  • is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.

  • existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.

  • check_first_frame (bool) –

Return type

None

create_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_inferences(…))

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – A unique identifier for this set of inferences.

  • data_url (Optional[str], optional) – A URL to the serialized inference entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.

  • inferences (Optional[Inferences], optional) – The inferences to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

Return type

None

create_or_update_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • data_url (Optional[str], optional) – A URL to the serialized dataset entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.

  • dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

  • is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.

  • seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.

  • is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.

  • existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.

  • check_first_frame (bool) –

  • update_type (str) –

Return type

None

create_or_update_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – A unique identifier for this set of inferences.

  • data_url (Optional[str], optional) – A URL to the serialized inference entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.

  • inferences (Optional[Inferences], optional) – The inferences to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

  • update_type (str) –

Return type

None

create_project(project_id, label_class_map, primary_task=None, secondary_labels=None, frame_links=None, label_links=None, default_camera_target=None, default_camera_position=None, custom_metrics=None, max_shown_categories=None, stratified_metrics=None, include_no_gt=None, metrics_confidence_threshold=None, metrics_iou_threshold=None, external_metadata=None)

Create a new project via the REST API.

Parameters
  • project_id (str) – project_id

  • label_class_map (LabelClassMap) – The label class map used to interpret classifications.

  • primary_task (Optional[str], optional) – Any specific primary task for a non-object detection or classification task. Can be ‘2D_SEMSEG’ or ‘CLASSIFICATION’ or ‘CLASSIFICATION_WITH_GEOMETRY’ or None.

  • secondary_labels ([type], optional) – List of secondary labels in classification tasks

  • frame_links (Optional[List[str]], optional) – List of string keys for links between frames

  • label_links (Optional[List[str]], optional) – List of string keys for links between labels

  • default_camera_target (Optional[List[float]], optional) – For 3D scenes, the default camera target

  • default_camera_position (Optional[List[float]], optional) – For 3D scenes, the default camera position

  • custom_metrics (Optional[ Union[CustomMetricsDefinition, List[CustomMetricsDefinition]] ], optional) – Defines which custom metrics exist for this project, defaults to None.

  • max_shown_categories (Optional[int], optional) – For categorical visualizations, set the maximum shown simultaneously. Max 100.

  • stratified_metrics (Optional[List[StratifiedMetricsDefinition]], optional) – Defines what object-level attributes to stratify metrics over.

  • metrics_confidence_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with IOU) to match your ground truth (GT) labels with your inference labels. Defaults to 0.1 if not specified.

  • metrics_iou_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with confidence) to match your ground truth (GT) labels with your inference labels. Defaults to 0.5 if not specified.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the project itself

  • include_no_gt (Optional[bool]) –

Return type

None

current_abstract_dataset_process_step_status(project_id, dataset_id)

Returns the process steps statuses for a given dataset or inferenceset

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

A set of process step statuses that exist for a given abstract dataset

Return type

Dict[str, Any]

current_dataset_process_state(project_id, dataset_id)

Current processing state of a dataset.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

current_inferences_process_state(project_id, dataset_id, inferences_id)

current processing state of inferences.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

dataset_exists(project_id, dataset_id)

Check if a dataset exists.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

Whether the dataset already exists.

Return type

bool

delete_dataset(project_id, dataset_id)

Mark a dataset for deletion

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Return type

None

delete_project(project_id)

Mark a project for deletion

Parameters

project_id (str) – project_id

Return type

None

get_dataset(project_id, dataset_id)

Get existing dataset for a project.

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – dataset_id

Returns

The dataset info.

Return type

dict

get_dataset_ingest_error_logs(project_id, dataset_id)

Get ingest error log entries for a dataset.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

List of error entries

Return type

list[dict]

get_datasets(project_id, include_archived)

Get existing datasets for a project.

Parameters
  • project_id (str) – The project id.

  • include_archived (Optional[bool]) –

Returns

A list of dataset info for the project.

Return type

list

get_frame(project_id, dataset_id, frame_id)

Get frame info (for datasets only, not inferences)

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – The dataset id.

  • frame_id (str) – The frame id.

Return type

Dict[str, Any]

get_frame_ids(project_id, dataset_id)

Get frame ids in a dataset (for datasets only, not inferences)

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – The dataset id.

Return type

List[str]

get_inferences_ingest_error_logs(project_id, dataset_id, inferences_id)

Get ingest error log entries for an inference set.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

List of error entries

Return type

list[dict]

get_issue_manager(project_id)

Get an issue manager object.

Parameters

project_id (str) – Project ID to manage.

Returns

The issue manager object.

Return type

IssueManager

get_metrics_manager(project_id)

Get a metrics manager object.

Parameters

project_id (str) – Project ID to manage.

Returns

The metrics manager object.

Return type

MetricsManager

get_project(project_id)

Get detailed info about a specific project

Returns

detailed info about a project

Return type

Dict[str, Any]

Parameters

project_id (str) –

get_projects()

Get info about existing projects

Returns

Project Info

Return type

list of dict

inferences_exists(project_id, dataset_id, inferences_id)

Check if a set of inferences exists.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

Whether the inferences id already exists.

Return type

bool

is_dataset_processed(project_id, dataset_id)

Check if a dataset is fully processed.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

If the dataset is done processing.

Return type

bool

is_inferences_processed(project_id, dataset_id, inferences_id)

Check if a set of inferences is fully processed.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

If the inferences set is done processing.

Return type

bool

preview_frame(project_id, labeled_frame, inference_frame=None)

prints out a URL that lets you preview a provided frame in the web browser Useful for debugging data and image url issues.

Parameters
  • project_id (str) – Name of project to be associated for this frame preview (for label class association)

  • labeled_frame (LabeledFrame) – Labeled Frame desired for preview in web-app

  • inference_frame (Optional[InferencesFrame], optional) – Labeled Inference Desired for preview in web-app. Defaults to None.

Return type

None

project_exists(project_id)

Checks whether a project exists.

Parameters

project_id (str) – project_id

Returns

Does project exist

Return type

bool

set_credentials(*, token=None, app_id=None, app_key=None, api_key=None)

Set credentials for the client.

Parameters
  • api_key (str, optional) – A string for a long lived API key. Defaults to None.

  • token (str, optional) – A JWT providing auth credentials. Defaults to None.

  • app_id (str, optional) – Application ID string. Defaults to None.

  • app_key (str, optional) – Application secret key. Defaults to None.

Raises

Exception – Invalid credential combination provided.

Return type

None

update_dataset_metadata(project_id, dataset_id, external_metadata)

Update dataset metadata

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – The dataset id.

  • external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_inferences_metadata(project_id, inferences_id, external_metadata)

Update dataset metadata

Parameters
  • project_id (str) – The project id.

  • inferences_id (str) – The inferences id.

  • external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_label_class_map_colors(project_id, changed_label_classes)

Updates label class colors of a specific project

Parameters
  • project_id (str) – The project id.

  • changed_label_classes (List[ClassMapUpdateEntry]) – The list of label classes with changed colors. Must be a subset of the project’s overall label class map

Return type

None

update_project_metadata(project_id, external_metadata)

Update project metadata

Parameters
  • project_id (str) – The project id.

  • external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

upload_asset_from_filepath(project_id, dataset_id, filepath)

Upload an asset from a local file path. This is useful in cases where you have data on your local machine that you want to mirror in aquarium.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • filepath (str) – The filepath to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

upload_asset_from_url(project_id, dataset_id, source_url)

Upload an asset from a private url. This is useful in cases where you have data easily accessible on your network that you want to mirror in aquarium.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • source_url (str) – The source url to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

class aquariumlearning.CollectionClient(*args, **kwargs)

Client class that interacts with the Aquarium REST API. Also handles extra work around collecting samples for collection campigns

Parameters

api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.

add_to_streaming_dataset(project_id, dataset_id, dataset=None, embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, update_type='SNAPSHOT', is_anon_mode=False, upload_version=None, validate_frames=False)

Append or update a streaming dataset with the provided data urls. Dataset must be have already been created using STREAMING pipeline_mode.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself

  • is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.

  • upload_version (str, optional) – A name for a distinct set of frames that are new or updated within the dataset, for comparison down the line. Defaults to last-created upload version.

  • validate_frames (bool, optional) – Flag to tell the client to pre-check that all queued frames conform to a project’s primary_task params. Defaults to False

  • update_type (str) –

Return type

None

create_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_dataset(…))

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • data_url (Optional[str], optional) – A URL to the serialized dataset entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.

  • dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

  • is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.

  • seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.

  • is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.

  • existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.

  • check_first_frame (bool) –

Return type

None

create_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_inferences(…))

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – A unique identifier for this set of inferences.

  • data_url (Optional[str], optional) – A URL to the serialized inference entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.

  • inferences (Optional[Inferences], optional) – The inferences to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

Return type

None

create_or_update_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • data_url (Optional[str], optional) – A URL to the serialized dataset entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.

  • dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

  • is_unlabeled_indexed_dataset (bool, optional) – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.

  • seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.

  • is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.

  • existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.

  • check_first_frame (bool) –

  • update_type (str) –

Return type

None

create_or_update_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='BATCH', update_type='SNAPSHOT')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – A unique identifier for this set of inferences.

  • data_url (Optional[str], optional) – A URL to the serialized inference entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.

  • inferences (Optional[Inferences], optional) – The inferences to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself

  • pipeline_mode (str, optional) – BATCH or STREAMING

  • update_type (str) –

Return type

None

create_project(project_id, label_class_map, primary_task=None, secondary_labels=None, frame_links=None, label_links=None, default_camera_target=None, default_camera_position=None, custom_metrics=None, max_shown_categories=None, stratified_metrics=None, include_no_gt=None, metrics_confidence_threshold=None, metrics_iou_threshold=None, external_metadata=None)

Create a new project via the REST API.

Parameters
  • project_id (str) – project_id

  • label_class_map (LabelClassMap) – The label class map used to interpret classifications.

  • primary_task (Optional[str], optional) – Any specific primary task for a non-object detection or classification task. Can be ‘2D_SEMSEG’ or ‘CLASSIFICATION’ or ‘CLASSIFICATION_WITH_GEOMETRY’ or None.

  • secondary_labels ([type], optional) – List of secondary labels in classification tasks

  • frame_links (Optional[List[str]], optional) – List of string keys for links between frames

  • label_links (Optional[List[str]], optional) – List of string keys for links between labels

  • default_camera_target (Optional[List[float]], optional) – For 3D scenes, the default camera target

  • default_camera_position (Optional[List[float]], optional) – For 3D scenes, the default camera position

  • custom_metrics (Optional[ Union[CustomMetricsDefinition, List[CustomMetricsDefinition]] ], optional) – Defines which custom metrics exist for this project, defaults to None.

  • max_shown_categories (Optional[int], optional) – For categorical visualizations, set the maximum shown simultaneously. Max 100.

  • stratified_metrics (Optional[List[StratifiedMetricsDefinition]], optional) – Defines what object-level attributes to stratify metrics over.

  • metrics_confidence_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with IOU) to match your ground truth (GT) labels with your inference labels. Defaults to 0.1 if not specified.

  • metrics_iou_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with confidence) to match your ground truth (GT) labels with your inference labels. Defaults to 0.5 if not specified.

  • external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the project itself

  • include_no_gt (Optional[bool]) –

Return type

None

current_abstract_dataset_process_step_status(project_id, dataset_id)

Returns the process steps statuses for a given dataset or inferenceset

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

A set of process step statuses that exist for a given abstract dataset

Return type

Dict[str, Any]

current_dataset_process_state(project_id, dataset_id)

Current processing state of a dataset.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

current_inferences_process_state(project_id, dataset_id, inferences_id)

current processing state of inferences.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

dataset_exists(project_id, dataset_id)

Check if a dataset exists.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

Whether the dataset already exists.

Return type

bool

delete_dataset(project_id, dataset_id)

Mark a dataset for deletion

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Return type

None

delete_project(project_id)

Mark a project for deletion

Parameters

project_id (str) – project_id

Return type

None

get_dataset(project_id, dataset_id)

Get existing dataset for a project.

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – dataset_id

Returns

The dataset info.

Return type

dict

get_dataset_ingest_error_logs(project_id, dataset_id)

Get ingest error log entries for a dataset.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

List of error entries

Return type

list[dict]

get_datasets(project_id, include_archived)

Get existing datasets for a project.

Parameters
  • project_id (str) – The project id.

  • include_archived (Optional[bool]) –

Returns

A list of dataset info for the project.

Return type

list

get_frame(project_id, dataset_id, frame_id)

Get frame info (for datasets only, not inferences)

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – The dataset id.

  • frame_id (str) – The frame id.

Return type

Dict[str, Any]

get_frame_ids(project_id, dataset_id)

Get frame ids in a dataset (for datasets only, not inferences)

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – The dataset id.

Return type

List[str]

get_inferences_ingest_error_logs(project_id, dataset_id, inferences_id)

Get ingest error log entries for an inference set.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

List of error entries

Return type

list[dict]

get_issue_manager(project_id)

Get an issue manager object.

Parameters

project_id (str) – Project ID to manage.

Returns

The issue manager object.

Return type

IssueManager

get_metrics_manager(project_id)

Get a metrics manager object.

Parameters

project_id (str) – Project ID to manage.

Returns

The metrics manager object.

Return type

MetricsManager

get_project(project_id)

Get detailed info about a specific project

Returns

detailed info about a project

Return type

Dict[str, Any]

Parameters

project_id (str) –

get_projects()

Get info about existing projects

Returns

Project Info

Return type

list of dict

inferences_exists(project_id, dataset_id, inferences_id)

Check if a set of inferences exists.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

Whether the inferences id already exists.

Return type

bool

is_dataset_processed(project_id, dataset_id)

Check if a dataset is fully processed.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

If the dataset is done processing.

Return type

bool

is_inferences_processed(project_id, dataset_id, inferences_id)

Check if a set of inferences is fully processed.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

If the inferences set is done processing.

Return type

bool

preview_frame(project_id, labeled_frame, inference_frame=None)

prints out a URL that lets you preview a provided frame in the web browser Useful for debugging data and image url issues.

Parameters
  • project_id (str) – Name of project to be associated for this frame preview (for label class association)

  • labeled_frame (LabeledFrame) – Labeled Frame desired for preview in web-app

  • inference_frame (Optional[InferencesFrame], optional) – Labeled Inference Desired for preview in web-app. Defaults to None.

Return type

None

project_exists(project_id)

Checks whether a project exists.

Parameters

project_id (str) – project_id

Returns

Does project exist

Return type

bool

sample_probabilities(frames)

Takes a list of Labeled Frames and stores scores for each based on each synced collection campaigns

Parameters

frames (List[LabeledFrame]) – a List of Labeled frames to score based on synced Collection Campaigns

Return type

Any

save_for_collection(override_sampling_threshold=None, target_sample_count=None, dry_run=False)

Based on the sampling threshold, take all sampled frames and upload those that score above the sampling threshold for each Collection Campaign.

Parameters
  • override_sampling_threshold (float, optional) – Override sampling threshold for all campaigns to save to server. If this value is set it will not use the sampling threshold set in UI for each Collection Campaign.

  • target_sample_count (int, optional) – Instead of collecting all samples above a specified similarity threshold, you can specify this option to save the N most similar samples.

  • dry_run (bool) –

Return type

None

set_credentials(*, token=None, app_id=None, app_key=None, api_key=None)

Set credentials for the client.

Parameters
  • api_key (str, optional) – A string for a long lived API key. Defaults to None.

  • token (str, optional) – A JWT providing auth credentials. Defaults to None.

  • app_id (str, optional) – Application ID string. Defaults to None.

  • app_key (str, optional) – Application secret key. Defaults to None.

Raises

Exception – Invalid credential combination provided.

Return type

None

sync_state(target_project_names=[], target_issue_uuids=[])

Downloads all collection campaigns and preps sampler with sample frames

Parameters
  • target_project_names (Optional[List[str]], optional) – List of project names whose collection campaigns should be sampled for

  • target_issue_uuids (Optional[List[str]], optional) – List of issue uuids whose collection campaigns should be sampled for. This takes precedence over ‘target_project_names’ and is mutually exclusive.

Return type

None

update_dataset_metadata(project_id, dataset_id, external_metadata)

Update dataset metadata

Parameters
  • project_id (str) – The project id.

  • dataset_id (str) – The dataset id.

  • external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_inferences_metadata(project_id, inferences_id, external_metadata)

Update dataset metadata

Parameters
  • project_id (str) – The project id.

  • inferences_id (str) – The inferences id.

  • external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_label_class_map_colors(project_id, changed_label_classes)

Updates label class colors of a specific project

Parameters
  • project_id (str) – The project id.

  • changed_label_classes (List[ClassMapUpdateEntry]) – The list of label classes with changed colors. Must be a subset of the project’s overall label class map

Return type

None

update_project_metadata(project_id, external_metadata)

Update project metadata

Parameters
  • project_id (str) – The project id.

  • external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

upload_asset_from_filepath(project_id, dataset_id, filepath)

Upload an asset from a local file path. This is useful in cases where you have data on your local machine that you want to mirror in aquarium.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • filepath (str) – The filepath to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

upload_asset_from_url(project_id, dataset_id, source_url)

Upload an asset from a private url. This is useful in cases where you have data easily accessible on your network that you want to mirror in aquarium.

Parameters
  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • source_url (str) – The source url to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

class aquariumlearning.CustomMetricsDefinition(name, metrics_type)

Definitions for custom user provided metrics.

Parameters
  • name (str) – The name of this metric.

  • metrics_type (str) – The metrics type, either ‘objective’ or ‘confusion_matrix’.

class aquariumlearning.Inferences

A container used to construct a set of inferences.

Typical usage is to create an Inferences object, add multiple InferencesFrames to it, then serialize the frames to be submitted.

add_frame(frame)

Add an InferencesFrame to this dataset.

Parameters

frame (InferencesFrame) – An InferencesFrame in this dataset.

Return type

None

write_embeddings_to_file(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_label_assets_to_file(filelike)

Write the frame’s label assets to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_to_file(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

class aquariumlearning.InferencesFrame(*, frame_id)
add_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, confidence=None, area=None, iscrowd=None, user_attrs=None, links=None)

Add a 2D bounding box.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • area (float, optional) – The area of the image.

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_classification(*, sensor_id, label_id, classification, confidence=None, user_attrs=None, secondary_labels=None)

Add an inference for 2D classification.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • secondary_labels (Optional[Dict[str, Any]]) –

Return type

None

add_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, confidence=None, user_attrs=None)

Add an inference for a 2D keypoints task.

A keypoint is a dictionary of the form:

‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • keypoints (list of dicts) – The keypoints of this detection

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_line_segment(*, sensor_id, label_id, classification, x1, y1, x2, y2, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a 2D line segment.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • x1 (int or float) – The x-coord of the first vertex in pixels

  • y1 (int or float) – The x-coord of the first vertex in pixels

  • x2 (int or float) – The x-coord of the first vertex in pixels

  • y2 (int or float) – The x-coord of the first vertex in pixels

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_polygon_list(*, sensor_id, label_id, classification, polygons, confidence=None, center=None, user_attrs=None)

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:
‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}
Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • polygons (list of dicts) – The polygon geometry

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • center (list of ints or floats, optional) – The center point of the instance

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_semseg(*, sensor_id, label_id, mask_url=None, mask_data=None)

Add an inference for 2D semseg. These should provide either a mask_url or a mask_data.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • mask_url (str, optional) – URL to the pixel mask png.

  • mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]

Return type

None

add_3d_classification(*, label_id, classification, confidence=None, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters
  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • coord_frame_id (optional, str) – The coordinate frame id.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add an inference for a 3D cuboid.

Parameters
  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • position (list of float) – the position of the center of the cuboid

  • dimensions (list of float) – the dimensions of the cuboid

  • rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.

  • confidence (float) – confidence of prediction

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (dict, optional) – Links between labels. Defaults to None.

  • coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per inference crop embedding

Parameters
  • label_id (str) – [description]

  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_custom_metric(name, value)

Add a custom metric for a given inference frame.

Parameters
  • name (str) – The name of the custom metric being added. Must match one of the custom_metrics already defined by the corresponding Project.

  • value (Union[float, List[Union[int, float]]]) – The value of the custom metric (either a float or 2d list of floats/integers).

Return type

None

add_frame_embedding(*, embedding, model_id='')

Add an embedding to this frame

Parameters
  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_inference_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, confidence, area=None, iscrowd=None, user_attrs=None)

Add an inference for a 2D bounding box.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • area (float, optional) – The area of the image.

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_classification(*, sensor_id, label_id, classification, confidence, user_attrs=None)

Add an inference for 2D classification.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, confidence, user_attrs=None)

Add an inference for a 2D keypoints task.

A keypoint is a dictionary of the form:

‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • keypoints (list of dicts) – The keypoints of this detection

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_line_segment(*, sensor_id, label_id, classification, x1, y1, x2, y2, confidence, iscrowd=None, user_attrs=None, links=None)

Add an inference for a 2D line segment.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • x1 (int or float) – The x-coord of the first vertex in pixels

  • y1 (int or float) – The x-coord of the first vertex in pixels

  • x2 (int or float) – The x-coord of the first vertex in pixels

  • y2 (int or float) – The x-coord of the first vertex in pixels

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • confidence (float) –

  • links (Optional[Dict[str, Any]]) –

Return type

None

add_inference_2d_polygon_list(*, sensor_id, label_id, classification, polygons, confidence, center=None, user_attrs=None)

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:
‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}
Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • polygons (list of dicts) – The polygon geometry

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • center (list of ints or floats, optional) – The center point of the instance

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_semseg(*, sensor_id, label_id, mask_url=None, mask_data=None)

Add an inference for 2D semseg.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • mask_url (str, optional) – URL to the pixel mask png.

  • mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]

Return type

None

add_inference_3d_classification(*, label_id, classification, confidence, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters
  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • coord_frame_id (optional, str) – The coordinate frame id.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add an inference for a 3D cuboid.

Parameters
  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • position (list of float) – the position of the center of the cuboid

  • dimensions (list of float) – the dimensions of the cuboid

  • rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.

  • confidence (float) – confidence of prediction

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (dict, optional) – Links between labels. Defaults to None.

  • coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_inference_text_token(*, sensor_id, label_id, index, token, classification, visible, confidence, user_attrs=None)

Add a label for a text token.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • index (int) – the index of this token in the text

  • token (str) – the text content of this token

  • classification (str) – the classification string

  • visible (bool) – is this a visible token in the text

  • confidence (float) – confidence of prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_text_token(*, sensor_id, label_id, index, token, classification, visible, confidence=None, user_attrs=None)

Add a label for a text token.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • index (int) – the index of this token in the text

  • token (str) – the text content of this token

  • classification (str) – the classification string

  • visible (bool) – is this a visible token in the text

  • confidence (float) – confidence of prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

to_dict()

Convert this label set into a dictionary representation.

Returns

dictified frame

Return type

dict

class aquariumlearning.Issue(name, dataset, elements, element_type, created_at=None, updated_at=None, reporter=None, assignee=None, state=None, issue_id=None, inference_set=None)

Definition for issue.

Parameters
  • name (str) – The issue name.

  • dataset (Optional[str]) – The dataset for this issue.

  • elements (List[IssueElement]) – The elements of the issue.

  • element_type (str) – The element type of the issue (“frame”, “crop”).

  • created_at (str) – The time of issue creation.

  • updated_at (str) – The time of last issue update.

  • reporter (str) – Email of issue creator.

  • assignee (Optional[str], optional) – Email of the person assigned the issue. Defaults to None.

  • state (str) – Current state of issue (“triage”, “inProgress”, “inReview”, “resolved”, “cancelled”). Defaults to “triage”.

  • issue_id (str) – The issue id.

  • inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.

class aquariumlearning.IssueElement(element_id, frame_id, element_type, dataset, status=None, inference_set=None, frame_data=None, crop_data=None, label_metadata=None)

Definition for issue element.

Parameters
  • element_id (str) – The element id.

  • frame_id (str) – The frame id of the element.

  • element_type (str) – The element type of the issue element (“frame” or “crop”).

  • dataset (str) – The base dataset an element is from. (Can be formatted as either “project_name.dataset_name” or just “dataset_name”)

  • inference_set (str) – The inference set an element is from (if any). (Can be formatted as either “project_name.inference_set_name” or just “inference_set_name”)

  • status (str) – (For read purposes, not element modification). The status of the element.

  • frame_data – (For read purposes, not element modification). JSON object that is based on either a LabeledFrame or InferencesFrame

  • crop_data – (For read purposes, not element modification). JSON object for the specific “frame_data” crop that a “crop”-type element is based on.

  • label_metadata – (For read purposes, not element modification). JSON object with confidence and IOU info if the element was created from a ground truth/inference comparison.

class aquariumlearning.IssueManager(client, project_id)

An issue manager for interacting with issues within a given project.

Parameters
  • client (Client) – An Aquarium Learning Python Client object.

  • project_id (str) – The project id associated with this manager.

add_elements_to_issue(issue_id, elements)

Add elements to an issue.

Parameters
  • issue_id (str) – The issue id.

  • elements (List[IssueElement]) – The elements to add to the issue.

Return type

None

create_issue(name, dataset, elements, element_type, inference_set=None)

Create an issue.

Parameters
  • name (str) – The issue name.

  • dataset (str) – The dataset for this issue.

  • elements (List[IssueElement]) – The initial elements of the issue.

  • element_type (str) – The element type of the issue (“frame” or “crop”).

  • inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.

Returns

The created issue id.

Return type

str

delete_issue(issue_id)

Delete an issue.

Parameters

issue_id (str) – The issue id.

Return type

None

get_issue(issue_id, exclude_frame_data=False)

Get a specific issue in the associated project. This will also include all associated frame metadata associated with each element.

Parameters
  • issue_id (str) – The issue id.

  • exclude_frame_data (bool) – Set to True to exclude full frame data from the issue element (e.g. to cut down on download size).

Returns

The issue data (including frame_data, crop_data, and label_metadata).

Return type

Issue

list_issues()

List issues in the associated project.

NOTE: this does NOT include the frame_data or crop_data information for the issue elements. (Use get_issue instead to see that info).

Returns

List of all issues data.

Return type

List[Issue]

remove_elements_from_issue(issue_id, elements)

Remove elements from an issue.

Parameters
  • issue_id (str) – The issue id.

  • elements (List[IssueElement]) – The elements to remove from the issue.

Return type

None

update_elements_status(issue_id, element_ids, new_status)

Update issue elements status.

Parameters
  • issue_id (str) – The issue id.

  • new_status (str) – The new status. (“unstarted”, “done”)

  • element_ids (List[str]) –

Return type

None

update_issue_name(issue_id, issue_name)

Update issue name.

Parameters
  • issue_id (str) – The issue id.

  • issue_name (str) – The new issue name.

Return type

None

update_issue_state(issue_id, issue_state)

Update issue state.

Parameters
  • issue_id (str) – The issue id.

  • issue_state (str) – The new issue state. (“triage”, “inProgress”, “inReview”, “resolved”, “cancelled”)

Return type

None

class aquariumlearning.LabelClassMap(*, entries=None)

A collection of ClassMapEntries that defines how to interpret classifications.

Parameters

entries (list of ClassMapEntry, optional) – List of classification entries. Defaults to None.

add_entry(entry)

Add a ClassMapEntry.

Parameters

entry (ClassMapEntry) – entry

Return type

None

static from_classnames(classnames)

A helper utility to generate a set of distinct colors given a list of classnames based on the number of classes

Parameters

classnames (List[str]) – List of classifications as strings

Returns

A LabelClassMap containing categorical colors for the provided classnames.

Return type

LabelClassMap

static from_classnames_max10(classnames)

A helper utility to generate a set of distinct colors given a list of classnames

Parameters

classnames (list of str) – List of up to ten classifications as strings

Returns

A LabelClassMap containing categorical colors for the provided classnames.

Return type

LabelClassMap

static from_classnames_max20(classnames)

A helper utility to generate a set of distinct colors given a list of classnames

Parameters

classnames (List[str]) – List of up to twenty classifications as strings

Returns

A LabelClassMap containing categorical colors for the provided classnames.

Return type

LabelClassMap

static from_classnames_turbo(classnames)

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the turbo color scheme, and will be assigned in order from dark to light.

Parameters

classnames (list of str) – List of classifications as strings.

Returns

A LabelClassMap containing colors for the provided classnames.

Return type

LabelClassMap

static from_classnames_viridis(classnames)

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the viridis color scheme, and will be assigned in order from dark to light.

Parameters

classnames (list of str) – List of classifications as strings.

Returns

A LabelClassMap containing colors for the provided classnames.

Return type

LabelClassMap

class aquariumlearning.LabeledDataset(*, pipeline_mode='BATCH')

A container used to construct a labeled dataset.

Typical usage is to create a LabeledDataset, add multiple LabeledFrames to it, then serialize the frames to be submitted.

add_frame(frame)

Add a LabeledFrame to this dataset.

Parameters

frame (LabeledFrame) – A LabeledFrame in this dataset.

Return type

None

write_crop_embeddings_to_file_streaming(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_embeddings_to_file(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_frame_embeddings_to_file_streaming(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_frames_to_file_streaming(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_label_assets_to_file(filelike)

Write the frame’s label assets to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_labels_to_file_streaming(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_to_file(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

class aquariumlearning.LabeledFrame(*, frame_id, date_captured=None, device_id=None)

A labeled frame for a dataset.

Parameters
  • frame_id (str) – A unique id for this frame.

  • date_captured (str, optional) – ISO formatted datetime string. Defaults to None.

  • device_id (str, optional) – The device that generated this frame. Defaults to None.

add_audio(*, sensor_id, audio_url, date_captured=None)

Add an audio “sensor” data to this frame.

Parameters
  • sensor_id (str) – The id of this sensor.

  • audio_url (str) – The URL to load this audio data (mp3, etc.).

  • date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_coordinate_frame_3d(*, coord_frame_id, position=None, orientation=None, parent_frame_id=None)

Add a 3D Coordinate Frame.

Parameters
  • coord_frame_id (str) – String identifier for this coordinate frame

  • position (Optional[Dict[POSITION, Union[int, float]]], optional) – Dict of the form {x, y, z}. Defaults to None.

  • orientation (Optional[Dict[ORIENTATION, Union[int, float]]], optional) – Quaternion rotation dict of the form {w, x, y, z}. Defaults to None.

  • parent_frame_id (Optional[str], optional) – String id of the parent coordinate frame. Defaults to None.

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per label crop embedding

Parameters
  • label_id (str) – [description]

  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_embedding(*, embedding, crop_embeddings=None, model_id='')

DEPRECATED! PLEASE USE add_frame_embedding and add_crop_embedding Add an embedding to this frame, and optionally to crops/labels within it.

If provided, “crop_embeddings” is a list of dicts of the form:

‘uuid’: the label id for the crop/label ‘embedding’: a vector of floats between length 0 and 12,000.

Parameters
  • embedding (list of floats) – A vector of floats between length 0 and 12,000.

  • crop_embeddings (list of dicts, optional) – A list of dictionaries representing crop embeddings. Defaults to None.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_frame_embedding(*, embedding, model_id='')

Add an embedding to this frame

Parameters
  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_geo_latlong_data(lat, lon)

Add a user provided EPSG:4326 WGS84 lat long pair to each frame

We expect these values to be floats

Parameters
  • lat (float) – lattitude of Geo Location

  • lon (float) – longitude of Geo Location

Return type

None

add_image(*, sensor_id, image_url, preview_url=None, date_captured=None, width=None, height=None)

Add an image “sensor” data to this frame.

Parameters
  • sensor_id (str) – The id of this sensor.

  • image_url (str) – The URL to load this image data.

  • preview_url (Optional[str], optional) – A URL to a compressed version of the image. It must be the same pixel dimensions as the original image. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

  • width (Optional[int], optional) – The width of the image in pixels. Defaults to None.

  • height (Optional[int], optional) – The height of the image in pixels. Defaults to None.

Return type

None

add_image_overlay(*, sensor_id, overlay_id, image_url, date_captured=None, width=None, height=None)

Add an image overlay for the given “sensor” to this frame.

Parameters
  • sensor_id (str) – The id of this sensor.

  • overlay_id (str) – The id of this overlay.

  • image_url (str) – The URL to load this image data.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

  • width (Optional[int], optional) – The width of the image in pixels. Defaults to None.

  • height (Optional[int], optional) – The height of the image in pixels. Defaults to None.

Return type

None

add_label_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D bounding box.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (dict, optional) – Links between labels. Defaults to None.

Return type

None

add_label_2d_classification(*, sensor_id, label_id, classification, secondary_labels=None, user_attrs=None)

Add a label for 2D classification.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • secondary_labels (dict, optional) – dictionary of secondary labels

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, user_attrs=None)

Add a label for a 2D keypoints task.

A keypoint is a dictionary of the form:

‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • keypoints (list of dicts) – The keypoints of this detection

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_line_segment(*, sensor_id, label_id, classification, x1, y1, x2, y2, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D line segment.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • x1 (int or float) – The x-coord of the first vertex in pixels

  • y1 (int or float) – The x-coord of the first vertex in pixels

  • x2 (int or float) – The x-coord of the first vertex in pixels

  • y2 (int or float) – The x-coord of the first vertex in pixels

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (Optional[Dict[str, Any]]) –

Return type

None

add_label_2d_polygon_list(*, sensor_id, label_id, classification, polygons, center=None, user_attrs=None)

Add a label for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:
‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}
Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • polygons (list of dicts) – The polygon geometry

  • center (list of ints or floats, optional) – The center point of the instance

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_semseg(*, sensor_id, label_id, mask_url=None, mask_data=None)

Add a label for 2D semseg.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • mask_url (str, optional) – URL to the pixel mask png.

  • mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]

Return type

None

add_label_3d_classification(*, label_id, classification, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters
  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • coord_frame_id (str, optional) – The coordinate frame id.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_3d_cuboid(*, label_id, classification, position, dimensions, rotation, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add a label for a 3D cuboid.

Parameters
  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • position (list of float) – the position of the center of the cuboid

  • dimensions (list of float) – the dimensions of the cuboid

  • rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (dict, optional) – Links between labels. Defaults to None.

  • coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_label_text_token(*, sensor_id, label_id, index, token, classification, visible, user_attrs=None)

Add a label for a text token.

Parameters
  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • index (int) – the index of this token in the text

  • token (str) – the text content of this token

  • classification (str) – the classification string

  • visible (bool) – is this a visible token in the text

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_obj(*, sensor_id, obj_url, coord_frame_id=None, date_captured=None)

Add a .obj file to the frame for text based geometry

Parameters
  • sensor_id (str) – sensor id

  • obj_url (str) – URL to where the object is located

  • coord_frame_id (Optional[str], optional) – ID for the coordinate frame. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_point_cloud_bins(*, sensor_id, pointcloud_url=None, kitti_format_url=None, intensity_url=None, range_url=None, coord_frame_id=None, date_captured=None)

Add a point cloud sensor data point to this frame, contained in dense binary files of little-endian values, similar to the raw format of KITTI lidar data. You can provide a combination of the following values, as long as at least either kitti_format_url or pointcloud_url are provided.

Parameters
  • sensor_id (str) – Sensor id

  • pointcloud_url (Optional[str]) – URL for the pointcloud: float32 [x1, y1, z1, x2, y2, z2, …]

  • kitti_format_url (Optional[str]) – URL for the pointcloud + intensity: float32 [x1, y1, z1, i1, x2, y2, z2, i2, …]

  • intensity_url (Optional[str]) – URL for the Intensity Pointcloud: unsigned int32 [i1, i2, …]

  • range_url (Optional[str]) – URL for the Range Pointcloud: float32 [r1, r2, …]

  • coord_frame_id (Optional[str], optional) – Id for the Coordinate Frame. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_point_cloud_pcd(*, sensor_id, pcd_url, coord_frame_id=None, date_captured=None)

Add a point cloud sensor data point to this frame, contained in PCD format. ascii, binary, and binary_compressed formats are supported. Numeric values for the following column names are expected: x, y, z, intensity (optional), range (optional)

Parameters
  • sensor_id (str) – sensor id

  • pcd_url (str) – URL to PCD formated data

  • coord_frame_id (Optional[str], optional) – The coordinate frame id. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_text(*, sensor_id, text, date_captured=None)

Add a text “sensor” data to this frame.

Parameters
  • sensor_id (str) – The id of this sensor.

  • text (str) – The text body.

  • date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_user_metadata(key, val, val_type=None)

Add a user provided metadata field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters
  • key (str) – The key for your metadata field

  • val (Union[str, int, float, bool]) – value

  • val_type (Literal["str", "int", "float", "bool"], optional) – type of val as string. Defaults to None.

Return type

None

add_user_metadata_list(key, val, list_elt_type=None)

Add a user provided metadata list field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters
  • key (str) – The key for your metadata field

  • val (Union[List[int], List[str], List[bool], List[float], Tuple[int], Tuple[str], Tuple[bool], Tuple[float]]) – value

  • list_elt_type (Literal["str", "int", "float", "bool"], optional) – type of list elements as string. Applies to all members of val. Defaults to None.

Return type

None

add_video(*, sensor_id, video_url, date_captured=None)

Add a video “sensor” data to this frame.

Parameters
  • sensor_id (str) – The id of this sensor.

  • video_url (str) – The URL to load this video data (mp4, webm, etc.).

  • date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

to_dict()

Convert this frame into a dictionary representation.

Returns

dictified frame

Return type

dict

validate_and_backfill_crop_embeddings()

Determines if there are frame embeddings but not Crop embeddings Backfills crop embeddings if Frame embeddings exist Returns True if backfill was necessary

Returns

returns if backfill on frame was necessary

Return type

bool

class aquariumlearning.MetricsManager(client, project_id)

A manager for interacting with metrics within a given project.

Contains the following constants:

MetricsManager.ORDER_CONF_DESC
MetricsManager.ORDER_CONF_ASC
MetricsManager.ORDER_IOU_DESC
MetricsManager.ORDER_IOU_ASC
MetricsManager.ORDER_BOX_SIZE_DESC
MetricsManager.ORDER_BOX_SIZE_ASC
Parameters
  • client (Client) – An Aquarium Learning Python Client object.

  • project_id (str) – The project id associated with this manager.

fetch_confusions(dataset_id, inferences_id, opts)

Fetch confusions for a given dataset + inference set pair.

The options define the parameters of the query. The options are a dictionary of the form:

{
    'confidence_threshold': Union[float, int],
    'iou_threshold': Union[float, int],
    'queries': List[QueryEntry],
    'ordering': QueryOrdering,
    'max_results': int (optional, defaults to 10,000),
    'cur_offset': int (optional, defaults to 0),
}

Confidence and iou thresholds can be any multiple of 0.1 between 0.0 and 1.0. Queries are a list of queries produced from helper methods such as

metrics_manager.make_confusions_query()

Ordering is defined by constants on the class object, such as:

metrics_manager.ORDER_CONF_DESC

Parameters
  • dataset_id (str) – The dataset id.

  • inferences_id (str) – The inference set id.

  • opts (ConfusionsOpts) – Options for the query.

Returns

All of your query results, up to max_results (default = 10k)

Return type

ConfusionsResultDict

make_cell_query(gt, inf)

Make a query entry for a specific confusion of gt as as inf.

Parameters
  • gt (str) – The classname of the ground truth class.

  • inf (str) – The classname of the inference class.

Returns

Result query entry.

Return type

QueryEntry

make_confused_as_query(name)

Make a query entry for all confusions as name. (inf == name, gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns

Result query entry.

Return type

QueryEntry

Parameters

name (str) –

make_confused_from_query(name)

Make a query entry for all confusions from name. (gt == name, gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns

Result query entry.

Return type

QueryEntry

Parameters

name (str) –

make_confusions_query()

Make a query entry for all confusions (gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns

Result query entry.

Return type

QueryEntry

make_correct_query()

Make a query entry for all correct detections/classifications (gt == inf)

Returns

Result query entry.

Return type

QueryEntry

make_false_negatives_query()

Make a query entry for all false negative detections.

These are cases without corresponding inference detections for a ground truth detection.

Returns

Result query entry.

Return type

QueryEntry

make_false_positives_query()

Make a query entry for all false positive detections.

These are cases without corresponding ground truth detections for an inference detection.

Returns

Result query entry.

Return type

QueryEntry

class aquariumlearning.StratifiedMetricsDefinition(name, ordered_values)

Definitions for stratified metrics given object-level attributes

Parameters
  • name (str) – The name of this attribute, which should match the attribute on object labels.

  • ordered_values (List[str]) – The ordered list of valid values to group by.