Aquarium Learning Python Client

class aquariumlearning.ClassMapEntry(*, name, class_id, color, train_name=None, train_id=None, category=None, category_id=None, has_instances=True, ignore_in_eval=False, train_color=None, keypoint_names=None, skeleton=None)

A description of how to interpret a classification.

In the common case, only three values are needed: - The string representation of the class - The int [0,255] representation of the class - What color to render it as.

In more complex cases, some classes may be ignored in evaluation, or collapsed together into a different class at inference time, or tracked as part of a larger category.

Parameters

name (str) – The string representation of the class
class_id (int) – The int representation of the class
color (Union[Tuple[int, int, int], List[int]]) – A length 3 list/tuple of RGB int values
train_name (Optional[int], optional) – The string representation of the class that this label will be inferred as. Defaults to None.
train_id (Optional[str], optional) – The int representation of the class that this label will be infererred as. Defaults to None.
category (Optional[str], optional) – The string representation of the parent category. Defaults to None.
category_id (Optional[int], optional) – The int representation of the parent category. Defaults to None.
has_instances (bool, optional) – Whether each label of this class is a separate instance. Defaults to True.
ignore_in_eval (bool, optional) – Whether to ignore this class while evaluating metrics. Defaults to False.
train_color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.
keypoint_names (Optional[List[str]], optional) – List of keypoint names that exist for this class. Indicies of keypoints in this list are used for the skeletons.
skeleton (Optional[List[Union[Tuple[int, int], List[int]]]], optional) – List of integer pairs that reference indicies in keypoint_names that represent the skeleton for this class.

to_dict()

Returns a dictified form of this data structure.

Returns: dict represenation of Class Map Entry
Return type: Dict[str, Any]

class aquariumlearning.ClassMapUpdateEntry(*, name=None, class_id=None, color=None, train_color=None, keypoint_names=None, skeleton=None)

A description of a change to be made to a classification. Classification can be referred to by name or id

Parameters

name (Optional[str], optional) – The string representation of the class
class_id (Optional[int], optional) – The int representation of the class
color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values
train_color (Optional[Union[Tuple[int, int, int], List[int]]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.
keypoint_names (Optional[List[str]], optional) – List of keypoint names that exist for this class. Indicies of keypoints in this list are used for the skeletons.
skeleton (Optional[List[Union[Tuple[int, int], List[int]]]], optional) – List of integer pairs that reference indicies in keypoint_names that represent the skeleton for this class.

to_dict()

Returns a dictified form of this data structure.

Returns: dict represenation of Class Map Update Entry
Return type: Dict[str, Any]

class aquariumlearning.Client(*, api_endpoint='https://illume.aquariumlearning.com/api/v1', **kwargs)

Client class that interacts with the Aquarium REST API.

Parameters

api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.
kwargs (Any) –

add_to_streaming_dataset(project_id, dataset_id, dataset=None, embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, is_anon_mode=False, upload_version=None, validate_frames=False, reuse_embeddings=False)

Append or update a streaming dataset with the provided data urls. Dataset must be have already been created using STREAMING pipeline_mode.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
dataset (Optional[LabeledOrUnlabeledDataset], optional) – The LabeledDataset or UnlabeledDataset to upload.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
upload_version (str, optional) – A name for a distinct set of frames that are new or updated within the dataset, for comparison down the line. Defaults to last-created upload version.
validate_frames (bool, optional) – Flag to tell the client to pre-check that all queued frames conform to a project’s primary_task params. Defaults to False
reuse_embeddings (bool) –

Return type

None

create_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='STREAMING', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_dataset(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledOrUnlabeledDataset], optional) – The LabeledDataset or UnlabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
is_unlabeled_indexed_dataset (bool, optional) – (DEPRECATED). Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.
is_unlabeled_indexed_dataset – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –

Return type

None

create_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='STREAMING')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_inferences(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.

Return type

None

create_or_update_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='STREAMING', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledOrUnlabeledDataset], optional) – The LabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.
is_unlabeled_indexed_dataset (bool, optional) – (DEPRECATED). Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –

Return type

None

create_or_update_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='STREAMING')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.

Return type

None

create_project(project_id, label_class_map, primary_task=None, secondary_labels=None, frame_links=None, label_links=None, default_camera_target=None, default_camera_position=None, custom_metrics=None, max_shown_categories=None, stratified_metrics=None, include_no_gt=None, metrics_confidence_threshold=None, metrics_iou_threshold=None, external_metadata=None, binary_classification_negative_class=None)

Create a new project via the REST API.

Parameters

project_id (str) – project_id
label_class_map (LabelClassMap) – The label class map used to interpret classifications.
primary_task (Optional[str], optional) – Any specific primary task for a non-object detection or classification task. Can be ‘2D_SEMSEG’ or ‘2D_INSTANCE_SEGMENTATION’ or ‘CLASSIFICATION’ or ‘CLASSIFICATION_WITH_GEOMETRY’ or ‘BINARY_CLASSIFICATION’ or ‘2D_OBJECT_DETECTION’.
secondary_labels ([type], optional) – List of secondary labels in classification tasks
frame_links (Optional[List[str]], optional) – List of string keys for links between frames
label_links (Optional[List[str]], optional) – List of string keys for links between labels
default_camera_target (Optional[List[float]], optional) – For 3D scenes, the default camera target
default_camera_position (Optional[List[float]], optional) – For 3D scenes, the default camera position
custom_metrics (Optional[ Union[CustomMetricsDefinition, List[CustomMetricsDefinition]] ], optional) – Defines which custom metrics exist for this project, defaults to None.
max_shown_categories (Optional[int], optional) – For categorical visualizations, set the maximum shown simultaneously. Max 100.
stratified_metrics (Optional[List[StratifiedMetricsDefinition]], optional) – Defines what object-level attributes to stratify metrics over.
metrics_confidence_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with IOU) to match your ground truth (GT) labels with your inference labels. Defaults to 0.1 if not specified.
metrics_iou_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with confidence) to match your ground truth (GT) labels with your inference labels. Defaults to 0.5 if not specified.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the project itself
binary_classification_negative_class (Optional[str]) – Required when primary_task is ‘BINARY_CLASSIFICATION’. The name of the negative class.
include_no_gt (Optional[bool]) –

Return type

None

current_abstract_dataset_process_step_status(project_id, dataset_id)

Returns the process steps statuses for a given dataset or inferenceset

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

A set of process step statuses that exist for a given abstract dataset

Return type

Dict[str, Any]

current_dataset_process_state(project_id, dataset_id)

Current processing state of a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

current_inferences_process_state(project_id, dataset_id, inferences_id)

current processing state of inferences.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

dataset_exists(project_id, dataset_id)

Check if a dataset exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

Whether the dataset already exists.

Return type

bool

delete_dataset(project_id, dataset_id)

Mark a dataset for deletion

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Return type

None

delete_project(project_id)

Mark a project for deletion

Parameters: project_id (str) – project_id
Return type: None

get_dataset(project_id, dataset_id)

Get existing dataset for a project.

Parameters

project_id (str) – The project id.
dataset_id (str) – dataset_id

Returns

The dataset info.

Return type

dict

get_dataset_ingest_error_logs(project_id, dataset_id)

Get ingest error log entries for a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

List of error entries

Return type

list[dict]

get_dataset_object_metadata_schema(project_id, dataset_id)

Get dataset object metadata schema

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Return type

List[Dict[str, str]]

get_datasets(project_id, include_archived)

Get existing datasets for a project.

Parameters

project_id (str) – The project id.
include_archived (Optional[bool]) –

Returns

A list of dataset info for the project.

Return type

list

get_frame(project_id, dataset_id, frame_id)

Get frame info (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_id (str) – The frame id.

Return type

LabeledFrame

get_frame_ids(project_id, dataset_id)

Get frame ids in a dataset (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.

Return type

List[str]

get_frames(project_id, dataset_id, frame_ids)

Get multple frame infos (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_ids (List[str]) – The list of frame ids.

Return type

Dict[str, LabeledFrame]

get_inferences_ingest_error_logs(project_id, dataset_id, inferences_id)

Get ingest error log entries for an inference set.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

List of error entries

Return type

list[dict]

get_issue_manager(project_id)

Get an issue manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The issue manager object.
Return type: IssueManager

get_metrics_manager(project_id)

Get a metrics manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The metrics manager object.
Return type: MetricsManager

get_project(project_id)

Get detailed info about a specific project

Returns: detailed info about a project
Return type: Dict[str, Any]
Parameters: project_id (str) –

get_projects()

Get info about existing projects

Returns: Project Info
Return type: list of dict

get_work_queue_manager(project_id)

Get a work queue manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The work queue manager object.
Return type: WorkQueueManager

inferences_exists(project_id, dataset_id, inferences_id)

Check if a set of inferences exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

Whether the inferences id already exists.

Return type

bool

is_dataset_archived(project_id, dataset_id)

Check if a dataset has been archived.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

If the dataset is archived. Returns False if dataset does not exist

Return type

bool

is_dataset_processed(project_id, dataset_id)

Check if a dataset is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

If the dataset is done processing.

Return type

bool

is_inferences_processed(project_id, dataset_id, inferences_id)

Check if a set of inferences is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

If the inferences set is done processing.

Return type

bool

preview_frame(project_id, labeled_frame, inference_frame=None)

prints out a URL that lets you preview a provided frame in the web browser Useful for debugging data and image url issues.

Parameters

project_id (str) – Name of project to be associated for this frame preview (for label class association)
labeled_frame (LabeledFrame) – Labeled Frame desired for preview in web-app
inference_frame (Optional[InferencesFrame], optional) – Labeled Inference Desired for preview in web-app. Defaults to None.

Return type

None

project_exists(project_id)

Checks whether a project exists.

Parameters: project_id (str) – project_id
Returns: Does project exist
Return type: bool

set_credentials(*, token=None, app_id=None, app_key=None, api_key=None)

Set credentials for the client.

Parameters

api_key (str, optional) – A string for a long lived API key. Defaults to None.
token (str, optional) – A JWT providing auth credentials. Defaults to None.
app_id (str, optional) – Application ID string. Defaults to None.
app_key (str, optional) – Application secret key. Defaults to None.

Raises

Exception – Invalid credential combination provided.

Return type

None

update_dataset_frames(project_id, dataset_id, update, upload_version=None)

Update LabeledFrames in a dataset. If a frame_id does not already exist in the dataset, its update will be dropped during processing. This method is useful for bulk-updating frame metadata: sensor data, external metadata, etc. To update specific labels, use update_dataset_labels

Parameters

update (List[LabeledFrame]) – The partial frame updates to be applied to this dataset.
project_id (str) –
dataset_id (str) –
upload_version (Optional[str]) –

Return type

None

update_dataset_labels(project_id, dataset_id, update, upload_version=None)

Update labels in a frame in a dataset. If a frame_id does not already exist in the dataset, its update will be dropped during processing. Updates are additive; new labels will be appended and existing labels will be updated with not-None values

Parameters

update (List[UpdateGTLabelSet]) – The partial label updates to be applied to this frame.
project_id (str) –
dataset_id (str) –
upload_version (Optional[str]) –

Return type

None

update_dataset_metadata(project_id, dataset_id, external_metadata)

Update dataset metadata

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_dataset_object_metadata_schema(project_id, dataset_id, object_metadata_fields)

Update dataset object metadata schema

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
object_metadata_fields (List[Dict[str, Any]]) – list of object metadata fields, formatted {“name”: “width_bucket”, “type”: “STRING”} Type must be one of STRING, BOOL, NUMERIC, INTEGER, FLOAT, or an array of a valid type ARRAY<type>.

Return type

None

update_frames_metadata(project_id, dataset_id, frame_metadata_updates)

Update the external metadata for labeled frames

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_metadata_updates (Dict[str, Dict[str, Any]]) – dictionary of frame_id to dictionary of metadata fields to change

Return type

None

update_inferences_metadata(project_id, inferences_id, external_metadata)

Update inference set metadata

Parameters

project_id (str) – The project id.
inferences_id (str) – The inferences id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_label_class_map_colors(project_id, changed_label_classes)

Updates label class colors of a specific project

Parameters

project_id (str) – The project id.
changed_label_classes (List[ClassMapUpdateEntry]) – The list of label classes with changed colors. Must be a subset of the project’s overall label class map

Return type

None

update_project_metadata(project_id, external_metadata)

Update project metadata

Parameters

project_id (str) – The project id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

upload_asset_from_filepath(project_id, dataset_id, filepath)

Upload an asset from a local file path. This is useful in cases where you have data on your local machine that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
filepath (str) – The filepath to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

upload_asset_from_url(project_id, dataset_id, source_url)

Upload an asset from a private url. This is useful in cases where you have data easily accessible on your network that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
source_url (str) – The source url to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

class aquariumlearning.CollectionClient(*args, **kwargs)

Client class that interacts with the Aquarium REST API. Also handles extra work around collecting samples for collection campigns

Parameters

api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.
args (Any) –
kwargs (Any) –

add_to_streaming_dataset(project_id, dataset_id, dataset=None, embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, is_anon_mode=False, upload_version=None, validate_frames=False, reuse_embeddings=False)

Append or update a streaming dataset with the provided data urls. Dataset must be have already been created using STREAMING pipeline_mode.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
dataset (Optional[LabeledOrUnlabeledDataset], optional) – The LabeledDataset or UnlabeledDataset to upload.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
upload_version (str, optional) – A name for a distinct set of frames that are new or updated within the dataset, for comparison down the line. Defaults to last-created upload version.
validate_frames (bool, optional) – Flag to tell the client to pre-check that all queued frames conform to a project’s primary_task params. Defaults to False
reuse_embeddings (bool) –

Return type

None

create_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='STREAMING', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_dataset(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledOrUnlabeledDataset], optional) – The LabeledDataset or UnlabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
is_unlabeled_indexed_dataset (bool, optional) – (DEPRECATED). Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.
is_unlabeled_indexed_dataset – Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –

Return type

None

create_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='STREAMING')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update. (backcompat shortcut for create_or_update_inferences(…))

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Does not work in streaming pipeline mode. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.

Return type

None

create_or_update_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True, external_metadata=None, pipeline_mode='STREAMING', is_unlabeled_indexed_dataset=False, seed_dataset_name_for_unlabeled_search=None, is_anon_mode=False, existing_embedding_version_uuid=None)

Create or update a dataset with the provided data urls. Dataset must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
data_url (Optional[str], optional) – A URL to the serialized dataset entries.
embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.
dataset (Optional[LabeledOrUnlabeledDataset], optional) – The LabeledDataset to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the dataset itself
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.
is_unlabeled_indexed_dataset (bool, optional) – (DEPRECATED). Whether this dataset is meant to be used as the seed dataset for unlabeled indexed search.
seed_dataset_name_for_unlabeled_search (Optional[str], optional) – (DEPRECATED). The name of the existing dataset or inf set whose issue elts will be used as the seed for unlabeled indexed search.
is_anon_mode (bool, optional) – flag to tell aquarium if url images are reachable from public internet or shared bucket. False if reachable, True if Not.
existing_embedding_version_uuid (str, optional) – uuid of the embedding version of an existing dataset that shares the same embedding space.
check_first_frame (bool) –

Return type

None

create_or_update_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True, external_metadata=None, pipeline_mode='STREAMING')

Create or update an inference set with the provided data urls. Dataset + Inferences must be created using STREAMING pipeline_mode to update.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – A unique identifier for this set of inferences.
data_url (Optional[str], optional) – A URL to the serialized inference entries.
embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.
inferences (Optional[Inferences], optional) – The inferences to upload.
wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.
wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.
embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.
delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the inferences itself
pipeline_mode (str, optional) – BATCH or STREAMING. Defaults to STREAMING.

Return type

None

create_project(project_id, label_class_map, primary_task=None, secondary_labels=None, frame_links=None, label_links=None, default_camera_target=None, default_camera_position=None, custom_metrics=None, max_shown_categories=None, stratified_metrics=None, include_no_gt=None, metrics_confidence_threshold=None, metrics_iou_threshold=None, external_metadata=None, binary_classification_negative_class=None)

Create a new project via the REST API.

Parameters

project_id (str) – project_id
label_class_map (LabelClassMap) – The label class map used to interpret classifications.
primary_task (Optional[str], optional) – Any specific primary task for a non-object detection or classification task. Can be ‘2D_SEMSEG’ or ‘2D_INSTANCE_SEGMENTATION’ or ‘CLASSIFICATION’ or ‘CLASSIFICATION_WITH_GEOMETRY’ or ‘BINARY_CLASSIFICATION’ or ‘2D_OBJECT_DETECTION’.
secondary_labels ([type], optional) – List of secondary labels in classification tasks
frame_links (Optional[List[str]], optional) – List of string keys for links between frames
label_links (Optional[List[str]], optional) – List of string keys for links between labels
default_camera_target (Optional[List[float]], optional) – For 3D scenes, the default camera target
default_camera_position (Optional[List[float]], optional) – For 3D scenes, the default camera position
custom_metrics (Optional[ Union[CustomMetricsDefinition, List[CustomMetricsDefinition]] ], optional) – Defines which custom metrics exist for this project, defaults to None.
max_shown_categories (Optional[int], optional) – For categorical visualizations, set the maximum shown simultaneously. Max 100.
stratified_metrics (Optional[List[StratifiedMetricsDefinition]], optional) – Defines what object-level attributes to stratify metrics over.
metrics_confidence_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with IOU) to match your ground truth (GT) labels with your inference labels. Defaults to 0.1 if not specified.
metrics_iou_threshold (Optional[float], optional) – In order to calculate metrics + confusion matrices, Aquarium uses this threshold (in combination with confidence) to match your ground truth (GT) labels with your inference labels. Defaults to 0.5 if not specified.
external_metadata (Optional[Dict[str, Any]], optional) – A JSON object that can be used to attach metadata to the project itself
binary_classification_negative_class (Optional[str]) – Required when primary_task is ‘BINARY_CLASSIFICATION’. The name of the negative class.
include_no_gt (Optional[bool]) –

Return type

None

current_abstract_dataset_process_step_status(project_id, dataset_id)

Returns the process steps statuses for a given dataset or inferenceset

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

A set of process step statuses that exist for a given abstract dataset

Return type

Dict[str, Any]

current_dataset_process_state(project_id, dataset_id)

Current processing state of a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

current_inferences_process_state(project_id, dataset_id, inferences_id)

current processing state of inferences.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

dataset_exists(project_id, dataset_id)

Check if a dataset exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

Whether the dataset already exists.

Return type

bool

delete_dataset(project_id, dataset_id)

Mark a dataset for deletion

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Return type

None

delete_project(project_id)

Mark a project for deletion

Parameters: project_id (str) – project_id
Return type: None

get_dataset(project_id, dataset_id)

Get existing dataset for a project.

Parameters

project_id (str) – The project id.
dataset_id (str) – dataset_id

Returns

The dataset info.

Return type

dict

get_dataset_ingest_error_logs(project_id, dataset_id)

Get ingest error log entries for a dataset.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

List of error entries

Return type

list[dict]

get_dataset_object_metadata_schema(project_id, dataset_id)

Get dataset object metadata schema

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Return type

List[Dict[str, str]]

get_datasets(project_id, include_archived)

Get existing datasets for a project.

Parameters

project_id (str) – The project id.
include_archived (Optional[bool]) –

Returns

A list of dataset info for the project.

Return type

list

get_frame(project_id, dataset_id, frame_id)

Get frame info (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_id (str) – The frame id.

Return type

LabeledFrame

get_frame_ids(project_id, dataset_id)

Get frame ids in a dataset (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.

Return type

List[str]

get_frames(project_id, dataset_id, frame_ids)

Get multple frame infos (for datasets only, not inferences)

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_ids (List[str]) – The list of frame ids.

Return type

Dict[str, LabeledFrame]

get_inferences_ingest_error_logs(project_id, dataset_id, inferences_id)

Get ingest error log entries for an inference set.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

List of error entries

Return type

list[dict]

get_issue_manager(project_id)

Get an issue manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The issue manager object.
Return type: IssueManager

get_metrics_manager(project_id)

Get a metrics manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The metrics manager object.
Return type: MetricsManager

get_project(project_id)

Get detailed info about a specific project

Returns: detailed info about a project
Return type: Dict[str, Any]
Parameters: project_id (str) –

get_projects()

Get info about existing projects

Returns: Project Info
Return type: list of dict

get_unlabeled_results_for_issue(*, issue_uuid, unlabeled_dataset_name, sort_status='UNSORTED', unlabeled_only=True, retrigger_search=False)

returns similar results from unlabeled dataset search

Parameters

issue_uuid (str) – the uuid of the seed issue
unlabeled_dataset_name (str) – the name of the unlabeled search dataset
sort_status (Optional[SortStatus]) – filter the returned collection frames by status (default UNSORTED)
unlabeled_only (Optional[bool]) – return only collection frames that haven’t been sent to labeling
retrigger_search (Optional[bool]) – retrigger similarity search (NOTE: this only works for searches against unlabeled datasets. It will poll + wait for the result)

Returns

List of CollectionFrames for a given issue

Return type

List[Dict[str, Any]]

get_work_queue_manager(project_id)

Get a work queue manager object.

Parameters: project_id (str) – Project ID to manage.
Returns: The work queue manager object.
Return type: WorkQueueManager

inferences_exists(project_id, dataset_id, inferences_id)

Check if a set of inferences exists.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

Whether the inferences id already exists.

Return type

bool

is_dataset_archived(project_id, dataset_id)

Check if a dataset has been archived.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

If the dataset is archived. Returns False if dataset does not exist

Return type

bool

is_dataset_processed(project_id, dataset_id)

Check if a dataset is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id

Returns

If the dataset is done processing.

Return type

bool

is_inferences_processed(project_id, dataset_id, inferences_id)

Check if a set of inferences is fully processed.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
inferences_id (str) – inferences_id

Returns

If the inferences set is done processing.

Return type

bool

preview_frame(project_id, labeled_frame, inference_frame=None)

prints out a URL that lets you preview a provided frame in the web browser Useful for debugging data and image url issues.

Parameters

project_id (str) – Name of project to be associated for this frame preview (for label class association)
labeled_frame (LabeledFrame) – Labeled Frame desired for preview in web-app
inference_frame (Optional[InferencesFrame], optional) – Labeled Inference Desired for preview in web-app. Defaults to None.

Return type

None

project_exists(project_id)

Checks whether a project exists.

Parameters: project_id (str) – project_id
Returns: Does project exist
Return type: bool

sample_probabilities(frames)

Takes a list of Labeled Frames and stores scores for each based on each synced collection campaigns

Parameters: frames (List[LabeledFrame]) – a List of Labeled frames to score based on synced Collection Campaigns
Return type: Any

save_for_collection(override_sampling_threshold=None, target_sample_count=None, dry_run=False)

Based on the sampling threshold, take all sampled frames and upload those that score above the sampling threshold for each Collection Campaign.

Parameters

override_sampling_threshold (float, optional) – Override sampling threshold for all campaigns to save to server. If this value is set it will not use the sampling threshold set in UI for each Collection Campaign.
target_sample_count (int, optional) – Instead of collecting all samples above a specified similarity threshold, you can specify this option to save the N most similar samples.
dry_run (bool) –

Return type

None

set_credentials(*, token=None, app_id=None, app_key=None, api_key=None)

Set credentials for the client.

Parameters

api_key (str, optional) – A string for a long lived API key. Defaults to None.
token (str, optional) – A JWT providing auth credentials. Defaults to None.
app_id (str, optional) – Application ID string. Defaults to None.
app_key (str, optional) – Application secret key. Defaults to None.

Raises

Exception – Invalid credential combination provided.

Return type

None

sync_state(target_project_names=[], target_issue_uuids=[])

Downloads all collection campaigns and preps sampler with sample frames

Parameters

target_project_names (Optional[List[str]], optional) – List of project names whose collection campaigns should be sampled for
target_issue_uuids (Optional[List[str]], optional) – List of issue uuids whose collection campaigns should be sampled for. This takes precedence over ‘target_project_names’ and is mutually exclusive.

Return type

None

update_dataset_frames(project_id, dataset_id, update, upload_version=None)

Update LabeledFrames in a dataset. If a frame_id does not already exist in the dataset, its update will be dropped during processing. This method is useful for bulk-updating frame metadata: sensor data, external metadata, etc. To update specific labels, use update_dataset_labels

Parameters

update (List[LabeledFrame]) – The partial frame updates to be applied to this dataset.
project_id (str) –
dataset_id (str) –
upload_version (Optional[str]) –

Return type

None

update_dataset_labels(project_id, dataset_id, update, upload_version=None)

Update labels in a frame in a dataset. If a frame_id does not already exist in the dataset, its update will be dropped during processing. Updates are additive; new labels will be appended and existing labels will be updated with not-None values

Parameters

update (List[UpdateGTLabelSet]) – The partial label updates to be applied to this frame.
project_id (str) –
dataset_id (str) –
upload_version (Optional[str]) –

Return type

None

update_dataset_metadata(project_id, dataset_id, external_metadata)

Update dataset metadata

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_dataset_object_metadata_schema(project_id, dataset_id, object_metadata_fields)

Update dataset object metadata schema

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
object_metadata_fields (List[Dict[str, Any]]) – list of object metadata fields, formatted {“name”: “width_bucket”, “type”: “STRING”} Type must be one of STRING, BOOL, NUMERIC, INTEGER, FLOAT, or an array of a valid type ARRAY<type>.

Return type

None

update_frames_metadata(project_id, dataset_id, frame_metadata_updates)

Update the external metadata for labeled frames

Parameters

project_id (str) – The project id.
dataset_id (str) – The dataset id.
frame_metadata_updates (Dict[str, Dict[str, Any]]) – dictionary of frame_id to dictionary of metadata fields to change

Return type

None

update_inferences_metadata(project_id, inferences_id, external_metadata)

Update inference set metadata

Parameters

project_id (str) – The project id.
inferences_id (str) – The inferences id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

update_label_class_map_colors(project_id, changed_label_classes)

Updates label class colors of a specific project

Parameters

project_id (str) – The project id.
changed_label_classes (List[ClassMapUpdateEntry]) – The list of label classes with changed colors. Must be a subset of the project’s overall label class map

Return type

None

update_project_metadata(project_id, external_metadata)

Update project metadata

Parameters

project_id (str) – The project id.
external_metadata (Dict[Any, Any]) – The new metadata

Return type

None

upload_asset_from_filepath(project_id, dataset_id, filepath)

Upload an asset from a local file path. This is useful in cases where you have data on your local machine that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
filepath (str) – The filepath to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

upload_asset_from_url(project_id, dataset_id, source_url)

Upload an asset from a private url. This is useful in cases where you have data easily accessible on your network that you want to mirror in aquarium.

Parameters

project_id (str) – project_id
dataset_id (str) – dataset_id
source_url (str) – The source url to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

class aquariumlearning.CustomMetricsDefinition(name, metrics_type)

Definitions for custom user provided metrics.

Parameters

name (str) – The name of this metric.
metrics_type (str) – The metrics type, either ‘objective’ or ‘confusion_matrix’.

class aquariumlearning.Inferences

A container used to construct a set of inferences.

Typical usage is to create an Inferences object, add multiple InferencesFrames to it, then serialize the frames to be submitted.

add_frame(frame)

Add an InferencesFrame to this dataset.

Parameters: frame (InferencesFrame) – An InferencesFrame in this dataset.
Return type: None

delete_frame(frame_id)

Delete a frame from this dataset.

Parameters: frame_id (str) – id of frame to delete.
Return type: None

write_embeddings_to_file(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_label_assets_to_file(filelike)

Write the frame’s label assets to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_to_file(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

class aquariumlearning.InferencesFrame(*, frame_id, reuse_latest_embedding=False, update_type='ADD')

Parameters

frame_id (str) –
reuse_latest_embedding (bool) –
update_type (Literal['ADD', 'MODIFY', 'DELETE']) –

add_2d_bbox(*, sensor_id='DEFAULT', label_id, classification, top, left, width, height, confidence=None, area=None, iscrowd=None, user_attrs=None, links=None)

Add a 2D bounding box.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
area (float, optional) – The area of the image.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_classification(*, sensor_id='DEFAULT', label_id, classification, confidence=None, user_attrs=None, secondary_labels=None)

Add an inference for 2D classification.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
secondary_labels (Optional[Dict[str, Any]]) –

Return type

None

add_2d_instance_seg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, instance_mapping, resize_mode=None)

Add an label for 2D instance segmentation. These should provide either a mask_url or a mask_data, and a instance mapping.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
instance_mapping (List[Dict]) –
a list of instances present in the mask, each is a numeric id, string classification, and optional dict of additional attributes. As an example of one instance: .. code-block:
```
{
    'id': 1,
    'classification': "Person",
    'attributes': {
        'is_standing': false,
    }
}
```
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_2d_keypoints(*, sensor_id='DEFAULT', label_id, classification, keypoints, confidence=None, top=None, left=None, width=None, height=None, polygons=None, center=None, user_attrs=None)

Add an inference for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
keypoints (list of dicts) – The keypoints of this detection
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
top (int or float, optional) – The top of the bounding box in pixels. Defaults to None.
left (int or float, optional) – The left of the bounding box in pixels. Defaults to None.
width (int or float, optional) – The width of the bounding box in pixels. Defaults to None.
height (int or float, optional) – The height of the bounding box in pixels. Defaults to None.
polygons (list of dicts, optional) – The polygon geometry. Defaults to None.
center (list of ints or floats, optional) – The center point of the polygon instance. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_line_segment(*, sensor_id='DEFAULT', label_id, classification, x1, y1, x2, y2, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a 2D line segment.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_polygon_list(*, sensor_id='DEFAULT', label_id, classification, polygons, confidence=None, center=None, user_attrs=None)

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_semseg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, resize_mode=None)

Add an inference for 2D semseg. These should provide either a mask_url or a mask_data.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_3d_classification(*, label_id, classification, confidence=None, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
coord_frame_id (optional, str) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add an inference for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float) – confidence of prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per inference crop embedding

Parameters

label_id (str) – [description]
embedding (List[float]) – A vector of floats of at least length 2.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_custom_metric(name, value)

Add a custom metric for a given inference frame.

Parameters

name (str) – The name of the custom metric being added. Must match one of the custom_metrics already defined by the corresponding Project.
value (Union[float, List[List[Union[int, float]]]]) – The value of the custom metric (either a float or 2d list of floats/integers).

Return type

None

add_frame_embedding(*, embedding, model_id='')

Add an embedding to this frame

Parameters

embedding (List[float]) – A vector of floats of at least length 2.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_inference_2d_bbox(*, sensor_id='DEFAULT', label_id, classification, top, left, width, height, confidence, area=None, iscrowd=None, user_attrs=None)

Add an inference for a 2D bounding box.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
area (float, optional) – The area of the image.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_classification(*, sensor_id='DEFAULT', label_id, classification, confidence, user_attrs=None)

Add an inference for 2D classification.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_instance_seg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, instance_mapping, resize_mode=None)

Add an inference for 2D instance segmentation.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
instance_mapping (List[Dict]) –
a list of instances present in the mask, each is a numeric id, string classification, and optional dict of additional attributes. As an example of one instance: .. code-block:
```
{
    'id': 1,
    'classification': "Person",
    'attributes': {
        'is_standing': false,
    }
}
```
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_inference_2d_keypoints(*, sensor_id='DEFAULT', label_id, classification, keypoints, confidence, top=None, left=None, width=None, height=None, polygons=None, center=None, user_attrs=None)

Add an inference for a 2D keypoints task. Can include a bounding box and/or polygon.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
keypoints (list of dicts) – The keypoints of this detection
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
top (int or float, optional) – The top of the bounding box in pixels. Defaults to None.
left (int or float, optional) – The left of the bounding box in pixels. Defaults to None.
width (int or float, optional) – The width of the bounding box in pixels. Defaults to None.
height (int or float, optional) – The height of the bounding box in pixels. Defaults to None.
polygons (list of dicts, optional) – The polygon geometry. Defaults to None.
center (list of ints or floats, optional) – The center point of the polygon instance. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_line_segment(*, sensor_id='DEFAULT', label_id, classification, x1, y1, x2, y2, confidence, iscrowd=None, user_attrs=None, links=None)

Add an inference for a 2D line segment.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
confidence (float) –
links (Optional[Dict[str, Any]]) –

Return type

None

add_inference_2d_polygon_list(*, sensor_id='DEFAULT', label_id, classification, polygons, confidence, center=None, user_attrs=None)

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_semseg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, resize_mode=None)

Add an inference for 2D semseg.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_inference_3d_classification(*, label_id, classification, confidence, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
coord_frame_id (optional, str) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add an inference for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float) – confidence of prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_inference_text_token(*, sensor_id='DEFAULT', label_id, index, token, classification, visible, confidence, user_attrs=None)

Add a label for a text token.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float) – confidence of prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_text_token(*, sensor_id='DEFAULT', label_id, index, token, classification, visible, confidence=None, user_attrs=None)

Add a label for a text token.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float) – confidence of prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_user_metadata(key, val)

Add a user provided metadata field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

key (str) – The key for your metadata field
val (Union[str, int, float, bool]) – value
val_type (Literal["str", "int", "float", "bool"], optional) – type of val as string. Defaults to None.

Return type

None

to_dict()

Convert this label set into a dictionary representation.

Returns: dictified frame
Return type: dict

validate_and_backfill_crop_embeddings()

Determines if there are frame embeddings but not Crop embeddings Backfills crop embeddings if Frame embeddings exist Returns True if backfill was necessary

Returns: returns if backfill on frame was necessary
Return type: bool

class aquariumlearning.Issue(name, dataset, elements, element_type, created_at=None, updated_at=None, reporter=None, assignee=None, state=None, issue_id=None, inference_set=None, comments=None, template_type=None, custom_template_fields=None)

Definition for issue.

Parameters

name (str) – The issue name.
dataset (Optional[str]) – The dataset for this issue.
elements (List[IssueElement]) – The elements of the issue.
element_type (str) – The element type of the issue (“frame”, “crop”).
created_at (str) – The time of issue creation.
updated_at (str) – The time of last issue update.
reporter (str) – Email of issue creator.
assignee (Optional[str], optional) – Email of the person assigned the issue. Defaults to None.
state (str) – Current state of issue (“triage”, “inProgress”, “inReview”, “resolved”, “cancelled”). Defaults to “triage”.
issue_id (str) – The issue id.
inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.
template_type (Optional[str]) – The template type for this issue. (read-only)
custom_template_fields (Optional[Dict[str, Any]]) – Custom fields associated with the issue template type. (read-only)
comments (Optional[Dict[str, Any]]) –

class aquariumlearning.IssueElement(element_id, frame_id, element_type, dataset, status=None, inference_set=None, frame_data=None, crop_data=None, label_metadata=None, comments=None, updated_at=None)

Definition for issue element.

Parameters

element_id (str) – The element id.
frame_id (str) – The frame id of the element.
element_type (str) – The element type of the issue element (“frame” or “crop”).
dataset (str) – The base dataset an element is from. (Can be formatted as either “project_name.dataset_name” or just “dataset_name”)
inference_set (str) – The inference set an element is from (if any). (Can be formatted as either “project_name.inference_set_name” or just “inference_set_name”)
status (str) – (For read purposes, not element modification). The status of the element.
frame_data (Optional[Dict[str, Any]]) – (For read purposes, not element modification). JSON object that is based on either a LabeledFrame or InferencesFrame
crop_data (Optional[Dict[str, Any]]) – (For read purposes, not element modification). JSON object for the specific “frame_data” crop that a “crop”-type element is based on.
label_metadata (Optional[Dict[str, Any]]) – (For read purposes, not element modification). JSON object with confidence and IOU info if the element was created from a ground truth/inference comparison.
comments (Optional[Dict[str, Any]]) – (For read purposes, not element modification). JSON object with comments.
updated_at (Optional[str]) – (str) (For read purposes, not element modification). The time this issue element was last updated.

class aquariumlearning.IssueManager(client, project_id)

An issue manager for interacting with issues within a given project.

Parameters

client (Client) – An Aquarium Learning Python Client object.
project_id (str) – The project id associated with this manager.

add_elements_to_issue(issue_id, elements)

Add elements to an issue.

Parameters

issue_id (str) – The issue id.
elements (List[IssueElement]) – The elements to add to the issue.

Return type

None

create_issue(name, dataset, elements, element_type, inference_set=None, issue_type='Bucket', work_queue_batch_size=None)

Create an issue.

Parameters

name (str) – The issue name.
dataset (str) – The dataset for this issue.
elements (List[IssueElement]) – The initial elements of the issue.
element_type (str) – The element type of the issue (“frame” or “crop”).
inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.
issue_type (Optional[str], optional) – The type of this issue. Defaults to “Bucket”.
work_queue_batch_size (Optional[int]) –

Returns

The created issue id.

Return type

str

create_work_queue(name, dataset, elements, element_type, inference_set=None, work_queue_batch_size=100)

Create a work queue issue.

Parameters

name (str) – The issue name.
dataset (str) – The dataset for this issue.
elements (List[IssueElement]) – The initial elements of the issue.
element_type (str) – The element type of the issue (“frame” or “crop”).
inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.
work_queue_batch_size (int) –

Returns

The created issue id.

Return type

str

delete_issue(issue_id)

Delete an issue.

Parameters: issue_id (str) – The issue id.
Return type: None

get_issue(issue_id, exclude_frame_data=False, include_comment_data=False)

Get a specific issue in the associated project. This will also include all associated frame metadata associated with each element.

Parameters

issue_id (str) – The issue id.
exclude_frame_data (bool) – Set to True to exclude full frame data from the issue element (e.g. to cut down on download size).
include_comment_data (bool) – Set to True to include comments on the issue and issue elements.

Returns

The issue data (including frame_data, crop_data, label_metadata, and comments).

Return type

Issue

list_issues()

List issues in the associated project.

NOTE: this does NOT include the frame_data or crop_data information for the issue elements. (Use get_issue instead to see that info).

Returns: List of all issues data.
Return type: List[Issue]

remove_elements_from_issue(issue_id, elements)

Remove elements from an issue.

Parameters

issue_id (str) – The issue id.
elements (List[IssueElement]) – The elements to remove from the issue.

Return type

None

update_elements_status(issue_id, element_ids, new_status)

Update issue elements status.

Parameters

issue_id (str) – The issue id.
new_status (str) – The new status. (“unstarted”, “done”)
element_ids (List[str]) –

Return type

None

update_issue_name(issue_id, issue_name)

Update issue name.

Parameters

issue_id (str) – The issue id.
issue_name (str) – The new issue name.

Return type

None

update_issue_state(issue_id, issue_state)

Update issue state.

Parameters

issue_id (str) – The issue id.
issue_state (str) – The new issue state. (“triage”, “inProgress”, “inReview”, “resolved”, “cancelled”)

Return type

None

class aquariumlearning.LabelClassMap(*, entries=None)

A collection of ClassMapEntries that defines how to interpret classifications.

Parameters: entries (list of ClassMapEntry, optional) – List of classification entries. Defaults to None.

add_entry(entry)

Add a ClassMapEntry.

Parameters: entry (ClassMapEntry) – entry
Return type: None

static from_classnames(classnames)

A helper utility to generate a set of distinct colors given a list of classnames based on the number of classes

Parameters: classnames (List[str]) – List of classifications as strings
Returns: A LabelClassMap containing categorical colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_max10(classnames)

A helper utility to generate a set of distinct colors given a list of classnames

Parameters: classnames (list of str) – List of up to ten classifications as strings
Returns: A LabelClassMap containing categorical colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_max20(classnames)

A helper utility to generate a set of distinct colors given a list of classnames

Parameters: classnames (List[str]) – List of up to twenty classifications as strings
Returns: A LabelClassMap containing categorical colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_turbo(classnames)

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the turbo color scheme, and will be assigned in order from dark to light.

Parameters: classnames (list of str) – List of classifications as strings.
Returns: A LabelClassMap containing colors for the provided classnames.
Return type: LabelClassMap

static from_classnames_viridis(classnames)

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the viridis color scheme, and will be assigned in order from dark to light.

Parameters: classnames (list of str) – List of classifications as strings.
Returns: A LabelClassMap containing colors for the provided classnames.
Return type: LabelClassMap

class aquariumlearning.LabeledDataset(*, pipeline_mode='STREAMING')

A container used to construct a labeled dataset.

Typical usage is to create a LabeledDataset, add multiple LabeledFrames to it, then serialize the frames to be submitted.

Parameters: pipeline_mode (str) –

add_frame(frame)

Add a LabeledFrame to this dataset.

Parameters: frame (LabeledFrame) – A LabeledFrame in this dataset.
Return type: None

delete_frame(frame_id)

Delete a frame from this dataset.

Parameters: frame_id (str) – A frame_id in the dataset.
Return type: None

write_crop_embeddings_to_file_streaming(filelike)

Write the frame’s crops’ embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_embeddings_to_file(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_frame_embeddings_to_file_streaming(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_frames_to_file_streaming(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_label_assets_to_file(filelike)

Write the frame’s label assets to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_labels_to_file_streaming(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_to_file(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

class aquariumlearning.LabeledFrame(*, frame_id, date_captured=None, device_id=None, update_type='ADD')

A labeled frame for a dataset.

Parameters

frame_id (str) – A unique id for this frame.
date_captured (str, optional) – ISO formatted datetime string. Defaults to None.
device_id (str, optional) – The device that generated this frame. Defaults to None.
update_type (Literal["ADD", "MODIFY", "DELETE"], optional) – For streaming datasets, the update type for the frame. Defaults to “ADD”. Frames that are ADDed completely overwrite any previous versions, while frames that are MODIFYed are applied on top of an existing frame. If you only want to partially update a frame (change metadata, add a new label, etc), use “MODIFY”

add_audio(*, sensor_id='DEFAULT', audio_url, date_captured=None, mirror_asset=False)

Add an audio “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
audio_url (str) – The URL to load this audio data (mp3, etc.).
date_captured (str, optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_coordinate_frame_2d(*, coord_frame_id, fx, fy, camera_model=None, position=None, orientation=None, camera_matrix=None, k1=None, k2=None, k3=None, k4=None, k5=None, k6=None, p1=None, p2=None, cx=None, cy=None, s1=None, s2=None, s3=None, s4=None, skew=None, parent_frame_id=None)

Add a 2D Coordinate Frame.

Parameters

coord_frame_id (str) – String identifier for this coordinate frame.
fx (Union[int, float]) – focal length x in pixels.
fy (Union[int, float]) – focal length y in pixels.
camera_model (Optional[CAMERA_MODELS], optional) – Either “fisheye” for the fisheye model, or “brown_conrady” for the pinhole model with Brown-Conrady distortion. Defaults to “brown_conrady”.
position (Optional[Dict[POSITION, Union[int, float]]], optional) – Dict of the form {x, y, z}. Defaults to {x: 0, y: 0, z: 0}.
orientation (Optional[Dict[ORIENTATION, Union[int, float]]], optional) – Quaternion rotation dict of the form {w, x, y, z}. Defaults to {x: 0, y: 0, z: 0, w: 1}.
camera_matrix (Optional[List[List[Union[int, float]]]], optional) – 4x4 row major order camera matrix mapping 3d world space to camera space (x right, y down, z forward). Keep in mind, if you pass in the camera matrix it will stack on top of the position/orientation you pass in as well. This is only needed if you cannot properly represent your camera using the position/orientation parameters. Defaults to None.
cx (Optional[Union[int, float]], optional) – optical center pixel x coordinate. Defaults to x center of image.
cy (Optional[Union[int, float]], optional) – optical center pixel y coordinate. Defaults to y center of image.
k1 (Optional[Union[int, float]], optional) – k1 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k2 (Optional[Union[int, float]], optional) – k2 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k3 (Optional[Union[int, float]], optional) – k3 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k4 (Optional[Union[int, float]], optional) – k4 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k5 (Optional[Union[int, float]], optional) – k5 radial distortion coefficient (Brown-Conrady). Defaults to 0.
k6 (Optional[Union[int, float]], optional) – k6 radial distortion coefficient (Brown-Conrady). Defaults to 0.
p1 (Optional[Union[int, float]], optional) – p1 tangential distortion coefficient (Brown-Conrady). Defaults to 0.
p2 (Optional[Union[int, float]], optional) – p2 tangential distortion coefficient (Brown-Conrady). Defaults to 0.
s1 (Optional[Union[int, float]], optional) – s1 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
s2 (Optional[Union[int, float]], optional) – s2 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
s3 (Optional[Union[int, float]], optional) – s3 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
s4 (Optional[Union[int, float]], optional) – s4 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
skew (Optional[Union[int, float]], optional) – camera skew coefficient (fisheye). Defaults to 0.
parent_frame_id (Optional[str], optional) – String id of the parent coordinate frame. Defaults to None.

Return type

None

add_coordinate_frame_3d(*, coord_frame_id, position=None, orientation=None, parent_frame_id=None)

Add a 3D Coordinate Frame.

Parameters

coord_frame_id (str) – String identifier for this coordinate frame
position (Optional[Dict[POSITION, Union[int, float]]], optional) – Dict of the form {x, y, z}. Defaults to None.
orientation (Optional[Dict[ORIENTATION, Union[int, float]]], optional) – Quaternion rotation dict of the form {w, x, y, z}. Defaults to None.
parent_frame_id (Optional[str], optional) – String id of the parent coordinate frame. Defaults to None.

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per label crop embedding

Parameters

label_id (str) – [description]
embedding (List[float]) – A vector of floats of at least length 2.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_embedding(*, embedding, crop_embeddings=None, model_id='')

DEPRECATED! PLEASE USE add_frame_embedding and add_crop_embedding Add an embedding to this frame, and optionally to crops/labels within it.

If provided, “crop_embeddings” is a list of dicts of the form:: ‘uuid’: the label id for the crop/label ‘embedding’: a vector of floats of at least length 2.

Parameters

embedding (list of floats) – A vector of floats of at least length 2.
crop_embeddings (list of dicts, optional) – A list of dictionaries representing crop embeddings. Defaults to None.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_frame_embedding(*, embedding, model_id='')

Add an embedding to this frame

Parameters

embedding (List[float]) – A vector of floats between length 0 and 12,000.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_geo_latlong_data(lat, lon)

Add a user provided EPSG:4326 WGS84 lat long pair to each frame

We expect these values to be floats

Parameters

lat (float) – lattitude of Geo Location
lon (float) – longitude of Geo Location

Return type

None

add_image(*, sensor_id='DEFAULT', image_url, preview_url=None, date_captured=None, width=None, height=None, coord_frame_id=None, mirror_asset=False)

Add an image “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
image_url (str) – The URL to load this image data.
preview_url (Optional[str], optional) – A URL to a compressed version of the image. It must be the same pixel dimensions as the original image. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
width (Optional[int], optional) – The width of the image in pixels. Defaults to None.
height (Optional[int], optional) – The height of the image in pixels. Defaults to None.
coord_frame_id (Optional[str], optional) – Id for the Coordinate Frame (only accepts coordinate frames of type “IMAGE” or “IMAGE_PROJECTION”). Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_image_overlay(*, sensor_id='DEFAULT', overlay_id, image_url, date_captured=None, width=None, height=None, mirror_asset=False, resize_mode=None)

Add an image overlay for the given “sensor” to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
overlay_id (str) – The id of this overlay.
image_url (str) – The URL to load this image data.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
width (Optional[int], optional) – The width of the image in pixels. Defaults to None.
height (Optional[int], optional) – The height of the image in pixels. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.
resize_mode (ResizeMode, optional) – if the overlay is a different size from the base image, define how to display it “fill”: stretch the overlay to fit the base image dimensions None: no change

Return type

None

add_label_2d_bbox(*, sensor_id='DEFAULT', label_id, classification, top, left, width, height, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D bounding box.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.

Return type

None

add_label_2d_classification(*, sensor_id='DEFAULT', label_id, classification, confidence=None, secondary_labels=None, user_attrs=None)

Add a label for 2D classification.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
secondary_labels (dict, optional) – dictionary of secondary labels
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_instance_seg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, instance_mapping, resize_mode=None)

Add a label for 2D instance segmentation.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
instance_mapping (List[Dict]) –
a list of instances present in the mask, each is a numeric id, string classification, and optional dict of additional attributes. As an example of one instance: .. code-block:
```
{
    'id': 1,
    'classification': "Person",
    'attributes': {
        'is_standing': false,
    }
}
```
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_label_2d_keypoints(*, sensor_id='DEFAULT', label_id, classification, keypoints, confidence=None, top=None, left=None, width=None, height=None, polygons=None, center=None, user_attrs=None)

Add a label for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
keypoints (list of dicts) – The keypoints of this detection
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
top (int or float, optional) – The top of the bounding box in pixels. Defaults to None.
left (int or float, optional) – The left of the bounding box in pixels. Defaults to None.
width (int or float, optional) – The width of the bounding box in pixels. Defaults to None.
height (int or float, optional) – The height of the bounding box in pixels. Defaults to None.
polygons (list of dicts, optional) – The polygon geometry. Defaults to None.
center (list of ints or floats, optional) – The center point of the polygon instance. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_line_segment(*, sensor_id='DEFAULT', label_id, classification, x1, y1, x2, y2, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D line segment.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_label_2d_polygon_list(*, sensor_id='DEFAULT', label_id, classification, polygons, confidence=None, center=None, user_attrs=None)

Add a label for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_semseg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, resize_mode=None)

Add a label for 2D semseg.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_label_3d_classification(*, label_id, classification, confidence=None, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
coord_frame_id (str, optional) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add a label for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_label_text_token(*, sensor_id='DEFAULT', label_id, index, token, classification, visible, confidence=None, user_attrs=None)

Add a label for a text token.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_obj(*, sensor_id, obj_url, coord_frame_id=None, date_captured=None, mirror_asset=False)

Add a .obj file to the frame for text based geometry

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
obj_url (str) – URL to where the object is located
coord_frame_id (Optional[str], optional) – ID for the coordinate frame. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_point_cloud_bins(*, sensor_id, pointcloud_url=None, kitti_format_url=None, intensity_url=None, range_url=None, coord_frame_id=None, date_captured=None, mirror_asset=False)

Add a point cloud sensor data point to this frame, contained in dense binary files of little-endian values, similar to the raw format of KITTI lidar data. You can provide a combination of the following values, as long as at least either kitti_format_url or pointcloud_url are provided.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
pointcloud_url (Optional[str]) – URL for the pointcloud: float32 [x1, y1, z1, x2, y2, z2, …]
kitti_format_url (Optional[str]) – URL for the pointcloud + intensity: float32 [x1, y1, z1, i1, x2, y2, z2, i2, …]
intensity_url (Optional[str]) – URL for the Intensity Pointcloud: unsigned int32 [i1, i2, …]
range_url (Optional[str]) – URL for the Range Pointcloud: float32 [r1, r2, …]
coord_frame_id (Optional[str], optional) – Id for the Coordinate Frame. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_point_cloud_pcd(*, sensor_id, pcd_url, coord_frame_id=None, date_captured=None, mirror_asset=False)

Add a point cloud sensor data point to this frame, contained in PCD format. ascii, binary, and binary_compressed formats are supported. Numeric values for the following column names are expected: x, y, z, intensity (optional), range (optional)

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
pcd_url (str) – URL to PCD formated data
coord_frame_id (Optional[str], optional) – The coordinate frame id. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed url assets.

Return type

None

add_text(*, sensor_id='DEFAULT', text, date_captured=None)

Add a text “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
text (str) – The text body.
date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_user_metadata(key, val, val_type=None)

Add a user provided metadata field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

key (str) – The key for your metadata field
val (Union[str, int, float, bool]) – value
val_type (Literal["str", "int", "float", "bool"], optional) – type of val as string. Defaults to None.

Return type

None

add_user_metadata_list(key, val, list_elt_type=None)

Add a user provided metadata list field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

key (str) – The key for your metadata field
val (Union[List[int], List[str], List[bool], List[float], Tuple[int], Tuple[str], Tuple[bool], Tuple[float]]) – value
list_elt_type (Literal["str", "int", "float", "bool"], optional) – type of list elements as string. Applies to all members of val. Defaults to None.

Return type

None

add_video(*, sensor_id='DEFAULT', video_url, date_captured=None, mirror_asset=False)

Add a video “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
video_url (str) – The URL to load this video data (mp4, webm, etc.).
date_captured (str, optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

to_dict()

Convert this frame into a dictionary representation.

Returns: dictified frame
Return type: dict

validate_and_backfill_crop_embeddings()

Determines if there are frame embeddings but not Crop embeddings Backfills crop embeddings if Frame embeddings exist Returns True if backfill was necessary

Returns: returns if backfill on frame was necessary
Return type: bool

class aquariumlearning.MetricsManager(client, project_id)

A manager for interacting with metrics within a given project.

Contains the following constants:

MetricsManager.ORDER_CONF_DESC
MetricsManager.ORDER_CONF_ASC
MetricsManager.ORDER_IOU_DESC
MetricsManager.ORDER_IOU_ASC
MetricsManager.ORDER_BOX_SIZE_DESC
MetricsManager.ORDER_BOX_SIZE_ASC

Parameters

client (Client) – An Aquarium Learning Python Client object.
project_id (str) – The project id associated with this manager.

fetch_confusions(dataset_id, inferences_id, opts)

Fetch confusions for a given dataset + inference set pair.

The options define the parameters of the query. The options are a dictionary of the form:

{
    'confidence_threshold': Union[float, int],
    'iou_threshold': Union[float, int],
    'queries': List[QueryEntry],
    'ordering': QueryOrdering,
    'max_results': int (optional, defaults to 10,000),
    'cur_offset': int (optional, defaults to 0),
}

Confidence and iou thresholds can be any multiple of 0.1 between 0.0 and 1.0. Queries are a list of queries produced from helper methods such as

metrics_manager.make_confusions_query()

Ordering is defined by constants on the class object, such as:

metrics_manager.ORDER_CONF_DESC

Parameters

dataset_id (str) – The dataset id.
inferences_id (str) – The inference set id.
opts (ConfusionsOpts) – Options for the query.

Returns

All of your query results, up to max_results (default = 10k)

Return type

ConfusionsResultDict

get_metrics(dataset_id, inferences_id, confidence_pct, iou_pct, scenario_ids=[], metrics_class=None)

Get metrics for an inference set, optionally across scenarios.

Parameters

dataset_id (str) – The dataset id.
inferences_id (str) – The inference set id.
confidence_pct (float or int) – The confidence threshold provided as a percent (ie 10 instead of 0.10).
iou_pct (float or int) – The IOU threshold provided as a percent (ie 10 instead of 0.10).
scenario_ids (None or List[str]) – Scenario names to fetch metrics for. If None, returns metrics for all scenarios for the dataset. Defaults to [], which only returns metrics for the complete inference set.
metrics_class (None or str) – Class name to fetch metrics for. Defaults to None which returns weighted avg.

Returns

Result metrics.

Return type

List[ScenarioMetricResultDict]

list_scenarios(dataset_id)

List all scenarios for a dataset.

Parameters: dataset_id (str) – The dataset id.
Returns: All scenarios for the dataset.
Return type: List[ScenarioInfo]

make_cell_query(gt, inf)

Make a query entry for a specific confusion of gt as as inf.

Parameters

gt (str) – The classname of the ground truth class.
inf (str) – The classname of the inference class.

Returns

Result query entry.

Return type

QueryEntry

make_confused_as_query(name)

Make a query entry for all confusions as name. (inf == name, gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns: Result query entry.
Return type: QueryEntry
Parameters: name (str) –

make_confused_from_query(name)

Make a query entry for all confusions from name. (gt == name, gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns: Result query entry.
Return type: QueryEntry
Parameters: name (str) –

make_confusions_query()

Make a query entry for all confusions (gt != inf)

This will only include cases where two matched detections exist, not for false positive or false negative detections.

Returns: Result query entry.
Return type: QueryEntry

make_correct_query()

Make a query entry for all correct detections/classifications (gt == inf)

Returns: Result query entry.
Return type: QueryEntry

make_false_negatives_query()

Make a query entry for all false negative detections.

These are cases without corresponding inference detections for a ground truth detection.

Returns: Result query entry.
Return type: QueryEntry

make_false_positives_query()

Make a query entry for all false positive detections.

These are cases without corresponding ground truth detections for an inference detection.

Returns: Result query entry.
Return type: QueryEntry

class aquariumlearning.StratifiedMetricsDefinition(name, ordered_values)

Definitions for stratified metrics given object-level attributes

Parameters

name (str) – The name of this attribute, which should match the attribute on object labels.
ordered_values (List[str]) – The ordered list of valid values to group by.

class aquariumlearning.UnlabeledDataset(*, pipeline_mode='STREAMING')

A container used to construct an unlabeled dataset.

Typical usage is to create an UnlabeledDataset, add multiple UnlabeledFrames to it, then serialize the frames to be submitted.

Parameters: pipeline_mode (str) –

add_frame(frame)

Add a LabeledFrame to this dataset.

Parameters: frame (LabeledFrame) – A LabeledFrame in this dataset.
Return type: None

delete_frame(frame_id)

Delete a frame from this dataset.

Parameters: frame_id (str) – A frame_id in the dataset.
Return type: None

write_crop_embeddings_to_file_streaming(filelike)

Write the frame’s crops’ embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_embeddings_to_file(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_frame_embeddings_to_file_streaming(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_frames_to_file_streaming(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_label_assets_to_file(filelike)

Write the frame’s label assets to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_labels_to_file_streaming(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

write_to_file(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters: filelike (filelike) – The destination file-like to write to.
Return type: None

class aquariumlearning.UnlabeledFrame(*, frame_id, date_captured=None, device_id=None, reuse_latest_embedding=False, update_type='ADD')

A frame for a dataset which is labeled with inferences rather than GT labels. Meant to be used with UnlabeledDataset.

Parameters

frame_id (str) – A unique id for this frame.
date_captured (str, optional) – ISO formatted datetime string. Defaults to None.
device_id (str, optional) – The device that generated this frame. Defaults to None.
reuse_latest_embedding (Optional[bool]) –
update_type (Literal['ADD', 'MODIFY', 'DELETE']) –

add_audio(*, sensor_id='DEFAULT', audio_url, date_captured=None, mirror_asset=False)

Add an audio “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
audio_url (str) – The URL to load this audio data (mp3, etc.).
date_captured (str, optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_coordinate_frame_2d(*, coord_frame_id, fx, fy, camera_model=None, position=None, orientation=None, camera_matrix=None, k1=None, k2=None, k3=None, k4=None, k5=None, k6=None, p1=None, p2=None, cx=None, cy=None, s1=None, s2=None, s3=None, s4=None, skew=None, parent_frame_id=None)

Add a 2D Coordinate Frame.

Parameters

coord_frame_id (str) – String identifier for this coordinate frame.
fx (Union[int, float]) – focal length x in pixels.
fy (Union[int, float]) – focal length y in pixels.
camera_model (Optional[CAMERA_MODELS], optional) – Either “fisheye” for the fisheye model, or “brown_conrady” for the pinhole model with Brown-Conrady distortion. Defaults to “brown_conrady”.
position (Optional[Dict[POSITION, Union[int, float]]], optional) – Dict of the form {x, y, z}. Defaults to {x: 0, y: 0, z: 0}.
orientation (Optional[Dict[ORIENTATION, Union[int, float]]], optional) – Quaternion rotation dict of the form {w, x, y, z}. Defaults to {x: 0, y: 0, z: 0, w: 1}.
camera_matrix (Optional[List[List[Union[int, float]]]], optional) – 4x4 row major order camera matrix mapping 3d world space to camera space (x right, y down, z forward). Keep in mind, if you pass in the camera matrix it will stack on top of the position/orientation you pass in as well. This is only needed if you cannot properly represent your camera using the position/orientation parameters. Defaults to None.
cx (Optional[Union[int, float]], optional) – optical center pixel x coordinate. Defaults to x center of image.
cy (Optional[Union[int, float]], optional) – optical center pixel y coordinate. Defaults to y center of image.
k1 (Optional[Union[int, float]], optional) – k1 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k2 (Optional[Union[int, float]], optional) – k2 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k3 (Optional[Union[int, float]], optional) – k3 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k4 (Optional[Union[int, float]], optional) – k4 radial distortion coefficient (Brown-Conrady, fisheye). Defaults to 0.
k5 (Optional[Union[int, float]], optional) – k5 radial distortion coefficient (Brown-Conrady). Defaults to 0.
k6 (Optional[Union[int, float]], optional) – k6 radial distortion coefficient (Brown-Conrady). Defaults to 0.
p1 (Optional[Union[int, float]], optional) – p1 tangential distortion coefficient (Brown-Conrady). Defaults to 0.
p2 (Optional[Union[int, float]], optional) – p2 tangential distortion coefficient (Brown-Conrady). Defaults to 0.
s1 (Optional[Union[int, float]], optional) – s1 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
s2 (Optional[Union[int, float]], optional) – s2 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
s3 (Optional[Union[int, float]], optional) – s3 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
s4 (Optional[Union[int, float]], optional) – s4 thin prism distortion coefficient (Brown-Conrady). Defaults to 0.
skew (Optional[Union[int, float]], optional) – camera skew coefficient (fisheye). Defaults to 0.
parent_frame_id (Optional[str], optional) – String id of the parent coordinate frame. Defaults to None.

Return type

None

add_coordinate_frame_3d(*, coord_frame_id, position=None, orientation=None, parent_frame_id=None)

Add a 3D Coordinate Frame.

Parameters

coord_frame_id (str) – String identifier for this coordinate frame
position (Optional[Dict[POSITION, Union[int, float]]], optional) – Dict of the form {x, y, z}. Defaults to None.
orientation (Optional[Dict[ORIENTATION, Union[int, float]]], optional) – Quaternion rotation dict of the form {w, x, y, z}. Defaults to None.
parent_frame_id (Optional[str], optional) – String id of the parent coordinate frame. Defaults to None.

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per label crop embedding

Parameters

label_id (str) – [description]
embedding (List[float]) – A vector of floats of at least length 2.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_embedding(*, embedding, crop_embeddings=None, model_id='')

DEPRECATED! PLEASE USE add_frame_embedding and add_crop_embedding Add an embedding to this frame, and optionally to crops/labels within it.

If provided, “crop_embeddings” is a list of dicts of the form:: ‘uuid’: the label id for the crop/label ‘embedding’: a vector of floats of at least length 2.

Parameters

embedding (list of floats) – A vector of floats of at least length 2.
crop_embeddings (list of dicts, optional) – A list of dictionaries representing crop embeddings. Defaults to None.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_frame_embedding(*, embedding, model_id='')

Add an embedding to this frame

Parameters

embedding (List[float]) – A vector of floats between length 0 and 12,000.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_geo_latlong_data(lat, lon)

Add a user provided EPSG:4326 WGS84 lat long pair to each frame

We expect these values to be floats

Parameters

lat (float) – lattitude of Geo Location
lon (float) – longitude of Geo Location

Return type

None

add_image(*, sensor_id='DEFAULT', image_url, preview_url=None, date_captured=None, width=None, height=None, coord_frame_id=None, mirror_asset=False)

Add an image “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
image_url (str) – The URL to load this image data.
preview_url (Optional[str], optional) – A URL to a compressed version of the image. It must be the same pixel dimensions as the original image. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
width (Optional[int], optional) – The width of the image in pixels. Defaults to None.
height (Optional[int], optional) – The height of the image in pixels. Defaults to None.
coord_frame_id (Optional[str], optional) – Id for the Coordinate Frame (only accepts coordinate frames of type “IMAGE” or “IMAGE_PROJECTION”). Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_image_overlay(*, sensor_id='DEFAULT', overlay_id, image_url, date_captured=None, width=None, height=None, mirror_asset=False, resize_mode=None)

Add an image overlay for the given “sensor” to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
overlay_id (str) – The id of this overlay.
image_url (str) – The URL to load this image data.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
width (Optional[int], optional) – The width of the image in pixels. Defaults to None.
height (Optional[int], optional) – The height of the image in pixels. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.
resize_mode (ResizeMode, optional) – if the overlay is a different size from the base image, define how to display it “fill”: stretch the overlay to fit the base image dimensions None: no change

Return type

None

add_label_2d_bbox(*, sensor_id='DEFAULT', label_id, classification, top, left, width, height, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D bounding box.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.

Return type

None

add_label_2d_classification(*, sensor_id='DEFAULT', label_id, classification, confidence=None, secondary_labels=None, user_attrs=None)

Add a label for 2D classification.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
secondary_labels (dict, optional) – dictionary of secondary labels
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_instance_seg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, instance_mapping, resize_mode=None)

Add a label for 2D instance segmentation.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
instance_mapping (List[Dict]) –
a list of instances present in the mask, each is a numeric id, string classification, and optional dict of additional attributes. As an example of one instance: .. code-block:
```
{
    'id': 1,
    'classification': "Person",
    'attributes': {
        'is_standing': false,
    }
}
```
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_label_2d_keypoints(*, sensor_id='DEFAULT', label_id, classification, keypoints, confidence=None, top=None, left=None, width=None, height=None, polygons=None, center=None, user_attrs=None)

Add a label for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
keypoints (list of dicts) – The keypoints of this detection
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
top (int or float, optional) – The top of the bounding box in pixels. Defaults to None.
left (int or float, optional) – The left of the bounding box in pixels. Defaults to None.
width (int or float, optional) – The width of the bounding box in pixels. Defaults to None.
height (int or float, optional) – The height of the bounding box in pixels. Defaults to None.
polygons (list of dicts, optional) – The polygon geometry. Defaults to None.
center (list of ints or floats, optional) – The center point of the polygon instance. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_line_segment(*, sensor_id='DEFAULT', label_id, classification, x1, y1, x2, y2, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D line segment.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_label_2d_polygon_list(*, sensor_id='DEFAULT', label_id, classification, polygons, confidence=None, center=None, user_attrs=None)

Add a label for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_semseg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, resize_mode=None)

Add a label for 2D semseg.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_label_3d_classification(*, label_id, classification, confidence=None, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
coord_frame_id (str, optional) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add a label for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_label_text_token(*, sensor_id='DEFAULT', label_id, index, token, classification, visible, confidence=None, user_attrs=None)

Add a label for a text token.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_obj(*, sensor_id, obj_url, coord_frame_id=None, date_captured=None, mirror_asset=False)

Add a .obj file to the frame for text based geometry

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
obj_url (str) – URL to where the object is located
coord_frame_id (Optional[str], optional) – ID for the coordinate frame. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_point_cloud_bins(*, sensor_id, pointcloud_url=None, kitti_format_url=None, intensity_url=None, range_url=None, coord_frame_id=None, date_captured=None, mirror_asset=False)

Add a point cloud sensor data point to this frame, contained in dense binary files of little-endian values, similar to the raw format of KITTI lidar data. You can provide a combination of the following values, as long as at least either kitti_format_url or pointcloud_url are provided.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
pointcloud_url (Optional[str]) – URL for the pointcloud: float32 [x1, y1, z1, x2, y2, z2, …]
kitti_format_url (Optional[str]) – URL for the pointcloud + intensity: float32 [x1, y1, z1, i1, x2, y2, z2, i2, …]
intensity_url (Optional[str]) – URL for the Intensity Pointcloud: unsigned int32 [i1, i2, …]
range_url (Optional[str]) – URL for the Range Pointcloud: float32 [r1, r2, …]
coord_frame_id (Optional[str], optional) – Id for the Coordinate Frame. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

add_point_cloud_pcd(*, sensor_id, pcd_url, coord_frame_id=None, date_captured=None, mirror_asset=False)

Add a point cloud sensor data point to this frame, contained in PCD format. ascii, binary, and binary_compressed formats are supported. Numeric values for the following column names are expected: x, y, z, intensity (optional), range (optional)

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
pcd_url (str) – URL to PCD formated data
coord_frame_id (Optional[str], optional) – The coordinate frame id. Defaults to None.
date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed url assets.

Return type

None

add_text(*, sensor_id='DEFAULT', text, date_captured=None)

Add a text “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
text (str) – The text body.
date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_user_metadata(key, val, val_type=None)

Add a user provided metadata field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

key (str) – The key for your metadata field
val (Union[str, int, float, bool]) – value
val_type (Literal["str", "int", "float", "bool"], optional) – type of val as string. Defaults to None.

Return type

None

add_user_metadata_list(key, val, list_elt_type=None)

Add a user provided metadata list field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

key (str) – The key for your metadata field
val (Union[List[int], List[str], List[bool], List[float], Tuple[int], Tuple[str], Tuple[bool], Tuple[float]]) – value
list_elt_type (Literal["str", "int", "float", "bool"], optional) – type of list elements as string. Applies to all members of val. Defaults to None.

Return type

None

add_video(*, sensor_id='DEFAULT', video_url, date_captured=None, mirror_asset=False)

Add a video “sensor” data to this frame.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint (optional if there is only 1 datapoint per frame)
video_url (str) – The URL to load this video data (mp4, webm, etc.).
date_captured (str, optional) – ISO formatted date. Defaults to None.
mirror_asset (Optional[str]), optional) – request this asset in this frame to be saved to Aquarium Storage. Useful for Short duration signed URL assets.

Return type

None

to_dict()

Convert this frame into a dictionary representation.

Returns: dictified frame
Return type: dict

validate_and_backfill_crop_embeddings()

Determines if there are frame embeddings but not Crop embeddings Backfills crop embeddings if Frame embeddings exist Returns True if backfill was necessary

Returns: returns if backfill on frame was necessary
Return type: bool

class aquariumlearning.UpdateGTLabelSet(*, frame_id, is_snapshot=True)

Parameters

frame_id (str) –
is_snapshot (bool) –

add_2d_bbox(*, sensor_id='DEFAULT', label_id, classification, top, left, width, height, confidence=None, area=None, iscrowd=None, user_attrs=None, links=None)

Add a 2D bounding box.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
area (float, optional) – The area of the image.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_classification(*, sensor_id='DEFAULT', label_id, classification, confidence=None, user_attrs=None, secondary_labels=None)

Add an inference for 2D classification.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
secondary_labels (Optional[Dict[str, Any]]) –

Return type

None

add_2d_instance_seg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, instance_mapping, resize_mode=None)

Add an label for 2D instance segmentation. These should provide either a mask_url or a mask_data, and a instance mapping.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
instance_mapping (List[Dict]) –
a list of instances present in the mask, each is a numeric id, string classification, and optional dict of additional attributes. As an example of one instance: .. code-block:
```
{
    'id': 1,
    'classification': "Person",
    'attributes': {
        'is_standing': false,
    }
}
```
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_2d_keypoints(*, sensor_id='DEFAULT', label_id, classification, keypoints, confidence=None, top=None, left=None, width=None, height=None, polygons=None, center=None, user_attrs=None)

Add an inference for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
keypoints (list of dicts) – The keypoints of this detection
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
top (int or float, optional) – The top of the bounding box in pixels. Defaults to None.
left (int or float, optional) – The left of the bounding box in pixels. Defaults to None.
width (int or float, optional) – The width of the bounding box in pixels. Defaults to None.
height (int or float, optional) – The height of the bounding box in pixels. Defaults to None.
polygons (list of dicts, optional) – The polygon geometry. Defaults to None.
center (list of ints or floats, optional) – The center point of the polygon instance. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_line_segment(*, sensor_id='DEFAULT', label_id, classification, x1, y1, x2, y2, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a 2D line segment.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_2d_polygon_list(*, sensor_id='DEFAULT', label_id, classification, polygons, confidence=None, center=None, user_attrs=None)

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_2d_semseg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, resize_mode=None)

Add an inference for 2D semseg. These should provide either a mask_url or a mask_data.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_3d_classification(*, label_id, classification, confidence=None, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float) – The confidence between 0.0 and 1.0 of the prediction
coord_frame_id (optional, str) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add an inference for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float) – confidence of prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per inference crop embedding

Parameters

label_id (str) – [description]
embedding (List[float]) – A vector of floats of at least length 2.
model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_label_2d_bbox(*, sensor_id='DEFAULT', label_id, classification, top, left, width, height, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D bounding box.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
top (int or float) – The top of the box in pixels
left (int or float) – The left of the box in pixels
width (int or float) – The width of the box in pixels
height (int or float) – The height of the box in pixels
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.

Return type

None

add_label_2d_classification(*, sensor_id='DEFAULT', label_id, classification, confidence=None, secondary_labels=None, user_attrs=None)

Add a label for 2D classification.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
secondary_labels (dict, optional) – dictionary of secondary labels
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_instance_seg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, instance_mapping, resize_mode=None)

Add a label for 2D instance segmentation.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
instance_mapping (List[Dict]) –
a list of instances present in the mask, each is a numeric id, string classification, and optional dict of additional attributes. As an example of one instance: .. code-block:
```
{
    'id': 1,
    'classification': "Person",
    'attributes': {
        'is_standing': false,
    }
}
```
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_label_2d_keypoints(*, sensor_id='DEFAULT', label_id, classification, keypoints, confidence=None, top=None, left=None, width=None, height=None, polygons=None, center=None, user_attrs=None)

Add a label for a 2D keypoints task.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
keypoints (list of dicts) – The keypoints of this detection
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
top (int or float, optional) – The top of the bounding box in pixels. Defaults to None.
left (int or float, optional) – The left of the bounding box in pixels. Defaults to None.
width (int or float, optional) – The width of the bounding box in pixels. Defaults to None.
height (int or float, optional) – The height of the bounding box in pixels. Defaults to None.
polygons (list of dicts, optional) – The polygon geometry. Defaults to None.
center (list of ints or floats, optional) – The center point of the polygon instance. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_line_segment(*, sensor_id='DEFAULT', label_id, classification, x1, y1, x2, y2, confidence=None, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D line segment.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
x1 (int or float) – The x-coord of the first vertex in pixels
y1 (int or float) – The x-coord of the first vertex in pixels
x2 (int or float) – The x-coord of the first vertex in pixels
y2 (int or float) – The x-coord of the first vertex in pixels
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

add_label_2d_polygon_list(*, sensor_id='DEFAULT', label_id, classification, polygons, confidence=None, center=None, user_attrs=None)

Add a label for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
polygons (list of dicts) – The polygon geometry
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_semseg(*, sensor_id='DEFAULT', label_id, mask_url=None, mask_data=None, resize_mode=None)

Add a label for 2D semseg.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

add_label_3d_classification(*, label_id, classification, confidence=None, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
coord_frame_id (str, optional) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add a label for a 3D cuboid.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str) – the classification string
position (list of float) – the position of the center of the cuboid
dimensions (list of float) – the dimensions of the cuboid
rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_label_text_token(*, sensor_id='DEFAULT', label_id, index, token, classification, visible, confidence=None, user_attrs=None)

Add a label for a text token.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_text_token(*, sensor_id='DEFAULT', label_id, index, token, classification, visible, confidence=None, user_attrs=None)

Add a label for a text token.

Parameters

sensor_id (str) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
label_id (str) – label_id which is unique across datasets and inferences.
index (int) – the index of this token in the text
token (str) – the text content of this token
classification (str) – the classification string
visible (bool) – is this a visible token in the text
confidence (float) – confidence of prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

to_dict()

Convert this label set into a dictionary representation.

Returns: dictified frame
Return type: dict

update_label_2d_bbox(*, label_id, sensor_id=None, classification=None, top=None, left=None, width=None, height=None, confidence=None, area=None, iscrowd=None, user_attrs=None, links=None)

Update an existing 2D bounding box. If the label_id does not already exist on the frame, it will be dropped.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
classification (str, optional) – the classification string
top (int or float, optional) – The top of the box in pixels
left (int or float, optional) – The left of the box in pixels
width (int or float, optional) – The width of the box in pixels
height (int or float, optional) – The height of the box in pixels
confidence (float, optional) – The confidence between 0.0 and 1.0 of the prediction
area (float, optional) – The area of the image.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

update_label_2d_classification(*, label_id, sensor_id=None, classification, confidence=None, user_attrs=None, secondary_labels=None)

Update an existing label for 2D classification.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
classification (str, optional) – the classification string
confidence (float, optional) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
secondary_labels (Optional[Dict[str, Any]]) –

Return type

None

update_label_2d_instance_seg(*, label_id, sensor_id=None, mask_url=None, mask_data=None, instance_mapping=None, resize_mode=None)

Update a label for 2D instance segmentation. These should provide either a mask_url or a mask_data, and a instance mapping. If the label_id does not already exist on the frame, it will be dropped.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
instance_mapping (List[Dict]) –
a list of instances present in the mask, each is a numeric id, string classification, and optional dict of additional attributes. As an example of one instance: .. code-block:
```
{
    'id': 1,
    'classification': "Person",
    'attributes': {
        'is_standing': false,
    }
}
```
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

update_label_2d_keypoints(*, label_id, sensor_id=None, classification=None, keypoints=None, confidence=None, top=None, left=None, width=None, height=None, polygons=None, center=None, user_attrs=None)

Partially update an inference for a 2D keypoints task. If the label_id does not already exist on the frame, it will be dropped.

A keypoint is a dictionary of the form:: ‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
classification (str, optional) – the classification string
top (int or float, optional) – The top of the box in pixels
left (int or float, optional) – The left of the box in pixels
width (int or float, optional) – The width of the box in pixels
height (int or float, optional) – The height of the box in pixels
keypoints (list of dicts, optional) – The keypoints of this detection
confidence (float, optional) – The confidence between 0.0 and 1.0 of the prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
polygons (Optional[List[Dict[Literal['vertices'], ~typing.List[~typing.Tuple[~typing.Union[int, float]]]]]]) –
center (Optional[List[Union[int, float]]]) –

Return type

None

update_label_2d_line_segment(*, label_id, sensor_id=None, classification=None, x1=None, y1=None, x2=None, y2=None, confidence=None, iscrowd=None, user_attrs=None, links=None)

Partially update an existing 2D line segment. If the label_id does not already exist on the frame, it will be dropped.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
classification (str, optional) – the classification string
x1 (int or float, optional) – The x-coord of the first vertex in pixels
y1 (int or float, optional) – The x-coord of the first vertex in pixels
x2 (int or float, optional) – The x-coord of the first vertex in pixels
y2 (int or float, optional) – The x-coord of the first vertex in pixels
confidence (float, optional) – The confidence between 0.0 and 1.0 of the prediction
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (Optional[Dict[str, Any]]) –

Return type

None

update_label_2d_polygon_list(*, label_id, sensor_id=None, classification=None, polygons=None, confidence=None, center=None, user_attrs=None)

Partially update an label for a 2D polygon list instance segmentation task. If the label_id does not already exist on the frame, it will be dropped.

Polygons are dictionaries of the form:

‘vertices’: List of (x, y) vertices (e.g. [[x1,y1], [x2,y2], …])

The polygon does not need to be closed with (x1, y1). As an example, a bounding box in polygon representation would look like:

{
    'vertices': [
        [left, top],
        [left + width, top],
        [left + width, top + height],
        [left, top + height]
    ]
}

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
classification (str, optional) – the classification string
polygons (list of dicts, optional) – The polygon geometry
confidence (float, optional) – The confidence between 0.0 and 1.0 of the prediction
center (list of ints or floats, optional) – The center point of the instance
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

update_label_2d_semseg(*, label_id, sensor_id=None, mask_url=None, mask_data=None, resize_mode=None)

Update an label for 2D semseg. These should provide either a mask_url or a mask_data. If the label_id does not already exist on the frame, it will be dropped.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
mask_url (str, optional) – URL to the pixel mask png.
mask_data (ndarray, optional) – ndarray of pixel mask data, shape [height, width]
resize_mode (ResizeMode, optional) – if the mask is a different size from the base image, define how to display it “fill”: stretch the mask to fit the base image dimensions None: no change

Return type

None

update_label_3d_classification(*, label_id, classification=None, confidence=None, coord_frame_id=None, user_attrs=None)

Update an existing label for 3D classification. If the label_id does not already exist on the frame, it will be dropped.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str, optional) – the classification string
confidence (float, optional) – The confidence between 0.0 and 1.0 of the prediction
coord_frame_id (str, optional) – The coordinate frame id.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

update_label_3d_cuboid(*, label_id, classification=None, position=None, dimensions=None, rotation=None, confidence=None, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Partially update an existing inference for a 3D cuboid. If the label_id does not already exist on the frame, it will be dropped.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
classification (str, optional) – the classification string
position (list of float, optional) – the position of the center of the cuboid
dimensions (list of float, optional) – the dimensions of the cuboid
rotation (list of float, optional) – the local rotation of the cuboid, represented as an xyzw quaternion.
confidence (float, optional) – (Unlabeled Datasets Only) The confidence between 0.0 and 1.0 of the proposed label.
iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.
links (dict, optional) – Links between labels. Defaults to None.
coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

update_label_text_token(*, label_id, sensor_id=None, index=None, token=None, classification=None, visible=None, confidence=None, user_attrs=None)

Partially update an existing label for a text token. If the label_id does not already exist on the frame, it will be dropped.

Parameters

label_id (str) – label_id which is unique across datasets and inferences.
sensor_id (str, optional) – ID of sensor that generates this datapoint. Must match base frame sensor_id. Do not use arg if not used in base frame.
index (int, optional) – the index of this token in the text
token (str, optional) – the text content of this token
classification (str, optional) – the classification string
visible (bool, optional) – is this a visible token in the text
confidence (float, optional) – confidence of prediction
user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

class aquariumlearning.WorkQueue(name, dataset, element_count, element_type, work_queue_id, batch_statuses, batch_assignees, elements=None, created_at=None, updated_at=None, inference_set=None)

Definition for work queue.

Parameters

name (str) – The work queue name.
dataset (Optional[str]) – The dataset for this work queue.
elements (List[WorkQueueElement]) – The elements of the work queue.
element_type (str) – The element type of the work queue (“frame”, “crop”).
element_count (int) – The number of elements in the work queue.
created_at (str) – The time of work queue creation.
updated_at (str) – The time of last work queue update.
work_queue_id (str) – The work queue id.
batch_statuses (Dict[str, str]) – The statuses of all batches in this queue.
batch_assignees (Dict[str, str]) – The emails of users assigned to batches if they have been assigned
inference_set (Optional[str], optional) – The inference set for this work queue. Defaults to None.

class aquariumlearning.WorkQueueElement(element_id, frame_id, element_type, dataset, batch_name, status, assignee=None, inference_set=None, label_metadata=None)

Definition for work queue element.

Parameters

element_id (str) – The element id.
frame_id (str) – The frame id of the element.
element_type (str) – The element type of the issue element (“frame” or “crop”).
dataset (str) – The base dataset an element is from. (Can be formatted as either “project_name.dataset_name” or just “dataset_name”)
batch_name (str) – The batch within the queue that the element belongs to
status (str) – The work status of the element. Can be queued, assigned, or done
assignee (Optional[str]) – (Optional[str]): The email of the person who is assigned to work on the element
inference_set (Optional[str]) – The inference set an element is from (if any). (Can be formatted as either “project_name.inference_set_name” or just “inference_set_name”)
label_metadata (Optional[Dict[str, Any]]) – (For read purposes, not element modification). JSON object with confidence and IOU info if the element was created from a ground truth/inference comparison.

class aquariumlearning.WorkQueueManager(client, project_id)

A work queue manager for interacting with work queues within a given project.

Parameters

client (Client) – An Aquarium Learning Python Client object.
project_id (str) – The project id associated with this manager.

get_work_queue(work_queue_id)

Get a specific work queue in the associated project.

Parameters: work_queue_id (str) – The work queue id.
Returns: The work queue data (including elements).
Return type: WorkQueue

list_work_queues()

List work queues in the associated project.

NOTE: this does NOT include any element data, just the element counts. (Use get_work_queue instead to see that info).

Returns: List of all work queues data.
Return type: List[WorkQueue]