Aquarium Learning Python Client

class aquariumlearning.ClassMapEntry(*, name, class_id, color, train_name=None, train_id=None, category=None, category_id=None, has_instances=True, ignore_in_eval=False, train_color=None)

A description of how to interpret a classification.

In the common case, only three values are needed: - The string representation of the class - The int [0,255] representation of the class - What color to render it as.

In more complex cases, some classes may be ignored in evaluation, or collapsed together into a different class at inference time, or tracked as part of a larger category.

Parameters

  • name (str) – The string representation of the class

  • class_id (int) – The int representation of the class

  • color (List[int]) – A length 3 list/tuple of RGB int values

  • train_name (Optional[str], optional) – The string representation of the class that this label will be inferred as. Defaults to None.

  • train_id (Optional[str], optional) – The int representation of the class that this label will be infererred as. Defaults to None.

  • category (Optional[str], optional) – The string representation of the parent category. Defaults to None.

  • category_id (Optional[int], optional) – The int representation of the parent category. Defaults to None.

  • has_instances (bool, optional) – Whether each label of this class is a separate instance. Defaults to True.

  • ignore_in_eval (bool, optional) – Whether to ignore this class while evaluating metrics. Defaults to False.

  • train_color (Optional[Tuple[int, int, int]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.

to_dict()

Returns a dictified form of this data structure.

Returns

dict represenation of Class Map Entry

Return type

Dict[str, Any]

class aquariumlearning.ClassMapUpdateEntry(*, name=None, class_id=None, color=None, train_color=None)

A description of a change to be made to a classification. Classification can be referred to by name or id

Parameters

  • name (Optional[str], optional) – The string representation of the class

  • class_id (Optional[int], optional) – The int representation of the class

  • color (Optional(List[int]), optional) – A length 3 list/tuple of RGB int values

  • train_color (Optional[Tuple[int, int, int]], optional) – A length 3 list/tuple of RGB int values for showing inferences of this class. Defaults to None.

to_dict()

Returns a dictified form of this data structure.

Returns

dict represenation of Class Map Update Entry

Return type

Dict[str, Any]

class aquariumlearning.Client(*, api_endpoint='https://illume.aquariumlearning.com/api/v1', **kwargs)

Client class that interacts with the Aquarium REST API.

Parameters

api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.

create_dataset(project_id, dataset_id, data_url=None, embeddings_url=None, dataset=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', preview_first_frame=False, delete_cache_files_after_upload=True, check_first_frame=True)

Create a dataset with the provided data urls.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • data_url (Optional[str], optional) – A URL to the serialized dataset entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized dataset embeddings. Defaults to None.

  • dataset (Optional[LabeledDataset], optional) – The LabeledDataset to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • preview_first_frame (bool, optional) – preview the first frame of the dataset in the webapp before continuing. Requires interaction.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

  • check_first_frame (bool) –

Return type

None

create_inferences(project_id, dataset_id, inferences_id, data_url=None, embeddings_url=None, inferences=None, wait_until_finish=False, wait_timeout=datetime.timedelta(seconds=7200), embedding_distance_metric='euclidean', delete_cache_files_after_upload=True)

Create an inference set with the provided data urls.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – A unique identifier for this set of inferences.

  • data_url (Optional[str], optional) – A URL to the serialized inference entries.

  • embeddings_url (Optional[str], optional) – A URL to the serialized inference embeddings. Defaults to None.

  • inferences (Optional[Inferences], optional) – The inferences to upload.

  • wait_until_finish (bool, optional) – Block until the dataset processing job finishes. This generally takes at least 5 minutes, and scales with the size of the dataset. Defaults to False.

  • wait_timeout (datetime.timedelta, optional) – Maximum time to wait for. Defaults to 2 hours.

  • embedding_distance_metric (str, optional) – Distance metric to use for embedding layout. Can be a member of [‘euclidean’, ‘cosine’]. Defaults to ‘euclidean’.

  • delete_cache_files_after_upload (bool, optional) – flag to turn off automatic deletion of cache files after upload. Useful for ipython notebook users that reload/re-attempt uploads. Defaults to True.

Return type

None

create_project(project_id, label_class_map, primary_task=None, secondary_labels=None, frame_links=None, label_links=None, default_camera_target=None, default_camera_position=None, custom_metrics=None, max_shown_categories=None, stratified_metrics=None)

Create a new project via the REST API.

Parameters

  • project_id (str) – project_id

  • label_class_map (LabelClassMap) – The label class map used to interpret classifications.

  • primary_task (Optional[str], optional) – Any specific primary task for a non-object detection or classification task. Can be ‘2D_SEMSEG’ or ‘CLASSIFICATION’ or None.

  • secondary_labels ([type], optional) – List of secondary labels in classification tasks

  • frame_links (Optional[List[str]], optional) – List of string keys for links between frames

  • label_links (Optional[List[str]], optional) – List of string keys for links between labels

  • default_camera_target (Optional[List[float]], optional) – For 3D scenes, the default camera target

  • default_camera_position (Optional[List[float]], optional) – For 3D scenes, the default camera position

  • custom_metrics (Optional[ Union[CustomMetricsDefinition, List[CustomMetricsDefinition]] ], optional) – Defines which custom metrics exist for this project, defaults to None.

  • max_shown_categories (Optional[int], optional) – For categorical visualizations, set the maximum shown simultaneously. Max 100.

  • stratified_metrics (Optional[List[StratifiedMetricsDefinition]], optional) – Defines what object-level attributes to stratify metrics over.

Return type

None

current_abstract_dataset_process_step_status(project_id, dataset_id)

Returns the process steps statuses for a given dataset or inferenceset

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

A set of process step statuses that exist for a given abstract dataset

Return type

Dict[str, Any]

current_dataset_process_state(project_id, dataset_id)

Current processing state of a dataset.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

current_inferences_process_state(project_id, dataset_id, inferences_id)

current processing state of inferences.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

semantic name of state of processing, percent done of job

Return type

Tuple[str, float]

dataset_exists(project_id, dataset_id)

Check if a dataset exists.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

Whether the dataset already exists.

Return type

bool

delete_dataset(project_id, dataset_id)

Mark a dataset for deletion

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Return type

None

delete_project(project_id)

Mark a project for deletion

Parameters

project_id (str) – project_id

Return type

None

get_dataset_ingest_error_logs(project_id, dataset_id)

Get ingest error log entries for a dataset.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

List of error entries

Return type

list[dict]

get_datasets(project_id)

Get existing datasets for a project.

Parameters

project_id (str) – The project id.

Returns

A list of dataset info for the project.

Return type

list

get_inferences_ingest_error_logs(project_id, dataset_id, inferences_id)

Get ingest error log entries for an inference set.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

List of error entries

Return type

list[dict]

get_issue_manager(project_id)

Get an issue manager object.

Parameters

project_id (str) – Project ID to manage.

Returns

The issue manager object.

Return type

IssueManager

get_project(project_id)

Get detailed info about a specific project

Returns

detailed info about a project

Return type

Dict[str, Any]

Parameters

project_id (str) –

get_projects()

Get info about existing projects

Returns

Project Info

Return type

list of dict

inferences_exists(project_id, dataset_id, inferences_id)

Check if a set of inferences exists.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

Whether the inferences id already exists.

Return type

bool

is_dataset_processed(project_id, dataset_id)

Check if a dataset is fully processed.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

Returns

If the dataset is done processing.

Return type

bool

is_inferences_processed(project_id, dataset_id, inferences_id)

Check if a set of inferences is fully processed.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • inferences_id (str) – inferences_id

Returns

If the inferences set is done processing.

Return type

bool

preview_frame(project_id, labeled_frame, inference_frame=None)

prints out a URL that lets you preview a provided frame in the web browser Useful for debugging data and image url issues.

Parameters

  • project_id (str) – Name of project to be associated for this frame preview (for label class association)

  • labeled_frame (LabeledFrame) – Labeled Frame desired for preview in web-app

  • inference_frame (Optional[InferencesFrame], optional) – Labeled Inference Desired for preview in web-app. Defaults to None.

Return type

None

project_exists(project_id)

Checks whether a project exists.

Parameters

project_id (str) – project_id

Returns

Does project exist

Return type

bool

set_credentials(*, token=None, app_id=None, app_key=None, api_key=None)

Set credentials for the client.

Parameters

  • api_key (str, optional) – A string for a long lived API key. Defaults to None.

  • token (str, optional) – A JWT providing auth credentials. Defaults to None.

  • app_id (str, optional) – Application ID string. Defaults to None.

  • app_key (str, optional) – Application secret key. Defaults to None.

Raises

Exception – Invalid credential combination provided.

Return type

None

update_label_class_map_colors(project_id, changed_label_classes)

Updates label class colors of a specific project

Parameters

  • project_id (str) – The project id.

  • changed_label_classes (List[ClassMapUpdateEntry]) – The list of label classes with changed colors. Must be a subset of the project’s overall label class map

Return type

None

upload_asset_from_filepath(project_id, dataset_id, filepath)

Upload an asset from a local file path. This is useful in cases where you have data on your local machine that you want to mirror in aquarium.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • filepath (str) – The filepath to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

upload_asset_from_url(project_id, dataset_id, source_url)

Upload an asset from a private url. This is useful in cases where you have data easily accessible on your network that you want to mirror in aquarium.

Parameters

  • project_id (str) – project_id

  • dataset_id (str) – dataset_id

  • source_url (str) – The source url to grab the assset data from

Returns

The URL to the mirrored asset.

Return type

str

class aquariumlearning.CollectionClient(*args, **kwargs)

Client class that interacts with the Aquarium REST API. Also handles extra work around collecting samples for collection campigns

Parameters

api_endpoint (str, optional) – The API endpoint to hit. Defaults to “https://illume.aquariumlearning.com/api/v1”.

sample_probabilities(frames)

Takes a list of Labeled Frames and stores scores for each based on each synced collection campaigns

Parameters

frames (List[LabeledFrame]) – a List of Labeled frames to score based on synced Collection Campaigns

Return type

None

save_for_collection(score_threshold=0.5)

Based on the score threshold, take all sampled frames and upload those that score above the score threshold for each Collection Campaign.

Parameters

score_threshold (float, optional) – Score threshold for all campaigns to save to server. Defaults to 0.7.

Return type

None

sync_state()

Downloads all collection campaigns and preps sampler with sample frames

Return type

None

write_to_file(frames, filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters

  • filelike (filelike) – The destination file-like to write to.

  • frames (List[Dict[str, any]]) –

Return type

None

class aquariumlearning.CustomMetricsDefinition(name, metrics_type)

Definitions for custom user provided metrics.

Parameters

  • name (str) – The name of this metric.

  • metrics_type (str) – The metrics type, either ‘objective’ or ‘confusion_matrix’.

class aquariumlearning.Inferences

A container used to construct a set of inferences.

Typical usage is to create an Inferences object, add multiple InferencesFrames to it, then serialize the frames to be submitted.

add_frame(frame)

Add an InferencesFrame to this dataset.

Parameters

frame (InferencesFrame) – An InferencesFrame in this dataset.

Return type

None

write_embeddings_to_file(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_to_file(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

class aquariumlearning.InferencesFrame(*, frame_id=None)

A frame containing inferences from an experiment.

Parameters

frame_id (str) – A unique id for this frame.

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per inference crop embedding

Parameters

  • label_id (str) – [description]

  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_custom_metric(name, value)

Add a custom metric for a given inference frame.

Parameters

  • name (str) – The name of the custom metric being added. Must match one of the custom_metrics already defined by the corresponding Project.

  • value (Union[float, List[Union[int, float]]]) – The value of the custom metric (either a float or 2d list of floats/integers).

add_embedding(*, embedding, crop_embeddings=None, model_id='')

DEPRECATED! PLEASE USE add_frame_embedding and add_crop_embedding Add an embedding to this frame, and optionally to crops/labels within it.

If provided, “crop_embeddings” is a list of dicts of the form:

‘uuid’: the label id for the crop/label ‘embedding’: a vector of floats between length 0 and 12,000.

Parameters

  • embedding (list of floats) – A vector of floats between length 0 and 12,000.

  • crop_embeddings (list of dicts, optional) – A list of dictionaries representing crop embeddings. Defaults to None.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_frame_embedding(*, embedding, model_id='')

Add an embedding to this frame

Parameters

  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_inference_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, confidence, area=None, iscrowd=None, user_attrs=None)

Add an inference for a 2D bounding box.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • area (float, optional) – The area of the image.

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_classification(*, sensor_id, label_id, classification, confidence, user_attrs=None)

Add an inference for 2D classification.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, confidence, user_attrs=None)

Add an inference for a 2D keypoints task.

A keypoint is a dictionary of the form:

‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • keypoints (list of dicts) – The keypoints of this detection

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_polygon_list(*, sensor_id, label_id, classification, polygons, confidence, center=None, user_attrs=None)

Add an inference for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: list of (x, y)

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • polygons (list of dicts) – The polygon geometry

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • center (list of ints or floats, optional) – The center point of the instance

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_2d_semseg(*, sensor_id, label_id, mask_url)

Add an inference for 2D semseg.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • mask_url (str) – URL to the pixel mask png.

Return type

None

add_inference_3d_classification(*, label_id, classification, confidence, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • confidence (float) – The confidence between 0.0 and 1.0 of the prediction

  • coord_frame_id (optional, str) – The coordinate frame id.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_inference_3d_cuboid(*, label_id, classification, position, dimensions, rotation, confidence, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add an inference for a 3D cuboid.

Parameters

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • position (list of float) – the position of the center of the cuboid

  • dimensions (list of float) – the dimensions of the cuboid

  • rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.

  • confidence (float) – confidence of prediction

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (dict, optional) – Links between labels. Defaults to None.

  • coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_inference_text_token(*, sensor_id, label_id, index, token, classification, visible, confidence, user_attrs=None)

Add a label for a text token.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • index (int) – the index of this token in the text

  • token (str) – the text content of this token

  • classification (str) – the classification string

  • visible (bool) – is this a visible token in the text

  • confidence (float) – confidence of prediction

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

to_dict()

Convert this frame into a dictionary representation.

Returns

dictified frame

Return type

dict

class aquariumlearning.Issue(name, dataset, elements, element_type, created_at=None, updated_at=None, reporter=None, assignee=None, state=None, issue_id=None, inference_set=None)

Definition for issue.

Parameters

  • name (str) – The issue name.

  • dataset (str) – The dataset for this issue.

  • elements (List[IssueElement]) – The elements of the issue.

  • element_type (str) – The element type of the issue (“frame”, “crop”).

  • created_at (datetime) – The time of issue creation.

  • updated_at (datetime) – The time of last issue update.

  • reporter (str) – Email of issue creator.

  • assignee (Optional[str], optional) – Email of the person assigned the issue. Defaults to None.

  • state (str) – Current state of issue (“triage”, “inProgress”, “inReview”, “resolved”, “cancelled”). Defaults to “triage”.

  • issue_id (str) – The issue id.

  • inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.

class aquariumlearning.IssueElement(element_id, frame_id, element_type, status=None, dataset=None, inference_set=None, frame_data=None, crop_data=None, label_metadata=None)

Definition for issue element.

Parameters

  • element_id (str) – The element id.

  • frame_id (str) – The frame id of the element.

  • element_type (str) – The element type of the issue element (“frame” or “crop”).

  • dataset (str) – (For read purposes, not element modification). The base dataset an element is from.

  • inference_set (str) – (For read purposes, not element modification). The inference set an element is from (if any).

  • status (str) – (For read purposes, not element modification). The status of the element.

  • frame_data – (For read purposes, not element modification). JSON object that is based on either a LabeledFrame or InferencesFrame

  • crop_data – (For read purposes, not element modification). JSON object for the specific “frame_data” crop that a “crop”-type element is based on.

  • label_metadata – (For read purposes, not element modification). JSON object with confidence and IOU info if the element was created from a ground truth/inference comparison.

class aquariumlearning.IssueManager(client, project_id)

An issue manager for interacting with issues within a given project.

Parameters

  • client (Client) – An Aquarium Learning Python Client object.

  • project_id (str) – The project id associated with this manager.

add_elements_to_issue(issue_id, elements)

Add elements to an issue.

Parameters

  • issue_id (str) – The issue id.

  • elements (List[IssueElement]) – The elements to add to the issue.

create_issue(name, dataset, elements, element_type, inference_set=None)

Create an issue.

Parameters

  • name (str) – The issue name.

  • dataset (str) – The dataset for this issue.

  • elements (List[IssueElement]) – The initial elements of the issue.

  • element_type (str) – The element type of the issue (“frame” or “crop”).

  • inference_set (Optional[str], optional) – The inference set for this issue. Defaults to None.

Returns

The created issue id.

Return type

str

delete_issue(issue_id)

Delete an issue.

Parameters

issue_id (str) – The issue id.

get_issue(issue_id)

Get a specific issue in the associated project. This will also include all associated frame metadata associated with each element.

Parameters

issue_id (str) – The issue id.

Returns

The issue data (including frame_data, crop_data, and label_metadata).

Return type

Issue

list_issues()

List issues in the associated project.

NOTE: this does NOT include the frame_data or crop_data information for the issue elements. (Use get_issue instead to see that info).

Returns

List of all issues data.

Return type

List[Issue]

remove_elements_from_issue(issue_id, elements)

Remove elements from an issue.

Parameters

  • issue_id (str) – The issue id.

  • elements (List[IssueElement]) – The elements to remove from the issue.

class aquariumlearning.LabelClassMap(*, entries=None)

A collection of ClassMapEntries that defines how to interpret classifications.

Parameters

entries (list of ClassMapEntry, optional) – List of classification entries. Defaults to None.

add_entry(entry)

Add a ClassMapEntry.

Parameters

entry (ClassMapEntry) – entry

from_classnames()

A helper utility to generate a set of distinct colors given a list of classnames based on the number of classes

Parameters

classnames (List[str]) – List of classifications as strings

Returns

A LabelClassMap containing categorical colors for the provided classnames.

Return type

LabelClassMap

from_classnames_max10()

A helper utility to generate a set of distinct colors given a list of classnames

Parameters

classnames (list of str) – List of up to ten classifications as strings

Returns

A LabelClassMap containing categorical colors for the provided classnames.

Return type

LabelClassMap

from_classnames_max20()

A helper utility to generate a set of distinct colors given a list of classnames

Parameters

classnames (List[str]) – List of up to twenty classifications as strings

Returns

A LabelClassMap containing categorical colors for the provided classnames.

Return type

LabelClassMap

from_classnames_turbo()

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the turbo color scheme, and will be assigned in order from dark to light.

Parameters

classnames (list of str) – List of classifications as strings.

Returns

A LabelClassMap containing colors for the provided classnames.

Return type

LabelClassMap

from_classnames_viridis()

A helper utility to generate a set of distinct colors given a list of classnames.

These colors are pulled from the viridis color scheme, and will be assigned in order from dark to light.

Parameters

classnames (list of str) – List of classifications as strings.

Returns

A LabelClassMap containing colors for the provided classnames.

Return type

LabelClassMap

class aquariumlearning.LabeledDataset

A container used to construct a labeled dataset.

Typical usage is to create a LabeledDataset, add multiple LabeledFrames to it, then serialize the frames to be submitted.

add_frame(frame)

Add a LabeledFrame to this dataset.

Parameters

frame (LabeledFrame) – A LabeledFrame in this dataset.

Return type

None

write_embeddings_to_file(filelike)

Write the frame’s embeddings to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

write_to_file(filelike)

Write the frame content to a text filelike object (File handle, StringIO, etc.)

Parameters

filelike (filelike) – The destination file-like to write to.

Return type

None

class aquariumlearning.LabeledFrame(*, frame_id, date_captured=None, device_id=None)

A labeled frame for a dataset.

Parameters

  • frame_id (str) – A unique id for this frame.

  • date_captured (str, optional) – ISO formatted datetime string. Defaults to None.

  • device_id (str, optional) – The device that generated this frame. Defaults to None.

add_audio(*, sensor_id, audio_url, date_captured=None)

Add an audio “sensor” data to this frame.

Parameters

  • sensor_id (str) – The id of this sensor.

  • audio_url (str) – The URL to load this audio data (mp3, etc.).

  • date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_coordinate_frame_3d(*, coord_frame_id, position=None, orientation=None, parent_frame_id=None)

Add a 3D Coordinate Frame.

Parameters

  • coord_frame_id (str) – String identifier for this coordinate frame

  • position (Optional[Dict[POSITION, Union[int, float]]], optional) – Dict of the form {x, y, z}. Defaults to None.

  • orientation (Optional[Dict[ORIENTATION, Union[int, float]]], optional) – Quaternion rotation dict of the form {w, x, y, z}. Defaults to None.

  • parent_frame_id (Optional[str], optional) – String id of the parent coordinate frame. Defaults to None.

Return type

None

add_crop_embedding(*, label_id, embedding, model_id='')

Add a per label crop embedding

Parameters

  • label_id (str) – [description]

  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_embedding(*, embedding, crop_embeddings=None, model_id='')

DEPRECATED! PLEASE USE add_frame_embedding and add_crop_embedding Add an embedding to this frame, and optionally to crops/labels within it.

If provided, “crop_embeddings” is a list of dicts of the form:

‘uuid’: the label id for the crop/label ‘embedding’: a vector of floats between length 0 and 12,000.

Parameters

  • embedding (list of floats) – A vector of floats between length 0 and 12,000.

  • crop_embeddings (list of dicts, optional) – A list of dictionaries representing crop embeddings. Defaults to None.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_frame_embedding(*, embedding, model_id='')

Add an embedding to this frame

Parameters

  • embedding (List[float]) – A vector of floats between length 0 and 12,000.

  • model_id (str, optional) – The model id used to generate these embeddings. Defaults to “”.

Return type

None

add_geo_latlong_data(lat, lon)

Add a user provided EPSG:4326 WGS84 lat long pair to each frame

We expect these values to be floats

Parameters

  • lat (float) – lattitude of Geo Location

  • lon (float) – longitude of Geo Location

Return type

None

add_image(*, sensor_id, image_url, preview_url=None, date_captured=None, width=None, height=None)

Add an image “sensor” data to this frame.

Parameters

  • sensor_id (str) – The id of this sensor.

  • image_url (str) – The URL to load this image data.

  • preview_url (Optional[str], optional) – A URL to a compressed version of the image. It must be the same pixel dimensions as the original image. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

  • width (Optional[int], optional) – The width of the image in pixels. Defaults to None.

  • height (Optional[int], optional) – The height of the image in pixels. Defaults to None.

Return type

None

add_label_2d_bbox(*, sensor_id, label_id, classification, top, left, width, height, iscrowd=None, user_attrs=None, links=None)

Add a label for a 2D bounding box.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (dict, optional) – Links between labels. Defaults to None.

Return type

None

add_label_2d_classification(*, sensor_id, label_id, classification, secondary_labels=None, user_attrs=None)

Add a label for 2D classification.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • secondary_labels (dict, optional) – dictionary of secondary labels

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_keypoints(*, sensor_id, label_id, classification, top, left, width, height, keypoints, user_attrs=None)

Add a label for a 2D keypoints task.

A keypoint is a dictionary of the form:

‘x’: x-coordinate in pixels ‘y’: y-coordinate in pixels ‘name’: string name of the keypoint

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • top (int or float) – The top of the box in pixels

  • left (int or float) – The left of the box in pixels

  • width (int or float) – The width of the box in pixels

  • height (int or float) – The height of the box in pixels

  • keypoints (list of dicts) – The keypoints of this detection

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_polygon_list(*, sensor_id, label_id, classification, polygons, center=None, user_attrs=None)

Add a label for a 2D polygon list instance segmentation task.

Polygons are dictionaries of the form:

‘vertices’: list of (x, y)

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • polygons (list of dicts) – The polygon geometry

  • center (list of ints or floats, optional) – The center point of the instance

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_2d_semseg(*, sensor_id, label_id, mask_url)

Add a label for 2D semseg.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • mask_url (str) – URL to the pixel mask png.

Return type

None

add_label_3d_classification(*, label_id, classification, coord_frame_id=None, user_attrs=None)

Add a label for 3D classification.

Parameters

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • coord_frame_id (str, optional) – The coordinate frame id.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_label_3d_cuboid(*, label_id, classification, position, dimensions, rotation, iscrowd=None, user_attrs=None, links=None, coord_frame_id=None)

Add a label for a 3D cuboid.

Parameters

  • label_id (str) – label_id which is unique across datasets and inferences.

  • classification (str) – the classification string

  • position (list of float) – the position of the center of the cuboid

  • dimensions (list of float) – the dimensions of the cuboid

  • rotation (list of float) – the local rotation of the cuboid, represented as an xyzw quaternion.

  • iscrowd (bool, optional) – Is this label marked as a crowd. Defaults to None.

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

  • links (dict, optional) – Links between labels. Defaults to None.

  • coord_frame_id (str, optional) – Coordinate frame id. Defaults to ‘world’

Return type

None

add_label_text_token(*, sensor_id, label_id, index, token, classification, visible, user_attrs=None)

Add a label for a text token.

Parameters

  • sensor_id (str) – sensor_id

  • label_id (str) – label_id which is unique across datasets and inferences.

  • index (int) – the index of this token in the text

  • token (str) – the text content of this token

  • classification (str) – the classification string

  • visible (bool) – is this a visible token in the text

  • user_attrs (dict, optional) – Any additional label-level metadata fields. Defaults to None.

Return type

None

add_obj(*, sensor_id, obj_url, coord_frame_id=None, date_captured=None)

Add a .obj file to the frame for text based geometry

Parameters

  • sensor_id (str) – sensor id

  • obj_url (str) – URL to where the object is located

  • coord_frame_id (Optional[str], optional) – ID for the coordinate frame. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_point_cloud_bins(*, sensor_id, pointcloud_url, intensity_url, range_url, coord_frame_id=None, date_captured=None)

Add a point cloud sensor data point to this frame, contained in dense binary files of little-endian values, similar to the raw format of KITTI lidar data.

Parameters

  • sensor_id (str) – Sensor id

  • pointcloud_url (str) – URL for the pointcloud: float32 [x1, y1, z1, x2, y2, z2, …]

  • intensity_url (str) – URL for the Intensity Pointcloud: unsigned int32 [i1, i2, …]

  • range_url (str) – URL for the Range Pointcloud: float32 [r1, r2, …]

  • coord_frame_id (Optional[str], optional) – Id for the Coordinate Frame. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_point_cloud_pcd(*, sensor_id, pcd_url, coord_frame_id=None, date_captured=None)

Add a point cloud sensor data point to this frame, contained in PCD format. ascii, binary, and binary_compressed formats are supported. Numeric values for the following column names are expected: x, y, z, intensity (optional), range (optional)

Parameters

  • sensor_id (str) – sensor id

  • pcd_url (str) – URL to PCD formated data

  • coord_frame_id (Optional[str], optional) – The coordinate frame id. Defaults to None.

  • date_captured (Optional[str], optional) – ISO formatted date. Defaults to None.

Return type

None

add_text(*, sensor_id, text, date_captured=None)

Add a text “sensor” data to this frame.

Parameters

  • sensor_id (str) – The id of this sensor.

  • text (str) – The text body.

  • date_captured (str, optional) – ISO formatted date. Defaults to None.

Return type

None

add_user_metadata(key, val, val_type=None)

Add a user provided metadata field.

The types of these metadata fields will be infered and they’ll be made available in the app for querying and metrics.

Parameters

  • key (str) – The key for your metadata field

  • val (Union[str, int, float, bool]) – value

  • val_type (Literal["str", "int", "float", "bool"], optional) – type of val as string. Defaults to None.

Return type

None

to_dict()

Convert this frame into a dictionary representation.

Returns

dictified frame

Return type

dict

class aquariumlearning.StratifiedMetricsDefinition(name, ordered_values)

Definitions for stratified metrics given object-level attributes

Parameters

  • name (str) – The name of this attribute, which should match the attribute on object labels.

  • ordered_values (List[str]) – The ordered list of valid values to group by.