Place Recognition package

Submodules

i_place_recognition module

class dronebuddylib.atoms.placerecognition.i_place_recognition.IPlaceRecognition(engine_configurations: EngineConfigurations)

Bases: IDBLFunction

Interface for place recognition functionality.

This interface defines the methods required for recognizing and remembering places using images. It allows for the association of names with recognized places and testing the memory of the recognition system.

recognize_place()

Recognizes places in an image.

remember_place()

Associates a name with a place in an image.

test_memory()

Tests the accuracy of the place recognition algorithm.

get_current_status()

Retrieves the current status of the place recognition engine.

abstract create_memory()

Create a memory database or structure for the place recognition engine.

This method creates a memory structure or database for the place recognition engine, which can be used to optimize future recognition tasks.

Returns:

A data structure or system representing the memory of the place recognition engine, useful for improving recognition efficiency and accuracy.

abstract get_current_status()

Get the current status of the place recognition engine.

This method provides the current operational status of the recognition engine, including any errors, operational mode, or other relevant status information.

Returns:

A description or object representing the current status of the place recognition engine. The exact return type and structure can be defined as needed.

abstract recognize_place(image) → RecognizedPlaces

Recognize places in an image.

Parameters:

image – The image containing places to be recognized.

Returns:

An object containing a list of recognized places with their details (e.g., names, bounding boxes).

Return type:

RecognizedPlaces

abstract remember_place(image=None, name=None) → bool

Associate a name with a place in an image.

This method is used to remember a place by associating it with a given name, facilitating easier identification in future recognitions.

Parameters:

• image – The image containing the place to be associated with a name.

• name (str) – The name to be associated with the place.

Returns:

True if the association was successful, False otherwise.

Return type:

bool

abstract test_memory() → dict

Test the accuracy of the place recognition algorithm.

This method evaluates the accuracy and performance of the place recognition algorithm by returning relevant metrics.

Returns:

A dictionary with accuracy-related numbers for the place recognition algorithm, including precision, recall, and F1 score, among others.

Return type:

dict

place_recognition_engine module

class dronebuddylib.atoms.placerecognition.place_recognition_engine.PlaceRecognitionEngine(algorithm: PlaceRecognitionAlgorithm, config: EngineConfigurations)

Bases: object

The PlaceRecognitionEngine class handles place recognition operations, facilitating the recognition and remembrance of places within images.

create_memory(changes=None)

Create a memory database or structure for the place recognition engine, optimizing future recognition tasks.

Returns:

A data structure or system representing the memory of the place recognition engine, useful for improving recognition efficiency and accuracy.

get_current_status()

Get the current operational status of the place recognition engine, including any relevant metrics or state information.

Returns:

An object or description providing insights into the current status of the place recognition engine, which may include its operational state, any errors, or performance metrics.

recognize_place(image)

Recognize places in an image, identifying and categorizing various locations depicted in the image.

Parameters:

image – The image containing places to be recognized.

Returns:

A list of recognized places, each potentially with associated metadata such as location name or coordinates.

remember_place(image=None, name=None, drone_instance=None, on_start=None, on_training_set_complete=None, on_validation_set_complete=None)

Remember a place by associating it with a name, facilitating its future identification and recall.

Parameters:

• image – The image containing the place.

• name (str) – The name to be associated with the place.

Returns:

True if the operation was successful, False otherwise.

place_recognition_knn_impl module

class dronebuddylib.atoms.placerecognition.place_recognition_knn_impl.PlaceRecognitionKNNImpl(engine_configurations: EngineConfigurations)

Bases: IPlaceRecognition

This class implements place recognition using the k-nearest-neighbors (KNN) algorithm, leveraging image features extracted via a pre-trained ResNet model.

When should I use this class? This implementation is ideal when you need to identify specific places or landmarks within a large dataset of known locations efficiently. It’s particularly useful for applications in robotics, drones, or any system requiring geographical awareness from visual cues.

Algorithm Description: The KNN classifier is trained on a dataset of images labeled with their corresponding places. For an unknown image,

the classifier predicts the place by finding the k most similar images in its training set

(based on the closest feature vectors under Euclidean distance) and performing a majority vote on their labels.

For instance, if k=3 and the three closest images in the training set to the given image are two images of the Eiffel Tower and one image of the Statue of Liberty, the result would be ‘Eiffel Tower’.

• This implementation can weight the votes according to the distance of neighbors,

giving closer neighbors more influence on the final outcome.

Usage:

1. Prepare a dataset of images for the places you want to recognize. Organize the images so that there is a sub-directory for each place within a main directory.

2. Use the ‘train’ method of this class to train the classifier on your dataset.

You can save the trained model to disk by specifying a ‘model_save_path’, allowing you to reuse the model without retraining.

3. To recognize the place depicted in a new, unknown image, call the ‘recognize_place’ method with the image as input.

NOTE: This implementation requires scikit-learn, NumPy, PyTorch, torchvision,

and PIL to be installed for machine learning operations and image processing. Ensure these packages are installed in your environment:

$ pip install scikit-learn numpy torch torchvision Pillow

A class implementing KNN-based place recognition.

This class uses k-nearest neighbors algorithm to recognize places by comparing the feature vectors of images extracted using a pre-trained ResNet model. It is capable of training a model with labeled images of places and predicting the place of an unknown image.

use_drone

Indicates if a drone is used to create the dataset.

Type:

bool

progress_event

Signals when the training process is done.

Type:

threading.Event

current_status

Holds the current status of the operation.

Type:

ExecutionStatus

configs

Configuration parameters for the engine.

Type:

EngineConfigurations

classifier_path = 'resources/models/classifiers/knn/trained_place_knn_model'

create_data_set(place_name, data_mode, drone_instance=None)

Generates a dataset for a given place name, optionally using a drone for image collection. Different modes support training, validation, and testing data collection.

Parameters:

• place_name (str) – The name of the place for which to create the dataset.

• data_mode (int) – The mode of dataset creation (e.g., training, validation).

• drone_instance – An optional drone instance to use for collecting images.

create_memory(changes=None)

Create a memory database or structure for the place recognition engine.

This method creates a memory structure or database for the place recognition engine, which can be used to optimize future recognition tasks.

Returns:

A data structure or system representing the memory of the place recognition engine, useful for improving recognition efficiency and accuracy.

current_status = None

extract_features_with_specified_models(image) → str

feature_extractor(img)

Extracts feature vectors from the given image using a pre-defined neural network model.

This method is crucial for converting raw images into a form that can be effectively used by the KNN classifier for place recognition.

Args:

img: The image from which to extract features, expected to be in a format compatible

with the pre-processing transformations.

Returns:

numpy.ndarray: The extracted features as a flat array.

feature_extractor_place(img)

Extracts feature vectors from the given image using a pre-defined neural network model.

This method is crucial for converting raw images into a form that can be effectively used by the KNN classifier for place recognition.

Args:

img: The image from which to extract features, expected to be in a format compatible

with the pre-processing transformations.

Returns:

numpy.ndarray: The extracted features as a flat array.

get_algorithm_name() → str

Get the algorithm name of the FaceRecognitionImpl class.

Returns:

The algorithm name.

Return type:

str

get_class_name() → str

Get the class name of the FaceRecognitionImpl class.

Returns:

The class name.

Return type:

str

get_current_status()

Get the current status of the place recognition engine.

This method provides the current operational status of the recognition engine, including any errors, operational mode, or other relevant status information.

Returns:

A description or object representing the current status of the place recognition engine. The exact return type and structure can be defined as needed.

get_optional_params() → list

Get the optional parameters for the FaceRecognitionImpl class.

Returns:

A list of optional parameters.

Return type:

list

get_required_params() → list

Get the required parameters for the FaceRecognitionImpl class.

Returns:

A list of required parameters.

Return type:

list

image_files_in_folder(folder)

load_test_data(test_dir)

Load images and true labels from the test directory.

preprocess_image(img)

progress_bar(done_event, title='Training Progress')

progress_event = <threading.Event object>

recognize_place(image) → RecognizedPlaces

Recognizes the place depicted in the given image. If the confidence of the prediction is below a given threshold, the place is classified as ‘unknown’.

Args:

image: The image of the place to recognize.

Returns:

RecognizedPlaces: The recognized place.

remember_place(image=None, name=None) → bool

Associate a name with a place in an image.

This method is used to remember a place by associating it with a given name, facilitating easier identification in future recognitions.

Parameters:

• image – The image containing the place to be associated with a name.

• name (str) – The name to be associated with the place.

Returns:

True if the association was successful, False otherwise.

Return type:

bool

test_classifier(future=None)

test_memory() → dict

Tests the memory capabilities of the classifier by evaluating its performance on a predefined test dataset. This method serves as a diagnostic tool to ensure the classifier is functioning as expected.

Returns:

A dictionary containing key performance metrics such as accuracy, precision, and F1 score of the classifier on the test dataset.

Return type:

dict

train(train_dir, model_save_path=None, n_neighbors=3, knn_algo='auto', weights='distance', classifier_index=0, changes=None, verbose=False, future=None)

Trains the KNN classifier on a set of labeled images stored in a directory structure. Each sub-directory within the train directory should represent a class (place), and contain images corresponding to that place.

Parameters:

• train_dir (str) – The directory containing the training dataset.

• model_save_path (str, optional) – Path to save the trained model.

• n_neighbors (int, optional) – Number of neighbors to use for k-nearest neighbors voting.

• knn_algo (str, optional) – Underlying algorithm to compute the nearest neighbors.

• weights (str, optional) – Weight function used in prediction.

• classifier_index (int, optional) – An index to uniquely identify the classifier.

• changes (str, optional) – Description of any changes or versioning info.

• verbose (bool, optional) – Enables verbose output during the training process.

• future (Future, optional) – A Future object for asynchronous execution.

Returns:

A dictionary containing performance metrics such as accuracy and precision of the trained model.

Return type:

dict

train_random_forest_classifier(train_dir, model_save_path=None, n_estimators=100, max_depth=None, random_state=42, criterion='gini', min_samples_split=2, classifier_index=None, changes='', verbose=False, future=None)

Trains the Random Forest classifier on a set of labeled images stored in a directory structure. Each sub-directory within the train directory should represent a class (place), and contain images corresponding to that place.

Parameters:

• train_dir (str) – The directory containing the training dataset.

• model_save_path (str, optional) – Path to save the trained model.

• n_estimators (int, optional) – The number of trees in the forest.

• max_depth (int, optional) – The maximum depth of the trees.

• random_state (int, optional) – Controls both the randomness of the bootstrapping of the samples used when building trees (if bootstrap=True) and the sampling of the features to consider when looking for the best split at each node.

• criterion (str, optional) – The function to measure the quality of a split.

• min_samples_split (int, optional) – The minimum number of samples required to split an internal node.

• classifier_index (int, optional) – An index to uniquely identify the classifier.

• changes (str, optional) – Description of any changes or versioning info.

• verbose (bool, optional) – Enables verbose output during the training process.

• future (Future, optional) – A Future object for asynchronous execution.

Returns:

A dictionary containing performance metrics such as accuracy and precision of the trained model.

Return type:

dict

use_drone = False

place_recognition_result module

class dronebuddylib.atoms.placerecognition.place_recognition_result.RecognizedPlaceObject(name: str, confidence: float)

Bases: object

get_confidence() → []

get_name() → str

class dronebuddylib.atoms.placerecognition.place_recognition_result.RecognizedPlaces(most_probable_place: RecognizedPlaceObject, probable_places: list[RecognizedPlaceObject])

Bases: object

get_most_likely() → RecognizedPlaceObject

get_recognized_places() → list[RecognizedPlaceObject]

Module contents