Generate Embeddings Using AI Models (GeoAI Tools)

Summary

Generates semantic embeddings for geospatial data using foundation models. The tool converts input data such as images and geographic locations into vector embeddings that capture semantic meaning. These embeddings can then be used in downstream embeddings based tools and workflows in ArcGIS Pro, such as similarity search, change detection, and find features by description.

Usage

This tool requires deep learning frameworks to be installed. To set up your machine to use deep learning frameworks in ArcGIS Pro, see Install deep learning frameworks for ArcGIS.

This tool requires a model definition file containing model information. The Input Model Definition File parameter value can be an Esri model definition JSON file (.emd) or a deep learning model package (.dlpk). The model files can be stored locally or hosted on ArcGIS Living Atlas of the World.
This tool supports the use of third-party models created using the model extensibility feature. This feature enables tasks—such as generating embeddings for locations, texts, images, and so on—using custom deep learning models that were not trained with tools supported by ArcGIS Pro. To learn more about creating a custom deep learning model file, see Use third-party language models with ArcGIS.
This tool can run on CPU or GPU; however, deep learning is computationally intensive and a GPU is recommended. To run this tool using GPU, set the Processor Type environment to GPU. If you have more than one GPU, specify the GPU ID environment instead.
For information about requirements for running this tool and issues you may encounter, see Deep Learning frequently asked questions.

Label	Explanation	Data type
Input Feature Class Or Raster	An input raster imagery or location feature layer for which embeddings will be generated.	Raster Dataset; Raster Layer; Mosaic Layer; Image Service; Map Server; Map Server Layer; Internet Tiled Layer; Feature Layer; Feature Class
Output Embeddings Feature Class	The output embeddings feature class storing generated embeddings in Esri-compatible embedding format.	Feature Class
Input Model Definition File	The foundation model that will be used to compute embeddings. The model determines the semantic representation of raw data.	File
Arguments (Optional)	Additional arguments that will be used by the model while performing inference. Value table columns: Grid Size—This is a model argument which defines the embedding grid size for raster inputs. One embedding is generated per embedding grid size covering the specified area in meters. Text Field—Field name whose values will be converted into embeddings while generating text embeddings.	Value Table

arcpy.geoai.GenerateEmbeddingsUsingAIModels(in_data, out_embeddings_feature_class, in_model_definition_file, {arguments})

Name	Explanation	Data type
in_data	An input raster imagery or location feature layer for which embeddings will be generated.	Raster Dataset; Raster Layer; Mosaic Layer; Image Service; Map Server; Map Server Layer; Internet Tiled Layer; Feature Layer; Feature Class
out_embeddings_feature_class	The output embeddings feature class storing generated embeddings in Esri-compatible embedding format.	Feature Class
in_model_definition_file	The foundation model that will be used to compute embeddings. The model determines the semantic representation of raw data.	File
arguments [arguments,...] (Optional)	Additional arguments that will be used by the model while performing inference. Value table columns: `grid_size`—This is a model argument which defines the embedding grid size for raster inputs. One embedding is generated per embedding grid size covering the specified area in meters. `text_field`—Field name whose values will be converted into embeddings while generating text embeddings.	Value Table

Code sample

GenerateEmbeddingsUsingAIModels example (stand-alone script)

The following example demonstrates how to use the GenerateEmbeddingsUsingAIModels function.

# Name: GenerateEmbeddingsUsingAIModels.py
# Description: Generates semantic embeddings for geospatial data using foundation models.
#
# Requirements: ArcGIS Pro Advanced license for generating location or textual embeddings and Image Analyst license for generating image embeddings

# Import system modules
import arcpy

# Set local variables
in_data = "C:/data/residential_area.tif"
out_embeddings_feature_class = "C:/project/GenerateEMbeddings.gdb/generated_embeddings"
in_model_definition_file = "C:/project/models/dinoV2.dlpk"

# Run GenerateEmbeddingsUsingAIModels
arcpy.geoai.GenerateEmbeddingsUsingAIModels(in_data, out_embeddings_feature_class, in_model_definition_file)

Environments

Extent, Processor Type, GPU ID, Parallel Processing Factor

Licensing information