[Bug]: Vector Search Index creation failed due to InternalServerError: 500 INTERNAL 13: INTERNAL

[JasperW01]

Contact Details

No response

File Name

gemini/use-cases/retrieval-augmented-generation/multimodal_rag_langchain.ipynb

What happened?

The following step failed with error of "InternalServerError: 500 INTERNAL 13: INTERNAL". The version of google-cloud-aiplatform installed is 1.44.0.

# https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings
DIMENSIONS = 768  # Dimensions output from textembedding-gecko

index = aiplatform.MatchingEngineIndex.create_tree_ah_index(
    display_name="mm_rag_langchain_index",
    dimensions=DIMENSIONS,
    approximate_neighbors_count=150,
    leaf_node_embedding_count=500,
    leaf_nodes_to_search_percent=7,
    description="Multimodal RAG LangChain Index",
)

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[JasperW01]

Full trackback print attached below:

---------------------------------------------------------------------------
InternalServerError                       Traceback (most recent call last)
Cell In[14], line 4
      1 # https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings
      2 DIMENSIONS = 768  # Dimensions output from textembedding-gecko
----> 4 index = aiplatform.MatchingEngineIndex.create_tree_ah_index(
      5     display_name="mm_rag_langchain_index",
      6     dimensions=DIMENSIONS,
      7     approximate_neighbors_count=150,
      8     leaf_node_embedding_count=500,
      9     leaf_nodes_to_search_percent=7,
     10     description="Multimodal RAG LangChain Index",
     11 )

File ~/.local/lib/python3.10/site-packages/google/cloud/aiplatform/matching_engine/matching_engine_index.py:558, in MatchingEngineIndex.create_tree_ah_index(cls, display_name, contents_delta_uri, dimensions, approximate_neighbors_count, leaf_node_embedding_count, leaf_nodes_to_search_percent, distance_measure_type, description, labels, project, location, credentials, request_metadata, sync, index_update_method, encryption_spec_key_name, create_request_timeout, shard_size)
    545 algorithm_config = matching_engine_index_config.TreeAhConfig(
    546     leaf_node_embedding_count=leaf_node_embedding_count,
    547     leaf_nodes_to_search_percent=leaf_nodes_to_search_percent,
    548 )
    550 config = matching_engine_index_config.MatchingEngineIndexConfig(
    551     dimensions=dimensions,
    552     algorithm_config=algorithm_config,
   (...)
    555     shard_size=shard_size,
    556 )
--> 558 return cls._create(
    559     display_name=display_name,
    560     contents_delta_uri=contents_delta_uri,
    561     config=config,
    562     description=description,
    563     labels=labels,
    564     project=project,
    565     location=location,
    566     credentials=credentials,
    567     request_metadata=request_metadata,
    568     sync=sync,
    569     index_update_method=index_update_method,
    570     encryption_spec_key_name=encryption_spec_key_name,
    571     create_request_timeout=create_request_timeout,
    572 )

File ~/.local/lib/python3.10/site-packages/google/cloud/aiplatform/base.py:863, in optional_sync.<locals>.optional_run_in_thread.<locals>.wrapper(*args, **kwargs)
    861     if self:
    862         VertexAiResourceNounWithFutureManager.wait(self)
--> 863     return method(*args, **kwargs)
    865 # callbacks to call within the Future (in same Thread)
    866 internal_callbacks = []

File ~/.local/lib/python3.10/site-packages/google/cloud/aiplatform/matching_engine/matching_engine_index.py:231, in MatchingEngineIndex._create(cls, display_name, contents_delta_uri, config, description, labels, project, location, credentials, request_metadata, sync, index_update_method, encryption_spec_key_name, create_request_timeout)
    220 create_lro = api_client.create_index(
    221     parent=initializer.global_config.common_location_path(
    222         project=project, location=location
   (...)
    226     timeout=create_request_timeout,
    227 )
    229 _LOGGER.log_create_with_lro(cls, create_lro)
--> 231 created_index = create_lro.result(timeout=None)
    233 _LOGGER.log_create_complete(cls, created_index, "index")
    235 index_obj = cls(
    236     index_name=created_index.name,
    237     project=project,
    238     location=location,
    239     credentials=credentials,
    240 )

File /opt/conda/lib/python3.10/site-packages/google/api_core/future/polling.py:261, in PollingFuture.result(self, timeout, retry, polling)
    256 self._blocking_poll(timeout=timeout, retry=retry, polling=polling)
    258 if self._exception is not None:
    259     # pylint: disable=raising-bad-type
    260     # Pylint doesn't recognize that this is valid in this case.
--> 261     raise self._exception
    263 return self._result

InternalServerError: 500 INTERNAL 13: INTERNAL

[JasperW01]

on the console of Vector Search, it shows an error mark as the screenshot

[JasperW01]

As comparison, tried the notebook of intro-textemb-vectorsearch.ipynb as well, and the vector search index creation also fails due to the same error.

[holtskinner]

@JasperW01 Thanks for the report. Does Cloud Logging have any more information about this failed operation?

[JasperW01]

Hi @holtskinner , sorry for the late response. the notification was buried by a lot of other repo messages.

No, I could't find any logging record in the Cloud Logging.