Updating index and querying raises error (Cohere Bedrock Embeddings).

[SMM-99]

I have a VectorStoreIndex that I update with new documents:

bedrock_embed_model = BedrockEmbedding(
    model_name="cohere.embed-multilingual-v3",
)
refreshed_docs = index.refresh_ref_docs(
    new_documents, embed_model=bedrock_embed_model
)

I also have this chat_engine built as follows:

bedrock_embed_model = BedrockEmbedding(
    model_name="cohere.embed-multilingual-v3",
)

retriever = VectorIndexRetriever(
    index=app.state.index,
    similarity_top_k=3,
    embed_model=bedrock_embed_model,

)

bedrock_llm = Bedrock(
    model="eu.anthropic.claude-3-5-sonnet-20240620-v1:0",
)


chat_memory = ChatMemoryBuffer.from_defaults(
    token_limit=10000,
    chat_store=app.state.chat_store,
    chat_store_key=email,
)


chat_engine = CondensePlusContextChatEngine(
    retriever=retriever, memory=chat_memory, llm=bedrock_llm, verbose=True
)

However, when I execute chat_engine.chat(query), it throws the following error: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (1315,) + inhomogeneous part. If I do not update the index, it works fine.

[dosubot[bot]]

Hey @SMM-99! I'm here to help you with any bugs, questions, or contributions. Let's tackle this issue together.

The error you're encountering, "setting an array element with a sequence," typically indicates that there is a mismatch in the expected shape of the embeddings when updating the index. This can happen if the embeddings generated by the BedrockEmbedding model for the new documents have a different shape or structure compared to the existing embeddings in the index.

Here are a few steps you can take to troubleshoot and resolve this issue:

1. Check Embedding Dimensions: Ensure that the embeddings generated by the BedrockEmbedding model for the new documents have the same dimensions as those already in the index. The embeddings should be consistent in size, as any discrepancy can lead to shape mismatches.

2. Verify Embedding Model Configuration: Double-check the configuration of the BedrockEmbedding model to ensure it is set up correctly and consistently for both the initial and new document embeddings. Any changes in model parameters or settings could affect the output shape.

3. Inspect New Document Embeddings: Before updating the index, inspect the embeddings of the new documents to confirm their shape. You can print or log the shape of these embeddings to verify they match the expected dimensions.

4. Consistency Across Updates: Ensure that the same embedding model and configuration are used consistently across all updates to the index. Any variation could lead to incompatible embeddings.

By ensuring that the embeddings are consistent in shape and configuration, you should be able to resolve the inhomogeneous shape error. If the issue persists, further investigation into the specific embeddings and their generation process may be necessary.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[SMM-99]

How can I inspect the shape of the embeddings for the old and new documents? @dosu