Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add More Error Logging When Indexing Fails #563

Merged
merged 1 commit into from
Aug 10, 2024

Conversation

jvanderen1
Copy link
Contributor

@jvanderen1 jvanderen1 commented Aug 2, 2024

In trying to debug an issue, I needed to add more logging to PGSync in order to see the problem. This PR adds additional logging similar to this one within the Elasticsearch community.

Before, I would see messages like:

BulkIndexError: ('1 document(s) failed to index.')

While this did indicate something was wrong with the document, I did not know which one or why it was problematic. With this change, I now see errors containing:

{..., 'reason': 'Document contains at least one immense term in field="description" ...}

This along with additional information such as the document shape (including id) have been valuable information.

Note: This logging will only be visible with ELASTICSEARCH_RAISE_ON_ERROR="false" and ELASTICSEARCH_RAISE_ON_EXCEPTION="false". This was thanks to the suggestion within #503 (comment)

@jvanderen1 jvanderen1 changed the title Add More Logging Add More Error Logging When Indexing Fails Aug 2, 2024
Copy link
Owner

@toluaina toluaina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! wish I'd have thought of that

@toluaina toluaina merged commit 5110dda into toluaina:main Aug 10, 2024
1 check passed
@jvanderen1 jvanderen1 deleted the jvanderen1-add-more-logging branch August 10, 2024 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants