Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): Log webhooks from syncs separate from the sync #3312

Open
wants to merge 26 commits into
base: master
Choose a base branch
from

Conversation

nalanj
Copy link
Contributor

@nalanj nalanj commented Jan 15, 2025

Log webhooks from syncs in a different log context than the sync itself.

I'll be putting together future PR(s) to handle webhooks from refresh token failures and new connections.

See https://linear.app/nango/issue/NAN-1547/log-nango-webhooks

How I tested it

  • Set up webhooks
  • Trigger a sync
  • You'll see the sync run, and then you'll see a separate webhook event.

Copy link

linear bot commented Jan 15, 2025

@nalanj nalanj self-assigned this Jan 15, 2025
@TBonnin
Copy link
Collaborator

TBonnin commented Jan 16, 2025

since we are improving the logging. Can we check if we could add some http logging that would help customers debug why calling the webhook endpoint failed?

@nalanj
Copy link
Contributor Author

nalanj commented Jan 16, 2025

@TBonnin absolutely - I'll dig into that too

@nalanj
Copy link
Contributor Author

nalanj commented Jan 16, 2025

@TBonnin better errors are covered in #3319

@@ -51,8 +51,8 @@
"devDependencies": {
"@eslint/js": "9.17.0",
"@types/node": "20.12.2",
"@typescript-eslint/eslint-plugin": "8.19.1",
"@typescript-eslint/parser": "8.19.1",
"@typescript-eslint/eslint-plugin": "8.20.0",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bumped these to try to fix the build failure you can see in the commit history. I figured I might as well keep the upgrade around.

throw new Error('Missing connection.id');
}

if (!nangoProps.syncConfig.id) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These checks are how I ended up resolving the build issue in CI. More than happy to discuss further, but basically the lint error I was seeing in the build history on commits in this PR wasn't happening on my computer, and I could never get it to reproduce. This worked around it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are just not up to date

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just merged and am running through CI! 🤞🏻

@nalanj nalanj requested a review from a team January 16, 2025 22:28
@nalanj nalanj marked this pull request as ready for review January 16, 2025 22:28
Copy link
Collaborator

@bodinsamuel bodinsamuel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍🏻
I feel like it's missing the same code for onFailure. Also sendSyncWebhook is used in webhook execution but not sure if it's part of this change?

packages/jobs/lib/execution/sync.ts Outdated Show resolved Hide resolved
throw new Error('Missing connection.id');
}

if (!nangoProps.syncConfig.id) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are just not up to date

throw new Error(`Failed to send webhook for sync: ${nangoProps.syncConfig.sync_name}`);
} else {
await webhookLogCtx.success();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think it could be self-contained inside sendSyncWebhook?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely - we'd just need to send a handful more parameters to sendSyncWebhook to help it have all the log context params it needs. I'm not sure if that's ideal with the package separation, but we've already got a function called sendSync so it's not like the webhook package doesn't know about syncs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bodinsamuel See ee9c652 for version where the log context is created inside of sendSync.

@nalanj nalanj requested review from bodinsamuel and a team January 17, 2025 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants