Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table resolvedTs is missing after truncate table DDL. #889

Open
asddongmen opened this issue Jan 16, 2025 · 1 comment
Open

Table resolvedTs is missing after truncate table DDL. #889

asddongmen opened this issue Jan 16, 2025 · 1 comment
Assignees
Labels
type/bug The issue is confirmed as a bug.

Comments

@asddongmen
Copy link
Collaborator

asddongmen commented Jan 16, 2025

What I did?

The integration test case multi_source is unstable, and I cannot determine the cause based on the current logs.

So I added a debug log in the eventStore to print the received resolvedTs and retested.

In the retest logs, I found that this dispatcher was not reset after registration.

[2025/01/16 02:44:50.689 +00:00] [INFO] [event_broker.go:293] ["dispatcher not reset"] [dispatcher=73875005148128948212000780908188640696]
[2025/01/16 02:45:50.689 +00:00] [INFO] [event_broker.go:293] ["dispatcher not reset"] [dispatcher=73875005148128948212000780908188640696]
[2025/01/16 02:46:50.689 +00:00] [INFO] [event_broker.go:293] ["dispatcher not reset"] [dispatcher=73875005148128948212000780908188640696]
[2025/01/16 02:47:50.689 +00:00] [INFO] [event_broker.go:293] ["dispatcher not reset"] [dispatcher=73875005148128948212000780908188640696]

I identified its tableID as 210.

[2025/01/16 02:43:57.523 +00:00] [INFO] [event_dispatcher_manager.go:434] ["new dispatcher created"] [ID=73875005148128948212000780908188640696] [changefeedID=5137061994273448495694501480027201430] [namespace=default] **[tableSpan="TableID:210** StartKey:\"t\\200\\000\\000\\000\\000\\000\\000\\377\\322_r\\000\\000\\000\\000\\000\\372\" EndKey:\"t\\200\\000\\000\\000\\000\\000\\000\\377\\322_s\\000\\000\\000\\000\\000\\372\" "] [startTs=455342903358128172]

I then searched for its resolvedTs log but found no results.

> cat cdc.log | grep "hyy advance resolved ts" | grep "tableID=210" | wc -l
0

However, I was able to find resolvedTs logs for other tables.

 cat cdc.log | grep "hyy advance resolved ts"  | wc -l
 757
 
 > cat cdc.log | grep "hyy advance resolved ts"  | tail
[2025/01/16 02:52:54.494 +00:00] [DEBUG] [event_store.go:521] ["hyy advance resolved ts"] [tableID=182] [dispatcherID=4509240733371491341320306463344308135] [resolvedTs=455343072729628675] [maxEventCommitTs=455342902611017742]
[2025/01/16 02:52:55.495 +00:00] [DEBUG] [event_store.go:521] ["hyy advance resolved ts"] [tableID=182] [dispatcherID=4509240733371491341320306463344308135] [resolvedTs=455343072991510531] [maxEventCommitTs=455342902611017742]
[2025/01/16 02:52:56.496 +00:00] [DEBUG] [event_store.go:521] ["hyy advance resolved ts"] [tableID=182] [dispatcherID=4509240733371491341320306463344308135] [resolvedTs=455343073253916676] [maxEventCommitTs=455342902611017742]

From this, I concluded that the resolvedTs for tableID=210 is missing.

Failed Test Link: https://github.com/pingcap/ticdc/actions/runs/12800697540/job/35688920752?pr=882
Log Link: https://github.com/pingcap/ticdc/actions/runs/12800697540/artifacts/2438463597

@asddongmen asddongmen added the bug Something isn't working label Jan 16, 2025
@asddongmen
Copy link
Collaborator Author

asddongmen commented Jan 16, 2025

I ran test based on #882 .

@asddongmen asddongmen added type/bug The issue is confirmed as a bug. and removed bug Something isn't working labels Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug The issue is confirmed as a bug.
Projects
None yet
Development

No branches or pull requests

2 participants