improve delete performance for clickhouse

[iuwqyir]

TL;DR

Improved the deletion logic in case of reorgs in ClickHouse by implementing partition-aware deletes and disabling lightweight delete synchronization.

What changed?

• Added lightweight_deletes_sync setting (value 0) to ClickHouse configuration. This will make the deletes async, so processing does not need to wait for it to finish
• Split generic deleteBatch function into specific delete functions for blocks, logs, transactions, and traces, which will improve performance due to having all of the order by keys defined
• Implemented two-step deletion process:
  1. Query for existing records that need deletion
  2. Execute targeted deletes using partition values and primary keys
• Added proper error handling and batch processing for deletions

Why make this change?

The previous deletion method was inefficient and could cause performance issues in ClickHouse.

[iuwqyir]

• automated clickhouse migrations #133
• improve delete performance for clickhouse #132 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.