Skip to content

Commit

Permalink
update MD by dispatch event pingcap/docs master
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions committed Jan 22, 2025
1 parent 8991056 commit 3d488ae
Show file tree
Hide file tree
Showing 3 changed files with 62 additions and 36 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,11 @@ This section introduces the configuration of a replication task.
- This configuration item affects configurations related to filter and sink.
- Default value: `false`
### `force-replicate`
- Specifies whether to forcibly [replicate tables without a valid index](/ticdc/ticdc-manage-changefeed.md#replicate-tables-without-a-valid-index).
- Default value: `false`
### `enable-sync-point` <span class="version-mark">New in v6.3.0</span>
- Specifies whether to enable the Syncpoint feature, which is supported starting from v6.3.0 and is disabled by default.
Expand Down
77 changes: 45 additions & 32 deletions markdown-pages/en/tidb/master/ticdc/ticdc-ddl.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,38 +11,51 @@ This document describes the rules and special cases of DDL replication in TiCDC.

Currently, TiCDC uses an allow list to determine whether to replicate a DDL statement. Only the DDL statements in the allow list are replicated to the downstream. The DDL statements not in the allow list are not replicated.

The allow list of DDL statements supported by TiCDC is as follows:

- create database
- drop database
- create table
- drop table
- add column
- drop column
- create index / add index
- drop index
- truncate table
- modify column
- rename table
- alter column default value
- alter table comment
- rename index
- add partition
- drop partition
- truncate partition
- create view
- drop view
- alter table character set
- alter database character set
- recover table
- add primary key
- drop primary key
- rebase auto id
- alter table index visibility
- exchange partition
- reorganize partition
- alter table ttl
- alter table remove ttl
In addition, TiCDC determines whether to replicate a DDL statement to the downstream based on whether the table has a [valid index](/ticdc/ticdc-overview.md#valid-index) and whether the configuration item [`force-replicate`](/ticdc/ticdc-changefeed-config.md#force-replicate) is set to `true`. When `force-replicate=true`, the replication task attempts to forcibly [replicate tables without a valid index](/ticdc/ticdc-manage-changefeed.md#replicate-tables-without-a-valid-index).

The following is the allow list of DDL statements supported by TiCDC. The abbreviations in the table:

- Y: Replication to the downstream is supported in this condition.
- N: Replication to the downstream is not supported in this condition.

> **Note**
>
> - When the upstream table has no valid index and `force-replicate=true` is not configured, the table will not be replicated. However, subsequent DDL statements (including `CREATE INDEX`, `ADD INDEX`, and `ADD PRIMARY KEY`) that create a valid index on this table will be replicated, which might cause inconsistency between downstream and upstream table schemas and lead to subsequent data replication failure.
> - DDL statements (including `DROP INDEX` and `DROP PRIMARY KEY`) that drop the last valid index will not be replicated, causing subsequent data replication to fail.
| DDL | A valid index exists | A valid index does not exist and `force-replicate` is `false` (default) | A valid index does not exist and `force-replicate` is set to `true` |
|---|:---:|:---:| :---: |
| `CREATE DATABASE` | Y | Y | Y |
| `DROP DATABASE` | Y | Y | Y |
| `ALTER DATABASE CHARACTER SET` | Y | Y | Y |
| `CREATE INDEX` | Y | Y | Y |
| `ADD INDEX` | Y | Y | Y |
| `DROP INDEX` | Y | N | Y |
| `ADD PRIMARY KEY` | Y | Y | Y |
| `DROP PRIMARY KEY` | Y | N | Y |
| `CREATE TABLE` | Y | N | Y |
| `DROP TABLE` | Y | N | Y |
| `ADD COLUMN` | Y | N | Y |
| `DROP COLUMN` | Y | N | Y |
| `TRUNCATE TABLE` | Y | N | Y |
| `MODIFY COLUMN` | Y | N | Y |
| `RENAME TABLE` | Y | N | Y |
| `ALTER COLUMN DEFAULT VALUE` | Y | N | Y |
| `ALTER TABLE COMMENT` | Y | N | Y |
| `RENAME INDEX` | Y | N | Y |
| `ADD PARTITION` | Y | N | Y |
| `DROP PARTITION` | Y | N | Y |
| `TRUNCATE PARTITION` | Y | N | Y |
| `CREATE VIEW` | Y | N | Y |
| `DROP VIEW` | Y | N | Y |
| `ALTER TABLE CHARACTER SET` | Y | N | Y |
| `RECOVER TABLE` | Y | N | Y |
| `REBASE AUTO ID` | Y | N | Y |
| `ALTER TABLE INDEX VISIBILITY` | Y | N | Y |
| `EXCHANGE PARTITION` | Y | N | Y |
| `REORGANIZE PARTITION` | Y | N | Y |
| `ALTER TABLE TTL` | Y | N | Y |
| `ALTER TABLE REMOVE TTL` | Y | N | Y |

## DDL replication considerations

Expand Down
16 changes: 12 additions & 4 deletions markdown-pages/en/tidb/master/ticdc/ticdc-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,17 +73,25 @@ The components in the architecture diagram are described as follows:

As shown in the architecture diagram, TiCDC supports replicating data to TiDB, MySQL, Kafka, and storage services.

## Valid index

Generally, TiCDC only replicates tables that have at least one valid index to the downstream. If an index in a table meets one of the following requirements, it is valid.

- A primary key (`PRIMARY KEY`) is a valid index.
- A unique index (`UNIQUE INDEX`) is valid if every column of the index is explicitly defined as non-nullable (`NOT NULL`) and the index does not have a virtual generated column (`VIRTUAL GENERATED COLUMNS`).

> **Note:**
>
> When you set [`force-replicate`](/ticdc/ticdc-changefeed-config.md#force-replicate) to `true`, TiCDC will forcibly [replicate tables without a valid index](/ticdc/ticdc-manage-changefeed.md#replicate-tables-without-a-valid-index).
## Best practices

- When you use TiCDC to replicate data between two TiDB clusters, if the network latency between the two clusters is higher than 100 ms:

- For TiCDC versions earlier than v6.5.2, it is recommended to deploy TiCDC in the region (IDC) where the downstream TiDB cluster is located.
- With a series of improvements introduced starting from TiCDC v6.5.2, it is recommended to deploy TiCDC in the region (IDC) where the upstream TiDB cluster is located.

- TiCDC only replicates tables that have at least one valid index. A valid index is defined as follows:

- A primary key (`PRIMARY KEY`) is a valid index.
- A unique index (`UNIQUE INDEX`) is valid if every column of the index is explicitly defined as non-nullable (`NOT NULL`) and the index does not have a virtual generated column (`VIRTUAL GENERATED COLUMNS`).
- Each table to be replicated by TiCDC has at least one [valid index](#valid-index).

- To ensure eventual consistency when using TiCDC for disaster recovery, you need to configure [redo log](/ticdc/ticdc-sink-to-mysql.md#eventually-consistent-replication-in-disaster-scenarios) and ensure that the storage system where the redo log is written can be read normally when a disaster occurs in the upstream.

Expand Down

0 comments on commit 3d488ae

Please sign in to comment.