v2.0.6

release v2.0.6

v2.1.2

Connectors:

• EXPERIMENTAL PREVIEW: support native postgres_sink in rust. #19328
• Support switching from connector='jdbc' for jdbc postgres sinks to connector='postgres'. #19703
• Performance improvements for native postgres sink. #19688

Fixes:

• Fix file scan s3 access issue. #20080
• Fix time travel garbage collection issue. #20107
• Fix MySQL meta store incompatibility issue. #20099

release v2.1.2

v2.1.1

Please use v2.1.2 instead

#20107 fixed a bug that may lead to data loss.

release v2.1.1

v2.0.5

release v2.0.5

v2.1.0

Please use v2.1.2 instead

#20107 fixed a bug that may lead to data loss.

For installation and running instructions, see Get started.

Main changes

SQL features

• Query syntax:
  • Technical preview: Supports ASOF JOIN to join and find the closest matching record by the event time or another ordered property. #18683
  • Supports AGGREGATE: prefixed scalar function in streaming aggregation. #18205
  • Supports using user-defined aggregate functions as window function. #18181
  • Supports blocking subscription cursors and configuring cursor timeouts. #18675
• SQL commands:
  • Enhances observability of cursors and subscription cursors by improving output results of SHOW SUBSCRIPTION CURSORS and SHOW CURSORS commands. #18896
• SQL functions & operators:
  • Technical Preview: Supports the TVF postgres_query . #18811
• System catalog:
  • Supports pg_catalog.pg_sequence. #18714
  • Supports information_schema.table_constraints. #18716
  • Supports information_schema.schemata. #18709

Connectors

• Breaking change: Changes scan.startup.mode=latest for NATS source connector to start consuming from next available message instead of last one. #18733
• Public preview: Supports shared Kafka sources, which can be disabled by session variable streaming_use_shared_source. #18749
• Supports recursively scanning file sources. #18324
• Supports schemaless ingestion for data in JSON format from Kafka sources by using INCLUDE payload clause. #18437
• Adds a set of options for NATS source connector based on the async_nats crate. #17615
• Adds a required option, consumer.durable_name, for NATS source connector. #18873
• Supports option max_packet_size for MQTT sources. #18520
• Supports option database.encrypt for SQL Server CDC sources. #18912
• Supports ingesting data from a partitioned table for PostgreSQL CDC sources. #18456
• Supports option auto.schema.change for PostgreSQL CDC sources to enable replicating Postgres table schema change. #18760
• Requires upstream table name to also be prefixed with database name when creating a SQL Server CDC table. #18868
• Adds JSON encode for file sinks, allowing users to sink JSON files into object storage. #18744
• Supports create_table_if_not_exists option for Iceberg sink connector. #18362
• Supports WebHDFS sinks. #18293
• Removes option bulk_write_max_entries for MongoDB sink and option default_max_batch_rows for DynamoDB sink. Adds options max_batch_item_nums and max_future_send_nums for DynamoDB sink. #17645
• Sets sink decoupling as the default policy for MongoDB, DynamoDB, and Redis sink connectors. #17645
• Supports option routing_column for ElasticSearch sinks, allowing a column to be set as a routing key. #18698
• Supports specifying batching strategy when sinking data in Parquet format. #18472

Installation and deployment

• Adds a CLI argument of --license-key-path for the meta node, enabling a background task to watch and reload license key from the specified file. #18768

Cluster configuration changes

• When visibility_mode is set to all, the latest uncommitted data will be queried, but consistency is no longer guaranteed between the tables. #18230
• Supports SET TIME ZONE INTERVAL '+00:00' HOUR TO MINUTE as equivalent to SET TIME ZONE UTC. #18705
• The etcd metastore is fully deprecated and unsupported. Users previously utilizing etcd metastore must manually migrate to a SQL backend (PostgreSQL, MySQL, or SQLite) to upgrade to v2.1.0

Full Changelog: v2.0.4...v2.1.0

v2.0.4

release v2.0.4

v2.0.3

release v2.0.3

v2.1.0-rc.2

release v2.1.0-rc.2

v2.1.0-rc.1

release v2.1.0-rc.1

v2.0.2

release v2.0.2