tidb-server v2.1.0-rc.1

• SQL Optimizer

  • Fix the issue that a wrong result is returned after the correlated subquery is decorrelated in some cases #6972
  • Optimize the output result of Explain #7011#7041
  • Optimize the choosing strategy of the outer table for IndexJoin #7019
  • Remove the Plan Cache of the non-PREPARE statement #7040
  • Fix the issue that the INSERT statement is not parsed and executed correctly in some cases #7068
  • Fix the issue that the IndexJoin result is not correct in some cases #7150
  • Fix the issue that the NULL value cannot be found using the unique index in some cases #7163
  • Fix the range computing issue of the prefix index in UTF-8 #7194
  • Fix the issue that result is not correct caused by eliminating the Project operator in some cases #7257
  • Fix the issue that USE INDEX(PRIMARY) cannot be used when the primary key is an integer #7316
  • Fix the issue that the index range cannot be computed using the correlated column in some cases #7357

• SQL Execution Engine

  • Fix the issue that the daylight saving time is not computed correctly in some cases #6823
  • Refactor the aggregation function framework to improve the execution efficiency of the Stream and Hash aggregation operators #6852
  • Fix the issue that the Hash aggregation operator cannot exit normally in some cases #6982
  • Fix the issue that BIT_AND/BIT_OR/BIT_XOR does not handle the non-integer data correctly #6994
  • Optimize the execution speed of the REPLACE INTO statement and increase the performance nearly 10 times #7027
  • Optimize the memory usage of time type data and decrease the memory usage of the time type data by fifty percent #7043
  • Fix the issue that the returned result is mixed with signed and unsigned integers in the UNION statement is not compatible with MySQL #7112
  • Fix the panic issue caused by the too much memory applied by LPAD/RPAD/TO_BASE64/FROM_BASE64/REPEAT #7171 #7266 #7409 #7431
  • Fix the incorrect result when MergeJoin/IndexJoin handles the NULL value #7255
  • Fix the incorrect result of Outer Join in some cases #7288
  • Improve the error message of Data Truncated to facilitate locating the wrong data and the corresponding field in the table #7401
  • Fix the incorrect result for decimal in some cases #7001 #7113 #7202 #7208
  • Optimize the point select performance #6937
  • Prohibit the isolation level of Read Commited to avoid the underlying problem #7211
  • Fix the incorrect result of LTRIM/RTRIM/TRIM in some cases #7291
  • Fix the issue that the MaxOneRow operator cannot guarantee that the returned result does not exceed one row #7375
  • Divide the Coprocessor requests with too many ranges #7454

• Statistics

  • Optimize the mechanism of statistics dynamic collection #6796
  • Fix the issue that Auto Analyze does not work when data is updated frequently #7022
  • Decrease the Write conflicts during the statistics dynamic update process #7124
  • Optimize the cost estimation when the statistics is incorrect #7175
  • Optimize the row count estimation #7233

• Server

  • Fix the bug in loading privilege information #6976
  • Fix the issue that the Kill command is too strict with privilege check #6954
  • Fix the issue of removing some binary numeric types #6922
  • Shorten the output log #7029
  • Handle the mismatchClusterID issue #7053
  • Add the advertise-address configuration item #7078
  • Add the GrpcKeepAlive option #7100
  • Add the connection or Token time monitor #7110
  • Optimize the data decoding performance #7149
  • Add the PROCESSLIST table in INFORMMATION_SCHEMA #7236
  • Fix the order issue when multiple rules are hit in verifying the privilege #7211
  • Change some default values of encoding related system variables to UTF-8 #7198
  • Make the slow query log show more detailed information #7302
  • Support registering tidb-serverrelated information in PD and obtaining this information by HTTP API #7082

• Compatibility

  • Support Session variables warning_count and error_count #6945
  • Add Scope check when reading the system variables #6958
  • Support the MAX_EXECUTION_TIME syntax #7012
  • Support more statements of the SET syntax #7020
  • Add validity check when setting system variables #7117
  • Add the verification of the number of PlaceHolders in the Prepare statement #7162
  • Support set character_set_results = null #7353
  • Support the flush status syntax #7369
  • Fix the column size of SET and ENUM types in information_schema #7347
  • Support the NATIONAL CHARACTER syntax of statements for creating a table #7378
  • Support the CHARACTER SET syntax in the LOAD DATA statement #7391
  • Fix the column information of the SET and ENUM types #7417
  • Support the IDENTIFIED WITH syntax in the CREATE USER statement #7402
  • Fix the precision losing issue during TIMESTAMP computing process #7418
  • Support the validity verification of more SYSTEM variables #7196
  • Fix the incorrect result when the CHAR_LENGTH function computes the binary string #7410
  • Fix the incorrect CONCAT result in a statement involving GROUP BY #7448
  • Fix the imprecise type length issue when casting the DECIMAL type to the STRING type #7451

• DML

  • Fix the stability issue of the Load Data statement #6927
  • Fix the memory usage issue when performing some Batch operations #7086
  • Improve the performance of the Replace Into statement #7027
  • Fix the inconsistent precision issue when writing CURRENT_TIMESTAMP #7355

• DDL

  • Improve the method of DDL judging whether Schema is synchronized to avoid misjudgement in some cases #7319
  • Fix the SHOW CREATE TABLE result in adding index process #6993
  • Allow the default value of text/blob/json to be NULL in non-restrict sql-mode #7230
  • Fix the ADD INDEX issue in some cases #7142
  • Increase the speed of adding UNIQUE-KEY index operation largely #7132
  • Fix the truncating issue of the prefix index in UTF-8 character set #7109
  • Add the environment variable tidb_ddl_reorg_priority to control the priority of the add-index operation #7116
  • Fix the display issue of `AUTO-INCREME...

tidb-server v2.0.6

Improvements

• Make “set system variable” log shorter to save disk space #7031
• Record slow operations during the execution of ADD INDEX in the log, to make troubleshooting easier #7083
• Reduce transaction conflicts when updating statistics #7138
• Improve the accuracy of row count estimation when the values pending to be estimated exceeds the statistics range #7185
• Choose the table with a smaller estimated row count as the outer table for Index Join to improve its execution efficiency #7277
• Add the recover mechanism for panics occurred during the execution of ANALYZE TABLE, to avoid that the tidb-server is unavailable caused by abnormal behavior in the process of collecting statistics #7228
• Return NULL and the corresponding warning when the results of RPAD/LPAD exceed the value of the max_allowed_packet system variable, compatible with MySQL #7244
• Set the upper limit of placeholders count in the PREPARE statement to 65535, compatible with MySQL #7250

Bug Fixes

• Fix the issue that the DROP USER statement is incompatible with MySQL behavior in some cases #7014
• Fix the issue that statements like INSERT/LOAD DATA meet OOM aftering opening tidb_batch_insert #7092
• Fix the issue that the statistics fail to automatically update when the data of a table keeps updating #7093
• Fix the issue that the firewall breaks inactive gPRC connections #7099
• Fix the issue that prefix index returns a wrong result in some scenarios #7126
• Fix the panic issue caused by outdated statistics in some scenarios #7155
• Fix the issue that one piece of index data is missed after the ADD INDEX operation in some scenarios #7156
• Fix the wrong result issue when querying NULL values using the unique index in some scenarios #7172
• Fix the messy code issue of the DECIMAL multiplication result in some scenarios #7212
• Fix the wrong result issue of DECIMAL modulo operation in some scenarios #7245
• Fix the issue that the UPDATE/DELETE statement in a transaction returns a wrong result under some special sequence of statements #7219
• Fix the panic issue of the UNION ALL/UPDATE statement during the process of building the execution plan in some scenarios #7225
• Fix the issue that the range of prefix index is calculated incorrectly in some scenarios #7231
• Fix the issue that the LOAD DATA statement fails to write the binlog in some scenarios #7242
• Fix the wrong result issue of SHOW CREATE TABLE during the execution process of ADD INDEX in some scenarios #7243
• Fix the issue that panic occurs when Index Join does not initialize timestamps in some scenarios #7246
• Fix the false alarm issue when ADMIN CHECK TABLE mistakenly uses the timezone in the session #7258
• Fix the issue that ADMIN CLEANUP INDEX does not clean up the index in some scenarios #7265
• Disable the Read Committed isolation level #7282

tidb-server v2.0.5

New Features

• Add the tidb_disable_txn_auto_retry system variable which is used to disable the automatic retry of transactions #6877

Improvements

• Optimize the cost calculation of Selection to make the result more accurate #6989
• Select the query condition that completely matches the unique index or the primary key as the query path directly #6966
• Execute necessary cleanup when failing to start the service #6964
• Handle \N as NULL in the Load Data statement #6962
• Optimize the code structure of CBO #6953
• Report the monitoring metrics earlier when starting the service #6931
• Optimize the format of slow queries by removing the line breaks in SQL statements and adding user information #6931
• Support multiple asterisks in comments #6931

Bug Fixes

• Fix the issue that KILL QUERY always requires SUPER privilege #6931
• Fix the issue that users might fail to login when the number of users exceeds 1024 #6986
• Fix an issue about inserting unsigned float/double data #6940
• Fix the compatibility of the COM_FIELD_LIST command to resolve the panic issue in some MariaDB clients #6929
• Fix the CREATE TABLE IF NOT EXISTS LIKE behavior #6928
• Fix an issue in the process of TopN pushdown #6923
• Fix the ID record issue of the currently processing row when an error occurs in executing Add Index #6903

tidb-server v2.1.0-beta

• SQL Optimizer
  • Optimize the selection range of Index Join to improve the execution performance
  • Optimize correlated subquery, push down Filter, and extend the index range, to improve the efficiency of some queries by orders of magnitude
  • Support Index Hint and Join Hint in the UPDATE and DELETE statements
  • Validate Hint TIDM_SMJ when no available index exists
  • Support pushdown of the ABS, CEIL, FLOOR, IS TRUE, and IS FALSE functions
  • Handle the IF and IFNULL functions especially in the constant folding process
• SQL Execution Engine
  • Implement parallel Hash Aggregate operators and improve the computing performance of Hash Aggregate by 350% in some scenarios
  • Implement parallel Project operators and improve the performance by 74% in some scenarios
  • Read the data of the Inner table and Outer table of Hash Join concurrently to improve the execution performance
  • Fix incorrect results of INSERT … ON DUPLICATE KEY UPDATE … in some scenarios
  • Fix incorrect results of the CONCAT_WS, FLOOR, CEIL, and DIV built-in functions
• Server
  • Add the HTTP API to scatter the distribution of table Regions in the TiKV cluster
  • Add the auto_analyze_ratio system variable to control the threshold value of automatic Analyze
  • Add the HTTP API to control whether to open the general log
  • Add the HTTP API to modify the log level online
  • Add the user information in the general log and the slow query log
  • Support the server side cursor
• Compatibility
  • Support more MySQL syntax
  • Make the bit aggregate function support the ALL parameter
  • Support the SHOW PRIVILEGES statement
• DML
  • Decrease the memory usage of the INSERT INTO SELECT statement
  • Fix the performance issue of PlanCache
  • Add the tidb_retry_limit system variable to control the automatic retry times of transactions
  • Add the tidb_disable_txn_auto_retry system variable to control whether the transaction tries automatically
  • Fix the accuracy issue of the written data of the time type
  • Support the queue of locally conflicted transactions to optimize the conflicted transaction performance
  • Fix Affected Rows of the UPDATE statement
  • Optimize the statement performance of insert ignore on duplicate key update
• DDL
  • Optimize the execution speed of the CreateTable statement
  • Optimize the execution speed of ADD INDEX and improve it greatly in some scenarios
  • Fix the issue that the number of added columns by Alter table add column exceeds the limit of the number of table columns
  • Fix the issue that DDL job retries lead to an increasing pressure on TiKV in abnormal conditions
  • Fix the issue that TiDB continuously reloads the schema information in abnormal conditions
  • Do not output the FOREIGN KEY related information in the result of SHOW CREATE TABLE
  • Support the select tidb_is_ddl_owner() statement to facilitate judging whether TiDB is DDL Owner
  • Fix the issue that the index is deleted in the Year type in some scenarios
  • Fix the renaming table issue in the concurrent execution scenario
  • Support the AlterTableForce syntax
  • Support the AlterTableRenameIndex syntax with FromKey and ToKey
  • Add the table name and database name in the output information of admin show ddl jobs

tidb-server v2.0.4

New Features

• Support the ALTER TABLE t DROP COLUMN a CASCADE syntax
• Support configuring the value of tidb_snapshot to TSO

Improvements

• Refine the display of statement types in monitoring items
• Optimize the accuracy of query cost estimation
• Configure the backoff max delay parameter of gRPC
• Support configuring the memory threshold of a single statement in the configuration file
• Refactor the error of Optimizer

Bug Fixes

• Fix the side effects of the Cast Decimal data
• Fix the wrong result issue of the Merge Join operator in specific scenarios
• Fix the issue of converting the Null object to String
• Fix the issue of casting the JSON type of data to the JSON type
• Fix the issue that the result order is not consistent with MySQL in the condition of Union + OrderBy
• Fix the compliance rules issue when the Union statement checks the Limit/OrderBy clause
• Fix the compatibility issue of the Union All result
• Fix a bug in predicate pushdown
• Fix the compatibility issue of the Union statement with the For Update clause
• Fix the issue that the concat_ws function mistakenly truncates the result

tidb-server v2.1.0-alpha

add pull request template (#6718)

tidb-server v2.0.3

New Features

• Support modifying the log level online
• Support the COM_CHANGE_USER command
• Support using the TIME type parameters under the binary protocol

Improvements

• Optimize the cost estimation of query conditions with the BETWEEN expression
• Do not display the FOREIGN KEY information in the result of SHOW CREATE TABLE
• Optimize the cost estimation for queries with the LIMIT clause

Bug Fixes

• Fix the issue about the YEAR type as the unique index
• Fix the issue about ON DUPLICATE KEY UPDATE in conditions without the unique index
• Fix the compatibility issue of the CEIL function
• Fix the accuracy issue of the DIV calculation in the DECIMAL type
• Fix the false alarm of ADMIN CHECK TABLE
• Fix the panic issue of MAX/MIN under specific expression parameters
• Fix the issue that the result of JOIN is null in special conditions
• Fix the IN expression issue when building and querying Range
• Fix a Range calculation issue when using Prepare to query and Plan Cache is enabled
• Fix the issue that the Schema information is frequently loaded in abnormal conditions

tidb-server v2.0.2

• Fix the issue of pushing down the Decimal division expression
• Support using the USE INDEX syntax in the Delete statement
• Forbid using the shard_row_id_bits feature in columns with Auto-Increment
• Add the timeout mechanism for writing Binlog

tidb-server v2.0.1

• Update the progress of Add Index to the DDL job information in real time
• Add the tidb_auto_analyze_ratio session variable to control the threshold value of automatic statistics update
• Fix an issue that not all residual states are cleaned up when the transaction commit fails
• Fix a bug about adding indexes in some conditions
• Fix the correctness related issue when DDL modifies surface operations in some concurrent scenarios
• Fix a bug that the result of LIMIT is incorrect in some conditions
• Fix a capitalization issue of the ADMIN CHECK INDEX statement to make its index name case insensitive
• Fix a compatibility issue about the UNION statement
• Fix a compatibility issue when inserting data of TIME type
• Fix a goroutine leak issue caused by copIteratorTaskSender in some conditions
• Add an option for TiDB to control the behaviour of Binlog failure
• Refactor the Coprocessor slow log，distinguish the scenario of tasks with long processing time and long waiting time
• Log nothing when meeting MySQL protocol handshake error. Avoid too many logs caused by load balancer keep alive mechanism
• Refine “Out of range value for column” error message
• Fix a bug when there is a subquery in an Update statement
• Change the behaviour of handling SIGTERM, do not wait for all queries to terminate anymore

tidb-server v2.0.0

• SQL Optimizer
  • Use more compact data structure to reduce the memory usage of statistics information
  • Speed up the loading statistics information when starting a tidb-server process
  • Support updating statistics information dynamically [experimental]
  • Optimize the cost model to provide more accurate query cost evaluation
  • Use Count-Min Sketch to estimate the cost of point queries more accurately
  • Support analyzing more complex conditions to make full use of indexes
  • Support manually specifying the Join order using the STRAIGHT_JOIN syntax
  • Use the Stream Aggregation operator when the GROUP BY clause is empty to improve the performance
  • Support using indexes for the MAX/MIN function
  • Optimize the processing algorithms for correlated subqueries to support decorrelating more types of correlated subqueries and transform them to Left Outer Join
  • Extend IndexLookupJoin to be used in matching the index prefix
• SQL Execution Engine
  • Refactor all operators using the Chunk architecture, improve the execution performance of analytical queries, and reduce memory usage.There is a significant improvement in the TPC-H benchmark result.
  • Support the Streaming Aggregation operators pushdown
  • Optimize the Insert Into Ignore statement to improve the performance by over 10 times
  • Optimize the Insert On Duplicate Key Update statement to improve the performance by over 10 times
  • Optimize Load Data to improve the performance by over 10 times
  • Push down more data types and functions to TiKV
  • Support computing the memory usage of physical operators, and specifying the processing behavior in the configuration file and system variables when the memory usage exceeds the threshold
  • Support limiting the memory usage by a single SQL statement to reduce the risk of OOM
  • Support using implicit RowID in CRUD operations
  • Improve the performance of point queries
• Server
  • Support the Proxy Protocol
  • Add more monitoring metrics and refine the log
  • Support validating the configuration files
  • Support obtaining the information of TiDB parameters through HTTP API
  • Resolve Lock in the Batch mode to speed up garbage collection
  • Support multi-threaded garbage collection
  • Support TLS
• Compatibility
  • Support more MySQL syntaxes
  • Support modifying the lower_case_table_names system variable in the configuration file to support the OGG data synchronization tool
  • Improve compatibility with the Navicat management tool
  • Support displaying the table creating time in Information_Schema
  • Fix the issue that the return types of some functions/expressions differ from MySQL
  • Improve compatibility with JDBC
  • Support more SQL Modes
• DDL
  • Optimize the Add Index operation to greatly improve the execution speed in some scenarios
  • Attach a lower priority to the Add Index operation to reduce the impact on online business
  • Output more detailed status information of the DDL jobs in Admin Show DDL Jobs
  • Support querying the original statements of currently running DDL jobs using Admin Show DDL Job Queries JobID
  • Support recovering the index data using Admin Recover Index for disaster recovery
  • Support modifying Table Options using the Alter statement