aerospike-SPDK patch based on 5.5.0.4

[k-hiwada]

What is aerospike-SPDK patch

The purpose of this patch, namely aerospike-SPDK patch, is to enable Aerospike users and developers to evaluate the merit of IO stack optimization designed for low latency SSDs, such as KIOXIA FL6. The patch is expected to reduce CPU cost for IO operations thanks to SPDK and expected to reduce CPU cost spent in pthread-switching thanks to user-level light-weight-context library, so that Aerospike users who uses low latency SSDs can get better transaction performance per server and better latency.

Main changes

(1) We abstract io-operations, such as pread/pwrite, in order to support other io methods, in order to support SPDK, a high-performance user-level-IO library, and implement SPDK-IO-backend. Since SPDK employs user-level asynchronous IO scheme, (2) we also abstract the thread layer, in order to support user-level context switching, and we implement Lthread backend. User-level context switching contributes to reducing context switching cost, but it often not compatible with blocking operations, such as blocking-io operations and resource locking operation like pthread_mutex_lock. In order to enable resource-locking on top of user-level context switching, (3) we also implement non-blocking version of mutex_lock, by using try_lock and yield.

Known problems

• Because of the adaptation to the SPDK library, we need bear-metal server to run aerospike-server. This problem could be solved by making other IO backend using io_uring or other IO libraries.
• SPDK/Lthread version of worker threads tend to occupy CPUs even if there are no requests. We'd better solve this problem by improving thread scheduler for user-level contexts.

Best Regards,