Skip to content

Commit

Permalink
pic small
Browse files Browse the repository at this point in the history
  • Loading branch information
wanghenshui committed Jul 21, 2024
1 parent febc95e commit 14f3edf
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 9 deletions.
12 changes: 6 additions & 6 deletions _posts/2024-07-19-2023-qcon-gz.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ kv分离现状
- 利用 Block Cache 实现数据同步


<img src="https://wanghenshui.github.io/assets/antkv1.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/antkv1.png" alt="" width="80%">

新的问题
- 中等大小(如256B)Value 情况下,Scan 仍然比 RocksDB 差很多
Expand All @@ -67,14 +67,14 @@ kv分离现状
- Level N-2 及以下的层级不做重写
- Level N-1 及以上的层级在 Compaction 时重写Value Log Files

<img src="https://wanghenshui.github.io/assets/antkv2.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/antkv2.png" alt="" width="80%">

- 针对 Scan 优化的重写:
- Compaction 过程中,对本轮参与的 Value Log Files 进行重叠记数
- 当发现某文件重叠记数超过阈值,则标记相关文件后续进行重写


<img src="https://wanghenshui.github.io/assets/antkv3.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/antkv3.png" alt="" width="80%">

收益 写入降低30% 但scan提升巨大

Expand All @@ -85,10 +85,10 @@ kv分离现状
Learned Index主要是要设计构建算法,这里需要展开一下


<img src="https://wanghenshui.github.io/assets/antkv-li1.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/antkv-li1.png" alt="" width="80%">


<img src="https://wanghenshui.github.io/assets/antkv-li2.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/antkv-li2.png" alt="" width="80%">

因为实际 SST 保存的 key 为 string 类型,非 integer,因此需要进行转换
- 要求
Expand All @@ -98,7 +98,7 @@ Learned Index主要是要设计构建算法,这里需要展开一下
- 字符串长度是随机的,并且可能很长


<img src="https://wanghenshui.github.io/assets/antkv-li3.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/antkv-li3.png" alt="" width="80%">


Learned Index非常小,读效率非常高
Expand Down
6 changes: 3 additions & 3 deletions _posts/2024-07-19-arch-summit2023-sh.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Workload分析和对应的挑战
- 防止个别超大metric影响整体可用性


<img src="https://wanghenshui.github.io/assets/bytedance-tsdb1.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/bytedance-tsdb1.png" alt="" width="80%">

如何线性扩展
- 二级一致性Hash分区
Expand All @@ -44,7 +44,7 @@ Workload分析和对应的挑战
- 不同维度的Metric可以拥有不同的二级Hash分片数


<img src="https://wanghenshui.github.io/assets/bytedance-tsdc.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/bytedance-tsdc.png" alt="" width="80%">

编码优化

Expand Down Expand Up @@ -87,7 +87,7 @@ Khronos存储引擎
- 兼容Tsdc,最低成本接入现有集群


<img src="https://wanghenshui.github.io/assets/bytedance-khronos1.png" alt="" width="60%">
<img src="https://wanghenshui.github.io/assets/bytedance-khronos1.png" alt="" width="80%">

- 每个Shard内部都是一棵独立的LSMT
- 一共分为三层
Expand Down

0 comments on commit 14f3edf

Please sign in to comment.