Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Valkey Over RDMA transport #477

Open
wants to merge 1 commit into
base: unstable
Choose a base branch
from

Commits on May 31, 2024

  1. Introduce RDMA transport

    Main changes in this patch:
    * implement server side of connection module only, this means we can *NOT*
      compile RDMA support as built-in.
    * add necessary information in README.md
    * support 'CONFIG SET/GET', for example, 'CONFIG Set rdma.port 6380', then
      check this by 'rdma res show cm_id' and valkey-cli(with RDMA support,
      but not implemented in this patch)
    * the full listeners show like():
        listener0:name=tcp,bind=*,bind=-::*,port=6379
        listener1:name=unix,bind=/var/run/valkey.sock
        listener2:name=rdma,bind=xx.xx.xx.xx,bind=yy.yy.yy.yy,port=6379
        listener3:name=tls,bind=*,bind=-::*,port=16379
    
    valgrind test works fine:
    valgrind --track-origins=yes --suppressions=./src/valgrind.sup
             --show-reachable=no --show-possibly-lost=no --leak-check=full
             --log-file=err.txt ./src/valkey-server --port 6379
             --loadmodule src/valkey-rdma.so port=6379 bind=xx.xx.xx.xx
             --loglevel verbose --protected-mode no --server_cpulist 2
             --bio_cpulist 3 --aof_rewrite_cpulist 3 --bgsave_cpulist 3
             --appendonly no
    
    performance test:
    server side: ./src/valkey-server --port 6379 # TCP port 6379 has no conflict with RDMA port 6379
                 --loadmodule src/valkey-rdma.so port=6379 bind=xx.xx.xx.xx bind=yy.yy.yy.yy
                 --loglevel verbose --protected-mode no --server_cpulist 2 --bio_cpulist 3
                 --aof_rewrite_cpulist 3 --bgsave_cpulist 3 --appendonly no
    
    build a valkey-benchmark with RDMA support(not implemented in this patch), run
    on a x86(Intel Platinum 8260) with RoCEv2 interface(Mellanox ConnectX-5):
    client side: ./src/valkey-benchmark -h xx.xx.xx.xx -p 6379 -c 30 -n 10000000 --threads 4
                 -d 1024 -t ping,get,set --rdma
    ====== PING_INLINE ======
    480561.28 requests per second, 0.060 msec avg latency.
    
    ====== PING_MBULK ======
    540482.06 requests per second, 0.053 msec avg latency.
    
    ====== SET ======
    399952.00 requests per second, 0.073 msec avg latency.
    
    ====== GET ======
    443498.31 requests per second, 0.065 msec avg latency.
    
    Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
    pizhenwei committed May 31, 2024
    Configuration menu
    Copy the full SHA
    a9e96f4 View commit details
    Browse the repository at this point in the history