Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP over io_uring fails on my CentOS 7 machine #29

Open
timvisee opened this issue Jul 2, 2020 · 0 comments
Open

TCP over io_uring fails on my CentOS 7 machine #29

timvisee opened this issue Jul 2, 2020 · 0 comments

Comments

@timvisee
Copy link

timvisee commented Jul 2, 2020

I'm trying to talk TCP using io_uring with rio. Now I got this working fine on my development machine, but it somehow fails to run on a remote server.

I did try some basic debugging but am failing to see what the problem is. I'm looking for any guidance on what I might try to debug this, or on things I should check.

Here's a simplified snippet of the TCP sender:

use std::{env, io, net::TcpStream};

// Set up io_uring
let config = rio::Config::default();
let ring = config.start()?;

// Open TCP stream
let stream = TcpStream::connect("127.0.0.1:6666")?;
stream.set_nonblocking(true)?;

// Keep sending buffer repeatedly
let buf = vec![0; 1024 * 8];
loop {
    let _written = ring.send(stream, &buf).wait()?;
}

On my development machine this successfully sends gigabytes per seconds. On the remote server this fails, it looks like it sends about 2.5MB, after which it fails to send anything more. I'm not seeing any errors or panics. These tests are done on the machine itself, with a sending and receiving process, LAN isn't involved here.

  • Development machine (succeeds):

    # lsb_release -a
    Distributor ID:	Ubuntu
    Description:	Ubuntu 20.04 LTS
    Release:	20.04
    Codename:	focal
    # uname -a
    Linux axiom 5.6.16-050616-generic #202006030730 SMP Wed Jun 3 07:35:14 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
  • Remote server (fails):

    # lsb-release -a
    Distributor ID: CentOS
    Description:    CentOS Linux release 7.8.2003 (Core)
    Release:        7.8.2003
    Codename:       Core
    # uname -a
    Linux localhost.localdomain 5.7.6-1.el7.elrepo.x86_64 #1 SMP Fri Jun 26 02:56:56 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux

Judging by the output of /proc/kallsyms support for io_uring is available in both kernels, as syscalls such as io_uring_setup are listed.

This is about sending data, however, receiving data (read_at) doesn't work either, nothing is received. I did test both the sending and receiving io_uring process by connecting it to a sending/receiving std TcpStream without using io_uring.

What could be happening here? I'm not sure where to look.

Posting this here since I'm using rio. Not sure if there's a better place to be posting this.

@timvisee timvisee changed the title TCP over io_uring fails on one of my machines TCP over io_uring fails on my CentOS 7 machine Jul 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant