Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

std.net: use send/recv for streams, support vectorized network io on windows #19751

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

truemedian
Copy link
Contributor

@truemedian truemedian commented Apr 23, 2024

The primary goal of this change is to use send and recv for network streams, this also enables passing per-message flags like MSG_NOSIGNAL instead of installing a SIGPIPE handler.

  • Alongside using recv and send, windows now uses WSARecvFrom and WSASendTo for readv and writev. This enables vectorized network I/O on windows.

Introduces a mostly platform agnostic iovec as std.net.iovec and std.net.iovec_const. This provides a singular interface to vectorized network I/O on windows and posix.

  • The only problem this introduces is windows using u32 for lengths and posix using usize. This requires any code currently using usize to @intCast into the length field.

Removes std.os.windows.ws2_32.WSARecvMsg, as it is not present in ws2_32.dll and must be fetched via WSAIoctl.

  • The sendto and recvfrom wrappers around WSASendTo and WSARecvFrom have been removed from std.os.windows. These functions are already provided by ws2_32 and are not needed.

A os-agnostic IoVec struct (re #7699) would likely be more convienent to use, but I could not design an interface that both works well and is reasonable to interact with.

A potential problem this PR introduces is passing WSABUF_const to WSASendTo and WSASend is that this type doesn't actually exist in windows. Windows makes no guarantees that the buffers will not be modified.

This enables the "general client/server API coverage" test on windows. The comment on the skip claims that the test was never passing, however during my tests the test passed 100% of the time.

Depends on #19768.

@truemedian truemedian force-pushed the std-net-sendrecv branch 2 times, most recently from 7e204ba to 74a9a6a Compare April 23, 2024 20:05
@truemedian truemedian changed the title std.net: use send/recv for streams, support vectorized io on windows std.net: use send/recv for streams, support vectorized network io on windows Apr 24, 2024
lib/compiler/objcopy.zig Outdated Show resolved Hide resolved
@truemedian truemedian force-pushed the std-net-sendrecv branch 5 times, most recently from b8dea14 to 5dae7fd Compare April 28, 2024 19:33
The primary goal of this change is to use `send` and `recv` for network
streams, this also enables passing per-message flags like `MSG_NOSIGNAL`
instead of installing a SIGPIPE handler.

- Alongside using recv and send, windows now uses WSARecvFrom and WSASendTo
  for readv and writev. This enables vectorized network I/O on windows.

Introduces a mostly platform agnostic iovec as `std.net.IoSlice` and
`std.net.IoSliceConst`. This provides a singular interface to vectorized
network I/O on windows and posix.

- The only problem this introduces is windows using `u32` for lengths
  and posix using `usize`. So windows is limited to 4GB slices while posix is not.

Removes std.os.windows.ws2_32.WSARecvMsg, as it is not present in ws2_32.dll
and must be fetched via WSAIoctl.

- The `sendto` and `recvfrom` wrappers around `WSASendTo` and `WSARecvFrom`
  have been removed from `std.os.windows`. These functions are already
  provided by `ws2_32` and are not needed.

This enables the "general client/server API coverage" test on windows.
The comment on the skip claims that the test was never passing, however
during my tests the test passed 100% of the time.
@daurnimator
Copy link
Collaborator

For what it's worth, the const-ness for IoSlice gets more complicated
with e.g. sendmsg where we need msghdr and msghdr_const
and then again with e.g. sendmmsg where we have e.g. mmsghdr_const vs msghdr_const

@truemedian
Copy link
Contributor Author

truemedian commented Apr 29, 2024

I decided to avoid writing definitions for msghdr and mmsghdr because nothing in std.net needs them (right now, they are necessary for sending flags with iovecs, but that can be easily hidden). I'm not sure a cross platform definition of msghdr is even necessary for TCP because WSASend/WSARecv already do vectorized IO and don't enable any extra features. And outside of Windows, []IoSlice can be ptrCast into an []iovec because they're guaranteed to have the same representation

std.net.Stream readv and writev are now implemented with sendmsg and
recvmsg.
This primarily removes strictly file-related errors from the error sets
and removes a layer of indirection in the kernel.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants