Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[catpowder] Enhancement: add buffer pooling and multiple ring entries to XDP backend #1495

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

kyleholohan
Copy link
Contributor

This PR allows for multiple RX/TX ring entries in the catpowder XDP backend. It also hooks the user memory region (UmemReg) into buffer pools to prevent additional buffer copies and allow for more efficient buffer allocation.

This PR addresses both #1438 and #1437.

One potential downside of this approach: receive buffers are a limited resource, and if the client holds too many receive buffers for too long, the buffer pool may become exhausted. In this case, the system will start to drop packets. Possibly this behavior should be configurable (i.e., the receive buffers should be copied out of the receive pool unless a "zero copy" config is enabled).

@kyleholohan kyleholohan added the enhancement Enhancement Request on an Existing Feature label Jan 31, 2025
@kyleholohan kyleholohan force-pushed the enhancement-catpowder-buffer-pool branch 2 times, most recently from ad40826 to a3d3e1b Compare February 12, 2025 17:29
@kyleholohan kyleholohan force-pushed the enhancement-catpowder-buffer-pool branch from a3d3e1b to be698e1 Compare February 12, 2025 18:08
Copy link

libos = catpowder
commit id = be698e1

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::inetstack::poll;inetstack::layer4::poll_once 21757259 626 251 12698854183 46 12691656874 84
bgc::inetstack::poll 10878629 1459 584 14981144829 54 2282290646 15
ioc::network::libos::push 16145 3286 1331 47213825 0 47213825 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch 5707 2894 1122 7319297 0 7315704 0
bgc::inetstack::tcp::established::background;tcp::established::background::retransmitter 11438 455 181 5470433 0 5470433 0
ioc::network::libos::pushto 671 6278 2591 3780327 0 3780327 0
ioc::network::libos::pop 9180 487 194 3352893 0 3352893 0
bgc::inetstack::tcp::established::background 11451 1909 751 12812274 0 3270787 0
bgc::inetstack::tcp::established::background;tcp::established::background::acknowledger 11451 734 289 3151298 0 3151298 0
ioc::network::libos::connect 42 42699 17364 1741675 0 1741675 0
bgc::inetstack::tcp::established::background;tcp::established::background::sender 11395 865 332 919756 0 919756 0
demikernel::sgaalloc 6503 152 61 879807 0 879807 0
demikernel::sgafree 8748 124 50 649175 0 649175 0
bgc::inetstack::tcp::passiveopen::background 42 17023 6742 451022 0 451022 0
ioc::network::libos::close 46 6708 2693 301483 0 301483 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch;udp::receive 192 341 138 70667 0 70667 0
ioc::network::libos::accept 39 2397 944 51544 0 51544 0
bgc::inetstack::arp::background 7 2195 875 13843 0 13843 0
bgc::inetstack::icmp::background 1 2386 952 2867 0 2867 0

Copy link

libos = catnap
commit id = be698e1

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::catnap::transport::epoll 13692169 2791 1162 6935386480 100 6935386480 100
ioc::network::libos::push 17600 466 194 7499080 0 7499080 0
ioc::network::libos::pop 15335 481 201 5574954 0 5574954 0
ioc::network::libos::connect 50 155188 63761 2754716 0 2754716 0
demikernel::sgaalloc 11292 123 52 1208894 0 1208894 0
demikernel::sgafree 11540 79 33 903230 0 903230 0
ioc::network::libos::pushto 1561 381 164 617151 0 617151 0
ioc::network::libos::accept 25 13191 5576 415904 0 415904 0
ioc::network::libos::close 18 22926 9566 324501 0 324501 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement Request on an Existing Feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant