Feature request
Support transferring pieces between peers over RDMA, in addition to the existing TCP and QUIC transports in dragonfly-client-storage. When a peer announces rdma as its download protocol, downstream peers should download pieces from it via RDMA using the Vortex protocol, implemented on top of datenlord/async-rdma.
dragonfly-client-storage currently supports two transports for piece exchange between peers:
dragonfly-client-storage/src/client/mod.rs — exposes tcp / quic client modules (TCPClient, QUICClient) that send DownloadPiece / DownloadPersistentPiece / DownloadPersistentCachePiece requests and stream piece content back.
dragonfly-client-storage/src/server/mod.rs — exposes tcp / quic server modules (TCPServer, QUICServer) that parse the Vortex TLV header, dispatch by Tag, and write PieceContent responses with rate limiting and metrics.
The wire format is the Vortex protocol (TLV: Packet ID (1B) | Tag (1B) | Length (4B) | Value (≤4GiB)), with piece sizes up to 1 GiB.
For high-throughput, latency-sensitive scenarios (AI model/dataset distribution, intra-datacenter clusters with RoCE/InfiniBand NICs), kernel TCP/QUIC stacks become the bottleneck. RDMA provides kernel-bypass, zero-copy transfer and significantly lower CPU overhead, which is a natural fit for piece content transfer.
Feature request
Support transferring pieces between peers over RDMA, in addition to the existing TCP and QUIC transports in dragonfly-client-storage. When a peer announces rdma as its download protocol, downstream peers should download pieces from it via RDMA using the Vortex protocol, implemented on top of datenlord/async-rdma.
dragonfly-client-storage currently supports two transports for piece exchange between peers:
dragonfly-client-storage/src/client/mod.rs — exposes tcp / quic client modules (TCPClient, QUICClient) that send DownloadPiece / DownloadPersistentPiece / DownloadPersistentCachePiece requests and stream piece content back.
dragonfly-client-storage/src/server/mod.rs — exposes tcp / quic server modules (TCPServer, QUICServer) that parse the Vortex TLV header, dispatch by Tag, and write PieceContent responses with rate limiting and metrics.
The wire format is the Vortex protocol (TLV: Packet ID (1B) | Tag (1B) | Length (4B) | Value (≤4GiB)), with piece sizes up to 1 GiB.
For high-throughput, latency-sensitive scenarios (AI model/dataset distribution, intra-datacenter clusters with RoCE/InfiniBand NICs), kernel TCP/QUIC stacks become the bottleneck. RDMA provides kernel-bypass, zero-copy transfer and significantly lower CPU overhead, which is a natural fit for piece content transfer.