Repository navigation
Add business QoS configuration for RDMA and URMA - #93
Draft
Spicy-cream wants to merge 3 commits into
Draft
Spicy-cream wants to merge 3 commits into
Spicy-cream wants to merge 3 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RDMA 的 SL 和 Traffic Class 固定为 0,URMA 的 Jetty priority 在进程内统一选择,业务无法分别配置发送策略。本 PR 增加 Channel 和 Server 级 QoS:客户端控制请求发送方向,服务端控制响应发送方向。
RdmaQosOptions 提供 service_level 0–15 和 traffic_class 0–255,默认均为 0,应用到 QP RTR 地址属性。UrmaQosOptions.priority 默认为 -1,自动选择首个包含 CTP 能力的 priority;同时包含其他路径能力的项也参与选择。显式值范围为 0–15,要求所选设备报告对应的 CTP 能力。配置错误时 Channel 初始化和 Server 启动返回错误。priority 的 SL 映射和调度顺序由设备及网络配置决定。
QoS 纳入连接签名,传递到直连、服务发现、连接池、短连接和服务端 accept 路径,资源重建时重新应用。URMA 预建 Jetty 仅供匹配其 priority 的连接使用,其他 priority 按需创建。v2/v3 握手沿用现有格式。初始化日志显示默认 priority,完成错误日志显示收发方向、完成标识、配置 priority 和实际 Jetty priority,性能示例记录所选配置。
中文和英文文档增加原生非 inline 与 inline 预检、自动 priority 单流基线、显式 priority 单流对照和竞争流量验证步骤。测试分别改变请求、响应方向,核对已建立的 URMA 连接和实际 priority,再结合设备队列计数器、吞吐和 P99 检查调度效果。
WSL Ubuntu 下生产静态库、RDMA 和 URMA 性能示例构建通过。Channel 87 项、RDMA mock 95 项、URMA mock 21 项测试通过,共 203 项。新增测试覆盖组合路径能力的自动选择,以及收发访问错误时的 priority 保留和诊断信息;7 项 URMA QoS 测试通过随机顺序验证。示例越界值、非 CTP priority 拒绝和 TCP RPC 烟测通过。URMA 单独启用时 endpoint 和 helper 通过 C++14 编译。
已有双机结果来自 PR92 和 UMDK 原生测试。两端内核及安装包版本一致,UMDK 为 26.06.0-B020,libummu 为 1.0.5。PR92 已完成 URMA 协商,服务端 SEND 报本地内存访问错误;原生 send_lat 使用 RM、CTP、共享 JFR、64 字节消息,在非 inline 和 inline 模式下均于服务端报 status 4。故障已在 brpc 之外复现,根因仍待定位。PR93 的硬件 priority 映射和竞争流量效果尚未验证。