Issues

• [需求] 建议支持https://<ip:port>/v2/xxxxxxxxxxxx/v1/chat/completions格式的大模型接口

  content_check_passedenhancement需求讨论
  AISBench/benchmark#584 · Filter by author yaoyanxiang opened on Sep 30, 2026
  2

  

• [需求] ais_bench_prefix_cache不支持sglang metrics，不支持sglang:prefill_effective_tokens_total等参数

  content_check_passedenhancement需求讨论
  AISBench/benchmark#583 · Filter by author Royce-sun opened on Sep 29, 2026 · AISBench 2026 Q4 Roadmap
  5

  

• [需求] aisbench适配MME数据集精度评测

  content_check_passedenhancement需求讨论
  AISBench/benchmark#582 · Filter by author jschen069 opened on Sep 29, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [疑问] [LiveCodeBench v6] 判题器不支持 sys.stdout.buffer，可能将正确程序判为运行错误

  content_check_passed
  AISBench/benchmark#579 · Filter by author mantuoluozk opened on Sep 29, 2026 · AISBench 2026 Q4 Roadmap
  8

• [疑问] [LiveCodeBench v6] 部分正确答案因非法测试用例或单一参考输出被误判

  content_check_passed
  AISBench/benchmark#576 · Filter by author mantuoluozk opened on Sep 28, 2026 · AISBench 2026 Q4 Roadmap
  7

• [需求] 希望在最后的统计看板体现request rate

  content_check_passedenhancement需求讨论
  AISBench/benchmark#540 · Filter by author qiudepei opened on Sep 14, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] 多模态解析的测试用例报告，需要自动解析多模态文件的分辨率参数

  content_check_passedenhancement需求讨论
  AISBench/benchmark#539 · Filter by author Happystone78 opened on Sep 14, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [主动优化] Agent数据集结果可视化解析+Badcase分类归因

  content_check_passedenhancement需求讨论
  AISBench/benchmark#523 · Filter by author ivanbao9783 opened on Sep 8, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] 希望能够支持现网流量会放的方式验证agent场景推理的处理能力

  content_check_failedenhancementin progress需求讨论
  Task

  AISBench/benchmark#498 · Filter by author chenghaitao1987 opened on Aug 26, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [enhancement] 虚拟内存占用检测是不是按照大小检测 而不是百分比

  content_check_passedenhancement需求讨论
  AISBench/benchmark#480 · Filter by author zhongzhouTan-coder opened on Aug 20, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] 支持量化场景下的压缩测评集快速精度评估能力

  content_check_passedenhancement需求讨论
  AISBench/benchmark#475 · Filter by author flame-hu opened on Aug 18, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] aisbench 支持 I2V视频生成的评测

  content_check_passedenhancement需求讨论
  AISBench/benchmark#470 · Filter by author ethan686 opened on Aug 17, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] agent驱动测评

  content_check_passedenhancement需求讨论
  AISBench/benchmark#435 · Filter by author joejoezrc opened on Aug 4, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求]

  content_check_passedenhancement需求讨论
  AISBench/benchmark#434 · Filter by author joejoezrc opened on Aug 4, 2026 · AISBench 2026 Q4 Roadmap
  5

  

• [RFC] [AISBench] 支持Prefix Cache场景数据集生成及性能测评能力技术方案设计

  content_check_failed
  AISBench/benchmark#432 · Filter by author Libotry opened on Aug 3, 2026 · AISBench 2026 Q4 Roadmap
  2

• [Roadmap] AISBench 2026 Q3 Roadmap

  content_check_failed
  AISBench/benchmark#431 · Filter by author Libotry opened on Aug 3, 2026 · AISBench 2026 Q4 Roadmap
  3

• [需求] 增加向量模型、重排序模型、多模态模型的性能测试

  content_check_failedenhancement需求讨论
  AISBench/benchmark#429 · Filter by author F0undLinks opened on Aug 3, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• 精度测评提供输出长度的分布

  content_check_passedenhancement需求讨论
  AISBench/benchmark#426 · Filter by author MOZhuai opened on Jul 29, 2026 · AISBench 2026 Q4 Roadmap
  4

  

• [feature] mooncaketrace特性支持py3.11 &py3.12

  content_check_passednot_fixed
  AISBench/benchmark#397 · Filter by author SJTUyh opened on Jul 14, 2026 · AISBench 2026 Q4 Roadmap
  3

• [需求] AISBench镜像支持测评agent数据集

  content_check_passedenhancement需求讨论
  AISBench/benchmark#390 · Filter by author SJTUyh opened on Jul 2, 2026 · AISBench 2026 Q4 Roadmap
  4

  

• [疑问] vbench评分结果如何获取单个视频的数据？

  content_check_passedquestionwait for reply
  AISBench/benchmark#348 · Filter by author mikasa955 opened on Jun 18, 2026 · AISBench 2026 Q4 Roadmap
  5

• [需求] 在配置虚拟数据集的时候能直接传入字符，比如52K,这样能自动解析

  content_check_passedenhancementpending
  AISBench/benchmark#275 · Filter by author Nagaloop opened on May 6, 2026 · AISBench 2026 Q4 Roadmap
  2

  

• [需求] 在 AISBench 中添加自动生成指定长度数据集的能力，且有真实语义供 mtp/eagle3/Speculative Inference 特性测试

  content_check_passedenhancement
  AISBench/benchmark#231 · Filter by author xuchi-0808 opened on Apr 4, 2026 · AISBench 2026 Q4 Roadmap
  3

  

• [需求] 希望支持aime2026数据集的精度测试

  content_check_passedenhancement
  AISBench/benchmark#211 · Filter by author chenweiqiang11 opened on Mar 23, 2026 · AISBench 2026 Q4 Roadmap
  4

  

• [需求] --search功能希望能够支持模糊匹配

  content_check_passedenhancementgood first issuehelp wanted
  Feature

  AISBench/benchmark#101 · Filter by author SJTUyh opened on Jan 9, 2026 · AISBench 2026 Q4 Roadmap
  3