dsr1-fp8-1k1k-max-tpt

concurrency 1,024 · 11 days ago

cron

passed

re-run

Commit

SHAa0c6e0b

chore: bump tokenspeed_mla 0.1.1 -> 0.1.6 (#28116)

kpham-sgl·11 days ago

PR #28116 · chore: bump tokenspeed_mla 0.1.1 -> 0.1.6

Run

GitHub Actions27455007552-1

Slurm job5303

GPUs48·prefill 16 / decode 32

ISL / OSL1024 / 1024

Logscron/27455007552-1/1k1k/dsr1-fp8-1k1k-max-tpt/2026-06-13/5303/sa-bench_isl_1024_osl_1024/

Metrics

28 captured

best_of

1.00

burstiness

1.00

completed

10,240

duration

259

max_concurrency

1,024

mean_e2el_ms

24,783ms

mean_itl_ms

1,233ms

mean_tpot_ms

25.25ms

mean_ttft_ms

1,531ms

median_e2el_ms

24,570ms

median_itl_ms

1,272ms

median_tpot_ms

25.34ms

median_ttft_ms

932ms

num_prompts

10,240

output_throughput

36,494tok/s

p99_e2el_ms

31,788ms

p99_itl_ms

1,546ms

p99_tpot_ms

26.01ms

p99_ttft_ms

7,768ms

peak_output_tokens_per_s

44,568s

request_throughput

39.58tok/s

std_e2el_ms

2,183ms

std_itl_ms

196ms

std_tpot_ms

0.71ms

std_ttft_ms

1,704ms

total_input_tokens

9,440,068

total_output_tokens

9,440,624

total_token_throughput

72,985tok/s