onnxruntime

This paper presents a systems study comparing C++ and PyTorch inference runtimes for reward model scoring in RLHF pipelines, finding that ONNXRuntime provides speedups on CPU while torch.compile leads on GPU, with batching strategy mattering more than language or runtime.