Tag
This paper examines the trade-off between model size and inference compute for grammar-constrained text-to-SQL tasks using small language models, finding that beam search outperforms sample+vote and that scaling model size generally yields better results than scaling inference compute.
This paper investigates the impact of constrained decoding on tool-call abstention, decomposing the effects of grammar constraints into stop and emission components, and evaluates performance on small open-weight models across languages.