Can we reconstruct a closed-source LLM tokenizer using only two oracles from the chat API?
Summary
This paper explores the feasibility of reconstructing the tokenizer of a closed-source large language model using only two oracle queries from its chat API, potentially revealing internal representation details.
Similar Articles
Stealing Reasoning Traces from Proprietary LLM APIs
This paper demonstrates a method to extract hidden reasoning traces from proprietary LLM APIs (Anthropic, OpenAI, Google) by replaying encrypted chain-of-thought blocks into weaker, jailbroken sibling models, recovering the stronger model's raw reasoning verbatim without attacking it directly.
Ask your LLM
The article discusses how to effectively query or interact with Large Language Models (LLMs) for various applications.
Don't let the LLM speak, just probe it (8 minute read)
The article introduces a technique that extracts hidden states from an LLM at the last prompt token to perform classification without text generation, using a small MLP to read the model's internal decision, enabling fast and cheap zero-shot classifiers.
Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models
This survey paper reviews decoding methods for large language models and vision-language models, focusing on inference-time approaches to enhance efficiency and control generation. It identifies paradigms, highlights challenges, and discusses future research directions.
ORCA: Evaluating LLMs on Data Science Code Translation
This paper introduces ORCA, a comprehensive benchmark for evaluating Large Language Models on data science code translation tasks, highlighting challenges and proposing an intent-augmented method to improve performance.