Tag
The article introduces dp_knapsack_sliding_hirschberg, a new activation memory budget solver for PyTorch that reduces peak RAM usage during memory planning by up to 20x compared to the default dp_knapsack solver, explaining how 0/1 knapsack dynamic programming is used to choose which operations to save or recompute.
Jakub Pachocki, chief scientist at OpenAI, addresses concerns about unmonitorability by stating that the computation graph depth in frontier models like Astra and GPT-4 is similar, and OpenAI emphasizes chain-of-thought monitoring.