@JeffLadish: We just discovered almost a million public URLs that OpenAI’s agents left behind when hacking Hugging Face, leaking cre…

We just discovered almost a million public URLs that OpenAI’s agents left behind when hacking Hugging Face, leaking credentials and attack details that could have allowed anyone who found them to compromise the company. 🧵

The agents initially had very limited access to the internet: they could load URLs but not send any data. Agents created a series of workarounds, using a link-shortener site to create almost a million URLs that, when chained together, let them execute code to hack Hugging Face.

The agents needed a browser to run their attack code. So they used a public screenshot website, which loads a virtual browser and takes a screenshot. But that virtual browser runs code, and so the agents could use it to send malicious payloads to Hugging Face’s servers.

Agents were able to use this link chaining + screenshot service trick to construct and execute extremely large pieces of code, at times chaining together more than 900 links.

To read the result of the requests they made, agents used many different techniques, such as converting a server’s response into a grid of pixels within the screenshot service’s own browser. The screenshot would then capture this grid and return it back as an image.

In one of the public traces, we see an agent encounter the following warning in a README.md. The agent ignored the warning and instead altered the file, adding a malicious configuration change in the header that directs the system to load a malicious file.

We recovered a script an agent used to search Hugging Face’s infrastructure for AWS credentials and other secrets, categorizing these into a list named “LOOT” and ranking them by their value. The agents also accessed and searched Hugging Face’s internal Slack.

Agents tried to access public datasets containing cybergym traces, but were blocked by a CAPTCHA. The agents installed a pre-trained image classifier model to try to bypass the problem, alongside experimenting with other solutions, before eventually abandoning their attempts.

Agents attempted to remove traces of their work. Many recovered payloads contain code to delete files initially used to deliver or trigger programs on Hugging Face workers.

Read our full report here: https://swarmtraces.org

Great work @alexscraping @bug39 @umasiii @SpencerKitts @Cormac_SB @collegraphy @she_llac

Here’s some of the agent actions we observed by reconstructing payloads. You can also explore the data yourself here: https://swarmtraces.org/viewer/

I’ve been extremely impressed with @alexscraping throughout. He reached out a couple weeks ago when he and his team found the initial links. He’s been incredibly thoughtful working with Hugging Face to ensure we redacted sensitive info while making sure the world got to see this