Unauthorized access to Hugging Face Spaces secrets

Hugging Face detected unauthorized access to Spaces, its platform for creating, sharing and hosting AI models and resources, affecting the secrets store that holds credentials for those apps. The company disclosed the intrusion in a blog post dated 31 May 2024 and said a subset of Spaces secrets may have been read without authorization.

What happened

Spaces secrets are the private values, such as keys and tokens, that apps on the platform use to reach protected accounts, tools and development environments. Hugging Face said its team spotted the unauthorized access in the week before the disclosure. The company has not published how the intruder got in or which secrets were read.

Impact

The number of affected users, Spaces or secrets is not publicly known; TechCrunch reported that this was unclear at the time. Hugging Face revoked a number of HF tokens found in the affected secrets and emailed the owners of revoked tokens. The company advised users to refresh any key or token, not only HF tokens.

Response

Hugging Face brought in outside forensic specialists and reported the incident to law enforcement and data protection authorities. It removed organization tokens entirely, moved Spaces secrets to a key management service, expanded detection and automatic invalidation of leaked tokens, and made fine-grained access tokens the default.

Lessons for security leaders

Hosted AI platforms often end up holding cloud keys and API tokens that developers paste in for convenience. Find out which of your teams store credentials in third-party ML hosting services, and require least-privilege, short-lived tokens for those integrations. After a provider discloses a secrets incident, rotate every credential stored there rather than only those the provider revoked, and check logs for use of the old values.