Cohere announces confidential computing public beta for Model Vault

Cohere says customer-side software checks cryptographically signed evidence of the processing environment’s hardware, firmware and software against an approved security policy before sending sensitive data. If the environment fails that check, the client refuses to send the request.

Protecting requests while models run

According to Cohere’s announcement, application payloads remain encrypted through intermediary infrastructure and can only be decrypted inside the attested confidential environment. The company says responses are protected on the return path. It also describes inference in a hardware-backed environment spanning the CPU and GPU, with protected GPU memory and encrypted traffic between them. Customers can create and manage encrypted vaults using the same Cohere APIs, the company says.

Cohere CTO Phil Blunsom argues that the setup protects customer data even from Cohere. The company cautions that confidential computing does not eliminate all risk or replace application security, compliance, contractual controls or data-residency requirements.