How Kythera Labs, a Databricks Built-On Partner, saves $2M+/year using Delta Sharing

Healthcare systems generate enormous amounts of sensitive data, but moving, sharing, and analyzing that data securely across organizations is still a major challenge. In this post, we'll look at how we at Kythera Labs use Databricks and Delta Sharing to manage more than 300 million patient records and support collaborations across healthcare and life sciences. The blog will cover the practical issues with older data-sharing methods, why we adopted Delta Sharing, and the impact it's had on our storage costs, efficiency, and real-time collaboration.

Making Data Work in Healthcare: Kythera's Approach

Kythera Labs is a data technology company that empowers healthcare and life sciences organizations with a unified, high-fidelity healthcare data platform for analysis. As a built-on Databricks Partner, we chose Databricks and Delta Sharing not just for internal data sharing but also to support seamless data exchange with external partners. Today, more than 80% of our customers use products built on the platform. We also support external collaborations, including organizations like Exact Sciences, using Delta Sharing across 50 active customer workspaces.

Why Delta Sharing?

Kythera Labs chose Delta Sharing to overcome significant challenges in securely sharing healthcare data. With over 300 million patient records spanning a decade of clinical history, traditional methods required creating and moving multiple full copies of datasets, driving storage costs into the hundreds of thousands of dollars and slowing delivery.

Delta Sharing changes that by enabling secure, real-time access to live data without creating duplicate copies. Instead of storing and maintaining separate datasets for each partner or environment, we can share a single, governed source of truth directly. This approach has allowed us to power internal teams and external collaborations with just 3.5 PB of storage, rather than the 20-plus PB otherwise required.

Another complexity is meeting our customers where they are on the cloud. Healthcare providers often operate in Azure, while many pharmaceutical companies run on AWS or GCP. Without a technology like Delta Sharing, delivering large datasets across clouds would mean costly transfers, complex ETL work, and multiple stale copies scattered across clouds. With Delta Sharing, we can instantly provide secure access to the same live dataset - no matter the cloud - while maintaining compliance and eliminating unnecessary copies.

This not only streamlines our internal workflows (moving from development to testing to production without re-copying data) but also makes it easy for customers to act faster, like instantly updating a cancer treatment model with the newest data.