M+E Connections

Databricks: Data, Analytics, AI on One Platform

San Francisco-based Databricks is on a mission: to help data teams solve the world’s toughest problems.

And with more than 7,000 organizations worldwide — including over 40 percent of the Fortune 500 — turning to Databricks for its data and AI solutions, it’s safe to say the mission’s going well.

Steve Sobel, RVP and global industry leader of communications, media and entertainment for Databricks, spoke with MESA about how the company combines the best elements of data lakes and data warehouses, the media and entertainment successes Databricks has enjoyed, and how AI is at the core of everything that Databricks does.

MESA: What was the impetus for Databricks, how did the company first come about in 2013?

Sobel: With origins in academia and the open-source community, Databricks was founded in 2013 by the original creators of popular open-source initiatives including Apache Spark, Delta Lake, and MLflow. As the world’s first and only lakehouse platform in the cloud, Databricks combines the best of data warehouses and data lakes to offer an open and unified platform for data and AI.

Today, more than 7,000 organizations worldwide — including Acxiom, Condé Nast, Disney, Paramount, Riot Games, and SEGA — rely on Databricks to enable massive-scale data engineering, collaborative data science, full-lifecycle machine learning, and business analytics.

MESA: The Lakehouse for Media & Entertainment Platform touts itself as combining the best elements of data lakes and data warehouses. For the media and entertainment industry, how does the platform help firms operate more efficiently and innovate faster?

Sobel: The Databricks Lakehouse for Media & Entertainment platform helps teams accelerate audience and advertiser outcomes in a number of ways:

• 360° view of your audience. The Lakehouse helps teams bring together all your structured and unstructured data — clickstream, demographic, social — in a single platform for analytics and AI. With a holistic view of the consumer journey, organizations can understand content preferences that help deliver more personalized experiences and develop more targeted advertising and engagement.

• Drive revenue from a content library. Media companies are built on unstructured data like video, images, and audio files so the ability to analyze unstructured data is essential for effective media asset management. A Lakehouse architecture can be used by marketers to leverage archived content for campaigns, production teams looking for existing content to include in new productions, and sales teams seeking IP they can sell to other media companies.

• Mitigate churn and increase ARPU. With an agile cloud-based platform, organizations can quickly and reliably process massive amounts of data and feed it to downstream systems to deliver a 1:1 experience on any channel at any time. As consumers’ expectations around real-time recommendations keep rising — and media companies fight for consumer attention — ensuring that personalization approaches real-time is becoming a requirement for many organizations.

• Put ML at the core of your business. The Databricks Lakehouse for Media & Entertainment helps teams unlock the power of machine learning to better understand consumer, employee, and advertiser needs. When all your data is centralized and seamlessly connected by a full suite of collaborative analytics and machine learning tools, data teams can work together to build powerful predictive models that drive new innovations in personalization, content monetization, and advertiser outcomes.

MESA: Where does artificial intelligence technology fit into what Databricks offers?

Sobel: AI is at the core of everything that we do at Databricks.

We believe that by expanding access to machine learning workloads, teams can better prepare and process data, streamline cross-team collaboration and standardize the full ML lifecycle from experimentation to production.

Our collaborative notebook environment with managed runtime for ML makes it easier for teams to get up and running, and our solution accelerators for common use cases including CLV, Churn, and Recommenders, accelerate the time it takes to go from concept to execution. 


MESA: What are some of Databricks’ favorite use-case examples (that you can share), where media and entertainment companies made especially good use of your services?

Sobel: Some of the most canonical examples are:

Comcast transforms their X1 home entertainment offering using voice, data, and AI to create personalized experiences

Riot Games uses Databricks to create an optimal in-game experience for League of Legends players. 

AT&T uses Databricks to stop fraud before it happens.

HBO Max uses Databricks to run real-time data analytics and content curation for 70M customers

Condé Nast creates personalized customer and advertiser experiences powered by machine learning

MESA: How has the pandemic impacted Databricks’ business, and in what ways has the company adjusted its services?

Sobel: Every industry on the planet has been impacted in some form or another by the pandemic. The role of data + AI in companies has become foundational to long-term strategic success as they seek to compete.

Teams that better understand their audience, leveraging actionable insights to increase the pace of innovation, drive greater personalization, and increase engagement and retention are more resilient as market conditions evolve.

At Databricks, we set out to make data + AI more accessible at scale. Throughout the pandemic, we’ve worked closely with our customers and partners — including industry leaders like Deloitte and Accenture — to build and optimize our platform to address the challenges of tomorrow, today. 


MESA: What’s next for Databricks, what advances or added services can we expect from the company on the horizon?

Sobel: Databricks continues to innovate on our Lakehouse platform. Recently, we announced Delta Lake was going fully open source, revealed Project Lightspeed, a next-generation data streaming engine for powerful, real-time analytics, and also added Data Cleanrooms to the Lakehouse for secure, collaborative data sharing.

These were just a few of the announcements we made at Data + AI Summit in June.

We’ll have more exciting updates to share in the coming months, we highly recommend following our blog and careers page – we’re hiring!