M+E Daily

Databricks used its keynotes at the annual Data + AI Summit in San Francisco on June 28 to highlight advancements that continue to be made in artificial intelligence (AI) and machine learning (ML).

“We’re calling it Generation AI because I really feel like this is a really, really special time,” said Ali Ghodsi, CEO and co-founder of Databricks. “In fact, I got a text this morning from someone that said, ‘what an amazing time we’re living in; this is the best time ever to be alive.’”

He added: “We called it Generation AI because I think we’re all part of this generation that’s actually going to form the future of the planet, the future of technology, the future of generative AI, machine learning and data…. We think every company in the future will be a data and AI company, and the people that are here, you’re presenting those companies. You are those that will shape the future of the planet.”

And with AI and other related technology, “we all can make the world much, much better,” he said, adding: “We can make everyone smarter. We can cure diseases and we can just raise the standard of living. Of course, there are issues with it as well, and we’ll get into them and we’ll talk about them, and hopefully we can deal with them.”

He went on to predict that, “all in all, I think actually this is going to be game changing for us and I think you are the ones, those of you who are here, you’re the ones [thar are] going to make it happen.”

Noting this is the 10th year of the event, he said it started at Spark Summit, then it became Sparkless AI Summit, then it became Data + AI Summit.

Fast forwarding to today, he pointed out Spark was donated to the Apache Foundation and celebrated at this conference 10 years ago. There are now 1 billion downloads a year of it, he noted. Meanwhile, Delta Lake “now has half a billion downloads a year” and ML Flow has 120 million downloads a year.”

He then introduced his first special guest, Microsoft CEO Satya Nadella. “All of us have been working on AI for a long time but this specific era of AI, the bet was quite frankly around the scaling laws on these foundation models. It was unclear even a few years ago whether they would work or not but they seem to be working.”

Nadella predicted: “There will be more breakthroughs but at least they are working. And when they’re working, as technologists, you take advantage of it.”

Databricks announcements at the event included LakehouseIQ, a knowledge engine that the company said learns the unique nuances of your business and data to power natural language access to it for a wide range of use cases.

Any employee within an organization can use LakehouseIQ to search, understand and query data in natural language. LakehouseIQ uses information about your data, usage patterns and org chart to understand your business’s jargon and unique data environment, and give significantly better answers than naive use of Large Language Models.