Join our newsletters to stay updated with the latest in AI.
Databricks is enhancing its gen AI developer tools for enterprises. At their annual data and AI conference, the company announced new features for its Mosaic AI platform, designed to help businesses deploy applications powered by large language models (LLMs).
Mosaic AI, stemming from Databricks’ $1.3 billion acquisition of MosaicML, has accelerated the development of generative AI. The latest updates focus on three main areas: developing compound AI systems, evaluating them across various metrics, and governing the entire process.
This initiative aims to create a comprehensive ecosystem that supports enterprises in building reliable generative AI applications from their data. It also boosts Databricks’ competitive edge against Snowflake, which has also been expanding in this area under CEO Sridhar Ramaswamy. Recently, Snowflake introduced its own enterprise-grade open LLM, “Arctic,” to rival Databricks’ DBRX.
For organizations keen on generative AI, leveraging internal data with powerful AI models holds great promise. However, achieving a good return on investment can be challenging, often resulting in applications that don’t deliver high-quality outputs within budget and privacy constraints.
To address this, companies have shifted to building compound AI systems based on retrieval augmented generation (RAG). These systems use various small models, retrievers, vector databases, and tools for evaluation, monitoring, security, and governance. Databricks has been enhancing Mosaic AI to support the development of these systems.
A few months ago, Databricks introduced Vector AI search as a serverless vector database integrated into its data platform. Now, it is adding Mosaic AI Model Training and Agent Framework.
The Model Training feature allows users to fine-tune small, open-source models to handle specific tasks or domains cost-effectively using the Databricks API or UI. The Agent Framework, integrated with Mosaic AI Vector Search and Model Serving, helps create high-quality RAG applications using these fine-tuned models.
The Agent Framework simplifies the evaluation of application quality through built-in AI-assisted evaluation and an intuitive tracing UI for feedback from human stakeholders. This enables easy incorporation of feedback, allowing developers to test and deploy their applications efficiently.
Databricks also introduced an AI Tools Catalog, which allows organizations to govern, share, and register tools via the Unity Catalog. These tools enhance compound AI systems by adding capabilities like code generation, web searching, and API calls. Any Python or SQL function registered in the Unity Catalog will be supported by the AI Tools Catalog, improving the overall quality of the responses.
Additionally, Databricks is introducing “Mosaic AI Gateway” to provide a unified interface for querying, managing, and deploying models. This gateway offers strong governance and monitoring capabilities, including usage tracking, rate limits, and safety filters.
Most of the new features of Mosaic AI, except the AI Tools Catalog, are in public preview and will become generally available soon. The tools catalog is in private preview for now. Other announcements at the event included Databricks AI/BI for AI-powered analytics, Databricks LakeFlow for data engineering, and an image generation model developed with Shutterstock.
The Databricks Data and AI Summit will run from June 10 to June 13, 2024.