Databricks
Data Platform
Data lakehouse platform, DBRX model, Mosaic ML acquisition.
Recent activity
-
Developers, a new layer called Omnigent in Databricks enables engineers to define an agent once, including the model, tools, policies, and limits, and run it across any harness, reducing the need to rebuild and manage multiple instances. Omnigent integrates with the Foundation Model APIs for unified cost and governance tracking. Additionally, a new web search component called Nimble, which can adapt to specific use cases and self-learn the best retrieval methods, can be integrated to improve the accuracy and efficiency of web search. AI summary
Read more → -
AIOps (Artificial Intelligence for IT Operations) combines AI and machine learning with observability to automate IT operations, detecting anomalies, correlating events, identifying root causes, and automating incident response, ultimately reducing downtime risk, cutting alert fatigue, and accelerating decision speed. AI summary
Read more → -
MATCH_RECOGNIZE is a new SQL operator available in Public Preview that allows detecting patterns and sequences from event data using regex-like pattern-matching, simplifying pattern detection and sequence analysis across various industries. AI summary
Read more → -
Databricks' Genie and AI business processes can operationalize ML insights into governed business operations for energy theft detection, enabling teams to accelerate the loop from flagged meter to recovered revenue to a safer household. A governed workflow connects interpretation, investigation prioritization, dispatch-ready reporting, and recovery workflows, with Lakebase maintaining live case state and recovery totals. Trusted answers and metrics are provided through Genie One, Unity Catalog, and Unity Gateway, enabling leaders to ask questions in plain English and receive answers grounded in governed data. AI summary
Read more → -
Databricks' marketing team uses Genie, an AI analytics assistant, 3x more often in decision-making, with over 85% adoption across the marketing organization. They achieved this by building a governed Marketing Lakehouse, documenting data and business context, encoding verified answers and examples, teaching Genie the language of their business, and continuously evaluating and improving the system through user feedback. AI summary
Read more → -
Managed Postgres should take routine database operations such as patching, scaling, failover, and backups off the database team's plate. Lakebase automates these operations on serverless infrastructure with automatic scaling, scale-to-zero, point-in-time recovery, branching, pgvector, and PostGIS. AI summary
Read more → -
Databricks now supports on-demand state repartitioning for Apache Spark Structured Streaming, allowing users to resize partitions without rebuilding checkpoint state, enabling more flexible tuning and scaling of stateful streaming queries. This feature is available in Databricks Runtime 18 and above with the RocksDB state store provider. AI summary
Read more → -
Conversational AI can help payer finance leaders decompose variances in medical loss ratio (MLR) across claims, utilization, cost, and population risk in minutes, without waiting on analysts or reports. However, AI needs payer-specific context to earn trust, and without it, AI amplifies confusion instead of resolving it. A unified payer intelligence foundation, combining governed data and AI capabilities with payer-specific data and business context, can help finance leaders understand what caused the variance and take corrective action. AI summary
Read more → -
Lakeflow Connect provides native, fully managed connectors for various SaaS applications, databases, and file sources, ingesting data directly into Databricks Platform. These connectors can be set up via a point-and-click UI or a simple API, and the ingested data is incrementally ingested as governed, managed tables in Unity Catalog. AI summary
Read more → -
Read restrictions and catalog labels from Apache Iceberg™ standardize delegated enforcement and governance context portability across engines and catalogs, respectively, addressing fragmented enforcement and enabling unified governance across the Open Lakehouse. AI summary
Read more → -
Databricks has improved the Lakebase Postgres compute cache, increasing throughput by up to 2x and reducing latency. The new cache path uses larger shared buffers, which can hold up to 75% of available DRAM, and enables autoscaling. This reduces double buffering and improves cache hit rates, resulting in lower CPU usage and faster access to data. AI summary
Read more → -
Here is a summary of the article in 3 plain sentences for a developer/AI-ML audience: Financial services leaders are asking about the governance and cost implications of implementing AI solutions, with questions including how to ensure AI systems are trustworthy, how to use real-time data without adding complexity, and how to manage AI costs as usage expands. To address these concerns, Databricks is showcasing its platform and AI capabilities at Sibos 2026, including its ability to provide governed data and AI for financial services workflows and its solutions for real-time data and AI governance. The company is also hosting executive meetings and providing demos at the event, which takes place September 28-October 1 in Miami, Florida. AI summary
Read more → -
Databricks provides a unified platform for end-to-end Solvency II reporting, connecting data ingestion, quality checks, reserving, capital calculation, quantitative reporting templates, own risk and solvency assessment, governance, approvals, and disclosure. This approach automates ingestion checks, provides a single control view, and supports governance, audit trails, and AI-assisted review. AI summary
Read more → -
To secure AI/BI dashboards for every viewer, use a single entitlements table to govern both embedded dashboards and direct SQL queries, signing a viewer or group scope into the __aibi_external_value. This approach uses default-deny, masking, and Unity Catalog row filters as layered controls to protect data. AI summary
Read more → -
Consort, an open-source agentic framework, uses Lakebase branching as its foundation for test-driven development on a real branching database. It integrates testing into the development workflow, allowing for integration testing to be performed against a live branch of the real database, reducing the need for mocks and improving the accuracy of tests. Consort enables teams to work in a more agile and efficient manner, with a clear separation of concerns between roles and a deterministic state machine that routes work and holds human-approval gates. AI summary
Read more → -
Adaptive Instructed-Retriever, a retrieval model, combines parallel single-step search with sequential search to achieve frontier-quality search at 2x lower latency, matching the quality of leading third-party models. The model uses online reinforcement learning to adaptively decide how much computation each user request requires, and can be fine-tuned by adjusting the magnitude of the step penalty. This approach enables data agents to spend additional search steps on complex queries where iterative reasoning can improve retrieval quality, while returning early on simpler queries to minimize latency. AI summary
Read more → -
Zepto uses Databricks and MLflow to build evaluation-first AI agents that manage 80%+ of support tickets, cut support costs by 65%, and deliver payback in under one month. They achieve this by building a dual-loop architecture that connects development and production loops with a strict quality gate, ensuring reliability, control cost, and risk management for large-scale agents. AI summary
Read more → -
Here is a summary of the article in 3 plain sentences for a developer/AI-ML audience: Choosing the correct SQL data type is crucial for data integrity, storage efficiency, and query performance, as it can prevent data corruption, reduce storage costs, and improve query speed; understanding the characteristics of different numeric, date/time, and character data types is essential for making informed decisions; by selecting the smallest data type that safely holds the data, developers can improve scalability and performance across analytics, BI, and ML workloads. AI summary
Read more → -
To build durable agents, Temporal and Lakebase can be combined to preserve agent progress, retry failed operations, and wait durably for human review. Temporal stores durable control flow for agent runs, while Lakebase stores the application-facing view of operational state, including current run status, messages, and evidence. AI summary
Read more → -
Decades-old separation between operational and analytical database systems is breaking down due to AI agents' need for real-time data access, and Lake Transactional/Analytical Processing (LTAP) unifies transactional and analytical workloads at the storage layer, allowing for serverless operational and analytical compute, and unified governance and cataloging. AI summary
Read more →