September 2024

  • Article

These features and Azure Databricks platform improvements were released in September 2024.

Note

Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.

September 30, 2024

Git folders and Repos (Legacy) now support 1GB and 20000 workspace assets per working branch.

Meta Llama 3.1 405B Instruct now supported on Databricks Function Calling

September 25, 2024

Meta Llama 3.1 405B Instruct now supported on Databricks Function Calling.

vector_search() now available to all customers (Public Preview)

September 25, 2024

The Public Preview for vector_search() is now available to all customers in regions where Mosaic AI Vector Search is supported. Customers no longer need to request access.

See vector_search function for how to query a Mosaic AI Vector Search index using SQL.

Meta Llama 3.2 3B and 1B models are supported in Model Serving

September 25, 2024

The Meta Llama 3.2 3B and Meta Llama 3.2 1B models are now supported in Foundation Model APIs provisioned throughput.

Meta Llama 3.2 3B and 1B models are supported in Mosaic AI Model Training

September 25, 2024

The Meta Llama 3.2 3B and Meta Llama 3.2 1B models are now supported in Mosaic AI Model Training. See Supported models.

Publish to Power BI is now generally available

September 24, 2024

The Publish to Power BI feature is now generally available. This feature allows users to seamlessly create semantic models from tables/schemas on Databricks and publish them directly to Power BI Service.

To get started, see Publish to Power BI Online from Azure Databricks.

Use @ to reference tables in Databricks Assistant prompts

September 24, 2024

To quickly reference tables in Assistant prompts, use the @ symbol. See Reference tables in prompts using @.

Prototype and export tool-calling GenAI Agents in AI Playground

September 24, 2024

Use the AI Playground to prototype, build, and then export tool-calling GenAI agents. You can now give your AI agent tools in the form of Unity Catalog functions and interact with the agent directly in AI Playground.

Export the AI agent to notebooks to iterate further, evaluate quality, and deploy it. See Create an AI agent and its tools.

Control external access to data in Unity Catalog using the new EXTERNAL USE SCHEMA privilege

September 18, 2024

The new EXTERNAL USE SCHEMA privilege enables you to restrict access to data in Unity Catalog when external processing engines like Iceberg clients or Microsoft Fabric use Unity Catalog open APIs or Iceberg APIs to access that data. See Control external access to data in Unity Catalog.

GTE v1.5 (English) embedding models are now supported in Foundation Model API provisioned throughput

September 13, 2024

Foundation Model APIs provisioned throughput now supports GTE v1.5 (English) models: gte-base-en-v1.5 and gte-large-en-v1.5.

Databricks Assistant Quick Fix debugs code inline

September 12, 2024

Assistant Quick Fix recommends single-line fixes for running code when it returns an error. Accept the fix and continue to run the code.

See Quick Fix.

Create budgets to monitor account spending (Public Preview)

September 11, 2024

Account admins can now create budgets to track spending in their Azure Databricks account. Budgets can include customized filters to track spending based on workspace and custom tags. See Use budgets to monitor account spending.

Mosaic AI Model Training is now available to all customers (Public Preview)

September 10, 2024

Mosaic AI Model Training, is now available to all customers in the following regions: centralus, eastus, eastus2, northcentralus, and westus. Customers no longer need to request access to use this feature in these regions.

Using Mosaic AI Model Training, you can use your own data to customize a foundation model to optimize its performance for your specific application. By fine-tuning or continuing training of a foundation model, you can train your own model using significantly less data, time, and compute resources than training a model from scratch. See Mosaic AI Model Training for foundation models.

AI Gateway is now Public Preview

September 9, 2024

Mosaic AI Gateway is now Public Preview. It is a centralized service that streamlines the usage and management of generative AI models within an organization.

AI Gateway brings governance, monitoring, and production readiness to model serving endpoints using the following features:

  • Permission and rate limiting to control who has access and how much access.
  • Payload logging to monitor and audit data being sent to model APIs using inference tables.
  • Usage tracking to monitor operational usage on endpoints and associated costs using system tables.
  • AI Guardrails to prevent unwanted data and unsafe data in requests and responses.
  • Traffic routing to minimize production outages during and after deployment.

Meta Llama 3.1 70B and 8B models supported in Mosaic AI Model Training

September 9, 2024

Meta Llama 3.1 70B and Meta Llama 3.1 8B models are now supported in Mosaic AI Model Training. See Supported models.

Extended AI-generated comments support

September 6, 2024

AI-generated comments support now includes catalogs, schemas, functions, models, and volumes in addition to tables and table columns. An inline assistant also helps edit comments in Catalog Explorer. See Add AI-generated comments to Unity Catalog objects.

Monitor clean room usage in the billable usage table

September 5, 2024

The system.billing.usage table now includes a usage_metadata.central_clean_room_id value, allowing you to monitor costs incurred by clean room usage. See Billable usage system table reference.

Databricks extension for Visual Studio Code is GA

September 4, 2024

The Databricks extension for Visual Studio Code is now generally available. The extension allows you to connect to your remote Azure Databricks workspaces from Visual Studio Code and then easily define, deploy, and run Databricks Asset Bundles, debug notebooks and run them as jobs, run files on clusters and as jobs, and synchronize local code to your workspace, all from the VSCode IDE.

To install the Databricks extension for Visual Studio Code and quickly get started, see What is the Databricks extension for Visual Studio Code?.

Function calling is now supported on Foundation Model APIs provisioned throughput

September 3, 2024

OpenAI compatible function calling is now available on Foundation Model APIs provisioned throughput. This launch also includes function calling support for the Llama 3.1 8B Instruct model on provisioned throughput workloads.

See Function calling on Azure Databricks.

System tables are now generally available

September 3, 2024

The Azure Databricks system tables platform is now generally available. This launch also includes the GA release of the system.billing.usage and system.billing.list_price tables. See Monitor usage with system tables.