Workspace libraries (legacy)

Workspace libraries serve as a local repository from which you create cluster-installed libraries. A workspace library might be custom code created by your organization, or might be a particular version of an open-source library that your organization has standardized on.

You must install a workspace library on a cluster before it can be used in a notebook or job. See Install a workspace library onto a cluster.

Workspace libraries in the Shared folder are available to all users in a workspace, while workspace libraries in a user folder are available only to that user.

How do workspace libraries differ from libraries stored with workspace files?

The term “workspace libraries” refers to any library registered to a workspace. Workspace libraries appear alongside notebooks and workspace files and follow ACL rules for their containing directory.

The term “workspace files” refers to any arbitrary file stored in the workspace. You can store Python .whl packages as workspace files and then register them as workspace libraries. See What are workspace files?.

You can upload libraries as workspace files. See Workspace files basic usage.

While you can upload many file types to workspace files, only Python .whl files can be installed using workspace files. See Cluster-scoped libraries.

In Databricks Runtime 13.3 LTS and above, you can use %pip to install Python wheel files stored as workspace files. See Install a package stored as a workspace file with %pip.

Create a workspace library

1. Right-click the workspace folder where you want to store the library.
2. Select Create > Library. The Create Library dialog appears.
3. Select the Library Source and follow the appropriate procedure:
   • Upload a library
   • Reference an uploaded library
   • PyPI package
   • Maven package
   • CRAN package

Upload a Jar, Python egg, or Python wheel file

1. In the Library Source button list, select Upload.
2. Select Jar, Python Egg, or Python Whl.
3. Optionally enter a library name.
4. Drag your Jar, Egg, or Whl to the drop box or click the drop box and navigate to a file. The file is uploaded to dbfs:/FileStore/jars.
5. Click Create. The library status screen displays.

Reference an uploaded JAR, Python egg, or Python wheel file

You can create a new workspace library by referencing JAR, Python egg, or Python wheel files stored in the DBFS root, on object storage, or with workspace files. Following this workflow registers a pointer to a library without uploading or moving files.

With Databricks Runtime 13.2 and above, you can also reference a volume in Unity Catalog. This capability is in Public Preview.

For shared access mode, you must add JARs to the allowlist. See Allowlist libraries and init scripts on shared compute.

1. Select File path/ADLS in the Library Source button list.
2. Select Jar, Python Egg, or Python Whl.
3. Optionally enter a library name.
4. Specify the path to the library, such as the following examples:
   • abfss://container-name@storage-account-name.dfs.core.windows.net/path/to/library.whl
   • /Workspace/path/to/library.whl
   • dbfs:/path/to/library.whl
   • /Volumes/<catalog>/<schema>/<volume>/<path_to_library_file>
5. Click Create. The library status screen displays.

For more on working with libraries in object storage, see Install libraries from object storage.

For more on workspace files, see Work with files on Azure Databricks.

Install a workspace library onto a cluster

In single user access mode, the identity of the assigned principal (a user or service principal) is used. In shared access mode, libraries use the identity of the user who installed the library. No-isolation shared access mode does not support volumes, but uses the same identity assignment as shared access mode.

To see all options for installing libraries on clusters, see Cluster libraries.

To install a library that already exists in the workspace, you can start from the cluster UI or the library UI:

Cluster

1. In the sidebar, click Compute.
2. Click a cluster name.
3. Click the Libraries tab.
4. Click Install New.
5. In the Library Source button list, select Workspace Library.
6. Select a workspace library.
7. Click Install.

Library

1. Go to the folder containing the library.
2. Click the library name.
3. Select the checkbox next to the cluster that you want to install the library on and click Install. Only running clusters appear in the list.

View workspace library details

1. Go to the workspace folder containing the library.
2. Click the library name.

The library details page shows the running clusters and the install status of the library. If the library is installed, the page contains a link to the package host. If the library was uploaded, the page displays a link to the uploaded package file.

Move a workspace library

1. Go to the workspace folder containing the library.
2. Right-click the library name and select Move. A folder browser displays.
3. Click the destination folder.
4. Click Move.

Delete a workspace library

To delete a workspace library:

1. Move the library to the Trash folder.
2. Either permanently delete the library in the Trash folder or empty the Trash folder.