Delta Lake logs in Warehouse in Microsoft Fabric
Applies to: Warehouse in Microsoft Fabric
Warehouse in Microsoft Fabric is built up open file formats. User tables are stored in parquet file format, and Delta Lake logs are published for all user tables.
The Delta Lake logs opens up direct access to the warehouse's user tables for any engine that can read Delta Lake tables. This access is limited to read-only to ensure the user data maintains ACID transaction compliance. All inserts, updates, and deletes to the data in the tables must be executed through the Warehouse. Once a transaction is committed, a system background process is initiated to publish the updated Delta Lake log for the affected tables.
How to get OneLake path
The following steps detail how to get the OneLake path from a table in a warehouse:
Open Warehouse in your Microsoft Fabric workspace.
In the Object Explorer, you find more options (...) on a selected table in the Tables folder. Select the Properties menu.
On selection, the Properties pane shows the following information:
- Name
- Format
- Type
- URL
- Relative path
- ABFS path
How to get Delta Lake logs path
You can locate Delta Lake logs via the following methods:
Delta Lake logs can be queried through shortcuts created in a lakehouse. You can view the files using a Microsoft Fabric Spark Notebook or the Lakehouse explorer in Synapse Data Engineering in the Microsoft Fabric portal.
Delta Lake logs can be found via Azure Storage Explorer, through Spark connections such as the Power BI Direct Lake mode, or using any other service that can read delta tables.
Delta Lake logs can be found in the
_delta_log
folder of each table through the OneLake Explorer in Windows, as shown in the following screenshot.
Limitations
- Table Names can only be used by Spark and other systems if they only contain these characters: A-Z a-z 0-9 and underscores.
- Column Names that will be used by Spark and other systems cannot contain:
- spaces
- tabs
- carriage returns
- [
- ,
- ;
- {
- }
- (
- )
- =
- ]
Related content
Feedback
https://aka.ms/ContentUserFeedback.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see:Submit and view feedback for