Kusto Ingest library overview

Applies to: ✅ Microsoft Fabric ✅ Azure Data Explorer

The Kusto Ingest library provides a client for ingesting data into your database. The library supports queued and direct ingestion. The ingestion methods are defined by the Kusto ingest client object.

For a list of interfaces and classes, see Kusto Ingest client reference.

Get the library

Select the tab for your preferred language.

Permissions

To ingest data into existing tables, you must have at least Database Ingestor or Table Ingestor permissions. To create a table, you must have at least Database User permissions. For more information, see Role-based access control.

Queued ingestion

The queued ingest client minimizes the dependencies of client code on the ingestion service. In this mode, ingestion is accomplished by submitting an ingestion message to an Azure queue, which is subsequently processed by the ingestion service. If any intermediate storage items are required, the ingest client generates them using the resources provided by the ingestion service.

Queued ingestion allows the ingestion requests to be persisted when the ingestion service is unavailable, and lets the ingestion service manage the ingestion load on your database. This method provides a mechanism to track the progress and outcome of every ingestion request, retries the ingestion on transient failures, and improves performance by efficient and controllable aggregation on inbound data.

Direct ingestion

The direct ingest client requires direct interaction with the ingestion service. In this mode, the ingestion service doesn't moderate or manage the data. Every ingestion request is translated into a command that is executed directly on the service. When synchronous methods are used, the method completion indicates the end of the ingestion operation.

Direct ingestion has low latency and doesn't involve aggregation. However, the client code has to implement retry or error handling logic, and the client code could overwhelm the database with requests as it's unaware of the capacity.