About Python Function App Event Hub Trigger settings

Jona 475 Reputation points
2023-11-28T23:08:22.58+00:00

Hi every one,

I'm developing locally a Python function app with an Event Hub trigger. I want the function to wait a batch has been filled or the wait time for reading new event finish, but none is working.

Could you give a hand??

This is my host.json:

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[3.*, 4.0.0)"
  },
  "extensions": {
    "eventHubs": {
      "maxEventBatchSize" : 100,
      "minEventBatchSize" : 25,
      "maxWaitTime" : "00:10:00",            
      "batchCheckpointFrequency" : 1,
      "prefetchCount" : 300,
      "initialOffsetOptions" : {
          "type" : "fromStart"
      },
      "clientRetryOptions":{
          "mode" : "exponential",
          "tryTimeout" : "00:01:00",
          "delay" : "00:00:00.80"
      }
    }
  }
}

After sending 30 messages to event hub via rest API, my storage look like this:

User's image

I don't think than an individual message should create a separate parquet file ... hence yout help on this.

Regards

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,978 questions
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
637 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Bruno Lucas 4,431 Reputation points MVP
    2023-11-29T09:19:00.9766667+00:00

    Hi

    there is no way to Guarantee that. if you are not managing to get that locally, it may get worse when you deploy and the function app start creating parallel instances.

    https://video2.skills-academy.com/en-us/azure/azure-functions/functions-bindings-event-hubs?tabs=isolated-process%2Cextensionv5&pivots=programming-language-csharp

    User's image

    What version is your function?

    User's image


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.