Hi
I'm using Event Hub REST API to send data to a stream. This happens in a Functions with BlobStorage trigger.I feel a little confused with some terminology which I've not found the proper documentation to finally fully understand.
I built this dashboard to get insights of my experiment. Basically, I'm sending differents data formats via Event Hub. This includes CSV, CSV compresed, JSON... and may be later Parquet and AVRO. The trigger on the BlobStorage reacts to CSV file uploads. After Event Hubs, there is another Functions that read from it and send the quantity of message to Power BI Service Push dataset API.
The objective es to determine which orquestation format performs the best with good metrics.
So my questions are:
- The metric Throttled Requests ¿ does it reject all the request? ¿or some of the events are sent properly and others not? I thinking about on single and batch events sendings
- The metric Capture Backlog ¿What is it?
I've reading the following docs:
I'm working with Event Hub Standard 1 TU and I uploaded two CSV files: one of 2,1MB size and other of 1,2MB size
The response of the API is as follows:
The response code is 413, and the message tells nothing about throttling. More over, the throttling metric won't show up on Azure Monitor; instead, it looks like a correct API CALL
If I upload a file smaller than 1 MB (as it should be to run fine), there is obviuosly no message error:
Key note: the main objective of my experiment is to demostrate that If I send a CSV serialized (or compressed, also test it by me), I would be able to send more events per unit time. Look how I've been able to send 2.311 rows, far beyond the limit of 1000 events per publication. This is because the entire serialized CSV file is considered as one single event
¿Why doesn't the REST API respond with the proper code, as indicated in the above docs?
Besides, I've not wanted to use Event Hub as an output binding, because I don't see anyway to perform error handling. The output binding for Event Hub just waits for a value to be returned and nothing else .. I don't know where to locate error handling.
In the other hand, the SDK has have a poorly performnce (sending a CSV of 10KB takes almost 3 second, whereas via REST API it takes almost 2 seconds to deliver a almost 1MB file size)
I know is quite much, However I hope you can help me understand this behaviour.
Regards