ADX - "Blob already received" in ingestion from event grid

Carlo Suriano 26 Reputation points
2022-03-14T15:32:02.993+00:00

Hi there,
I have an ADX ingestion pipeline set up that populates a single table from two event grids sources.
This is specified as suboptimal in docs (https://video2.skills-academy.com/en-us/azure/data-explorer/error-codes#category-blobalreadyreceived) and - in fact - I am experiencing some BlobAlreadyReceived failures.

Why is N:1 ingestion model suboptimal? Is there a way to configure it in order to avoid failures?
I was not able to find any other reference or configuration hint about this in docs.

Thank you

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
525 questions
0 comments
Accepted answer
  1. PRADEEPCHEEKATLA-MSFT 89,466 Reputation points Microsoft Employee
    2022-03-15T09:51:27.587+00:00

    Hello @Carlo Suriano ,

    Thanks for the question and using MS Q&A platform.

    Getting many errors of BlobAlreadyReceived indicates that there are many duplicate blobs sent to the same table. That's mean duplicate records on the table which is not optimal. You can set few EG sources to the same table, but in case that you refer same blob from the same SA to the same table few times using few EG sources- it indicates a suboptimal ingestion.

    Why is N:1 ingestion model suboptimal? Is there a way to configure it in order to avoid failures?

    N:1 is not suboptimal. Suboptimal configuration is to send the same data more than one time to the same table. Is it what you are doing? If so- do you indeed want to send data, several times to the same table?

    We saw customers defining on the same SA few EG obtainers with similar configuration, so they get duplications on the target table. This is suboptimal. Having few EG obtainers ingesting data from different blobs to the same table is totally fine.

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.