Azure Stream Analytics reference data input suddenly getting truncated to first 50 records after upgrade to V2 plan

79848876 11 Reputation points
2023-07-04T10:09:10.76+00:00
  1. An existing Stream Analytics job which has not been changes for a couple of years and working happily
  2. It supports a few hundred IoT devices "in the wild"
  3. As part of the job it relies on several (4x) reference data inputs
  4. All inputs are quick simple - along the lines of "select * from TABLE where ..."
  5. As of today, suddenly we have a problem - ALL reference data inputs truncate to the first 50 records apparently discarding the rest of the data.
  6. The SQL is correct and tested - it returns all records (between 69 and 230)
  7. In SQL server log, I could see stream analytics receiving all data as well

For me, it is critical - suddenly half of my live devices are marked as "offline" as the stream analytics cannot process them (lacks reference data)

Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
354 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Sander van de Velde | MVP 32,641 Reputation points MVP
    2023-07-04T22:45:53.64+00:00

    Hello @79848876,

    welcome to this moderated community forum.

    In the past, I have used Azure Stream Analytics reference data but did not encounter similar behavior.

    I recommend creating an Azure support ticket .

    One trick could be building a new Stream Analytics job from the ground up with the same logic.

    This eliminates any dependencies on the initial migration.

    If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.