expired messages not accessible but still show in approximage message counts

@voteplatypus Welcome to Microsoft Q&A, Thank you for posting your query here! Thanks for raising this question! Firstly, apologies for the delay in responding here and any inconvenience this issue may have caused. Kindly try the following steps to isolate the issue:

Valid values that can be entered in the Expires in field are between 1 second and 7 days. Select Message never expires to indicate a message that will remain in the queue until it is explicitly removed.

• Let us know how you are measuring the message count?

Approximate messages count will give you an approximate count of total messages in a queue and will include both visible and invisible messages.

One thing you could possibly do is PEEK at messages and it will return you a list of visible messages. However it will only return you a maximum of top 32 messages from the queue. So you logic to send notification message would work if the threshold is less than 32.

if you want to use ApproximateMessageCount to get result try this

CloudQueue q = queueClient.GetQueueReference(QUEUE_NAME);
q.FetchAttributes();
qCnt = q.ApproximateMessageCount;

Additional information: queue metrics are aggregated on service level other than queue level today. It’s in our roadmap but there is no concrete release plan yet.

The workaround is to run a reoccurring job to query the status of each queue. For example, query approximate message with Get Queue Metadata (REST API) - Azure Storage | Microsoft Learn.

Kindly let us know if you still have more questions on this., I wish to engage with you offline for a closer look and provide a quick and specialized assistance, please send an email with subject line “Attn:subm” to AzCommunity[at]Microsoft[dot]com referencing this thread and the Azure subscription ID, I will follow-up with you. Once again, apologies for any inconvenience with this issue.

Thanks for your patience and co-operation.

Hope this helps!

Kindly let us know if the above helps or you need further assistance on this issue.

-----------------------------------------------------------------------------------------------------------------------------------------------------------

Please don’t forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Hello

So I'm aware of peek and such as well I'm using all these things. I was just trying to dissect what appears to be a misbehavior/bug of the api.

So using golang.

Take

// Assume this pointer is connected properly already
queueUrl *azqueue.QueueURL 
props, _ := queueUrl.GetProperties(ctx)
messageCount := props.ApproximateMessagesCount()  // this reads header: x-ms-approximate-messages-count:int-value)

This will initially return 0 records on a brand new queue. Also I only have the One queue when testing this. No other queues in target subscription/resource group.
Then manipulate the Queue ONLY in the Azure Portal
Add a message (doesn't matter the contents) and run the above. Here's the flow

Add message (expire 7 days) (in portal)
messageCount == 1
Add message (expire 7 days)
messageCount == 2
Delete message (doesn't matter which)
messageCount == 1
Add message (expire 7 **seconds**)
messageCount == 2
wait about 10 seconds (in portal the 7sec expire message disappears)
messageCount == 2
Delete remaining visible message
messageCount == 1

And it stays that way forever more as far as I can tell.

thanks

@Sumarigo-MSFT

Could you share with us if there is any further update on this ? We want to use queue length for a scaling decision and hence need it to be reliable.

Hello Blue
I moved out of Azure to AWS due to it's excessive costs and bugs like this. They did contact me directly to "work through it" but if it can't be worked through on the public forum (so others can benefit as well) then I don't have time for it. Also the above code example showed the error very clearly. So I don't think they tried to reproduce.

I have not bothered testing since moving off Azure. But since it's been a bug for so long (issue linked in my first post above is from 2018) I'd assume it's still there. You can test using the above code flow easily though.

So I'd test before committing to it, but I'm also assuming it's still a bug. Oh and Peek won't work for scaling decisions in my opinion. As I recall it can only peek a max of 30? So if you have 3M messages you only know there are 30 more.

good luck!

Thank you for the detailed update @voteplatypus
From my analysis i am also disappointed at such a basic feature out of place, but i see no choice, but to avoid depending on it too.

We will still continue to use storage queues, but making sure our messages don't expire (high ttl), so that we don't get incorrect queue length counts, since our scaling logic does depend on it.