Azure function timer-based function, if timeout time and retry time are the same new instance being cancelled?

Bill Turner 0 Reputation points
2024-09-27T23:08:42.0066667+00:00

I have a long-running timer-based Azure function (yes, not good form for a long-running function, but...) The function timeout has been configured for 5 minutes in host.json, and the schedule starts the task every five minutes during a "run window" (i.e., 7PM to 8PM).

Since the function could time out, I've implemented cancellation by periodically checking with ThrowIfCancellationRequested - the catch logs that the function is being cancelled and re-throws.

What I have found is that the first invocation runs as expected. When the timeout occurs, the framework notifies that the timeout has been exceeded, that cancellation has been initiated, and that the function failed.

  • 2024-09-27T22:20:00.108 [Information] Executing 'StatementTask' (Reason='Timer fired at 2024-09-27T15:20:00.0952840-07:00', Id=1ff7645f-c2ed-4978-bed4-2458d94da172)
  • 2024-09-27T22:20:00.173 [Information] StatementTask started at: 9/27/2024 3:20:00 PM
  • ... <log messages during execution>
  • 2024-09-27T22:25:00.160 [Error] Timeout value of 00:05:00 exceeded by function 'StatementTask' (Id: '1ff7645f-c2ed-4978-bed4-2458d94da172'). Initiating cancellation.
  • 2024-09-27T22:25:00.181 [Error] Executed 'StatementTask' (Failed, Id=1ff7645f-c2ed-4978-bed4-2458d94da172, Duration=300082ms)Timeout value of 00:05:00 was exceeded by function: StatementTask

The timer then starts the next invocation of the task, but it immediately is terminated - it's like the cancellation token was the same between the two invocation!

  • 2024-09-27T22:25:00.223 [Information] Executing 'StatementTask' (Reason='Timer fired at 2024-09-27T15:25:00.2224013-07:00', Id=4ca19cf2-446a-4f8c-8ac8-8f5d42e61378)
  • 2024-09-27T22:25:00.223 [Information] StatementTask started at: 9/27/2024 3:25:00 PM
  • 2024-09-27T22:25:00.297 [Warning] 4ca19cf2-446a-4f8c-8ac8-8f5d42e61378 - StatementTask was cancelled - terminating
  • 2024-09-27T22:25:00.303 [Error] Executed 'StatementTask' (Failed, Id=4ca19cf2-446a-4f8c-8ac8-8f5d42e61378, Duration=81ms)System.OperationCanceledException : The operation was canceled.

After this, the timer kicks off the invocation at the scheduled time, and the cycle repeats.

Any suggestions on how to prevent the invocation immediately after the timeout from being cancelled as it starts?

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,953 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. hossein jalilian 6,830 Reputation points
    2024-09-27T23:24:25.0066667+00:00

    Thanks for posting your question in the Microsoft Q&A forum.

    It seems there may be an issue with how the cancellation token is being handled between function invocations.

    • Make sure each function invocation is using a new, separate cancellation token.
    • If you're using any static variables or singleton instances to manage state, ensure they are reset between function invocations.
    • For long-running tasks, consider using Durable Functions which are designed to handle extended execution times
    • Ensure all tasks are properly completed or cancelled at the end of each invocation

    Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.