[Errno 10054] An existing connection was forcibly closed by the remote host while requesting images from URL

Nam Ly 46 Reputation points
2021-03-07T13:25:24.537+00:00

Hi,
I tried to download the tagged images using Custom Vision API and in Python. I set the limit to 256 images. However, after 60-90 images, this error would show up:
[Errno 10054] An existing connection was forcibly closed by the remote host

Can I receive further help?

This is my code. I have edited out the project id and training key as I think this is private information.

https://pastebin.com/yVfG85hM

Update:
I added conn.close after data = response.read().decode(encoding). It is the later processing that is the issue.

I did a google search regarding urlretrieve() and [Errno 10054] An existing connection was forcibly closed by the remote host. There is a solution that add edthe proxy to urllib.request. This works most of the time. When it get [Errno 10054], the images downloaded are around 100 images, which improved compared to without proxy. I do not know why this is the case though. After adding the proxy, I get another type of error urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

Resources:
https://stackoverflow.com/questions/27732075/python-3-windows-7-connectionreseterror-winerror-10054-an-existing-connect/27732243
https://stackoverflow.com/questions/22967084/urllib-request-urlretrieve-with-proxy

Azure AI Custom Vision
Azure AI Custom Vision
An Azure artificial intelligence service and end-to-end platform for applying computer vision to specific domains.
247 questions
0 comments
Accepted answer
  1. romungi-MSFT 45,961 Reputation points Microsoft Employee
    2021-03-08T08:10:38.787+00:00

    @Nam Ly The above code seems to be keeping the connection open until the processing is done for all the images on the client side. Could you try to close the connection after the response from the API i.e after data = response.read().decode(encoding)?

    I think since the connection is open until all your images or items are processed on client side in the for loop the API is closing the connection after the default limit.
    The initial response object should contain all the image details from the API, could you consider to print this response and check if the later processing is the issue?

    Increasing the time out on your http.client call is an option but that can still keep the connection open for a long time until all images in the response object are processed.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.