SSL Troubleshooting for IIS Web Sites contd...

  • Artykuł

Recently a colleague of mine was working on a customer's case which was a
Critical level incident. High pressure job, huhh!

The issue was with SSL not working for one of their web sites. They were
seeing "Page cannot be displayed" when trying to access this site over SSL. It
worked just fine over HTTP.

In the System event log we were seeing this intermittently:

Event Type: Error
Event Source: W3SVC
Event
Category: None
Event ID: 1114
Description:
One of the IP/Port
combinations for site 'NNNNN' has already been configured to be used
by
another program. The other program's SSL configuration will be used.

We troubleshot on this issue for hours without luck :-(. We tried all the
steps I guess as mentioned here .

Here is what all we tried:

  • Checked the Certificate properties to ensure it was a valid one. It was
    good.
  • Yet, replaced the current certificate with a new one, still no luck.
  • Here customer had all the sites running under different IP addresses. Rest
    of the other sites were working over SSL, except this one :-(.
  • We ran SSLDiag which gave a misleading error.
  • We tried running the site on a different SSL port, still no luck.
  • We setup the securebindings metabase property for the web site in question,
    still no luck.
  • We ran netstat -ano to check for any other process listening on this port,
    everything looked clean. refer this.
  • We disabled all the 3rd party non-MS services, restarted Windows Server in
    selective startup mode, no luck.
  • We installed Windows Server 2003 Service Pack 1 32-bit Support Tools on the
    server    , ran the httpcfg query iplisten. It gave a clean output, no specific
    IP entries listed by it.
  • Restarted IIS/HTTP services umpteen number of times during the course of
    troubleshooting, no luck whatsoever. Even reboot was done a couple of
    times.

Finally after few hours of troubleshooting we decided to run this site on a
different IP address (we had thought of this earlier but our customer was under
a constraint) and hurray it worked this time!!!. Now everything was set but we
had a lingering question in mind as to why, why, why this site did not work on
that IP address we had. It had an entry in the Advanced TCP/IP Settings, was a
valid one in all the sense to our best knowledge.

Finally we figured out that there was a problem with the IIS SSL
listener.

To get a list of IP and port configuration binded to a certificate, run
"httpcfg query ssl". Here is an excerpt from a technet article:

The HTTP API enables applications to communicate over HTTP without
using Microsoft Internet Information Services (IIS). Applications can
register to receive HTTP requests for particular URLs, receive HTTP
requests, and send HTTP responses. The HTTP API includes SSL support so
applications can also exchange data over secure HTTP connections
without depending on IIS. It is also designed to work with I/O
completion ports. ....Such meta-information is maintained by the HTTP API in a metastore, and
is used to locate certificates for certificate exchange in HTTPS
sessions.

Below is a sample of a working and non-working
scenario:
------------------------------------------------------------------------------

\Program Files\Support Tools> httpcfg.exe query ssl

Working scenario:

IP                      : 192.168.100.118:443
Hash                  :
c96667684997887f 5b889b7b3f737c8c4da5f16
Guid                  :
{4dc3e181-e14b-4a21-b022-59fc669b0914}
CertStoreName           :
MY
CertCheckMode           : 0
RevocationFreshnessTime :
0
UrlRetrievalTimeout     : 0
SslCtlIdentifier       
:
SslCtlStoreName         :
Flags                   : 0

Non-working scenario:

IP                     : 192.168.100.234:443
Hash               
:
Guid                : {00000000-0000-0000-0000-000000000000}
CertStoreName :
(null)
CertCheckMode : 0
RevocationFreshnessTime :
0
UrlRetrievalTimeout : 0
SslCtlIdentifier : (null)
SslCtlStoreName :
(null)
Flags : 0

Here Hash will have the same value as the Thumbprint in your SSL certificate. You will notice that the Guid is all zero in a
non-working scenario. You may see the Hash either having some
value or blank. Even if we remove the certificate from the web site, and then
run "httpcfg query ssl", the site with all Guid as all "0" will
still be listed. If you see the GUID as "{0000...............000}, there is a
problem.

We need to remove this entry by running the command "httpcfg delete ssl -i
<IP:Port Number>". In the above example, we need to type "httpcfg delete
ssl -i 192.168.100.234:443". Once we remove it, then we need to reinstall the
certificate back on to the web site.

Also once certificate is installed, in the cmd prompt type in "httpcfg query
ssl" to confirm the GUID is no longer all 0.

This fixed the issue for the web site on the failing IP address.

Hope this helps someone.

Till next time, Cheers!

Comments

  • Anonymous
    June 01, 2008
    Dude, nice finding, it will really help in future, if at all I come across this type of issue. Will try it at my end.

  • Anonymous
    June 01, 2008
    Congrats for the solution, and thanks for posting it!

  • Anonymous
    June 02, 2008
    Good going. Its really helpful.

  • Anonymous
    July 01, 2008
    You're a life savior!!! Just what I needed!

  • Anonymous
    September 10, 2008
    Wow.  I just spent 4 hours trying to figure this out.  You're a lifesaver!

  • Anonymous
    September 26, 2008
    Hello, Not in IT just curious to how this stuff works. I am missing what appears to be a simple step/concept. Any and all clarification is welcome and appreciated!! General subdomain/host header question: My Senerio: Main site: test.com    HomeDir:  c:test  IP: x.x.x.x subsite: driving.test.com  Site files: c:testdriving  same IP Main site created in IIS (any version i guess) sub site setup using host headers: driving.test.com and www.driving.test.com binded to main site IP When "http://driving.test.com" is entered in browser how does the default page in the sub directory (c:testdriving) get configured to load? Do you specify home dir somewhere? Please fill in the blank for me Thnx, vin

  • Anonymous
    September 28, 2008
    Hi Vin, Are you seeing any error message when you go to the above URL? You don;t have to configure anything else here. If you have a site with the name driving.test.com, it should take you to the correct web site, provided you have a host header entry to differentiate various sites.

  • Anonymous
    February 05, 2009
    Having the same type of problem (we see the 0 guid and want to delete), but the delete command is failing with error code 2. What else could be holding it open?

  • Anonymous
    February 05, 2009
    Robert, Error code 2 indicates "Error_File_Not_Found". The combination of IP:PORT that you are trying is it valid on this server? Do you see it as listed in there. Perhaps you can share the complete command stack that you tried.

  • Anonymous
    February 05, 2009
    Yes it was valid. We worked around by changing to a different IP address in the pool and it works. For some reason, the system would not give up that address and we do not have complete control over the machine or the network so have to call it in to get it rebooted to see if it will release...

  • Anonymous
    March 24, 2009
    The comment has been removed

  • Anonymous
    September 14, 2009
    THANK YOU! I spent hours Friday trying to figure out why our www server was putting out an SSL certificate FROM ANOTHER MACHINE after I installed our new certificate from an authority. I ran through your previous post and everything checked out. A commenter mentioned the event logs... so I found the same error message you posted about here (but I don't have any other services running on that port). So I tried your httpcfg trick. In my case, I had data coming back bound to 0.0.0.0:443, but it was not the thumbprint of the new cert that I installed. So I used your command to delete it and then remove and re-assigned my new cert and BINGO! Thanks again very much for both of these posts!

  • Anonymous
    November 21, 2009
    I tried your solution, but I am getting HttpDeleteServiceConfiguration completed with 1019, which translates into "System could not allocate the required space in a registry log.” according to windows error messaging. The server is a development server, and it only had 200 megs free on the C: drive. It probably ran out of drive space at some point. More details are on serverfault: http://serverfault.com/questions/86975/problem-deleting-host-ssl-certs-in-iis-with-httpcfg-ssl-delete

  • Anonymous
    November 22, 2009
    The comment has been removed

  • Anonymous
    January 14, 2010
    The comment has been removed

  • Anonymous
    January 17, 2010
    The comment has been removed

  • Anonymous
    November 01, 2010
    Great solution, saved me hours today.  Much obliged!

  • Anonymous
    March 24, 2017
    Great solution, saved me from having to take more drastic measures! In my case there were two IPs bound to 4430.0.0.0:443172.20.0.9:443I ran "httpcfg delete ssl -i 0.0.0.0:443" to correct the issue, as suggested in your article.Thanks for posting!