Hotmail, Live, Outlook and Bing connection errors, and our security.

Pale Moon releases and site news
User avatar
Pale Moon guru
Pale Moon guru
Posts: 30036
Joined: 2011-08-28, 17:27
Location: Tranås, SE

Hotmail, Live, Outlook and Bing connection errors, and our security.

Post by Moonchild » 2017-05-29, 18:21

Today, our users started seeing connectivity errors when trying to connect to most Microsoft on-line services like Hotmail, Onedrive, Outlook, Microsoft Live, and even the https version of the Bing search engine. The culprit? misconfigured servers on Microsoft's side, specifically their so-called "stapled OCSP responses".

Now, this gets technical rather quickly, so a quick summary of what this is all about:

Certificate revocation and OCSP

Server certificates (used to verify secure https connections) can get revoked by their owners before they expire, e.g. when a private key gets compromised or a certificate is mis-issued by someone who doesn't have the authority to issue a certificate for that domain. There are several mechanisms available for this, of which the most widely-used one is the OCSP protocol. Because individual lookups to OCSP servers are not without issues (connectivity issues as well as privacy concerns), a better way to handle certificate revocation checks was created, where the secure server you are connecting to can provide an authenticated (signed) response along with the certificate that verifies that it hasn't been revoked.
This is called a "stapled OCSP response". It is a relatively short-lived OCSP response that gets sent along with the certificate information to a browser, which can then verify that the certificate is still valid without having to make extra connections or doing extra lookups to an OCSP server operated by the certificate authority.

To make this work, a server has to have the correct intermediate certificate installed on the server that belongs to the certificate presented to the public, and usually the server needs a full certificate chain available to be able to staple the proper response with a valid and verifiable signature.

Stapled OCSP responses have been accepted and processed by all main web browsers for a long time already (at least a few years), including Pale Moon, IE, Firefox, Opera, Chrome, and other Chromium-based browsers.

What happened with the Microsoft domains?

What happened is that servers for the domains mentioned did not use the correct certificate chain to sign their stapled OCSP responses. As a result, connections to the related https servers started to fail. But, notably, only from browsers using NSS (like Pale Moon and Firefox). Chrome didn't complain (more on that later). Edge was apparently also fine, but I haven't looked into why that is, myself.

From a browser's point of view, this should be considered (very) bad, because it looks like some other party (not being the authority that issued the certificate) is trying to tell the browser that a certificate isn't revoked. This party could be an attacker that is trying to use a revoked (mis-issued) certificate, for example.

Now, considering all browsers can be expected to support stapled responses, this highlighted a rather disturbing security issue with mainstream browsers: Apparently, only Pale Moon and Firefox (and rebuilds) are doing the correct thing. Stapling is defined in the standard specification RFC 6066, and that RFC states (in section 8) the following:
RFC 6066 wrote:Clients requesting an OCSP response and receiving an OCSP response in a "CertificateStatus" message MUST check the OCSP response and abort the handshake if the response is not satisfactory with bad_certificate_status_response(113) alert. This alert is always fatal.
This means that, without option, browsers must at all times abort the connection with an error if they ask for a stapled OCSP response, the server sends one, and that response is invalid for any reason. Additionally, the "This is always fatal" part means that no other method is allowed to be attempted to retry the connection.

The initial response from @Outlook on Twitter was:
This has something to do with the advanced security feature in Firefox. We recommend that you reach out to their support.
Well, not so much "advanced" since other browsers have also had this "advanced security feature" for years, so Firefox isn't being an odd one out there, but apparently "advanced" in such a way that it seems to be the only correctly implemented one.

Why didn't Chrome complain?

Chrome and Chromium-based browsers like Opera etc. didn't complain about this because apparently, Chromium isn't RFC compliant. They implement stapled OCSP response parsing, but seem to be ignoring invalid responses. It's unknown at this point if those responses are simply classified valid (with no further checking) and assuming the certificate isn't revoked in that case (which would be a serious security risk!) or if it retries the connection or uses a different method to validate (which would still go against the RFC spec which mandates the connection must be aborted -- and would not indicate attempted forgery to the user either) or no validation at all except what happens to be in their so-called "CRLSet", a Google controlled and very limited list of revoked certificates, simply assuming everything else valid (also a security risk).

A bug was opened to get answers about that and/or get this fixed. Chromium bug #727255

I need to access my hotmail/outlook/live account NOW! Help?!

For Pale Moon users who cannot wait for Microsoft techs to fix this issue, and must get access right now, no fear!
You can work around the problem in Pale Moon (and Firefox) by temporarily disabling OCSP stapling, making it fall back to other forms of checking (regular OCSP/CRL lookups) and avoiding the misconfiguration of stapling in the Microsoft servers.

To do so:
  1. In the address bar, type about:config and press enter.
  2. Confirm the warning that is presented and that you'll be careful.
  3. In the list of preferences, find security.ssl.enable_ocsp_stapling
  4. Double-click the line to set the value to false
Remember to set this back to its default value when you no longer need to work around this issue.
"Son, in life you do not fight battles because you expect to win, you fight them merely because they need to be fought." -- Snagglepuss