Firefox & HTTP pipelining

[Moonchild]

Excuse the very lengthy post. It serves to illustrate a point, I swear!

This was mildly amusing (if sad) so I thought I'd share this all with you.

There is this 10 year old bugzilla bug, bug #264354, dealing with HTTP pipelining, the bug report asking to enable it by default. It was very recently marked WONTFIX by the powers that be at the Mozilla Corporation. Before I go into the details, a little fill-in as to the background of the topic:

• HTTP pipelining is part of the HTTP/1.1 protocol standard which has been out for a very long time (hence the age of this bug). It's clearly defined in RFC 2616, and is available in just about all servers and browsers on the market.
• Pale Moon has had pipelining enabled for a very long time already (several years) because it is several orders of magnitude more efficient than plain HTTP "one element at a time". Over time, I've found a good, balanced set of settings for pipelining as the implementation in Firefox evolved, effectively and efficiently using it for web traffic that works for everyone.
• Google's SPDY protocol does something similar in a more complex way, but it is a protocol that is very restricted in its use, and very few websites on the web actually offer the protocol (even if it's available in the most-used web server software out there). Most importantly, SPDY only works over HTTPS when certain conditions are met.
• HTTP/2.0 is in development, which is not going to be a human-readable protocol and much more complex to implement in software, although it offers better efficiency for HTTP transports. The current work on that draws on the basics employed in SPDY. But: It is currently not even a proposed standard or draft (that has a rough estimate of November 2014) and will likely not see any sort of practical or widespread implementation in servers or clients for at least a few years, if ever.

The pipelining bug recently saw some renewed activity, actually by someone called RH offering to help move the bug forward since the last two comments by myself and one other adding feedback that pipelining was working very well in a practical sense on a very large number of different systems (this offer to help prompting the WONTFIX and resulting in some discussion afterwards), of which I will highlight the most important things here for you to read, and see how Mozilla is shooting itself in the foot (yet again).

this bug should have been closed a while ago - pipelines have been passed by multiplexing approaches such as spdy and http/2. The inherent head of line blocking challenges in pipelines make them perform too inconsistently.

NEW --> RESOLVED, WONTFIX

I know that stream multiplexing {sic: this is what HTTP/2 is supposed to offer and what SPDY offers in a limited way} is an alternative approach to what pipelining achieves.

So, because people are too dumb to implement HTTP/1.1 (RFC 2616, which clearly specifies pipelining) correctly, let's create HTTP/2.0 and let Google force it. VERY VERY VERY WISE.

HTTP/2 = not implemented by anyone, not even a standards proposal yet.
SPDY = HTTPS only, and having its own set of pitfalls.

Regular HTTP traffic (what, 95+% of all traffic on the web?) could definitely use the advantages of pipelining, so marking this WONTFIX in light of a yet non-proposed standard is incredibly short-sighted, IMHO. Pipelining is implemented in gecko, it works (and works very well), it's part of the HTTP/1.1 standard, so use it already

pipelines inherently suffer from hol problems even when implemented bug free.

To use them correctly you need to know what the rtt is going to be, the size of the object being transferred, and the effective bandwidth of the pipe at that moment. That's pretty unknowable.

...

So desktop isn't really a good match at this time.. I'm going to leave it on for mobile for now as the higher rtts tip the balance in its favor - but only by a little bit. trying to refine it further is not a great plan when multiplexing approaches are fixing its fundamental problems - we're investing in http/2 instead as the path forward on this problem.

I don't agree with your assessment of google's role here.

Indeed, there are issues with pipelining and it's not a perfect solution for all problems. However, it has been specified by the standard and in the average case, it would offer good performance improvements. I agree that it's not a very useable solution {sic: would not be a good final solution for HTTP transports in the long run} and multiplexing (along with other improvements by HTTP/2.0) are better solutions.

But: Given that it's not even possible to implement HTTP/1.1 (which includes pipelining) correctly within 10+years (on server as well as on client side), how would it EVER be possible to implement HTTP/2.0 correctly, especially considering that large parts of HTTP/2 incorporate HTTP/1.1 syntax?

> I don't agree with your assessment of google's role here.

HTTP/2.0 is based on SPDY, which is only successful because Google created it and enforced it via Chrome, isn't it?

SPDY is successful because it works well and everyone who is already implementing it in their web server software (Apache, NGinX, etc) is allowing users to reap the benefits. Pipelining is a woulda-coulda-shoulda solution that should have had it's day in the sun years ago but something better has come along. On paper, pipelining is great. In practice, it's a fustercluck. Too many chefs have spoiled the pot.

I'd been using it enabled at my org for many years but SPDY is just the next natural evolution to what Pipelining was *supposed* to be. The odds of web servers out there actually using Pipelining is slim to none. Even Microsoft is conceding their answer to SPDY / Pipelining is no match for SPDY. SPDY/4 will likely be what HTTP/2.0 will become and we'll all be the better for it. There's nothing complex about it. It's simple implementation. But please do not make the argument that SPDY is a Google only baby, it's not. Anyone can now implement it and it will be an open and ratified standard by, likely, 2015. It's pointless to wast man-hours on a dead standard.

Sales pitch aside, in 2014, use of SPDY was down from 1% in April 2013 to 0.6/0.7% now. If it was that fantastic, you'd expect greater adoption. So no, it's neither a widely implemented protocol, nor increasingly used, despite Google's pushing.

As for pipelining's potential hol issues, the Firefox implementation has implemented rescheduling of elements to a different pipeline and fallback to non-pipelined for a reason, in case there are "holdups". If you set the pipelining parameters in a sane way (keep pipes short and use several in parallel) it results in a big practical win, even with short rtts.

In all practical respects enabling pipelining, even if not a "holy grail" solution, will be beneficial for everyone except a very small percentage running into technical problems (usually proxy related or talking to ancient servers that don't implement http/1.1 properly and are pretty much broken).

I think you should be careful not to have tunnel-vision here. "But http/2 will be awesome" is nice, but doesn't help us now, nor for some years to come.

>...use of SPDY was down from 1% in April 2013
> to 0.6/0.7% now.

That does not reflect our userbase. {sic: really now? }

> As for pipelining's potential hol issues, ...

I know better than anyone. I wrote it. Its good - but not good enough.

Just for an example, activating pipeling for https will immediately make my ISP's account management interface completely broken. And it happens to be the largest ISP in Australia.

Saying that they're stupid might be truthful, but doesn't change that fact.
------
I agree that SPDY is not perfect - annoyingly it only works for servers that have a verifiable identity on the internet (ie. the server has to know its global address and be able to prove it), which is a larger scope than HTTP and not all HTTP servers work like that.

But if pipelining doesn't work, then it doesn't work

(In reply to Patrick McManus [:mcmanus] from comment #52)
> That does not reflect our userbase.

http://w3techs.com/technologies/details/ce-spdy/all/all
So, either your metrics are way wrong, or somehow Firefox users only visit SPDY-enabled sites. Which of the two is more likely?

> I know better than anyone. I wrote it. Its good - but not good enough.
Is it better than plain HTTP's one-element-at-a-time? I think so. And just brute-forcing it by opening a different TCP connection for each element is hardly a solution, either (and probably slower, anyway, for small elements). {sic: not to forget the fact that most residential gateways and wireless routers will choke big time on that}

(In reply to Radek 'sysKin' Czyz from comment #53)
If the fact is that they need to update their management interface's webserver to something that supports a 10+ year old standard, or configure it properly, then I'm not saying it isn't a fact. I'm saying that that is something that can be fixed easily by affected providers, and should be fixed. They may not be aware. And I think it would take very little time for them to correct that -- but that's hardly something that should hold back a good technological solution and standard.

> But if pipelining doesn't work, then it doesn't work
Pipelining DOES work. And it works very well. See my post (and others) before where pipelining as-implemented in Firefox is and has been enabled for years already, with barely any ill effects.

Sorry, I wasn't clear there: the "fact" I was talking about was that this specific website will not work with pipelining on.

And for a consumer product like Firefox, it is not acceptable to release a version where this doesn't work, when it worked on the previous version. (this is obviously not a fact but my opinion).

Would you support integrating other standards violations (and to repeat it one more time: pipelining is a MUST according to RFC 2616: "A server MUST send its responses to those [pipelined] requests in the same order that the requests were received."), too just to keep the admins of your ISP happy?

> And for a consumer product like Firefox, it is not acceptable to release a...
So Firefox' CSS rendering should have never switched from quirks mode to standards mode? There were so many Web sites that stopped "working"...

(In reply to Radek 'sysKin' Czyz from comment #56)
> And for a consumer product like Firefox, it is not acceptable to release a...
By that reasoning, any non-compliant server would be a blocker for any product progress. That's not how things work. Regressions do happen, and if they are caused by broken servers, then the servers need to be fixed, not the browser.
Pipelining is part of the standard. If the management server breaks, the server is not RFC compliant.
...
The fact that it doesn't work properly on a very small percentage of non-compliant/broken servers (and that is indeed a fact) should not mean everyone else has to be disadvantaged.

At which point a "huff'n'puff" ivory-tower attitude person stepped in with the following:

Please stop spamming this bug.

It is very simple:

Pipelining has been an undeniable pain in the ass. Nobody has gotten it working properly without hacks and even then problems pop up. It should work, but it is nonetheless a mess. It would be nice if we could get it running now, but obviously that hasn't happened. THERE IS NO POINT IN DEBATING THIS. Yes, servers should be fixed, but they aren't. Yes, heuristics to get it working are possible, but they're still not idiot-proof. There is nothing productive in rambling on this topic here.

The guy in charge made the decision that we failed to get this to the point where it could be turned on by default. That's his decision. Mozilla has limited resources. Working on a coherent strategy to fix the issues pipelining aims to fix through a standard that doesn't have the same issues is the way forward. Yeah, it kinda sucks that we won't get this on by default in the meantime, but that's just the way it's going to be. You can still turn it on for yourself if you want. Nobody is talking about yanking it yet. {sic: "yet"...}

SPDY is an option and it does work now. Yeah, it isn't as widely used, though the tiny fraction of the Internet that does use it has a LOT of users, so there is a disproportionate benefit from it. Yeah, it's not ideal either. {sic: the tiny fraction which happens to be Mozilla's "limited resources" main benefactor. Yup.}

Unless you are a peer on this project there is nothing productive you can post here. Please stop. You're just wasting people's time and flooding inboxes. If you want to debate this at length, do so somewhere else.

In short "We don't want to hear your arguments. We decided that this (Google SPDY) is Good (because it works on a few high-traffic sites that all happen to be Google buddies), should be the only thing used, and you should shut up and go away". Nice, eh?
After that little essay, I decided to let Mozilla shoot themselves in the foot (again) instead of actually keeping an open forum for fellow developers and peers who have practical experience in these matters.

[dark_moon]

Mozilla is so argh. Why in hell is mozilla going to such a bad company?
Where is the mozilla what build a browser for and with users, like at the beginning?

Thats bad and sick like your Earthlink support mail. Thanks for sharing this!

[megaman]

Mozilla leaves me without words. (Other than these saying that Mozilla leaves me without words.)