A more compatible user agent string

Talk about code development, features, specific bugs, enhancements, patches, and similar things.
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.

This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.

Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
access2godzilla

A more compatible user agent string

Unread post by access2godzilla » 2014-10-18, 18:46

I'm aware of the need for distinguishing PM from FF, and I fully support the change in the GUID. However, site owners are very often reluctant to use compatibility checks and keep on using user agent detection.

In the light of this, and to reflect the present direction of Pale Moon, could this be set as the default UA string:

Code: Select all

Mozilla/5.0 ([...]) Gecko/[...] PaleMoon/[...] (Unlike Firefox/32.0)
The UA would still maintain compatibility with most sites, while also speaking a lot about Pale Moon itself xD (I'm using that UA myself on my home computer which has v25 installed.)

It may not be so much of a good idea to put "Firefox/32.0" in there, but putting "Firefox" could be at least considered.

Sariel

Re: A more compatible user agent string

Unread post by Sariel » 2014-10-18, 19:25

Well, that is pretty much the default UA string in PMv25.0.1 outside the Firefox Compatibility Mode and site-specific overrides.

access2godzilla

Re: A more compatible user agent string

Unread post by access2godzilla » 2014-10-19, 10:54

Pale Moon identifying itself as Firefox 24 would be incorrect, because it has improvements ported over from later versions of Firefox. Hence the recommendation.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35649
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: A more compatible user agent string

Unread post by Moonchild » 2014-10-19, 11:26

If your idea is to satisfy UA scripts, while having a human-readable "jest" in it, then a /version portion is required.
I can't even imagine how many UA scripts would break more horribly than they already do now if it finds Firefox but not /nn.nn - so that would be way worse for Pale Moon users (undefined path instead of unknown browser).

It also really doesn't tackle the problem, the reluctance of website operators to stop using bad practices, and only tackles the symptoms.

The same UA sniffing websites when seeing Firefox/32.0 or similar will "jump to conclusions" on which specific subset of HTML/CSS is implemented, assuming Pale Moon has the same subset as Firefox 32.0 -- which is likely not the case; Pale Moon will have some things implemented that Firefox does not, and the other way around. A good example of why UA sniffing is bad.

Also, it's not like it's difficult to switch to a better detection that even makes webdesign easier by converting necessary unsupported features to supported ones, like using the pre-made Modernizr library, html5shim, and the likes. Hands-free stuff, really, and compatible with any browser, even ones completely unknown to the website owners. No to sound like I'm blatantly plugging one particular library over others, it's an example. Why keep reinventing the wheel?
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

access2godzilla

Re: A more compatible user agent string

Unread post by access2godzilla » 2014-10-19, 12:05

Moonchild wrote:If your idea is to satisfy UA scripts, while having a human-readable "jest" in it, then a /version portion is required.
I don't really know if a version portion is required. IE11, for example, uses:

Code: Select all

Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko
Note the absence of the "/{date}" in it. (BTW, the jest was suggested because everyone uses "like foobar" in their UA strings. PM has its roots in FF but it isn't FF, so why tell its "like Firefox"?)
Why keep reinventing the wheel?
I'm not a huge fan of libraries, but in any case your point stands: feature detection should be used instead of user agent detection. However, telling that to website owners is pretty much an exercise in futility...

User avatar
Sajadi
Board Warrior
Board Warrior
Posts: 1227
Joined: 2013-04-19, 00:46

Re: A more compatible user agent string

Unread post by Sajadi » 2014-10-20, 10:32

More simpler way... Why not implementing from start a

general.useragent.override

override with a "browser clone UA" - any of recent browsers would be enough. Best would be the most recent Chrome/Safari User agent.

That would also work for most people just out of the box and more or less flawless with a majority of pages.

After all it is the topic which many many many people do complain recently. So ignoring this matter and not offering them such a "out-of-the-box-solution" leads only to more anger, confusion or simply switching to other browsers, and i am sure no one want that to happen. And for many people, just giving the suggestion to "go and install an add-on for user agent changes" also does not look very professional and competent. And many users which do not like to fiddle around with settings or third party software would be thankful for that.

So, this feature request would increase usability and instantly kill off a lot of complaining.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35649
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: A more compatible user agent string

Unread post by Moonchild » 2014-10-20, 11:36

We are not ignoring this issue, we are very well aware of it, however, it is something that is an increasingly complex issue for any new browser if UA sniffing remains and increases as a practice. Every browser is being forced to lie about what it is -- then what's the point?

The "out of the box" solution will be a checkbox to enable Firefox compatibility mode in the next version. In Options -> Advanced.
It will still be "off" by default, but it's taking a page from the book of SeaMonkey to have an easily toggleable option to insert Firefox/nn.nn into the UA if people either don't want to go into about;config or if they frequent too many unresponsive sites. The main sites that keep being repeated though are Google & Netflix.
It doesn't really matter how "big" these guys are, it's still a small number of sites that no longer work as expected.

A few more UA strings will also be added as temp workarounds for some prominent sites while we wait for replies from corporate offices.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
Trinoc
Fanatic
Fanatic
Posts: 199
Joined: 2013-10-24, 18:09
Location: UK

Re: A more compatible user agent string

Unread post by Trinoc » 2014-10-20, 12:39

I think the user agent string should be thought of as advertising which features a browser is capable of supporting compatibly with other browsers, rather than calling it "faking" another browser, which seems somehow sneaky.

New Tobin Paradigm

Re: A more compatible user agent string

Unread post by New Tobin Paradigm » 2014-10-20, 13:17

There are better more modern ways of determing exactly what the client supports that makes ua detection moot and obsolete .. there is no reason in this day and age that we should be using a ua to guess so broadly a feature set when we can determin it exactly via other means

User avatar
Trinoc
Fanatic
Fanatic
Posts: 199
Joined: 2013-10-24, 18:09
Location: UK

Re: A more compatible user agent string

Unread post by Trinoc » 2014-10-20, 13:43

Matt A Tobin wrote:There are better more modern ways of determing exactly what the client supports that makes ua detection moot and obsolete .. there is no reason in this day and age that we should be using a ua to guess so broadly a feature set when we can determin it exactly via other means
True, but it's not going to go away any time soon, so I guess we'll have to live with it. In fact, maybe that's all the more reason to set it once and for all to something that works with whatever most web sites do with it these days, then it can quietly become obsolete as web sites (hopefully) stop using it.

New Tobin Paradigm

Re: A more compatible user agent string

Unread post by New Tobin Paradigm » 2014-10-20, 14:04

The ua still remains in user control and the knowelge is all over the web not to mention this forum so yeah not too bothered with it

User avatar
Trinoc
Fanatic
Fanatic
Posts: 199
Joined: 2013-10-24, 18:09
Location: UK

Re: A more compatible user agent string

Unread post by Trinoc » 2014-10-20, 14:22

Matt A Tobin wrote:The ua still remains in user control and the knowelge is all over the web not to mention this forum so yeah not too bothered with it
All I'd say is this: There is a whole potential user base out there of disaffected Firefox users who have either tried Australis and hated it, or heard about it and decided they won't like it. Many of these will be desperate for a browser which looks and feels like pre-Australis Firefox and, most importantly, works out of the box rendering all current web pages at least as well as the last pre-Australis Firefox did. Any web sites that revert to lower feature levels will put some potential users off. A good impression in the first few minutes of use is the most important thing to keep a user.

Once a user has had Palemoon for a while they will become attached to it and prepared to tinker a bit as web sites and other browsers move on, but the first impression has to be good.

So I ask: apart from the political reason of "not wanting to look like a fake Firefox", is there any technical reason why the default user agent setting in future PM releases should not be one that advertises compatibility with Firefox up to whatever pre-Australis level (28? 31?) it can handle?

Sariel

Re: A more compatible user agent string

Unread post by Sariel » 2014-10-20, 20:12

Trinoc wrote:So I ask: Apart from the political reason of "not wanting to look like a fake Firefox", is there any technical reason why the default user agent setting in future PM releases should not be one that advertises compatibility with Firefox up to whatever pre-Australis level (28? 31?) it can handle?
The thing is there will be no support for pre-Australis Firefox, since the new ESR (31) already has Australis on, which means that websites that only support the latest release+few versions back will be dropping ESR 24 (Pale Moon's base) support soon. This means that while we can lie about being ESR 31, that lie might give rise to incompatibility issues, since we obviously won't be/nor have all of th features the actually supported browsers will. Which is why Pale Moon needs his unique user agent, in order to gain support (and actually appear in browser-use stats), which would hopefully help with gaining support from many of lazier sites/companies.

User avatar
Trinoc
Fanatic
Fanatic
Posts: 199
Joined: 2013-10-24, 18:09
Location: UK

Re: A more compatible user agent string

Unread post by Trinoc » 2014-10-20, 20:39

Sariel wrote:The thing is there will be no support for pre-Australis Firefox, since the new ESR (31) already has Australis on, which means that websites that only support the latest release+few versions back will be dropping ESR 24 (Pale Moon's base) support soon. This means that while we can lie about being ESR 31, that lie might give rise to incompatibility issues, since we obviously won't be/nor have all of th features the actually supported browsers will. Which is why Pale Moon needs his unique user agent, in order to gain support (and actually appear in browser-use stats), which would hopefully help with gaining support from many of lazier sites/companies.
I'm not suggesting lying about anything. I see that Palemoon 24 advertises compatibility with Firefox 28.0, and I presume anything that was supported by PM24 is also supported by PM25, so PM25 could safely advertise Firefox 28.0 but possibly nothing later (unless PM25 adds compatibility with a later but still pre-Australis Firefox, in which case that later version could be advertised).

Any web sites which add features specific to Australis will (or should) key them to whatever higher version of Firefox is necessary to support them, which means they will not be used with PM. Depending on how much the site wants to support older formats, either it will send Firefox 28 compatible stuff to PM or it will fall back to "other browser" safe mode. Either way we would be no worse off than the current situation in which it would always fall back to "other browser" safe mode, even on sites that could support Firefox 28 features.

The important thing, as far as I can see, is to make sure that the transition to PM25 does not cause any web sites to be rendered less well than they were under PM24. As for the future, what will be will be, but whatever it is, it will have had the same effect on both PM24 and PM25, and it is something we will have to deal with when it happens.

Sariel

Re: A more compatible user agent string

Unread post by Sariel » 2014-10-20, 21:55

Sadly, websites don't work how you'd want them to and they won't even take into account soon Firefox versions under 31, thus keeping Firefox 28.0 in the general user agent might not be pointless in the short-term, but it actually goes against Pale Moon's survival in the long-term. While I can see your point of trying to keep the new users, I'm glad the Pale Moon team took the step of differentiating themselves before it was too late (since they didn't wait until every major site dropped support for non-31&up ie Australis Firefox versions). Otherwise, we would eventually be forced to having the "other/old browser version of every single non-standard complying/UA sniffing site, sites not working, which would eventually force us to move to another browser.

TL;DR We can't wait till Pale Moon is broken to fix it.

User avatar
Trinoc
Fanatic
Fanatic
Posts: 199
Joined: 2013-10-24, 18:09
Location: UK

Re: A more compatible user agent string

Unread post by Trinoc » 2014-10-20, 22:24

Sariel wrote:TL;DR We can't wait till Pale Moon is broken to fix it.
OK, but I still can't see a single situation where including Firefox 28 (or whatever) in the UA string would make things worse than leaving it out. Web sites that ignore UAs will ignore it. Web sites (if any) that specifically support Palemoon will read the Palemoon version and ignore the Firefox version. Web sites that assume everyone is on Australis and send stuff specific to Australis will fail on Palemoon whether Firefox is in the UA string or not. But there are some web sites (as we've already seen here) that correctly interpret the Firefox version and use features which both pre-Australis Firefox and Palemoon can display correctly, but in the absence of Firefox in the UA string causes them to fall back to sending fewer features.

Maybe I'm missing something, but can you please talk me through any plausible situation in which having Firefox in the UA makes things worse than not having it? And if there are any such cases, how likely are they to happen compared to the cases (some of which we already know about) where missing out Firefox makes it render worse than putting it in.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35649
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: A more compatible user agent string

Unread post by Moonchild » 2014-10-21, 07:50

Example situation:
Firefox 28 supports features a, b, and c, but not d.
Pale Moon 25 supports features a, b, and d, but not c.

If a website with fine-grained UA checking finds "Firefox/28" in it, it assumes features present are a, b, and c, but not d, and serves page code relying on c and using a workaround for d. This will not render well in Pale Moon, because c is not supported, and the workaround for d might not work either because it could be mutually exclusive with actually having the feature.

If, instead, the website would use capabilities detection (e.g. using @supports in CSS), it will render correctly on both browsers, since it will find (a,b,c,!d) when Firefox users visit, and (a,b,!c,d) when Pale Moon users visit. No assuming needed, and the only time it would render incorrect would be if either browser lies about what it really supports.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
Trinoc
Fanatic
Fanatic
Posts: 199
Joined: 2013-10-24, 18:09
Location: UK

Re: A more compatible user agent string

Unread post by Trinoc » 2014-10-21, 11:29

Moonchild wrote:Example situation:
Firefox 28 supports features a, b, and c, but not d.
Pale Moon 25 supports features a, b, and d, but not c.
OK, clearly I'm missing something about how this stuff is supposed to work. I was under the impression that (in this example) a browser should only advertise "Firefox/28" if it supports all of the features that Firefox 28 supports (in this case including c).

I always assumed that the fact that PM24 had "Firefox/28" in its UA string, I could count on it to render correctly any page that Firefox 28 could render correctly.

If instead PM (24 or 25) only supports all features of (say) Firefox 24 then surely the UA string should contain "Firefox/24". As for any additional features in PM but not in Firefox 24, any web site which had been updated to recognise "Palemoon" in the UA string would key the features it used on this rather than the Firefox version.

It seems odd that the UA string should mean anything less than "This browser support all of the features of all of the browser types listed here".

Obviously other ways of determining features (such as CSS) are better, but as long as there are web sites out there that use UA then it makes sense to support it.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35649
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: A more compatible user agent string

Unread post by Moonchild » 2014-10-21, 20:11

A divergent product will inherently not have all the features of another product, regardless of version. This is the very reason why "having to lie about which browser you are" is bad.
Pale Moon is not "Firefox with added features on top of everything it has", it is "A Mozilla based browser with different features". If you are looking for the "lowest common denominator" then you will be looking further back since some features have never been in Pale Moon. In the example above, it would need to be the version that only supports a and b, since that is what perfectly matches.

So it's not a matter of finding "what will have 100% coverage of the related Firefox product" but rather finding "what it is functionally closest to" if you are going to have to lie about what you are. It's about striking a balance, and it depends greatly on the website you're visiting and what assumptions that website makes of the browser based on the arbitrary string it receives.
If you want it to be "This browser support all of the features of all of the browser types listed here" then Pale Moon won't actually be able to carry any "Firefox" ID in the UA string, because it has, from its inception, always been built with a number of unwanted components left out.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

CraigPD
Lunatic
Lunatic
Posts: 292
Joined: 2013-01-01, 19:03
Location: Mexico

Re: A more compatible user agent string

Unread post by CraigPD » 2014-10-24, 16:42

Moonchild wrote:The main sites that keep being repeated though are Google & Netflix.
It doesn't really matter how "big" these guys are, it's still a small number of sites that no longer work as expected.

A few more UA strings will also be added as temp workarounds for some prominent sites while we wait for replies from corporate offices.
MC, I see you've added an override for netflix.com in 25.0.2 but nothing for google.com. A change of strategy?