Cloudflare checks broken again?

[digitalaudiorock]

In order for the captcha to work, just add "general.useragent.override.challenges.cloudflare.com" > "Mozilla/5.0 (%OS_SLICE% rv:56.0) Gecko/20100101 Firefox/68.0 Waterfox/56.6.2022.11" (unquoted) in about:config
Holy shit, cloudflare...

OMG. This worked for me on the site I've had issues with this time around. Thanks!...but yea, this crap is inexcusable.

Tom

[tristan9]

I can't imagine the average person running a website sees it much differently... they see being asked not to use CloudFlare as being asked to be more vulnerable to hackers and reduce speed, and would see any such request as extremely suspicious or at least inconvenient.

From the point of view of someone actually running a large website with our our own in-house CDN, these website operators are right to pick CF, unfortunately.
And that's even when I use or have used CF for other projects, and found its performance regularly subpar in various locations (or for specific services they offer). And its observability being nothing but catastrophic unless you pay a lot of money for their entreprise plan.

But for your average person running a wordpress/bb/... site for fun? It's seriously an uphill battle to DIY it all to the same level.
Yes, you could set up your own reverse proxies with their local caches, have very tight firewalls on all of them, then wire up a WAF somewhere in your edge, set up tight-but-loose-enough ratelimits, and then keep all of this updated regularly. And we do just that.
But it's a monumental timesink, requires a lot of expertise, and requires updating whenever there's new threats even aside from regular updates. Updates which also alternate between fixing stuff and breaking other stuff, and small-time website operators don't have dev environments perfectly mimicking their live environment. etc.
Oh and it's also a lot of extra resources to pay for along the way as all of this analysis/tracking/etc isn't magically processed.
And yet we still get owned with absolutely 0 recourse when some random fuck clicks the 350Gbps L4 attack button on their shitty booter.

And since the other big players of the web either don't need help (they are big enough that they already made their own CDN, and have teams dedicated to it), or they even sell their own CDN product, the result is that no one with decision power is interested in trying to improve things at all.
Because doing so is hard, time-consuming, requires convincing people that are notoriously annoying whenever you suggest changing anything (administrations and large ISPs come to mind). All for exactly 0 business benefit to them.
Note that here it doesn't mean they are *against* improvement. But they don't have a business case for the large time investment in both technology and advocacy they'd have to do.

Basically, the problem seems to be that CloudFlare is using extensive feature detection to make sure that the constellation of features supported by a browser lines up exactly with one of the browsers they support. In some ways, feature detection which was always hailed as a solution to the problems of relying on user-agents, is turning out to be worse for Pale Moon, because in practice websites are using combinations of features and their implementation details to determine the exact browser engine and turn away any browser engine they don't recognize as potential malware. At least with user agents we could spoof them to get past the sniffing, with this they are actually challenging us to do every single individual thing their supported browsers do in precisely the way they do it as a way of determining what engine we are on, whether they actually need/use that functionality or not, with the point being to filter out unsupported browser engines.

It's unlikely that their setup is based around testing for a set of known-good browsers like that tbh. (No matter how they might market/describe how it works.)
Most likely, the approach CF takes (and everyone else for that matter) is a statistical one, such as:
1. Collect actual behavior during TLS handshake, JS challenges, etc. per advertised UA
2. Clean up and categorize samples, to keep only the "real" profiles (would be behavior exhibited by the vast majority of the samples that pass captchas for a given advertised UA)
3. Flag the properties that only a few samples (or captcha-failing ones, or abusive ones, ...) exhibited as signs of spoofing
4. Deploy new rules based on the new set of "known" behavior-per-advertised-UA alongside blocking profiles (rather than UAs) associated with negative behaviour

To make this work with minimal rate of false-positives, especially at the scale CF operates, you need a ton of data however. And even then it's never perfect.
And you can't reliably test it all in house because of the ridiculous number of possible OS x Device settings x Browser x Browser settings. Even if I'm sure they specifically test quite a few of the most popular cases, and then do very gradual rollouts.
Eventually during-and-after they are deploying, they likely look at false-positives, and PM unfortunately-but-understandably does not sit at the top of their priority list.

Yet that's not them having anything against PM specifically. It's more that mechanically it's much more likely to be flagged by this kind of approach, and less likely to get fixed quickly, as it probably has:
- a much-higher-than-average amount of traffic that *does* have spoofed UA (by necessity due to shitty sites sniffing UAs, yes, but still)
- a much smaller footprint overall, so its FPs aren't looked at in priority

In the end, your worries are justified though, and it is only going to get worse over time indeed.
But even so I wouldn't be too quick to blame CF. They might have set out to fight a symptom of the current Internet's issues (bots, DoS, ...) rather than the root cause of it, but it's a really tough job as-is.
And for what it's worth, the only ones that are trying to fix a root cause (malicious browsers) are Google with WEI. Which is not encouraging because they are doing it mostly to fight adblockers and the likes...
So pick your poison I guess.

The only way is to keep reporting PM issues to CF for now, so that they eventually get to it on their false-positives list, and fix it.
Or try and find a friend of a friend who knows someone inside CF who can fast-track the issue past their support bureaucracy.

[Moonchild]

In order for the captcha to work, just add "general.useragent.override.challenges.cloudflare.com" > "Mozilla/5.0 (%OS_SLICE% rv:56.0) Gecko/20100101 Firefox/68.0 Waterfox/56.6.2022.11" (unquoted) in about:config
Holy shit, cloudflare...

Other than that I feel completely powerless at this point

We still appreciate you and your work)

I've added this override to our dynamic user-agent updates which should have the workaround be picked up automatically by all installations; considering CF still hasn't responded.

[Moonchild]

It's unlikely that their setup is based around testing for a set of known-good browsers like that tbh. (No matter how they might market/describe how it works.)

It's literally on their website what they consider "supported"(1), and their conversation with me had me believe that they have a specific test suite they use (which doesn't include Pale Moon(2))

(1) https://developers.cloudflare.com/waf/reference/cloudflare-challenges/#supported-browsers
(2)

Basically, since Pale Moon is based on an older version of Firefox, we had no system in place to test for it.
We do state in our documentation which browsers are supported, and those are the browsers we test for

Of course I pointed out in reply that they really shouldn't do that because their short list of "supported browsers" is literally only major commercial ones, and UXP/Goanna is fully independently developed and won't exactly match, suggesting they add Pale Moon to their test suite. I also pointed out their challenges are deployed literally on the billions of web pages they proxy (and not just their own web interfaces where such a check would be acceptable if they wanted to not support other browser for managing CF by customers) whenever the website owner either uses defaults and it gets triggered for any of their arbitrary reasons, or explicitly enables higher security/"protection"...
Although I was never given another reply after that referring to "this is what we support" and was pretty much ghosted.

[Frugal]

I haven't applied the override, yet every one of the several sites mentioned in this thread I can now access - whereas just a few hours ago I couldn't get past the captcha loop on any of them. Suggest therefore Cloudflare have actually fixed something.

[Moonchild]

I haven't applied the override, yet every one of the several sites mentioned in this thread I can now access - whereas just a few hours ago I couldn't get past the captcha loop on any of them. Suggest therefore Cloudflare have actually fixed something.

I added the Waterfox override to our dynamic updates system for user agents to provide a global workaround for everyone since I don't know if or when this would get addressed from CF's side. That doesn't mean CF fixed it yet.

[Goodydino]

Since they written in the topic that cloudflare works on waterfox classic...
PM 33.0.1 x64, win7
Based on the results of a couple of quick tests - with "network.http.useragent.global_override" = "Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/68.0 Waterfox/56.6.2022.11" CF-captcha checking works fine. Tested on 4chan and fanfiction.net
Not tested with site-specific useragent, only with global override (maybe I'll check later, not much time right now)
Maybe someone could use

If that version of Waterfox is based on Firefox 68, should the rv number not be 68 rather than 56?

[Moonchild]

If that version of Waterfox is based on Firefox 68

far as I know it's based on Firefox 56. the /68 is just for compatversion, I think.

[Goodydino]

I tried that Waterfox override by adding it to User Agent Switcher (which I have patched and it works) for the global override. Cloudflare still does not work with Mac Pale Moon with the fanfiction.net site.

[athenian200]

From the point of view of someone actually running a large website with our our own in-house CDN, these website operators are right to pick CF, unfortunately.
And that's even when I use or have used CF for other projects, and found its performance regularly subpar in various locations (or for specific services they offer). And its observability being nothing but catastrophic unless you pay a lot of money for their entreprise plan.

And since the other big players of the web either don't need help (they are big enough that they already made their own CDN, and have teams dedicated to it), or they even sell their own CDN product, the result is that no one with decision power is interested in trying to improve things at all.
Because doing so is hard, time-consuming, requires convincing people that are notoriously annoying whenever you suggest changing anything (administrations and large ISPs come to mind). All for exactly 0 business benefit to them.
Note that here it doesn't mean they are *against* improvement. But they don't have a business case for the large time investment in both technology and advocacy they'd have to do.

Not saying they are wrong to choose CF, but rather that CF itself has chosen a mechanism that can't really be adapted to us. There's not even a specific web technology we're missing that they could tell us to adopt, because if I understand correctly they are basically generating a fingerprint here, and shutting us out because our fingerprints don't match anything they have on file. It appears to be a problem that our browser is too unique and thus stands out too much.

My issue is that I just don't see a solution to this problem that doesn't involve either begging website owners not to use CF, or begging our users to avoid websites that do. It seems very much like we're going to have to throw up our hands and tell our users to give up and use Chrome for about 60% of the web instead of just 40% of the web on account of this.

It's unlikely that their setup is based around testing for a set of known-good browsers like that tbh. (No matter how they might market/describe how it works.)
Most likely, the approach CF takes (and everyone else for that matter) is a statistical one, such as:
1. Collect actual behavior during TLS handshake, JS challenges, etc. per advertised UA
2. Clean up and categorize samples, to keep only the "real" profiles (would be behavior exhibited by the vast majority of the samples that pass captchas for a given advertised UA)
3. Flag the properties that only a few samples (or captcha-failing ones, or abusive ones, ...) exhibited as signs of spoofing
4. Deploy new rules based on the new set of "known" behavior-per-advertised-UA alongside blocking profiles (rather than UAs) associated with negative behaviour

That kind of approach obviously can't work with Pale Moon, though. It seems like it's trying to do what I suggested, but in a roundabout and unreliable way that involves statistics and methods that they won't disclose to us.

In the end, your worries are justified though, and it is only going to get worse over time indeed.
But even so I wouldn't be too quick to blame CF. They might have set out to fight a symptom of the current Internet's issues (bots, DoS, ...) rather than the root cause of it, but it's a really tough job as-is.
And for what it's worth, the only ones that are trying to fix a root cause (malicious browsers) are Google with WEI. Which is not encouraging because they are doing it mostly to fight adblockers and the likes...
So pick your poison I guess.

Yeah, unfortunately their solution to that tough job seems to have been to give up and go scorched earth against alternate browser engines. WEI would have the same problems as OAuth2 does for e-mail, in that every browser would likely have to jump through a lot of hoops to be validated in such a way that a proper third-party would vouch for the identity and be trusted by the server. It basically goes back to having to buy trust at an expensive price that indie browsers or e-mail clients just can't afford. At this point, what we seem to be getting is kind of like pseudo-WEI that's less reliable through the back door on centralized private infrastructure anyway.

So while WEI would likely lock us out completely on websites that adopted it, the alternatives to WEI employed by CF seem to be following the same path and trying to do what it was going to do anyway, but doing it badly with statistics rather than doing it "correctly" with third-party attestations. I'm honestly not sure what is worse here, WEI or CF's backdoor alternative to it using statistical methods that only work for major browsers who are likely the same ones that could afford the cost of obtaining third-party attestations anyway.

The only way is to keep reporting PM issues to CF for now, so that they eventually get to it on their false-positives list, and fix it.
Or try and find a friend of a friend who knows someone inside CF who can fast-track the issue past their support bureaucracy.

Well, they might not care enough to listen, possibly hoping that if they ignore the issue long enough, people will quit using Pale Moon because of how critical CF is to webmasters and choose their favorite websites over their preferred browser. :/

[Enobarbous]

I added the Waterfox override to our dynamic updates system for user agents to provide a global workaround for everyone since I don't know if or when this would get addressed from CF's side. That doesn't mean CF fixed it yet.

Maybe CF also solved the problem on their side: I checked on a clean portable version of pm33, with an empty ua-update.json, and checked that the original PM useragent is passed in the request header - the cloudflare check succeeds (even without the requirement to check the "I am a human" box). Looks like a silent fix
It looks like your support request was successful. Let's hope it's not for a couple of days...

[Frugal]

I added the Waterfox override to our dynamic updates system for user agents to provide a global workaround for everyone since I don't know if or when this would get addressed from CF's side. That doesn't mean CF fixed it yet.

Interesting! Then there's something going on that I don't understand, because ..

1. In about:support under Important Modified Preferences I have

general.useragent.updates.enabled false
general.useragent.updates.lastupdated 1664829997462 (= 3/10/22)

2. Nowhere in my about:support or about:config is there any mention of the
strings cloudflare or challenges or waterfox.

3. Restarting PM from a nailed-down read-only profile (last modified weeks ago)
then running tcpdump to capture connecting to http://4chan.org the only
useragent string I see being passed is

User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Goanna/5.0 Firefox/68.0 PaleMoon/30.0.0

4. For good measure I then locally blacklisted palemoon.org

dig dua.palemoon.org ..

;; ANSWER SECTION:
dua.palemoon.org. 10 IN HINFO "This query has been locally blocked" "by dnscrypt-proxy"

then restarted PM again (with the nailed-down read-only profile), checked it
definitely couldn't talk to dua.palemoon.org, then ran tcpdump again and saw
the same useragent string again, and still it happily connected to 4chan.org
and several of the other "problem" sites that are behind CF captcha.

So I'm baffled what's happening.

[stefan11111]

Seems to work now.

[cannonmc]

Yup. Only just checked here so haven't changed anything on PM but seems to be working for me now.

Thank heavens, Opera really is a messy browser

[Moonchild]

Nowhere in my about:support or about:config is there any mention of the
strings cloudflare or challenges or waterfox.

Dynamic UA updates are transparent. They won't show in preferences.

PaleMoon/30.0.0

I don't know if CF accepts that string or not. They are not communicating with me.

It's -possible- that they updated their code at the exact same time as me adding the override, but without hearing from them I can't tell if that timing leads to your confusion or not (or if this override is now no longer necessary)

[noellarkin]

I'm not too well-versed with these things, so I'm sorry if this is a naive question: how difficult would it be to set PaleMoon up so it always mimics the latest version of Firefox in the UserAgent passed in HTTP requests, and also mimics the FF fingerprint as closely as possible?
It seems to me that CF is going to keep prioritizing the most common browsers, Chrome and Firefox.
Since PM's engine is more similar to FF than it is to Chrome, would it be extremely difficult or impossible for Pale Moon to emulate FF's fingerprint out-of-the-box? Wouldn't that be a good long-term solution?

[Kand_in_Sky]

Seems they fixed it, it works (for) now

[Moonchild]

ok.. removing the dynamic override again then