Cloudflare checks broken again?

[digitalaudiorock]

There just ARE no words for this. Am I correct that that old post on their community forum is closed and won't accept any new replies??

EDIT: Never mind. Just saw the new thread there.

The good news for me is that shortly after the last go around with this 1337x.to stopped using this nonsense...however I use another site that's breaking.

Cloudflare has a lot of fucking nerve pretending to be the fucking gate keepers of the Internet when their bullshit isn't even browser agnostic.

Tom

[Goodydino]

(emphasis mine), i.e. latest version of Google Chrome, Mozilla Firefox, Apple's Safari, Microsoft Edge, and Google Chrome on Android and Apple Safari on iOS.
Anything else is fair game to block as far as they are concerned the way it's worded, and the only solution is to use a "major browser" according to their definition. Previously they were adamant they weren't intentionally blocking independent forks or browsers, but it seems they are just clamping down on it now.

Can Cloudflare tell what the browser is if its user-agent reads as Firefox or Chrome? What would tell it otherwise?
I am talking here about the real user-agent in the navigator information, not just the field in the browser's request header. Navigator info can be dug out by javascript.

[dim22]

Waterfox portable uses firefox 56 and there are no problems with CloudFlare. So it does not depend on which user agent you have

[Moonchild]

They do use user-agent strings as one of their criteria to "detect legitimate web clients" (they literally say so on their captcha info page) but it's not all they do. To the best of my knowledge they now do a very strict check on individual browser behaviour (e.g. exact match of javascript support etc.) meaning they are literally only accepting alternative browsers that are an exact match to what they call "major browsers". Since we have individual development our feature set does not match Firefox (we have many more features than old Waterfox, but it doesn't match exactly what Firefox's latest versions have...)

[Moonchild]

Needless to say, while we have used CloudFlare to help with bandwidth needs for our servers, I'll be working on terminating my pro account with them after our upcoming release. This has been totally unacceptable and I withdraw my endorsement of them after this debacle. Might mean our updates/sites may become slower as server load will increase (and I may need to get heavier VDSes...) but so be it.

[athenian200]

Overall though, what does this mean for Pale Moon as a project going forwards? Here's what worries me...

First, we had a very difficult phase where it took us a long time to implement critical new web functionality for various websites. We mostly overcame this and showed that given enough time, we can implement the functionality we need as websites start using it. That was something that was difficult, but possible. The worst sites to use with Pale Moon were always the big, corporate sites that always use the latest frameworks.

But now this CloudFlare thing is different. Lots of smaller, more independent websites rely on CloudFlare and don't have a ready alternative because they fear hackers knowing their server's real IP and can't eat the bandwidth costs of running a site with its IP exposed directly to the Internet, basically. They are totally dependent upon a CDN to stay online, and they don't have their own, so they rely on a centralized service like this one to stay online. I remember I tried arguing with that John Connor guy about it once, and he just didn't get it and thought his point about security and speed trumped everything else and I was being dumb, so I can't imagine the average person running a website sees it much differently... they see being asked not to use CloudFlare as being asked to be more vulnerable to hackers and reduce speed, and would see any such request as extremely suspicious or at least inconvenient.

So basically, the problem is this... Pale Moon doesn't work particularly well with the big websites that can afford their own CDN that is a little more reasonable than CloudFlare, and doesn't fall back to such ridiculous measures. Further, the CDN relied on by most smaller websites PM users would otherwise prefer to use is now hostile to Pale Moon, and there aren't ready alternatives that webmasters are likely to switch to.

Ultimately, this reminds me of a conversation I had with my Dad a long time ago that really got on my nerves, when I was complaining about Google controlling everything with Android and Chrome. He pointed out that Google controlled everything because they built out the foundations that allowed stuff made by various companies to work together, while all the others were focusing on working inside silos promoting their own offerings. I then complained that was actually a terrible situation, because it was like they controlled an entire "grid" that the whole Internet and smartphone ecosystems runs on, and can decide what does or doesn't run on it. He just shrugged and said, "Well, it's not your grid, is it? Of course you don't get a choice what runs on it, or what they trust to run on it." And that was the end of the argument, there wasn't much I could say to it. He also went further and said that any attempt to build your own grid wouldn't work, because that would be seen as just building another failed silo like those other companies trying to compete with Google did, so now doing things Google's way is seen as being interoperable, and doing things any other way is seen as isolating yourself in a silo.

Overall, it seems like this is the Achilles' heel of the entire project in a nutshell, and I was kind of mad at my Dad for identifying it and dismissing the concern so carelessly, but the fact is it's hard to argue with his logic. We're suffering more and more "not your grid" problems over time. We don't control web standards, Google more or less does. We don't control what browsers CloudFlare chooses to allow on their service, CloudFlare does. Which means to a large extent, what our browser is allowed to do is limited to what hostile third-parties think it should be allowed to do, and it becomes a self-fulfilling prophecy.

Worse, as these problems compound, our userbase gets skewed towards unreasonable users on ancient hardware or software who also think we are maintaining legacy support for Firefox 52 and get angry at us for deviating from that even a little bit. It's like most of the world is just unable to process the idea of an independent browser as a concept and keeps thinking we are basically a continuation of our fork point, and the few that do understand what we're doing think it's a dangerous or bad idea and don't want to tolerate the existence of such a thing.

[frostknight]

Not really. It's primarily just a reverse proxy service. It doesn't de-anonymize users or sniff them. And in fact it protects website owners from quite a bit of malicious user behaviour. I also haven't heard of "insecurities". Yes, it does mean centralization of web traffic which isn't necessarily a good thing and could lead to regional segregation, but ultimately it is just a proxy tool with inherent advantages and disadvantages. So before you echo "pile of crap" I suggest you take a bit more time to understand what it is and is not.

To be fair, the de-anonymize part happens with tor and some proxies. That part I know to be accurate. The rest? Idk...

I do know that information has been leaked from cloudflare quite a few times, beyond that though, you would be right and I know little about it.

[andyprough]

Worse, as these problems compound, our userbase gets skewed towards unreasonable users on ancient hardware or software who also think we are maintaining legacy support for Firefox 52 and get angry at us for deviating from that even a little bit.

As the proverb says, the squeaky wheel gets the grease. Even so, I assume that a lot of us have newer hardware and current software and we prefer to use Pale Moon over other browsers for a wide variety of reasons. Including: easier to set up for privacy; lower memory usage, 'definitely not from any of the FANGG companies'; more secure without chromium's constantly exploited vulnerabilities; almost no instances of 'change for the sake of change'; virtually no additions of unnecessary 'features' like VPN or AI or 'Safe Browsing' monitoring; consistent GUI; a more powerful set of currently maintained extensions; easy to build and configure; all open software viewable on a publicly available git repository; modern features such as jpgxl support that are not available elsewhere. Maybe some of these advantages could be marketed and there might be an inflow of new users. The rollout of MV3 is likely to make a sizeable number of users look for alternatives.

Lots of smaller, more independent websites rely on CloudFlare and don't have a ready alternative because they fear hackers knowing their server's real IP and can't eat the bandwidth costs of running a site with its IP exposed directly to the Internet, basically.

I've been blocking everything from Cloudflare by default for years and rarely have to unblock anything from it to allow me access to a site. Most sites just work whether I'm blocking Cloudflare or not. I don't know why that is. I've had the same experience with blocking everything from Google. Maybe you or someone else that understands this stuff much better than myself can weigh in.

[athenian200]

As the proverb says, the squeaky wheel gets the grease. Even so, I assume that a lot of us have newer hardware and current software and we prefer to use Pale Moon over other browsers for a wide variety of reasons. Including: easier to set up for privacy; lower memory usage, 'definitely not from any of the FANGG companies'; more secure without chromium's constantly exploited vulnerabilities; almost no instances of 'change for the sake of change'; virtually no additions of unnecessary 'features' like VPN or AI or 'Safe Browsing' monitoring; consistent GUI; a more powerful set of currently maintained extensions; easy to build and configure; all open software viewable on a publicly available git repository; modern features such as jpgxl support that are not available elsewhere. Maybe some of these advantages could be marketed and there might be an inflow of new users. The rollout of MV3 is likely to make a sizeable number of users look for alternatives.

Yeah, you are definitely among the users who I think really understand what Pale Moon is about. Avoiding nonsense like "Google Safe Browsing," stuff controlled by big tech companies, support for more powerful extensions, JPEG-XL support that is held back in other places because of vested interests in WebP, etc. I just hope those users who are frustrated with MV3 choose to come here, we really need people who can appreciate this.

I've been blocking everything from Cloudflare by default for years and rarely have to unblock anything from it to allow me access to a site. Most sites just work whether I'm blocking Cloudflare or not. I don't know why that is. I've had the same experience with blocking everything from Google. Maybe you or someone else that understands this stuff much better than myself can weigh in.

I've tried blocking everything from Google in the past. The main thing that broke was actually captcha services. Sometimes if I e-mailed the webmaster and told them Google's captcha wasn't working for me because I was behind a firewall that blocked all Google services, they would grumble and verify my account manually. The problem with Cloudflare seems to be that websites can make it so that you have to go through Cloudflare in order to even access their website and don't offer a direct IP as a fallback if you block CloudFlare. Maybe most sites just aren't using a service like CloudFlare in the first place and we just hear disproportionately from those that do.

[andyprough]

I've tried blocking everything from Google in the past. The main thing that broke was actually captcha services.

Yeah, I occasionally have to unblock either Google or Cloudflare to get past a captcha, but that's been getting more rare as most sites just work for some reason even when eMatrix shows I'm actively blocking Google and Cloudflare on that site. I'm not really sure why - maybe I've just subconsciously moved away from the more obvious captcha sites over the years because they are so freaking annoying.

One more thing Pale Moon does amazingly well is handle the entire suite of Proton apps - Mail, Drive, Pass, Calendar - smooth as butter and low memory usage. And Disroot also. And Pale Moon works flawlessly with Proton VPN. Anyone that's really into the Proton and Disroot ecosystems like myself would benefit. And I've said before, Pale Moon handles the 'Logos' Biblical scholar research web app without using insane amounts of resources or failing to load the WebGL content like the way nearly every other browser mis-handles it. I'll bet a lot of users have their own sites that perform much better on Pale Moon than on other browsers.

[suzyne]

My path to Pale Moon is a little different from many here. I couldn't care any less about the history of Firefox or a retro look, but I am a kind of infatuated with doing everything in a browser and not installing software (if it can possibly be avoided).

And as someone who puts together and maintains websites, discovering that Pale Moon has both FTP and SSH extensions got me on board.

I love how scaling the interface of FireFTP is easy because it is part of the browser! With Cyberduck that I was using before, it was always too chunky or tiny depending on the screens I was working on.

As a free bonus Pale Moon is also great for testing, by making sure my sites work on it, I know that they will work anywhere.

I must have jinxed myself in my previous comment here because I got my first Cloudflare "are you human loop" yesterday, and it is worrying.

Even though personally, I am not getting cloudflare checks during my web browsing in Pale Moon

[yan242]

Hi,

I don't understand why Cloudflare tell me i'm not an human being. I'm and i want use Palemoon with Cloudflare! .

[Moonchild]

Lots of smaller, more independent websites rely on CloudFlare and don't have a ready alternative because they fear hackers knowing their server's real IP and can't eat the bandwidth costs of running a site with its IP exposed directly to the Internet, basically.

The thing is, as a CloudFlare user myself I know very well that the challenges and the proxying are not at all directly connected. You can have one without the other. CF can be your CDN (hiding server IPs and preventing direct attacks) without all of the alleged security "features" of CF.
But this is a problem of CF pushing to use these features by default, and a problem of scale -- every affected website can individually disable what has now been broken, but they aren't going to because we're a small fry and CF will keep pushing that their "web application firewall" is the holy grail of security for websites. This needs a lot more evangelism than we can provide as a small project, and that puts us in a really bad position with CF controlling so much of the internet space (that "grid") at the moment.
The main issue and difference with the Google question is though that CF is transparent to the end user, unlike Google. CF services aren't made part of websites, they sit between the website and the client. In that way it's essentially different and all that much harder to mitigate issues with.

[synthesizer_joe]

I came back from a camping trip to find I was stuck in a loop at the cloudflare verification stage when trying to access the captcha to post on 4chan.
I'm both glad to see this isn't just a "me-problem" but also disappointed that it's something overarching, out of your/our control.
Is there any workaround to circumvent this at the user's end?

To clarify, I love palemoon, it's general functionality and usability is really good, but in addition, a significant reason I use it is for moral and ethical reasons. Like, you actually seem to give a shit about privacy, freedom of speech and not being a totalitarian cyberpunk nightmare.
I don't want to switch over to firefox, partially for the above mentioned reasons but also because 4chan does not display correctly on firefox. I feel like this is almost a dig at 4chan, like if you want to use it it's got to be through "spyware" and/or you get a janky interface that makes it difficult to use. Probably coincidence, but in this era I wouldn't be surprised.

Anyways, TL;DR Has anyone got any solutions or workarounds at the user end?

[back2themoon]

Has anyone got any solutions or workarounds at the user end?

As of now, no.

[athenian200]

Has anyone got any solutions or workarounds at the user end?

The only solution or workaround I can imagine, even in theory, would be creating some kind of extension that loads CloudFlare checks in a different browser engine like Chromium, gets you past them, and then after getting past the check sends the actual web page back to Pale Moon for rendering. Even if we somehow did that and got it to work, I'm sure it wouldn't work for long and would be blocked.

Basically, the problem seems to be that CloudFlare is using extensive feature detection to make sure that the constellation of features supported by a browser lines up exactly with one of the browsers they support. In some ways, feature detection which was always hailed as a solution to the problems of relying on user-agents, is turning out to be worse for Pale Moon, because in practice websites are using combinations of features and their implementation details to determine the exact browser engine and turn away any browser engine they don't recognize as potential malware. At least with user agents we could spoof them to get past the sniffing, with this they are actually challenging us to do every single individual thing their supported browsers do in precisely the way they do it as a way of determining what engine we are on, whether they actually need/use that functionality or not, with the point being to filter out unsupported browser engines.

I don't see a solution other than to maybe create an extension of some kind that uses something based on Chromium to get past the check, and then feeds the resulting website back into Pale Moon rather than rendering it in the Chromium-based engine after its gotten a green light, handing off the session back to Pale Moon after the valid state has been confirmed and "tricking" CloudFlare. And even if we did do that, it would probably be an "arms race" between us and CloudFlare and they would try to stop this approach in much the way websites try to stop ad-blockers with anti-adblock. What CloudFlare seems to be demanding is that every browser be based on a browser engine that their team has already vetted, and move to a newer version of that engine when they say it's time. They just straight up do not allow independent browser engines and say that user-agents they don't recognize are likely to be malware clients that don't follow the web standards and compromise security. It's like when Google called Pale Moon a "browser toolbar" even though that doesn't make sense. It's also kind of like how Epyrus got accused by SpamAssassin of "impersonating Mozilla" just because the user agent string contained the word "Mozilla" in reference to what it was based on, like it always has historically. There just seems to be this increasing attitude that if you are not working under the radar of an established organization to build the application you're building, you don't have a right to build it and what you're building shouldn't be trusted not to be malicious. It's effectively doing an end-run around the concept of open source by making the source code useless to any organization other than the one that is recognized as the proper steward of the code with the right to release new versions of applications based on the latest version.

Basically, lots of online services overall seem to have decided to take a scorched earth approach to preventing access from bots and malware by insisting that people only use modern versions of well-known browsers, using clever tricks to make sure that the browser engine is identical to what they support, and whitelisting those as allowed user agents. And it will be a moving target because malware authors will inevitably get better at fooling CloudFlare into thinking they are an older version of Chromium, so the check will inevitably have to "catch" the malware using newer and newer features found in newer and newer web clients, meaning that there is basically no way we can keep up, because even the malware user-agents they are fighting using this approach likely have bigger teams than we do and will be able to defeat CloudFlare's new checks before we can and more effectively impersonate the feature set of a supported Chrome build or something.

It comes down to this... a big piece of centralized infrastructure that will be very hard to avoid interacting with has put its foot down and told Pale Moon users that, no, they can't use whatever browser they want if they want service from CloudFlare, and since a lot of websites you may want to use have chosen to rely on CloudFlare in such a way that CloudFlare has to be willing to provide service to your browser for you to access their site, they have effectively joined with them in putting their foot down and saying no to Pale Moon. We have no leverage here, there's not even a pretense of open standards... CloudFlare is private infrastructure, like if a privately-owned toll road were the only way to get from one town to another, and they reserved the right to deny vehicles service if the driver isn't willing to subject themselves to a background check and a credit check, and/or if the manufacturer of the car isn't a company on their approved list. The only real difference being that all the websites have actually chosen to be hidden behind CloudFlare's private infrastructure and rely on them to turn away the riff-raff, unlike the situation of a town where it literally cannot move to a public road.

Now that I think about it, It bears more of a resemblence to the Widevine situation than it does to the WebComponents situation, in that websites are relying on what is effectively black box technology controlled by private interests that we aren't approved by in order to vet whether it's safe to show their content to us. Only it's not technically a black box because there's no encryption and the key is in theory open source, but in practice changes too fast for an independent implementation to keep up.

[gepus]

Out of curiosity I checked 4chan with Firefox ESR 102.15.1. and got the same loop. It gets blocked by cloudf*ck.
Maybe someone can test with Firefox ESR 115. I assume it won't work either. Probably only v123 or upwards will work.

[sunstarunicorn]

It comes down to this... a big piece of centralized infrastructure that will be very hard to avoid interacting with has put its foot down and told Pale Moon users that, no, they can't use whatever browser they want if they want service from CloudFlare, and since a lot of websites you may want to use have chosen to rely on CloudFlare in such a way that CloudFlare has to be willing to provide service to your browser for you to access their site, they have effectively joined with them in putting their foot down and saying no to Pale Moon. We have no leverage here, there's not even a pretense of open standards... CloudFlare is private infrastructure, like if a privately-owned toll road were the only way to get from one town to another, and they reserved the right to deny vehicles service if the driver isn't willing to subject themselves to a background check and a credit check, and/or if the manufacturer of the car isn't a company on their approved list. The only real difference being that all the websites have actually chosen to be hidden behind CloudFlare's private infrastructure and rely on them to turn away the riff-raff, unlike the situation of a town where it literally cannot move to a public road.

Now that I think about it, It bears more of a resemblence to the Widevine situation than it does to the WebComponents situation, in that websites are relying on what is effectively black box technology controlled by private interests that we aren't approved by in order to vet whether it's safe to show their content to us. Only it's not technically a black box because there's no encryption and the key is in theory open source, but in practice changes too fast for an independent implementation to keep up.

I think the fact that Pale Moon is User-First and Privacy-First is the real issue, here.

Absolutely, the centralization of the Internet and the creation of 'walled gardens' has been a rapidly growing issue. 'Private corporations' are using their power and leverage in an attempt to force people to play by their rules (or else). The problem is that, in many areas, consumers are pushing back against the corporations. 'Worse', they are using the Internet to do it. Thus, the corporations are desperate to prevent people from using browsers that don't play by 'their rules'.

This is because it's much easier to control a few centralized browsers (which are already run by Big Tech) than to enforce their standards on all the alternative browsers out in the Internet Wilds.

A week ago, when the initial problem with Cloudflare was fixed, Moonchild remarked on how Cloudflare didn't seem to understand that Pale Moon is its own browser and no longer simply a 'Firefox clone'. Seems very foolish - I mean, because of how old Pale Moon is, it should be obvious that Pale Moon is an independent browser and no longer reliant on the old Firefox code. I think Cloudflare understands that just fine. But they have to pretend that they don't understand that very basic fact, because then they can jerk Pale Moon around and treat Pale Moon like a second class browser.

Much like people have been getting censored on different Big Tech websites, now the corporations are trying to censor browsers.

I don't know what the best response to this censorship should be, but the fact that Pale Moon is being censored can absolutely be a selling point for Pale Moon. There has been a strong movement against censorship of all stripes in many countries and the freedom of the Internet is a key component of modern day freedom. There are absolutely people out there who would love to use a browser that is so User and Privacy first that it's being targeted.

No, that doesn't change the situation with Cloudflare, but if Pale Moon can get a larger userbase, then Pale Moon starts getting the leverage it needs to push back against Cloudflare's Internet Garden. There might even be websites which are currently using Cloudflare, but would be willing to drop Cloudflare because they are censoring entire browsers.

I am discouraged by Cloudflare 'putting their foot down', but I'm certainly not going to bow to their bullying. Pale Moon is my primary browser and that's not about to change.

Moonchild, I know we have a thread open on the Cloudflare Community Forum, but are there any other avenues which we can use to apply pressure on Cloudflare?

[billmcct]

Out of curiosity I checked 4chan with Firefox ESR 102.15.1. and got the same loop. It gets blocked by cloudf*ck.
Maybe someone can test with Firefox ESR 115. I assume it won't work either. Probably only v123 or upwards will work.

Works for me on Firefox 115.9.1-ESR.

[gepus]

Works for me on Firefox 115.9.1-ESR.

What's your OS?