Bypass Cloudflare with Saved Cookies

[tellu-white]

I've noticed that more and more sites are using Cloudflare verification (unfortunately). Even "stackoverflow.com" has started using it!

I have a "Custom Button" that deletes the cookies of the domain used by the page in the current tab - when I close the tab. Because of this, I have to go through Cloudflare verification every time I open a page that belongs to that domain. There are cases where this check takes quite a long time. To stop wasting time with these checks, I made a new add-on ("Bypass Cloudflare with Saved Cookies") with which I can save in "prefs.js" the cookies of the domains that are important to me, so I can reload in browser the cookies of a domain before opening a page that belongs to that domain.

This add-on also has a "Shortcut Key" ("Shift + Z") which can be useful in case a Cloudflare verification seems to run forever (and cannot be stopped with the mouse - by closing the TAB). The "Shortcut Key = Shift + Z" tries to stop the Cloudflare code from executing, then copies the URL from the TAB in question to the Clipboard and replaces the checked page with a blank page ("about:blank").

This add-on uses the same idea as the "Bypass Cloudflare with Firefox Cookies" add-on ( https://forum.palemoon.org/viewtopic.php?f=71&t=32098 ), but it is meant to be used if the Cloudflare verification works properly in Pale Moon.

Screenshots:

"Bypass Cloudflare with Saved Cookies":

01.png

Right-Click on Button (open a list with "Preferences"):

02.png

Click on Button (open a list with add-on usage alternatives):

03.png

The "OPTIONAL" entry in the list opened with Right-Click is useful if the user has set "Permission" so that it is forbidden to load cookies in Pale Moon by a specific domain. This setting causes Cloudflare to refuse checking for that page (that domain) and display the message "Please enable Cookies and reload the page":

04.png

Screenshot with the option to load cookies in browser (Click on Button):

05.png

Screenshot with the option to delete cookies from "prefs.js" (Right-Click on Button):

06.png

Anyone interested in testing this add-on ("Bypass Cloudflare with Saved Cookies 3.5") can download it here:

https://www.mediafire.com/file/oeh9rweh7f9q4bt/bypass_cloudflare_with_saved_cookies_3_5.zip/file

[tellu-white]

I've made a new version of this add-on in which I've added a new entry in the "context menu" for the case when you Right-Click on a link that (maybe) targets a page that requires Cloudflare verification (the new entry is called "Bypass Cloudflare with Saved Cookies"). After Clicking on this new option in the "context menu", the respective page is opened without Cloudflare check, if cookies were previously saved for the respective page (domain) with the option "Save Cookies of Current Page in prefs.js" (which is opened with Right-Click on the add-on button).

If there are no cookies saved for the respective domain, after clicking on the new entry in the context menu the message "There are no Cookies saved for URL = ..." will be displayed.

Screenshots:

1. I passed a Cloudflare check successfully for a random page "stackoverflow.com" and then I saved cookies in "prefs.js" with "Save Cookies of Current Page in prefs.js" (which is opened with Right-Click on the add-on button).

01.png

2. I did a Google search with the text "cookies expiration time", then I deleted all cookies from Pale Moon:

02.png

3. I right-clicked on the first link "stackoverflow" and then I clicked on the new entry in the "context menu" ("Bypass Cloudflare with Saved Cookies"):

03.png

4. This action caused the add-on to copy cookies from "prefs.js" to Pale Mooon, then to open the page targeted by that link:

04.png

5. I closed the "stackoverflow" page then I deleted again all cookies:

05.png

6. I right-clicked the same link and then used the "Open Link in New Tab" option, which is hard-coded in the browser:

06.png

7. This time the Cloudflare verification was activated again:

07.png

8. If this new add-on option is used on a link that has not yet passed the Cloudflare check (or does not use Cloudflare), then the message "There are no Cookies saved for URL = ..." will be displayed:

08.png

09.png

The new version of the add-on ("Bypass Cloudflare with Saved Cookies 4.3") can be downloaded here:

https://www.mediafire.com/file/omoa896me3hmkc5/bypass_cloudflare_with_saved_cookies_4_3.zip/file

[tellu-white]

This is a bugfix.

Download link ( Bypass Cloudflare with Saved Cookies 4.5 ):

https://www.mediafire.com/file/d4qmgljydzf1e03/bypass_cloudflare_with_saved_cookies_4_5.zip/file

[MasterSlenderTR]

Thank you for making the extension! I think the extension would be more user-friendly if it had those cloudflare cookies built-in.

[tellu-white]

Thank you for making the extension! I think the extension would be more user-friendly if it had those cloudflare cookies built-in.

You're welcome! I'm glad you find it useful.

I didn't make built-in cookies because websites (including those using Cloudflare) update cookies after a certain time. On most sites I've tested (with this add-on), this time interval is a long one, which is why this add-on is useful.

When cookies are updated by a particular site, a new Cloudflare check must be successfully passed on any page of that site, after which the new cookies must be saved again with the option "Save Cookies of Current Page in prefs.js" (which is opened with Right-Click on the add-on button).

As I mentioned in the post for "Bypass Cloudflare with Firefox Cookies" add-on ( https://forum.palemoon.org/viewtopic.php?f=71&t=32098 ), I came across a site ( https://tinyurl.com/ ) that does the Cloudflare check on every visit to their page. In such cases, this add-on is of no use. Fortunately, I haven't seen any site other than "tinyurl" that has such behavior.

[Moonchild]

websites (including those using Cloudflare) update cookies after a certain time

It's actually an inherent feature of CloudFlare's cookies. They include an epoch timestamp which will determine after what time the cookie is no longer valid in their system. That's by design so cookies can't be hardcoded in bots to bypass CF checks. How exactly this is further weaved into the cookie data is unknown outside CF (for obvious reasons) and I gather it also includes some form of location/IP marker, but it makes the idea of baking cookies into an extension completely moot.

[tellu-white]

It's actually an inherent feature of CloudFlare's cookies. They include an epoch timestamp which will determine after what time the cookie is no longer valid in their system. How exactly this is further weaved into the cookie data is unknown outside CF (for obvious reasons).

My add-on adds a year to "cookie.expires" for all cookies saved in "prefs.js".

I did a test with a "stackoverflow.com" page. After loading cookies from "prefs.js", eight cookies are loaded in Pale Moon, five of which contain in their name the string "cf" (probably an abbreviation of "CloudFlare"). After opening the page "stackoverflow.com", only one more cookie is added - but this is of type "cookie.isSession = true". I also noticed a change to the value of "cookie.expires" (compared to the values saved in "prefs.js") for only one cookie, the one with the name "__cf_bm". For this cookie, the expiration time is equal to the time at the moment of loading the "stackoverflow.com" page plus 30 minutes.

My add-on has been working for many days on "stackoverflow.com" site with the same cookies saved in "prefs.js", so I don't know after how long the cookies related to Cloudflare verification expire, nor if the expiration time modified by my add-on could be taken into account by Cloudflare.

[tellu-white]

This is a bugfix.

Download link ( Bypass Cloudflare with Saved Cookies 4.6 ):

https://www.mediafire.com/file/k25c21p2ec9a070/bypass_cloudflare_with_saved_cookies_4_6.zip/file

[tellu-white]

The "new entry in the context menu" option of the add-on does not work for sponsored links in Google Search results pages. These pages contain JavaScript code (addEventListener mousedown) that modifies the HREF of an "A" tag when the sponsored link is right-clicked (or clicked). This Google code works as follows (modified by me to be easily understood):

Code: Select all

function get_closest_a_tag(clicked_element){
	return clicked_element ? clicked_element.closest("A") : null
}

function change_HREF_to_Google_Search_Results(event){
	var a_tag = get_closest_a_tag(event.target);
	
	if(a_tag){
		if(a_tag.dataset.agdh && a_tag.dataset.rw){
			// alert(a_tag.dataset.rw);
			
			a_tag.href = a_tag.dataset.rw;
		}
	}
}

window.document.documentElement.addEventListener("mousedown", change_HREF_to_Google_Search_Results);

You can see that the Javascript code uses the "data-agdh" and "data-rw" attributes of the "A" tags (found only in sponsored links) to modify the HREF value. This Javascript code is no longer executed if we remove the "data-agdh" and "data-rw" attributes from the Google Search results page. We can do this using the "Intercept & Modify HTTP Response 4.3" add-on ( viewtopic.php?f=70&t=31829&start=20#p258461 ), with the following filter:

Code: Select all

[`/google.com/`, `/.*/`, [`/data-agdh=".*?"/g`, ``], [`/data-rw=".*?"/g`, ``]]

Screenshots:

I. Tag "A" before and after right-click on the sponsored link, in case the above filter is not used:

Before right-click on the sponsored link:

01.png

After right-click on the sponsored link:

02.png

Notice the change of the HREF value after right-click on the link:

https://www.udemy.com/course/ultimate-webrtc-bootcamp-for-beginners/

becomes

https://www.googleadservices.com/pagead/aclk?sa=...

After click on "Bypass Cloudflare with Saved Cookies":

03.png

II. The filter above in the add-on "Intercept & Modify HTTP Response 4.3":

04.png

III. Tag "A" before and after right-click on the sponsored link, in case we use the add-on "Intercept & Modify HTTP Response 4.3" with the filter posted above:

Before right-click on the sponsored link:

05.png

After right-click on the sponsored link:

06.png

After click on "Bypass Cloudflare with Saved Cookies":

07.png

***

Note: This work-around is only necessary if the link (in the Google Search results page) opens a sponsored page that uses Cloudflare verification. However, this work-around should be used because we cannot know in advance whether this is the case or not.

[back2themoon]

...I've added a new entry in the "context menu"...

Please consider adding an option to disable this entry, tellu-white. Just a matter of keeping things as neat as possible in the context-menu. By the way, I didn't really understand what this right-click does:

After Clicking on this new option in the "context menu", the respective page is opened without Cloudflare check, if cookies were previously saved...

But, if these cookies have already been saved, isn't a simple left-click enough? What's the difference?

[tellu-white]

...I've added a new entry in the "context menu"...

I didn't really understand what this right-click does:

After Clicking on this new option in the "context menu", the respective page is opened without Cloudflare check, if cookies were previously saved...

But, if these cookies have already been saved, isn't a simple left-click enough? What's the difference?

As I said in my first post here, "I have a Custom Button that deletes the cookies of the domain used by the page in the current tab - when I close the tab. Because of this, I have to go through Cloudflare verification every time I open a page that belongs to that domain."

Many Pale Moon users tend to delete cookies frequently. These users face the same problem. To overcome this problem, my add-on saves cookies in the "prefs.js" file and not in the "cookies.sqlite" database, where Pale Moon saves them. Thus, even if the user deletes cookies in Pale Moon (i.e., from the "cookies.sqlite" database), the add-on will be able to reload them from "prefs.js" (if they were previously saved with the option "Right-Click on add-on Button + Save Cookies of Current Page in prefs.js"). Only after the cookies have been reloaded from "prefs.js" (using the add-on) will a simple click on the link be enough to bypass Cloudflare verification.

...I've added a new entry in the "context menu"...

Please consider adding an option to disable this entry, tellu-white. Just a matter of keeping things as neat as possible in the context-menu.

I will not disable this entry from the "context menu" because it is very useful in Google Search results pages. I will explain why it is useful. Let's take an example where I used the add-on and saved cookies (in the "prefs.js" file) for the domain "drunkenslug.com" after passing the Cloudflare verification. Then, I deleted all cookies from Pale Moon (i.e., from the "cookies.sqlite" database), after which I searched Google for "drunkenslug.com."

Screenshots:

Cookies for the domain "drunkenslug.com" saved using the add-on (in the "prefs.js" file) after passing the Cloudflare verification.

001.png

Cookies from Pale Moon (i.e., from the "cookies.sqlite" database) after searching Google for "drunkenslug.com" (I had previously deleted all cookies saved by Pale Moon).

002.png

To reload the cookies saved by the add-on in "prefs.js" and open the https://drunkenslug.com/ page, we need to copy the link to that page, then Click on the Add-On Button and use "Enter URL of a Page with SAVED Cookies + Open THAT Page" option. Let's see what happens if we want to copy that link from the Google Search results page.

003.png

004.png

005.png

As you can see, when you right-click (or click) on a link in the Google Search results page, Google automatically changes the URL of the page to add tracking parameters. Because of this, the copied link cannot be used in the add-on. In this example, the copied link will look like this:

https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://drunkenslug.com/&ved=2ahUKEwjZyJmUqMSPAxWA_bsIHV7oGAYQFnoECAsQAQ&usg=AOvVaw2hPjdgUm51bKksOD8XrMzG

006.png

We can see that the domain of this link has been changed from "drunkenslug.com" to "google.com", so we will have to copy the link https://drunkenslug.com/ from somewhere else - not from the Google Search results page (and thus we will waste time searching for that link).

When using the "Right-Click on Link + Bypass Cloudflare with Saved Cookies" option, the add-on will extract the URL of the searched page from the URL modified by Google (the one with "tracking parameters"), and then will load the cookies for that domain (if they were previously saved in the "prefs.js" file). This way, it will only take two clicks to open the page.

007.png

008.png

***

Note: You can also use the "Click on Add-On Button + Load Cookies for a Top-Level Domain from prefs.js" option before opening the page https://drunkenslug.com/, but if you want to open it in a new tab (so that you don't lose the other Google Search results), you will need 7 clicks instead of the 2 clicks (mentioned above).

***

If, however, I have not convinced you of the utility of this option ("Bypass Cloudflare with Saved Cookies" in the "context menu"), you can remove this entry with the following code, copied into the "userChrome.css" file:

Code: Select all

#menuitem_bypass_cloudflare_with_saved_cookies {
	display:none !important;
}

Screenshot:

009.png

[back2themoon]

I will not disable this entry from the "context menu" because it is very useful... ...If, however, I have not convinced you of the utility of this option...

Thanks for the detailed info. Slight misunderstanding here. You didn't need to convince me, since I never doubted its usefulness or asked for you to disable it, but for the option to disable it. You did provide that css code, so thanks for that. It's perfectly fine. I just don't use Google as a middle man for these particular pages - I visit them directly. I mean, if we have a specific extension for them after all the Cloudflare fuss, it is safe to assume we know about them and do not need Google any more. To be honest, I don't see why your Bypass Cloudflare extension would need to deal with Google at all. It seems irrelevant (and perhaps even complicating things rather than simplifying them), but that is your choice of course. I understand that others may be using Google that way.

Several extensions provide the option the remove their context-menu entries. However, they usually provide alternative ways to access those settings. Unless I misunderstood, in this case it seems there is no other way to access this function in your extension (ok, there's the 7-click method). So, it makes sense to not provide an option to remove it because the function will then become invisible to the user.

But, you also have to think what will happen if every extension keeps adding context menu entries! In any case, the css code is fine.

[River Moon]

Off-topic:

Google automatically changes the URL of the page to add tracking parameters.

I would urge anyone using Google to deGoogle the links...

Code: Select all

	eli = document.getElementsByTagName("a");
	for(i = 0, li = eli.length; i < li; i++)
	{
		if (eli[i].href.indexOf("/url?") != -1) eli[i].href = href.searchParams.get('url');
	}

https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://drunkenslug.com/&ved=2ahUKEwjZyJmUqMSPAxWA_bsIHV7oGAYQFnoECAsQAQ&usg=AOvVaw2hPjdgUm51bKksOD8XrMzG

...would become...

https://drunkenslug.com/
 
No one wants to see the www.google.com/url stuff
 

[back2themoon]

I am still a bit confused as to how the extension is meant to work. Correct me where I am wrong. For example:

1. Visit e.g. drunkenslug.com and wait for the CF check to complete. It takes some time.
2. Use the extension to save the drunkenslug.com CF-related cookies in its special prefs.js section.
3. Delete all history/cookies etc. for drunkenslug.com.

If I understand this correctly, the extension now comes into play so we can visit drunkenslug.com again and pass the CF check way more quickly. And here's the question:

a) Does the extension detect the new visit to drunkenslug.com automatically and uses its prefs.js cookies on its own, or:
b) Does the user need to manually use the extension at this point, to load those prefs.js cookies?

[back2themoon]

If it's the second, may I suggest adding an option to "Load Cookies for all saved Hosts"? It should make things quicker, right?

[tellu-white]

I don't see why your Bypass Cloudflare extension would need to deal with Google at all.

As I said in my first post here:

I've noticed that more and more sites are using Cloudflare verification (unfortunately). Even "stackoverflow.com" has started using it!
...
I have a "Custom Button" that deletes the cookies of the domain used by the page in the current tab - when I close the tab. Because of this, I have to go through Cloudflare verification every time I open a page that belongs to that domain.

If I had done a Google search like this: "Cloudflare site:stackoverflow.com", all the links on the Google Search results page would have targeted "stackoverflow" pages. If I had closed such a "stackoverflow" page after finding that I couldn't find the information I wanted, I would have had to repeat the procedure of loading the cookies from the "prefs.js" file before opening another "stackoverflow" page using a link from the Google Search page with the search results (mentioned above). To avoid having to repeat this procedure, the simplest solution was to add the "Right-Click on Link + Bypass Cloudflare with Saved Cookies" option.

Note: Currently, "stackoverflow" has stopped using Cloudflare verification, but similar situations may occur with other domains.

If I understand this correctly, the extension now comes into play so we can visit drunkenslug.com again and pass the CF check way more quickly. And here's the question:

a) Does the extension detect the new visit to drunkenslug.com automatically and uses its prefs.js cookies on its own, or:
b) Does the user need to manually use the extension at this point, to load those prefs.js cookies?

If it's the second, may I suggest adding an option to "Load Cookies for all saved Hosts"? It should make things quicker, right?

It's the second option, but it's not a good idea to load cookies for all hosts. This option is in contradiction with the option to delete cookies from the browser very often (as I do myself - see the explanation above). On the other hand, users who keep cookies until they expire no longer have to go through the Cloudflare verification process (until they expire), so they do not need this add-on. When cookies expire, users who use the add-on must go through the Cloudflare verification process again, just like users who do not use the add-on.

[tellu-white]

I would urge anyone using Google to deGoogle the links...

Code: Select all

	eli = document.getElementsByTagName("a");
	for(i = 0, li = eli.length; i < li; i++)
	{
		if (eli[i].href.indexOf("/url?") != -1) eli[i].href = href.searchParams.get('url');
	}

The code you posted does not work for two reasons. Let's see what they are, using a real example. I do a Google Search with the following text:

Code: Select all

"Bypass Cloudflare with Saved Cookies" site:forum.palemoon.org/viewtopic.php?f=71&t=32499

After loading the results page (actually a single result), I right-click on the link, so Google modifies that URL by changing the host from "palemoon" to "google" and adding tracking parameters.

01.png

I am using the "Custom Buttons Enhanced 0.0.6" add-on ( https://forum.palemoon.org/viewtopic.php?f=71&t=32626 ) to test your code.

02.png

03.png

We can see that the HREF modified by Google remains untouched after executing your code (by clicking on the button).

1. The first reason your code does not work in a Pale Moon add-on is that you used the code "document.getElementsByTagName('a')" to access the "a" tags, but in Pale Moon add-ons you must use the code "content.document.getElementsByTagName('a')". I am testing this statement with a Custom Button (this behavior is identical in a Pale Moon extension).

04.png

05.png

As you can see, your version of the code does not find any "a" tags on the page.

2. The second reason your code isn't working is because "href" isn't defined in the code snippet below:

Code: Select all

... = href.searchParams.get('url');

Even if you had used

Code: Select all

... = eli[i].href.searchParams.get('url');

instead of "href," your code still wouldn't have worked because in this case "href" is a string, so that code would have thrown an error too:

06.png

07.png

For your code to work, it must be modified as follows:

Code: Select all

try{
	var eli = content.document.getElementsByTagName("a");
	var li = eli.length;

	for(var i = 0; i < li; i++)
	{
		if (eli[i].href.indexOf("/url?") != -1)
		{
			var obj_URL = new URL(eli[i].href);
			
			eli[i].href = obj_URL.searchParams.get("url");
		}
	}
	
} catch(err){
	alert(err.message);
}

08.png

09.png

[River Moon]

The code you posted does not work ...

The code I posted is a very small part of a User script I use under Basilisk that does many manipulations to Google search. I can assure it works fine – at least it does under Basilisk.

Even this old war-horse still works: Redirect Cleaner

But the point I was making is that I don't think you should tolerate Google's adulterated links.

[Gemmaugr]

Off-topic:

The code you posted does not work ...

The code I posted is a very small part of a User script I use under Basilisk that does many manipulations to Google search. I can assure it works fine – at least it does under Basilisk.

Even this old war-horse still works: Redirect Cleaner

But the point I was making is that I don't think you should tolerate Google's adulterated links.

Agreed that google should be tolerated as little as possible. I use Pure URL and adding ?sa= and &usg= to the garbage fields list seem to strip most of the tracking parameters out of the link.

[back2themoon]

It's the second option, but it's not a good idea to load cookies for all hosts. This option is in contradiction with the option to delete cookies from the browser very often (as I do myself)

Ok, I am still confused here. I understand that you delete cookies often. I do too every now and then. But this is about something else.

Let's say I have 10 Cloudflare websites I want to login after having cleared Pale Moon's cookies. I visit these websites via Bookmarks or by typing their url directly. So, let's leave Google out of this.

Why would this be preferable:

Use the extension ten times to load the prefs.js cookies for ten websites separately...

instead of:

...use the extension one time to load all available prefs.js cookies in one go, then proceed to login to those websites? (which can be all launched quickly and at once by the way via Session Manager or a simple Bookmarks folder).