Pure URL for Palemoon

[moonbat]

Not possible, it would need a rewrite to start including site specific settings.

[andyprough]

Not possible, it would need a rewrite to start including site specific settings.

That's what I figured, not a problem, I can just toggle expansion when I want to visit that site.

[moonbat]

Changelog for 3.3.3

• Fix adding blank garbage fields in options.

[fatboy]

Good Day folks,

I have been using Pale Moon to browse amazon.ca and I see PureURL fails to remove some of the tracking url. It appears as an appendice to the URL itself. Here is an example:

Dirty URL:
https://www.amazon.ca/DECKER-Toaster-Ar ... 447&sr=8-5

Clean URL:
https://www.amazon.ca/DECKER-Toaster-Ar ... 00V5VEYBE/

The tracking url is everything from the "ref" part:
ref=sr_1_5?keywords=toaster&qid=1656612447&sr=8-5

[moonbat]

This is what I get for clean and dirty URLs you pasted -

Code: Select all

https://www.amazon.ca/DECKER-Toaster-Artisan-Black-TR1278BD/dp/B00V5VEYBE/ref=sr_1_5?keywords=toaster&qid=1656612447&sr=8-5
https://www.amazon.ca/DECKER-Toaster-Artisan-Black-TR1278BD/dp/B00V5VEYBE/

It's working as expected - everything after and including ref is stripped out.

[fatboy]

Hey Moonbat,

Thanks for the feedback. On my end it isn't working. I am using the Debain repo build on Antix Linux?

What do you think can be the cause for this?

[andyprough]

Hey Moonbat,

Thanks for the feedback. On my end it isn't working. I am using the Debain repo build on Antix Linux?

What do you think can be the cause for this?

Yeah it's weird. I'm on the same antiX distro with the Debian Pale Moon build right now, and if I search for the toaster on DuckDuckGo and click on it there it takes me to the Amazon page without the ref.... ending. If I search from within the Amazon page and click on it, then it sends me to a url that does include the ref.... ending. I've also got Pure URL installed and enabled.

[moonbat]

Please post sample original and processed links from the Pure URL logs (enable logging in its preferences, clear the error console (Tools->Developer tools->Error console) and reload the page in question) inside code tags so they're easy to see at a glance. I'll try to reproduce it here and see what's wrong. The extension being a fork, I've only added user interface changes (the options dialog, toolbar buttons/menus) and the expansion of shortened URLs; the link parsing logic is largely the same as the original.

[nicolaasjan]

Same issue here with that Amazon link...
Nothing Pure URL related in error console.

Code: Select all

Time stamp: 07-07-22 08:18:37 Error: The Components object is no longer supported. It will be removed shortly. Source file: https://www.amazon.ca/DECKER-Toaster-Artisan-Black-TR1278BD/dp/B00V5VEYBE/ref=sr_1_5?keywords=toaster&qid=1656612447&sr=8-5
Line: 9003

Time stamp: 07-07-22 08:55:19
Warning: Error: WebGL: Failed to create WebGL context: WebGL is currently disabled.
Source file: https://www.amazon.ca/DECKER-Toaster-Artisan-Black-TR1278BD/dp/B00V5VEYBE/ref=sr_1_5?keywords=toaster&qid=1656612447&sr=8-5
Line: 1

Time stamp: 07-07-22 08:55:19
Warning: Error: WebGL: Failed to create WebGL context: WebGL is currently disabled.
Source file: https://www.amazon.ca/DECKER-Toaster-Artisan-Black-TR1278BD/dp/B00V5VEYBE/ref=sr_1_5?keywords=toaster&qid=1656612447&sr=8-5
Line: 1

Had to add ref@amazon.ca to the preferences though (wasn't there by default)., but that didn't help.

[Edit]
When I open that link in Firefox, which has the latest uBlock Origin, I get:
https://www.amazon.ca/DECKER-Toaster-Artisan-Black-TR1278BD/dp/B00V5VEYBE/ref=sr_1_5

[andyprough]

Please post sample original and processed links from the Pure URL logs (enable logging in its preferences, clear the error console (Tools->Developer tools->Error console) and reload the page in question) inside code tags so they're easy to see at a glance. I'll try to reproduce it here and see what's wrong. The extension being a fork, I've only added user interface changes (the options dialog, toolbar buttons/menus) and the expansion of shortened URLs; the link parsing logic is largely the same as the original.

I'm pasting the output of the Error Console. The first paste is from a clean url that was clicked through from DuckDuckGo, and the second one is from a url with a referral link that was clicked through from within Amazon. They are both for the same toaster item that @fatboy linked to previously. I'm not familiar with the Error Console messages, so I'm pasting everything in an organized manner. If I'm pasting it incorrectly let me know and I can give you something else.

----------------------------------
Here's getting the link from DuckDuckGo, which Pure URL cleans of the referral tags:

Code: Select all

Errors: 
Timestamp: 07/07/2022 12:17:03 AM
Error: The Components object is deprecated. It will soon be removed.
Source File: https://www.amazon.com/BLACK-DECKER-TR1278BD-2-Slice-Toaster/dp/B00V5VEYBE
Line: 7039

Timestamp: 07/07/2022 12:17:06 AM
Error: TypeError: NetworkError when attempting to fetch resource.

Timestamp: 07/07/2022 12:17:07 AM
Error: TypeError: NetworkError when attempting to fetch resource.

Warnings:
Timestamp: 07/07/2022 12:17:02 AM
Warning: Use of getPreventDefault() is deprecated.  Use defaultPrevented instead.
Source File: https://images-na.ssl-images-amazon.com/images/I/61lemL2h6EL.js?AUIClients/AmazonUIjQuery&KK9dlo3A#412402-T1.412405-T1
Line: 91

Timestamp: 07/07/2022 12:17:03 AM
Warning: Strict-Transport-Security: The site specified a header that could not be parsed successfully.
Source File: https://m.media-amazon.com/images/G/01/csm/showads.v2.js?adtag=csm&adflag=-google-adsense.
Line: 0

Messages:
[PureURL:]Original link: https://www.aboutamazon.com/?utm_source=gateway&utm_medium=footer&token=about
Cleaned link:https://www.aboutamazon.com/?token=about

[PureURL:]Original link: https://sustainability.aboutamazon.com/?utm_source=gateway&utm_medium=footer&ref_=susty_footer
Cleaned link:https://sustainability.aboutamazon.com/?ref_=susty_footer

[PureURL:]Original link: https://www.fountain.com/jobs/amazon-delivery-service-partner?utm_source=amazon.com&utm_medium=footer
Cleaned link:https://www.fountain.com/jobs/amazon-delivery-service-partner

[PureURL:]Original link: https://logistics.amazon.com/marketing?utm_source=amzn&utm_medium=footer&utm_campaign=home
Cleaned link:https://logistics.amazon.com/marketing

[PureURL:]Original link: https://pro.imdb.com/?ref_=amzn_nav_ftr
Cleaned link:https://pro.imdb.com/

---------------------------------------------------------
Here's getting the link from inside Amazon, where the referral info stays at the end of the URL:

Code: Select all

Errors:
Timestamp: 07/07/2022 12:17:03 AM
Error: The Components object is deprecated. It will soon be removed.
Source File: https://www.amazon.com/BLACK-DECKER-TR1278BD-2-Slice-Toaster/dp/B00V5VEYBE
Line: 7039

Timestamp: 07/07/2022 12:17:06 AM
Error: TypeError: NetworkError when attempting to fetch resource.

Timestamp: 07/07/2022 12:17:07 AM
Error: TypeError: NetworkError when attempting to fetch resource.

Timestamp: 07/07/2022 12:20:59 AM
Error: Content Security Policy: The page’s settings blocked the loading of a resource at self (“style-src blob: *”). Source: tab-size: 4.
Source File: view-source:https://images-na.ssl-images-amazon.com/images/I/61lemL2h6EL.js?AUIClients/AmazonUIjQuery&KK9dlo3A
Line: 0
Source Code:
tab-size: 4

Timestamp: 07/07/2022 12:20:59 AM
Error: Content Security Policy: The page’s settings blocked the loading of a resource at self (“style-src blob: *”).
Source File: view-source:https://images-na.ssl-images-amazon.com/images/I/61lemL2h6EL.js?AUIClients/AmazonUIjQuery&KK9dlo3A
Line: 0

Timestamp: 07/07/2022 12:27:41 AM
Error: The Components object is deprecated. It will soon be removed.
Source File: https://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Dgarden&field-keywords=TR1278BD&crid=374HXEAL6R9L9&sprefix=tr1278bd%2Cgarden%2C296
Line: 5567

Timestamp: 07/07/2022 12:27:44 AM
Error: TypeError: NetworkError when attempting to fetch resource.

Timestamp: 07/07/2022 12:27:44 AM
Error: TypeError: NetworkError when attempting to fetch resource.

Warnings:
Timestamp: 07/07/2022 12:17:02 AM
Warning: Use of getPreventDefault() is deprecated.  Use defaultPrevented instead.
Source File: https://images-na.ssl-images-amazon.com/images/I/61lemL2h6EL.js?AUIClients/AmazonUIjQuery&KK9dlo3A#412402-T1.412405-T1
Line: 91

Timestamp: 07/07/2022 12:17:03 AM
Warning: Strict-Transport-Security: The site specified a header that could not be parsed successfully.
Source File: https://m.media-amazon.com/images/G/01/csm/showads.v2.js?adtag=csm&adflag=-google-adsense.
Line: 0

Timestamp: 07/07/2022 12:27:43 AM
Warning: Use of getPreventDefault() is deprecated.  Use defaultPrevented instead.
Source File: https://images-na.ssl-images-amazon.com/images/I/61lemL2h6EL.js?AUIClients/AmazonUIjQuery&KK9dlo3A#412402-T1.412405-T1
Line: 91

Timestamp: 07/07/2022 12:27:44 AM
Warning: This site appears to use a scroll-linked positioning effect. This may not work well with asynchronous panning; see https://developer.mozilla.org/docs/Mozilla/Performance/ScrollLinkedEffects for further details and to join the discussion on related tools and features!
Source File: https://www.amazon.com/s?k=TR1278BD&i=garden&crid=374HXEAL6R9L9&sprefix=tr1278bd%2Cgarden%2C296&ref=nb_sb_noss
Line: 0

Messages:
[PureURL:]Original link: https://www.aboutamazon.com/?utm_source=gateway&utm_medium=footer&token=about
Cleaned link:https://www.aboutamazon.com/?token=about

[PureURL:]Original link: https://sustainability.aboutamazon.com/?utm_source=gateway&utm_medium=footer&ref_=susty_footer
Cleaned link:https://sustainability.aboutamazon.com/?ref_=susty_footer

[PureURL:]Original link: https://www.fountain.com/jobs/amazon-delivery-service-partner?utm_source=amazon.com&utm_medium=footer
Cleaned link:https://www.fountain.com/jobs/amazon-delivery-service-partner

[PureURL:]Original link: https://logistics.amazon.com/marketing?utm_source=amzn&utm_medium=footer&utm_campaign=home
Cleaned link:https://logistics.amazon.com/marketing

[PureURL:]Original link: https://pro.imdb.com/?ref_=amzn_nav_ftr
Cleaned link:https://pro.imdb.com/

OpenGL compositor Initialized Succesfully.
Version: 4.6 (Compatibility Profile) Mesa 20.3.5
Vendor: Intel
Renderer: Mesa Intel(R) HD Graphics 620 (KBL GT2)
FBO Texture Target: TEXTURE_2D

[PureURL:]Original link: https://www.aboutamazon.com/?utm_source=gateway&utm_medium=footer&token=about
Cleaned link:https://www.aboutamazon.com/?token=about

[PureURL:]Original link: https://sustainability.aboutamazon.com/?utm_source=gateway&utm_medium=footer&ref_=susty_footer
Cleaned link:https://sustainability.aboutamazon.com/?ref_=susty_footer

[PureURL:]Original link: https://www.fountain.com/jobs/amazon-delivery-service-partner?utm_source=amazon.com&utm_medium=footer
Cleaned link:https://www.fountain.com/jobs/amazon-delivery-service-partner

[PureURL:]Original link: https://logistics.amazon.com/marketing?utm_source=amzn&utm_medium=footer&utm_campaign=home
Cleaned link:https://logistics.amazon.com/marketing

[PureURL:]Original link: https://pro.imdb.com/?ref_=amzn_nav_ftr
Cleaned link:https://pro.imdb.com/

OpenGL compositor Initialized Succesfully.
Version: 4.6 (Compatibility Profile) Mesa 20.3.5
Vendor: Intel
Renderer: Mesa Intel(R) HD Graphics 620 (KBL GT2)
FBO Texture Target: TEXTURE_2D

[moonbat]

So did you try to add the ref_ field to the list of garbage fields, which exists for exactly this reason? There's even a tooltip that explains how to add domain specific filters. You can't expect the extension to mind read each and every possible junk parameter across the entire internet - the default list just has the most common ones.

When you use any software - explore the settings and options first to get an idea of its capabilities; be it this or any other extension or Pale Moon itself or your operating system.

[andyprough]

So did you try to add the ref_ field to the list of garbage fields, which exists for exactly this reason? There's even a tooltip that explains how to add domain specific filters. You can't expect the extension to mind read each and every possible junk parameter across the entire internet - the default list just has the most common ones.

When you use any software - explore the settings and options first to get an idea of its capabilities; be it this or any other extension or Pale Moon itself or your operating system.

Good point, hadn't thought about it, it works so well out of the box that I forget about its many options a bit too easily.

Edit: Looks like after several tries of different versions of the garbage field, and then finally removing and re-installing Pure URL, I got ref@amazon.com to remove the referral material.

[fatboy]

Hi Moonbat, I have already tried adding rules to the garbage list before I posted here.

I am still having issues with the extension?

Some of the rules I see are:
ref@amazon.ca
ref@amazon.in
ref@amazon.com
ref_src

Any suggestions on what rules I should add for pureURL to work?

[moonbat]

If you find a parameter that isn't being filtered, then first see if it's there in the list, if not add it. It pays not to add 'ref' without a domain filter because I've seen sites use it for non tracking purposes and then they mysteriously break. On Amazon, there's ref and ref_, and you have to add them for each Amazon domain that you use.
I kept the list short with the most commonly used defaults, but there may be others so I made it user configurable. None of the similar extensions around for other browsers let you add your own filters, in fact some of them rely on the developer pushing out updates with every new version.

[Moonchild]

some of them rely on the developer pushing out updates with every new version.

Trying to maintain such a list for every user is folly. It'll never be complete.
People will just have to cater the list to their own usage patterns so your approach is exactly right IMHO

[nicolaasjan]

some of them rely on the developer pushing out updates with every new version.

Trying to maintain such a list for every user is folly. It'll never be complete.
People will just have to cater the list to their own usage patterns so your approach is exactly right IMHO

Indeed.
Just look at the sheer amount of rules in the 3 lists, that can be enabled/added in the webextension version of uBlock Origin, that has support for '$removeparam=' :

Code: Select all

AdGuard URL Tracking Protection → 570 used out of 669
Actually Legitimate URL Shortener Tool → 1,902 used out of 1,921
ClearURLs for uBo → 658 used out of 666

[moonbat]

Trying to maintain such a list for every user is folly. It'll never be complete.
People will just have to cater the list to their own usage patterns so your approach is exactly right IMHO

Currently the list of fields is stored as a string preference, my concern is running into its limits eventually as one adds more parameters. Do you think it's a good idea performance-wise to rework the extension to use an sqlite database for the rules? That way I can also have a 'factory default' copy to let the user revert to defaults if required.

[Moonchild]

Preferences were never meant to be used to store large swathes of data, so using an sqlite database will be both faster and more resilient.
In fact if you store too much data in a preference the platform will warn you about the impact

[BenFenner]

Interesting article about Facebook's tracking URL parameter. Looks like Firefox and Brave screwed us...
https://tech.slashdot.org/story/22/07/1 ... g-measures

I guess we FB users get what we deserve. :.(

[andyprough]

Interesting article about Facebook's tracking URL parameter. Looks like Firefox and Brave screwed us...
https://tech.slashdot.org/story/22/07/1 ... g-measures

I guess we FB users get what we deserve. :.(

According to the Ghacks article which Slashdot points to, it was not Firefox or Brave that screwed you, but Facebook itself by encrypting its url tracking parameters: https://www.ghacks.net/2022/07/17/faceb ... nt-4531269

Previously, Facebook used the parameter fbclid for tracking purposes. Now, it uses URLs such as https://www.facebook.com/ghacksnet/post ... %2CO%2CP-R instead.

The main issue here is that there it is no longer possible to remove the tracking part of the URL, as Facebook merged it with part of the required web address. Removing the entire construct after the ? would open the main Facebook page of Ghacks Technology News, but it won't open the linked post.

Looks like you should be able to reduce much of the damage by deleting facebook's cookies immediately after leaving their website. Pale Moon has some good extensions to do that for you, including one of my personal favorites, Crush Those Cookies: https://addons.palemoon.org/addon/crush-those-cookies/