ηMatrix/eMatrix support thread

Dedicated board for extension releases/support threads

Moderators: FranklinDM, Lootyhoof

Forum rules
Please do not create new topics here unless you are an extension author in need of a dedicated releases&support thread!
vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-04-23, 11:37

I think fixing it is worth the trouble.
If you get some time to try, see which extensions cause this issue.
I tried with only Adblock Latitude, but the icon would keep being there, so maybe it's a combination of multiple things.

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-04-23, 13:15

Attention Basilsik users:
The new version changes how the extension interacts with the UI.
If you see any problem, please report it here as soon as possible.
Thanks.

federatedfly

Top-level domain grouping.

Unread post by federatedfly » 2019-05-12, 03:12

I have a problem related to top-level domain grouping.

I live in Brazil, and most sites here have the top-level domain .com.br.
Since eMatrix applies rules by domain, it should deem the site ahnegao.com.br as the 1st-party domain. And, since I globally unblocked 1st-party domains, the browser should connect to all of its subdomains (cdn.ahnegao.com.br and www.ahnegao.com.br) and block everywhere else. But instead, eMatrix thinks that ahnegao is a subdomain of the website com.br (such website does not exist) and groups ahnegao.com.br alongside with any other brazilian website which has .com.br as top-level domain.

What is worse is that ahnegao.com.br is (just like most other blogs) full of adds and statistics trackers, and some of these ads come from .com.br ad providers. In the image bellow you can see the domain premiumads.com.br, where the blog gets its ads from. Since I have unblocked content from 1st-party domains and eMatrix thinks com.br is the 1st-party domain and, therefore, premiumads is one of its subdomains, the page connects to this PremiumAds website. It also connects to boostbox.com.br, which is a "statistics" tracker.
eMatrix window at ahnegao.com.br
eMatrix window at ahnegao.com.br
I could not find anything related to configuring the list of words eMatrix consider as top-level domains.

PS: the site name is ahnegao, not ahegao. There's an n in it!
Last edited by federatedfly on 2019-05-12, 14:24, edited 3 times in total.

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-05-12, 09:36

That's indeed a problem.
Unfortunately a solution will take time because that's a part of the code I have to study as I don't know how it works yet.
But in general I think I can simply "fuse" any top-level domain in the form "co.tld" or "com.tld", since they are very common for regional domains (like co.uk or co.jp)
When the new version is out, it'll have this fix, but as I said it'll take a while.

JustOff

Re: ηMatrix/eMatrix support thread

Unread post by JustOff » 2019-05-12, 11:58

vannilla wrote:
2019-05-12, 09:36
I think I can simply "fuse" any top-level domain in the form "co.tld" or "com.tld", since they are very common for regional domains (like co.uk or co.jp)
You'd better use nsIEffective​TLDService for this.

federatedfly

Re: ηMatrix/eMatrix support thread

Unread post by federatedfly » 2019-05-12, 12:02

vannilla wrote:
2019-05-12, 09:36
That's indeed a problem.
Unfortunately a solution will take time because that's a part of the code I have to study as I don't know how it works yet.
Thank you for the quick response. I'll wait for the next release.

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-05-12, 13:48

Fortunately finding where TLDs are handled was easy, but changing that part might be a problem.
JustOff wrote:
2019-05-12, 11:58
You'd better use nsIEffective​TLDService for this.
I discovered that uMatrix (and therefore eMatrix) uses a suffix list which is downloaded from https://publicsuffix.org/ and which is then used to extract the various parts from a full URL.
Is it safe to replace this list with the built-in one using nsIEffectiveTLDService? Essentially, is the built-in list sufficiently up-to-date?
The function used to get the TLD currently in use in eMatrix seems to be wrong, but before fixing it I want to know if I should bother or if I should just use the built-in service instead.

JustOff

Re: ηMatrix/eMatrix support thread

Unread post by JustOff » 2019-05-12, 15:41

Actually, using https://publicsuffix.org is preferred, but perhaps uMatrix/eMatrix does not handle it properly, as "com.br" is there.

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-05-12, 18:42

JustOff wrote:
2019-05-12, 15:41
Actually, using https://publicsuffix.org is preferred, but perhaps uMatrix/eMatrix does not handle it properly, as "com.br" is there.
I see, thanks.
And as I said in my previous post:
The function used to get the TLD currently in use in eMatrix seems to be wrong

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-05-15, 19:05

Version 2.0.0 is available. It will be uploaded to Phoebus a few minutes after this post is published.
Very brief changelog: https://gitlab.com/vannilla/ematrix/tags/2.0.0

PLEASE NOTE:
The new version underwent some behavioural changes, hence the increase in the major version (from 1.x to 2.x)
Due to this, there is a chance old installations might break. Please be patient and report any issue you might experience.
I didn't see anything while testing, but that doesn't really give any guarantees.

federatedfly

Re: ηMatrix/eMatrix support thread

Unread post by federatedfly » 2019-05-16, 20:25

vannilla wrote:
2019-05-15, 19:05
Version 2.0.0 is available.

[...] there is a chance old installations might break. Please be patient and report any issue you might experience.
Nice! I will use it and report here any issue I undergo.
Thanks!

User avatar
plushkava
Apollo supporter
Apollo supporter
Posts: 46
Joined: 2015-07-31, 04:53
Location: Clown World

Re: ηMatrix/eMatrix support thread

Unread post by plushkava » 2019-05-17, 02:06

Unless using a throwaway profile, I would warn users of uBlock Origin 1.16.4.10 to prevent eMatrix from updating to 2.0.0, for the time being, as they appear to be incompatible. I'm curious to know whether anyone else can confirm my findings.

Ez-waker

Re: ηMatrix/eMatrix support thread

Unread post by Ez-waker » 2019-05-17, 02:51

vannilla wrote:
2019-05-15, 19:05
Version 2.0.0 is available. It will be uploaded to Phoebus a few minutes after this post is published.
Very brief changelog: https://gitlab.com/vannilla/ematrix/tags/2.0.0

PLEASE NOTE:
The new version underwent some behavioural changes, hence the increase in the major version (from 1.x to 2.x)
Due to this, there is a chance old installations might break. Please be patient and report any issue you might experience.
I didn't see anything while testing, but that doesn't really give any guarantees.
Yes, I can confirm the incompatibility. (Thanks very much for taking on this extension for us, by the way!)

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-05-17, 11:23

Thanks for the report.
I'll take a look at it once I'm at my computer (I'm on a different device right now.)
I think I know what's up, seeing that it appens after a restart.
I'll post updates on the issue on gitlab.
Unfortunately I can't take v2.0.0 out because extensions can't be deleted from Phoebus (that I know of), but I'll do my best to fix it quickly.

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: ηMatrix/eMatrix support thread

Unread post by vannilla » 2019-05-17, 12:11

Sorry for the double post.
A fix has been applied, and version 2.0.1 was released just now.
I tried the steps listed in the gitlab issue, and it seems to work as intended. Please try it out.
Thanks.

User avatar
plushkava
Apollo supporter
Apollo supporter
Posts: 46
Joined: 2015-07-31, 04:53
Location: Clown World

Re: ηMatrix/eMatrix support thread

Unread post by plushkava » 2019-05-17, 13:05

vannilla wrote:
2019-05-17, 12:11
A fix has been applied, and version 2.0.1 was released just now.
I tried the steps listed in the gitlab issue, and it seems to work as intended. Please try it out.
With uBlock Origin installed, I tested the upgrade path from eMatrix 1.4.0 to 2.0.1 and it seems to be working perfectly now. :thumbup:

Ez-waker

Re: ηMatrix/eMatrix support thread

Unread post by Ez-waker » 2019-05-17, 14:11

vannilla wrote:
2019-05-17, 12:11
A fix has been applied, and version 2.0.1 was released just now.
I tried the steps listed in the gitlab issue, and it seems to work as intended. Please try it out.
Thanks.
I just upgraded to 2.0.1 and everything seems to be working well. I'll report back if there are any bumps. Thank you for the quick fix!

New Tobin Paradigm

Re: ηMatrix/eMatrix support thread

Unread post by New Tobin Paradigm » 2019-05-17, 14:40

Phoebus 2.1 will have the ability to mark a version as busted. For now, be more careful about your releases.

I will however slide in an add-on userDisable option into the current code that will disable the whole add-on from the system until you release an update so you can do SOMETHING to stop proliferation of a bad current release.

userDisable to the site, aus, and integration when true will act like it doesn't exist or more properly has active set to false.

That I can do today.

User avatar
adesh
Board Warrior
Board Warrior
Posts: 1277
Joined: 2017-06-06, 07:38

Re: ηMatrix/eMatrix support thread

Unread post by adesh » 2019-05-17, 15:03

vannilla,

My eMatrix got updated from 1.4.0 to 2.0.1 automatically and I lost all of my settings. Browser was not restarted (not prompted) and extension is functioning normally.

This is not a complain, just to let you know. I'll create my rules as and when required.

New Tobin Paradigm

Re: ηMatrix/eMatrix support thread

Unread post by New Tobin Paradigm » 2019-05-17, 17:15

Did he not say he was changing where extension data is stored? Guess he never wrote a migration for it.

Uhh try going into your profile directory with the browser not open and go to extension-data/ and move ematrix.sqlite to ematrix-data/. That should resolve your issue.

Though, I would suggest to the developer to revert it back and quickly to it where it was. Although this was a bullshit webextension prep convention it does make sense for extension-data/[your-addon].sqlite and a good convention to keep around.

Either that or write a migration routine to move the db from the old to new location. Which shouldn't be difficult but will be work.

Establish Preference extensions.ematrix.dbMigrationDone as false in default prefs.

Code: Select all

pref = Services.pref.getBoolPref("extensions.ematrix.dbMigrationDone", false);
if (!pref) {
  oldDB = existence of file in old location;
  if (oldDB) {
    newDB = existence of db in new location; 
    if (newDB) {
      rename newDB to ematrix.sqlite.bak;
    }

    move old db file to new location;

    if (old directory is empty) {
      remove old directory;
    }

  migrated = existence of sqlite file in new location;
  
  if (!migrated) {
    Components.utils.reportError("Unable to migrate existing database...");
  }
  
  set pref to true;
}

// create/load db file routine
That is generally how you would handle it.

Locked