Info on DNS blacklisting in browser

Users and developers helping users with generic and technical Pale Moon issues on all operating systems.

Moderator: trava90

Forum rules
This board is for technical/general usage questions and troubleshooting for the Pale Moon browser only.
Technical issues and questions not related to the Pale Moon browser should be posted in other boards!
Please keep off-topic and general discussion out of this board, thank you!
Lucio Chiappetti
Astronaut
Astronaut
Posts: 654
Joined: 2014-09-01, 15:11
Location: Milan Italy

Info on DNS blacklisting in browser

Unread post by Lucio Chiappetti » 2020-07-17, 08:05

I thought to know about DNS (I configured the one of my institute) and about antispam e-mail blacklisting (I did the same for spamassassin in the past) ... but I am realizing I am not familiar at all with DNS blacklisting in browsers (or in web servers ?).

Recently (now that I am forced to work from home) I tried to access (from PM) a couple of NASA sites (fits.gsfc.nasa.gov and heasarc.gsfc.nasa.gov) and got a "site unreachable" message. When I asked my colleagues whether they knew of some problems, I was told that: (a) everything works normally from the institute (which is on the academic GARR network); (b) it works normally from home using some providers; (c) it does not work for those using as provider TIM (the main Italian Telecom company).

In fact a search finds rumours like this https://community.tim.it/t5/MODEM-ROUTE ... m-p/173088 (in Italian) that TIM has been inserted in some blacklists and this is causing the problem. At DNS level.

In fact if I am on ssh at the institute a ping, tracepath, host or nslookup resolves the affected sites addresses perfectly. If I try the same from home I get "time out - no servers can be reached". It is clearly a DNS problem as using another DNS on the fly e.g. host -d fits.gsfc.nasa.gov 8.8.8.8 it works.

Now before embarking in changing the default DNS (TIM ones) either on the router, or on the laptop (which unfortunately is an Ubuntu one ... I say unfortunately because Ubuntu handles DNS in an odd unfamiliar way ... it is not enough to ediy /etc/resolv.conf as I was used on OpenSuse), I tried a simpler approach i.e. I inserted the IP addresses and host names of the two interesting sites in /etc/hosts statically.

Now this works at the base level (ping and tracepath resolve the address) but NOT within the browser (still times out) and even with the "host" command it appears that there is a double lookup. I mean ...
  • if I do host xxxx on a normal site I get a reply from 127.0.0.53#53 (which is the Ubuntu way to forward to the provider real DNS
  • if I do host xxxx on an arbitrary NASA site I get time out no servers can be reached
  • if I do host xxxx on the sites added in /etc/host I get a first reply from 127.0.0.53#53 returning the value from /etc/host and then a time out no servers can be reached (is it checking twice ? doing also a reverse ?)
So I wonder whether repointing the DNS will cure my problem at browser level or still be blocked by this apparent double lookup (or simply because the server will detect my IP is in a range assigned to TIM and block that. And anyhow I'd like to know if a browser, namely Pale Moon, does something more complicated that a plain single dns lookup.
The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. (G.B. Shaw)

Lurker_01
Fanatic
Fanatic
Posts: 122
Joined: 2015-06-12, 14:59
Location: Uruguay

Re: Info on DNS blacklisting in browser

Unread post by Lurker_01 » 2020-07-17, 08:17

Moonchild wrote:
2019-12-20, 18:00
... the browser does cache DNS lookups unless you explicitly disable that in the browser (which you can do for testing , of course, but not recommended to disable it for normal use because it will hammer your DNS server).
As a simple user i cant find anything more from a very quick forum search

User avatar
Pentium4User
Board Warrior
Board Warrior
Posts: 1114
Joined: 2019-04-24, 09:38

Re: Info on DNS blacklisting in browser

Unread post by Pentium4User » 2020-07-17, 08:19

Pale Moon uses the DNS server set in the system.
In Ubuntu it is 127.0.0.53 which is systemd-resolve.
systemd-resolve gets the DNS server via NetworkManager or Netplan.
If you just installed Ubuntu and did not configure Netplan, you have to use NetworkManager to change the DNS.
Default is DHCP for IPv4 and IPv6.
Click on the network icon on the upper bar in Ubuntu and then select "edit connections".
Then select your connection and go to the IPv4.
Then change it from automatically to Automatic (DHCP) addresses only.

Then you can type the IP of a non-blacklistung DNS server in the downer field "DNS server".
Use an IPv4 address for it.

Then select IPv6 and select (Automatic, addresses only, NOT DHCP only).
The type the IPv6 address of a trusted DNS server in the downer field.
The profile picture shows my Maico EC30 E ceiling fan.

User avatar
Admin
Site Admin
Site Admin
Posts: 405
Joined: 2012-05-17, 19:06

Re: Info on DNS blacklisting in browser

Unread post by Admin » 2020-07-17, 09:50

If DNS would be blacklisted, you would get a "host not found", and not a connection timeout, no?
Did you know that moral outrage triggers the pleasure centers of the brain? It's unlikely you can actually get addicted to outrage, but there is plausible evidence that you can become strongly predisposed to it.
Source: https://www.bbc.co.uk/programmes/p002w557/episodes/downloads - "The cooperative species" and "Behaving better online"
Image

Lucio Chiappetti
Astronaut
Astronaut
Posts: 654
Joined: 2014-09-01, 15:11
Location: Milan Italy

Re: Info on DNS blacklisting in browser

Unread post by Lucio Chiappetti » 2020-07-17, 10:30

I just tried NOW to go to fits.gsfc.nasa.gov, whose IP is currently set in /etc/hosts (i.e. resolved by ping or tracepath) with the Developer->Network tool active. The tool shows nothing, PM gives "Problem loading page" with "The server at fits.gsfc.nasa.gov is taking too long to respond." ... I am afraid this means somebody on the way is blocking my dynamic IP assigned by TIM, therefore I presume that moving from a TIM DNS to a public DNS won't help.
The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. (G.B. Shaw)

User avatar
Pentium4User
Board Warrior
Board Warrior
Posts: 1114
Joined: 2019-04-24, 09:38

Re: Info on DNS blacklisting in browser

Unread post by Pentium4User » 2020-07-17, 10:35

Lucio Chiappetti wrote:
2020-07-17, 10:30
I just tried NOW to go to fits.gsfc.nasa.gov, whose IP is currently set in /etc/hosts (i.e. resolved by ping or tracepath) with the Developer->Network tool active. The tool shows nothing, PM gives "Problem loading page" with "The server at fits.gsfc.nasa.gov is taking too long to respond." ... I am afraid this means somebody on the way is blocking my dynamic IP assigned by TIM, therefore I presume that moving from a TIM DNS to a public DNS won't help.
As Admin mentioned, there should be another error message. Maybe TIM does block the HTTP/HTTPS traffic to the site.
Ask your provider.
The profile picture shows my Maico EC30 E ceiling fan.

User avatar
adesh
Board Warrior
Board Warrior
Posts: 1277
Joined: 2017-06-06, 07:38

Re: Info on DNS blacklisting in browser

Unread post by adesh » 2020-07-17, 11:08

I think it's not DNS issue rather your public IP / network is blocked (being on blocklist?) by the site in question.

Lucio Chiappetti
Astronaut
Astronaut
Posts: 654
Joined: 2014-09-01, 15:11
Location: Milan Italy

Re: Info on DNS blacklisting in browser

Unread post by Lucio Chiappetti » 2020-07-17, 11:54

I guess it's both. If I try a wget anysite.nasa.gov I get a "Temporary failure in name resolution", if I do a wget fits.gsfc.nasa.gov I get a successful resolution of the name (via my /etc/host) then it times out on port 443. Which for me means a workaround like changing my DNSs won't be effective.

The IP is dynamic and changes every time, I checked the current one (which I can get with a "who" when ssh'ed on my institute machine) and is indeed in a couple of blocklists. As said in the URL reported in the first post, (1) this seems the case for most TIM networks, (2) contacting their support is a lost cause (suggestion is to send registered mail and waiting 10 days). Also I am not their customer ... the SIM I have has been supplied by my institute (and is a TIM SIM because TIM is the MEPA government provider).

It is not critical, and I will find another solution.
The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. (G.B. Shaw)

Locked