Unreadable characters

[Phlip]

WIth Palemoon 28.5.0 (64 bit) for Manjaro:

On the page https://www.linkedin.com/in/consultantlinked-in/ the beginning of each line of the summary appears as unrecognized characters.
(on the same system, in Firefox 67.0, by some trick, the same characters are perfectly readable in bold)

Thank you for your work.

[Lucio Chiappetti]

Do you mean the first character (to me appears like a triangle) or the rest of the "title" ? to me appears as normal text in bold italic. However, they appear as such also in "View source". I guess it depends on the font or character sets in your browser preference.
I wanted to know which Unicode characters they are, but if I save source the resulting file has too long lines for my editor. If I cut and paste to an Unicode capable terminal like urxvt, the triangle shows up but the other characters are empty square boxes (they are not recognized by my system fonts ?).

Cutting and pasting in an Unicode utility http://unicode.scarfboy.com/ the text (e.g. M of word MON) is identified as MATHEMATICAL SANS-SERIF BOLD ITALIC CAPITAL M 1D648. Strange ... beyond my mark.

[Lunokhod]

I can see the characters in unstable PM, and I have these font related packages installed, so I guess it must be in there somewhere and as Manjaro is Arch based you would have more or less the same packages available. (Some in the list are not actually fonts but things to interact with fonts.) Sometimes I get unreadable characters so I don't have every possible font character myself. Some font packages are really big, having too many installed makes for slow updates.

Code: Select all

$ pacman -Qs font
local/adobe-source-code-pro-fonts 2.030ro+1.050it-5
    Monospaced font family for user interface and coding environments
local/cantarell-fonts 1:0.111-1
    Humanist sans serif font
local/fontconfig 2:2.13.1+12+g5f5ec56-1
    A library for configuring and customizing font access
local/freetype2 2.10.0-2
    Font rasterization library
local/gsfonts 20180524-2
    (URW)++ Core Font Set [Level 2]
local/libfontenc 1.1.4-1
    X11 font encoding library
local/libxfont2 2.0.3-1
    X11 font rasterisation library
local/libxft 2.3.3-1
    FreeType-based font drawing library for X
local/ttf-dejavu 2.37-2
    Font family based on the Bitstream Vera Fonts with a wider range of characters
local/xorg-bdftopcf 1.1-1 (xorg xorg-apps)
    Convert X font from Bitmap Distribution Format to Portable Compiled Format
local/xorg-font-util 1.3.1-2 (xorg-fonts xorg)
    X.Org font utilities
local/xorg-font-utils 7.6-5
    Transitional package depending on xorg font utilities
local/xorg-fonts-100dpi 1.0.3-4 (xorg)
    X.org 100dpi fonts
local/xorg-fonts-75dpi 1.0.3-4 (xorg)
    X.org 75dpi fonts
local/xorg-fonts-alias 1.0.3-2
    X.org font alias files
local/xorg-fonts-encodings 1.0.4-5 (xorg-fonts xorg)
    X.org font encoding files
local/xorg-mkfontscale 1.2.1-1 (xorg-apps xorg)
    Create an index of scalable font files for X
local/xorgproto 2018.4-1
    combined X.Org X11 Protocol headers

[Phlip]

Yes, I do mean the "title", after the triangle.
If it can help explain the mystery, here are some screen captures:
- FF "view source"
- FF rendered page
- PM "view source"
- PM rendered page
(with PM and FF installed on the same system)
(Ah, and the behavior is the same whatever the user agent mode in PM)

[Night Wing]

After clicking on the link, the page renders properly for me without any unreadable characters.

I'm using 64 bit linux Pale Moon 28.5.0 running in 64 bit linux Mint 19.1 (Tessa) Xfce. The site sees my user agent as Firefox Compatibility mode and the only two addons/extensions I use are NoSquint and uBlock Origin.

[Lucio Chiappetti]

Here everything displays OK in PM. So since you can see them with FF, it means you have the adequate fonts somewhere on your system. Go to preferences->content->advanced in PM and equivalent in FF (wherever it be, I abandoned FF long ago) and make sure they are the same ...

... anyhow it is rather silly from the website author side to use funny unicode encoding to emulate simple things like italic and bold !
... (rant mode on) back in old NCSA Mosaic times site authors won't care how client browser will display their logical look (rant mode off)

[plushkava]

Here everything displays OK in PM. So since you can see them with FF, it means you have the adequate fonts somewhere on your system.

That's not necessarily a safe assumption because Firefox has been known to bundle fonts. The correct way to test for coverage of a given codepoint is to query the fontconfig system directly. Taking Phlip's second attachment as an example, the first unresolved codepoint shown is U+1D648 (MATHEMATICAL SANS-SERIF BOLD ITALIC CAPITAL M). To check for coverage thereof, enter the following command in a terminal:

fc-list ':charset=1D648'

Doing so will print a list of fonts that cover that particular codepoint, if any. Otherwise, it will print nothing, in which case it will be necessary to install a covering font, such as Noto Sans Math. Given access to a Windows machine, I would recommend grabbing its Segoe UI Symbol font (seguisym.ttf), because it covers many of the things that libre fonts do not.

[Phlip]

@Lucio Chiappetti : Oh, Yes, I fully agree with your rant!

@plushkava : Thanks, it does work:
fc-list ':charset=1D648'
/usr/share/fonts/mathjax/HTML-CSS/Gyre-Termes/woff/GyreTermesMathJax_SansSerif-Regular.woff: TeX Gyre Termes Math,Gyre Termes MathJax SansSerif:style=Regular
/usr/share/fonts/mathjax/HTML-CSS/STIX-Web/woff/STIXMathJax_SansSerif-BoldItalic.woff: STIX MathJax SansSerif:style=Bold Italic
/usr/share/fonts/mathjax/HTML-CSS/Gyre-Pagella/woff/GyrePagellaMathJax_SansSerif-Regular.woff: TeX Gyre Pagella Math,Gyre Pagella MathJax SansSerif:style=Regular
/usr/share/fonts/mathjax/HTML-CSS/Asana-Math/woff/AsanaMathJax_SansSerif-Regular.woff: Asana Math,Asana MathJax SansSerif:style=Regular
/usr/share/fonts/mathjax/HTML-CSS/Latin-Modern/woff/LatinModernMathJax_SansSerif-Regular.woff: Latin Modern Math,Latin Modern MathJax SansSerif:style=Regular

MathJax and "STIX MathJax" fonts do not display the text, but selecting in "Content / Advanced / Sans-serif" either "Asana Math", "Latin Modern Math", "Tex Gyre Termes Math" or "Tex Gyre Pagella Math" give the same readable result as FF (...which has ordinary BitStream Vera fonts selected. Go figure! I suppose it's a result of FF bundling fonts, as you mentioned.)

I'm so happy using PaleMoon for everything (even LinkedIn ), I just keep FF in a corner to have a comparison point when something goes wrong. And thanks to this forum, I can see a little under the hood, and understand some of what's going on... Thanks a lot!

[plushkava]

One important thing to keep in mind is that fontconfig can fall through to any available font in order to render a particular character, even though the application may have been configured to use a particular font by default. In principle, your default font preferences in Pale Moon should not matter and they should not need to be changed.

It is possible to instruct Pale Moon to report the fonts that were selected in the course of rendering a document, which might help you to debug this problem further. As long as you're using bash 4.2 or later, the following command will generate a suitable document for testing:

printf '<html><head><meta charset="utf-8"></head><body>\U1D648</body></html>' > ~/test.html

Load test.html in Pale Moon, press Ctrl+Shift+I to bring up the inspector, then click the Fonts tab on the right-hand side. I did so and found that the "Segoe UI Symbol" font was chosen and that the character was rendered correctly. I also installed and tested the GyreTermesMathJax_SansSerif-Regular.woff font (by making all other suitable fonts unavailable to fontconfig) and repeated the test. This time, "DejaVu Math TeX Gyre" was selected and, again, the character was rendered correctly. I made no changes to Pale Moon's font preferences.

All in all, the fontconfig engine can be a complex beast; it may be that it is not falling through to the font that would be expected in your case - perhaps due to misconfiguration or bad defaults on the part of your distro.

[Phlip]

I downloaded "Segoe UI Symbol" from sharefonts.net and found that, on my system, it does not display correctly this character, either in PM or LibreOffice. Maybe we don't have the same version of the font? The zipped ttf file is dated July 18, 2016.

Using the font tab of the inspector, I've found that FF displays it by falling back on the "Latin Modern Math" font: LatinModernMathJax_SansSerif-Regular

As it is the first time this happens, I will just leave my default font in PM as "Asana Math" (which looks good to me), at least until my distro goes through a major update.

[Lucio Chiappetti]

@plushkava : Thanks, it does work:
fc-list ':charset=1D648'
... list of fonts deleted

Curious that my openSUSE Leap 42.3 has fonts which allow PM to show the incriminated page but fc-list ':charset=1D648' replies "Unable to parse the pattern"

[plushkava]

Curious that my openSUSE Leap 42.3 has fonts which allow PM to show the incriminated page but fc-list ':charset=1D648' replies "Unable to parse the pattern"

Your fontconfig package is probably too old. I think that this pattern syntax requires at least version 2.11.91. Unfortunately, I don't know how to define this pattern in a way that would work in older versions. Sorry.

[Lucio Chiappetti]

2.11.1 indeed. Where is pattern syntax documented ? not in the man page (just for education, or curiosity)

[plushkava]

I'm not aware of any comprehensive documentation but the pattern matching syntax is mentioned here. It doesn't present the information in any way that makes the usage I've described obvious. Still …

Fontconfig provides a textual representation for patterns that the library can both accept and generate. The representation is in three parts, first a list of family names, second a list of point sizes and finally a list of additional properties …

<families>-<point sizes>:<name1>=<values1>:<name2>=<values2>...

Further …

<charset>

This element holds at least one <int> element of an Unicode code point or more.

In fact, I saw an example of how to use :charset in a git commit message and it is sometimes mentioned elsewhere. To specify more than one element, as they put it, a space can be used. Ranges are also accepted. For example:

Code: Select all

fc-list ':charset=1D648 1D649' # fonts that cover U+1D648 and U+1D649
fc-list ':charset=1D63C-1D655' # fonts that cover the range U+1D63C - U+1D655

[Lunokhod]

Checking that here shows the ttf-dejavu package provides it for me, which also happens to be the only specified font package dependency of the Chromium browser package. The Firefox package requires the more general ttf-font which is provided by this package and some alternative ttf font packages too.

[Axatax]

I see this all time, and it's usually where characters that deal with navigation within the site normally go.

The best way to see this is at walmart.com. All the characters at the top navigation bar display as things like chess pieces and astrological symbols and Chinese or Arabic characters, etc.

[vannilla]

I see this all time, and it's usually where characters that deal with navigation within the site normally go.

The best way to see this is at walmart.com. All the characters at the top navigation bar display as things like chess pieces and astrological symbols and Chinese or Arabic characters, etc.

If you get those characters rather than, say, arrows and such, then the problem is that the site wants to use a custom font that replaces those characters with arrows etc, but for some reason the font is unavailable.