33.2.0 bug with Cyrillic text and digits

[_yuyu_]

In Pale Moon 33.2.0 on Windows when a series of digits occurs in a Cyrillic text, those digits cannot be highlighted individually, but only as a whole block. For example: абвгд 12345 иклмн
That was not the case with PM 33.1.1.

[RealityRipple]

Sounds like it's related to the changes in UXP PR#2514, since that dealt with preventing multi-character sequence "clusters" from being split up, but I don't know why standard Cyrillic characters would trigger it. There is a known problem with some emoji characters not being detected due to the underlying Unicode (ICU) version, but I don't think it would mess up so badly as to mark any of the Cyrillic set as part of the various Emoji and Pictographic lists.

Currently, I'm running into issues just trying to get a working build (some Visual Studio update baloney) but I'll try to do some tests and see if I can narrow this down.

Tested. Yes, this seems to be caused by the latest Emoji-related changes. I'm narrowing the scope of the problem down, comparing FF behavior in some areas, and determining the best course of action.

Firefox does suffer from the same bug, but only with the skin tone set, which is the way their IsClusterExtender() function works. Apparently, so does Chrome. And Notepad in Windows 11. I'm detecting a theme.

[Moonchild]

Off-topic:

only with the skin tone set

If you ask me, this whole skin tone modifier stuff is stupid and unnecessary. Aside from creating racial profiling by people using certain emoji, which I'm sure was the opposite of what was intended when people pushed hard for being "inclusive" in their emoji sets, demanding human emoji get their skin colour instead of the generic, accepted, banana yellow for all, why would this be necessary to express anything?

[RealityRipple]

Off-topic:

only with the skin tone set

If you ask me, this whole skin tone modifier stuff is stupid and unnecessary. Aside from creating racial profiling by people using certain emoji, which I'm sure was the opposite of what was intended when people pushed hard for being "inclusive" in their emoji sets, demanding human emoji get their skin colour instead of the generic, accepted, banana yellow for all, why would this be necessary to express anything?

I completely agree, but Unicode has decided to make everyone's life hell instead.

Also, long-winded, overly verbose issue with a quick solution, or a better-than-other-browsers-but-needs-a-little-research solution: https://repo.palemoon.org/MoonchildProductions/UXP/issues/2538.

[Moonchild]

Thanks for making that extensive analysis and write-up. See my comment on it to answer your questions there, and keep implementation discussion on the repo.

[SemiKebab]

Another problem, I am encountering issues in a textearea on Wikipedia, where in some cases, I cannot properly navigate/select the characters. It behaves a bit like RTL text (the caret "jumps" to the opposite side), but it's even impossible to do the precise text selection I want.

It seems to occur around sequences of digits, and if the « ↗ » character is present in the textarea (but depending on the spaces/newlines around, the bug happens or not).

I strongly guess this bug is related to the recent changes about handling of emojis. Though, I suppose you already have better test cases than the one I reported here.

[SemiKebab]

RealityRipple, thank you for the great work you have made in issue #2538 and PR 2539 about this!

As a side note, about this comment:

(not that I think filenames with emoji in them is in any way sane, but that's something for a different discussion...)

Just to provide an example, creators on YouTube often put emojis in their videos' titles. And indeed, I dislike this too, it causes many issues (for example, with such filenames on Android phone).

[Moonchild]

The fact that people use them in filenames and that YouTube is too dumb to translate or sanitize video titles to file names doesn't make the practice any more sane.