canary.mangadex.dev

For support with specific websites

Moderator: trava90

Forum rules
Please always mention the name/domain of the website in question in your topic title.
Please one website per topic thread (to help keep things organized). While behavior on different sites might at first glance seem similar, they are not necessarily caused by the same.

Please try to include any relevant output from the Toolkit Error Console or the Developer Tools Web Console using the following procedure:
  1. Clear any current output
  2. Navigate or refresh the page in question
  3. Copy and paste Errors or seemingly relevant Warnings into a single [ code ] block.
User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

canary.mangadex.dev

Unread post by tristan9 » 2023-08-26, 03:36

During a routine dev build check for compat with Pale Moon, I noticed a failure to load our public test build successfully.

PM 32.3.1 was used, with caches wiped and no particular profile/settings changed from the defaults.

Unfortunately the browser console isn't particularly useful, with only these errors:

Code: Select all

Content Security Policy: Ignoring “'unsafe-inline'” within script-src: ‘strict-dynamic’ specified  (unknown)
Content Security Policy: Ignoring “https:” within script-src: ‘strict-dynamic’ specified  (unknown)
Content Security Policy: Ignoring “http:” within script-src: ‘strict-dynamic’ specified  (unknown)
Except we definitely don't set a content-security-policy header, nor do we use meta-equiv HTML tags either.

I checked it the following as well with the same results:
- dev build (ie no minification/treeshaking/etc)
- local server (both dev and prod builds)

Maybe I'm missing some debug flag on Pale Moon to see what is going on, so feel free to peek here: https://canary.mangadex.dev

A couple more details:

I do see one extra (harmless on our end) warning within our code as discrepancy between current prod and dev:
- on dev, there is a warning about a feature flag being used but lacking a value
- on prod, it is all ok
But this isn't what breaks the rendering (that code hasn't changed, and I tested disabling its usage locally and it still failed the rendering).

And indeed, most pages are broken in a more interesting (or at least clearer) way than the homepage, with a cryptic "e is not defined" error.

Adding some debugging logs on our end, I can get the following to be spat out:

Code: Select all

Failed list fetch TypeError: res is undefined
Stack trace:
assertSuccess@http://localhost:3000/_nuxt/src/apiHelpers.ts:13:1
searchGeneric@http://localhost:3000/_nuxt/src/api/generic.ts:7:10
After much digging downstream, things go wrong around https://github.com/unjs/ofetch/blob/31c ... ch.ts#L160 when calling globalThis.fetch and using the return value.

It is invoked with the following:

Code: Select all

context.request = "https://api.mangadex.org/group?limit=32&offset=0"
context.options = Object { params: undefined, headers: undefined, query: undefined, method: "GET" }
At line 244 (so right after native fetch returns) we have:

Code: Select all

context = Object { request: "https://api.mangadex.org/group?limi…", options: Object, response: Response, error: undefined }
context.response = Response { type: "cors", url: "https://api.mangadex.org/group?limi…", redirected: false, status: 200, ok: true, statusText: "OK", headers: Headers, bodyUsed: false }
Then the local variable hasBody is defined as such:

Code: Select all

const hasBody = context.response.body && !nullBodyResponses.has(context.response.status) && context.options.method !== "HEAD";
But it... evaluates to undefined on the next line...?
Rather puzzling to me. Maybe I'm just misusing the PM debugger somehow, but this essentially prevents any response from being consumed since it's acting like they're all 204s as a result.

Can't dig much more on my own from here on out.
Last edited by tristan9 on 2023-08-26, 03:49, edited 1 time in total.

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-08-26, 04:53

Oh dammit I mislicked the "mark as solved" button instead of the edit one... Sigh. It is not solved, if a moderator passes by and can undo this.

Also, naturally, we do not return a 204 there, and PM's network tab shows that it is indeed all fine.

User avatar
Kris_88
Keeps coming back
Keeps coming back
Posts: 940
Joined: 2021-01-26, 11:18

Re: canary.mangadex.dev

Unread post by Kris_88 » 2023-08-26, 05:15

ReadableStream api is not supported by Pale Moon.
Probably Response.body is undefined.

https://developer.mozilla.org/en-US/doc ... ponse/body

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-08-31, 05:22

Well that makes sense, although it is a bit unfortunate. Seeing as we don’t explicitly use it ourselves, it is presumably coming from the fetch API wrapper (ofetch) or Nuxt and this will soon cause more issues.

While I appreciate that it’s a bit odd to do stream transformation, since it’s (by the looks of the MDN compatibility matrix) at least 7 years old, it doesn’t seem entirely like something to blame on a library for relying upon. Oh well.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35651
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: canary.mangadex.dev

Unread post by Moonchild » 2023-08-31, 07:37

tristan9 wrote:
2023-08-31, 05:22
it doesn’t seem entirely like something to blame on a library for relying upon.
It's always the problem in the current web ecosystem.
The problem is that APIs are being used left, right and centre for purposes they were not actually designed for but that web devs seem to be compelled to use because "it's there" and... I guess because they'd want to flex their creative muscles?
Many frameworks are using collections of APIs that really don't seem to be necessary. If you yourself already find it odd that stream transformation is being used, then that is because it is. the Streams API was designed for a very specific use case where client side data manipulation in-transit is essential to be able to present it to the user -- I can think of some scenarios that would be especially useful in terms of obfuscated delivery of data, for example. If it's being used in a utilitarian fashion, then that really falls outside of its intended scope.
In such situations it becomes a matter of "pass the buck". The website passes the buck to the framework devs. The framework devs pass the buck to us for not being Chrome and having 100% of the obscene set of specs implemented (many of which were penned by Google implementation-first).
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

dbsoft
Project Contributor
Project Contributor
Posts: 419
Joined: 2020-02-21, 17:35

Re: canary.mangadex.dev

Unread post by dbsoft » 2023-09-29, 04:53

I have this working in my development tree, so hopefully should have https://mangadex.org/ working in Pale Moon again fairly soon.

KlarkKentThe3rd
Astronaut
Astronaut
Posts: 556
Joined: 2018-04-20, 20:31

Re: canary.mangadex.dev

Unread post by KlarkKentThe3rd » 2023-09-29, 08:45

dbsoft wrote:
2023-09-29, 04:53
I have this working in my development tree, so hopefully should have https://mangadex.org/ working in Pale Moon again fairly soon.
I am full of hope. And patience.

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-09-29, 20:01

dbsoft wrote:
2023-09-29, 04:53
I have this working in my development tree, so hopefully should have https://mangadex.org/ working in Pale Moon again fairly soon.
Nice. Feel free to ask if you need any support for it.

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-10-20, 19:33

I see the compat with ReadableStream is now slated for 32.5.0.

Are there official (or unofficial) nightly builds somewhere against which we can check compat while we wait? The reason is that we're also still updating things as we go, and it'd be good to avoid the bad surprise of breaking PM compat again just as you fix it...

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35651
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: canary.mangadex.dev

Unread post by Moonchild » 2023-10-20, 20:00

We don't have nightly/unstable builds but if it'd help I can build one for you (Windows 32 or 64 bit)
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-10-22, 01:46

Would be helpful yes; Win 64 works.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35651
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: canary.mangadex.dev

Unread post by Moonchild » 2023-10-22, 11:21

"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-10-22, 17:59

Thanks. I gave it a try and it seems to not have broken anything compared to our current prod :thumbup:

There's still a lot of CSS quirks (probably us using CSS functions in places where those are still rather new), and I notice that PM doesn't implement alpha channel support for WebM videos, but at least nothing broken by a new API that I can see at a glance.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35651
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: canary.mangadex.dev

Unread post by Moonchild » 2023-10-22, 19:05

tristan9 wrote:
2023-10-22, 17:59
alpha channel support for WebM videos
videos with alpha channel is bonkers. :D
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2194
Joined: 2018-05-05, 13:29

Re: canary.mangadex.dev

Unread post by vannilla » 2023-10-22, 20:38

Moonchild wrote:
2023-10-22, 19:05
videos with alpha channel is bonkers. :D
Off-topic:
No way someone actually did that lmao :lol:

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-10-22, 21:29

I would agree, if GIF and APNG weren’t both absolutely awful in terms of efficiency…

But with a user-customizable background color, we can’t just have 2 pre-made video files for animations… hence videos with alpha channels.

That said support remains quite… finicky; so I didn’t expect PM to support it; just mentioned it as I saw WebM with VP9 is supported, which itself is rather recent.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35651
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: canary.mangadex.dev

Unread post by Moonchild » 2023-10-22, 22:03

I'm not sure what you're using it for. APNG works fine with us and isn't terrible re: performance but file sizes are relatively large. Gif should die already.
Both our WebP and jpeg-XL implementation support animation and alpha for transparent animated images, so... use that?

https://jpegxl.info/test-page/

Just keep video formats for actual video, alright?
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-10-22, 22:18

APNG works fine with us and isn't terrible re: performance but file sizes are relatively large
Files being about 10x larger than videos is what I meant yeah. But I can always do that for PM specifically I suppose.
Gif should die already.
Ack
Just keep video formats for actual video, alright?
What really is the difference between an animated image and a video without sound? As far as I'm concerned, one is 1.5MB when the other is 300KB*. That's roughly what went into this call.
Again, not that I expect PM to support something this niche. But it remains a fact that animated image formats are really crappy compared to video ones, at the time I'm writing this.
Ovbiously wouldn't use videos for the small animated emojis on a forum like here, but for very few larger animations (think 1 per page at most), they seem to be the only option to get something decent out.

*: After ~2 days of experimentation, it turns out that for the same exact animation (ie animation with transparency, 15s loop 512px width @ 15fps, and allowing something around 80% quality), optimization limits were around:
- GIF: 1.15MB (with custom palette use and gifsicle optimization with max effort)
- APNG: 1.7MB
- WebM VP8: 3xxKB
- WebM VP9: 1xxKB
- MOV HEVC w/ HVC1 specifically (precise needs for Safari...): 5xxKB

So yeah, it's quite clear-cut...

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35651
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: canary.mangadex.dev

Unread post by Moonchild » 2023-10-22, 22:43

tristan9 wrote:
2023-10-22, 22:18
What really is the difference between an animated image and a video without sound?
The difference is intended scope of use.
Animated images focus on delivering exactly what it says: animated (individual) images. This has a focus on reproducing each frame with minimal distortion, and is more often than not meant to be used for relatively small areas of display, and integration with other page content -- hence the stress on transparency there.
Video focuses on higher resolution and frame rate for delivering motion-first (streamed) media where distortion is perfectly acceptable, and is meant to be used for isolated content consumption (or even full-screen), and alpha channels would be subject to pretty bad artefacting because of stream compression. I'm not even sure how an alpha channel would blend in with YUV... I'm assuming they'd use a separate channel "tacked on" and have it use its own compression? Sounds really like a kludge in that case.

Now, because "gif" sites have long since converted animated images to video (also because actual pseudo-video gifs were a pervasive bane) and been serving those pseudo-video gifs actually as MP4 precisely because they did not want to 10-fold their bandwidth use, there's been some blurring of the lines going on, but even in those cases they would not be using transparency for those pseudo-video gifs (which was a terrible abuse of the GIF format, to be fair, but lacking proper web video way back when, people used what they had). the use of "gif" for that context should have died when html5 introduced <video>, but there was a lot of momentum keeping "video gifs" alive.

If bandwidth is a concern, you can/should use WebP or jpeg-xl for animated images (the former is widely supported in all current browsers, at least. I'm not sure how well jpeg-xl support is at the moment) as they can use lossy animated formats with transparency, drastically cutting down file sizes at the cost of a little bit of clarity (subject to compression settings). Or, if your animation is linework/vector, you could look into actually using SVG instead.

EDIT: By the way, I'm also seriously concerned about the performance impact of video with transparency - you can't really use any of the normal acceleration techniques for smooth video playback in that case or offload things to the GPU, as you'd have to send each video frame through the compositor... lower-end hardware would definitely struggle.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tristan9
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2023-06-08, 09:29

Re: canary.mangadex.dev

Unread post by tristan9 » 2023-10-23, 04:03

Moonchild wrote:
2023-10-22, 22:43
tristan9 wrote:
2023-10-22, 22:18
What really is the difference between an animated image and a video without sound?
The difference is intended scope of use.
Animated images focus on delivering exactly what it says: animated (individual) images. This has a focus on reproducing each frame with minimal distortion, and is more often than not meant to be used for relatively small areas of display, and integration with other page content -- hence the stress on transparency there.
Video focuses on higher resolution and frame rate for delivering motion-first (streamed) media where distortion is perfectly acceptable, and is meant to be used for isolated content consumption (or even full-screen), and alpha channels would be subject to pretty bad artefacting because of stream compression.
That is a distinction yes, but a bit of a weak one I think.
Rarely did someone ever use a GIF *because* they cared about perfect rendering of every single frame (excluding 2-5 frames loops kind of cases).
And in our case it is motion-first yes; otherwise we would be able to render a reasonably-sized GIF and have it work much more easily.
Moonchild wrote:
2023-10-22, 22:43
I'm not even sure how an alpha channel would blend in with YUV... I'm assuming they'd use a separate channel "tacked on" and have it use its own compression? Sounds really like a kludge in that case.
YUV 4:2:0 + an alpha channel, as far as I can see, yes.
As for the artifacting, it is non-existent in our case but it's only ever 0% or 100% so maybe that's why.
Moonchild wrote:
2023-10-22, 22:43
If bandwidth is a concern, you can/should use WebP or jpeg-xl for animated images (the former is widely supported in all current browsers, at least. I'm not sure how well jpeg-xl support is at the moment) as they can use lossy animated formats with transparency, drastically cutting down file sizes at the cost of a little bit of clarity (subject to compression settings). Or, if your animation is linework/vector, you could look into actually using SVG instead.
Bandwidth isn't a concern to us, but it is to users on crappy network connections. At least when it results in THAT big of a size difference.
Haven't looked at JXL because support is still quite limited, but fair point that it could be a good option to add to the list of sources. Though I somehow can guess that "support for JXL" and "support for JXL animations" will be two very different buckets, and a third one for transparency on top... (just like AVIF support is only for still images atm on Safari).
SVG animations are unfortunately a no-go in terms of size either (and frankly sound like the most expensive of all imaginable options).
Moonchild wrote:
2023-10-22, 22:43
EDIT: By the way, I'm also seriously concerned about the performance impact of video with transparency - you can't really use any of the normal acceleration techniques for smooth video playback in that case or offload things to the GPU, as you'd have to send each video frame through the compositor... lower-end hardware would definitely struggle.
How does this differ from any old GIF with transparency though? Though I suppose GIFs are usually very low FPS.