Bug - Fetch API keepalive call during the beforeunload fails

Talk about code development, features, specific bugs, enhancements, patches, and similar things.
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.

This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.

Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-20, 22:21

In Chromium-based browsers (Chrome, Brave, Falcon nightly) and most up to date Gecko-based browsers (Firefox, Waterfox) this works, but on Pale Moon it doesn't.

This code calls a backend method during the "beforeunload" event via the Fetch API.

What this does and why:
I work as a corporate Backend Programmer, and I need to remove something from the session (which isn't available at the frontend...) when the tab with the page is closed, the browser is closed, the page is refreshed or the URL is redirected somewhere else (for example when clicking a link). Our last senior Frontend Programmer left the company months ago. Woe is me.

sendBeacon cannot be used due to its limits and the web application security. Synchronous ajax requests no longer work in the "beforeunload" event for Chromium-based browsers. Which left me with using Fetch API with keepalive.

I did leave the Czech comments for posterity. PS: Google translate doesn't work all that well for Czech. ;)

Code: Select all


 window.addEventListener("beforeunload", function(event) {
                
            *here should be code that gets attributeName and attributeValueToRemove, but believe it or not, the rest of the code has been extracted from a multitude of functions and these two params are just Strings, so feel free to use whatever instead*
                
            var data = new FormData();
            var token = $("meta[name='_csrf']").attr("content");

            //------------- POZOR - následující věci jsou extrémně háklivé na cokoliv!! Funguje to divně!----------------

            // add your data
            //Nefunguje, píše to Required String parameter 'attributeName' is not present.
/*          data.append("attributeName", attributeName);
            data.append("attributeValueToRemove", attributeValueToRemove);*/

            // add the auth token - u nás dle formulářů
            data.append("_csrf", token);

            //sendBeacon je extrémně omezený a nefunguje s naším zabezpečením aplikace, takže nelze použít, přestože tohle
            // je přesně jeho účelem
            //navigator.sendBeacon(url("/session/removeFromSessionArrayDuringUnload"), data);

            //Protože ajaxové volání je v synchronní podobě při unload eventech zakázáno a asynchronně nedojde do konce,
            // a sendBeacony nefungují s naším zabezpečením aplikace, tak použijeme Fetch API.
            //https://github.com/w3c/beacon/pull/27#issuecomment-241549321
            //https://stackoverflow.com/questions/38027231/how-to-make-navigator-sendbeacon-use-get-method
            //https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch
            //https://stackoverflow.com/questions/40893537/fetch-set-cookies-and-csrf

            const headers = new Headers({
                'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
                'Connection': 'keep-alive',
                'X-CSRF-TOKEN': token
            });

            //PS: potencionální TODO Chrome se může kdykoliv rozhodnout Fetch API při on before unload eventu omezit nebo
            //                       zakázat, tak jak to udělal se synchronním ajaxem při této události. Správně se tady
            //                       má použít sendBeacon, jenže ten je tak naschvál omezený, že se při našem zabezpečení
            //                       (csrf token, atd.) není možné spojit s backendem, nebo aspoň mě se to celé hodiny
            //                       nedařilo. Tzn. je možné, že dobudoucna bude nutné tohle předělat, když se něco
            //                       takového autoři Chromu rozhodnou provést a zakázat.

            //Proč jsou parametry v url viz komentář u data.append("attributeName"
            fetch(url("/session/removeFromSessionArrayDuringUnload?attributeName=" + attributeName + "&attributeValueToRemove=" + attributeValueToRemove), {
                method: "POST",
                cache: 'no-cache',
                body: data /*{
                //Nefunguje, píše to Required String parameter 'attributeName' is not present.
                    "attributeName": attributeName,
                    "attributeValueToRemove": attributeValueToRemove,
                    "_csrf": token
                }*/,
                headers: headers,
                credentials: 'include',
                mode: 'cors',
                keepalive: true,// i.e. don't terminate when fetch group is terminated - díky tomuhle se dotaz dodělá
                                // až do konce, ikdyž se třeba zavírá prohlížeč
            }).then(r => console.log("removeFromSessionArrayDuringUnload has finished!"));

            //Hack pro firefox - https://stackoverflow.com/questions/67754620/fetch-keep-alive-is-not-working-as-expected
            const time = Date.now();
            while ((Date.now() - time) < 500) {}

            //Viz dlouhé TODO někde výše, proč není použit beacon.
            //navigator.sendBeacon(url("/session/removeFromSession?attributeName=" + attributeName + "&attributeValueToRemove=" + attributeValueToRemove));
                
                
});

I have no idea how to file a bug report for Pale Moon, so I'm just putting it here. It's 23:00 and I need to return to work.

User avatar
Kris_88
Lunatic
Lunatic
Posts: 277
Joined: 2021-01-26, 11:18

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by Kris_88 » 2022-01-21, 07:25

What about using window.onunload ?
https://javascript.info/onload-ondomcontentloaded

Code: Select all

let analyticsData = { /* object with gathered data */ };

window.addEventListener("unload", function() {
  navigator.sendBeacon("/analytics", JSON.stringify(analyticsData));
});

User avatar
Kris_88
Lunatic
Lunatic
Posts: 277
Joined: 2021-01-26, 11:18

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by Kris_88 » 2022-01-21, 08:05

Or maybe better use the visibilitychange event.
https://stackoverflow.com/questions/419 ... load-event
https://developer.mozilla.org/en-US/doc ... sendBeacon

In any case, there is no guarantee your promise will be executed.
fetch(...).then(r => console.log("removeFromSessionArrayDuringUnload has finished!"));
Last edited by Kris_88 on 2022-01-21, 08:29, edited 1 time in total.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 32026
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by Moonchild » 2022-01-21, 08:25

AFAICT Fetch keepalives also don't work in Firefox, as they still have an open bug for that that hasn't been addressed, so I'm surprised that you say it works there.
In general, relying on critical operations to occur when users navigate away from pages or unload documents isn't a good idea; especially if you want to do it asynchronously, since it would require the browser to keep workers alive in a service worker type fashion which we explicitly don't do (for very obvious tracking reasons if nothing else).
You may have to do some detection and use synchronous calls in our case that Chrome has stopped supporting (why is beyond me -- this is a good example where you'd need synchronous processing).

@Kris_88: you didn't pay attention - sendBeacon isn't something they can use; see OP.
"You will observe with concern how long a useful truth may be known and exist before it is generally received and practiced on." -- Benjamin Franklin
"Compromise and collaboration lie at the heart of all great endeavours" -- Kassandra
Image

User avatar
Kris_88
Lunatic
Lunatic
Posts: 277
Joined: 2021-01-26, 11:18

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by Kris_88 » 2022-01-21, 08:42

Moonchild wrote:
2022-01-21, 08:25
@Kris_88: you didn't pay attention - sendBeacon isn't something they can use; see OP.
I have seen. But maybe OP was mistaken in this.

And it is also possible that the request works, but the promise does not work.

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-21, 10:20

Kris_88 wrote:
2022-01-21, 08:05
In any case, there is no guarantee your promise will be executed.
fetch(...).then(r => console.log("removeFromSessionArrayDuringUnload has finished!"));
On Chromium-based browsers - it is. I've been able to stall closing the tab or the browser for entire minutes by having a breakpoint at the BACKEND inside the Java method the Fetch API calls to.

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-21, 10:25

Kris_88 wrote:
2022-01-21, 07:25
What about using window.onunload ?
https://javascript.info/onload-ondomcontentloaded

Code: Select all

let analyticsData = { /* object with gathered data */ };

window.addEventListener("unload", function() {
  navigator.sendBeacon("/analytics", JSON.stringify(analyticsData));
});
Can't be used for reasons that I am unable to disclose. Even if it could be, why would I do that? It works in all up-to-date Chromium-based browsers, it works in Firefox and its derivatives (like Waterfox), then why should I change stuff SPECIFICALLY for Pale Moon (and possibly all of its Goanna offshoots)? How would I explain time spent on that (I need to make hourly work reports on what I do) to my manager? For years, our primary web application browser target was IE (UGH) and now our customer has finally (in December 2021...) moved to Edge, so that is the primary supported browser. I just found it weird that the code doesn't work on Pale Moon, so I've decided to give you guys an echo.

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-21, 10:27

Moonchild wrote:
2022-01-21, 08:25
AFAICT Fetch keepalives also don't work in Firefox, as they still have an open bug for that that hasn't been addressed, so I'm surprised that you say it works there.
I have a hack for that which works well enough not for me to worry about it.

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-21, 10:35

Kris_88 wrote:
2022-01-21, 08:42
Moonchild wrote:
2022-01-21, 08:25
@Kris_88: you didn't pay attention - sendBeacon isn't something they can use; see OP.
I have seen. But maybe OP was mistaken in this.

And it is also possible that the request works, but the promise does not work.
The request doesn't work in Pale Moon, as the attribute is not removed from the session (which is what the call to backend does - all requests are logged in the backend and the backend doesn't even get the request if Pale Moon is used), causing later snafus and the reason why I even found out about it.

Both request and promise work in Chromium-based browsers and yadayadayada, I've already said it too many times. I can see the log text from the promise in the web console as the tab closes.

sendBeacon cannot be used, as it is extremely limited and I cannot modify what it sends to the server enough to be acceptable by security measures on the server - the server is contacted, but logs that it refused to process the request. This is something I cannot change on the server side. Thus sendBeacon cannot be used.

PS: I forgot to mention what version of Pale Moon I'm using: 29.4.4 64-bit

I'm a big fan of Pale Moon, by the way. Too bad I cannot use Chromium extensions, otherwise it could very well have been my main browser.

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-21, 10:44

Moonchild wrote:
2022-01-21, 08:25
You may have to do some detection and use synchronous calls in our case that Chrome has stopped supporting (why is beyond me -- this is a good example where you'd need synchronous processing).
Oh, yeah. Originally I did use synchronous calls, and it worked perfectly. In Waterfox. Then I tried Chrome and Brave and found out that they rather recently made the decision that synchronous calls are not to be made from a set of events, the one I use was among them. When I found out about it, I was very, VERY angry. You are rather perceptive, so you have probably even spotted the link I gave to them doing this in version 80.

Because Edge is the main target browser for our corporate web application, I was forced to use the second route.

Bollocks.

I could probably only use the Fetch API for Chromium-based browsers by parsing the navagent (and use synchronous calls for everything else), but that is a fragile solution, as many browsers (Brave!) hide the fact in some manner, shape or form.

But as I have said, we do hourly reports on our progress per each phase of each contract in a project, so my project manager probably wouldn't see this as progress, as it really isn't.

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-21, 11:09

I'm a big fan of Pale Moon, by the way. Too bad I cannot use Chromium extensions, otherwise it could very well have been my main browser.
PPS: (can't edit a reply that hasn't been approved yet, I think)
I DO however use Pale Moon as my main browser when booted under Windows XP 32-bit in the form of New Moon.
(non-virtualized XP. A 15+year-old installation moved from drive to drive as drives failed, still running smooth, just with a bit of regular maintenance done once per a few months... getting it running on modern hardware is rather tricky nowadays! Thousands of old games installed and running great (without exaggeration))

This could very well be offtopic, but it wouldn't be that much if I could edit one of my latest posts and put it there. Just a little bit of background info. The WinXP part is for people not to yell at me to use "Wyndows 13, the bestest operatyng sistem evur!", but they probably will anyway.

User avatar
pale guru
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2021-11-06, 11:10
Location: Tyskland

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by pale guru » 2022-01-22, 05:17

Before I open a new thread for this, a sidenote:

Wouldn't it be recommended to set

dom.disable_beforeunload → true

as a default setting?

Ie. Shopify is a great user of before_unload via XMLHttprequest to track user which page is visited how long.
… tanning in dimmed LCD light. – Evry 1′s a beginner, baby, that's the truth…

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-22, 17:57

pale guru wrote:
2022-01-22, 05:17
Before I open a new thread for this, a sidenote:

Wouldn't it be recommended to set

dom.disable_beforeunload → true

as a default setting?

Ie. Shopify is a great user of before_unload via XMLHttprequest to track user which page is visited how long.
So that even more stuff that works in other browsers won't work in Pale Moon and derivatives? Do we want another Internet Explorer?
pale guru wrote:
2022-01-22, 05:17
Wouldn't it be recommended to set
Recommended by whom?
pale guru wrote:
2022-01-22, 05:17
Ie. Shopify is a great user of before_unload via XMLHttprequest to track user which page is visited how long.
Never even heard of shopify. The tab won't event close unless the beforeunload event finishes (in Chromium-based browsers). If the user doesn't get suspicious at his tab refusing to close and goes to other pages, then it should be his problem.

Default settings shouldn't break compatibility, at least in my head.

"He who forfeits freedom for safety shall have neither."

User avatar
pale guru
Hobby Astronomer
Hobby Astronomer
Posts: 22
Joined: 2021-11-06, 11:10
Location: Tyskland

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by pale guru » 2022-01-27, 02:29

Default settings shouldn't break compatibility,
Is there any serious use of this beforeUnload event, other than tracking users?
… tanning in dimmed LCD light. – Evry 1′s a beginner, baby, that's the truth…

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-27, 10:23

pale guru wrote:
2022-01-27, 02:29
Default settings shouldn't break compatibility,
Is there any serious use of this beforeUnload event, other than tracking users?
Yes, web application analytics is usually given as an example.

It is used to do something when the user leaves the page in some way (closing the tab, closing the browser, refreshing the page, clicking on a link,...).

In my specific case, I have to remove something from the session.

There are legitimate uses that work under Chromium-and-Gecko-based browsers but would break in Goanna-based browsers if what you think should be set as default would be set as a default. And that is a bad thing.

If a website doesn't work or has bugs under Palemoon, but works fine in Chromium-and-Gecko-based browsers, do you think the users will spend hours trying to figure out why it doesn't work, or just try out a different browser instead?

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 32026
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by Moonchild » 2022-01-27, 11:41

pale guru wrote:
2022-01-27, 02:29
Is there any serious use of this beforeUnload event, other than tracking users?
templayer wrote:
2022-01-27, 10:23
Yes, web application analytics is usually given as an example.
I don't think application analytics is something a web client should worry about compatibility with. If you insist on gathering telemetry/analytics on user visits, there's going to be work needed for what is optional. If a user terminates the browser (and possibly on tab close too), there also won't be any beforeunload event fired (that only happens on navigation) so it's at best a very unreliable metric to use.
templayer wrote:
2022-01-21, 10:27
I have a hack for that which works well enough not for me to worry about it.
Well then, why did you say in your OP:
In Chromium-based browsers (Chrome, Brave, Falcon nightly) and most up to date Gecko-based browsers (Firefox, Waterfox) this works, but on Pale Moon it doesn't.
...because clearly it doesn't work in "most up to date Gecko-based browsers". Why did you present it as a bug specifically for us when it wasn't?

So you're saying it doesn't work in Firefox but you have a hack for that. What is this hack? Why doesn't it (the hack) work on Pale Moon?
Also... why are you expecting us to solve this already non-unified behaviour in the core to cater to optional analytics data gathering that most users of Pale Moon would actually want to avoid anyway?
"You will observe with concern how long a useful truth may be known and exist before it is generally received and practiced on." -- Benjamin Franklin
"Compromise and collaboration lie at the heart of all great endeavours" -- Kassandra
Image

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-27, 11:55

Moonchild wrote:
2022-01-27, 11:41
pale guru wrote:
2022-01-27, 02:29
Is there any serious use of this beforeUnload event, other than tracking users?
templayer wrote:
2022-01-27, 10:23
Yes, web application analytics is usually given as an example.
I don't think application analytics is something a web client should worry about compatibility with. If you insist on gathering telemetry/analytics on user visits, there's going to be work needed for what is optional. If a user terminates the browser (and possibly on tab close too), there also won't be any beforeunload event fired (that only happens on navigation) so it's at best a very unreliable metric to use.
templayer wrote:
2022-01-21, 10:27
I have a hack for that which works well enough not for me to worry about it.
So you're saying it doesn't work in Firefox but you have a hack for that. What is this hack? Why doesn't it work on Pale Moon?
Also... why are you expecting us to solve this already non-unified behaviour in the core to cater to optional analytics data gathering that most users of pale moon would actually want to avoid anyway?
1. It triggers both on tab closing and browser closing if the browser closes gracefully.
2. For Palemoon to be a viable alternative browser.

To reiterate - I know it is hard to understand, but the analytics doesn't have to be some evil.... I don't even know how to write it. When I wrote analytics, I mean non-user-misusable-data analytics, because I'm taking this from a web-application standpoint in a closed corporate environment. I didn't mean user tracking or going for user's information. It's hard to explain. All of my browsers use adblocks, Decentraleyes, stricter pop-up blockers, etc. Analytics doesn't automatically mean tracking the user, etc.

Every single event in a browser can be misused with malicious intentions.

But I see I have failed to explain this properly. Well, I will just have to add that Palemoon and Goanna-based browsers are not supported.

It's too bad, I liked Palemoon.

Bye.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 32026
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by Moonchild » 2022-01-27, 12:34

1. in Chrome. Because Chrome prioritizes analytics and tracking.
2. Not triggering this event asynchronously on some background thread after the content has already unloaded does not make Pale Moon any less viable of a browser. Its task is to display web content. Its task is not to satisfy arbitrary analytics gathering.

You also made a point to not explain what hack you're using for Firefox and why that hack doesn't work in Pale Moon, or why you initially presented this as a Pale Moon bug when it was not unique to us.
You also seem to be unwilling to make any concessions for non-Chrome browsers or a "one size fits all" solution (i.e. not considering using synchronous calls just because Chrome stopped supporting them -- so which browser has a bug there? I could argue it's Chrome having the bug)
You haven't failed to explain this at all. It's very clear. But you've not answered crucial questions coming forth from your explanation that might actually lead to a solution for Pale Moon.

Also, I'm aware any event is a double-edged sword, and "onbeforeunload" has been abused by trap sites too; but i leave it enabled by default for a reason because it HAS good uses - just doesn't work in the way you want it to and our users would, as stated, rather avoid silent metrics gathering by web servers when they close/navigate away.

But it seems you're not wanting to consider anything else. As a result you're now, as far as I gather from your tone, explicitly going to block Pale Moon in whatever code you're writing just because you can't gather your analytics "the Chrome way"...
And people wonder why I am reluctant to discuss technical aspects in the browser these days. :problem:
"You will observe with concern how long a useful truth may be known and exist before it is generally received and practiced on." -- Benjamin Franklin
"Compromise and collaboration lie at the heart of all great endeavours" -- Kassandra
Image

User avatar
templayer
Moongazer
Moongazer
Posts: 11
Joined: 2022-01-20, 20:59
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by templayer » 2022-01-27, 13:11

Moonchild wrote:
2022-01-27, 12:34
As a result you're now, as far as I gather from your tone, explicitly going to block Pale Moon in whatever code you're writing just because you can't gather your analytics "the Chrome way"...
I just read this last sentence and it really, really annoyed me. ANALYTICS IS USED AS AN EXAMPLE IN THE LINKS IN THE CODE I HAVE PROVIDED.

OUR COMPANY DOESN'T USE THE EVENT IN QUESTION FOR ANALYTICS. AT ALL.

If you REALLY need to know: We have an array of UUIDs for edited database entities in the session for our web application. On certain pages, leaving the page removes a UUID from the said array. There is no retarded user tracking in any shape or form!

"Our" analytics my ass.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 32026
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Bug - Fetch API keepalive call during the beforeunload fails

Unread post by Moonchild » 2022-01-27, 14:04

Oh, fantastic, so now because I've assumed something because I wasn't told anything concrete you think it's OK to shout at me?
Great going. Very professional behaviour there.

Honestly, at this point I really don't care what critical function you've hooked up to an event that's not guaranteed to fire; whether analytics or anything else. I've explained multiple times already it's not a good idea to rely on asynchronous callbacks on content that is no longer active, regardless of how or for what you use this event. Browsers tend to garbage/cycle collect disconnect events out of necessity. Assuming every browser has the exact same behaviour as Chrome has been an error in judgement on your behalf and giving me flak over it will not get you anywhere. In fact, what little incentive there was to look into making this kind of behaviour possible is effectively gone after that spat.
I wish you good luck with your fragile setup. I hope for you that the Chrome implementation never changes again or you'll be facing the same problem.
"You will observe with concern how long a useful truth may be known and exist before it is generally received and practiced on." -- Benjamin Franklin
"Compromise and collaboration lie at the heart of all great endeavours" -- Kassandra
Image

Locked