Anyone interested in a patch to disable GTK2 on distros that don't support it?

[athenian200]

So, I created a PR here...

https://repo.palemoon.org/MoonchildProd ... pulls/2408

And what it basically does, is allow building on systems that don't have GTK2 available, possibly at the expense of compatibility with GTK2-based NPAPI plugins. This is mostly about providing options to people on distros without GTK2 who may want to build themselves, and not care about the risk of some NPAPI plugins ceasing to function. I suspect GTK3 plugins will work fine.

While the implications of disabling this are largely unknown, the option should be on the table, IMO.

Would anyone be interested in helping me test this out to make sure it works as intended? That is, nothing should change at all if you don't specify "--disable-gtk2" in your .mozconfig, but if you do specify it, then you should be able to build without any gtk2 packages on your system.

[dbsoft]

This is probably a good idea for alternate architectures too. My understanding is this is predominantly used for supporting the Flash plugin, since it is GTK2.

Platforms like FreeBSD never had a the Flash plugin at all, so this feature is mostly useless, so I'd prefer to build without GTK2 plugin support.

Historically on FreeBSD to use the Flash plugin you'd use the Linux binary compatibility system, in which case you'd be running the Linux binary and not the FreeBSD one.

[micwoj92]

Definitely interesting. Distro that I use still supports GTK2, I don't use any NPAPI plugins and I don't use Pale Moon that much nowadays, but it is always nice to drop that unneeded (36MB in my case) gtk2 (build)dependency.
Out of curiosity I checked and only packages that needed GTK2 on my system were LXDE packages. I have now switched these to GTK3 version.
Anyway, back on topic: I have tried building Pale Moon with this patch, however it fails "make package" as file bin/gtk2/libmozgtk.so is no longer built.

Code: Select all

Error: /build/palemoon-gtk3/src/pmbuild/palemoon/installer/package-manifest:60: Missing file(s): bin/gtk2/libmozgtk.so
Traceback (most recent call last):
  File "/build/palemoon-gtk3/src/Pale-Moon/platform/toolkit/mozapps/installer/packager.py", line 423, in <module>
    main()
  File "/build/palemoon-gtk3/src/Pale-Moon/platform/toolkit/mozapps/installer/packager.py", line 374, in main
    copier.add(mozpath.join(respath, 'removed-files'), removals)
  File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/build/palemoon-gtk3/src/Pale-Moon/platform/python/mozbuild/mozpack/errors.py", line 131, in accumulate
    raise AccumulatedErrors()
mozpack.errors.AccumulatedErrors
make[2]: *** [/build/palemoon-gtk3/src/Pale-Moon/platform/toolkit/mozapps/installer/packager.mk:41: stage-package] Error 1
make[2]: Leaving directory '/build/palemoon-gtk3/src/pmbuild/palemoon/installer'
make[1]: *** [/build/palemoon-gtk3/src/Pale-Moon/platform/toolkit/mozapps/installer/packager-uxp.mk:16: make-archive] Error 2
make[1]: Leaving directory '/build/palemoon-gtk3/src/pmbuild/palemoon/installer'
make: *** [/build/palemoon-gtk3/src/Pale-Moon/platform/../palemoon/build.mk:9: package] Error 2

After patching palemoon/installer/package-manifest.in locally I have normal Arch Linux pkg.tar.zst package.

On my system the resulting package is is whopping 27232 bytes smaller.

[athenian200]

Definitely interesting. Distro that I use still supports GTK2, I don't use any NPAPI plugins and I don't use Pale Moon that much nowadays, but it is always nice to drop that unneeded (36MB in my case) gtk2 (build)dependency.
Out of curiosity I checked and only packages that needed GTK2 on my system were LXDE packages. I have now switched these to GTK3 version.
Anyway, back on topic: I have tried building Pale Moon with this patch, however it fails "make package" as file bin/gtk2/libmozgtk.so is no longer built.

That's not included in this patch, that would need to be a PR made against the Pale Moon repo, not the UXP repo. I was planning to do that after the main patch is finished. But the eventual patch will include changes to the package manifest in Pale Moon to make including that dependent on MOZ_ENABLE_GTK2. I just don't want to jump the gun and open a PR for the changes on the Pale Moon side if the patch winds up not being accepted into UXP, because of some kind of issues that come up during testing, etc...

EDIT: This is an example of what the package-manifest.in file will look like if it's accepted, though.

https://repo.palemoon.org/athenian200/e ... 7082ffab1e

I figure it won't hurt anything to mess up my own application with an extra commit or two, while I would rather keep MC's stuff as clean as possible and not touch anything until the UXP commit is accepted.

[frostknight]

Is it already possible to disable GTK3 in builds? If so, then being able to disable gtk2 probably is wise also.

[athenian200]

Is it already possible to disable GTK3 in builds? If so, then being able to disable gtk2 probably is wise also.

Hmm... well, on most platforms popular other than Linux, GTK isn't used at all. So this is a non-issue on Mac and Windows, GTK is never enabled.

When you do a GTK2 build, nothing from GTK3 is pulled in. GTK2 builds were the default for several years for Mozilla and it took them a long time to switch to GTK3.

The issue is, GTK2 is pulled in when you do a GTK3 build, primarily because not pulling it in breaks GTK2 NPAPI plugins. For that reason, Mozilla did not allow the codebase to be built without GTK2 for as long as NPAPI was supported upstream. They only removed the GTK2 requirement when they killed NPAPI entirely. We never got around to changing that, mostly because we all disagreed on how to approach this and it was easier to just leave it as-is.

I'm trying to find a middle ground here, a way to remove the GTK2 dependency while still leaving support for NPAPI plugins that don't depend on GTK2. But building without the GTK2 library at all shouldn't be the default, simply because it potentially breaks critical functionality and we have no idea what the implications are, which plugins if any would still work, etc. However, it should be possible for self-builders to choose.

[frostknight]

The issue is, GTK2 is pulled in when you do a GTK3 build, primarily because not pulling it in breaks GTK2 NPAPI plugins. For that reason, Mozilla did not allow the codebase to be built without GTK2 for as long as NPAPI was supported upstream. They only removed the GTK2 requirement when they killed NPAPI entirely. We never got around to changing that, mostly because we all disagreed on how to approach this and it was easier to just leave it as-is.

Maybe you guys should get around to changing that if its a huge issue. Although its not my choice, but its something to ponder for the future maybe?

I mean, is it possible to make npapi work without gtk2? I would be surprised if that's a no.

[athenian200]

Maybe you guys should get around to changing that if its a huge issue. Although its not my choice, but its something to ponder for the future maybe?

I mean, is it possible to make npapi work without gtk2? I would be surprised if that's a no.

I'm pretty confident it is possible, and in fact I believe some limited testing was done with similar changes, and it was found a lot of plugins still worked (though the person who said so wasn't very trustworthy and I cannot vouch for them). Mozilla just didn't care to try...

[Moonchild]

I mean, is it possible to make npapi work without gtk2? I would be surprised if that's a no.

The NPAPI itself does not rely on any particular set of widgets, but the problem is that plugins themselves are compiled to use GTK2! So the binary/compiled plugins are the issue here: they will not work if the NPAPI host does not support GTK2 as the plugins will make calls into the host to provide specific widgets and interfaces.
So the limiting factor here are the plugins themselves, not the API or some technical limitation of NPAPI. You can compile NPAPI plugins that use gtk3, instead, but obviously with how nobody but us still supports NPAPI plugins, this is all legacy stuff that will not see any further development from the plugin creators.

[micwoj92]

I have found 1 NPAPI plugin which works without gtk2, so other people can test it.
https://code.videolan.org/videolan/npapi-vlc version 3.0.3
Must be built with ./configure --without-gtk

It is not compatible with gtk3, so that's why build without gtk.
This unfortunately does not work in wayland (using xwayland).
Simple test page:

Code: Select all

<!DOCTYPE html>
<html>
<title>VLC Mozilla plugin test page</title>
<body>
<embed type="application/x-vlc-plugin"
       width="640"
       height="480"
       target="%path-to-your-video%"
       id="vlc" />
<script type="text/javascript">

npapi-vlc.png

[athenian200]

Linux is like one of the worst platforms to do binary plugins on... widget library updates break things. Glibc updates break things. Moving to Wayland breaks things. Getting Flash or Java plugins to work on Linux is quickly becoming almost as bad as trying to run that old SimCity 3000 CD from Loki Games on a modern Linux distro. :/

So, I tried the Java plugin. It still works absolutely fine on GTK3 builds with GTK2 enabled, but doesn't work with GTK2 disabled. So thus far I don't really have a good example of a plugin that works with this patch on Linux.

The only thing I've been able to demonstrate is that at least the browser doesn't crash when the Java and Flash plugins fail. It just complains about invalid casts from GdkX11 to GdkWayland, and spits console errors about random features not being supported in GTK3.

So, would it be fine to merge this feature, with the caveat that we don't enable it by default (for obvious reasons), and make it very clear that it effectively breaks Flash and Java on Linux, and probably only works with self-compiled plugins from now on? I figure if someone really wants to make an NPAPI plugin for Linux, they will figure it out, or tell us how to fix it if there really is more broken than just the plugins depending on GTK2.

EDIT: Actually, I tried using X11... with GTK2 disabled, the Java plugin plugin works on X11, but not on Wayland. With GTK2 enabled, Java works on Wayland as well. Bizarre, but worth noting. So cutting GTK2 support actually hurts Wayland compatibility... somehow. Flash doesn't work regardless. My impression is that the plugins that aren't failing on X11 but are failing on Wayland are probably falling back to Xlib or something when GTK2 fails. So there, at least now we know that it does kind of work with Xlib. Kind of a weird conclusion, but it proves that GTK2 is not needed for NPAPI.

[Moonchild]

Given all that, I'm really not sure if we want this? "Here's a build switch that will break your plugins without outright cutting plugin support"... How useful is that?

[athenian200]

Given all that, I'm really not sure if we want this? "Here's a build switch that will break your plugins without outright cutting plugin support"... How useful is that?

Understood. You have a good point. Honestly, I'd rather it stay as-is if touching this puts that on the table. I had forgotten about those discussions... if I'd remembered that was the conclusion reached, I probably would have been more reluctant to bring it up again...

I don't really want to create patch like that, but I probably should... sigh. It's easy to do, it's just I was really hoping to find a way to not have to go that far. I use NPAPI more than I use XUL, I'm not really thrilled that things are going towards it being dead on Unix just because one old toolkit is unmaintained.

EDIT: After thinking about it more, I really do need to go ahead and make an option to disable NPAPI support. It's the only real way to create an option to build without GTK2... anything else is just wishful thinking that maybe someone will create GTK3 NPAPI plugins someday. That patch won't be quick though... disabling NPAPI entirely is a lot more involved than what I did before. Not to say that it's difficult, just time-consuming.

[Moonchild]

Well, don't get me wrong, if offered i'd probably accept it if the work is already done, since it's entirely an "at your own risk and peril" build config option, and if gtk2 becomes a hard blocker to be able to build and run the platform on some flavours of Linux (including official binaries) which I could potentially see some purist *nix distros pushing for, then we'd need it anyway, if for nothing but to keep compatibility for the run-time environment, then we'd need this anyway.
Continuing that line of thought though, those kinds of distros would likely also be pushing hard for Wayland? In that case it'd be even more broken... Lots of things to think about for future use.

Honestly I really wish Linux would get its act together but there are too many captains on that ship and nobody wants to give up their position.

[athenian200]

Well, don't get me wrong, if offered i'd probably accept it if the work is already done, since it's entirely an "at your own risk and peril" build config option, and if gtk2 becomes a hard blocker to be able to build and run the platform on some flavours of Linux (including official binaries) which I could potentially see some purist *nix distros pushing for, then we'd need it anyway, if for nothing but to keep compatibility for the run-time environment, then we'd need this anyway.
Continuing that line of thought though, those kinds of distros would likely also be pushing hard for Wayland? In that case it'd be even more broken... Lots of things to think about for future use.

Honestly I really wish Linux would get its act together but there are too many captains on that ship and nobody wants to give up their position.

Yeah, exactly my thinking here. Wayland only is about to become a much more popular thing because Fedora is dropping X11 support. Which means GTK3 becomes the new minimum because it was the first version of GTK to offer Wayland support.

And as of right now, we don't actually have proper Wayland support, so any hypothetical future plugins wouldn't be able to take advantage of Wayland even if we kept them, because at the moment we don't actually run in Wayland natively, we run through XWayland. And not many people developing new plugins are going to want to target a weird XWayland GTK3 for running them in...

At least for the moment, X11 is likely to stick around longer on other Unix-based operating systems (I'm pretty confident Solaris/illumos will be on X11 for as long as they exist). But it looks like it is going to be going away pretty fast on most major Linux distros now that Fedora is dropping it.

But yeah, it does seem like the option of disabling NPAPI just to reduce the amount of "hooks" into a specific toolkit or display manager might be prudent to have, at least while doing initial implementation. There's nothing saying we can't try to revive NPAPI on Wayland at some point in the future, but for now what most of them are going to want is a way to disable GTK2, and since that will break plugins, the best way to do that is to have a --disable-npapi option rather than --disable-gtk2 option.

The problem with Linux is it's just chaos... I mean, if the thing I dislike most about Windows is that Microsoft changes things unnecessarily that I think don't need to be changed, Linux is far from a safe haven from that sort of thing. They almost do the Chromium thing where they kill off smaller projects with constant API churn and breaking backwards compatibility, but with the whole OS rather than just the web.

[dbsoft]

The problem with Linux is it's just chaos... I mean, if the thing I dislike most about Windows is that Microsoft changes things unnecessarily that I think don't need to be changed, Linux is far from a safe haven from that sort of thing. They almost do the Chromium thing where they kill off smaller projects with constant API churn and breaking backwards compatibility, but with the whole OS rather than just the web.

This is my complaint too, with projects in the same sphere like GTK as well. GTK is the hardest API to support for my library since I have to completely rewrite it every few years. I have GTK2, GTK3 and GTK4 modules because every major version they deprecate or remove a significant portion of the API. So I am forced to rewrite using the new widgets they add because the existing ones are going to disappear or having a boatload of deprecation warnings.

[athenian200]

So, I have an update... the good news is that I was able to get the codebase to build with NPAPI (mostly) disabled on both Windows and Linux. I even checked that it works with GTK2 and GTK3. So if you want to build with GTK2 and no NPAPI for some reason, you could. I figure if that work is actually going to be done, it might as well work with everything.

The bad news is... it required so many ifdefs that my changes will be hard to follow, it has not been tested on Mac, and the platform's behavior may change in unpredictable ways for people who build it this way. It does seem like we're between a rock and a hard place here. Either we take a patch that pretty much breaks extensions while claiming to just disable GTK2 (which I will admit is kinda hacky), or we take on a mammoth patch that touches every part of the codebase with ifdefs for an option we probably won't build with officially anyway. Well, I might actually ship Epyrus on Linux with it if it goes in, since it's already GTK3-only anyway, and I don't believe there are any NPAPI plugins that support Epyrus... but I would have to do research to make sure it doesn't hurt anything.

Even if we don't wind up merging this, I will go ahead and leave it available as a potential patch for anyone who wants it... since I already did the work before looking back and realizing how out of control it had gotten.

https://repo.palemoon.org/athenian200/U ... sable_work

Note that for some weird reason, the moz.build files look wonky... some of the lines don't have the amount of space I would expect and seem misaligned. I swear they look absolutely fine on my host system, but for some reason when I pushed my branch up, they look wrong in Gitea. I think it has something to do with my copy/pasting stuff from the original moz.build files, and them using tabs in some places. It doesn't seem to affect building in my testing, but it is something I would want to go back and fix before submitting a PR...

Off-topic:
Sigh... why did they all have to build their NPAPI plugins against an unreliable library like GTK2, that expires as fast as milk? Couldn't they have just drawn their own widgets or something? It's such a waste...

[Moonchild]

Hey, I'm fine with a big patch adding #ifdefs, even if it touches the whole code base. There's no problem with that and if it allows this to just switch off npapi when doing gtk3-only, then that would be the best outcome.
This kind of adding conditionals isn't too hard to review (or test, or bugfix) either. so by all means make it a PR.

[Moonchild]

Note that for some weird reason, the moz.build files look wonky...

it's the infamous TAB vs SPACES nonsense.

[athenian200]

It doesn't appear to be going well so far. It seems like dbsoft is hitting the roadblocks I hit when trying to make/test this and is having a lot of trouble getting past them...

And if someone as experienced as dbsoft can't figure out the trick to getting the build system to preprocess IPDL files in different ways when switching between build options (basically killing the objdir by any means necessary to get it to start from scratch), then it may not be reasonable to expect the average user to do anything other than assume it's broken and file an issue...

I do see now why I was warned a long time ago not to mess with conditional preprocessing of IPDL files. While you technically can do it, it's very hard for most people to get the build right when switching between build options, and so you'll get constant bug reports from developers who can't understand what the build system is doing or how to kick it back into working. And since this is a developer/self-builder facing option, that means people not being able to get it to work will likely be the norm. Still, I am really not satisfied with just settling for either leaving functionality baked in that may not always be desired unconditionally, or ripping it out unconditionally. That's frustratingly ham-fisted and seems to be what the developers of the codegen expected people would settle for.

Maybe if I could find a way to make it so that when this option is specified, it automatically deletes the entire objdir on every ./mach build?

The one piece of good news is that it appears to at least avoid breaking anything when --disable-npapi is not specified, even on Mac and FreeBSD... so at least no problems should be introduced for those who don't use the option, which I am guessing would be the majority.