I Made A Mistake

[Basilisk-Dev]

As mentioned in this topic, I transitioned the Basilisk release builds from being built in CentOS 7 to Slackware 14.2.

I learned yesterday that Slackware 14.2 EOL is 2024-01-01 from this LinuxQuestions.org post.

Seems I'll have to research a new x86 environment for Linux builds again. I guess I learned my lesson there.

I have a few questions:
1 - Does anyone have any suggestions on LTS Linux operating systems that I could use? I'd like to use an older version to retain older Glibc compatibility. I'd like to have the ability to run GCC 11 if possible, but GCC 9 is fine too.
2 - What glibc and libstdc++ versions does your currently used desktop distribution use?
3 - What are your thoughts on continuing to build on an environment that no longer receives security updates?

I'd like to hear your thoughts especially on the last point, I don't personally think it's a security risk if you are using a binary that was built on an older distribution as long as your current distribution still gets security updates. That would be like saying that it's a security risk to run a binary compiled for Windows XP on Windows 11 solely because it was compiled on an old compiler that targeted Windows XP.

[Admin]

While I don't use Linux as a daily driver, myself, I do think for point 3 that it would be just fine to use an older version of Linux as build environment, if you keep it dedicated to that task and don't do anything risky or have it actively exposed to inbound traffic. In this case, I think compatibility of the resulting binaries is more important than the environment itself.

Anyway, just my 2¢ on the matter.

[athenian200]

I have a few questions:
1 - Does anyone have any suggestions on LTS Linux operating systems that I could use? I'd like to use an older version to retain older Glibc compatibility. I'd like to have the ability to run GCC 11 if possible, but GCC 9 is fine too.
2 - What glibc and libstdc++ versions does your currently used desktop distribution use?
3 - What are your thoughts on continuing to build on an environment that no longer receives security updates?

Are you planning to use it as a daily driver? Or just as a compilation environment?

Currently I use Oracle Linux 7 and GCC 7 to compile Epyrus. Mostly it's based on hearing that was the environment Travis used for Pale Moon at one point, except I used Oracle Linux in place of CentOS because they're both RHEL clones, and I have an upgrade path to Oracle Linux 8. I think I picked it out for testing GRE stuff right in the middle of the chaos going on with CentOS 8, and just wound up repurposing it for Epyrus builds. I don't think Rocky Linux existed at the time, and AlmaLinux was too new for me to trust it. The glibc on offer is a version of 2.17 that still gets a few critical security updates. There's also a compat-glibc of 2.12 you can link against if you need to compile binaries that will run on RHEL 6, but I don't use that.

As for libstdc++, one of the major reasons we use RHEL-based distros is because they allow you to use a newer GCC version via devtoolset without introducing dependencies on a newer libstdc++ than what came with RHEL. In other distros, you generally link against the libstdc++ that shipped with whatever version of GCC you use. So if you upgrade GCC on most other distros, the resulting binary breaks every Linux distro that doesn't have a new enough libstdc++.

If you inspect our binaries with readelf or something, you should see that none of them have a dependency on anything beyond GLIBCXX_3.4.19, which means it will run on any system that ships with the GCC 4 libstdc++ libraries or later. While actually being compiled with a devtoolset that corresponds to GCC 7 or 9, which would normally require the user to have libstdc++ with GLIBCXX_3.4.24 or newer.

Now, I actually do have a Linux environment I run on one of my older PCs that won't run OpenIndiana, and I didn't want to buy a Windows 10 license for it anyway. The Linux I play around with for personal use on desktop is openSUSE Leap. It ships with glibc 2.31 and still uses GCC 7 as the system compiler, but has an option to install GCC 10 or 11 as an additional compiler. It feels surprisingly similar to Magia/Mandriva (my old standby back when I used Linux a lot more) in terms of being very user-friendly in an old-school desktop way, but a bit more polished because the company behind it is still around. So in other words, just because I would pick a RHEL fork for compiling Epyrus doesn't mean I would actually want to run such an environment as a desktop.

I would say as far as LTS Linux operating systems, the main choices you have are SUSE Enterprise Linux, Debian, Ubuntu, and RHEL along with its forks. And those RHEL forks have special characteristics that seem to make them uniquely suited to using newer GCC without introducing dependencies on newer libstdc++. I have no idea if any other distros do that or not.

If you're just running the Linux distro in a VM on top of whatever OS you run as a daily driver (even another Linux version), then it's not a big deal. But if you are running it as a daily driver, using a RHEL-fork that only gets security updates (or maybe none at all) could become a bit painful. The cool thing about Linux, though, is you don't even need a full VM to use a different Linux distro as a compilation environment. I think you could get away with just a Docker container if it's setup properly.

EDIT: If you do wind up using a RHEL-clone, it probably needs to be at least version Oracle Linux 8/RHEL 8/Rocky 8 going forwards... we are moving to C++17, and while nothing seems to require the ABI yet (probably by pure luck), this will eventually break RHEL 7 and require a newer libstdc++ version, in a way that devtoolset cannot work around. Actually, a little research has turned up that RHEL 7's devtoolkit-7 doesn't even fully support the C++11 ABI, so it's amazing we haven't had more problems on Linux. Maybe we've been getting away with just API support for newer C++ while not needing a newer ABI at all.

[andyprough]

There's a page that keeps track of the LTS dates for the different distros: https://www.geekersdigest.com/updated-l ... pport-lts/

With Debian having free ELTS support you have a lot of options - everything from Debian 8 Jessie until 2025 to the current Debian 12 Bookworm until probably 2032. You could mix and match your needs for different packages and different lengths of support.

[Basilisk-Dev]

Are you planning to use it as a daily driver? Or just as a compilation environment?

Just for compilation.

I don't think Rocky Linux existed at the time, and AlmaLinux was too new for me to trust it. The glibc on offer is a version of 2.17 that still gets a few critical security updates. There's also a compat-glibc of 2.12 you can link against if you need to compile binaries that will run on RHEL 6, but I don't use that.

Good to know, I'll look into compat-glibc. I used to use Oracle Linux 8 as a desktop system as I prefer LTS distros to bleeding edge distros if I am going to use Linux.

I was using CentOS 7 as the build environment before I switched it over to Slackware.

Off-topic:
To be honest at this point I've been so disappointed to modern Linux distributions breaking things through updates that I've considered switching to Windows for my desktop OS. Windows is inconsistent from a GUI standpoint but it can run every program I want to run, even some older programs that are almost 30 years old that were originally intended to be ran on Windows 95/98/NT4.

As for libstdc++, one of the major reasons we use RHEL-based distros is because they allow you to use a newer GCC version via devtoolset without introducing dependencies on a newer libstdc++ than what came with RHEL.

I actually didn't know that. That sounds ideal.

The cool thing about Linux, though, is you don't even need a full VM to use a different Linux distro as a compilation environment. I think you could get away with just a Docker container if it's setup properly.

I am already using Docker for the Basilisk Linux builds actually.

https://repo.palemoon.org/Basilisk-Dev/ ... asilisk.sh

EDIT: If you do wind up using a RHEL-clone, it probably needs to be at least version Oracle Linux 8/RHEL 8/Rocky 8 going forwards... we are moving to C++17, and while nothing seems to require the ABI yet (probably by pure luck), this will eventually break RHEL 7 and require a newer libstdc++ version, in a way that devtoolset cannot work around.

Good to know. I think switching to a newer distribution like CentOS 8 might break some backwards compatibility for some users, but at some point you have to move forward. I can't cater to users intentionally using outdated software forever.

With Debian having free ELTS support you have a lot of options - everything from Debian 8 Jessie until 2025 to the current Debian 12 Bookworm until probably 2032. You could mix and match your needs for different packages and different lengths of support.

The last time I used Ubuntu or Debian was around Debian 6/7 and Ubuntu 14.04. On both operating systems, upgrades completely broke the entire systems to the point that they wouldn't even boot. I also frequently ran into weird apt-get issues. It put such a bad taste in my mouth that I made the decision to never use deb/dpkg based distros again.

That was almost a decade ago, so it's possible that things may have improved now. I might give Debian a shot again at some point if I am not happy with Oracle Linux. Thanks for the suggestion.

[athenian200]

Good to know, I'll look into compat-glibc. I used to use Oracle Linux 8 as a desktop system as I prefer LTS distros to bleeding edge distros if I am going to use Linux.

I was using CentOS 7 as the build environment before I switched it over to Slackware.

Oracle Linux 8 is actually one of the candidates for our next build environment. I believe MC was testing it out? Don't know if Travis is running anything RHEL-8 based yet or not, I haven't talked to him in a while. I know he's using GCC 9 now, and I've already moved to GCC 10 on the SunOS builds of Pale Moon. I'm still using GCC 7 for the time being on Epyrus for Linux simply because I figure I will blow the entire thing away in favor of Oracle Linux 8 when it's time to switch compilers and GCC 7 stops working or becomes unreliable.

Yeah, Linux tends to break things too often if you're not running an LTS distro, though the LTS distros often lack a lot of creature comforts, like choice of software, etc. Honestly, the only thing stopping me from running something like Oracle Linux 8 as a desktop system is that I don't care for GNOME, and I don't think they really support anything else.

Off-topic:
To be honest at this point I've been so disappointed to modern Linux distributions breaking things through updates that I've considered switching to Windows for my desktop OS. Windows is inconsistent from a GUI standpoint but it can run every program I want to run, even some older programs that are almost 30 years old that were originally intended to be ran on Windows 95/98/NT4.

Off-topic:
That's exactly why it's so hard for me to move away from Windows. At most it seems like I can only ever play around with other operating systems on spare computers, but always need at least a few Windows machines to get real work done.

I am already using Docker for the Basilisk Linux builds actually.

https://repo.palemoon.org/Basilisk-Dev/ ... asilisk.sh

Yeah, and if you're already familiar with Oracle Linux 8, setting that up should work fine.

Good to know. I think switching to a newer distribution like CentOS 8 might break some backwards compatibility for some users, but at some point you have to move forward. I can't cater to users intentionally using outdated software forever.

Yeah, generally I find that isn't a problem with 64-bit Linux users who are using GTK3... but the ones that are stubbornly sticking to 32-bit... those are the ones that are most likely also running a very old glibc or something and will complain if you move to building on newer stuff. That's one reason why I only bother building a 64-bit GTK3 package for Linux, because I know sooner or later the 32-bit people will be mad at me if I start supporting them and then suddenly need a version of a package that their system can't run. I figure it's better to take the fallout of refusing to support them to start with, than to offer it and take it away.

The last time I used Ubuntu or Debian was around Debian 6/7 and Ubuntu 14.04. On both operating systems, upgrades completely broke the entire systems to the point that they wouldn't even boot. I also frequently ran into weird apt-get issues. It put such a bad taste in my mouth that I made the decision to never use deb/dpkg based distros again.

That's my experience with Debian and Debian-based distros like Ubuntu as well. They work great out of the box, but if you try to upgrade them, you'll brick your system. RPM-based systems are a little better, but it really seems with Linux in general, you have to be prepared to lose all your data when you update the system and be prepared to reinstall. Linux does give you a choice about upgrading, sure, but upgrades are also more likely to cause problems than they are on Windows. And the more you customize your system, the more likely it is something will break when you upgrade.

[trava90]

Oracle Linux 8 is actually one of the candidates for our next build environment. I believe MC was testing it out? Don't know if Travis is running anything RHEL-8 based yet or not, I haven't talked to him in a while. I know he's using GCC 9 now, and I've already moved to GCC 10 on the SunOS builds of Pale Moon.

I'm currently still using CentOS-7 for release builds, though I do intend to setup an Oracle Linux 8 build environment soon. Will likely update to a newer compiler (exact version TBD) at the same time, as I've been running personal builds of both Pale Moon and Epyrus built with GCC 11, 12, and 13 for many months now with no obvious downsides.

[mrnhmath]

I've started experimenting with EL8 a while ago and I haven't hit a single issue with the base GCC 8 toolchain. I know DPMO says it's deprecated but it's been fine for my personal builds.

[Drugwash]

Can't say anything about LTS Linux distros, never was much interested in that aspect.

About glibc - can't find it in Synaptic by that name, just some docs package that mentions v2.27-3ubuntu1.6. However found some info on the web, ran ldd --version | head -n1 which yielded ldd (Ubuntu GLIBC 2.27-3ubuntu1.6) 2.27 so I guess it's safe to say that 2.27 is the version on this system.

About libstdc++ - there are a couple versions installed on my Mint 19.2 x64 (based on Ubuntu 18.04). Not sure which is the default one:
- libstdc++5 v1:3.3.6-30
- libstdc++6 v8.4.0-1ubuntu1~18.04

About building on an environment that no longer receives security updates - personally I don't care much about this overhyped security but to each their own. As long as the building environment doesn't directly reflect in the product itself I'd say it's OK. I'd rather have a tool to work with, and when necessary other means of protection, than no tool at all.

That said, I had just updated to latest version (2023.12.09) earlier and upon restart the browser wouldn't run. Started from terminal I got the error:

XPCOMGlueLoad error for file [...]/basilisk/libxul.so:
/usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by [...]/basilisk/libxul.so)
Couldn't load XPCOM.

Downloaded older versions (all of them GTK3) from the archive site which is extremely slow. Tried:
2023.11.05 - same error
2023.10.03 - same error
2023.09.15 - same error
2023.09.12 - same error
2023.07.18 - success! (this was my previous version before the update)

For completeness sake I also tried the first available GTK2 version (2023.09.15) which yielded the same error. Safe to assume all further versions would do the same.

That's about it.

[athenian200]

That said, I had just updated to latest version (2023.12.09) earlier and upon restart the browser wouldn't run. Started from terminal I got the error:

XPCOMGlueLoad error for file [...]/basilisk/libxul.so:
/usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by [...]/basilisk/libxul.so)
Couldn't load XPCOM.

You would need the libstdc++ from GCC 9.1.0 or later to run those Basilisk binaries. That means whatever distro was used to compile them, the linker trick I described above simply didn't work, and things seem to have regressed back to the whole "You have to have the exact same libstdc++ as what it was compiled with" that we were trying very hard to avoid by sticking with CentOS 7 on Pale Moon. I should say, though, that supporting Linux as a developer is a really sucky experience, between people expecting multiple GTK versions and using a variety of libc versions, so try and cut Basilisk-Dev a little slack.

[Drugwash]

You would need the libstdc++ from GCC 9.1.0 or later to run those Basilisk binaries. That means whatever distro was used to compile them, the linker trick I described above simply didn't work, and things seem to have regressed back to the whole "You have to have the exact same libstdc++ as what it was compiled with" that we were trying very hard to avoid by sticking with CentOS 7 on Pale Moon. I should say, though, that supporting Linux as a developer is a really sucky experience, between people expecting multiple GTK versions and using a variety of libc versions, so try and cut Basilisk-Dev a little slack.

Unfortunately no GCC higher than 8 seems to be available for my particular platform, and that only pertains to the dev environment, not runtime. So that's that.

I do understand the whole mess that Linux is. Didn't intend to take it on the Basilisk dev, just stated my findings for the record in case anything was relevant.
Thank you for the info.

[athenian200]

Unfortunately no GCC higher than 8 seems to be available for my particular platform, and that only pertains to the dev environment, not runtime. So that's that.

Out of curiousity, which platform is it? I bet you I can find some files from a newer GCC that would run on there with a bit of experimentation. Not ideal, but I have a lot of experience messing with everything from missing DLLs in Windows to this kind of stuff on Linux.

[Moonchild]

I should say, though, that supporting Linux as a developer is a really sucky experience, between people expecting multiple GTK versions and using a variety of libc versions, so try and cut Basilisk-Dev a little slack.

My Sig:

"Linux makes everything difficult."

[Drugwash]

It's Mint 19.2 x64 Cinnamon (based on Ubuntu 18.04/Bionic) as mentioned somewhere above. Already EOL.

Back on Windows I did my share of experimenting with upgraded libraries - running 98SE up to 2016-2017 kinda required jumping through lots of hoops - but in Linux I try to be a bit more... conservative. Not to say I didn't stretch the cord a couple times when I really really wanted to get some code compiled or some application run, but considering a previous bad experience with an earlier Mint that got FUBAR due to too many KDE libraries in a Cinnamon environment - don't do this at home, children! - I am trying my best to refrain from screwing up this setup too, because I have no backup whatsoever (no free space) and as such wouldn't be able to get it back as it was. Might as well hang myself on the tallest tree in the garden if that happened (knock on wood ).

So thank you for your offer, really kind of you, but for now guess I'll just leave well enough be. Besides, Basilisk is not my main - or only - browser so there's no emergency on the horizon.

My Sig:

Lyceus Anubite wrote:
"Linux makes everything difficult."

Right on spot. But the alternative(s)... Ah... Shudder. No thank you siree.