New FreeBSD port

[OlCe1]

PR was OK for a system including llvm 8. This error occurs on a shiny new one, with llvm 10.

[New Tobin Paradigm]

Don't use clang. You been told.

[Moonchild]

PR was OK for a system including llvm 8. This error occurs on a shiny new one, with llvm 10.

Why are you building with clang and not gcc like we assumed you were from previous conversation?

[OlCe1]

I'm *not* building with LLVM/Clang. But I'm building *against* libc++ headers from LLVM/Clang. The easy to correct error was from a bad interaction between these headers and mozalloc.h.

There is another error in jemalloc with very recent FreeBSD, due to RTLD_DEEPBIND being available. I just de-activated the check to pass this one. Did not find anything in the tree that used RTLD_DEEPBIND, except some code in WebRTC (did not check yet if this code is really applicable on FreeBSD).

Now testing. Didn't have much time this week.

[Moonchild]

I'm *not* building with LLVM/Clang. But I'm building *against* libc++ headers from LLVM/Clang.

Ahhh okay. Excuse my lack of knowledge of which lib belongs to what. I'm primarily a Windows person.

I just de-activated the check to pass this one. Did not find anything in the tree that used RTLD_DEEPBIND, except some code in WebRTC (did not check yet if this code is really applicable on FreeBSD).

WebRTC is third-party code that isn't included in Pale Moon releases so you don't have to concern yourself with it (unless you plan on supporting applications building on UXP that include WebRTC on FreeBSD, but that shouldn't be a concern right now anyway).

[SlySven]

Thank you very much OiCe1 for your work on this. I triple boot FreeBSD|GNU / Linux|Windows 10 and the loss of PM from FreeBSD a while ago really messed with my browsing on that OS - not being willing to run the proper NotChrome I had to fall back to "Cliqz" which was a partially gelded NotChrome that phoned home a bit less but had the "Ghostery" add-on welded on - trouble is that had now gone the way of the dinosaur vis-a-vis large lump of rock/ice around 65M years ago.

I work on a FOSS project that primary targets the Big 3 OSes but which I have gotten compiled on FreeBSD - but it ain't anywhere near going into /usr/ports yet. I've had the opposite compiler problem in that I can get Clang/LLVM 10 (and in the past the older ones) to build my project but getting anything runable with any GCC has so for proven impossible - at best I can get it to compile and link cleanly but it just crashed and burns when run. Anyhow it seems you probably know more about that sort of thing than I do. For the record, in my experience, CLang/LLVM is the standard compiler on FreeBSD - like on MacOS and using GCC there does seem to be harder.

May i ask: is the ability to use the Sync feature only allowed for "Official Branded" builds? I cannot recall whether the old version that FreeBSD had in its "Ports" was an official Pale Moon or an unofficial New Moon build (I am beginning to think it was the latter) and I cannot recall whether that was synced to the ones on my other OS on the same PC and on other machines on my LAN.

[OlCe1]

Hi,

I've finally gotten to testing, and, with the proposed small fixes, everything builds and works flawlessly.

Here is the new PR, corresponding to 2 separate small issues (one commit for each).

Could you please review it?

Regards.

[OlCe1]

Thank you very much OiCe1 for your work on this.

Thanks. Took quite a bit of time.

Anyhow it seems you probably know more about that sort of thing than I do. For the record, in my experience, CLang/LLVM is the standard compiler on FreeBSD - like on MacOS and using GCC there does seem to be harder.

I'm quite experienced with that kind of things yes. I maintained for years FreeBSD systems where most ports were built with GCC and linked against libstdc++, after the switch to clang/LLVM, so I know first-hand how nightmarish things can become. But compatibility either way (building with GCC and linking against libc++, or building with clang/LLVM and linking against libstdc++) is now good (the former works, and the latter also with a recent libstdc++).

You're right in theory, which should usually translate into some high probability. But for this particular project, this doesn't seem to be the case. Months ago, I built everything with clang 8 and I got almost immediate crashes. Now, we have newer versions, and I have more standard setups, so it could work. But I prefer to use GCC (9), because I've not have any problem with it, and upstream uses GCC on non-Windows platforms.

May i ask: is the ability to use the Sync feature only allowed for "Official Branded" builds? I cannot recall whether the old version that FreeBSD had in its "Ports" was an official Pale Moon or an unofficial New Moon build (I am beginning to think it was the latter) and I cannot recall whether that was synced to the ones on my other OS on the same PC and on other machines on my LAN.

This I don't know. But hopefully, we may get an official build approval, if things go well enough. Let's see.

[OlCe1]

But otherwise, yes:

Let us see your mozconfig.

So here it is again a bit updated:

Code: Select all

_BUILD_64=1
mk_add_options MOZ_OBJDIR=/usr/ports/www/palemoon/work/pale-moon/pmbuild

ac_add_options --enable-application=palemoon
ac_add_options --enable-optimize="-w -O2"
ac_add_options --enable-default-toolkit=cairo-gtk3
ac_add_options --enable-jemalloc
ac_add_options --enable-strip
ac_add_options --enable-devtools
ac_add_options --disable-eme
ac_add_options --disable-webrtc
ac_add_options --disable-gamepad
ac_add_options --disable-tests
ac_add_options --disable-debug
ac_add_options --disable-necko-wifi
ac_add_options --disable-updater
ac_add_options --with-pthreads
ac_add_options --enable-phoenix-extensions

ac_add_options --enable-official-branding
export MOZILLA_OFFICIAL=1

# Below this line, build options depending on port options, as well as make and
# build flags, will be appended.
ac_add_options --enable-alsa
ac_add_options --disable-pulseaudio
export CPP="cpp9"
export CC="gcc9"
export CXX="g++9"
export CFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing"
export CXXFLAGS="-O2 -pipe -fstack-protector-strong -nostdinc++ -I/usr/include/c++/v1 -nodefaultlibs -lc++ -lcxxrt -lm -lc -lgcc -pthread -lpthread"
export LDFLAGS="-lgcc -fstack-protector-strong -L/usr/local/lib/gcc9"
export LD="/usr/local/bin/ld"
export AS="/usr/local/bin/as"
export AR="/usr/local/bin/ar"
export RANLIB="/usr/local/bin/ranlib"
export OBJDUMP="/usr/local/bin/objdump"
export NM="/usr/local/bin/nm"

The exports at the end are plumbery to compile with GCC 9 while linking against libc++ headers, except for

Code: Select all

-fstack-protector-strong

, which is just standard FreeBSD's policy.

Is it ok as it is?

[Moonchild]

Looks like the mozconfig as-stated is fine. A few unnecessary options there that are defaults (less is more) but they won't hurt so w/e.

The exports at the end are plumbery to compile with GCC 9 while linking against libc++ headers, except for

-fstack-protector-strong

, which is just standard FreeBSD's policy.

That shouldn't be a problem. It's a mature compiler option of gcc to trade off some performance to add redundant stack checking (which isn't needed because we have our own stack mechanisms that go way beyond that); to avoid any lengthy and unnecessary arguments over which is better/preferred, it's fine to double up on it.
FYI: we explicitly disable these mechanisms at the compiler level on Windows because the sanity checking in-tree makes them unnecessary, but if FreeBSD policy states this must always be used then go right ahead. You'll probably lose about 5% of performance overall as a result, maybe a little more.

[OlCe1]

Looks like the mozconfig as-stated is fine. A few unnecessary options there that are defaults (less is more) but they won't hurt so w/e.

I just took exactly the options listed on the Linux build instructions page, and did no further check on which ones are the default. If some options are not necessary now, or their presence deceptive on whether there is really some choice to be made, I think it would be best to update this page directly, so that everyone gets the info.

That shouldn't be a problem. It's a mature compiler option of gcc to trade off some performance to add redundant stack checking (which isn't needed because we have our own stack mechanisms that go way beyond that); to avoid any lengthy and unnecessary arguments over which is better/preferred, it's fine to double up on it.
FYI: we explicitly disable these mechanisms at the compiler level on Windows because the sanity checking in-tree makes them unnecessary, but if FreeBSD policy states this must always be used then go right ahead. You'll probably lose about 5% of performance overall as a result, maybe a little more.

Exactly, if this is ever a problem, I can raise the argument then. Current general policy is that SSP can be disabled by the maintainer only if enabling it breaks the port.

Going to push the port later this day (then, it will have to be reviewed and committed by someone else).

[Moonchild]

If some options are not necessary now, or their presence deceptive on whether there is really some choice to be made, I think it would be best to update this page directly, so that everyone gets the info.

Like I said being explicit can't hurt, so it doesn't matter. And no, all of these are real choices - kind of silly to suggest otherwise.

[OlCe1]

Hi, some (currently bad) news.

Port was finally integrated on Monday... to be withdrawn only 4 hours later. Some guy suddenly woke up and said Tauthon was violating Python 2.7 interpreter's expiration date, two months after it got into the ports tree. This has triggered some discussion among the ports tree managers, with still no public outcome.

I'm trying to find a way that could be acceptable to them. So this is again ongoing...

[Moonchild]

There's no solution for using anything but python 2.7 to build our tree, period. So if they decide it can't be in portage and subsequently blocking a required dep for UXP, then I really think they should re-think their approach to what packages they allow. I understand they don't want old packages exposed to the user, but they do need to allow them to build other packages, IMO.

I wish you the best of luck sorting that out.

[New Tobin Paradigm]

So Tauthon IS viable for building UXP? If so that is actually of GREAT BENEFIT. Not so much for BSD Ports that is bullshit.. But yeah.

[OlCe1]

So Tauthon IS viable for building UXP? If so that is actually of GREAT BENEFIT. Not so much for BSD Ports that is bullshit.. But yeah.

Yeah, I said this a while ago.

It is viable... with some patches though. I made them for the port, I can open a PR here when I have more time if you want. Current patches are "rough" in the sense that they will break using regular Python 2.7. Should not take much work to change that if wanted.

[New Tobin Paradigm]

It would be best if both work. It is extremely difficult to properly build Python on Windows vs being trivial on *nix. So I would greatly appreciate it.

[SlySven]

There's no solution for using anything but python 2.7 to build our tree, period. So if they decide it can't be in portage and subsequently blocking a required dep for UXP, then I really think they should re-think their approach to what packages they allow. I understand they don't want old packages exposed to the user, but they do need to allow them to build other packages, IMO.

The trouble is that Python2 is a year past End-Of-Life (see https://www.python.org/downloads/) and any active OS platform ishas migrating away from it - so it is going to be come increasingly hard to rely on it in platform's offerings as they will rightly spot that there are not going to be any security fixes and forbid its inclusion in packages when the successor is already widely available. Whilst there may be a residue Python27 in the FreeBSD /usr/ports tree I can see it is possible not unreasonable to say that that will block the addition of a NEW port - whether there can be any wiggle room around the fact that it is resurrecting a previous port that was deleted because it was out-of-date and this version is now up to date remains to be seen...

Port was finally integrated on Monday... to be withdrawn only 4 hours later. Some guy suddenly woke up and said Tauthon was violating Python 2.7 interpreter's expiration date, two months after it got into the ports tree. This has triggered some discussion among the ports tree managers, with still no public outcome.

I must have gotten in during that 4 hour Window - after a 3 hour build I am now using it from FreeBSD (12.2-RELEASE-p3) - what was really apprieciated is that it picked up from my prevous PaleMoon-28 install and synced up to my other installs on both the same machine in other Oses and other PCs.

Only issue I have found is that enabling smell-checking is red-lining every word I type - fair enough I do sezs da proper Queen's English and not wot those Yanks use the other side of the Pond. FTR I do have the language dictionary installed (and set to my default) and most of the infrastructure does seem to be working as I can "Add word to dictionary" and that removes that mispelling mark for that word everywhere as expected (I've been involved in adding Huspell based support for the FOSS project I code for so have some idea of what goes on under the surface for this.)

[Moonchild]

We are fully aware that several tools in our toolchain are (very) old. The problem is only that there is no distinction between build toolchains and runtime toolsets on various distros, meaning that it becomes impossible to build otherwise up-to-date applications just because a tool in the build toolchain is considered undesirable for run-time use.
I'm not suggesting anyone keep obsolete packages available for daily use. But it does cause a problem if they also can't be used in specific build environments that need them. Distro maintainer decisions to flat-out block old versions from even being available is therefore an architectural/administrative problem.

This is especially a problem for complex build systems that can't easily be translated to later versions of tools; and expecting us to throw away a perfectly good working system based on an arbitrary availability decision is a hard sell.

[SlySven]

Ah, I wasn't previously aware that Tauthon is a sort of workaround to get Python 2 code to build in a Python 3 environment. That being the case though, doesn't it has to be a temporary stepping stone whilst the existing Python 2 code is refactored/revised to be Python 3 code. I've virtually no experience of Python but the closest parallel in my experience has been migrating a project from Qt4 to Qt5 libraries - and that was not much fun either since the Qt people never really gave people a stable target to aim at!

This is especially a problem for complex build systems that can't easily be translated to later versions of tools; and expecting us to throw away a perfectly good working system based on an arbitrary availability decision is a hard sell.

But it isn't an arbitary availablity decision though is it? Reading https://snarky.ca/why-python-3-exists makes me realise that there were fundamental issues with how version 2 handled string data which does't work so well (since, I am guessing, the invention of Unicode?) It looks like a nettle that the PM developers are going to have to grasp and it seems that PM is going to need to have overhaul that tool chain - or get bogged down in accumulating technical debt. I haven't the foggiest idea how much work is involved in updating your tool-chain but please can you reassure us that it is in the TODO list somewhere?