Pale Moon forum

Posted: **2020-07-13, 04:15**

https://build.opensuse.org/package/show ... r/palemoon

It's not urgent right now, since the earlier successful Sid builds for 28.10.0 are still in its repository, but the issue will probably also migrate to Debian Testing, and might affect the next release. The build logs are available to anyone if you poke around the "failed" link, but do you want me to see if I can isolate the error messages?

bgstack was having the same problem on his Devuan unstable builds there the last time I looked...

Posted: **2020-07-13, 04:37**

This is from your x86-64 build, but I notice something similar in the others too:

Code: Select all

[  168s] ../../coreconf/rules.mk:288: *** multiple target patterns.  Stop.
[  168s] make[8]: Leaving directory '/usr/src/packages/BUILD/platform/security/nss/lib/nss'
[  168s] make[7]: *** [../coreconf/rules.mk:67: private_export] Error 2
[  168s] make[7]: Leaving directory '/usr/src/packages/BUILD/platform/security/nss/lib'
[  168s] make[6]: *** [Makefile:451: private_export-nss/lib] Error 2
[  168s] make[6]: Leaving directory '/usr/src/packages/BUILD/obj-x86_64-pc-linux-gnu/config/external/nss'
[  168s] make[5]: *** [/usr/src/packages/BUILD/platform/config/recurse.mk:79: config/external/nss/export] Error 2
[  168s] make[5]: *** Waiting for unfinished jobs....

For whatever reason, it seems something is suddenly wrong with rules.mk having "multiple target patterns." I'm not sure what's different... a new compiler version, a new version of GNU Make, etc? Do you think it was a change in Debian's toolchain that caused this, or a change in UXP's code? I wouldn't expect the code has changed, so that would leave a toolchain issue. I wouldn't know where to start looking for that since I'm not on Linux myself these days, but it seems like figuring out what caused this to fail is important to fixing the problem.

From this, it looks like the builds started failing on June 30th?

Posted: **2020-07-13, 10:54**

I don't see a specific error in the build log, just that it seems to fail somewhere in NSS.

This is just a shot in the dark, but I noticed that these warnings crop up right around where the build errors out:

Code: Select all

warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.

Posted: **2020-07-13, 11:42**

Sounds like make has changed breaking parallel building (and I'm pretty sure our tree in some places relies on the jobserver being available, but don't pin me on that)

Posted: **2020-07-13, 17:38**

For reference, make 4.3 was released earlier this January. Announcement and relnotes here, including some details on breaking changes and deprecations. Debian Testing is still on make 4.2.1.

I tried spinning up a Debian VM, switching my package repos over to unstable (sid), and building Pale Moon. Interestingly enough I haven't run into any errors during configuration, building, or packaging. What version of make is installed on that machine, Steven?

Posted: **2020-07-14, 06:13**

The OBS builds are done in the "cloud"--I'm not doing them locally. It will be whatever version of make is current in Sid. I checked, and Sid still has gcc-9 as the default compiler, so it's not gcc-10 causing the issue.

Maybe the jobserver messages are a red herring, since I think I see the same messages in the successful builds.

Re make: the log says:

Code: Select all

[   34s] [115/433] installing make-4.3-4

Hmmmm--Sid did an update of make on June 30, which is very suspicious, plus that jobserver change sure looks like it may have borked the builds now. 4.3 has been in Sid since late May, though:

Code: Select all

make-dfsg (4.3-4) unstable; urgency=high

  * Cherry picked [SV 58232] Disable inheritance of jobserver FDs for
    recursive make from ther development branch
  * Bug fix: "FTBFS: E: Build killed with signal TERM after 150 minutes of
    inactivity", thanks to Lucas Nussbaum (Closes: #963335).

 -- Manoj Srivastava <srivasta@debian.org>  Tue, 30 Jun 2020 00:58:36 -0700

I guess a workaround on the OBS would be a single threaded build.

Posted: **2020-07-14, 09:31**

stevepusser wrote: ↑
2020-07-14, 06:13
I guess a workaround on the OBS would be a single threaded build.

And hope it doesn't take too long? Build systems tend to have timeouts to prevent "hung" builds.

Posted: **2020-07-14, 19:47**

https://lwn.net/Articles/810384/ sounds relevant. For those who do not wish to navigate through the link, it is quoted here.

StefanBr on Jan 23, 2020 15:59 UTC wrote: > The jobserver protocol isn't hard to implement manually, even if not using make.

Unfortunately, the jobserver protocol is broken.

Job tokens are passed down via inherited file descriptors -

Caveat 1: If the called program is not $(MAKE) (verbatim), the file descriptors are closed on exec (FD_CLOEXEC), and the chain ends there. You can opt-in by adding a '+' in front of your command, but then you have to do that on every elegible command. Hint - none of the common Makefile generators does that.

Caveat 2: If you happen to call make indirectly (e.g. via a script called from the Makefile), unfortunately the '--jobserver-auth=<readfd>,<writefd>' will still be in the MAKEFLAGS env var although make had just closed the corresponding file descriptors. Two things can happen:

a) The fds no longer refer to valid file descriptors - make will detect this
b) The fds have been reused in the meantime - make will find valid file descriptors. If the fd is not readable (e.g. referring to a write end of a pipe), make will start busy looping. If the fd is readable, it will spawn a new job for every byte read (now figure what will happen if it points to a 2 kByte source file ...)

Also, other projects have run into similar-sounding errors. https://github.com/nodejs/node/issues/22457, where it was fixed by some reverting some Makefile changes in their code: https://github.com/nodejs/node/pull/22458/files. It is worth noting that nodejs had this issue before make 4.3 was released.

tl;dr: I'm no expert, but it looks like "$(MAKE)" might be needed instead of bare "make" invocations.

Posted: **2020-07-15, 03:34**

I remembered that I can probably fix it by just adding the former working 4.3-3 (sources at snapshot.debian.org) version to my repo for Sid with an epoch of "1:" at the start of the version that will make it higher than 4.3-4.

Edit: well, that port of the older make also fails to build on Sid.

Posted: **2020-07-15, 11:02**

As mentioned previously, I'm able to build with make 4.3-4 on a vanilla sid VM without an issue.

What branch are you building? I have successfully built master, but will try building release instead to see if I can reproduce your issue.

Edit: Also what is the mozconfig you're using to build?

Edit 2: I checked my build of release on Sid and it still builds correctly, so now I'm wondering if it is something in our .mozconfigs that is different.

Posted: **2020-07-16, 04:27**

Moonchild wrote: ↑
2020-07-14, 09:31

stevepusser wrote: ↑
2020-07-14, 06:13
I guess a workaround on the OBS would be a single threaded build.
And hope it doesn't take too long? Build systems tend to have timeouts to prevent "hung" builds.

Nah, I've had things like llvm and Chromium builds on the OBS take many hours, even with 8 threads, so PM is nothing compared to those. I just set up a single threaded build on Sid, and it seems to have got past the jobserver error.

Posted: **2020-07-16, 04:33**

Isengrim wrote: ↑
2020-07-15, 11:02
As mentioned previously, I'm able to build with make 4.3-4 on a vanilla sid VM without an issue.

What branch are you building? I have successfully built master, but will try building release instead to see if I can reproduce your issue.

Edit: Also what is the mozconfig you're using to build?

Edit 2: I checked my build of release on Sid and it still builds correctly, so now I'm wondering if it is something in our .mozconfigs that is different.

You can poke around the repo and download the PM debian tarball, which contains the mozconfig file:

https://download.opensuse.org/repositor ... _Unstable/

Single threaded PM builds seem to work fine in an experiment on the OBS--how many cores does your VM see? I tried to forwardport make 4.3-3 from snapshot.debian.org--that's the one that failed.

Posted: **2020-07-16, 12:10**

stevepusser wrote: ↑
2020-07-16, 04:33
You can poke around the repo and download the PM debian tarball, which contains the mozconfig file:

https://download.opensuse.org/repositor ... _Unstable/

Single threaded PM builds seem to work fine in an experiment on the OBS--how many cores does your VM see? I tried to forwardport make 4.3-3 from snapshot.debian.org--that's the one that failed.

My VM is currently configured to use 2 processors and 8 GB RAM. In my mozconfig I have MOZ_MAKE_FLAGS="-j3"; I can try fiddling with this and other options to see if I can break my build.

Posted: **2020-07-16, 17:18**

I tried building with the mozconfig you pointed to in the Debian tarball. (I also upped the number of processors for the VM to 4, just for fun.) The build still succeeded. I don't think the build system spits out a full log, so I tee'd it to a file - see attached.

Other than those jobserver warnings, I don't see anything suspect...

Posted: **2020-07-17, 03:10**

I can't really figure it out--sometimes the OBS has build problems that a VM on a local machine doesn't. I think they're using VMs on some kind of cluster of machines, but don't really know.

I can see it's not restricted to Pale Moon, either: https://build.opensuse.org/package/show ... kports/efl

Edit: and now it's working for EFL on Sid. PM builds are still failing.

Posted: **2020-07-18, 03:45**

28.11.0 builds are now in my repo. I added a _constraints file to my palemoon-sid single threaded build which seems to have restricted the pool of builders for it to faster machines, so the builds were done in 2 to four hours--not completely unreasonable. Packaging on my laptop with sbuild took about 26 minutes, though a good part of that is having sbuild set up the build environment, including downloading and installing the many build-dependencies automatically, then cleaning up after itself after the build.

Posted: **2020-07-26, 03:41**

I did come up with a little code for the debian/rules to use a detect a build on Sid and use a mozconfig that enforces a single build there, but will wait until the next PM update to avoid forcing another upgrade on users that won't make any difference to them. I suppose a fancy way to do that would be use sed to change just one mozconfig file on the fly...

Code: Select all

# Use a custom mozconfig for Sid to build with one thread.
# Builds on Debian derivatives may have to add their own release name.
distrelease := $(shell lsb_release -cs)
ifeq ($(distrelease),$(filter $(distrelease),sid unstable))
override_dh_auto_configure:
	cp debian/mozconfig-sid debian/mozconfig
endif

override_dh_auto_build:
	cp debian/mozconfig mozconfig
	make -f client.mk build

Posted: **2020-07-26, 14:24**

Instead of copying you can simply "export MOZCONFIG=/path/to/mozconfig" and it will pick it up. i use that all the time for multiple builds from one tree, even doing them parallel.

Posted: **2020-07-27, 03:27**

That's cool--so for a Sid build I just have it use the debian/mozconfig-sid file? That would be simpler.

Pale Moon forum

Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid

Re: Hmmm--Pale Moon 28.10.0 is now failing to build on Debian Sid