A linux build framework for Pale Moon

Users and developers helping users with generic and technical Pale Moon issues on all operating systems.

Moderator: trava90

Forum rules
This board is for technical/general usage questions and troubleshooting for the Pale Moon browser only.
Technical issues and questions not related to the Pale Moon browser should be posted in other boards!
Please keep off-topic and general discussion out of this board, thank you!
Unix Ronin

Re: A linux build framework for Pale Moon

Unread post by Unix Ronin » 2016-12-22, 19:28

The thing that's really puzzling me is that up to 26.5.0, it was building perfectly. Then bump to 27.0.0, and suddenly it's blowing up before it even gets off the taxiway. I'm really curious what the key change was.

Unix Ronin

Re: A linux build framework for Pale Moon

Unread post by Unix Ronin » 2017-01-13, 21:37

I believe I've identified the root cause of the problem. The short answer seems to be that in palemoon 27.x, mach is starting out by trying to parse my entire shell environment, and its parser is too crude to accomplish the task and is failing. WHY it is trying to parse my entire environment is another question.

...And the next step in the answer is that while mach is parsing my environment, it comes across a declaration inside one of my shell functions of a variable named subdir, and for some reason thinks that everything from that part on is a multiline declaration of that variable. Still working on figuring out how to fix that.

I temporarily worked around that by removing the function from my environment that it's seizing up on, and it APPEARS to be building.

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-01-14, 05:38

Google for the combined terms (no quotes)...

Code: Select all

bash environment shellshock 2014
e.g. http://arstechnica.com/security/2014/09 ... nix-in-it/ and https://www.symantec.com/connect/blogs/ ... nerability and a lot more.
There's a right way
There's a wrong way
And then there's my way

New Tobin Paradigm

Re: A linux build framework for Pale Moon

Unread post by New Tobin Paradigm » 2017-01-14, 15:11

you can just not use mach.. You can still do it the old way by just using make -f client.mk. I can't recall how to do packaging directly from make but it will come to me. However, it is recommended you do use mach and supported environments.

Unix Ronin

Re: A linux build framework for Pale Moon

Unread post by Unix Ronin » 2017-01-14, 19:25

I filed a bug against mach already, and I have a temporary workaround. With the workaround, my ebuild works using mach. Mach really should be fixed with a smarter parser, but in the short term I can refactor the shell function that it barfed on.

Now i need to figure out which extensions that were working on 26.5.0 PaleMoon 27 is choking on.

Unix Ronin

Re: A linux build framework for Pale Moon

Unread post by Unix Ronin » 2017-01-15, 03:29

Turns out there were about three extensions I have in 26.5.0 that are hard-incompatible with 27.0, and about eight others that claim to have no version that supports it. Amusingly, one of the hard-incompatible ones is mozilla's add-on compatibility reporter. Bluhell Firewall, DNS Flusher, ExExceptions, Weekly Browsing Schedule, and ShareMeNot are among the other 27.x incompatibilities.

fillerup

Re: A linux build framework for Pale Moon

Unread post by fillerup » 2017-01-21, 06:42

thanks for the link here. will you revise this for 27.1's imminent release, or is that not necessary?

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-01-21, 07:47

fillerup wrote:thanks for the link here. will you revise this for 27.1's imminent release, or is that not necessary?
If you have the option...

Code: Select all

ac_add_options --enable-gstreamer
... in mozconfig.txt, delete it for 27.1.0. My understanding is that gstreamer 1.x can still be built with 27.1.0, but defaults to not being built. And if the transition to ffmpeg goes OK, the gstreamer code will be removed entirely from Pale Moon source sometime after 27.1.0.

That's all I'm aware of right now. I'll post here if/when I find out about any other changes.
There's a right way
There's a wrong way
And then there's my way

User avatar
trava90
Contributing developer
Contributing developer
Posts: 1736
Joined: 2013-05-20, 18:19
Location: Somewhere in Sector 001

Re: A linux build framework for Pale Moon

Unread post by trava90 » 2017-01-21, 15:10

Actually, GStreamer support is built by default so the --enable-gstreamer option is redundant anyway. 27.1 will still be built with GStreamer support, but it will be disabled by default (it can be enabled by flipping a pref if need be). Unless some serious regressions are found in the 27.1 release, GStreamer support will be removed entirely from 27.2.

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-02-23, 19:58

Hi all!
I'm having some trouble trying to build Palemoon for a Debian x32 API.
I followed all the steps here, but my build stop very early with:
...
/opt/BUILD/pmmain/native/pmbuild/_virtualenv/bin/python /opt/BUILD/pmmain/pm27.1.1/config/pythonpath.py -I/opt/BUILD/pmmain/pm27.1.1/othe
/opt/BUILD/pmmain/pm27.1.1/python/mozbuild/mozbuild/action/xpidl-process.py --cache-dir ../../../xpcom/idl-parser ../../../dist/i
../../../dist/include ../../../dist/bin/components .deps \
xulapp nsINativeAppSupport
xultmpl.xpt
/opt/BUILD/pmmain/native/pmbuild/_virtualenv/bin/python /opt/BUILD/pmmain/pm27.1.1/config/pythonpath.py -I/opt/BUILD/pmmain/pm27.1.1/othe
/opt/BUILD/pmmain/pm27.1.1/python/mozbuild/mozbuild/action/xpidl-process.py --cache-dir ../../../xpcom/idl-parser ../../../dist/i
../../../dist/include ../../../dist/bin/components .deps \
xultmpl nsIXULBuilderListener nsIXULSortService nsIXULTemplateBuilder nsIXULTemplateQueryProcessor nsIXULTemplateResult nsIXULTem
zipwriter.xpt
/opt/BUILD/pmmain/native/pmbuild/_virtualenv/bin/python /opt/BUILD/pmmain/pm27.1.1/config/pythonpath.py -I/opt/BUILD/pmmain/pm27.1.1/othe
/opt/BUILD/pmmain/pm27.1.1/python/mozbuild/mozbuild/action/xpidl-process.py --cache-dir ../../../xpcom/idl-parser ../../../dist/i
../../../dist/include ../../../dist/bin/components .deps \
zipwriter nsIZipWriter
make[6]: Leaving directory '/opt/BUILD/pmmain/native/pmbuild/config/makefiles/xpidl'
make[5]: Leaving directory '/opt/BUILD/pmmain/native/pmbuild/xpcom/xpidl'
make[4]: Leaving directory '/opt/BUILD/pmmain/native/pmbuild'
/opt/BUILD/pmmain/pm27.1.1/config/recurse.mk:36: recipe for target 'export' failed
make[3]: *** [export] Error 2
make[3]: Leaving directory '/opt/BUILD/pmmain/native/pmbuild'
/opt/BUILD/pmmain/pm27.1.1/config/rules.mk:541: recipe for target 'default' failed
make[2]: *** [default] Error 2
make[2]: Leaving directory '/opt/BUILD/pmmain/native/pmbuild'
/opt/BUILD/pmmain/pm27.1.1/client.mk:398: recipe for target 'realbuild' failed
make[1]: *** [realbuild] Error 2
make[1]: Leaving directory '/opt/BUILD/pmmain/pm27.1.1'
client.mk:171: recipe for target 'build' failed
make: *** [build] Error 2


I have no clue what is going on, can someone help me ??
Thanks!

User avatar
stevenpusser
Project Contributor
Project Contributor
Posts: 903
Joined: 2015-08-01, 18:33

Re: A linux build framework for Pale Moon

Unread post by stevenpusser » 2017-02-24, 20:33

There are already Debian package builds in the openSUSE build service repository: https://build.opensuse.org/package/show ... r/palemoon

If you read the instructions about adding the repository carefully, you will see they include the actual repository address that you can paste into a browser and open: http://download.opensuse.org/repositori ... ebian_8.0/

which includes the source files that the OBS builds all the Ubuntu and Debian packages from. If you're interested, you can download them and rebuild them yourself, using the Debian packaging tools. If you want to change anything, you can make them in the debian folder, such as making changes in the mozconfig option file, or look at the rules file to see how the package is being built. Be aware that debhelper usually does a lot of stuff automatically behind the scenes, such as setting the default install directory as /usr. Customizing the automatic debhelper options is usually handled by overrides in the rules file.

About your particular build error, all I can see is that you are trying to build it in /opt, which is a system directory, so I would guess that you are trying to build it as root or with sudo. Generally, many programs will have trouble when built that way, so I would try building it somewhere in your regular user's home folder as that regular user.

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-02-25, 04:06

JotaMG wrote:Hi all!
I'm having some trouble trying to build Palemoon for a Debian x32 API.
I followed all the steps here, but my build stop very early with:
Are you using version 2 of the framework? I can't go back and edit the first post in the thread to indicate that the first version is obsolete. I did a new version on December 9. It's approximately the 1st post on the 2nd page of the thread. The direct link to it is viewtopic.php?f=37&t=13898&start=20
There's a right way
There's a wrong way
And then there's my way

User avatar
trava90
Contributing developer
Contributing developer
Posts: 1736
Joined: 2013-05-20, 18:19
Location: Somewhere in Sector 001

Re: A linux build framework for Pale Moon

Unread post by trava90 » 2017-02-25, 07:08

Off-topic:
Walter Dnes wrote:I can't go back and edit the first post in the thread to indicate that the first version is obsolete.
First post has been edited to include note about (and link to) the newer version.

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-02-25, 09:33

Walter Dnes wrote:
JotaMG wrote:Hi all!
I'm having some trouble trying to build Palemoon for a Debian x32 API.
I followed all the steps here, but my build stop very early with:
Are you using version 2 of the framework? ... The direct link to it is viewtopic.php?f=37&t=13898&start=20
Yes I was!
Will try again with the new version.

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-02-25, 09:50

stevepusser wrote:There are already Debian package builds in the openSUSE build service repository: https://build.opensuse.org/package/show ... r/palemoon
...
About your particular build error, all I can see is that you are trying to build it in /opt, which is a system directory, so I would guess that you are trying to build it as root or with sudo. Generally, many programs will have trouble when built that way, so I would try building it somewhere in your regular user's home folder as that regular user.
Thank you, did not know about OBS, it seems very usefull, and gives one more option to try build PM.
Maybe it was not very explicit from my previous post, I'm trying to build for the x32 ABI (https://wiki.debian.org/X32Port).
I'm building as user, I've give full rights to /opt/BUILD, but even so, do you think I should try to build in user's home folder instead??

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-02-25, 11:42

JotaMG wrote: Yes I was!
Will try again with the new version.
Same error:

Code: Select all

... 
1:47.41 xul.xpt
 1:47.61 xulapp.xpt
 1:47.74 xultmpl.xpt
 1:48.02 zipwriter.xpt
 1:48.16 /home/user/BUILD/pmmain/pmsrc/config/recurse.mk:36: recipe for target 'export' failed
 1:48.16 make[3]: *** [export] Error 2
 1:48.16 /home/user/BUILD/pmmain/pmsrc/config/rules.mk:541: recipe for target 'default' failed
 1:48.16 make[2]: *** [default] Error 2
 1:48.16 /home/user/BUILD/pmmain/pmsrc/client.mk:398: recipe for target 'realbuild' failed
 1:48.16 make[1]: *** [realbuild] Error 2
 1:48.16 client.mk:171: recipe for target 'build' failed
 1:48.16 make: *** [build] Error 2
 1:48.18 0 compiler warnings present.
Will try using OBS...

User avatar
stevenpusser
Project Contributor
Project Contributor
Posts: 903
Joined: 2015-08-01, 18:33

Re: A linux build framework for Pale Moon

Unread post by stevenpusser » 2017-02-25, 18:56

Oh...the OBS only provides Intel i386 and amd64 builds for the standard developer. You are supposed to be able to add several more architectures if you ask them in an email, but I don't know if x32 is among them.

I know that from looking at Debian package source that the Debian developers very often have to add patches to programs' source to get them to build on the many platforms that Debian supports. Mozilla seems to care about only a few arches, so Firefox has quite a few of these.

You can try rebuilding my source packages from the OBS on an x32 platform--download the orig.tar.xz and the debian.tar.xz file in my second link.

Set up a build directory in your home folder with no spaces in its PATH, such as /home/palemoon/buildit, and put the two downloaded archives in it.

Extract the two archives and move the /debian folder into the extracted PM source folder.

Install the build-depends. It's assumed that the Debian packaging tools are going to be already installed, so aren't listed in /debian/control--there's a metapackage that will get them called "packaging-dev".

Install the extra packages needed to build PM as listed in debian/control; Debian Stretch users need to make sure they have the gcc, etc 4.9 packages installed to not use the standard gcc-6 in there. Otherwise, it needs

autoconf2.13,
python (>= 2.7),
unzip,
zip,
pkg-config,
libgtk2.0-dev (>= 2.14),
libdbus-1-dev (>=0.60),
libdbus-glib-1-dev (>= 0.60),
yasm (>= 1.1),
libasound2-dev,
libpulse-dev,
libxt-dev,
mesa-common-dev

Once those are installed, enter the extracted PM source folder and as a standard use (not root or with sudo) issue the build command:

Code: Select all

debuild -uc -us
I have a feeling that the build process will probably die with a gcc error because it will need patches for the x32 platform, but who knows? Maybe this will work... Also note that PM will use all available processor cores to parallelize the build, so if the build dies and you go back through the .build build log file to try and find the error, it could be in a thread that errored out some time before the rest of the build threads stopped in the log.

If it works, you'll end up with a .deb file back up in the directory with the orig.tar.xz file--where the build log also is.

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-02-26, 00:21

JotaMG wrote:
JotaMG wrote: Yes I was!
Will try again with the new version.

[...snip...]

Same error:
Can you tar up your "build", "mozconfig.txt", and "customize" (if you used one) and attach them to a post.? I'll look at them and see if I can find anything relevant. First cd to the directory containing "build" and "mozconfig.txt", and then execute the command...

Code: Select all

tar -cvzf pm.tar.gz build mozconfig.txt customize
...to make the tarball. Don't worry about the warning if you don't have "customize".
There's a right way
There's a wrong way
And then there's my way

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-02-26, 21:39

@walter, stevepusser

I've done lots of experiments, these are the ones that had been more successful:
(no need to tar files, my changes are minimal, compared to your versions)

in build

Code: Select all

export LDFLAGS="-mx32 -Wl,-O1,--sort-common,--as-needed,-z,relro"
 ...
make -f client.mk build &> /dev/stdout | tee ${startdir}/buildlog.txt
in mozconfig.txt

Code: Select all

export CFLAGS="-mx32 -O2 -march=nocona -floop-parallelize-all -fpredictive-commoning -ftree-loop-distribution -ftree-vectorize"
export CXXFLAGS="-mx32 ${CFLAGS}"
export LD=ld.gold
export LDFLAGS="-mx32 -Wl,-O1,--sort-common,--as-needed,-z,relro"
When I add the -mx32 flag, the build goes past the previous zipwriter.xpt error, and, as stevepusser correctly predicts, other problems arise, like this one for instance:

Code: Select all

../../dist/include/js/HashTable.h:556:9: error: static assertion failed: unexpected word size, new hashing strategy required to properl
         static_assert(sizeof(word) == 8,
         ^
../../dist/include/js/HashTable.h:559:36: warning: right shift count >= width of type
         return HashNumber((word >> 32) ^ word);
                                    ^
/opt/BUILD/pmmain/build/pmsrc/config/rules.mk:936: recipe for target 'Unified_cpp_dom_cellbroadcast0.o' failed

Size of long and pointers is 8 bytes for x86-64 and 4 bytes for x32, so just change
static_assert(sizeof(word) == 8 to static_assert(sizeof(word) == 4
and the drums beat for a while more, until hit this:

Code: Select all

/usr/include/x86_64-linux-gnux32/bits/sysctl.h:19:3: error: #error "sysctl system call is unsupported in x32 kernel"
 # error "sysctl system call is unsupported in x32 kernel"
   ^
/opt/BUILD/pmmain/build/pmsrc/config/rules.mk:904: recipe for target 'evutil_rand.o' failed
make[5]: *** [evutil_rand.o] Error 1
this one I don't know how to deal with??
does this help?
https://bugs.debian.org/cgi-bin/bugrepo ... 7881;msg=7

Another thing I don't quite understand why, is that I have to comment
#ac_add_options --with-pthreads
because it tells me that my system does not have pthreads support, but the pthreads lib is installed !??
So, this seems to be as far as I'm able to go for myself without further help.

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-02-27, 14:35

JotaMG wrote:

Code: Select all

/usr/include/x86_64-linux-gnux32/bits/sysctl.h:19:3: error: #error "sysctl system call is unsupported in x32 kernel"
 # error "sysctl system call is unsupported in x32 kernel"
   ^
/opt/BUILD/pmmain/build/pmsrc/config/rules.mk:904: recipe for target 'evutil_rand.o' failed
make[5]: *** [evutil_rand.o] Error 1
this one I don't know how to deal with??
does this help?
https://bugs.debian.org/cgi-bin/bugrepo ... 7881;msg=7

Another thing I don't quite understand why, is that I have to comment
#ac_add_options --with-pthreads
because it tells me that my system does not have pthreads support, but the pthreads lib is installed !??
So, this seems to be as far as I'm able to go for myself without further help.
"x32" is different enough that it's almost cross-compiling. The sad part is that it's being "standards-compliant, in not supporting sysctl, which is being deprecated in linux. So it's not "bug-compatible" to x86 and x86_64. Having said that, some spelunking in the source code turns up the option...

Code: Select all

ac_add_options --enable-x32
Can you try it and see if it helps?
There's a right way
There's a wrong way
And then there's my way

Locked