Linux built fm source; mozconfig won't take MOZ_OBJDIR from env variable Topic is solved

[Walter Dnes]

In case anyone else wants to try, the git pull is 1.9 GIGABYTES. After the build process completes the *.tar.bz2 files are...

1. 64-bit 40,008,722 bytes (i3 desktop)
2. 32-bit 42,039,686 bytes (my netbook)

I've got both 64-bit and 32-bit builds running. They seem to work OK. One quirk I noticed is that it seemed to have forgotten my Wordpress login+password (automatic formfill). I had to clear the password on the Wordpress profile, and it rembers now after I entered it once. But it has remembered my Slashdot password properly.

The netbook feels a bit slower, but that may be psychological. I've run into one issue so far, which Travis has helped me work around. I use almost the identical environment for both of my desktop PCs (64-bit Gentoo linux), and the 32-bit VM that builds for my netbook. To make it generic, I have the build script calculate and export environmental variables "cpucount" and "objdir" at build time. In mozconfig.txt, they get picked up like so...

Code: Select all

mk_add_options MOZ_MAKE_FLAGS="-j${cpucount}"
mk_add_options MOZ_OBJDIR=${objdir}

The build script normally copies mozconfig.txt to pmsrc/.mozconfig This has worked as expected, until v27 alpha. It seems that part of the build process picks up MOZ_OBJDIR properly, and part doesn't. It starts off OK, with the correct directory, but another portion can't find it,and the build dies. Gzipped log is attached.

But if MOZ_OBJDIR is hard-coded into pmsrc/.mozconfig, it builds fine. So I commented out

Code: Select all

mk_add_options MOZ_OBJDIR=${objdir}

in mozconfig.txt, and changed a couple of lines in the build script. Rather than copying mozconfig.txt to pmsrc/.mozconfig, it uses bash to write the MOZ_OBJDIR line to pmsrc/.mozconfig and then appends mozconfig.txt to it.

[Moonchild]

My own .mozconfig for Tycho has:

Code: Select all

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/build-x86

...which works just fine (for building on Windows, at any rate)

What is {$objdir} in your case? does it carry across to your build script?

[Walter Dnes]

What is {$objdir} in your case? does it carry across to your build script?

The flow is that values are calculated in the main "build" script, and exported as environmental variables, which are then picked up by the mozconfig file. Note that by default, there is no "customize" file. I use it only when building for non-native CPUs, e.g. building for my Atom inside a VM on my desktop. The build script includes the following 2 snippets...

Code: Select all

cpucount=`grep -c "^flags" /proc/cpuinfo`
(( cpucount++ ))
export cpucount

The number of cpu cores varies from machine to machine. I set the number of parallell jobs to core count plus 1.

Code: Select all

export objdir="${PWD}/pmbuild/"fi

In bash scripts ${PWD} is the Present Working Directory. In my case ${PWD} is "/home/waltdnes/palemoon/tycho" and "${PWD}/pmbuild/" should be "/home/waltdnes/palemoon/tycho/pmbuild/".

For the 26.3.3 build, mozconfig.txt contains the lines...

Code: Select all

mk_add_options MOZ_MAKE_FLAGS="-j${cpucount}"
mk_add_options MOZ_OBJDIR=${objdir}

Here's the start of the buildlog in the failure case

0:00.86 /usr/bin/gmake -f client.mk MOZ_PARALLEL_BUILD=5 -s
0:01.32 client.mk:201: /home/waltdnes/palemoon/tycho/pmbuild/.mozconfig.mk: No such file or directory
0:01.79 Clobber not needed.
0:03.11 Adding client.mk options from /home/waltdnes/palemoon/tycho/pmsrc/.mozconfig:
0:03.11 AUTOCLOBBER=1
0:03.11 MOZ_CO_PROJECT=browser
0:03.11 MOZ_MAKE_FLAGS=-j5
0:03.11 MOZ_OBJDIR=/home/waltdnes/palemoon/tycho/pmbuild
0:03.11 OBJDIR=/home/waltdnes/palemoon/tycho/pmbuild
0:03.11 FOUND_MOZCONFIG=/home/waltdnes/palemoon/tycho/pmsrc/.mozconfig
0:03.50 cd /home/waltdnes/palemoon/tycho/pmbuild
0:03.50 /home/waltdnes/palemoon/tycho/pmsrc/configure
0:03.81 Adding configure options from /home/waltdnes/palemoon/tycho/pmsrc/.mozconfig

Note the lines...
0:03.11 MOZ_MAKE_FLAGS=-j5
0:03.11 MOZ_OBJDIR=/home/waltdnes/palemoon/tycho/pmbuild
0:03.11 FOUND_MOZCONFIG=/home/waltdnes/palemoon/tycho/pmsrc/.mozconfig
0:03.50 cd /home/waltdnes/palemoon/tycho/pmbuild

I have 4 cores on my current desktop. 4 + 1 = 5, so that is passed properly. It also gets MOZ_OBJDIR properly. In the file I attached with my initial message, it goes on quite happily, building jemalloc, libffi, and icu. But it eventually runs into a brick wall with...

0:20.39 js/src> checking for malloc_usable_size... (cached) yes
0:20.39 js/src> updating cache /home/waltdnes/palemoon/tycho/pmbuild/config.cache
0:20.39 js/src> creating ./config.status
0:20.67 Traceback (most recent call last):
0:20.67 File "./config.status", line 980, in <module>
0:20.67 config_status(**args)
0:20.67 File "/home/waltdnes/palemoon/tycho/pmsrc/python/mozbuild/mozbuild/config_status.py", line 135, in config_status
0:20.68 emitter = TreeMetadataEmitter(env)
0:20.68 File "/home/waltdnes/palemoon/tycho/pmsrc/python/mozbuild/mozbuild/frontend/emitter.py", line 93, in __init__
0:20.68 mozinfo.find_and_update_from_json(config.topobjdir)
0:20.68 File "/home/waltdnes/palemoon/tycho/pmsrc/testing/mozbase/mozinfo/mozinfo/mozinfo.py", line 164, in find_and_update_from_json
0:20.69 build = MozbuildObject.from_environment()
0:20.69 File "/home/waltdnes/palemoon/tycho/pmsrc/python/mozbuild/mozbuild/base.py", line 181, in from_environment
0:20.69 raise ObjdirMismatchException(topobjdir, _config_topobjdir)
0:20.69 mozbuild.base.ObjdirMismatchException: Objdir mismatch: /home/waltdnes/palemoon/tycho/pmbuild != /home/waltdnes/palemoon/tycho/pmsrc/js/src
0:20.71 *** Fix above errors and then restart with\
0:20.71 "/usr/bin/gmake -f client.mk build"
0:20.71 /home/waltdnes/palemoon/tycho/pmsrc/client.mk:361: recipe for target 'configure' failed
0:20.71 gmake[2]: *** [configure] Error 1
0:20.71 /home/waltdnes/palemoon/tycho/pmsrc/client.mk:375: recipe for target '/home/waltdnes/palemoon/tycho/pmbuild/Makefile' failed
0:20.71 gmake[1]: *** [/home/waltdnes/palemoon/tycho/pmbuild/Makefile] Error 2
0:20.71 client.mk:171: recipe for target 'build' failed
0:20.71 gmake: *** [build] Error 2
0:20.73 0 compiler warnings present.

See the attached file with the first post for the full details.

[Moonchild]

Unless I'm mistaken, you're trying to have the OBJDIR outside of the SRCDIR - I don't think that's supported by the Mozilla build system.

[New Tobin Paradigm]

While I don't know if it was officially supported to have an obj-dir outside of source.. You CAN get the same effect by setting MOZ_OBJDIR to a directory path relative to source directory.. ie ../obj-dir. Also, it may not pick up env vars that are specified outside of the mozconfig... That is why we have to EXPORT for a couple things sometimes.

But yeah, just set your outside obj-dir to a directory using a relative path from topsrcdir and it will work.

[Walter Dnes]

I can confirm that...

Code: Select all

mk_add_options MOZ_OBJDIR=../pmbuild

works as expected, and builds properly. Where's the green+white checkmark for "issue solved"? BTW, the developer wiki http://developer.palemoon.org/Developer ... Moon/Linux suggests

Code: Select all

mk_add_options MOZ_OBJDIR=/home/$USER/pmbuild/

Maybe that can be changed. The reason I wanted to get away from hard-coded absolute paths was that one day I cloned a working dir, and forgot to change the MOZ_OBJDIR entry in the new mozconfig.txt. Net result, the binary code being compiled from the new working directory was sent to pmbuild/ in the old working directory. That's my version of "OOPS progamming"

EDIT: I found the "Accept answer" checkmark.

[New Tobin Paradigm]

If the wiki needs a correction then register and edit the page please.

[Walter Dnes]

If the wiki needs a correction then register and edit the page please.

I'd prefer to wait at little while, to test it. My previous approach worked for several months, before hitting problems. Also, I'm currently working on documenting an overall set of small utilities and instructions that I've put together for building Pale Moon. I need this for personal use, so that I can build Pale Moon "at the drop of a hat" on either of my desktops, and in a VM (for my Atom netbook). It would also be useful for anybody doing "contributed builds" for linux. It's 99% generic, with all customizations in 1 or 2 files. When it's finished, this documentation would replace most of the linux instructions now in the wiki. It's currently almost 300 lines long, not counting the short utility scripts.

It may be counter-intuitive, but I seem to document stuff best when I'm learning it. I have to think things through, and follow the steps. Once I get to the point where I can do a process "without thinking", I'm hard-pressed to explain exactly what I'm doing.

[New Tobin Paradigm]

If you are gonna make major changes to that page please submit them to me first.