29.4.2 source tarball changed

Talk about code development, features, specific bugs, enhancements, patches, and similar things.
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.

This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.

Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
User avatar
Bababo26
Moongazer
Moongazer
Posts: 9
Joined: 2021-02-12, 11:49

29.4.2 source tarball changed

Unread post by Bababo26 » 2021-11-11, 19:53

On Nov 9 I downloaded a copy of the 29.4.2 source from http://archive.palemoon.org/source, the file was named palemoon-29.4.2-source.tar.xz. Today when I attempted to download the 29.4.2.1 release using the same naming convention I discovered that you no longer use a dash in the file name, no problem. While discovering this I noticed the 29.4.2 source code file had also changed name, was a different size and modified on Nov 10.

Being paranoid I downloaded the new 29.4.2 source code file palemoon-29.4.2.source.tar.xz and discovered that the directory structure had changed, it now places the source code into a subdirectory palemoon-source which might explain the change but when I did a quick diff against the Nov 9 version I got other changes

Code: Select all

$ diff -rq palemoon-29.4.2-source palemoon-29.4.2.source/palemoon-source
Only in palemoon-29.4.2.source/palemoon-source: .ycm_extra_conf.py
Files palemoon-29.4.2-source/mach and palemoon-29.4.2.source/palemoon-source/mach differ
Only in palemoon-29.4.2-source/palemoon/branding: beta
Only in palemoon-29.4.2-source/palemoon/branding: unstable
Only in palemoon-29.4.2-source/platform: .gitattributes
Only in palemoon-29.4.2-source/platform/db: mork
Only in palemoon-29.4.2-source/platform: docs
Only in palemoon-29.4.2-source/platform/dom/heapsnapshot: .gitattributes
Only in palemoon-29.4.2-source/platform: ldap
Only in palemoon-29.4.2-source/platform: mailnews
Only in palemoon-29.4.2-source/platform/media/libaom/src: .gitattributes
Only in palemoon-29.4.2-source/platform/media/libaom/src/build: .gitattributes
diff: palemoon-29.4.2.source/palemoon-source/platform/media/libav/README: No such file or directory
Only in palemoon-29.4.2-source/platform/python/pystache: .gitmodules
diff: palemoon-29.4.2.source/palemoon-source/platform/testing/mozharness/configs/single_locale/linux32.py: No such file or directory
Only in palemoon-29.4.2-source/platform: xulrunner
Which is concerning. Is this the correct code? Why was the code changed post release? Would it be possible to provide GPG/PGP signatures for source code tarball as you do for the binary releases so we can check authenticity?

Mark


Note the 2 errors from the diff were caused by symlinks pointing to non existent files

Code: Select all

$ ls -l palemoon-29.4.2.source/palemoon-source/platform/media/libav/README
lrwxrwxrwx 1 mark users 23 Nov  8 19:27 palemoon-29.4.2.source/palemoon-source/platform/media/libav/README -> palemoon-sourceEADME.md
$
$ ls -l  palemoon-29.4.2.source/palemoon-source/platform/testing/mozharness/configs/single_locale/linux32.py
lrwxrwxrwx 1 mark users 22 Nov  8 19:20 palemoon-29.4.2.source/palemoon-source/platform/testing/mozharness/configs/single_locale/linux32.py -> palemoon-sourceinux.py
which look odd, the Nov 9 versions look correct

Code: Select all

$ ls -l palemoon-29.4.2-source/platform/media/libav/README
lrwxrwxrwx 1 mark users 9 Nov  8 19:27 palemoon-29.4.2-source/platform/media/libav/README -> README.md
$
$ ls -l  palemoon-29.4.2-source/platform/testing/mozharness/configs/single_locale/linux32.py
lrwxrwxrwx 1 mark users 8 Nov  8 19:20 palemoon-29.4.2-source/platform/testing/mozharness/configs/single_locale/linux32.py -> linux.py
The 29.4.2.1 source tarball also has the incorrect symlinks.

Code: Select all

$ ls -l palemoon-29.4.2.1.source/palemoon-source/platform/media/libav/README
lrwxrwxrwx 1 mark users 23 Nov  8 19:27 palemoon-29.4.2.1.source/palemoon-source/platform/media/libav/README -> palemoon-sourceEADME.md
$
$ ls -l  palemoon-29.4.2.1.source/palemoon-source/platform/testing/mozharness/configs/single_locale/linux32.py
lrwxrwxrwx 1 mark users 22 Nov  8 19:20 palemoon-29.4.2.1.source/palemoon-source/platform/testing/mozharness/configs/single_locale/linux32.py -> palemoon-sourceinux.py

New Tobin Paradigm

Re: 29.4.2 source tarball changed

Unread post by New Tobin Paradigm » 2021-11-11, 20:39

Because I devised a script to compress the source so that it excluded stuff like leftover artifacts, items that are not compiled into Pale Moon like mailnews components and with maximum compression for smaller downloads and less space taken on the archive server.

Also when you untar it it matches our build documentation which cites a "palemoon-source" directory.

Moonchild simply recompressed using that script. Which is all in prep for some infrastructure improvements coming in the next few months regarding download and update services that should make things much easier on everyone.

Additionally, there shouldn't BE any symlinks but I will look into it.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: 29.4.2 source tarball changed

Unread post by Moonchild » 2021-11-11, 21:46

There is no reason for concern.
Both tarballs are a complete source to build the browser and all components. The new upload is merely trimmed down a little as explained by Tobin, removing parts of the source that are neither necessary nor relevant to the browser build.
Full buildability and correct operation with a fresh extraction of the tarball resulting from the script has been confirmed and it will produce the exact executable form you can download from our site.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
Bababo26
Moongazer
Moongazer
Posts: 9
Joined: 2021-02-12, 11:49

Re: 29.4.2 source tarball changed

Unread post by Bababo26 » 2021-11-11, 22:13

New Tobin Paradigm wrote:
2021-11-11, 20:39
Because I devised a script to compress the source so that it excluded stuff like leftover artifacts, items that are not compiled into Pale Moon like mailnews components and with maximum compression for smaller downloads and less space taken on the archive server.
Ah the change to mach. Its a bit disconcerting having files in the sources tarball newer than the release.
New Tobin Paradigm wrote:
2021-11-11, 20:39
Additionally, there shouldn't BE any symlinks but I will look into it.
The problem is you have not escapted the . in your transformation regexp, so it should be

Code: Select all

MCP_TAR_BASEDIR="s/^\\./$MCP_APP-source/"
When testing the above there was a complaint that --exclude-vcs was being ignored because it only affects following parameters. So you probably really want

Code: Select all

MCP_TAR_COMMAND="tar cfJv ../$MCP_TAR_FILENAME --exclude-vcs . --transform $MCP_TAR_BASEDIR --warning=no-file-changed"
Mark

New Tobin Paradigm

Re: 29.4.2 source tarball changed

Unread post by New Tobin Paradigm » 2021-11-11, 22:33

I will give that a try. Thank you.

Luckily the 2 symlinks (which I have no idea how they were committed as such) are not critical to anything. The readme to readme.md is obvious and mozharness will likely be gotten rid of as we clean up more un-used mozjunk related to their unknown and irrelevant build automation shit they have changed at least twice anyway.

Locked