Intended archive format change for Linux

Support and discussions for the x86/x64 Linux version of Pale Moon and specific Linux distribution questions related to the browser.

Moderator: trava90

Forum rules
If your question is about general use of the browser and not specific to Linux, then please use the General Support board.

What archiving format should be used for the Linux tarballs?

Poll ended at 2019-11-29, 18:49

bzip2 (current, larger files)
4
9%
xz (proposed, smaller files)
23
49%
I don't have a preference
20
43%
My distro doesn't support xz! (reply below)
0
No votes
 
Total votes: 47

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 28147
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Intended archive format change for Linux

Post by Moonchild » 2019-11-15, 18:49

Considering all supported distros to the best of my knowledge have xz support for tarballs (although maybe not always installed by default), I'd like to change the Linux tarball archive format from bz2 to xz for smaller downloads and (on our side as well) less needed bandwidth. But, I'm unsure as to how the Linux users would view such a change, so I'm asking y'all: xz or bz2, or you don't really care either way? Please use the poll to make your choice -- no need to announce in replies what you've chosen, it's redundant.

Note: If your distro doesn't have xz support at all, please let us know, and mention which distro you are using. Please check first if it might be available but simply not installed.
"There will be times when the position you advocate, no matter how well framed and supported, will not be accepted by the public simply because you are who you are." -- Merrill Rose
Image

daftman1
Moongazer
Moongazer
Posts: 9
Joined: 2016-08-20, 19:57

Re: Intended archive format change for Linux

Post by daftman1 » 2019-11-15, 20:11

Why not use zstd? Arch Linux recently switched their package manager to use it instead of xz, they claim the encryption/decryption speeds are faster while compression level is comparable to xz.

User avatar
New Tobin Paradigm
Knows the dark side
Knows the dark side
Posts: 8064
Joined: 2012-10-09, 19:37
Location: I'm.. over there.

Re: Intended archive format change for Linux

Post by New Tobin Paradigm » 2019-11-15, 20:15

Because these are generic binaries intended to run on many systems and seeing as I have never heard of that format before today.. Who knows what all supports it. xz by now should be well supported by all distros and is deff supported in the majority of them.

I also don't think using Facebook technology is a good thing.
Your computer has too much computer in it and not enough typewriter.
Image

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 28147
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Intended archive format change for Linux

Post by Moonchild » 2019-11-15, 20:29

daftman1 wrote:
2019-11-15, 20:11
encryption/decryption speeds
No encryption is used so this is not an argument anyway.
Like Tobin I never even heard of this format before your mention of it, meaning it's likely hardly adopted anywhere, not mature, and certainly unsuited for software distribution to a broad audience. In addition, after 1 minute research, it's clear that it's not meant for archive compression, it's meant for stream/transport/etc. compression ("real-time"). Compression ratios are also not the same as LZMA (considerably less[1]) which makes it a non-starter for this purpose. Not sure why Arch would decide to switch to it in favor of xz for packages, unless they are suffering from Magpie Syndrome.

[1]I did some basic comparison with a typical data set (/dist/bin of a built Pale Moon). Zip = 41.4 MB; zstd (default setting at fast) = 36.6 MB; zstd (ultra) = 30.4 MB; 7z (ultra) = 27.6 MB -- of note is that compression speed of zstd at ultra was very slow (~1 MB/s) compared to 7z (~3.6 MB/s), on my system.
"There will be times when the position you advocate, no matter how well framed and supported, will not be accepted by the public simply because you are who you are." -- Merrill Rose
Image

daftman1
Moongazer
Moongazer
Posts: 9
Joined: 2016-08-20, 19:57

Re: Intended archive format change for Linux

Post by daftman1 » 2019-11-15, 21:01

Sorry I meant compression/decompression speeds (instead of encryption) but of course you are correct.
I took the statements from the Arch announcement as given while reality looks much less favourable for zstd - even speed doesn't seem as good as claimed (https://community.centminmod.com/thread ... etc.17259/).

User avatar
New Tobin Paradigm
Knows the dark side
Knows the dark side
Posts: 8064
Joined: 2012-10-09, 19:37
Location: I'm.. over there.

Re: Intended archive format change for Linux

Post by New Tobin Paradigm » 2019-11-15, 22:01

Considering that wasn't one of the options, I dunno why you suggested it.
Your computer has too much computer in it and not enough typewriter.
Image

vannilla
Board Warrior
Board Warrior
Posts: 1186
Joined: 2018-05-05, 13:29

Re: Intended archive format change for Linux

Post by vannilla » 2019-11-15, 22:57

If I'm not mistaken, the Linux kernel itself started using XZ with version 5, or maybe even earlier.

bgstack15
Moon lover
Moon lover
Posts: 91
Joined: 2018-01-22, 23:04

Re: Intended archive format change for Linux

Post by bgstack15 » 2019-11-17, 00:46

No preference. Can use tar -J everywhere I've needed to. Thanks for checking with the users!

User avatar
Lunokhod
Lunatic
Lunatic
Posts: 296
Joined: 2017-04-20, 21:25
Contact:

Re: Intended archive format change for Linux

Post by Lunokhod » 2019-11-17, 03:58

Code: Select all

Original archive:
$ time tar -xjf 2019-11-11-palemoon-unstable-latest.linux-x86_64.tar.bz2 
real	0m4.879s
user	0m4.782s
sys	0m0.503s
Retar here for comparison in case of OS oddity
$ tar -cJf  xzpalemoon.tar.xz palemoon/
$ tar -cjf bpalemoon.tar.bz2 palemoon/
$ rm -r palemoon/
$ time tar -xJf xzpalemoon.tar.xz 
real	0m2.912s
user	0m2.790s
sys	0m0.579s
$ rm -r palemoon/
$ time tar -xjf bpalemoon.tar.bz2 
real	0m4.871s
user	0m4.801s
sys	0m0.476s
The bz2 is 47M while the xz is 40M, so that suggests both faster extraction and download times for xz. The concept works well here.

User avatar
New Tobin Paradigm
Knows the dark side
Knows the dark side
Posts: 8064
Joined: 2012-10-09, 19:37
Location: I'm.. over there.

Re: Intended archive format change for Linux

Post by New Tobin Paradigm » 2019-11-17, 04:45

When you consider using 7z on windows at ultra and xz on linux at max a single Pale Moon release can save upwards of 40 megs of server and archive storage.. 50 once GTK3 build is gotten around to. Of course smaller downloads mean less transfer both server and user side and it extracts faster than bz2.. Well that is a plus..

Good.
Your computer has too much computer in it and not enough typewriter.
Image

User avatar
loxodont
Astronaut
Astronaut
Posts: 619
Joined: 2014-07-26, 23:03
Location: Mare Serenitatis

Re: Intended archive format change for Linux

Post by loxodont » 2019-11-17, 13:58

I voted "no preference" as Pale Moon internal updates worked very well (since updater was dropped) and I guess if I need unpacking xz files my linuxes will do.

Axatax
Apollo supporter
Apollo supporter
Posts: 40
Joined: 2018-05-17, 14:05

Re: Intended archive format change for Linux

Post by Axatax » 2019-11-17, 19:52

Who cares?

In the age of symmetical gigabit fiber you can UUENCODE for all I care.

Axatax
Apollo supporter
Apollo supporter
Posts: 40
Joined: 2018-05-17, 14:05

Re: Intended archive format change for Linux

Post by Axatax » 2019-11-17, 19:53

Axatax wrote:
2019-11-17, 19:52
Who cares?

In the age of symmetical gigabit fiber you can UUENCODE for all I care. It takes a whole 3 seconds to download.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 28147
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Intended archive format change for Linux

Post by Moonchild » 2019-11-17, 21:47

Axatax wrote:
2019-11-17, 19:52
In the age of symmetical gigabit fiber you can UUENCODE for all I care.
The ignorance displayed here is quite astonishing.
"There will be times when the position you advocate, no matter how well framed and supported, will not be accepted by the public simply because you are who you are." -- Merrill Rose
Image

User avatar
stevepusser
Project Contributor
Project Contributor
Posts: 675
Joined: 2015-08-01, 18:33
Location: California

Re: Intended archive format change for Linux

Post by stevepusser » 2019-11-17, 22:22

Most distros should include parallel compressors for the various formats to take advantage of modern CPUs, but the only two I've used for xz are pxz and pixz.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 28147
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Intended archive format change for Linux

Post by Moonchild » 2019-11-17, 23:27

stevepusser wrote:
2019-11-17, 22:22
Most distros should include parallel compressors for the various formats to take advantage of modern CPUs, but the only two I've used for xz are pxz and pixz.
I'm not entirely sure how this relates to the question at hand...
"There will be times when the position you advocate, no matter how well framed and supported, will not be accepted by the public simply because you are who you are." -- Merrill Rose
Image

bgstack15
Moon lover
Moon lover
Posts: 91
Joined: 2018-01-22, 23:04

Re: Intended archive format change for Linux

Post by bgstack15 » 2019-11-18, 14:05

The user commented about various archiving tools used for the proposed archive format. That sounds relevant to me. Also, it might provide information to people who are not yet aware of such tools to open xz files. If you want a survey without general discussion, go use surveymonkey.

Lucio Chiappetti
Lunatic
Lunatic
Posts: 353
Joined: 2014-09-01, 15:11
Location: Milan Italy

Re: Intended archive format change for Linux

Post by Lucio Chiappetti » 2019-11-18, 15:04

While in general I'm not so keen in the "yet another compression tool" which each distro and version seem to use (compare e.g. the archiving of syslog and alike), it is easy to use a command like apropos xz to get all the commands needed. Or man xzcat if you dare to guess.
The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. (G.B. Shaw)

User avatar
karlchen
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2019-01-16, 15:55

Re: Intended archive format change for Linux

Post by karlchen » 2019-11-18, 18:54

Finding xz (XZ Utils) 5.1.0alpha on my older LTS systems (Ubuntu 16.04.6 / Mint 18.1), I decided to be a daredevil and opt for the xz format. ;)

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 28147
Joined: 2011-08-28, 17:27
Location: Tranås, SE
Contact:

Re: Intended archive format change for Linux

Post by Moonchild » 2019-11-18, 19:05

For the record, you don't need to "learn new tools" or read manuals for this change. you'd still be using good ol' tar to extract the tarball, just with a different flag (capital J) to indicate it's xz compressed. So you can get off of your proverbial soapboxes.

We won't be switching to anything else. xz/lzma has been tried and tested and is mature enough to consider at this point, that's why I posed the question. It's not supposed to be about any alternatives or how distros handle stuff internally when encountering the compressor or archiving tools or what not.
Off-topic:
And no, I won't be using any user-tracking big name survey engine that will Spam You To Death Forever After You've Used Them Once™ because they want to sell you something-as-a-service. All I ask is that everyone please keep discussion here ON TOPIC, i.e. directly related to the posed question in the original post. If y'all can't do that, then you risk more unilateral decisions in the future about the browser and its release engineering.
"There will be times when the position you advocate, no matter how well framed and supported, will not be accepted by the public simply because you are who you are." -- Merrill Rose
Image

Locked