What optimization level should I use for Epyrus?

[Moonchild]

if you're interested at all in building Epyrus, I think you'll be surprised by how familiar the build process is...

http://www.epyrus.org/build.html

I had a look at this since it doesn't really require much brainpower and I want to stay somewhat busy when up. A few small remarks about the config on Windows:
1) --disable-eme is no longer a thing and will error the builds
2) The WIN32_REDIST_DIR is incorrect; current visual studio versions use {root VS dir}\VC\redist\MSVC\14.nn.xxxxx\$BUILD_ARCH\Microsoft.VC143.CRT where 14.nn.xxxxx depends on which update of VS you have installed (my current one is 14.38.33135)

Also, as an aside, for general optimization, you may want to think about using --disable-skia since a mail client won't actually use canvas drawing (and there's still the cairo fallback for that if needed) which can seriously cut down on build time for some, as well as reduce resulting binary size, complexity and potential bugs. Similarly, you may want to consider not building AV1 because I don't see that kind of codec playback being necessary for mail display?

[athenian200]

I had a look at this since it doesn't really require much brainpower and I want to stay somewhat busy when up. A few small remarks about the config on Windows:
1) --disable-eme is no longer a thing and will error the builds
2) The WIN32_REDIST_DIR is incorrect; current visual studio versions use {root VS dir}\VC\redist\MSVC\14.nn.xxxxx\$BUILD_ARCH\Microsoft.VC143.CRT where 14.nn.xxxxx depends on which update of VS you have installed (my current one is 14.38.33135)

Interestingly, both of those things are correct on my own .mozconfig (including the path to the CRT), but they are just wrong on the website. I'm really not sure how that old path got on there, but the lines you said you were missing are in my actual .mozconfig that I build Epyrus with. I mean, I know way back when I first built Epyrus I had it wrong, but I wrote the build instructions way after that, so that shouldn't even be in there. So weird... but yeah, I just happen to have it hardcoded in my case, and never bothered to turn it into a variable like was done later for Pale Moon.

Also, as an aside, for general optimization, you may want to think about using --disable-skia since a mail client won't actually use canvas drawing (and there's still the cairo fallback for that if needed) which can seriously cut down on build time for some, as well as reduce resulting binary size, complexity and potential bugs. Similarly, you may want to consider not building AV1 because I don't see that kind of codec playback being necessary for mail display?

That's a really interesting idea, the skia thing. I think the only way AV1 might be needed is if someone embedded a video file in an e-mail, but I'm not sure if Epyrus would theoretically support video playback in that case, or if anyone would use that functionality if it did. I should probably test what happens if someone tries embedding an AV1 video in an e-mail. If it doesn't work regardless, I should be safe to disable that.

[Navigator]

I have an AVX build of the current Epyrus version up here, if anyone wants to try it...

https://www.addons.epyrus.org/download/ ... n64-avx.7z

How much administrative overhead it would cost to provide a couple of different builds for each release?

[athenian200]

How much administrative overhead it would cost to provide a couple of different builds for each release?

Less than it would be for Pale Moon, honestly. Really, the main problem is Linux. Pale Moon has to do GTK2 and GTK3 versions of any build they offer, so they have to either enforce AVX on all builds or offer four versions because they don't want to go with some kind of odd logic like "GTK2 is the 32-bit of Linux" or something. On the other hand, I already said no to GTK2 for Linux, so I would only need to do one extra build.

The problem is that because of the way I do releases, all of them are a huge administrative overhead and require a lot of manual work on my part when I get ready to do one. Here's the process of doing an Epyrus release:

1. Decide which platform commit to use, and update the platform commit pointer on the Epyrus repo.

2. Update version.txt within the Epyrus repo to reflect the new Epyrus version.

3. Verify that everything builds and runs properly on all platforms.

4. Create a new release tag on the Epyrus repo referring to this version so I can roll back and see exactly what went into this version later on.

5. Open up MozillaBuild on Windows, and cd into the Epyrus source code directory on my hard drive.

6. Open up Git Bash, and also cd into the Epyrus source code directory on my hard drive.

7. Type git checkout Epyrus2.x.x_Release in the Git Bash window.

8. Type git submodule update in the Git Bash window.

9. Type notepad .mozconfig and verify my .mozconfig is correctly setup for a 32-bit build so I can do that first.

10. Type ./mach build in the MozillaBuild window to do a 32-bit build.

11. Wait for the build to finish, and then use ./mach run to verify everything built properly.

12. Type ./mach package && ./mach installer && ./mach mar && ./mach mozpackage to generate the base 7z package (required to generate the installer and mar), the Windows installer, the update MAR, and a zip file for distribution.

13. Type ls obj*/dist to make sure everything is in the directory that should be.

14. Repeat step 9, but this time setup my .mozconfig for a 64-bit build.

15. Repeat step 10.

16. Repeat step 11.

17. Repeat step 12.

18. Repeast step 13.

19. Use the ./mach updater-xml command I built into my perl-based mach stub to generate update xml files for both the 32-bit and 64-bit builds after they are built. (I couldn't figure out shell scripting well enough to do it as a cross-platform thing, and perl is a build requirement anyway).

20. Type the currently released version of Epyrus and verify the files were generated correctly.

21. Open up a virtual machine running CentOS 7 and get to a terminal window.

22. cd into the Epyrus source code directory.

23. Repeat steps 7 and 8, but in a GNOME terminal window instead of a Git Bash window obviously.
24. Type vim .mozconfig, verifying that I have a 64-bit build and everything is correct.

25. Type scl enable devtoolset-7 bash to use a newer version of GCC than what comes with CentOS 7.

26. Type ./mach build, and prepare for the build to take an unusually long time because my CentOS 7 VM can randomly be dog slow for unpredictable reasons... sometimes it builds in 20 or 30 minutes, sometimes it takes over an hour, and I never know which it will be.

27. Use ./mach run to verify everything is working.

28. Type ./mach package && ./mach mar to generate a tarball and an update MAR.

29. Finally, type ./mach updater-xml again to generate the update xml needed for Linux, entering the currently released version of Epyrus carefully and making sure not to enter the version I'm building.

30. Upload the Linux-based files, the update mar, the tarball, and the update xml to OneDrive so I can access it on Windows.

31. Open WinSCP on Windows.

32. Login to the Epyrus hosting provider with username and password.

33. Carefully navigate to where AUS looks for updates.

34. Create a new directory named for the version I'm about to release within the subdirectory associated with each platform, and make sure to put an empty update.xml file there as place holder, otherwise users will complain about weird pop-ups saying that updating failed.

35. Go into the directory named for the version that's currently released, and paste the update.xml and MAR file I generated for each platform into the correct folder, being careful not to mix things up and deliver the wrong bitness/OS to the wrong people.

36. While I'm at it, go ahead and upload the the new installers/tarballs/zip files of newer versions for download on my Downloads page, editing the HTML and making sure to update the link and remove the tarballs/installers of the older versions, avoiding the need for page formatting changes with each release or using up extraneous disk space on my hosting service by offering older releases.

36. Check the Windows 32-bit and 64-bit versions update correctly myself on my own machines to make sure they updated properly, and just watch the forums nervously for a couple hours to see if Linux users complain, since the Linux build is almost always correct due to it being a single build with no counterparts, and the process being very different compared with 64-bit and 32-bit Windows.

37. Announce the new release on the forums, and I'm done, hopefully for a few months... I don't enjoy this process at all because I really can't afford to make a mistake here, and I'm not the most detail-oriented person in the world. So I try to avoid updating Epyrus more often than necessary because... well, look at this list.

[Bilbo47]

Here's the process of doing an Epyrus release:

Wow, thanks for posting that! It shows the detail of a major unknown in contemplating building major apps for release, let alone in building a private instance. Converting this bit from unknown to known makes it less scary, but it's still a barrier to entry. Most build-systems I've seen say like "this is as automated/easy as we could make it" but it still seems much too manual and detailed for use by general developers.

[athenian200]

Wow, thanks for posting that! It shows the detail of a major unknown in contemplating building major apps for release, let alone in building a private instance. Converting this bit from unknown to known makes it less scary, but it's still a barrier to entry. Most build-systems I've seen say like "this is as automated/easy as we could make it" but it still seems much too manual and detailed for use by general developers.

Well, it wasn't really intended as a guide, more as an illustration of why I am trying to keep the number of builds to a minimum... truth is, I was actually trying to illustrate that the Epyrus release process is painful and manual enough that any additional builds make my life harder.

It was even worse before I created a way to auto-generate the update.xml files. At first I had to do everything I listed above, and then also run various terminal commands to get the SHA sum and size in bytes of MAR files, change the version number manually in a template, etc.

My problem when it comes to development is that I am the kind of person that is smart enough to follow a convoluted series of steps and figure out how to change something and make it do what I want, but I wind up getting burned out when I have to repeat the same complicated process to achieve the same result over and over again. I'm good at wading into a complicated subject and understanding the moving parts well enough to get what I want... once. But I'm not good at streamlining things for efficiency or reducing the amount of pain involved in reproducing the result for myself and others.

[Moonchild]

It was even worse before I created a way to auto-generate the update.xml files.

I initially created XML files manually as well which is a royal pain. Thankfully Tobin (with some help from me figuring out a re-entrancy issue on a high-request-rate server) spent some time writing the AUS script for us which now basically make it a hands-free deal; the XMLs are generated server-side from the .mar and build .json files directly (and cached) so it just needs to have new .mar/.json uploaded, the old ones deleted, and then the cache cleared (just deleting of the written-out XML manifest) for an update.
If you're interested in that I'll happily send it over. it's PHP.

I wind up getting burned out when I have to repeat the same complicated process to achieve the same result over and over again

That's the problem with being smart. It's an issue I've had trouble conveying to employers because there's still this wrong understanding of being overqualified for certain tasks - repetition burns you out faster than a candle with 10 wicks.

[RuudNL]

I have an AVX build of the current Epyrus version up here, if anyone wants to try it...

https://www.addons.epyrus.org/download/ ... n64-avx.7z

Nice. Super responsive on my 6th gen i7 platform. I would welcome this as the default optimised build

[Basilisk-Dev]

Here's the process of doing an Epyrus release:

Good lord. I was really appalled as to why multiple builds seemed like such a big deal to you earlier in the thread, I see why now. You're doing all of this manually.

I do the Windows builds manually, and manually copy the Windows builds to my Linux desktop from the Windows VM.

On the Linux side I have some internal scripts I've written to automate all of this. I literally run one command and it builds both 64-bit GTK builds, both aarch64 GTK builds, and optionally if I pass in an argument it will do CPU optimized builds. It also will SCP the builds to the server if I want it to.

Sorry to hear you're having to spend so much time to do Epyrus builds. I'd be happy to help make the process easier for you.

I do recommend the PHP script Moonchild mentioned, I have a fork of an older version of the script that I am using and it really does speed up the process significantly for the release XML generation.

[athenian200]

Good lord. I was really appalled as to why multiple builds seemed like such a big deal to you earlier in the thread, I see why now. You're doing all of this manually.

I do the Windows builds manually, and manually copy the Windows builds to my Linux desktop from the Windows VM.

On the Linux side I have some internal scripts I've written to automate all of this. I literally run one command and it builds both 64-bit GTK builds, both aarch64 GTK builds, and optionally if I pass in an argument it will do CPU optimized builds. It also will SCP the builds to the server if I want it to.

Sorry to hear you're having to spend so much time to do Epyrus builds. I'd be happy to help make the process easier for you.

I do recommend the PHP script Moonchild mentioned, I have a fork of an older version of the script that I am using and it really does speed up the process significantly for the release XML generation.

For what it's worth, one of the first things I'm planning to work on when school lets out is my own way of automating all this. I already have portions of it more automated than it used to be, I just need to put the finishing touches on it.

I'd prefer not to introduce any dependencies on PHP if I can help it, especially since I'm thinking I'm close to having my own solution... but yeah, I really should have put more effort into this before Epyrus got as far along as it has. Now I'm kind of stalled out because of school (and my Dad visiting, I can't get anything done when he's around because him showing up always means various construction/remodeling projects will be happening, and thus I don't know what to expect from one day to the next), so it's been longer than I would like since the last release.