AMO and Zamboni, The Serpent Saga

[access2godzilla]

Why not run the AMO code directly on your servers? Have you had any problems with it?

(The reason for my insistence upon those features is that Pale Moon should appear to be a polished product, not a half-baked one. The impression one would get from a half-finished site will not attract new users.)

Also, regarding moderation: what to do with those extensions that track the user? Allow them as long as the user is informed of it and given an option to opt out?

[New Tobin Paradigm]

Why not run the AMO code directly on your servers? Have you had any problems with it?[...]

Remora was what I wanted to run originally since it is a PHP solution but it is severly bit rotted and is based on the barely maintained PHPCake framework. As for Zamboni.. it is a horridly massive, convoluted, python mess that is exceptionally difficult to setup and maintain and very heavy on resources and frankly has junk in it for the Firefox Marketplace as well that we simply do not need. A solution tailored to the needs of Pale Moon is what is needed but until a few weeks ago I wasn't sure how this would shape up or when it would be come needed. Also I do not know python well enough to even attempt to run it. (If you want an example of how ungodly unstable and convoluted Mozilla's python junk is allow me to draw the example of the Sync Server)

What is needed now is a central location for addons for Pale Moon in some form or another. While I intend to devise a more complete AMO like solution that caters to not only front end users but also addon developers them selves this has to be done properly and sustainably.

This won't be half baked perse but just limited initially for developers. The First Generation addons site while having to do more of a manual management will be acceptable in the short term. I have at length discussed this with Moonchild and it is better to start small and work twords something more all-encompassing over time as to do it right.

I can't see the Second Generation site being in any useable state for at least a year. Due to choices Mozilla has made we literally must re-invent the wheel here. But faced with that reality isn't it better to have something soon (like within a month) than have nothing for like a year?

Keep in mind the Pale Moon Project has evolved over time and what was once acceptable in the past isn't anymore and like Mozilla way back when alot of this stuff is new to all of us. Creating an infrastructure isn't something that can be haphazardly done and if you try and rush things you end up with Present Day Mozilla.

[...](The reason for my insistence upon those features is that Pale Moon should appear to be a polished product, not a half-baked one. The impression one would get from a half-finished site will not attract new users.)[...]

When AMO started it also wasn't as all-encompassing either. Some may see it as being "half-baked" but one must avoid the tendency to compare something that is already established (and has many problems) with something that is new.

Also, the goals for the Pale Moon addons site (like Pale Moon it's self) are not necessarily 1:1 with the goals or decisions Mozilla made with AMO.

[...]Also, regarding moderation: what to do with those extensions that track the user? Allow them as long as the user is informed of it and given an option to opt out?

Answer:

Also, in the beginning we must be careful until rules and policies can be established about what exactly gets put up on the addons site as well as staff responsibilities and the like.

EDIT: For interest sake.. Here is an original announcement for when AMO then called Mozilla Update was first launched.. It was a manual process too initially but did get management features over time and we will too http://forums.mozillazine.org/viewtopic.php?p=541078

[access2godzilla]

Also I do not know python well enough to even
attempt to run it.

You need to run it on a Linux box. The rest of it should be pretty much copy-pasting commands.

It should be possible to disable the FF marketplace, IIRC.

[Moonchild]

Also I do not know python well enough to even
attempt to run it.

You need to run it on a Linux box. The rest of it should be pretty much copy-pasting commands.

It should be possible to disable the FF marketplace, IIRC.

Maybe you can get a bare-bones framework running that way, but it will also be a behemoth even when just starting out already, and it's known to have issues and be terribly inflexible in certain areas. For people not intimately familiar with both python and the exact way it's implemented, it's like running a black box. If something goes wrong, then that's it - end of the line. Maintenance will either not exist or will have to be outsourced in case of trouble. That is just not a good choice. The difference between running it (i.e.: having the process start) and running it (having full control over it and being able to maintain and troubleshoot) is massive; I think you're thinking of the former, while Tobin is thinking of the latter.

I'm not even mentioning the fact that what is running on Mozilla servers is likely not even the same as what is available in the repos, and has been adapted and optimized, and probably bugfixed. And the differences are, of course, not known to anyone trying to set it up themselves, nor is the way Mozilla has set it up.
(and if you think that's not true, I've had a fun time trying to build the jpake key server. They have it running, but the repo was severely bitrotted and wouldn't properly build at all)

It's simply not a good solution, the same way that Mozilla Sync in python isn't a good solution and will be kicked to the curb due to its instability and resource use and also lack of correct documentation.

[access2godzilla]

Nice setup you've got there

Another thing (although probably this isn't acceptable since you've probably put a lot of effort into the addons site): how about using zamboni for the 1st generation website? I had a look at their git repo, and it doesn't seem to be as bad as the Sync server. Also time and resources are not sufficient at present, so running zamboni may be a satisfactory solution for the time being. (A custom solution should be needed in the long run though. Zamboni is somewhat complicated in nature.)

I could look into this, if you need me to.

[Moonchild]

how about using zamboni for the 1st generation website?

We looked at it and Zamboni is simply unacceptable regardless of generation of the site.

It comes with a lot of extraneous stuff we don't need, and will be a similar python mess Sync is. There's a good reason why many AMO feature requests aren't addressed or addressed very slowly, e.g. the response to one bug I filed that should have been easy if it was e.g. PHP: "Due to resource constraints we are closing bugs which we won't realistically be able to fix."
If the entire team at MozCo can't realistically fix a relatively simple feature request, we can't hope to run that black box in a custom setting, even temporary.

It'll be less work to set up something of our own in e.g. a CMS package than to try and run Zamboni.

[access2godzilla]

the response to one bug I filed that should have been easy if it was e.g. PHP: "Due to resource constraints we are closing bugs which we won't realistically be able to fix." [The] entire team at MozCo can't realistically fix a relatively simple feature request[.]

They just want to reduce their list of bugs by closing off random bugs. It has nothing to do about not being able to fix it or about "resource constraints" (which happens to be Mozilla's favourite excuse).

[Moonchild]

the response to one bug I filed that should have been easy if it was e.g. PHP: "Due to resource constraints we are closing bugs which we won't realistically be able to fix." [The] entire team at MozCo can't realistically fix a relatively simple feature request[.]

They just want to reduce their list of bugs by closing off random bugs. It has nothing to do about not being able to fix it or about "resource constraints" (which happens to be Mozilla's favourite excuse).

I'm sorry but I go by what they tell me. If they say resource constraints prevent them from implementing something, then I have to assume they are telling the truth.

More importantly it'd be running a "black box" python mess, though.

Off-topic:
The more I see about python being used for large projects, the less I am convinced the language is suited for it. It's great for small tools, but trying to implement a full server implementation in python is... well, it's shown to be as fragile as one would expect when using something for what it doesn't seem to be designed for. I think a comparison can be drawn with implementing a web server in shell scripts Sure, it works and can be done, but it won't be stable when trying to scale it.

[New Tobin Paradigm]

Nice setup you've got there

Another thing (although probably this isn't acceptable since you've probably put a lot of effort into the addons site): how about using zamboni for the 1st generation website? I had a look at their git repo, and it doesn't seem to be as bad as the Sync server. Also time and resources are not sufficient at present, so running zamboni may be a satisfactory solution for the time being. (We do need our custom solution in the long run though. Zamboni is somewhat complicated in nature.)

I could look into this, if you need me to.

Not to shoot down your idea or reject your help out of hand but I must emphatically reject Zamboni or any Mozilla python-based solution for this site. They code things with less than no regard for stability or sustainability as well as keeping in mind they have in relative comparison unlimited resources not to mention CentOS support is virtually non-exsistant.

There are next to no support systems for such a system and MozCo people who DO know about it will likely jerk us around out of spite because it isn't related to Firefox which they have historically done to not only us but to the poor sods on comm-central.

Regardless, I appreciate your offer.

[access2godzilla]

I understand the fact that Zamboni is unstable and poorly written. (Writing everything in Python? I would atleast expect them to use apache/nginx etc. for the server and use python scripts for the rest.) But creating a similar system for Pale Moon will require time and resources, something that happens not to be in abundance in the present.

Consider the fact that even the most basic of all systems will have to remember info based on extension IDs (this isn't a configuration I've seen being used on CMSes: most store by names or a custom ID), check whether the correct ID is used when developers upload their add-on; make some basic checks to determine compatibility with Pale Moon versions, allow for multiple developers to edit the extension page, and also provide the extension update information.

All this is doable, but I don't think it's possible to do it immediately. This is why I suggested the use of Zamboni at the beginning; eventually a custom solution would replace it.

Regarding support... Linux support, as has always been, limited to scouring forums and blog posts; unless you buy a copy of RHEL. In any case, usage of Zamboni will be a temporary arrangement and the need for support thus is very much reduced due to this.

[New Tobin Paradigm]

I understand the fact that Zamboni is unstable and poorly written. (Writing everything in Python? I would atleast expect them to use apache/nginx etc. for the server and use python scripts for the rest.) But creating a similar system for Pale Moon will require time and resources, something that happens not to be in abundance in the present.

Consider the fact that even the most basic of all systems will have to remember info based on extension IDs (this isn't a configuration I've seen being used on CMSes: most store by names or a custom ID), check whether the correct ID is used when developers upload their add-on; make some basic checks to determine compatibility with Pale Moon versions, allow for multiple developers to edit the extension page, and also provide the extension update information.

All this is doable, but I don't think it's possible to do it immediately. This is why I suggested the use of Zamboni at the beginning; eventually a custom solution would replace it.

Regarding support... Linux support, as has always been, limited to scouring forums and blog posts; unless you buy a copy of RHEL. In any case, usage of Zamboni will be a temporary arrangement and the need for support thus is very much reduced due to this.

The first generation add-ons site will BE a traditional site controlled and administrated by a small group of people.. We will leverage the forum for those aspects that the site initially will lack. Starting small and building and growing gradually. This is the same approach that Pale Moon its self uses.

Even though it will be a standard off the shelf CMS it will be structured carefully in a way that can be translated to a more sophisticated and custom system. (Think of a hybrid of concepts expressed and exemplified by Mozillazine and the old MozDev site with basic concepts of AMO evolving over time twords our custom solution)

This will very much be a community project and primarily the forum will be used for add-on support and communication between us and the developers to begin with.

This is fairly outlined (internally at BinOC and touched on here) of the progressive evolution of the add-ons site concept and lifecycle.

We are not MozCo and have no obligation to follow MozCo procedures historical or current. To expect us to do so or expect us to choose the arguably simple and easy path is just not being realistic.

BOTTOM LINE: Zamboni will NOT be used.. Period.

[access2godzilla]

I would expect the 1st generation site to at least support automatic updates of add-ons and identify compatible versions of Pale Moon. If that's supported, the custom system developed would be ready to go. Otherwise it would really cause some inconvenience on the part of users.

[Moonchild]

Add-on developers will have to do a few things manually and in deliberation with staff initially. The human factor and community communication is not a bad thing.
You can't create an automated system like you are expecting (and I think you may be the only one expecting this from the new site...) overnight.

Otherwise, especially without (1), it would be better to just host them in AMO

That's not possible since Pale Moon is not a supported application by their system.

[New Tobin Paradigm]

I would expect the 1st generation site to at least:

• Support automatic updates of add-ons and identify compatible versions of Pale Moon.
• Allow multiple developers to edit the properties of add-on.

If the above are supported, the custom system developed would be ready to go. Otherwise, especially without (1), it would be better to just host them in AMO (which I'm sure you don't want).

Add-on update checking is a priority but would be handled by a standalone script which when it is ready the static uri path will be altered to point to that instead of the AMO version. Which of course would also invoke the AMO version as well..

And in what category would you suggest a Pale Moon specific add-on would be posted on AMO under? AMO ONLY Supports addons for Firefox, Thunderbird, and Seamonkey (and as for the latter two saying they are treated like second class citizens would be the BEST CASE on a good day.)

Our Add-ons site will be very much a supplicant to AMO in the beginning.. As I have constantly stated, these things will come with time.

Given that I have 15 thousand add-ons to parse and narrow down to Firefox specific and then to Firefox 4+ and also cross reference the most important add-ons users state they use with those that are confirmed as broken on v25 as of today.. I have a lot of work ahead of me beyond just putting a site online. (This is the case even if we DID use that python mess.)

Though if you wish to help in this process it could allow us to get the features you want in place quicker. Without community support in this project I am left with my self and what little resources and personnel I can gather from Binary Outcast.

[access2godzilla]

You can't create an automated system like you are expecting overnight.

And that is the reason why I was insisting on the use of something that's already built, like Zamboni or Remora (assuming that it is maintained...).

these things will come with time.

I understand your point: that in some time, the addons site will get an auto-update mechanism. However, one thing particularly concerns me -- if you are using an off-the-shelf CMS, can you really have an effective auto-update mechanism? Achieving such a thing would require that each addon listing be tied with its extension ID and I haven't seen such configurations where CMS pages are tied to an user provided parameter (except for the page title). It would also require the system refuse uploads of addons with a different ID than the one specified.

I have 15 thousand addons to parse

Parsing 15000 addons by hand is not a practical solution. It obviously has to be automated in some manner. It also appears that except for SDK addons, most other addons work fine, so automation may not be entirely unacceptable.

[Moonchild]

You can stop insisting on Zamboni or Remora.

The issue with using either of those systems (or something similarly complex) as a temporary solution is that they take a large amount of work to set up and will demand an equally large amount of work from add-on developers to set up their profile, submit add-ons, get vetted, etc., which will then all be thrown away again. That work is best put in a permanent (long term) solution instead of trying to get this temp thing to run stable (with all the extra fluff we don't need to begin with).

[New Tobin Paradigm]

We are gonna continue to run in circles. Now, a2g, if you would like to offer any help and suggestions or time to the current game plan for the add-ons site please feel free to open up a new thread (or use one of the existing ones).

tl;dr All that is to be said has been said and it is appreciated but it won't change the reality of the situation or direction.

Topic Locked.