A (potential?) solution for SDK extensions...

fillerup · Unread post by **fillerup** » 2017-01-08, 14:05

do you have any timeframe for when the beta will be released? 27.0.3's inability to play youtube videos consistently is driving me insane

half-moon · Unread post by **half-moon** » 2017-01-08, 14:43

fillerup wrote:do you have any timeframe for when the beta will be released? 27.0.3's inability to play youtube videos consistently is driving me insane

I have no issue playing youtube videos.

fillerup · Unread post by **fillerup** » 2017-01-08, 15:24

it's a linux specific issue related to gstreamer, if you recall the thread where previous thread where i asked about your computer OS situation

Unread post by **Moonchild** » 2017-01-08, 15:32

Please keep this thread on-topic!

Unread post by **Moonchild** » 2017-01-08, 15:40

JustOff wrote:In view of the above, I would propose to take the next logical step: return the path to PMkit to its original location: 'resource://gre/modules/commonjs/'. This will simplify the jpm usage (autogeneration of bootstrap.js) and will give a good sign to developers that they are welcome to attend here in contrast with Mozilla which throws them out. If we will use the common path (as it was in Pale Moon 24.x-26.5) developers will be able to smoothly adapt their add-ons to Pale Moon and new users will be able to find their usual extensions more easily.

The problem is that changing the gre path will break browser internals, and especially the devtools.
"smoothly adapting" and "thinking a GUID change is all that's needed" (transplanting) are two different things, too. If people are serious about this being taken over from Mozilla, then someone should take JPM and change it to add support for Pale Moon/PMkit for auto-generation. That would be a task to be performed by the tool, not the target application it aims to create extension packages for. I'd gather that adding an option to JPM to target Pale Moon would be 100x less involved than trying to change the internal paths in the browser back to the (too ambiguous IMO anyway) "commonjs" naming.

kizo07 · Unread post by **kizo07** » 2017-01-08, 16:25

Although YouTube working excelent here, since my childhood I haven't was so excited over something, as I'm now over 27.1. Thanks to Just Off, I feel it's gonna be as a candy box...flavor after flavor, day after day...mmm. Pale Moon user, Just Happy user

Thanks again, and in advance, mr. Add-On!

...♪♫ ...Dancing In the Moonlight, la,la,tra,la,la...yeah...♪♫♥♫♪

JustOff · Unread post by **JustOff** » 2017-01-09, 01:19

Moonchild wrote:The problem is that changing the gre path will break browser internals, and especially the devtools.

I have carefully studied the full history of integration of Jetpack in Tycho and then in Pale Moon 27, all current dependencies with devtools and with the rest of the code, and I can say with full responsibility that my proposal is safe and requires minimal changes.

I have applied the changes and checked that all is working properly for several days.

"smoothly adapting" and "thinking a GUID change is all that's needed" (transplanting) are two different things, too.

You are right, but with the latest PMkit updates Pale Moon 27 became more compatible with SDK than ever before, and even more compatible than with other (non SDK) add-ons from AMO that are allowed to run even without proper GUID. Yes, I mean that the chance to successfully run the SDK-based add-on in Pale Moon is greater than for classic extensions for Firefox, while all of them will probably require some adaptation.

If people are serious about this being taken over from Mozilla, then someone should take JPM and change it to add support for Pale Moon/PMkit for auto-generation. That would be a task to be performed by the tool, not the target application it aims to create extension packages for.

Sorry, but I think that isn't best idea to make a fork just for the one string patch. But that's not the point.

I'd gather that adding an option to JPM to target Pale Moon would be 100x less involved than trying to change the internal paths in the browser back to the (too ambiguous IMO anyway) "commonjs" naming.

Don't get me wrong, I have not intoxicated by success with PMkit and I'm not trying to impose my view just because you allow to argue. I truly think it's very important to ensure highest possible compatibility of extensions that we can provide without negative impact on the project itself. And in this particular case the new benefits will cost literally nothing.

We have about one year until Mozilla will make AMO completely useless, but we can't expect that all developers will be so kind (or so wise) to support completely separate version for Pale Moon during this time. Let's take Self-Destructing Cookies as an example. For years it has a part of code specially targeted to Pale Moon that allowed to run it in both browsers. I'm pretty sure that if this was impossible to implement then Pale Moon users simple were ignored. There are many other such add-ons, not only SDK-related. Yes, this is not our problem, the only users will suffer. So let's be wiser: if we will use common path to PMkit we will show our strength, not the weakness.

I could give many more examples and arguments, but I don't want to take your time, as and so my posts here are usually too large. I just kindly ask you once again to consider the option to return PMkit to the commonly used path and I promise not to bother you more on this subject until the next release.

dark_moon · Unread post by **dark_moon** » 2017-01-09, 09:29

I dont have the technical knowledge but i dont see any reason why Pale Moon doesnt use the great work which JustOff make.
I also would say we even need it because of addon compatibility. Yes its not the work for browser to work with addons, but in this time it is needed and would give Pale Moon a great power and also show many addon developer a easy way.

JustOff · Unread post by **JustOff** » 2017-01-09, 10:59

dark_moon wrote:I dont have the technical knowledge but i dont see any reason why Pale Moon doesnt use the great work which JustOff make.

You are wrong, all my patches related to extensions and PMkit are already in master branch of Pale Moon, we are just discussing the most appropriate way to align with external world.

I also would say we even need it because of addon compatibility. Yes its not the work for browser to work with addons, but in this time it is needed and would give Pale Moon a great power and also show many addon developer a easy way.

Thanks for your support, but the task is to keep the balance, this is why we can only advise, while Moonchild is one who is in charge for decisions.

Unread post by **Moonchild** » 2017-01-09, 23:30

JustOff wrote:You are right, but with the latest PMkit updates Pale Moon 27 became more compatible with SDK than ever before, and even more compatible than with other (non SDK) add-ons from AMO that are allowed to run even without proper GUID. Yes, I mean that the chance to successfully run the SDK-based add-on in Pale Moon is greater than for classic extensions for Firefox, while all of them will probably require some adaptation.

The former is good news.

The latter is something you have to understand is directly related to how limited SDK extensions are. If you are abstracting code from the underlying technology, then any alignment of the abstraction layer will automatically align the extensions building on it.

JustOff wrote:I have applied the changes and checked that all is working properly for several days.

Very good!

JustOff wrote:All this was initially introduced by this commit.

Of note, that is not a commit I was directly involved in. It is also part of a larger PR that does a lot more.

Tobin, can you chime in here please, is reverting that commit and aligning the internal gre paths with Firefox going to have any unforeseen snags or not?
Logically, since this is just an abstract path determined at the manifest level(?), it wouldn't really matter where the actual JS/JSMs live...

New Tobin Paradigm · Unread post by **New Tobin Paradigm** » 2017-01-10, 03:36

I want to for the record say that I am completely against this SDK revival. I feel it is a step backwards and sidestepping the issue that the SDK is restrictive, unpredictable, and largely unmaintainable and utterly inferior to just using the Toolkit directly.

I have spent several years watching how the SDK has utterly destroyed add-ons and knowledge of how they actually work with the browser.

As for the gre paths.. it is completely abstract as the sdk is.. The internal gre paths don't matter as that is merely set on the bootstrap level. I changed the path in the hopes of it being eventually removed but merely retained as a couple of internal functions require it.

I wanted to see an end to this nonsense in its entirety not another level added on top of it. If you want to realign it and basically fully reinstate the sdk or mutate it into yet another thing with this so called PMKit I cannot stop it not merely because I am not in a position to do so anymore but obviously a misguided sector of the community wants it so badly because no one outside maybe wost can be bothered to do proper extensions that replace the handful of "required" sdk ones.

But to answer your question on a purely technical level.. Assuming the general block for Firefox SDK Add-ons remains in xpinstaller you can if desired change the internal path back. But you must make sure you change all references back. I don't understand the reason for reversion though except to push the SDK back into Mainstream Pale Moon development legitimizing it again.

If this is not answering the question please tell me and I will try and rephrase.

Unread post by **Moonchild** » 2017-01-10, 08:46

I wanted to see an end to this nonsense in its entirety not another level added on top of it. If you want to realign it and basically fully reinstate the sdk or mutate it into yet another thing with this so called PMKit I cannot stop it not merely because I am not in a position to do so anymore but obviously a misguided sector of the community wants it so badly{...}

The problem isn't necessarily that it is a misguided direction (because that really is the case, I agree), but rather that it is a part of the Firefox legacy. A legacy that has had several years of extension development build on it. A legacy we can't just arbitrarily ignore, no matter how undesired it is.

Remember, the decision was not made lightly and mostly spurred on by the technical hurdles in place for not having a front-end to talk to through the SDK. That is very much true for the SDK extension support we offered in v26 -- cfx-based ones -- but having looked at the JPM way of handling things the abstraction was taken a level further which actually makes it less tied-in with how the FE works, and provided the interface in the actual jetpack modules translates well to our FE, we can shim what's needed since we will be providing the necessary code, not the extension.

There is obviously community support to make this alignment happen, and desire by end-users to have this (although that is secondary, no matter how many ghostery users there are). As usual this is a balance decision, and it will not come without cost.

For example, if we offer this PMkit extension framework to extension developers it should still be considered a non-evolving framework. It will be there only to "make legacy SDK extensions work" and to offer a smallest-hurdle transitional technology to native Pale Moon extension development as opposed to enforcing a rewrite immediately (which is likely not going to happen; extension devs are more likely to abandon or rewrite in WebExtensions format if feasible), but it will not be there to offer an evolving or improving framework. SDK/PMkit extension development will be and remain deprecated in our view.

Another drawback is that this will, in effect, freeze internal devtools development. If there are changes required to the jetpack modules for them, then that can't be catered to and whomever does devtools development will have to make independent solutions not reliant on the jetpack modules. For me this is less of a concern since "they work" and are not owned by anyone at the moment (aside from platform update bugfixes being put in).

This will also require dedication to support it and documentation to help extension developers understand the differences. I believe we currently have this community effort being put forward and it will explicitly rely on continued support by the people who started this motion. Our core developer team can not, at any point in time, take on extension development/troubleshooting tasks for the jetpack modules since they are provided as-is, and in an otherwise unsupported fashion by us.

But to answer your question on a purely technical level.. Assuming the general block for Firefox SDK Add-ons remains in xpinstaller you can if desired change the internal path back. But you must make sure you change all references back.

That's good then! As far as I'm concerned we should do this then and not make the hurdle "to make it work" larger than it necessarily has to be.

JustOff · Unread post by **JustOff** » 2017-01-10, 09:27

Matt A Tobin wrote:I want to for the record say that I am completely against this SDK revival.
...
As for the gre paths.. it is completely abstract as the sdk is.. The internal gre paths don't matter as that is merely set on the bootstrap level.

While I totally disagree with your position, I must to say that I highly appreciate your honest professional answer on topic.

Moonchild wrote:This will also require dedication to support it and documentation to help extension developers understand the differences. I believe we currently have this community effort being put forward and it will explicitly rely on continued support by the people who started this motion.

I can't speak on behalf of the entire community, but I personally intend to continue to give the necessary attention to this subject.

That's good then! As far as I'm concerned we should do this then and not make the hurdle "to make it work" larger than it necessarily has to be.

Thank you so much for your balanced decision! I'm ready to open the corresponding PR as soon as #818 will be resolved, that will unlock the tree.

Unread post by **Moonchild** » 2017-01-10, 09:42

I've had a chat with Tobin directly and there's another solution that will come at less cost for the core (but is potentially more complex for extension developers):

An option previously not considered here would be to have the extension SDK in a separate framework/foundation extension that defines the commonjs paths with its own copy of the SDK modules. Tobin apparently looked at that when he was still directly involved in development but at the time that wasn't feasible (with cfx-style integration, for the same reasons we dropped it entirely), but with the later method of the SDK supplying everything, in lieu of requiring each extension to include their own copy of the SDK modules, it should be possible.

That would cut away the cost for the core, and would prevent devtools from being locked in their current state and being directly affected. It could be a middle ground; the downside there is that end users will have to install the foundation extension and any extensions building on it should check if it is available/installed before trying to use it (and providing a message if not).

[10:25] mattatobin: I think JustOff should just include a foundation extension that has his own modified copy of the sdk and should leave what is in toolkit alone.. it serves devtools and nothing else
[10:26] mattatobin: but that is my opinion
[10:26] mattatobin: in other news I am having fun programming this solution for better xref management
[10:26] wolfbeast: You mean providing his own resource/commonjs set?
[10:26] mattatobin: something I am gonna use personally as well
[10:26] mattatobin: yeah
[10:27] wolfbeast: Its a good suggestion.
[10:27] wolfbeast: Offer it up.
[10:27] mattatobin: as it would not rely on browser releases
[10:28] wolfbeast: It would make having that installed a prerequisite for any such extensions.
[10:28] mattatobin: a solution somewhere between bundling it and every extension having a copy
[10:28] mattatobin: middle ground
[10:28] mattatobin: we all love it

So before rushing ahead with this with PRs, have a look at this option and see if it's feasible?

JustOff · Unread post by **JustOff** » 2017-01-10, 10:31

Moonchild wrote:An option previously not considered here would be to have the extension SDK in a separate framework/foundation extension that defines the commonjs paths with its own copy of the SDK modules. [..] It could be a middle ground; the downside there is that end users will have to install the foundation extension and any extensions building on it should check if it is available/installed before trying to use it (and providing a message if not).

That was exactly my first idea when I began to study the possibility of running SDK-based add-ons in Pale Moon 27.0.0b1. I played with it for several days, tried to load the framework both as XUL and restartless, but unfortunately the attempt to get a workable solution has failed, because we can't control the order of starting add-ons and some init procedures are running in parallel. And even if we could resolve this (which will require huge intervention into the browser core), the possible result will definitely lead to slower startup not only for add-ons, but for whole application.

Unread post by **Moonchild** » 2017-01-10, 10:54

JustOff wrote:unfortunately the attempt to get a workable solution has failed, because we can't control the order of starting add-ons

Right, because it's all async out of necessity.

Ohwell! It was a good idea, if unrelated people came up with the exact same that only confirms it

Pity it isn't feasible.

Issue #818 was merged, so you're free to continue.

New Tobin Paradigm · Unread post by **New Tobin Paradigm** » 2017-01-10, 16:33

I am not sure init order would be an issue.. If the foundation extension is toolkit not bootstrap and merely provides resource uri in chrome.manifest it should be there by the time any bootstrap extensions load up. Because toolkit chrome.manifest loading is one of the first things that happen isn't it so that resources are available as soon as possible.

EDIT:

JustOff for the sake of humoring me could you create a companion bootstrap extension that loads resources from this modified copy of jetpack contained in this toolkit extension and see if as a Toolkit extension with merely a resource entry in chrome.manifest indeed has an ordering problem?

The resource URIs are:
resource://jetpack/commonjs/
resource://jetpack/sdk/

Fedor2 · Unread post by **Fedor2** » 2017-01-10, 17:40

Is the minversion 27.1 of jetpack-foundation-1.0a1.xpi means that it can be used only on 27.1 and not on 27.0.3?

New Tobin Paradigm · Unread post by **New Tobin Paradigm** » 2017-01-10, 19:19

Right so this is useless for anyone outside of a potential test.. Why have 6 people downloaded it..

JustOff · Unread post by **JustOff** » 2017-01-10, 19:23

Matt A Tobin wrote:JustOff for the sake of humoring me could you create a companion bootstrap extension that loads resources from this modified copy of jetpack contained in this toolkit extension and see if as a Toolkit extension with merely a resource entry in chrome.manifest indeed has an ordering problem?

~~Here it is: infocon-1.3.1-ft.xpi,~~ it works well if manually loaded, but not on browser restart ...

Pale Moon forum

A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...

Re: A (potential?) solution for SDK extensions...