Plugin servers / getting plugins to a system

godboko71

  • Fence accomplished!
  • Hello
  • Posts: 361
Re: Plugin servers / getting plugins to a system
« Reply #15, on October 17th, 2011, 05:01 PM »
*Upload from Computer* Could be a Jquary type pop in and be a separate page as a fall back when JavaScript is disabled or not working correctly. Just a thought. Either way though I think it can be its own page.
Thank you,
Boko

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Plugin servers / getting plugins to a system
« Reply #16, on October 17th, 2011, 05:13 PM »
Quote from Arantor on October 14th, 2011, 01:24 AM
OK, let's recap.

.zip vs .tar.gz.
There are more rugged solutions than read_zip_data() out there, one of which is ZipArchive. It's not the only solution, however, and it won't be that hard to find such a solution.

My theory of using only one file type in general just means that it simplifies the process and the code, and minimises the overall number of hassles people have, IMHO.
Sure.
OTOH, the only argument I have against this -- the plugin server would see a significant bandwidth increase if it can't server gzipped content...
Quote
Should uploading plugins from your computer be a separate page to downloading from a plugin server?
I'd tend to put everything into the same page, but really that's because it's already the case in SMF.
Other than that... I don't really bother either way.
Quote
Should a plugin be able to list the plugin server or servers where it can be found?
Hmm... I don't know :^^;:
It would have to be seen, first, if anyone creates 'independent' plugin servers that host plugins by multiple users.
Other than that, it'd be just simpler for us to provide RSS feeds on the Wedge plugin server with an author filter. i.e. people could be notified when their favorite authors release new plugins or new versions. Totally missing from the SMF customization site, eh...
Quote
(Just a quick note, I took a look today at how WP handles plugin updates on the fly. Interestingly, it's practically the same as I'd outlined Wedge would have to be doing, that the download occurs, the old plugin is disabled, the new one enabled, and the old plugin then deleted. I still think the user should perform those steps, though, rather than entirely automating it. I don't know. Might have to experiment with how well that works out for us.)
We could add a 'noob-friendly' option to do everything automatically.
If it fails --> manual processing.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Plugin servers / getting plugins to a system
« Reply #17, on October 17th, 2011, 05:25 PM »
I'm really not that enthusiastic about putting a jQuery widget in for something like that. For uploading multiple plugins, maybe, but anything more than that is IMO unnecessary, as this is something I don't want to spend vast amounts of time on, knowing that most users won't upload multiple plugins at once, probably won't even upload plugins all that often since it should be possible to handle from downloading better...
Quote
Sure.
OTOH, the only argument I have against this -- the plugin server would see a significant bandwidth increase if it can't server gzipped content...
Well... plugins would still be compressed, and you have to get to a largeish package before it makes a big difference. SD 2.0 for example is 553K for ZIP vs 404K for .tar.gz, and mostly that's because it has a lot of files in it, of which most won't compress all that well.

The other thing is preference. The majority of plugin authors are likely to be on Windows where making a .zip file is easier.[1]

The alternative is forcing .tar.gz across the board, which is smaller generally. Our plugin server could unpack zip and repack as tar.gz on upload. Those running custom servers would have to just deal with it, I guess.
Quote
I'd tend to put everything into the same page, but really that's because it's already the case in SMF.
Other than that... I don't really bother either way.
Well, if it remains as one page, it desperately needs renaming. I don't really have a problem with renaming it, just that my first instinct was to make it a separate page, but I'm increasingly agreeing with the view that it doesn't need to be.
Quote
It would have to be seen, first, if anyone creates 'independent' plugin servers that host plugins by multiple users.
Other than that, it'd be just simpler for us to provide RSS feeds on the Wedge plugin server with an author filter. i.e. people could be notified when their favorite authors release new plugins or new versions. Totally missing from the SMF customization site, eh...
Well, this is where it gets into the technicalities of providing support. I suggested doing it as a REST style but I don't think that's viable in the long run. In fact, I'm sensing it would have to pretty much be a SOAP-type request to the server to cope with everything.

What that ultimately means is that the requester sends an HTTP POST with the body being a block of XML. (Or POST vars. Doesn't really matter. The key point is that you make a POST with one or more variables attached.) Then you get a block of XML back in some form.

SOAP formalises this process, and I don't think we need go quite that far, but certainly it needs more than a simple URL request, since the process has to account for various kinds of filtering.
Quote
We could add a 'noob-friendly' option to do everything automatically.
If it fails --> manual processing.
I wasn't really going to do anything else. It's not like it's that hard to automate that process, especially since the user will have likely had to provide their FTP/SFTP details at some point (could even do it ahead of time so that the details are known before even trying to download)
 1. Don't flame me and tell me that it is or isn't easier on Linux. I honestly don't care. Most people here will be using Windows on the desktop, and it's directly built into the file manager and has been for years. I don't recall the same finesse on Linux distros.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

spoogs

  • Posts: 417
Re: Plugin servers / getting plugins to a system
« Reply #18, on October 17th, 2011, 05:46 PM »
Quote
Well, if it remains as one page, it desperately needs renaming.
This is what I've been thinking all along^^ the only real issue is the name of the page. I can't speak much as how things should be handled with the zip vs gz stuff but as far as the layout presented to the user 1 page is best. SMF's layout isn't really that bad (though it could use a face lift) it's really just the name of the page that's an issue.

IIRC live's theme Grace actually went a step further by providing the options for acquiring packages right above the list of mods which I thought was interesting looking.

As far as the name of the page... maybe View/Install Plugins or Acquire Plugins or something of the sort I suppose.
Stick a fork in it SMF

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278

spoogs

  • Posts: 417

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Plugin servers / getting plugins to a system
« Reply #21, on October 17th, 2011, 06:37 PM »
More to the point, neither have I, and I think I can safely (and very legitimately) say I know SMF's package system better than most people. In fact, I'm still not entirely clear what actual use it does have. If something is serving packages through URL, you either download it to your PC and upload it, or it's going to be a package server anyway.

The *only* scenario I can envisage is for the case of a package not on a package server and you're using a phone to add a plugin. Which, frankly, is asking for trouble anyway.

spoogs

  • Posts: 417
Re: Plugin servers / getting plugins to a system
« Reply #22, on October 17th, 2011, 06:43 PM »
I actually typed up almost the same thing but edited it to the simple 1-liner above :P

I agree it's a more natural reaction to download the pack as you're already there than to copy the url. The phone option hadn't crossed my mind but then again I've never even attempted to get the to ACP from my phone anyway.

If that option is going to be kept it can probably be made the last option on the page then.

Dr. Deejay

  • Happy new year all!
  • Posts: 118
Re: Plugin servers / getting plugins to a system
« Reply #23, on October 17th, 2011, 06:53 PM »
I use it often when I'm too lazy to download the packages to my pc and find the according package. But even if it's going to stay (I'm guess it will be removed because most people don't use it), it sometimes just don't work

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Plugin servers / getting plugins to a system
« Reply #25, on October 17th, 2011, 10:03 PM »
Quote from Arantor on October 17th, 2011, 05:25 PM
The other thing is preference. The majority of plugin authors are likely to be on Windows where making a .zip file is easier.
Upload zip -> Wedge server receives zip -> Wedge server repackages it as tgz -> Wedge server serves tgz when sending data across servers, and serves zip on request (manual download.)
Quote
The alternative is forcing .tar.gz across the board, which is smaller generally. Our plugin server could unpack zip and repack as tar.gz on upload.
Just as I suggested, lol... ;)
Quote
What that ultimately means is that the requester sends an HTTP POST with the body being a block of XML. (Or POST vars. Doesn't really matter. The key point is that you make a POST with one or more variables attached.) Then you get a block of XML back in some form.
Isn't that what xhr is all about...? :P

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Plugin servers / getting plugins to a system
« Reply #26, on October 17th, 2011, 10:10 PM »
Quote
Upload zip -> Wedge server receives zip -> Wedge server repackages it as tgz -> Wedge server serves tgz when sending data across servers, and serves zip on request (manual download.)
Yup, that way we'd be providing tgz across the board. Though I don't see a need to offer both, just do one.
Quote
Isn't that what xhr is all about
On the browser side, yes, but I was thinking about doing it on the server instead, to avoid having to cope with cross-domain requests where it won't be sending everything in the URL...
Re: Plugin servers / getting plugins to a system
« Reply #27, on October 17th, 2011, 11:58 PM »
OK, so I've thought about this a bit more, after taking some time to review fetch_web_data a little bit, since that's quite a neat little function.[1]

I can't quite make up my mind whether to use HTTP headers as part of the request (and make it a GET) or put the contents in a POST body, either should be doable, though I guess using a GET is more semantically accurate.[2]

In other news I'm thinking of expanding fetch_web_data to support getting an arbitrary number of bytes at the start of a file and making it use the Range headers. The reason? Oversized, non attached images. When someone posts using those, the URL is fetched, the entire image is loaded into memory and its size established with the GD image info functions... but that means downloading the entire image. For the purposes of getting its size, getting the first 1.5K should be sufficient and we can do it ourselves from the file headers.
 1. I mean, in an ideal world, it would be using cURL and no hosts would be putting stupid restrictions in that necessitate such measures, but in a cURL-less, stupid-host-filled world, it's pretty neat.
 2. Especially as GET methods are supposed to be idempotent.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Plugin servers / getting plugins to a system
« Reply #28, on October 19th, 2011, 10:26 AM »
Re: cUrl, here's a 3-year-old post by yours truly about it :P
http://www.simplemachines.org/community/index.php?topic=282969.msg1858080#msg1858080
Latest mod version: http://custom.simplemachines.org/mods/index.php?mod=1569

Use GET if you feel like it's better. You know you want it :P

Range header support in fwd would be cool. aeva_getMedia() already supports it and that's pretty cool :) I thought SMF had it in the first place... My mistake.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Plugin servers / getting plugins to a system
« Reply #29, on October 19th, 2011, 10:59 AM »
Well, if I add cURL support (by default) to fetch_web_data, I'll be sure to add in the facility to add arbitrary headers which means range support is a step away if anything wants it. The plugin manager will need that if I use GET, and then for the places outside media which use it, it's pretty much a one-liner to deal with.

Debating whether or not to convert fetch_web_data into a class so that setting things like URL, headers etc don't make for a very long function call, though they would make for 2-3 lines of calling instead of the current 1-2 (allowing for inclusion of Subs-Package as it stands right now)