HTTP Multipart Downloader utility

[cynagen]

Ever play on a server like SwitchCraft? Ever work with large files that can't be downloaded via HTTP.GET because they're larger than the size limit set on the server? Have a ridiculously low size limit set and still wanna get that one large file loaded? Well provided you have the space (extra large disk sizes or Computronics Tapes), here's your utility!

WARNING: This downloader is inaccurate and has a very slim chance to actually miss between 1 and X bytes of data where X is the number of splits. To try to reduce the chances of the multipart download from failing, reduce the number of splits or keep splits to even numbers!

HTTP Multipart Downloader

Code Select Expand

pastebin get YqqeA0x9 multipart.lua
Load using:

Code Select Expand

multipart=require("multipart.lua")Usage:

Code Select Expand

multipart.get(url [string], headers? [table], binary? [boolean])Returns:
On success: Binary/text data stream
On error:

-1	Generic HTTP_FAILURE
-2	No Accept-Range support detected before receiving Content-Range headers, server likely does not support ranging.
-206	Received an unexpected HTTP response code (we're expecting 206 Partial Content, hence -206)

I'm not really going to go into more detail, this is a pretty basic library, but adds some sorely needed functionality (I get why it was requested.) When provided with a valid link, it will do some HTTP HEAD magics to figure out the file size and such, then split it up according to multipart.splitNum and multipart.maxPartSize. Defaults provided work well enough, and maxPartSize has been configured for SwitchCraft's limits, though can be adjusted at any time by redefining the value before running .get()! Additionally, if the file ends up being larger than .splitNum*.maxPartSize, the script will automatically increase the .splitNum automatically to strictly comply with .maxPartSize, so no need to worry about doing your own calculations for number of parts. (You can set .splitNum to 1 to just let the library automatically expand as needed.)

Looking forward to feedback!

Received some feedback from Wojbie to advise that this library will not function if a server does not support the Range feature either by choice or lack of. You have been so informed. I have updated the code to set up a conditional to ensure we know we support ranging before continuing. To that end I have also added a list of the return codes other than your expected output.