Titanium Community Questions & Answer Archive

We felt that 6+ years of knowledge should not die so this is the Titanium Community Questions & Answer Archive

Large file download on mobile

Hi. I'm tearing my hair out trying to figure this one out…

I'm trying to download a 4 meg(ish) file to local (mobile) storage from a remote server. The following code works fine for smaller files, but with larger ones I get an outOfMemory exception. This is because Ti saves the data to memory before committing it to the file… which is no good for me.

My code follows (cut down, but you get the idea)…

var videoFile = Titanium.Filesystem.getFile(localFilename);
var httpClient = Ti.Network.createHTTPClient();
httpClient.onreadystatechange = function(){
    if(httpClient.readyState == 4){
        videoFile.write(httpClient.responseData);
    }
};
httpClient.open("GET", theRemoteFileURL);
httpClient.send(null);

Like i say… the above works fine for small files, but gets messy for larger files.

I found the following post… http://developer.appcelerator.com/question/26001/copying-a-file-from-a-server-to-your-harddisk
but this is for Ti Desktop… i tried the code anyway and just got nothing but trouble. It would seem that Filesystem.getFileStream isnt implemented in Mobile, nor is HTTPClient.receive. (I would have been able to see this without testing if I at all trusted the documentation to be in any way reliable, btw(!)).

I was hoping to maybe set an interval to Append to the file every X seconds and then maybe flush the HttpClient (to clear memory) - but from what I've seen I can't append to a file, nor flush the stream.

Does anybody have any info that can help?

Also, is there anywhere else I can get better documentation… I use KitchenSink when possible but the demos sometimes arent extensive enough.

Cheers.

— asked June 15th 2010 by Steffan Luczyn
  • download
  • filesystem
  • httpclient
  • mobile
0 Comments

9 Answers

  • I'm still encountering this problem in Android, so I dug through the Titanium 1.5.1 source for the HTTPClient, and came up with a (not entirely elegant) work-around.

    Here's an example onload function for the HTTPClient:

    xmlHttpObj = Ti.Network.createHTTPClient();
    xmlHttpObj.onload = onSuccess;
    ...
    filename = "filename.txt";
    ...
    
    function onSuccess()
    {
      Ti.API.info("Successfully downloaded file");
      Ti.API.info("Saving file");
    
      // This method results in Out of Memory Error for large files
      //var f = Ti.Filesystem.getFile(
      //        Ti.Filesystem.getExternalStorageDirectory(),
      //        filename);
      //f.write(xmlHttpObj.responseData);
    
      // Use the fact that TiHTTPClient internally saves to a
      // file to move that temporary file where we want it.
      // responseData is a TiBlob object, which can get data from
      // multiple sources, including files (TYPE == 1).
      if (xmlHttpObj.responseData.type == 1)
      {
        var f = Ti.Filesystem.getFile(
                   xmlHttpObj.responseData.nativePath);
        var dest = Ti.Filesystem.getFile(
                      Ti.Filesystem.getExternalStorageDirectory(),
                      filename);
        if (dest.exists)
          dest.deleteFile();
        f.move(dest.nativePath);
      }
      else
      {
        var f = Ti.Filesystem.getFile(
                   Ti.Filesystem.getExternalStorageDirectory(),
                   filename);
        f.write(xmlHttpObj.responseData);
      }
      // Do not access xmlHttpObj.responseData after this point,
      // since the file backing it might have been moved in the
      // previous step.
    
      Ti.API.info("Successfully saved file");
    }
    

    There needs to be at least twice the file size of space left on the device, since Titanium's file.move() implementation copies the file before removing the original.

    And just to note, I've only done some simple testing with this code. I can't guarantee that it will always work, although I don't know why it wouldn't!

    — answered February 14th 2011 by Jens Taylor
    permalink
    3 Comments
    • Jens, I just did a test download of a 28MB file and it worked perfectly! Thanks very much. I just voted this up. I had not dug into the source code yet because with iOS solved, this had not escalated to the top of my priority pile yet.

      — commented February 14th 2011 by Doug Handy
    • had the same problem, this seems to work fine, thanks. however, i don't understand how Ti determines the responseData.type. does it depend on size? i've observed that smaller files have a responseData.type == 2. anyway. –

      — commented March 24th 2011 by u no
    • Internally, the HTTPClient stores the response to a request depending on the headers it receives for the response's size. If it's under a certain (configurable?) size, it's simply stored in memory. If it's larger, it gets stored to a file.

      The HTTPClient then passes either the memory or the file location off to a TiBlob, which uses that as backing. So in the small file size case, the TiBlob is backed by memory. In the large file size case, it's backed by a file.

      — commented March 24th 2011 by Jens Taylor
  • https://appcelerator.lighthouseapp.com/projects/32238/tickets/1019-httpclient-crashes-app-with-substantial-downloads

    it's a known ticket apparently so you have to wait for it to be resolved or give up Titanium mobile

    — answered June 15th 2010 by Guillaume LAFOUTRY
    permalink
    1 Comment
    • I was hoping that there may be a workaround i.e. writing to the file in chunks and clearing the recievedData in httpClient.

      — commented June 15th 2010 by Steffan Luczyn
  • I am stuck on this as well. Being able to download large file to iphone is a very important feature of my app.
    I am now reading the module guide to write a module to handle this. So sad to go back to that damn objective-c again.

    — answered November 27th 2010 by jason hu
    permalink
    0 Comments
  • I've solved this problem in other apps (not titanium, but similar issue) by implementing a REST service that allows me to get chunks of the file via service calls. Then I make consecutive calls to the service and append the data on each chunk received. It can be time consuming to do, but you get other benefits as well like the ability to continue downloads where they left off if a connection is dropped, etc.

    — answered November 27th 2010 by Javier Muniz
    permalink
    1 Comment
    • i thought about this in the beginning, but as Steffan L said, Titanium's File method doesn't allow appending data to an existing file. So that wiped out this possibility too.

      I made some progress on my first module code, i can invoke a method and fire event, however, the asynchronous httpconnection doesn't callback inside the module. and I feel helpless here with so little information available in this module guide.

      — commented November 27th 2010 by jason hu
  • finally my prototype module for downloading large file is working. I still need to add more whistles and bells to it.
    it will download file to disk, report progress, and resume download
    I will publish it once I get it done.

    — answered November 28th 2010 by jason hu
    permalink
    1 Comment
    • Hey Jason, any progress on this?

      — commented December 11th 2010 by Scott M
  • Jason,

    This sounds wonderful. Which platform(s) are you targeting with your module? Is it still possible to run the project in the simulator while the module is named in the project's tiapp.xml file?

    — answered November 28th 2010 by Doug Handy
    permalink
    2 Comments
    • I only tested it in iPhone 4, but I need it to run in android later, yes, it can be simply added to tiapp.xml just like the admob plugin someone published.
      The module is pretty simple to use and looks like everything works as expected, even the resume downloading.
      so far, I only tested it in simulator with "titanium run" command, can't appreciate titanium more, they made it so easy, all the troubles I had were my own dumb mistakes.
      I will publish the module zip later once I fully test it in device mode.
      I know a lot of people are struggling with learning titanium, I am one of them, so I hope this can ease some pain for you guys.

      — commented November 28th 2010 by jason hu
    • I'd love to see this too. I'm finding this problem with JSON responses larger than 500k. Any advice on workarounds the meantime?

      — commented December 2nd 2010 by Stephen Baker
  • It looks like there's a fix in 1.5.0 for this:

    https://github.com/appcelerator/titanium_mobile/commit/05ab7461dab9a38200a27d082389d2885be951f8

    — answered December 11th 2010 by Scott M
    permalink
    0 Comments
  • Hey Did anyone ever solve this issue? Jason, have you posted your module anywhere for us to play with? Thanks

    — answered January 28th 2011 by Dave F
    permalink
    0 Comments
  • Checkout the file download example in the latest Kitchen sink. You use a xhr.file = (path); after open and before send and it will automatically stream the file to that path, this will stop you running out of memory.

    — answered June 9th 2011 by Mark Henderson
    permalink
    4 Comments
    • According to the API docs as of 1.7.2, this is only applicable to IOS.
      This leaves the android download problem.

      — commented July 27th 2011 by Stephen Feather
    • True, but see the top reply with 2 votes. It shows sample code for how you can achieve the same effect in Android, without using the .file property available for iOS.

      — commented July 27th 2011 by Doug Handy
    • Doug, agreed.

      We are still writing or hunting for work arounds for an incomplete (read non-equal) android implementation. This work around is STILL required (was written 6 months ago) after 1.5.0, 1.6.0, 1.7.x Sdk releases, and you know what? It still seems to be required in 1.8.0. (and no one start preaching about beta junk, i loaded it to see if it was fixed).

      — commented July 27th 2011 by Stephen Feather
    • Don't shoot the messenger :)

      I was just trying to help and point out that at least a work-around exists in this case. My SWAG is that it will remain this way until a Pro or Enterprise subscriber raises it as an issue.

      And frankly, I am OK with that in this case simply because I do know of a work-around. So while parity between the OS versions is a laudable goal, I personally am more concerned about things where there is no work-around.

      Maybe that is just me.

      — commented July 28th 2011 by Doug Handy
The ownership of individual contributions to this community generated content is retained by the authors of their contributions.
All trademarks remain the property of the respective owner.