Stream large file (database) to server

You must Login before you can answer or comment on any questions.

Android isn't very good at base64Encoding a large file (15-20mb). My app ends up with large databases as it's a data intensive app that has to work offline. Very rarely something goes wrong and the user can't sync with the main server, in this case I need to be able to get the database sent to me to debug it.

I seem to have to base64Encode a file (as a blob) in order to xhr upload it. Is this always the case or is there a way around it (I'm sending the file to a .Net asmx webservice)?

Is it possible to stream upload a blob through xhr on Android? Does anyone have an example, the docs aren't clear in this namespace?

Maybe I can base64Encode chunks of the file blob to a text file and when that's complete, stream upload the large file, so at no point does the whole file need to be held in memory.

Any help much appreciated!

— asked 10 months ago by Mark Henderson
1 Comment
  • OK so I see that it's because I'm using SOAP that it has to be base64Encoded.. So is it possible to stream an xhr post? for example using, I'll try using WCF and MTOM instead if so:

    xhr.send({media:finalZip.read()});
    or
    xhr.send(finalZip.read());

    — commented 10 months ago by Mark Henderson

1 Answer

I wrote this code which will base64Encode a file of any size as it does it in chunks, may be of use to someone:

var instream = finalZip.open(Titanium.Filesystem.MODE_READ);
    var outfile = Titanium.Filesystem.getFile(Ti.Filesystem.getExternalStorageDirectory() + 'test.txt');
    if (outfile.exists()) {
        outfile.deleteFile();
    }
    var outstream = outfile.open(Titanium.Filesystem.MODE_WRITE);   
    var buffer = Titanium.createBuffer({length: 1024});
    // Read and write chunks.
    var size = 0;
    while ((size = instream.read(buffer)) > -1) {
        var str = Ti.Codec.decodeString({length:buffer.length, position:0, source:buffer});
        var encoded = Ti.Utils.base64encode(str);
        Ti.API.info('encoded.length:' + encoded.length)
        var bufferNew = Titanium.createBuffer({length:encoded.length});
        var bytes = Ti.Codec.encodeString({source:encoded.text,dest:bufferNew})
        bufferNew.length = bytes;
 
 
      outstream.write(bufferNew);
      Titanium.API.info('Wrote ' + size + ' bytes');
    }
    // Cleanup.
    instream.close();
    outstream.close();
I ended up streaming the file to the WebService like this:
var finalZip = Titanium.Filesystem.getFile(pathToYourFile);
xhr.send({data:finalZip});
This method will send the bytes as a multipart/form body but it adds 170 characters to the message body. They are supposed to be mimeType etc.. headers but they appear in the body. I add my own headers using setRequestHeader(), and then on the server where I convert the message body to a file I remove the first 170 bytes so the file is intact. Fiddler saved the day here!

Your Answer

Think you can help? Login to answer this question!