Titanium Community Questions & Answer Archive

We felt that 6+ years of knowledge should not die so this is the Titanium Community Questions & Answer Archive

Android XML Parsing

Hi,

I'm having problems parsing a relatively large XML file on the Android platform.
The XML I'm trying to parse is about 1.5 megabytes - and while I walk through each node with a recursive function, the data is saved to a database node by node.

This is my recursive function:

function parseXMLData(xml, level)
{

    var xmlElem = xml;

    if (xmlElem.nodeName == 'page') 
    {
        savePageToDatabase(xmlElem, level);
    }

    if (xmlElem.nodeName == 'element' && xmlElem.getAttribute('dataStructure') == 'Absatz') 
    {
        saveContentToDatabase(xmlElem);
    }

    if (xmlElem.nodeName == 'element' && xmlElem.getAttribute('ctype') == 'text') 
    {
        saveTextToDatabase(xmlElem);
    }

    if (xmlElem.hasChildNodes()) 
    {
        ++level;
    }
    else 
    {
        --level;
    }

    for (var i = 0; i < xmlElem.childNodes.length; ++i) 
    {
        parseXMLData(xmlElem.childNodes.item(i), level);
    }

}

Now what I get is a java.lang.OutOfMemoryError Exception that ends the parsing of the XML when I try to walk thorugh that 1.5 megabyte file. What irritates me, is that the code works fine for smaller XML files (400kb).

AND: This code works just perfectly fine for the same large file on iOs devices!

Is there any way to free some memory or has anybody got a good solution to this problem?

— asked July 21st 2011 by Peter Hamm
  • android
  • large file
  • out of memory
  • parsing
  • xml
3 Comments
  • Would you please detail your environment, as requested in the Q&A Question Checklist?

    Many thanks

    — commented July 22nd 2011 by Paul Dowsett
  • Sure, here we go:

    It's a mobile app that I'm porting from iOS to Android 2.3.3
    Titanium SDK is 1.7.1 and I'm running the Studio on a Mac (Lion). But I dont think that's relevant since the memory error exists within the emulator as well as on the device itself (Samsung Galaxy pad GT-P1000 FW 2.2)

    — commented July 22nd 2011 by Peter Hamm
  • No, don't worry, Peter, Lion is not the issue here. :)

    — commented July 22nd 2011 by Paul Dowsett

2 Answers

  • Peter

    Are you using Titanium's parseString? Where is this code?

    You are probably going to have problems parsing an XML file as large as 1.5MB for this reason, I'm afraid. The recommended approach is always to use JSON, as it incurs far less overhead. Is it possible for you to move to it?

    Cheers

    — answered July 21st 2011 by Paul Dowsett
    permalink
    3 Comments
    • If you are going to write a webservice for Android to convert the XML file to JSON, then I would recommend you use it for iOS also. This will save the user's bandwidth, and potentially improve the app's responsiveness.

      — commented July 22nd 2011 by Paul Dowsett
    • Try installing the latest Continuous Build, to determine whether the XML issue, or other related issue, has been addressed that could improve your situation.

      Cheers

      — commented July 22nd 2011 by Paul Dowsett
    • Thanks for your tip, I now tested it with the latest build (1.7.2.v20110720070250), but the issue still is there.

      This is a console excerpt, maybe you can get some information out of it?:

      D/dalvikvm(  352): GC_FOR_MALLOC freed 1K, 17% free 13610K/16327K, external 8281K/10329K, paused 122ms
      I/dalvikvm-heap(  352): Forcing collection of SoftReferences for 1832-byte allocation
      I/dalvikvm-heap(  352): Clamp target GC heap from 25.914MB to 24.000MB
      D/dalvikvm(  352): GC_FOR_MALLOC freed 9K, 17% free 13600K/16327K, external 8281K/10329K, paused 139ms
      E/dalvikvm-heap(  352): Out of memory on a 1832-byte allocation.
      I/dalvikvm(  352): "kroll$1: app://app.js" prio=5 tid=10 RUNNABLE JIT
      I/dalvikvm(  352):   | group="main" sCount=0 dsCount=0 obj=0x405d63c8 self=0x2bc2a0
      I/dalvikvm(  352):   | sysTid=362 nice=0 sched=0/0 cgrp=default handle=2806672
      I/dalvikvm(  352):   | schedstat=( 10082652045 6879371575 2466 )
      I/dalvikvm(  352):   at java.util.ArrayList.add(ArrayList.java:~159)
      I/dalvikvm(  352):   at org.apache.harmony.xml.dom.InnerNodeImpl.insertChildAt(InnerNodeImpl.java:128)
      I/dalvikvm(  352):   at org.apache.harmony.xml.dom.InnerNodeImpl.appendChild(InnerNodeImpl.java:50)
      I/dalvikvm(  352):   at org.apache.harmony.xml.parsers.DocumentBuilderImpl.appendText(DocumentBuilderImpl.java:390)
      I/dalvikvm(  352):   at org.apache.harmony.xml.parsers.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:271)
      I/dalvikvm(  352):   at org.apache.harmony.xml.parsers.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:360)
      I/dalvikvm(  352):   at org.apache.harmony.xml.parsers.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:360)
      
      [...]
      
      I/dalvikvm(  352):   at org.apache.harmony.xml.parsers.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:133)
      I/dalvikvm(  352):   at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:107)
      I/dalvikvm(  352):   at ti.modules.titanium.xml.XMLModule.parse(XMLModule.java:63)
      I/dalvikvm(  352):   at ti.modules.titanium.network.TiHTTPClient.getResponseXML(TiHTTPClient.java:563)
      I/dalvikvm(  352):   at ti.modules.titanium.network.HTTPClientProxy.getResponseXML(HTTPClientProxy.java:65)
      I/dalvikvm(  352):   at ti.modules.titanium.network.HTTPClientProxyBindingGen$6.dynamicGet(HTTPClientProxyBindingGen.java:334)
      I/dalvikvm(  352):   at org.appcelerator.kroll.KrollDynamicProperty.safeInvoke(KrollDynamicProperty.java:41)
      I/dalvikvm(  352):   at org.appcelerator.kroll.KrollDynamicProperty.get(KrollDynamicProperty.java:87)
      I/dalvikvm(  352):   at org.appcelerator.kroll.KrollProxy.getDynamicProperty(KrollProxy.java:243)
      I/dalvikvm(  352):   at org.appcelerator.kroll.KrollProxy.get(KrollProxy.java:175)
      I/dalvikvm(  352):   at org.appcelerator.kroll.KrollObject.get(KrollObject.java:67)
      I/dalvikvm(  352):   at org.mozilla.javascript.ScriptableObject.getProperty(ScriptableObject.java:2073)
      I/dalvikvm(  352):   at org.mozilla.javascript.ScriptRuntime.getObjectProp(ScriptRuntime.java:1523)
      I/dalvikvm(  352):   at org.mozilla.javascript.ScriptRuntime.getObjectProp(ScriptRuntime.java:1509)
      I/dalvikvm(  352):   at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1398)
      I/dalvikvm(  352):   at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:854)
      I/dalvikvm(  352):   at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:164)
      I/dalvikvm(  352):   at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:426)
      I/dalvikvm(  352):   at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3177)
      I/dalvikvm(  352):   at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:162)
      I/dalvikvm(  352):   at org.appcelerator.titanium.kroll.KrollCallback.callSync(KrollCallback.java:139)
      I/dalvikvm(  352):   at org.appcelerator.titanium.kroll.KrollCallback$1.run(KrollCallback.java:164)
      I/dalvikvm(  352):   at android.os.Handler.handleCallback(Handler.java:587)
      I/dalvikvm(  352):   at android.os.Handler.dispatchMessage(Handler.java:92)
      I/dalvikvm(  352):   at android.os.Looper.loop(Looper.java:123)
      I/dalvikvm(  352):   at org.appcelerator.titanium.kroll.KrollHandlerThread.run(KrollHandlerThread.java:86)
      I/dalvikvm(  352): 
      I/dalvikvm-heap(  352): Clamp target GC heap from 25.916MB to 24.000MB
      D/dalvikvm(  352): GC_CONCURRENT freed 4K, 17% free 13603K/16327K, external 8281K/10329K, paused 12ms+6ms
      D/dalvikvm(  352): GC_CONCURRENT freed 6445K, 57% free 7177K/16327K, external 8281K/10329K, paused 8ms+7ms
      

      — commented July 22nd 2011 by Peter Hamm
  • Hi Paul,

    thanks for your answer.

    I'm actually retrieving the XML file from a webserver via Ti.Network.createHTTPClient and then call parseXMLData(this.responseXML, 0); in the onload event for the first time to start the recursion.

    What I might do now is to write some server side script that transforms the XML to a JSON File and then try to parse that, just for Android… as it all works fine on iOS. Might there be somewhere be a memory leak on Android?

    — answered July 22nd 2011 by Peter Hamm
    permalink
    2 Comments
    • Peter, please use the comment facility rather than answering your own questions. Thanks :)

      — commented July 22nd 2011 by Paul Dowsett
    • check this thread. there's a code there that parses xml to json - even though i'm having issues in Android - possibly context issue.
      http://developer.appcelerator.com/question/94541/pushing-a-text-string-onto-an-array-causes-error.html#answer-216093

      — commented August 10th 2011 by Joe Da Silva
The ownership of individual contributions to this community generated content is retained by the authors of their contributions.
All trademarks remain the property of the respective owner.