Titanium Community Questions & Answer Archive

We felt that 6+ years of knowledge should not die so this is the Titanium Community Questions & Answer Archive

Large XML response parsing crashes iPone app - memory leak?

I have narrowed a problem down with parsing XML from a fairly large XML response. The app I am running crashes on XML.parseString, when I got a good amount of reading and parsing of XML nodes.

In a callback from a getSoapResponse, I have created, the data is saved to disk and read back up for parsing.

What I have done so far:

  1. I tried parsing running down the DOM of the full XML response (varying from 15 to 1500 items). When crossing more than 50 items, the app crashes, when parsing this way. And this makes the app crash randomly.
  2. I have then tried reducing memory footprint by reading the response and quickly save it to persistent storage (disk). Then I read the XML file 1024 bytes at a time, looking for the entities I want and passing them off to the below method. In this way, I can reproduce consistency in the crash. When I have read 346112 bytes, and passed the contents of my buffer off for extraction, the app crashes. Now, this made me think there was something malformed about the entity, but I took the xml and ran it through the parser for each of the entities recognized in the file (statically set the input string). Same thing: It crashed, but in a different position.

Is this because of a memory leak?

The data looks like this (and sorry, I have to obscure the customer data, so this is not exactly matching, only meant as an example):

<clsFPLListItem>
    <Id>354xxxx</Id>
    <FlightLogId>XXXXXX-EVE-OSL</FlightLogId>
    <ATCId>XXXNNX</ATCId>
    <PPSName />
    <SchTimeDep>201112120545</SchTimeDep>
    <ATCDep>XXXX</ATCDep>
    <ATCTime>0545</ATCTime>
    <ATCDest>XXXX</ATCDest>
    <ATCCtot>0831</ATCCtot>
    <ATCAddr>XXXXXXXX XXXXXXXX</ATCAddr>
    <MissionIDInt>1</MissionIDInt>
    <Status>SRM</Status>
    <ATCDOF>2011-12-12T00:00:00</ATCDOF>
    <ActionNeeded>1</ActionNeeded>
    <Action />
    <Remarks />
    <Cancelled>0</Cancelled>
    <StatusList>
        <clsFPLStatus>
            <FPLId>0</FPLId>
            <Status>SRM</Status>
            <UnitName>EUROCONTROL</UnitName>
        </clsFPLStatus>
    </StatusList>
    <ATCEET>0151</ATCEET>
    <PPSRemark>XXX-XXXX  J</PPSRemark>
    <FLTNumber>XXXNNN</FLTNumber>
</clsFPLListItem>

The piece of code, which breaks looks like this, where fileContents is the above and entityName is "clsFPLListItem":

    mdfl.net._extractFlightListXMLEntity = function(fileContents,entityName){
        var xmlDoc = Ti.XML.parseString(fileContents);
        xmlDoc = xmlDoc.getFirstChild();
        fileContents = null; // Throw away the input and free memory
        var Username = Ti.App.Properties.getString("Username");
        if(xmlDoc.evaluate){
            Ti.API.debug('... dom evaluates true');
        } else {
            Ti.API.debug('... dom evaluates false');
        }
        Ti.API.debug('xmlDoc name:' + xmlDoc.nodeName);
        var items = []; 
...
The code keeps extracting node values etc. from here by creating a nodelist.

The statement:
var xmlDoc = Ti.XML.parseString(fileContents);
Seems to be the one breaking it, as narrowing it down and commenting it in and out provokes the crash.

This is the last lines of the debug console:
[DEBUG] Will send this to XML extraction:<clsFPLListItem><Id>3542174</Id><FlightLogId>NAX671S-AGP-CPH</FlightLogId><ATCId>NAX671S</ATCId><PPSName /><SchTimeDep>201112121215</SchTimeDep><ATCDep>LEMG</ATCDep><ATCTime>1215</ATCTime><ATCDest>EKCH</ATCDest><ATCCtot /><ATCAddr>EUCHZMFP EUCBZMFP</ATCAddr><MissionIDInt>1</MissionIDInt><Status>ACK</Status><ATCDOF>2011-12-12T00:00:00</ATCDOF><ActionNeeded>0</ActionNeeded><Action /><Remarks /><Cancelled>0</Cancelled><StatusList><clsFPLStatus><FPLId>0</FPLId><Status>ACK</Status><UnitName>EUROCONTROL</UnitName></clsFPLStatus></StatusList><ATCEET>0332</ATCEET><PPSRemark>AGP-CPHN  J </PPSRemark><FLTNumber>NAX3671</FLTNumber></clsFPLListItem>

[DEBUG] App Spec: <DTiPhoneSimulatorApplicationSpecifier 0x1003022b0> specified by path /Users/egilrausner/Documents/Titanium Studio Workspace/MDFL test/build/iphone/build/Debug-iphonesimulator/MDFL test.app
[DEBUG] SDK Root: <DTiPhoneSimulatorSystemRoot 0x100301350> path=/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk version=5.0 name=Simulator - iOS 5.0
[DEBUG] using device family iphone
[DEBUG] Session started
[DEBUG] Session did end with error Error Domain=DTiPhoneSimulatorErrorDomain Code=1 "The simulated application quit." UserInfo=0x100137dc0 {NSLocalizedDescription=The simulated application quit., DTiPhoneSimulatorUnderlyingErrorCodeKey=-1}
[INFO] Application has exited from Simulator

I'm running Titanium Studio, build: 1.0.6.201110251616

— asked December 12th 2011 by Egil Rausner
  • crash
  • iphone
  • memory
  • xml
1 Comment
  • um maybe you are actually running out of resources on the phone. 340k is a lot of data to load in one go. On a device which has max 1mb of memory.

    Sorry but Ithink you may need to rethink your approach, standard don't download more than 10 k in one go… you may be able to do this in small chunks but putting that amount into persistent data is really a no no.

    Sorry if this comes across a little blunt but I don't think it is anything other than you shouldn't really be even trying this.

    T

    — commented December 12th 2011 by Trevor Ward

3 Answers

  • I agree with trevor, you need a better API, you should not really be trying to process all of this xml data at once.

    BUT if you are pressed, you can try using xpath to get specific data elements one at a time, I suspect that might not have the same memory requirement, BUT I could be wrong

    — answered December 13th 2011 by Aaron Saunders
    permalink
    2 Comments
    • Hi Aaron,

      Would appreciate if you give an example of using xpath to get specific data elements.

      Thanks

      — commented December 13th 2011 by Zarir Bhesania
    • Trevor and Aaron,

      Well, thanks - good answers. I've already asked the backend team to get a sequenced payload, so that I can get an arbitrary number of items at a time, until all data is loaded. In that way you are right, it's way too much data on a mobile anyways.

      But, as for your comment on persistence, I've saved the responseXML to disk, which works fine. The problem arises after iterating on the response. Note that I have created a buffered read, which works on one item at a time, to avoid consuming excess memory. I think I will also keep this after receiving the sequenced response.

      So, the question is, why does it crash after a certain number of parsing elements time after time, when I've freed up the memory and saved the response to disk?

      Aaron, by xpath do you mean traversing the DOM 2.x using a nodelist? I'v done that, so maybe you mean something different?

      — commented December 13th 2011 by Egil Rausner
  • Ok, I ran the app in Xcode and got the following in Thread 6:
    "Thread 6 KrollContext<kroll$1>:Program received signal: "EXC_BAD_ACCESS".

    The last output on the console is:

    [Switching to process 1575 thread 0x1b003]
    [Switching to process 1575 thread 0x1b003]
    sharedlibrary apply-load-rules all
    Warning: the current language does not match this frame.
    Current language:  auto; currently c++
    (gdb)
    

    Now, this might be a stupid question.. Is the XML parser module thread safe?

    I'm no expert in Instruments, but I ran the app through a leaks profile, and it turned out well.

    Is anyone from Appcelerator listening in on this issue?

    — answered December 14th 2011 by Egil Rausner
    permalink
    0 Comments
  • Turned out to be an error in the API below. The item had been reported and solved here: http://jira.appcelerator.org/browse/TIMOB-4751

    By upgrading to SDK 1.8 RC2 the problem with EXEC_BAD_ACCESS no longer exists. Now I only have to deal with some memory being eaten. The app goes from 102 mb free memory to 8 mb in 100 iterations on the above mentioned piece of XML. This might be natural, but at least I don't have to deal with some mystery bug down below decks.

    — answered December 17th 2011 by Egil Rausner
    permalink
    0 Comments
The ownership of individual contributions to this community generated content is retained by the authors of their contributions.
All trademarks remain the property of the respective owner.