Large XML response parsing crashes iPone app - memory leak?
I have narrowed a problem down with parsing XML from a fairly large XML response. The app I am running crashes on XML.parseString, when I got a good amount of reading and parsing of XML nodes.
In a callback from a getSoapResponse, I have created, the data is saved to disk and read back up for parsing.
What I have done so far:
- I tried parsing running down the DOM of the full XML response (varying from 15 to 1500 items). When crossing more than 50 items, the app crashes, when parsing this way. And this makes the app crash randomly.
- I have then tried reducing memory footprint by reading the response and quickly save it to persistent storage (disk). Then I read the XML file 1024 bytes at a time, looking for the entities I want and passing them off to the below method. In this way, I can reproduce consistency in the crash. When I have read 346112 bytes, and passed the contents of my buffer off for extraction, the app crashes. Now, this made me think there was something malformed about the entity, but I took the xml and ran it through the parser for each of the entities recognized in the file (statically set the input string). Same thing: It crashed, but in a different position.
Is this because of a memory leak?
The data looks like this (and sorry, I have to obscure the customer data, so this is not exactly matching, only meant as an example):
<clsFPLListItem>
<Id>354xxxx</Id>
<FlightLogId>XXXXXX-EVE-OSL</FlightLogId>
<ATCId>XXXNNX</ATCId>
<PPSName />
<SchTimeDep>201112120545</SchTimeDep>
<ATCDep>XXXX</ATCDep>
<ATCTime>0545</ATCTime>
<ATCDest>XXXX</ATCDest>
<ATCCtot>0831</ATCCtot>
<ATCAddr>XXXXXXXX XXXXXXXX</ATCAddr>
<MissionIDInt>1</MissionIDInt>
<Status>SRM</Status>
<ATCDOF>2011-12-12T00:00:00</ATCDOF>
<ActionNeeded>1</ActionNeeded>
<Action />
<Remarks />
<Cancelled>0</Cancelled>
<StatusList>
<clsFPLStatus>
<FPLId>0</FPLId>
<Status>SRM</Status>
<UnitName>EUROCONTROL</UnitName>
</clsFPLStatus>
</StatusList>
<ATCEET>0151</ATCEET>
<PPSRemark>XXX-XXXX J</PPSRemark>
<FLTNumber>XXXNNN</FLTNumber>
</clsFPLListItem>
The piece of code, which breaks looks like this, where fileContents is the above and entityName is "clsFPLListItem":
mdfl.net._extractFlightListXMLEntity = function(fileContents,entityName){
var xmlDoc = Ti.XML.parseString(fileContents);
xmlDoc = xmlDoc.getFirstChild();
fileContents = null; // Throw away the input and free memory
var Username = Ti.App.Properties.getString("Username");
if(xmlDoc.evaluate){
Ti.API.debug('... dom evaluates true');
} else {
Ti.API.debug('... dom evaluates false');
}
Ti.API.debug('xmlDoc name:' + xmlDoc.nodeName);
var items = [];
...
The code keeps extracting node values etc. from here by creating a nodelist.
The statement:
var xmlDoc = Ti.XML.parseString(fileContents);
Seems to be the one breaking it, as narrowing it down and commenting it in and out provokes the crash.
This is the last lines of the debug console:
[DEBUG] Will send this to XML extraction:<clsFPLListItem><Id>3542174</Id><FlightLogId>NAX671S-AGP-CPH</FlightLogId><ATCId>NAX671S</ATCId><PPSName /><SchTimeDep>201112121215</SchTimeDep><ATCDep>LEMG</ATCDep><ATCTime>1215</ATCTime><ATCDest>EKCH</ATCDest><ATCCtot /><ATCAddr>EUCHZMFP EUCBZMFP</ATCAddr><MissionIDInt>1</MissionIDInt><Status>ACK</Status><ATCDOF>2011-12-12T00:00:00</ATCDOF><ActionNeeded>0</ActionNeeded><Action /><Remarks /><Cancelled>0</Cancelled><StatusList><clsFPLStatus><FPLId>0</FPLId><Status>ACK</Status><UnitName>EUROCONTROL</UnitName></clsFPLStatus></StatusList><ATCEET>0332</ATCEET><PPSRemark>AGP-CPHN J </PPSRemark><FLTNumber>NAX3671</FLTNumber></clsFPLListItem>
[DEBUG] App Spec: <DTiPhoneSimulatorApplicationSpecifier 0x1003022b0> specified by path /Users/egilrausner/Documents/Titanium Studio Workspace/MDFL test/build/iphone/build/Debug-iphonesimulator/MDFL test.app
[DEBUG] SDK Root: <DTiPhoneSimulatorSystemRoot 0x100301350> path=/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.0.sdk version=5.0 name=Simulator - iOS 5.0
[DEBUG] using device family iphone
[DEBUG] Session started
[DEBUG] Session did end with error Error Domain=DTiPhoneSimulatorErrorDomain Code=1 "The simulated application quit." UserInfo=0x100137dc0 {NSLocalizedDescription=The simulated application quit., DTiPhoneSimulatorUnderlyingErrorCodeKey=-1}
[INFO] Application has exited from Simulator
I'm running Titanium Studio, build: 1.0.6.201110251616
3 Answers
-
I agree with trevor, you need a better API, you should not really be trying to process all of this xml data at once.
BUT if you are pressed, you can try using xpath to get specific data elements one at a time, I suspect that might not have the same memory requirement, BUT I could be wrong
-
Ok, I ran the app in Xcode and got the following in Thread 6:
"Thread 6 KrollContext<kroll$1>:Program received signal: "EXC_BAD_ACCESS".The last output on the console is:
[Switching to process 1575 thread 0x1b003] [Switching to process 1575 thread 0x1b003] sharedlibrary apply-load-rules all Warning: the current language does not match this frame. Current language: auto; currently c++ (gdb)
Now, this might be a stupid question.. Is the XML parser module thread safe?
I'm no expert in Instruments, but I ran the app through a leaks profile, and it turned out well.
Is anyone from Appcelerator listening in on this issue?
-
Turned out to be an error in the API below. The item had been reported and solved here: http://jira.appcelerator.org/browse/TIMOB-4751
By upgrading to SDK 1.8 RC2 the problem with EXEC_BAD_ACCESS no longer exists. Now I only have to deal with some memory being eaten. The app goes from 102 mb free memory to 8 mb in 100 iterations on the above mentioned piece of XML. This might be natural, but at least I don't have to deal with some mystery bug down below decks.