Titanium Community Questions & Answer Archive

We felt that 6+ years of knowledge should not die so this is the Titanium Community Questions & Answer Archive

SQLite multiple INSERT

Hi all !

I need to insert a lot of lines into a SQLite database.
I currently do this :

for (var i=0; i<xml_length; i++) {
    db.execute('INSERT INTO myTable ("id", "value") VALUES ("'+xml.id+'", "'+xml.value+'")';
}

It works, but it could be much faster to execute multiple inserts at a time.

I tried queries such as:

INSERT INTO myTable ("id", "values") VALUES ("1", "test"), ("2", "test"), ("3", "test");

or:

BEGIN;
INSERT INTO myTable ("id", "values") VALUES ("1", "test")
INSERT INTO myTable ("id", "values") VALUES ("2", "test")
INSERT INTO myTable ("id", "values") VALUES ("3", "test")
COMMIT;

or even:

INSERT INTO myTable ("id", "values") VALUES ("1", "test", "2", "test", "3", "test");

It seems to me like a basic database feature, but I can't find the right syntax to use.

— asked January 25th 2011 by Adrien Mas
  • insert
  • iphone
  • multiple
  • sqlite
0 Comments

2 Answers

  • Accepted Answer

    Appcelerator uses a standard sqlite3 database engine. I'd suggest you start by looking at its online documentation for the insert statement. Each INSERT will create a single row unless you are using a SELECT clause instead of VALUES().

    To wrap several inserts within a transaction, I use this syntax:

    db.execute("BEGIN IMMEDIATE TRANSACTION");
    db.execute(sql);   // repeat as necessary
    db.execute(sql);
    db.execute(sql);
    db.execute("COMMIT TRANSACTION");
    

    You may want to peruse the syntax for various sqlite3 commands here.

    — answered January 25th 2011 by Doug Handy
    permalink
    4 Comments
    • You've kind of missed the point of the original post. Calling the same sql statement multiple times like that is inefficient, since the statement needs to be prepared again for every item of data. Wrapping the inserts in a transaction is faster than relying on autocommit, but many sqlite wrappers also provide a bulk insert feature, and that's what the OP seems to be asking for. Unfortunately I don't believe Ti has such a feature at this time.

      — commented January 25th 2011 by Damien Elmes
    • (that said, the suggestion to wrap the inserts in an outer transaction is probably going to save a lot more time than removing redundant statement preparations)

      — commented January 25th 2011 by Damien Elmes
    • I didn't so much miss the point of the OP as not realize there are any wrappers for sqlite3 which provide a bulk insert. I did intentionally point out multiple rows can be inserted at once using a SELECT variant although it wasn't clear from the OP's example that would be practical. Just in case, I tried to point him to the sqlite3 documentation.

      — commented January 25th 2011 by Doug Handy
    • Sorry, I could have chosen my words better. I just meant to say "I think the OP was actually asking about X."

      — commented January 25th 2011 by Damien Elmes
  • Thanks for your answer Doug.
    Your way to do it is really clean. And it's much faster than what I did before.

    While waiting for an answer, I found another way to do it. It's not as clean as yours but it seems a bit faster.
    The goal is to create a query such as :

    INSERT INTO myTable
    SELECT "1", "test"
    UNION
    SELECT "2", "test"
    ...
    

    It works, there seems to be a limit around 500 lines inserted, but it works.

    So I did a little Benchmark for inserting 1544 lines with the same datas (I did the test 5 times for each methods) :

    • Multiples INSERTs : 2500ms
    • Using BEGIN and COMMIT : 90ms
    • Using SELECT and UNION : 40ms

    Anyway, I think I'll use your method because your code is cleaner and much easier to maintain. Here is what I had to do :

    var newQuery = true, query = '';
    for (i=0; i<locations_length; i++) {
        if (newQuery) {
            query = 'INSERT INTO myTable (id, name, code)';
            newQuery = false;
        }
        else {
            query += ' UNION';
        }
    
        query += ' SELECT "'+values[i][0]+'", "'+values[i][1]+'", "'+values[i][2]+'"';
    
        if (i!=0 && i%499==0) {
            db.execute(query);
            newQuery = true;
        }
    }
    
    //executing remaining lines
    if (i%499!=0) {
        db.execute(query);
    }
    
    — answered January 25th 2011 by Adrien Mas
    permalink
    1 Comment
    • Thank you Adrien

      This is very helpful for me for Inserting the multiple data in sqlite

      can you help me for updating multiple data same as this?

      Thanks :)

      — commented August 1st 2012 by Dhaval Kumar
The ownership of individual contributions to this community generated content is retained by the authors of their contributions.
All trademarks remain the property of the respective owner.