CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 3 of 3
  1. #1
    Join Date
    Jan 2002
    Location
    USA
    Posts
    150

    SPEED - Importing delimited text file manually vs programatically - access/sql server

    Hello,

    In microsoft access and enterprise manager you have the option of importing delimited text files. In access you use the "file->GetExternal Data" option and in enterprise manager you just right click on a table and press "all tasks->import data" and then select "text file", etc. To import huge text files (a few hundred megabytes) it is very fast. Even when the tables are indexed. However, I haven't been able to duplicate anywhere near this speed in code. I've tried everything (ado.net, ado, command objects, etc) and it's all about 10x slower. What I'm thinking is there is something built in that just makes the import MUCH faster than having a program call 'insert' queries on each individual row. So, the question is:

    How can I duplicate that speed? Why are enterprise manager, and microsoft access able to do this so quickly? And what can I do to do the same thing? It is too much to call an insert query for every single row (there are about 2 million rows in the file i'm using now). Is there ANY other way?!!

    Thanks in advance

  2. #2
    Join Date
    Jan 2002
    Location
    USA
    Posts
    150

    Re: SPEED - Importing delimited text file manually vs programatically - access/sql server

    ok so I found the DTS packages, and they do the job. But they're only for SQL server. Does anyone know if they can be used with microsoft access (or if there's an equivalent)?

  3. #3
    Join Date
    Jan 2002
    Location
    USA
    Posts
    150

    Re: SPEED - Importing delimited text file manually vs programatically - access/sql server

    nevermind... I got it. In case anyone needs this later, I'm now using a macro in ms access and calling that from code.. so the sql server solution would be DTS and in ms access the solution would be a text file import macro. To my knowledge, any data import method other than these is over 10X slower because it has to run many, many insert queries, rather than using built in utilities made for large data transfer.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured