ContextSwitchDeadLock please help

[techtalk7]

I built a TickImporter class to import very large csv files of price data into a trading software for charting.

The function that does all the actual importing is TickImporter.ImportToDataStore().
When calling ImportToDataStore() from a basic Console Application it performs very well and does what it should.

However, I realized it would be very time consuming to import the data by hand because the data comes in many files that need to be organized, merged into one (TickImporter overwrites so importing them separately is not an option), then actually import the file which will take a few hours inside ImportToDataStore() looping through the millions of lines, and then do this again for each of over 20 symbols. As you can imagine this will takes days with a program and possibly weeks manually!

So I decided to build a WPF app to make this easier for other members on the forum that may want to import these same files (which are publicly available). All they have to do is drag all the raw downloaded files into the app and it does the dirty work, organizing, merging, and importing.

However, I noticed it performs significantly slower than its Console Application counterpart (only 5kb/sec as compared to something like 200 kb/sec on the Console App!)

Not only that but after ~350kb I get ContextSwitchDeadLock:

The CLR has been unable to transition from COM context 0x9670e8 to COM context 0x967258 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations.


Like I said TickImporter.ImportDataToStore() is the main computation intensive area, I get the deadlock while program is inside it.

Im not knowledgeable in threading yet so if you will explain how to put ImportDataToStore() on a different thread please provide code.

And what is with this performance??? Its extremely slow even at the very beginning looks something like 7kb/sec and once it gets over 250kb it drops down to like 3kb/sec and crashes 350ish,. Theres nothing wrong with ImportDataToStore(), its lightning fast in a console app it gets to 350kb before I can blink..


Thank you