Datagrid / Datatable gets slower and slower when refilling / resizing?!?

I am importing data from files into datagrids/tables and then I have some selection oppertunities on the datatable. But if I import over 5 files after eachother that are supposed to overwrite eachother in the grid, rowselection takes longer and longer! (When clicking the grey spot in front of each row). When selecting a cell and then selecting the grey spot, it is still fast. It's only when i press the grey spot first.

This also happens if I programically add more columns to the table. After adding about 5 columns, rowselection gets noticably slower.

Is this normal? Am i supposed to use some kind of dispose function or somthing to get rid of the old table? I really need this not to happen.

I have a small event when i press the grey spot that gets the selected indexes and adds rowdata to an arraylist. Maybe that has something to do with it.