C# .NET - .NET C# console application to load data from flat files into Oracle
Asked By Ismael on 19-Feb-14 08:31 AM
I'm working in a .NET C# console application to load data from flat files into Oracle, it will be like 100,000 records per day and every row will be upserted (merge). So I would like to know if you have any suggestions on which would be an efficient way of loading the data, without making it too slow and without putting at risk the data integrity.
Do you think I should serialize every row into objects first or just load it directly? I'm not allowed to use SSIS or any other tool, I has to be a custom solution.
Robbe Morris replied to Ismael on 19-Feb-14 11:56 AM
100k rows isn't all that many. Just read the text file in and perform your insert/update. I tend to attempt an update first and if the statement doesn't return any affected rows, then perform the insert. This removes the SELECT query you would typically run when deciding whether to insert or update. One less sql statement x 100k helps performance.