I don't know if this heading makes sense, but my situation is this:
I am writing a .Net program to query an Oracle database for a huge amount of history (about two years of daily data) and I need to create a delimited text file from this returned query to send to the client to populate their new database (the final file size will be about 4 GB) .
My current code is as follows:
Dim strSQLQuery as string = (My query to get all the data)
Dim cmd = New OracleCommand(strSQLQuery, conn)
...
Using Reader As OracleDataReader = cmd.ExecuteReader()
... write to text file ...
End Using
This code works fine, but my problem is that the query returns such a huge set of records, and therefore I'm afraid to run out of memory in the instruction cmd.ExecuteReader().
My question is, is there any way in .Net, and not process the entire request and immediately return the entire recordset in order to make the recordset return in more significant fragments or something like that?
Of course, one solution would be to split the query itself (which I could do), but I am also interested to find out if there is a better solution that already exists in .Net before leaving this route ...
Also, although this code is written in VB, I am equally comfortable with solutions in VB or C #.
Thank!!!
source
share