All geek questions in one place

Import a small number of records from a very large CSV file into Biztalk 2006

I have a Biztalk project that imports an incoming CSV file and uploads it to a database table. Import works fine, but I need to store about 200-300 records from a file with more than a million lines. My orchestration discards these lines, but the problem is that the flat file that I import is still 250 MB, and when converting to XML using the usual flat file conveyor, it takes many hours to process, and sometimes to output to the server memory.

Is there something I can do to make the user pipeline itself drop lines that I don't need? The very first element in each CSV line is one of several lines, and I want to save lines starting with a specific line.

Thanks for any help you can provide.

+3
source share
1 answer

A custom pipeline component would certainly be the best solution; but it will have to be executed at the decoding stage before the disassembler component.

100% - (, , ), CVS , , ( MemoryStream), , , FileStream BizTalk, .

+4

Source: https://habr.com/ru/post/1740211/

More articles:


All Articles