Reading file changes in real time using .NET.

Question

I have a CSV file that is often updated (20 to 30 times per minute). I want to insert newly added rows into the database as soon as they are written to the file.

The FileSystemWatcher class listens for file system change notifications and can raise an event whenever there is a change in the specified file. The problem is that FileSystemWatcher cannot determine exactly which rows were added or deleted (as far as I know).

One way to read these lines is to save and compare the number of lines between the changes and read the difference between the last and second last change. However, I am looking for a cleaner (possibly more elegant) solution.

Answer 1

I wrote something very similar. I used FileSystemWatcher to receive change notifications. Then I used FileStream to read the data (by tracking the last position in the file and doing this before reading new data). Then I add the read data to the buffer, which automatically extracts the complete lines, and then displays them in the user interface.

Note: "this.MoreData (..) is an event whose listener adds to the aforementioned buffer and handles a full row extraction.

Note. As already mentioned, this will only work if changes are always added to the file. Any deletions will cause problems.

Hope this helps.

   public void File_Changed( object source, FileSystemEventArgs e )
    {
        lock ( this )
        {
            if ( !this.bPaused )
            {
                bool bMoreData = false;

                // Read from current seek position to end of file
                byte[] bytesRead = new byte[this.iMaxBytes];
                FileStream fs = new FileStream( this.strFilename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite );

                if ( 0 == this.iPreviousSeekPos )
                {
                    if ( this.bReadFromStart )
                    {
                        if ( null != this.BeginReadStart )
                        {
                            this.BeginReadStart( null, null );
                        }
                        this.bReadingFromStart = true;
                    }
                    else
                    {
                        if ( fs.Length > this.iMaxBytes )
                        {
                            this.iPreviousSeekPos = fs.Length - this.iMaxBytes;
                        }
                    }
                }

                this.iPreviousSeekPos = (int)fs.Seek( this.iPreviousSeekPos, SeekOrigin.Begin );
                int iNumBytes = fs.Read( bytesRead, 0, this.iMaxBytes );
                this.iPreviousSeekPos += iNumBytes;

                // If we haven't read all the data, then raise another event
                if ( this.iPreviousSeekPos < fs.Length )
                {
                    bMoreData = true;
                }

                fs.Close();

                string strData = this.encoding.GetString( bytesRead );
                this.MoreData( this, strData );

                if ( bMoreData )
                {
                    File_Changed( null, null );
                }
                else
                {
                    if ( this.bReadingFromStart )
                    {
                        this.bReadingFromStart = false;
                        if ( null != this.EndReadStart )
                        {
                            this.EndReadStart( null, null );
                        }
                    }
                }
            }
        }

Answer 2

, FileSystemWatcher . , .., .

? , . , , , .

Answer 3

, NTFS :

NTFS , . NTFS , . , .

TechNet. PInvoke .NET.

Answer 4

, , diff, , . , http://www.mathertel.de/Diff/, , - , . db.

Answer 5

, . , , .

.

Answer 6

FileSystemWatcher. , , .. , , , .

? , . , . - , . "" , , , . . , . , - - , (.. ), .