Using FileStream and Memory

Question

I wrote the following program, the purpose of which is to create a file with the size of the output with some random data. The program works great and does what it should do. However, I do not understand why it consumes 5 GB of RAM (see screenshot of my task manager). Although I am writing a file with random data, I am not creating new objects. What am I missing? I would expect this program to not save memory at all.

The big problem that I am facing right now is that in the middle of the file generation process, the machine dies ...

class Program
{
    static void Main(string[] args)
    {
        CreateFile("test.dat", 10 * 1024 * 1024);
    }

    public static void CreateFile(string path, long approximativeFileSizeInKb)
    {
        RandomNumberGenerator randomNumber = RandomNumberGenerator.Create();

        byte[] randomData = new byte[64 * 1024];

        int numberOfIteration = 0;
        randomNumber.GetNonZeroBytes(randomData);

        using (FileStream fs = File.Create(path, 64 * 1024))
        {
            while (numberOfIteration++ * 64 < approximativeFileSizeInKb)
            {
                fs.Write(randomData, 0, randomData.Length);
            }
        }
    }
}

Answer 1

, :

using (FileStream fs = File.Create(path, 64 * 1024))

using (FileStream fs = File.Create(path, 64 * 1024, FileOptions.WriteThrough))

, .

Answer 2

.

FileSystem.Write , , .

FileSystem.Flush, , .

. RAID- , RAID 5 6 64 .

( , FileOptions.WriteThrough File.Create.)

Answer 3

Windows, , ...

Answer 4

... , Windows , . , "" , FileStream, , . Windows , , , . , ... , -, , .

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Runtime.InteropServices;
using Microsoft.Win32.SafeHandles;

namespace ConsoleApplication2
{
    class Program
    {
        [DllImport("KERNEL32", SetLastError = true)]
        public extern static int CloseHandle(IntPtr hObject);
        [DllImport("kernel32", SetLastError = true)]
        public static extern unsafe IntPtr CreateFile(
            string FileName,           // file name 
            uint DesiredAccess,        // access mode 
            uint ShareMode,            // share mode 
            IntPtr SecurityAttributes, // Security Attr 
            uint CreationDisposition,  // how to create 
            uint FlagsAndAttributes,   // file attributes 
            IntPtr hTemplate // template file   
            );
        static void Main(string[] args)
        {
            const uint FILE_FLAG_NO_BUFFERING = 0x20000000;
            const uint FILE_FLAG_WRITE_THROUGH = 0x80000000;
            Random r = new Random(0);
            IntPtr f = CreateFile(@"e:\test\temp.bin",
                             (uint)FileAccess.Write,
                             (uint)FileShare.None,
                             IntPtr.Zero,
                             (uint)FileMode.Create,
                              FILE_FLAG_NO_BUFFERING,
                             IntPtr.Zero);
            using (FileStream fs = new FileStream(f,FileAccess.Write,false,1024*1024))
            {
                int blocksize = 1024 * 1024;
                byte[] val = new byte[blocksize];
                for (int i = 0; i < blocksize; i++)
                {
                    val[i] = 1;
                }
                while (true)
                {
                    for (int i = 0; i < 1000; i++)
                    {
                        for (int j = 0; j < blocksize; j++)
                        {
                            fs.WriteByte(val[i]);
                        }
                    }
                    Console.WriteLine("Enter s to stop");
                    ConsoleKeyInfo k = Console.ReadKey();
                    if (k.KeyChar == 's')
                    {
                        break;
                    }
                }
            }
            CloseHandle(f);
            Console.WriteLine("done");
            Console.ReadKey();
        }
    }
}

Answer 5

FileStream , . , , .

; 10 ( 64 ) 5 . - , ? ?

Answer 6

Gonzalo, 1 .

Do you have antivirus enabled? AV can scan the .dat file as it is being written, resulting in data being buffered in memory during the scan, resulting in a huge increase in memory usage. If you suspect that AV is part of the problem, try changing the file extension to something else .dat (e.g. .txt).

Another thing to try is to add a call to fs.Flush () after fs.Write (...).