C # How to loop a large set of folders and files recursively without using a huge amount of memory

Question

I want to index all my music files and store them in a database. I have this function that I call provocatively, starting from the root of my music disc.

i.e.

start > ReadFiles(C:\music\);

ReadFiles(path){
   foreach(file)
      save to index;

   foreach(directory)
      ReadFiles(directory);
}

This works fine, but when I run the program, the amount of used memory grows and grows, and finally, my system runs out of memory.

Does anyone have a better approach that doesn't need 4 GB of RAM to complete this task?

Best regards, Tys

Answer 1

An Alxandr queue based solution should work fine.

.NET 4.0, Directory.EnumerateFiles, , :

void ReadFiles(string path)
{
    IEnumerable<string> files =
        Directory.EnumerateFiles(
            path,
            "*",
            SearchOption.AllDirectories); // search recursively

    foreach(string file in files)
        SaveToIndex(file);
}

Answer 2

. .., , ?

, .

Answer 3

. ( ), . , . , , , , , . .

- :

Queue<string> dirs = new Queue<string>();
dirs.Enqueue("basedir");
while(dirs.Count > 0) {
    foreach(directory)
        dirs.Enqueue(directory);
    ReadFiles();
}

Answer 4

, , EnumerateFiles() , - . , :

public static List<string> getFiles(string path, List<string> files)
{
    IEnumerable<string> fileInfo = null;
    IEnumerable<string> folderInfo = null;
    try
    {
        fileInfo = Directory.EnumerateFiles(str);
    }
    catch
    {

    }
    if (fileInfo != null)
    {
        files.AddRange(fileInfo);
        //recurse through the subfolders
        fileInfo = Directory.EnumerateDirectories(str);
        foreach (string s in folderInfo)
        {
            try
            {
                getFiles(s, files);
            }
            catch
            {

            }
        }
    }
    return files;
}

:

List<string> files = new List<string>();
files = folder.getFiles(path, files);

: http://msdn.microsoft.com/en-us/library/vstudio/bb513869.aspx.

: http://social.msdn.microsoft.com/Forums/vstudio/en-US/ae61e5a6-97f9-4eaa-9f1a-856541c6dcce/directorygetfiles-gives-me-access-denied?forum=csharpgeneral. Stack ( , ), , , . , C D .

Answer 5

These can be nodal folders that lead to an endless loop when recursing, but I'm not sure, check this out and see for yourself. Link: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/mklink