C # and LINQ; Grouping files into 5MB groups

Question

Greetings, I am trying to write a Linq query to run in a list of file names that returns a list of files grouped into 5 MB chunks. Therefore, each group will contain a list of file names whose total / total MB does not exceed 5 MB.

I agree with Linq, but I do not know where to start. Help

DirectoryInfo di = new DirectoryInfo (@"x:\logs");
List<string> FileList = di.GetFiles ("*.xml")
var Grouped = FileList =>

Answer 1

Yes, you can do it with LINQ.

var groupedFiles = files.Aggregate(
    new List<List<FileInfo>>(),
    (groups, file) => {
        List<FileInfo> group = groups.FirstOrDefault(
           g => g.Sum(f => f.Length) + file.Length <= 1024 * 1024 * 5
        );
        if (group == null) {
            group = new List<FileInfo>();
            groups.Add(group);
        }
        group.Add(file);
        return groups;
    }
);

. , FileInfo, 5 . , . , OrderBy(f => f.Length) Aggregate , , .

Answer 2

fooobar.com/questions/20884/... . . group by. , LINQ , .

, . File, LINQ. Linq 4.0 group-by-in, , .

Answer 3

:

• Define a type that takes the file size as input and returns a value that increases with the specified maximum and is reset. (This type is responsible for maintaining its own state.)
• Group by the values ​​returned by this type.

Code example:

// No idea what a better name for this would be...
class MaxAmountGrouper
{
    readonly int _max;

    int _id;
    int _current;

    public MaxAmountGrouper(int max)
    {
        _max = max;
    }

    public int GetGroupId(int amount)
    {
        _current += amount;
        if (_current >= _max)
        {
            _current = 0;
            return _id++;
        }

        return _id;
    }
}

Using:

const int BytesPerMb = 1024 * 1024;

DirectoryInfo directory = new DirectoryInfo(@"x:\logs");
FileInfo[] files = directory.GetFiles("*.xml");

var grouper = new MaxAmountGrouper(5 * BytesPerMb);
var groups = files.GroupBy(f => grouper.GetGroupId((int)f.Length));

foreach (var g in groups)
{
    long totalSize = g.Sum(f => f.Length);
    Console.WriteLine("Group {0}: {1} MB", g.Key, totalSize / BytesPerMb);
    foreach (FileInfo f in g)
    {
        Console.WriteLine("File: {0} ({1} MB)", f.Name, f.Length / BytesPerMb);
    }
    Console.WriteLine();
}

Answer 4

I would first drop the list of files into an SQL table. Something like this, but with a size column:

CREATE TABLE #DIR (fileName varchar(100))

INSERT INTO #DIR
EXEC master..xp_CmdShell 'DIR C:\RTHourly\*.xml /B'

Then these will be the following statements:

SELECT *,
CASE WHEN SIZE < 5 THEN 1
WHEN SIZE < 10 THEN 2
...
END AS Grouping
FROM #DIR
ORDER BY Grouping, FileName, Size

There is a special security setting that you must change on SQL Server. See Blog Post HERE.