Avoiding extra space allocation when using standard collections and storing millions of items

Question

Avoiding extra space allocation when using standard collections and storing millions of items

I use standard collections such as List <T>, HashSet <T> and the dictionary <TKey, TValue> to store 100,000 thousand items, up to a million. The case with a specific case is that the set of elements, which was huge in the beginning, grows very slowly and remains in memory for a long time. So the problem that I am facing (even though LOH allocation / fragmentation is another problem) is that these collections consume a lot of memory due to their internal logic, which doubles the allocated memory every time it ends out of free space (in fact, it doubles and looks for the nearest prime number). In my case, when adding is rare, saving extra memory is useless.

Here is a simplified version of how I deal with a list:

public static void Add_GrowSlow<T>([NotNull] this List<T> list, T item, int growStep)
{
    if (list == null) throw new ArgumentNullException(nameof(list));
    if (growStep <= 0)
        throw new ArgumentOutOfRangeException(nameof(growStep));

    var count = list.Count;
    if (list.Capacity == count)
    {
        if (count > 10000)
        {
            list.Capacity = count + growStep;
        }
    }

    list.Add(item);
}

, HashSet/Dictionary . - , ? PowerCollections, .

: , : nuget , , , ,. BCL :

class HashSet<T>
{
    // ...
    protected virtual void IncreaseCapacity() {...}
}

class Dictionary<TKey, TValue>
{
    // ...
    protected virtual void Resize() {...}
}