A complex tree structure causes the GC to pause indefinitely

Question

A complex tree structure causes the GC to pause indefinitely

I study self-study on my own, and currently I am implementing a reverse mode of automatic differentiation as a practice.

The way the program works is, in essence, overloading general expressions, such as multiplication, addition, etc., and creating a tree, the nodes of which will subsequently be called up from the top down. This is probably the first time I've used closure in F # and I really like them, but unfortunately I suspect they are choking on the GC, although I don't know how to check it. I would prefer to ask first what to redesign the algorithm so that it does not use them, since they are very convenient.

The specified tree is created multiple times during program execution, and I am counting on a garbage collector to get rid of it.

At the bottom of the program is the main loop that educates the 2-layer network on the XOR problem. Calling GC.Collect () before the loop or after the loop, and then running it again forces it to go into an indefinite break.

Here are the data structures that I use. I would like some advice if my guesses were correct, and I should redesign the algorithm so as not to use closure or something else does not work.

type dMatrix(num_rows:int,num_cols,dArray: DeviceMemory<float32>) = 
    inherit DisposableObject()

    new(num_rows,num_cols) =
        new dMatrix(num_rows,num_cols,worker.Malloc<float32>(num_rows*num_cols))

    member t.num_rows = num_rows
    member t.num_cols = num_cols
    member t.dArray = dArray

    override net.Dispose(disposing:bool) =
        if disposing then
            dArray.Dispose()

type Df_rec = {
    P: float32 
    mutable c : int 
    mutable A : float32
    }

type DM_rec = {
    P: dMatrix 
    mutable c : int 
    mutable A : dMatrix
    }

type Rf =
    | DfR_Df_DM of Df_rec * (float32 -> dMatrix) * RDM
    | DfR_Df_Df of Df_rec * (float32 -> float32) * Rf

and RDM = 
    | DM of DM_rec
    | DMRb of DM_rec * (dMatrix -> dMatrix) * (dMatrix -> dMatrix) * RDM * RDM // Outside node * left derivative function * right derivative func * prev left node * prev right node.
    | DMRu of DM_rec * (dMatrix -> dMatrix) * RDM

It uses many of the features such as below. Sgemm - a cuBLAS sgemm wrapper. fl out and fr out are closures. They are convenient, but the GC may find it difficult to clean the tree.

let matmult (a: RDM) (b:RDM) =
    let mm va vb =
        let c = sgemm nT nT 1.0f va vb
        let fl out = sgemm nT T 1.0f out vb // The derivative with respect to the left. So the above argument gets inserted from the right left. Usually error * input.
        let fr out = sgemm T nT 1.0f va out // The derivative with respect to the right. So the above argument gets inserted from the right side. Usually weights * error.
        DMRb(DM_rec.create c,fl,fr,a,b)
    let va = a.r.P
    let vb = b.r.P
    mm va vb