Golang: How to get the total size of a directory?

Question

So, I'm trying to get the total size of the directory using golang, so far I have this

var dirSize int64 = 0

func readSize(path string, file os.FileInfo, err error) error {
    if !file.IsDir() {
        dirSize += file.Size()
    }
    return nil
} 

func DirSizeMB(path string) float64 {
    dirSize = 0
    filepath.Walk(path, readSize)
    sizeMB := float64(dirSize) / 1024.0 / 1024.0
    sizeMB = Round(sizeMB, .5, 2)
    return sizeMB
}

So the question is, will the dirSize global variable lead to problems, and if so, how can I move it to the scope of the DirSizeMB function?

Answer 1

Using a global like at best is bad practice. It is also a race if DirSizeMBcalled simultaneously.

A simple solution is to use closure, for example:

func DirSize(path string) (int64, error) {
    var size int64
    err := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error {
        if !info.IsDir() {
            size += info.Size()
        }
        return err
    })
    return size, err
}

Playground

You can assign a closure to a variable if you think it looks better.

Answer 2

If you want to use a variable, you can do this:

func DirSizeMB(path string) float64 {
    var dirSize int64 = 0

    readSize := func(path string, file os.FileInfo, err error) error {
        if !file.IsDir() {
            dirSize += file.Size()
        }

        return nil
    }

    filepath.Walk(path, readSize)    

    sizeMB := float64(dirSize) / 1024.0 / 1024.0

    return sizeMB
}

Answer 3

, , - DirSizeMB readSize , . .

func DirSizeMB(path string) float64 {
    sizes := make(chan int64)
    readSize := func(path string, file os.FileInfo, err error) error {
        if err != nil || file == nil {
            return nil // Ignore errors
        }
        if !file.IsDir() {
            sizes <- file.Size()
        }
        return nil
    }

    go func() {
        filepath.Walk(path, readSize)
        close(sizes)
    }()

    size := int64(0)
    for s := range sizes {
        size += s
    }

    sizeMB := float64(size) / 1024.0 / 1024.0

    sizeMB = Round(sizeMB, 0.5, 2)

    return sizeMB
}

http://play.golang.org/p/zzKZu0cm9n

?

, , filepath.Walk readSize. , , , goroutines (, , , ). , , concurrency, , .

The answer @DaveC gives shows how to do this, using closure on a local variable, solves the problem of having a global variable, so multiple simultaneous calls to DirSize would be safe. Docs for Walk explicitly states that the walk function works on files in a deterministic order, so its solution is enough for this problem, but I will leave this as an example of how to make it safe to execute an internal function at the same time.