Why is my Go program running much worse than expected in the following scenario?

Question

I solved a simple question in C, Go and Python and compared the results. The solution just needs to have two equations inside the if - else block. Below is the code for my solution in:

C

python

go

I compared the way the three languages ​​work with floating point results and therefore made this script to generate test cases and this to compare the results, two at a time.

Strange is the time taken to run the first 3 scripts. Program C obviously goes through a couple of seconds. Python takes about 2.5-3 seconds. However, Go takes about 24-25 seconds to run the program in test cases generated using the generate_test_cases.py script.

I thought Go would be somewhere between C and Python, as far as running time is concerned. I am doing something inefficiently with my Go code, and if so, then what?

PS I also ran the above three programs without file processing operations and all the same result, i.e. Go takes an extremely long time compared to the other two.

Answer 1

Like Max, my strong suspicion was that Go's slowness was due to poor I / O performance. I tested this hypothesis:

 package main import "fmt" import "os" import "time" func main(){ now := time.Now() input,_ := os.Open("testing/test_cases.txt") defer input.Close() output,_ := os.Create("testing/Goutput.txt") defer output.Close() var ncases int var p float64 fmt.Fscanf(input,"%d",&ncases) fmt.Println("Opened files in ", time.Since(now), "seconds") now = time.Now() cases := make([]float64, ncases) fmt.Println("Made array in ", time.Since(now), "seconds") now = time.Now() for i := 0; i < ncases; i++ { fmt.Fscanf(input,"%f",&cases[i]) } fmt.Println("Read data in ", time.Since(now), "seconds") now = time.Now() for i := 0; i < ncases; i++ { p = cases[i] if p >= 0.5 { cases[i] = 10000 * (1-p) * (2*p-1) + 10000 } else { cases[i] = p*(1-2*p)*10000 + 10000 } } fmt.Println("Processed data in ", time.Since(now), "seconds") now = time.Now() for i := 0; i < ncases; i++ { fmt.Fprintln(output, cases[i]) } fmt.Println("Output processed data in ", time.Since(now), "seconds") } 

Running it, he made this conclusion:

  Opened files in 2.011228ms seconds
 Made array in 109.904us seconds
 Read data in 4.524544608s seconds
 Processed data in 10.083329ms seconds
 Output processed data in 1.703542918s seconds

So, it looks like on my machine all the math comes up after about 10 ms, but I / O is slow, confirming the hypothesis. As Yann noted in the comments, there are more likely to be faster options than fmt .

Update: For example, wrapping input and output with bufio "Readers and Writers":

 binput := bufio.NewReader(input) boutput := bufio.NewWriter(output) 

and using binput and boutput for buffered I / O, your original version runs on my machine in 2.1 seconds, which is slightly faster than Python 2.7.

Update 2: I noticed that I get different results by simply switching to buffered I / O.

• Turns out you also need to tweak the format strings to include \n , as in version C. I think this is actually more correct anyway, but it looks like you can leave with it until you are buffered.

• It is also important Flush() your buffer output, which I made, but not mentioned earlier.

Here is my full buffer solution:

 package main import "fmt" import "os" import "bufio" import "time" func main(){ now := time.Now() nbinput, _ := os.Open("testing/test_cases.txt") defer nbinput.Close() nboutput, _ := os.Create("testing/Goutput.txt") defer nboutput.Close() binput := bufio.NewReader(nbinput) boutput := bufio.NewWriter(nboutput) var ncases int var gain, p float64 fmt.Fscanf(binput,"%d\n",&ncases) for i := 0; i < ncases; i++ { fmt.Fscanf(binput, "%f\n", &p) if p >= 0.5 { gain = 10000 * (1-p) * (2*p -1) } else { gain = p*(1-2*p)*10000 } fmt.Fprintln(boutput, gain+10000) } boutput.Flush() fmt.Println("Took ", time.Since(now), "seconds") } 

Answer 2

I think the following lines are slower in go.

  fmt.Fscanf(input,"%f",&p) fmt.Fprintln(output,gain+10000) 

There is magic when you do IO. They look synchronous, but they are actually asynchronous. Go rotine performs an asynchronous request and returns control to the scheduler. The planner is looking for another goroutine awaiting control, but there is only one io wait. Thus, the scheduler loops do nothing.

If you had 2, 10 or 100 simultaneous goroutines, then they will see better performance.