Is it difficult to implement Go concurrency templates in Scala?

Question

There is no doubt about this; the Go syntax is much simpler than Scala. It also has less language features. I really like the ease with which you can write concurrency code with Go.

As it turned out, the executable code does not block the code (see http://norvig.com/21-days.html#Answers ), and both Go and Scala are very good at this.

My question is how you can write programs in Scala that behave exactly like Go programs by implementing the same concurrency patterns. The first thing that comes to mind is the use of Futures in the same way with channels.

I'm looking for

• possible implementations of Go concurrency templates in Scala
• if Go constructors are difficult to accurately mimic in Scala
• code snippets

Any help is greatly appreciated.

[Edit] Some examples of Go concurrency templates http://talks.golang.org/2012/concurrency.slide

Fan in

func fanIn(input1, input2 <-chan string) <-chan string { c := make(chan string) go func() { for { select { case s := <-input1: c <- s case s := <-input2: c <- s } } }() return c } 

Timeout granularity (one channel and the whole conversation)

Replicate service calls between multiple instances and return the value of the first response. (this uses a set of patterns)

All with: No locks. No condition variables. No callbacks. (Scala Futures use callbacks)

Answer 1

Go has concurrency functions built into the main language, while Scala uses the parallel package and concurrency primitives from Java java.util.concurrent .

In Scala, it is idiomatic for using either concurrency or an Actor Model , whereas Go concurrency on Hoare, reporting sequential processes .

Although the concurrency primitives between the two languages ​​do not match, there seems to be some similarities.

In Go, concurrency is usually achieved using Goroutines and Channels . There are also other more traditional low-level synchronization primitives, such as mutexes and waiting groups .

In Scala, as far as I know, any class declared "Runnable" will be run in a separate thread and will not be blocked. It is functionally similar to goroutines.

In Scala, Queues can be used to transfer information between procedures similar to channels in Go.

EDIT: As pointed out by Chuck , the “crucial difference between Scala Queues and Go channels is that by default, Go channel block on write until something is ready to read from them and block reading until something is ready to write them". This would have to be written to any Scala channel implementation.

EDIT 2: As Maurício Linhares points out: "You can do concurrency without visible callbacks in Scala using Async-github.com/scala/async - but you can't do it without callbacks at all, it's just not possible considering the way JVM is implemented . ".

Thanks to everyone for the constructive comments.

For more details see:

• http://golang.org/doc/effective_go.html#concurrency
• http://www.scala-lang.org/api/current/index.html#scala.concurrent.package
• http://twitter.imtqy.com/scala_school/concurrency.html
• https://github.com/scala/async
• http://akka.io/docs/

Answer 2

The short answer is no, it’s not difficult.

As you know, concurrency through message passing can work with blocking or non-blocking synchronization primitives. Go channels can do both things: they can be unbuffered or buffered - you choose.

JVM languages ​​say a lot that non-blocking concurrency is always superior. This is not so at all; it's just a feature of the JVM that threads on it are quite expensive. In response, most JVM concurrency APIs provide only a non-blocking model, although this is unfortunate.

For a relatively modest concurrency up to, say, 1000 JVM threads, concurrency blocking can work very efficiently even on the JVM. Since this style does not include any callbacks, it is easy to write and then read later.

The excellent JCSP library from the University of Canterbury is a good way to write Java / Scala / ... programs using CSP . This is the same style as Go; JCSP channels are very similar to Go channels, providing the option of unbuffered or buffered (or rewritten fixed buffers) sizes. Its select is called Alternative and has been verified by the right JCSP developers through formal analysis.

But since the JVM cannot really support more than 1000 or so threads, this is not suitable for some areas of the application. But then, go ...

---

Footnote: The current version of JCSP is v1.1rc5 in the Maven repositories, contrary to what the JCSP site says.

Answer 3

A good implementation is nontrivial.

those. you can implement one in "lock" mode, where each go blocking algorithm (wait channel) actually blocks the execution thread. Implementation will be trivial but useless.

An alternative is a mechanism that allows you to asynchronously wait for a "pause / resume" execution thread. Since we do not have built-in support for continuations in the JVM, the implementation of this is quite complicated and requires transformation of the AST or bycoding of the bytecode.

To implement approach No. 1 (i.e., with AST conversion on top of SIP-22 async), you can see https://github.com/rssh/scala-gopher (warning: I'm the author).

Answer 4

There appears to be a third-party lib (Netflix) that provides reactive extensions for Scala (but also Java and other JVM languages). Observed RX values ​​can be tested similarly to Go channels.

https://github.com/Netflix/RxJava/tree/master/language-adaptors/rxjava-scala

Documentation is also useful, providing visual representations of common patterns.

Answer 5

Implementing the CSP concurrency style on the JVM is not easy, whether for Java or for Scala. The reason is that CSP is based on threads with a reduced context, which are often called green threads. The reduced context consumes less memory, which means that you can run much greener threads than OS threads or Java threads (1 Java thread corresponds to 1 OS thread). I once tried: with 4 GB of RAM, you can run about 80,000 gorts (the green thread option in Go) compared to about 2,000 Java threads.

Now, why does it matter? The idea in CSP is that if there is no data in any channel, there is “only” one green stream, lost, which now sits on that channel until it receives an input. Say you have access to a web application of 40,000 users. 80,000 Goroutines, which can be run on a machine with 4 GB of RAM, can process these 40,000 connections on the spot (1 incoming connection and one outgoing connection). Without green threads, you need much more memory or more servers.

Another point in green streams is that you just don't need to worry if the green stream is on the channel, since you have so many. Now with channel-oriented code, you can look at the code that is behind the asynchronous one, as if it were synchronous. The flow of messages through pipes is as simple as any other method calls. Robert Pike explains this well in this Youtube-Video at around 29:00. This makes CSP-style parallel code much easier from the start, as well as easier to find concurrency related errors.

Another question is the continuation. Let's say you have a function that consumes data from two channels in a row and somehow calculates the data. Now the first channel has data, and the second - no. Therefore, when the second channel receives data, the language runtime should skip inside the function to the place where the second channel transmits the data to the function. In order to be able to do this, the runtime must remember where to jump, and he needed to store the data taken from the first channels and restore them, because they are calculated along with the data of the second channel. This can be done on the JVM using continuation libraries that use byte code injection to stitch intermediate results and remember where to go. One library for Java nad Kotlin that can do this is Quasar: http://docs.paralleluniverse.co/quasar/ Quasar also has fibers that serve as a means for something like green threads on the JVM. Quasar’s developer is Ron Presler, who was hired by Oracle to work on Projekt Loom: http://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.html The idea of ​​this project is to support fibers and extensions at the JVM level, which would make the fibers more efficient, and introducing the byte code to continue is less cumbersome.

Then Kotlin also has Coroutines: https://kotlinlang.org/docs/reference/coroutines.html Kotlin Couroutines also implements the fibers and extensions provided by the Kotlin compiler, so the developer does not need to help the CSP library (e.g. Quasar), knowing which function is required to enter a byte code.

Unfortunately, Kotlin Couroutines are for Kotlin only and cannot be used outside of it. Therefore, they are not available for other JVM languages. Quasar does not work with Scala, since inserting byte code for Scala for continuations will be much more complicated than for Java or Kotlin, since Scala is a much more complex language. At least these are the arguments made by the Quasar developer.

So, the best thing to do, like Scala, is to stick with Akka or wait for Project Loom to complete. Then, some Scala people can begin to implement CSP for Scala at a level that really implements CSP. At the time of writing, Project Loom works, but is not yet officially approved by Oracle. Therefore, it is still unclear whether some future JDK will contain those things that are necessary for a full-blown CSP.