Multicore and Concurrency - Languages, Libraries, and Development Methods

Question

The architecture of the processor architecture has changed; several cores are a trend that will change the way we develop software. I did multi-threaded development in C, C ++ and Java, I did multi-processor development using various IPC mechanisms. Traditional approaches to the use of threads, apparently, do not allow the developer to use hardware that supports a high degree of concurrency.

What languages, libraries, and development methods do you know about this help to alleviate the traditional problems of creating parallel applications? I obviously think about issues like dead ends and race conditions. Also interesting are design methods, libraries, tools, etc., which help to actually take advantage of and ensure the availability of available resources. Simply creating a reliable, reliable, multi-threaded application does not guarantee that it uses all available cores.

What I have seen so far:

• Erlang : process based, IPC messaging, concurrency actor model
• Dramatis : Member Model Library for Ruby and Python
• Scala : functional programming language for JVM with added concurrency support
• Clojure : a functional programming language for the JVM with a member library.
• Termite : Erlang process process port and message flowing to Schema

What else do you know about what worked for you and what do you think is interesting to watch?

Answer 1

I would suggest two paradigm shifts:

Program Transactional Memory

You can take a look at the concept of Transactional Memory (STM). The idea is to use an optimistic concurrency: any operation executed in parallel with another tries to shut down in an isolated transaction; if at some point another transaction was made that invalidates the data on which the transaction is running, the transaction work is discarded and the transaction is executed again.

I think the first widely known implementation of the idea (if not the proof of concept and the first) is like this in Haskell: Documents and presentations about transactional memory in Haskell . Many other implementations are listed in the Wikipedia article STM .

Event loops and promises

Another very different way of working with concurrency is implemented in the programming language [E] ( http://en.wikipedia.org/wiki/E_(programming_language%29) .

Note that his way of working with concurrency, as well as other parts of the language design, is largely based on the Actor model.

Answer 2

You mentioned Java, but you only mention topics. Have you looked at the Java parallel library? It comes bundled with Java 5 and above.

This is a very good library containing ThreadPools, CopyOnWriteCollections, to name a few. Check out the documentation in the Java Tutorial . Or, if you want, Java documents .

Answer 3

Some materials based on Scala:

• PiLib: Hosted Language for the Pi-Calculus Concurrency Style
• Event-based programming without control inversion
• Actors bringing together themes and events
• Scala Multicast Actors: Architecture and Implementation
• Deploying Associations Using Extensible Template Matching
• Communication Scala Objects (revised)

Answer 4

I used processing for Python. It mimics the thread module API and is therefore fairly easy to use.

If you happen to use map/imap or a generator / list, converting your code to use processing very simple:

 def do_something(x): return x**(x*x) results = [do_something(n) for n in range(10000)] 

can be parallelized with

 import processing pool = processing.Pool(processing.cpuCount()) results = pool.map(do_something, range(10000)) 

which will use as many processors as necessary to calculate the results. There are also lazy ( Pool.imap ) and asynchronous options ( Pool.map_async ).

There is a queue class that implements Queue.Queue , and Queue.Queue like workers.

Gotchas

processing based on fork() , which should be emulated on Windows. Objects are unpickle using pickle / unpickle , so you need to make sure this works. To form a process that has already received resources may not be what you need (for example, connecting to a database), but in general it works. It works so well that it was added in Python 2.6 on an accelerated path (see PEP-317 ).

Answer 5

Intel Threading Building Blocks for C ++ looks very interesting to me. It offers a much higher level of abstraction than raw streams. O'Reilly has a very good book if you like dead tree documentation. See Also Any experiments with streaming network building modules? .

Answer 6

I would say:

Models: threads + general state, actors + messaging, transactional memory, card / reduction? Languages: Erlang, Io, Scala, Clojure, Reia Libraries: Retlang, Jetlang, Kilim, Cilk ++, fork / join, MPI, Kamaelia, Terracotta p>

I maintain a concurrency link blog about such things (Erlang, Scala, Java threading, actor model, etc.) and post a couple of links per day:

http://concurrency.tumblr.com

Answer 7

Question: What parallel programming model do you recommend today to take advantage of tomorrow's multi-core processors? has already been asked. I also gave the following answer.

Kamaelia is a Python development environment for building applications with many interacting processes.

Kamaelia - Concurrency Has Been Helpful, Fun

At Kamaelia, you build systems from simple components that interact with each other . This speeds up development, makes maintenance much easier, and also means that you create naturally parallel software . It is intended for access to any developer, including beginners. It also makes it fun :)

Which systems? Network servers, clients, desktop applications, Pygame-based games, transcoding systems and pipelines, digital television systems, spam eradication tools, training tools and much more :)

Here's a video from Pycon 2009. It starts by comparing Kamaelia with Twisted and Parallel Python, and then gives you an introduction to the Kamaelia demo.

Easy Concurrency with Kamaelia - Part 1 (59:08)
Easy Concurrency with Kamaelia - Part 2 (18:15)

Answer 8

I have been doing parallel programming in Ada for almost 20 years.

The language itself (not tied to the library) supports threads ("tasks"), several planning models, and several synchronization paradigms. You can even create your own synchronization schemes using the built-in primitives.

You can think of Ada rendezvous as a kind of procedurally oriented means of synchronization, and protected objects are more object-oriented. Rendezvous monitors are similar to the old CS-concept, but much more powerful. Protected objects are special types with synchronization primitives that allow you to create things in the same way as OS locks, semaphores, events, etc. However, it is powerful enough that you can also invent and create your own synchronization objects, depending on your exact needs.

Answer 9

I closely monitor Parallel Extensions for .NET and Parallel LINQ .

Answer 10

I know Reia , an Erlang-based language, but more like Python / Ruby.

Answer 11

This question is closely related to, if not a duplicate. What parallel programming model do you recommend today to take advantage of multiprocessor processors tomorrow?

Answer 12

In Java there is an actor library that you know . And did you know that J ava is a functional language? ;)

Answer 13

OpenMP

It processes the threads for you, so you only worry about which parts of your C ++ application you want to run in parallel.

eg.

 #pragma omp parallel for for (int i=0; i < SIZE; i++) { // do something with an element } 

the above code will start the for loop on as many threads as you said while OpenMP was running, so if SIZE is 100 and you have a quad-core box, 25 elements will be executed for the loop on each core.

There are several other parallel extensions for different languages, but the ones that interest me the most are those that run on your video card. This is real parallel processing :) (examples: GPU ++ and libSh )

Answer 14

C ++ 0x will provide std::lock functions to lock more than one mutex together. This will help eliminate the deadlock due to out of order locking. In addition, the C ++ 0x thread library will have promises, futures, and packaged tasks that allow the thread to wait for the result of an operation performed on another thread without any locks at the user level.

Answer 15

I started with .Net Parallel Extensions. However, this is CTP, and it is changing in every new reality. Now I am using C # with examples of ThreadPool, BackgroundWorker and Thread. I am rewriting several critical processes in a medium-sized application. I did not know where to start. However, I bought an e-book version of the book Programming in C # 2008 and 2005, by Gaston C. Hillar - Packt Publishing - http://www.packtpub.com/beginners-guide-for-C-sharp-2008 -and-2005-threaded-programming / book , 7 days ago. I bought an e-book from publishers, but now the book is available on Amazon.com. Highly recommended for C # programmers. I downloaded the code and I started doing the exercises. The book is a good guide with lots of code for practice. I read the first 6 chapters. He tells stories, while he explains the most complex concepts. It's good. Nice to read. I could see that the Core 2 Quad Q6700 achieves 98% processor utilization in C # using 4 simultaneous threads! This is easier than I thought. I am impressed with the results you can achieve using many cores at the same time. I recommend the book to anyone interested in starting multi-core or multi-threaded programming using C #.

Answer 16

multiprocessing is a python library that simplifies multi-core programming as mentioned in another answer.

A program written with python multiprocessing can be easily modified to deliver work in the cloud, rather than to local kernels. piCloud uses this to provide great on-demand processing power in the cloud: you just need to change 2 lines of your code.

So, here’s a takeaway: when choosing a library for multi-core, you may ask if the cloud approach makes sense.