Mnesia: how to block multiple lines at once so that I can write / read a "consistent" set of records

Question

HOW I WANT THAT I ASKED MY QUESTION TO START WITH

Take a table with 26 keys, az and let them have integer values. Create a process, Ouch, that does two things over and over.

• In one transaction, write random values ​​for a, b, and c so that these values ​​always add up to 10
• In another transaction, read the values ​​for a, b, and c and complain if their values ​​don't add up to 10

If you deploy a few more of these processes, you will see that a, b and c are very quickly in a state where their values ​​do not add up to 10. I believe that there is no way to ask mnesia "to lock these 3 records before recording (or read) ", you can only mnesia block records, because they get to them (so to speak), which allows you to set a set of record values ​​to violate my restriction" should sum to 10 ".

If I'm right, solutions to this problem include

• block the whole table before writing (or reading) a set of 3 records - I do not like to block the whole table for 3 speeches,
• Create a process that keeps track of who reads or writes which keys and protects bulk operations from anyone who writes or reads until the operation is complete. Of course, I would have to make sure that all processes used this ... shit, I think that means I wrote my own AccessMod as the fourth parameter for / 4 activity, which looks like a non-trivial exercise.
• Something else that I'm not smart enough to understand.

thoughts?

Ok, I'm an ambitious rookie Erlang, sorry if this is a stupid question, but

I am creating an application-specific distributed cache, and I need to be able to write sets of Key, Value pairs in one transaction, and also retrieve sets of values ​​in one transaction. In other words, I need 1) Write 40 keys, values ​​to the cache and make sure that no one else can read or write any of these 40 keys during this operation with several keys; and also, 2) Read 40 keys in one operation and get 40 values, knowing that all 40 values ​​have remained unchanged from the beginning of this read operation to its end.

The only way I can do this is to lock the entire table at the top of the fetch_keylist ([ListOfKeys]) or at the top of the write_keylist ([KeyValuePairs], but I don’t want to do this because I have many processes running their own at the same time multi-user reads and writes, and I don’t want to lock the entire table at any time when any process needs to read / write a relatively small subset of the records.

reference

Trying to be more clear: I don't think it's just using vanilla transactions

I think I'm asking a more subtle question than this. Imagine that I have a process that iterates through 10 records in a transaction, fixing them as they appear. Now imagine that this process begins, but before he repeats the third record. ANOTHER process updates the third record. This will be great, because the transactions go because the first process has not blocked the 3rd record (for now), and the OTHER process has changed it and released it before the first process reached it. I want to guarantee that as soon as my first process starts, no other process can touch 10 records until the first process is executed with them.

SOLVING THE PROBLEM - I IDIOT ... I suppose ... Thank you all for your patients, especially Hynek -Pichi- Vychodil! I prepared my test code to show the problem, and I could actually reproduce the problem. Then I simplified the code for readability, and the problem disappeared. I could not reproduce the problem again. This is both confusing and mysterious for me, because I had this problem for several days. In addition, mnesia never complained that I perform operations outside of a transaction, and I have no dirty transactions anywhere in my code, I have no idea how I was able to introduce this error into my code!

I hit the concept of Isolation in the head and have no doubt that it exists again.

Thanks for the education.

In fact, it turns out the problem was using try / catch in mnesia operations inside a transaction. See here for more details.

Answer 1

Transaction Mnesia will do just that thing for you. This is a transaction if you are not performing dirty operations. So just put your write and read operations in one transaction, and mnesia will take a break. All operations in one transaction are performed as one atomic operation. Mnesia's transaction isolation level is what is sometimes called "serializable," i.e. The strongest level of isolation.

Edit:

You seem to have missed one important point about parallel processes in Erlang. (To be fair, this is not only true in Erlang, but also in any truly parallel environment, and when someone argues differently, this is not a true parallel environment.) You cannot distinguish which action occurs earlier and what the second, if You do not do some synchronization. Only in this way can you perform this synchronization using messaging. You have guaranteed only one message message in Erlang, the ordering of messages sent from one process to another process. This means that when you send two messages M1 and M2 from process A to process B they arrive in the same order. But if you send message M1 from A to B and message M2 from C to B , they can arrive in any order. Just because you can even say which message you sent first? Even worse, if you send a message M1 from A to B and then M2 from A to C and when M2 arrives at C sends M3 from C to B you did not guarantee that M1 will reach B to M3 . Even this will happen in one virtual machine in the current implementation. But you cannot rely on it, because it is not guaranteed and can change even in the next version of VM, only thanks to the implementation of message passing between different schedulers.

It illustrates the problems of ordering events in parallel processes. Now back to the mnesia transaction. Mnesia surgery should be free from the side effects of fun . This means that there can be no messages sent outside the transaction. Thus, you cannot determine which transaction starts from the beginning and at the start. The only thing you can say if the transaction is successful, and they order, you can only determine by its effect. When you consider this, your subtle explanation does not make sense. One transaction will read all the keys in the atomic operation, even if it is implemented as reading one key by one in the implementation of the transaction, and your write operation will also be performed as an atom operation. You cannot determine if the fourth key was recorded in the second transaction after you read the 1st key in the first transaction, because there it is not observed externally. Both transactions will be executed in a specific order as a separate atomic operation. From an external point of view, all keys will be read at the same moment in time, and it is mnesia's job to force it. If you send a message from within a transaction, you are violating the mnesia transaction property, and you cannot be surprised that it will behave strangely. To be specific, this message can be sent many times.

Edit2:

If you deploy even a few of these processes, you will see that a, b and c are very quickly in a state where their values ​​do not add up to 10.

I'm curious why you think this will happen, or did you test it? Show me your test case and I will show my:

 -module(transactions). -export([start/2, sum/0, write/0]). start(W, R) -> mnesia:start(), {atomic, ok} = mnesia:create_table(test, [{ram_copies,[node()]}]), F = fun() -> ok = mnesia:write({test, a, 10}), [ ok = mnesia:write({test, X, 0}) || X <- [b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z]], ok end, {atomic, ok} = mnesia:transaction(F), F2 = fun() -> S = self(), erlang:send_after(1000, S, show), [ spawn_link(fun() -> writer(S) end) || _ <- lists:seq(1,W) ], [ spawn_link(fun() -> reader(S) end) || _ <- lists:seq(1,R) ], collect(0,0) end, spawn(F2). collect(R, W) -> receive read -> collect(R+1, W); write -> collect(R, W+1); show -> erlang:send_after(1000, self(), show), io:format("R: ~p, W: ~p~n", [R,W]), collect(R, W) end. keys() -> element(random:uniform(6), {[a,b,c],[a,c,b],[b,a,c],[b,c,a],[c,a,b],[c,b,a]}). sum() -> F = fun() -> lists:sum([X || K<-keys(), {test, _, X} <- mnesia:read(test, K)]) end, {atomic, S} = mnesia:transaction(F), S. write() -> F = fun() -> [A, B ] = L = [ random:uniform(10) || _ <- [1,2] ], [ok = mnesia:write({test, K, V}) || {K, V} <- lists:zip(keys(), [10-AB|L])], ok end, {atomic, ok} = mnesia:transaction(F), ok. reader(P) -> case sum() of 10 -> P ! read, reader(P); _ -> io:format("ERROR!!!~n",[]), exit(error) end. writer(P) -> ok = write(), P ! write, writer(P). 

If this does not work, it will be a really serious problem. There are serious applications, including payment systems, that rely on it. If you have a test case that shows that it is broken, report an error on the [e-mail]. page

Answer 2

Have you tried mnesia events ? You can force the subscriber to subscribe to mnesia Table Events especially write , so as not to interrupt the recording process. Thus, mnesia simply sends a copy of what was written in real time to another process that checks what values ​​are there at any given time. take a look at this:

  subscriber () ->
     mnesia: subscribe ({table, YOUR_TABLE_NAME, simple}),
     %% OR mnesia: subscribe ({table, YOUR_TABLE_NAME, detailed}),
     wait_events (). 

 wait_events () ->
 receive
      %% For simple events
     {mnesia_table_event, {write, NewRecord, ActivityId}} -> 
         %% Analyse the written record as you wish
         wait_events ();
      %% For detailed events
     {mnesia_table_event, {write, YOUR_TABLE, NewRecord, [OldRecords], ActivityId}} ->
         %% Analyse the written record as you wish
         wait_events ();
     _Any -> wait_events ()
 end.

Now you create your analyzer as a process like this:

  spawn (? MODULE, subscriber, []).

This makes the whole process work without blocking the process, mnesia does not require locking any tables or records, because now you have a writer and analyser . All this will work in real time. Remember that there are many other events that you can use, if you want, according to the pattern matching them in the receiver body wait_events() .
It is possible to build a heavy duty gen_server or a complete application designed to receive and analyze all your mnesia events. It is usually better to have one capable subscriber than many unsuccessful event subscribers. If I understand well that you asked the question correctly, this unblocking solution meets your requirements.

Answer 3

mnesia: read / 3 with write locks seems enough.

A Mnesia transaction is implemented using read and write locks, and the locks are well formed (lock lock until the end of the transaction). Thus, the isolation level is serializable.

The granularity of the locks depends on each record, as long as you gain access to the primary key.