Re: Want to know something about mlock or shmctl via SHMLOCK

On Sun, 25 Dec 2005 14:32:04 +0100, sorcerer <nagual.hsu@xxxxxxxxx> wrote:

enrio@xxxxxxxxx 寫道：

sorcerer skrev:
>From what you say it sounds like you will just pass the data on to
another process, which will be waiting for it, or possibly first
ensuring somehow that the other process is "ready for it".

But what about using shared memory in the two processes? If the amount
of data is large, isn't shared memory the fastest?  That would be copy
once - to the shared memory. Or rather, generate once - in the shared
memory?


The amount of real data is large but the amount of  the auxiliarydata is small.  The auxiliary data includes the information abount thesize of the real data and the production time of the real data . Theproducer process only reads the real data from some hardware,  put thereal data in the shared memory, and generates the auxiliary data, andthen send the auxiliary data to the consumer processes which are readyfor it.

Yes, the real data is generated once (in a shared ring buffer).
The auxiliary data isn't. Yes, I can put the auxiliary data into anotherring buffer too.  By this way, consumer processes only read the latestauxiliary data from some sort of shared index.

Then what more you need is synchronization. What about using a futex?
Yes. However, we use kernel 2.4. :( 

Oh. :(

So, there are multiple consumer processes. Must each piece of data
be consumed by exactly one consumer?

The following userspace-only code assigns an unsigned 32-bit integer to
each piece of work. Consumers each reserve a piece of work for themselves,
ahead of time. The producer, after producing a piece X, signals the
consumer that reserved X.  Occasionally, the producer could find no
consumer is ready to accept X, in which case it does not signal any
process.  If multiple pieces are produced and still no consumer ready
to consume it, all timing constraints get broken, but not because of
bugs in this code. A consumer that finds a piece of work is already
available does not wait for a signal before continuing.

The code relies on well-known properties of modulo 2^32 arithmetic.

Untested :)

#define NLOG2 8           /* much less than 32 */
#define NAUX (1UL<<NLOG2) /* larger than N_CONSUMERS; must be power of two */
#define AUXMASK (NAUX-1UL)
#define N_CONSUMERS 35    /* each consumer has an id, 0 <= id < N_CONSUMERS */
#define MAX_BACKLOG 1     /* less than NAUX */

typedef struct {
 .../* auxiliary data */
} aux_t;

volatile aux_t aux[NAUX];                    /* in shared memory */
volatile uint32_t aux_read, aux_write;       /* in shared memory */
volatile uint32_t aux_reserved[N_CONSUMERS]; /* in shared memory */

/* Consumers call this. May sleep */
int get_work(int consumer_id)
{
   uint32_t my_aux_read;
   asm("atomically my_aux_read = ++read_aux");
   my_aux_read--;     /* want the old value */
   aux_reserved[consumer_id] = my_aux_read;

   while (1) {
     /* Positive when consumers are behind producer, i.e. data is available */
     int32_t work_available = (int32_t) (aux_write - my_aux_read);

     /* Notice that if aux_read is ahead of aux_write but wraps to small
      * integers before aux_write does, work_available is still negative.
      * If aux_write is ahead and wraps, work_available is still positive,
      * as long as the ideal unwrapped aux_read and aux_write are within
      * +/- NAUX of each other */

     if (work_available > 0)
       break;

     /* The signal is blocked at all times except inside sigsuspend */
     sigsuspend(mask);
   }
   return aux_reserved[consumer_id];
}

/* Producer calls this. Does not sleep */
void issue_work(/* aux data */)
{
   uint32_t work_id = aux_write;
   aux[work_id & AUX_MASK] = /* aux data */;
   asm(" atomically increment aux_write ");

   /* Works also when any of the two has wrapped around to small integers. */
   int32_t consumers_available = (int32_t)(aux_read - aux_write);

   if (consumers_available > 0) {
      /* A consumer has reserved this piece of work */
      int cons;
      for (cons = 0; cons < N_CONSUMERS; cons++)
         if (aux_reserved[cons] == work_id)
            break;

      kill(pid_of(cons), the_signal);
   }
   else if (consumers_available < -MAX_BACKLOG)
      SERIOUS_FAILURE("Issue more consumers, or buy faster computers!");
}

The danger here is too few signals, not too many.  "Too many" could result
if the consumer did not call sigsuspend() in the previous call of get_work(),
so there is a signal pending left over from then.  The consumer simply loops
and suspends again. This needs not cost any context switch.

Too few signals would happen if the producer finds no-one has reserved the
piece of work just issued, so there is no-one it can signal. In detail,
the producer issues piece X, increasing aux_write to X+1.  It looks at
aux_read and finds aux_read <= X.  For this to be wrong, there must be a
consumer who finds aux_read == X, increases it to X+1, and looks at
aux_write but finds aux_write <= X. But aux_write was atomically increased
to X+1 before the producer inspected aux_read and found aux_read <= X.
After the consumer has atomically increased aux_read to X+1 it cannot find
aux_write still <= X.

You mean you doubt it is free of bugs?  (I fail to see how being "no
linux VMM expert" helps avoiding bugs.) Or did I miss something?
Yes. I doubt it is free of bugs. 

> I did not take the
> reference of ptrace.c while I implemented that.

Sorry, what do you mean? 

/usr/src/linux/kernel/ptrace.c uses get_user_pages as some sort of
memory copy.   I should take that as a reference when I implementedmemory copy between processes.


Ah, now all is clear.

-Enrique
.

Relevant Pages

  • Problems with shared memory to communicate between two Linux tasks...
    ... I recently tried to use shared memory to communicate between two Linux ... program called shm1.c is a consumer and shm2.c is the producer. ...
    (comp.os.linux.misc)
  • Re: Problem of fast data transfer
    ... I mean message queue, shared memory. ... Nils have rightly said simple shared memory is fast, but synchronization using semaphore added over head. ... As I have some basics of writing char device drivers, I was wondering whether I can use software char device concept. ... and halt the producer until the consumer is ...
    (comp.os.linux.development.apps)
  • Re: Multithreading - Problem with notifyAll() and wait()
    ... NotifyAlland waitstatements (producer notifies and consumer ... private Queue queue = new Queue; ... Vector vectorQueue = new Vector; ...
    (comp.lang.java.programmer)
  • Re: Gas prices up to pay for bosses
    ... kilter profits become the more competition intensified. ... of the producer ought to be attended to, only so far as it may be necessary ... for promoting that of the consumer. ...
    (uk.politics.misc)
  • Re: Is this group dying?
    ... consumer queue toward the dispatcher thread and a single-producer/ ... that producer has put a new "next" node in place. ... I want to implement the wait-free factory as a multi-threaded daemon process ...
    (comp.programming.threads)