[concurrency-interest] implementing a DB write-behind algorithm

Alexandru Popescu the.mindstorm.mailinglist at gmail.com
Tue May 2 20:10:57 EDT 2006


This is gonna be stupid, but I've seen it used on different places
(sourcecode included), and couldn't find a definition [blushing/]:
CAS=Compair-and-swap?

./alex
--
.w( the_mindstorm )p.


On 5/2/06, studdugie <studdugie at gmail.com> wrote:
> Another option is to use a CLQ w/ an AtomicBoolean or (now that I
> think about it) an AtomicInteger (AI).
>
> The logic goes like this. Thread has database write so it increments
> AtomicInteger and compares it against guard variable
> (MAX_CONCURR_WRITERS). If AI <= MAX_CONCURR_WRITERS then enter write
> loop and drain the CLQ.
>
> The code may look something like this.
>
> void dbPut(DbData data)
> {
>     dbWriteCLQ.offer( data );
>
>     try
>     {
>         if(AI.incrementAndGet() <= MAX_CONCURR_WRITERS )
>             while(null !=(data = dbWriteCLQ.poll()))
>                 doDatabaseWrite( data );
>     }
>     finally
>     {
>         AI.decrementAndGet();
>     }
> }
>
> I call it the hijack approach because instead of having one or more
> specialist threads to do the job you use a CAS to "hijack" any thread
> that happens to be passing by at the wrong (or right) place at the
> wrong (or right) time.
>
> Like I said in the first paragraph, you could also use an
> AtomicBoolean to do the hijacking (CAS) but that limits you to one
> thread.
>
> Obviously hijacking isn't the best solution all the time because
> you've pulled a thread away from its normal flow and that flow may be
> time sensitive (ex. responding to an HTTP request). But if you have
> complete control over every thread that can get hijacked you may be
> able to get away w/ it. Your best bet (as always) is slap a profiler
> on the code w/ varying loads after you've determined your throughput
> minimum.
>
> Love, peace, and hair grease,
>
> Dane
>
> On 4/30/06, Alexandru Popescu <the.mindstorm.mailinglist at gmail.com> wrote:
> > Hi!
> >
> > I firstly have to confess that when getting to concurrency related
> > problems, I am getting confused quite quickly :-).
> >
> > Now, the current problem I am trying to solve is: I am trying to
> > figure out how to implement a DB write-behind strategy. Multiple
> > processes will post records to be written to the DB, but the actual
> > writes should happen on a separate process. So, far I was thinking
> > about 2 possible approaches:
> > a) continous write-behind: multiple processes write to a queue which
> > is continously polled by a separate process. When an element is found
> > on the queue, than the write process removes it from queue and
> > attempts to write it to the DB.
> >
> > To have this done, I was looking in the direction of ConcurrentLinkedQueue.
> >
> > b) batched write-behind: multiple processes post to a size-bounded
> > queue. When the max size is reached, the original queue is passed to
> > the parallel write process and replaced with a new queue.
> >
> > To have this done, I was looking in the direction of
> > LinkedBlockingQueue with an additional atomic operation of swapping
> > the old queue with the new empty one.
> >
> > My question is: am I looking in the right direction or I am completely
> > wrong. Any ideas and help are highly appreciated.
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> >
> > _______________________________________________
> > Concurrency-interest mailing list
> > Concurrency-interest at altair.cs.oswego.edu
> > http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
> >
>



More information about the Concurrency-interest mailing list