[concurrency-interest] ConcurrentSkipListMap performance

Ted Yu ted_yu at yahoo.com
Tue Oct 25 22:03:17 EDT 2011


Peter:
We use the following in MemStore so that we can narrow our search:
      SortedSet<KeyValue> ss = kvset.tailSet(firstKv);
where kvset is KeyValueSkipListSet which is built on ConcurrentSkipListMap

If you're really interested, see http://hbase.apache.org/book/regions.arch.html 8.6.4.1
You can also get the source code by following http://hbase.apache.org/source-repository.html

Cheers



________________________________
From: Peter Hawkins <hawkinsp at cs.stanford.edu>
To: Ted Yu <ted_yu at yahoo.com>
Cc: "concurrency-interest at cs.oswego.edu" <concurrency-interest at cs.oswego.edu>
Sent: Tuesday, October 25, 2011 6:20 PM
Subject: Re: [concurrency-interest] ConcurrentSkipListMap performance

Hi...

Just out of interest, why does HBase need an ordered concurrent map?
Is there a description of what the relevant module does somewhere?

I was looking for users of ConcurrentSkipListMap via a Google code
search recently but didn't find an awful lot; on a cursory examination
most of the clients I found didn't actually need concurrent ordered
iteration and arguably would have performed better with a
ConcurrentHashMap.

Cheers,
Peter

On Sun, Oct 23, 2011 at 12:01 PM, Ted Yu <ted_yu at yahoo.com> wrote:
> HBase expects ConcurrentSkipListMap to have good performance.
> See the following discussion.
>
> Comments are welcome.
>
> ---------- Forwarded message ----------
> From: Akash Ashok <thehellmaker at gmail.com>
> Date: Sun, Oct 23, 2011 at 2:57 AM
> Subject: Re: SILT - nice keyvalue store paper
> To: dev at hbase.apache.org
>
>
> I was running some similar tests and came across a surprising finding. I
> compared reads and write on ConcurrentSkipListMap ( which the memstore uses)
> and synchronized TreeMap ( Which was literally treemap synchronized).
> Executed concurrent reads, writes and deletes on both of them.
> Surprisingly synchronized treeMap performed better, though just slightly
> better, than ConcurrentSkipListMap which KeyValueSkipListSet uses.
>
> Here are the output of a few runs
>
> Sometimes the difference was considerable
> Using HBaseMap it took 20438ms
> Using TreeMap it took 11613ms
> Time Difference:8825ms
>
> And sometimes the difference was negligible
> Using HBaseMap it took 13370ms
> Using TreeMap it took 9482ms
> Time Difference:3888ms
>
> I've attaching the test  java file which I wrote to test it.
> This might be a very minor differece but still surprising considering the
> fact that ConcurrentSkipListMap uses fancy 2 level indexes which they say
> improves the deletion performance.
>
> And here are the details about the test run.
> 100 Threads each fetching 10,000 records
> 100 threads each adding 10,000 records.
> 100 threads each deletin 10,000 records
> ( Reads, Writes and deletes simultaneously )
>
> Cheers,
> Akash A
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20111025/c2de4841/attachment-0001.html>


More information about the Concurrency-interest mailing list