PlanetJava
Custom Search

java-javagroups-devel
[Top] [All Lists]

Re: [jgroups-dev] [javagroups-users] A memcached implementation in JGrou

Subject: Re: [jgroups-dev] [javagroups-users] A memcached implementation in JGroups
Date: Thu, 04 Sep 2008 14:10:49 -0500
The JVM memory allocator is efficient, and it even has support for large 
pages (-XX:+UseLargePages) if TLB usage is a concern. The overall 
footprint in Java, however, would of course be bigger.
Also, direct buffers are often worse than regular heap, since you have 
the malloc() cost and the extra copying.
Bela Ban wrote:
> Correct, but that's a feature of Java versus C in general, and not 
> PartitionedHashMap in particular.
> 
> memcached uses something similar to a buddy memory allocation scheme 
> ([1]), which is great, but they need to make sure they don't waste 
> memory. For instance, if you always allocate pages of 500 bytes, then 
> this mechanism is not the best, because the smaller pages won't get 
> used, and the larger pages are wasted, unless they get fragmented into 
> smaller ones.
> 
> I'll take the stance that, unless you know exactly what the avg size of 
> your app's memory requirements is, the OS does a better job at 
> allocating memory and in addition you'll benefit from future 
> improvements in the mem allocator code of your OS.
> 
> memcached probably shines when you know exactly what the memory pages 
> sizes are and you change the src to accommodate that.
> 
> I'd also claim that even with GC, this is very useful, because the few 
> GC cycles are a good tradeoff against having to go to the DB.
> 
> Note that we could implement something like memcached's memory 
> allocator, too: grab direct memory (ByteBuffer.allocateDirect() / 
> MappedByteBuffer), divide it up into lists of fixed sizes (buddy pools) 
> and then use those buffers to store data. Direct memory is allocated 
> outside of the Java heap, so it will never get garbage collected, but 
> TBH I'm not sure this is a good idea. I mean, we're coming back to Java 
> versus C here. There's a reason I switched and a big part is garbage 
> collection and the avoidance of dangling pointers.
> 
> I've attached the doc describing memcached's memory allocation strategy.
> 
> [1] http://en.wikipedia.org/wiki/Buddy_memory_allocation
> 
> Hanson Char wrote:
>> One of the major benefits of using the native memcached is that unlike 
>> a JVM, GC (full GC in particular) can be entirely avoided.  Wouldn't 
>> that benefit be lost if a memcached impl is done entirely in Java ?
> 
-- 
Jason T. Greene
JBoss, a division of Red Hat
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Javagroups-development mailing list
msgmiddle
<Prev in Thread] Current Thread [Next in Thread>
  • [jgroups-dev] A memcached implementation in JGroups
    • Re: [jgroups-dev] [javagroups-users] A memcached implementation in JGroups
      • Re: [jgroups-dev] [javagroups-users] A memcached implementation in JGroups,
        • Re: [jgroups-dev] [javagroups-users] A memcached implementation in JGroups
    • Re: A memcached implementation in JGroups
      • Re: [jgroups-dev] [javagroups-users] A memcached implementation in JGroups
      • Re: [jgroups-dev] [javagroups-users] A memcached implementation in JGroups
Current Sitemap | © 2012 planetjava | Contact | Privacy Policy