• New Feature
  • Status: Open
  • 2 Major
  • Resolution:
  • ehcache-core
  • Reporter:
  • September 21, 2009
  • 1
  • Watchers: 1
  • October 11, 2011

Attachments

Description

Please see attached file. The purpose of this extension is similar to patch 2046973 (MemoryStore Expiry thread), however it does so without iterating though all elements at fixed intervals. With large caches such an approach is wasteful (creating iterators for large caches consumes resources, ConcurrentHashMap pretty much has to do an internal copy when providing non-fail-fast iterators) and many times unnecessary. This extension insteads schedules removal using a Timer so elements are only removed when necessary. There’s no polling involved either, waits are performed indirectly via Object.wait()

We’ve been using this in our production systems with fine results. This source has a hack to disable the keyArray sampling eviction policy in the MemoryStore for large caches (> 5000) using reflection and Field.setAccessible() - this is not pretty but it’s the only way to do this without actually modifying MemoryStore. The reason is that this eviction algorithm is rendered pretty much useless once we begin removing elements and keys before the eviction algorithm expects this to happen. Normally, MemoryStore never evicts the ConcurrentHashMap until it’s *really* full, even if it’s full of expired elements. In that scenario, the keySampling eviction algorithm works fine most of the time. But the approach breaks down when we eagerly clear expired elements. Sourceforge Ticket ID: 2798687 - Opened By: mads1980 - 30 May 2009 02:17 UTC

Comments

Steve Harris 2009-10-21

Take a look and see if it makes sense

gluck 2010-01-20

Transferred patch over from SourceForge

gluck 2010-01-20

The patch is implemented as a CacheExtension. This is nice because it can be configured if required on a cache by cache basis. The extension can be added and noted in ehcache.xml and in the doco and then used if someone needs an expiry thread. Because this patch interacts with a Cache it should work fine regardless of whether it is standalone or distributed.

I have asked Manuel to complete a Contributor Agreement.

Manuel Dominguez Sarmiento 2010-05-24

Not that we’re in any kind of rush to get this built into a release (we donated this extension and have been successfully been using it in production for over a year), but the Fix version is marked for 2.0.0 and we can’t find this in the current release (2.1.0) - has this been incorporated into the project?

gluck 2010-05-24

Manuel

Did you complete the Agreement? Let me know if you have and I could can merge the patch in.

Manuel Dominguez Sarmiento 2010-05-24

Hi Greg, yes I did on Jan. 20th - I just forwarded it again with the latest patch to contributors lists dot terracotta dot org as well as gluck gregluck dot com

gluck 2010-08-16

Mads

Ready to do this now. Can you please resubmit a patch against trunk. Things have changed a fair bit, so will need this redone.

Manuel Dominguez Sarmiento 2010-08-17

Hi Greg, I’ve uploaded the current version we have working on our production systems against EhCache 2.2.0 - I haven’t checked the trunk yet - are there any significant changes in the upcoming 2.2.1 release which could render this patch incompatible? If so please let us know so that we can look at it.

Fiona OShea 2010-09-01

Moving all unresolved “Fix Revision 2.2.1” to fix revision “unknown” as we are releasing Magnum first which is 2.3. Currently not sure which fix version these will actually be in, but they are targeted for Fremantle release

Manuel Dominguez Sarmiento 2011-02-22

This new attached version properly takes into account TTI expiration. The previous version worked fine with TTL, but did not account for changing expiration timestamps due to TTI.

Manuel Dominguez Sarmiento 2011-02-22

BTW, just for the record, the keyArray sampling and Field.setAccessible() hacks have no longer been relevant for a long time, and the attached patches do not contain the hacks to get around those issues. It’s been a while, but I think those hacks were necessary for the EhCache 1.6 branch, since later version fixed those issues.