• New Feature
  • Status: Open
  • 2 Major
  • Resolution:
  • ehcache-core
  • prodmgmt
  • Reporter: jborger
  • May 19, 2013
  • 1
  • Watchers: 6
  • May 20, 2013

Description

We had an out of memory condition on a cache put in production. The thing was that the in memory cache was sized by bytes to 4 MB and the entry which was tried to be put turned out to be 800 MB, we saw that in a heap dump. Also, there was a large IdentityHeapMap and a Stack on the heap. The thread dump showed that the ObjectGraphWalker.walk was active.

We have configured sizeOfPolicy maxDepthExceededBehavior=”continue”, otherwise the cache can grow much larger than we want.

So, it turns out that the sizeOf calculation to determine if the 800 MB object fits in the 4 MB cache, tries to count all the 800 MB: takes a long time and then generates an out of memory error. I would say, after counting 4 MB, it is clear that it does not fit and we can stop counting.

Alex Snaps writes a reply: I’m slightly confused how this triggered an OOME, but I guess if the “entry object graph” is composed of many small objects, I guess the IdentityHashMap’s entries could end up consuming quite some memory. Obviously the keys & values in there are of “no overhead” as they are consuming memory you’ve allocated already yourself.

On a more general note, the feature was indeed never meant to account for such a usecase. This is a very “niche” problem you have I’d say. Having a 800MB Object graph and trying to jam it in a 4MB bound cache seems kinda surprising. Failing quickly would be indeed feasible I guess… I propose you create a new feature request on jira and PM would need to prioritize this.

As an additional note, we’ve tried to keep the sizeOf as agnostic as possible from ehcache itself. The sizeOfPolicy being already “leaking” a little too much concern into the sizing engine, I think we’d need to refactor the code in order to support this feature, and avoid further coupling. But other than that, having the sizeOfPolicy be extended to support further conditions that would trip it, doesn’t seem crazy at all indeed.

Comments