EHC ❯ Сyclic dead lock on three threads when use disk cache
-
Bug
-
Status: New
-
2 Major
-
Resolution:
-
ehcache
-
-
drb
-
Reporter: azalesky
-
November 09, 2016
-
0
-
Watchers: 1
-
November 09, 2016
-
Attachments
Description
We use Ehache 2.10.3 as second level cache for Hibernate under Tomcat.
ehcache.xml and excerpt from threads dump are attached.
The following happens: Thread http-nio-8080-exec-120 executes Cache.put(). It successfully replaced value in ConcurrentHashMap, received lock 0x0000000708f21468 in OnHeapCachingTier$Fault.get and requested readLock before DiskCache update. But it is already locked by thread http-nio-8080-exec-118 with writeLock 0x00000006c1465f28. So thread http-nio-8080-exec-120 is waiting for thread http-nio-8080-exec-118.
Thread http-nio-8080-exec-125 executes Cache.put() with the same key. It tried to execute ConcurrentHashMap.internalReplace, set LOCKED flag in ConcurrentHashMap node and notify OnHeapCachingTier. OnHeapCachingTier has synchronization in OnHeapCachingTier$Fault.get which is locked by 0x0000000708f21468. So thread http-nio-8080-exec-125 is waiting for thread http-nio-8080-exec-120.
Thread http-nio-8080-exec-118 executes Cache.removeAll. It received wrtieLock 0x00000006c1465f28, and started to clear ConcurrentHashMap. During clear it tries to set LOCKED flag in the first node in a bin but it appears to be locked thread http-nio-8080-exec-125. So thread http-nio-8080-exec-118 is waiting for thread http-nio-8080-exec-125 and we got a circle.
After that all other threads stuck in writeLock given to thread http-nio-8080-exec-118 and system became unresponsive.