deadlock in objectID sequencer

Type: Bug
Status: Closed
Priority: 1 Critical
Resolution: Fixed
Component/s:
Labels:

Assignee: kkannaiy
Reporter: teck
Created: July 13, 2009
Votes: 0
Watchers: 2
Updated:
February 12, 2013
Resolved:
July 17, 2009

Attachments

teck (84.00 k) text/plain p2out.log
teck (46.00 k) application/x-gzip testsuite-log-trunk-alltests-rev13077-rh5mo1-20090712-2300.com.tctest.PartialCollectionsCloneTest.log.gz

Description

I’ll attach the logs. This happened in a system test (PartialCollectionsCloneTest)

    INFO     Found one Java-level deadlock:
    INFO     =============================
    INFO     "WorkerThread(object_id_request_response_stage,0)":
    INFO       waiting to lock monitor 0x09aed124 (object 0x95c67990, a com.tc.util.sequence.BatchSequence),
    INFO       which is held by "Thread-26"
    INFO     "Thread-26":
    INFO       waiting to lock monitor 0x8e588e60 (object 0x95c679b0, a com.tc.object.idprovider.impl.RemoteObjectIDBatchSequenceProvider),
    INFO       which is held by "WorkerThread(object_id_request_response_stage,0)"
    INFO
    INFO     Java stack information for the threads listed above:
    INFO     ===================================================
    INFO     "WorkerThread(object_id_request_response_stage,0)":
    INFO     	at com.tc.util.sequence.BatchSequence.setNextBatch(BatchSequence.java:69)
    INFO     	- waiting to lock <0x95c67990> (a com.tc.util.sequence.BatchSequence)
    INFO     	at com.tc.object.idprovider.impl.RemoteObjectIDBatchSequenceProvider.handleEvent(RemoteObjectIDBatchSequenceProvider.java:40)
    INFO     	- locked <0x95c679b0> (a com.tc.object.idprovider.impl.RemoteObjectIDBatchSequenceProvider)
    INFO     	at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:142)
    INFO     "Thread-26":
    INFO     	at com.tc.object.idprovider.impl.RemoteObjectIDBatchSequenceProvider.requestBatch(RemoteObjectIDBatchSequenceProvider.java:32)
    INFO     	- waiting to lock <0x95c679b0> (a com.tc.object.idprovider.impl.RemoteObjectIDBatchSequenceProvider)
    INFO     	at com.tc.util.sequence.BatchSequence.requestNextBatch(BatchSequence.java:64)
    INFO     	at com.tc.util.sequence.BatchSequence.requestMoreIDsIfNecessary(BatchSequence.java:44)
    INFO     	at com.tc.util.sequence.BatchSequence.next(BatchSequence.java:31)
    INFO     	- locked <0x95c67990> (a com.tc.util.sequence.BatchSequence)
    INFO     	at com.tc.object.idprovider.impl.ObjectIDProviderImpl.next(ObjectIDProviderImpl.java:21)
    INFO     	at com.tc.object.ClientObjectManagerImpl.nextObjectID(ClientObjectManagerImpl.java:1140)
    INFO     	at com.tc.object.ClientObjectManagerImpl.basicCreateIfNecessary(ClientObjectManagerImpl.java:1084)
    INFO     	at com.tc.object.ClientObjectManagerImpl.basicCreateIfNecessary(ClientObjectManagerImpl.java:1099)
    INFO     	- locked <0x95c67db0> (a com.tc.object.ClientObjectManagerImpl)
    INFO     	at com.tc.object.ClientObjectManagerImpl.access$600(ClientObjectManagerImpl.java:79)
    INFO     	at com.tc.object.ClientObjectManagerImpl$AddManagedObjectAction.basicVisit(ClientObjectManagerImpl.java:1047)
    INFO     	at com.tc.object.ClientObjectManagerImpl$BaseAction.visit(ClientObjectManagerImpl.java:1033)
    INFO     	at com.tc.object.Traverser.traverse(Traverser.java:93)
    INFO     	at com.tc.object.ClientObjectManagerImpl.traverse(ClientObjectManagerImpl.java:950)
    INFO     	at com.tc.object.ClientObjectManagerImpl.create(ClientObjectManagerImpl.java:296)
    INFO     	at com.tc.object.ClientObjectManagerImpl.lookupOrCreateIfNecesary(ClientObjectManagerImpl.java:356)
    INFO     	at com.tc.object.ClientObjectManagerImpl.lookupOrCreate(ClientObjectManagerImpl.java:334)
    INFO     	at com.tc.object.ClientObjectManagerImpl.lookupRootOptionallyCreateOrReplace(ClientObjectManagerImpl.java:875)
    INFO     	at com.tc.object.ClientObjectManagerImpl.lookupOrCreateRoot(ClientObjectManagerImpl.java:664)
    INFO     	at com.tc.object.ClientObjectManagerImpl.lookupOrCreateRoot(ClientObjectManagerImpl.java:647)
    INFO     	at com.tc.object.bytecode.ManagerImpl.lookupOrCreateRoot(ManagerImpl.java:332)
    INFO     	at com.tc.object.bytecode.ManagerImpl.lookupOrCreateRoot(ManagerImpl.java:311)
    INFO     	at com.tc.object.bytecode.ManagerUtil.lookupOrCreateRoot(ManagerUtil.java:92)
    INFO     	at com.tctest.runner.AbstractTransparentApp.__tc_setappIds(AbstractTransparentApp.java)
    INFO     	at com.tctest.runner.AbstractTransparentApp.<init>(AbstractTransparentApp.java:24)
    INFO     	at com.tctest.runner.AbstractErrorCatchingTransparentApp.<init>(AbstractErrorCatchingTransparentApp.java:19)
    INFO     	at com.tctest.PartialCollectionsCloneTest$PartialCollectionsCloneTestApp.<init>(PartialCollectionsCloneTest.java:46)
    INFO     	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    INFO     	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    INFO     	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    INFO     	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    INFO     	at com.tc.simulator.app.DSOApplicationBuilder.newApplication(DSOApplicationBuilder.java:73)
    INFO     	- locked <0x94f847f8> (a com.tc.simulator.app.DSOApplicationBuilder)
    INFO     	at com.tc.simulator.container.Container.newExecutionInstance(Container.java:190)
    INFO     	at com.tc.simulator.container.Container.startInstances(Container.java:204)
    INFO     	at com.tc.simulator.container.Container.run(Container.java:90)
    INFO     	- locked <0x94f84a10> (a com.tc.simulator.container.Container)
    INFO     	at java.lang.Thread.run(Thread.java:619)
    INFO
    INFO     Found 1 deadlock.

Comments

Tim Eck 2009-07-13

another example

Saravanan Subbiah 2009-07-13

This seems critical to fix.

Kalai Kannaiyan 2009-08-13

verified the fix with svn rev13161.