[ZODB-Dev] possible deadlock with Relstorage + Oracle ?

[Mauro Amico]

Hi there,

We currently use Plone 4.2 with Relstorage and Oracle RAC, sometimes
(like once a week) it
happens that Zope threads remains locked and Zope stops to serve new
requests. When that happens, it is frequent that the phenomenon appears
over all the instances, on different servers.

Currently we have configured 2 Zope threads per instance. In attach
there is log reported by haufe.requestmonitoring,
with traces, always the same, when the instances stop to work.

As first action I'm thinking to increase to three threads per instance,
other ideas or directions to investigate?

Regards,
Mauro.

Products:
------------

* RelStorage-1.5.1-py2.7
* cx_Oracle-5.1.2-py2.7-linux-x86_64
* Oracle Instance Client Shared Library 64-bit - 11.2.0.3.0
* ZODB3 3.10.5-py2.7-linux-x86_64
* Zope2 2.13.19-py2.7

Logs:
-------

2013-11-25T13:10:52 WARNING RequestMonitor.DumpTrace Long running request
Request 583318 "/@@healthcheck" running in thread 139854656571136 since
28.9825799465s
Python call stack (innermost first)
Module relstorage.adapters.oracle, line 322, in restart_load
Module relstorage.storage, line 252, in _restart_load_and_call
Module relstorage.storage, line 1204, in _restart_load_and_poll
Module relstorage.storage, line 1230, in poll_invalidations
Module ZODB.Connection, line 492, in _flush_invalidations
Module ZODB.Connection, line 1045, in open
Module ZODB.DB, line 751, in open
Module App.ZApplication, line 43, in __bobo_traverse__
Module ZPublisher.BaseRequest, line 406, in traverse
Module ZPublisher.Publish, line 49, in publish
Module ZPublisher.Publish, line 7, in _patched_publish
Module ZPublisher.Publish, line 237, in publish_module_standard
Module ZPublisher.Publish, line 443, in publish_module
Module ZServer.PubCore.ZServerPublisher, line 31, in __init__

2013-11-25T13:10:52 WARNING RequestMonitor.DumpTrace Long running request
Request 583319 "/@@healthcheck" running in thread 139854648178432 since
28.5510978699s
Python call stack (innermost first)
Module threading, line 127, in acquire
Module ZODB.DB, line 725, in open
Module App.ZApplication, line 43, in __bobo_traverse__
Module ZPublisher.BaseRequest, line 406, in traverse
Module ZPublisher.Publish, line 49, in publish
Module ZPublisher.Publish, line 7, in _patched_publish
Module ZPublisher.Publish, line 237, in publish_module_standard
Module ZPublisher.Publish, line 443, in publish_module
Module ZServer.PubCore.ZServerPublisher, line 31, in __init__