Class AutoGCObjectReader

source code

ZODB.serialize.ObjectReader --+
                              |
                             AutoGCObjectReader

ZODB has semipeculiar behavior wherein the object cache is only garbage collected at the transaction boundaries. If, within a transaction, one wishes to read a number of objects greater than the configured object cache size, the cache, and therefore memory, will simply grow with each read object.

This makes sense, when you think about it. Removing cached objects out from under existing references could have horrible effects, so it is only safe when beginning a new transaction (or aborting an existing one). Unfortunately, we have several cases where we need to read an enormous number of objects within a transaction, but don't need to write anything.

This class replaces a ZODB.Connection.Connection's existing ObjectReader. It will garbage-collect the cache every n objects. To enforce integrity, it will also abort any open transaction after cleaning the cache. It is safe ONLY when you are certain that the open transaction has not modified any objects.