ZODB Components: Modules loaded by one Request were being GC'ed while executing in another Request. (!785) · Merge Requests · nexedi / erp5

ZODB Components: Modules loaded by one Request were being GC'ed while executing in another Request.

Until this commit, loading a ZODB Component would add it to the current Request object to prevent its reference counter reaching 0 (thus its global being reset to None and then being later GC'ed) when a reset happens in another Request.

This problem was found when investigating testFunctionalConfigurator failure when installing bt5s as requested by the Configurator:

Request R1 calls Zuite_waitForActivities importing extension.erp5.ERP5Zuite (M1). => Add M1 to R1._module_cache_set. => M1 ref counter equals to 2 (sys.modules and R1._module_cache_set references).
R1 terminates and is GC'ed. => M1 ref counter equals to 1 (sys.modules).
Request R2 runs Configurator configuring the Site.
testFunctionalConfigurator calls Zuite_waitForActivities to wait for the Configurator to finish (request R3 which may take ~15 minutes). This calls time.sleep() in a loop where 'time' module is imported at top-level.
R2 installs bt5 triggering reset. => M1 ref counter equals to 0. ===> M1 global variables are reset to None and thus 'time' is set to None raising an Exception in the next call of time.sleep() in the loop.

The easiest way would be to have a hook on sys.modules dict lookup and thus add M1 to R2._module_cache_set when being imported, but this is not supported... Instead create a global cache on erp5.component package.

Tests are passing: https://nexedi.erp5.net/test_result_module/20181023-84CC06B4 .

@nexedi

Until this commit, loading a ZODB Component would add it to the current Request
object to prevent its reference counter reaching 0 (thus its global being reset to
None and then being later GC'ed) when a reset happens in another Request.

This problem was found when investigating testFunctionalConfigurator failure
when installing bt5s as requested by the Configurator:
  1. Request R1 calls Zuite_waitForActivities importing extension.erp5.ERP5Zuite (M1).
     => Add M1 to R1._module_cache_set.
     => M1 ref counter equals to 2 (sys.modules and R1._module_cache_set references).
  2. R1 terminates and is GC'ed.
     => M1 ref counter equals to 1 (sys.modules).
  3. Request R2 runs Configurator configuring the Site.
  4. testFunctionalConfigurator calls Zuite_waitForActivities to wait for the
     Configurator to finish (request R3 which may take ~15 minutes). This calls
     time.sleep() in a loop where 'time' module is imported at top-level.
  5. R2 installs bt5 triggering reset.
     => M1 ref counter equals to 0.
        ===> M1 global variables are reset to None and thus 'time' is set to None
             raising an Exception in the next call of time.sleep() in the loop.

The easiest way would be to have a hook on sys.modules dict lookup and thus add M1
to R2._module_cache_set when being imported, but this is not supported... Instead
create a global cache on erp5.component package.

Tests are passing: https://nexedi.erp5.net/test_result_module/20181023-84CC06B4 .