1. 02 May, 2017 38 commits
  2. 01 May, 2017 2 commits
    • Jérome Perrin's avatar
      upgrader: upgrade catalog schema right after installing BTs · ba541ad4
      Jérome Perrin authored
      93e30e5e introduce a  new `user` table that is listed as a *search table*, so until this table is created all catalog queries are failing, and alarm tool, which relies on catalog fail with such a traceback:
      
      ```
      ERROR TimerService Process timer error
      Traceback (most recent call last):
        File "parts/erp5/product/TimerService/TimerService.py", line 102, in process_timer
          DateTime(prev_tick), DateTime(next_tick))
        File "parts/erp5/product/ERP5/Tool/AlarmTool.py", line 175, in process_timer
          self.tic()
        File "parts/erp5/product/ERP5/Tool/AlarmTool.py", line 135, in tic
          for alarm in self.getAlarmList(to_active=1):
        File "parts/erp5/product/ERP5/Tool/AlarmTool.py", line 111, in getAlarmList
          alarm_date={'query':now,'range':'ngt'}
        File "parts/erp5/product/ERP5Catalog/CatalogTool.py", line 702, in unrestrictedSearchResults
          return ZCatalog.searchResults(self, **kw)
        File "parts/erp5/product/ZSQLCatalog/ZSQLCatalog.py", line 1091, in searchResults
          return catalog.searchResults(REQUEST, **kw)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2585, in searchResults
          **kw
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2554, in queryResults
          **kw
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2418, in buildSQLQuery
          ignore_unknown_columns=ignore_unknown_columns,
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2394, in buildEntireQuery
          query=self.buildQuery(kw, ignore_empty_string=ignore_empty_string, ignore_unknown_columns=ignore_unknown_columns),
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2295, in buildQuery
          result = self.buildSingleQuery(key, value)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2087, in buildSingleQuery
          search_key, related_key_definition = self.getColumnSearchKey(key, search_key_name)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2049, in getColumnSearchKey
          related_key_definition = self.getRelatedKeyDefinition(key)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1999, in getRelatedKeyDefinition
          for entire_definition in self.getSQLCatalogRelatedKeyList([key]):
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1935, in getSQLCatalogRelatedKeyList
          column_map = self._getSQLCatalogRelatedKeySet()
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 130, in wrapper
          result = transactional_cache[cache_id] = method(wrapped_self)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1908, in _getSQLCatalogRelatedKeySet
          column_map = self.getColumnMap()
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 130, in wrapper
          result = transactional_cache[cache_id] = method(wrapped_self)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1146, in getColumnMap
          for field in table_dict[table]:
      KeyError: 'user'
      ```
      
      This means that even if upgrader's post upgrade constraint are supposed to fix this by calling `portal_catalog.upgradeSchema`, because this constraint rely on alarm tool, it's already too late.
      
      The suggested way to fix this is to also call [`portal_catalog.upgradeSchema`](https://lab.nexedi.com/nexedi/erp5/blob/c99fb9163503c5afdef59ecb124b3060b05330a4/bt5/erp5_upgrader/SkinTemplateItem/portal_skins/erp5_upgrader/TemplateTool_checkTableConsistency.py#L7) in the same transaction that the transaction upgrading business templates.
      
      I have not completely removed `TemplateToolTableConsistencyConstraint`, because maybe there will be cases where just call post upgrade alarm to execute data migration steps after installing a business template manually. But you guys think it's better to completely merge `TemplateToolTableConsistencyConstraint` with `TemplateToolBusinessTemplateInstallationConstraint` I would be OK with that too.
      
      Of course, we cannot just make
      `TemplateToolTableConsistencyConstraint` an `upgrade` constraint, because there would be no guarantee that it's called after `TemplateToolBusinessTemplateInstallationConstraint`.
      
      Also, I have not considered making [`getColumnMap` and friends](https://lab.nexedi.com/nexedi/erp5/blob/master/product/ZSQLCatalog/SQLCatalog.py#L1142) resilient to the case where a table listed in search tables does not exist, because it's would just have been failing later anyway and it's better to fail early in such case.
      
      cc: @vpelletier @jm @seb @tiwariayush @gabriel
      
      /reviewed-on nexedi/erp5!247
      ba541ad4
    • Jérome Perrin's avatar
      Lower priority of mass workflow transition activity · f88d9684
      Jérome Perrin authored
      We had an incident in a in instance were a user changed state of 70K invoices using https://www.erp5.com/howto/erp5-developer-howto/erp5-HowTo.Change.Workflow.State.Of.Multiple.Documents  and leads to user not receiving the reports they requested.
      
      `Folder_modifyWorkflowStatus` creates a lot of `callMethodOnObjectList` activities (for all the selected documents) with priority 2, then these activities will cause more reindex an expand activities.
      
      Until all these `callMethodOnObjectList` are processed, no new activities with priority > 2 were processed. Some "important for users" activities such as erp5_deferred_style reports where waiting in the queue.
      
      I believe we should just set a lower priority to these `callMethodOnObjectList`, eventhough I considered a more clever way of giving the priority:
      
       - if the number of selected documents is reasonably small, process them with a very high priority, this way the user can see his document changing state almost immediately as when using the synchronous change state. This should not cause congestion because there are not too many documents.
       - when there are a lot of selected documents, process them with a very low priority, because anyway it will take time and user will not "wait" for documents to change state.
      
      ... but I realized this is trying to be too clever.
      
      So any objections to just lower priority here ?
      
      /reviewed-on nexedi/erp5!235
      f88d9684