1. 10 Feb, 2021 11 commits
  2. 09 Feb, 2021 1 commit
    • Vincent Pelletier's avatar
      CMFActivity: Simplify validation queries further. · e4273c58
      Vincent Pelletier authored
      The query planner does not seem to notice that we are trying to know if
      any row exists matching a set of dependency values, and it keeps scanning
      multiple row for each value - which is unproductive.
      So split dependency queries from (pseudo-code)
        WHERE <column{,s}> IN <values{, pairs}>
      to unions of
        WHERE <column{,s}> = <value{, pair}> LIMIT 1
      which produces query plans which do stop immediately when finding a
      candidate row.
      On a serialization_tag query with 40 values and real-world indexations,
      this reduces the number of rows scanned by mariadb from 500 (<10%
      efficiency) to 40 (100% efficiency).
      The produced SQL is significantly larger (~3x, around 500kB on
      real-world sample data, but may vary a lot depending on value length),
      but if this has any effect is is more than compensated by the improved
      query plan efficiency.
      e4273c58
  3. 08 Feb, 2021 3 commits
  4. 05 Feb, 2021 3 commits
  5. 04 Feb, 2021 6 commits
    • Romain Courteaud's avatar
      erp5_web_renderjs_ui: fixup! do not expand every field's div · a4df1d40
      Romain Courteaud authored
      Ensure float/integer field can align right on the listbox
      See nexedi/erp5@cfbc621b
      a4df1d40
    • Georgios Dagkakis's avatar
      erp5_web_service: Migrate DocumentConnection to components · 1fe01629
      Georgios Dagkakis authored
      This was removed from products in previous commits
      1fe01629
    • Jérome Perrin's avatar
      trade: extend "Lines Report" to allow filtering by use · 4723100a
      Jérome Perrin authored
      Nowadays instead of creating many portal types, we use `use` category to
      separate different types of movements, so it makes sense to filter by use,
      the same way that we already were able to filter by portal type.
      4723100a
    • Jérome Perrin's avatar
      trade: delegate size on Base_viewTradeFieldLibrary/my_dialog_mode_portal_type · d8343f59
      Jérome Perrin authored
      Let's be consistent and use the default size.
      d8343f59
    • Jérome Perrin's avatar
      trade: add simple test for "Lines Report" · de0abcb9
      Jérome Perrin authored
      de0abcb9
    • Jérome Perrin's avatar
      Fix Form.proxifyField creating inconsistent proxy fields · ed84c574
      Jérome Perrin authored
      When `Form.proxifyField` is used to make a proxy field from a field which has only a TALES and no value, it creates a proxy field with inconsistent internal data structures where `has_value` method does not work as expected. As a consequence, proxy fields to list fields are often broken in the ods/odt rendering.
      
      Here is a scenario to reproduce the problem:
      
      * add `Base_viewDummyFieldLibrary` ERP5 Form
      * add a `LinesField`, with id `your_test_base_field`. Change `size` to 1 (not really required but I did this in screenshots here)
      * add another `LinesField`, with id `your_test_field` and in this field:
         * set a `default` (`2`) in values, it's only needed to visualise the problem in ods style:
      ![set default and no items](/uploads/0cb9e229b0ca6761bb15f3d8845ea3f8/image.png)
         * set `items` in TALES (`python: [('one', '1'), ('two', '2')]`) , this will be the problem
      ![set items in TALES](/uploads/93d90f57e15afdcca60324675b59086a/image.png)
       * So far, when this is rendered as ODS, everything is fine:
      ![ODS before](/uploads/ce387cd7a5146585104470bf099f890b/image.png) 
      this is also what we see in html style view: ![html before](/uploads/650330951ed286576f8366bbd5c44139/image.png)
       * use the proxify action, to make `your_test_field` be a proxy field to `your_test_base_field`
      ![proxify action](/uploads/9ff7b29141ce7f170f340c842fd76830/image.png)
       * result of proxify action looks good, both for values:
      ![values after proxify](/uploads/a9f7df3cd2d0af1e036aeff352624760/image.png) and for TALES:
      ![TALES after proxify](/uploads/aeaaa45cc9e91e9327781d85ec1efd57/image.png)
       * but after this rendering the form with ODS becomes broken:
      ![ODS after proxify](/uploads/d4e7be3251c596de35333526d4f6277a/image.png) even though it looks fine with html views.
      
      `erp5_ods_style` knows how to render list fields and it uses the "display" (two) and not the "value" (2), this is done [here](https://lab.nexedi.com/nexedi/erp5/blob/451ce4137dc1d006bc2cd155535523d51a928150/bt5/erp5_ods_style/SkinTemplateItem/portal_skins/erp5_ods_style/field_ods_macro.zpt#L60) with a check depending on `field.has_value("items")`.
      
      `has_value` is implemented like [this](https://lab.nexedi.com/nexedi/erp5/blob/451ce4137dc1d006bc2cd155535523d51a928150/product/ERP5Form/ProxyField.py#L598-611) for proxy fields:
      ```python
        def has_value(self, id):
          """
          Return true if the field defines such a value.
          """
          result = None
          if (id in self.widget.property_names) or \
             (not self.is_delegated(id)):
            result = ZMIField.has_value(self, id)
          else:
            proxy_field = self.getTemplateField()
            if proxy_field is not None:
              result = proxy_field.has_value(id)
          return result
      ```
      
      and like [this](https://lab.nexedi.com/nexedi/erp5/blob/451ce4137dc1d006bc2cd155535523d51a928150/product/Formulator/Field.py#L80-86) for traditional fields:
      
      ```python
          def has_value(self, id):
              """Return true if the field defines such a value.
              """
              if self.values.has_key(id) or self.form.has_field(id):
                  return 1
              else:
                  return 0
      ```
      
      when the value is defined on the proxy field and not delegated to template field, the condition is `self.values.has_key(id)`.
      
      But `Form.proxifyField` when transforming a traditional field in a proxy field does not keep the `id` in `self.values` if it's only needed in `self.tales`. This is the root cause of this problem, if we inspect the field, it is something like this:
      
      ```
      (Pdb) self
      <ProxyField at /erp5/portal_skins/custom/Base_viewDummyFieldLibrary/your_test_field>
      (Pdb) pp self.tales
      {'field_id': '',
       'form_id': '',
       'items': <Products.Formulator.TALESField.TALESMethod object at 0x7ffa705c7450 oid 0x572626 in <Connection at 7ffa42c1a610>>}
      (Pdb) pp self.values
      {'default': '2',
       'field_id': 'your_test_base_field',
       'form_id': 'Base_viewDummyFieldLibrary',
       'title': 'Test Field'}
      (Pdb) self.has_value('items')
      0
      (Pdb) 
      ```
      
      If we edit the proxy field, it will repair itself, because the proxy field edit method maintain `self.tales` and `self.values` consistent, but this is not the case  with`Form.proxifyField`, which mutate directly `.tales` and `.values` and can make them have different keys - which is not supposed to happen.
      
      The problem is that most of the proxy fields we have have been generated by `Form.proxifyField`, so for most of our fields the XML data of business template has this inconsistency. We could have took the easy way and make change `ProxyField.has_value` to understand this case, but it's probably better to fix the data. 
      `erp5_hal_json` also uses `field.has_value` in some places, so it's better to fix at field level and not to address this in `erp5_ods_style` and `erp5_odt_style`.
      
      
      These changes:
       - add a little more test coverage for `ProxyField.has_value` ( at first I thought the problem was only in `has_value`)
       - Fix `Form.proxifyField`
       - add a `ProxyField.checkConsistency` to check that `.values` and `.tales` are in sync - or more exactly that all entries from `.tales` are also in `.values`, because this is what cause the problem with `has_value`.
       - re-export all proxy fields after cleaning up their `.values` and `.tales`, using `checkConsistency(fixit=True)` on all proxy fields.
      
      See merge request nexedi/erp5!1352
      ed84c574
  6. 03 Feb, 2021 2 commits
  7. 02 Feb, 2021 14 commits