Skip to content

W
wendelin
Commit bce74d9d authored by Ekaterina's avatar Ekaterina
Browse files
Options

add: tutorial tests

parent 14a475aa
Hide whitespace changes
Inline Side-by-side
Showing with 781596 additions and 0 deletions
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/portal_skins/erp5_wendelin_data_lake/Ingestion_ingestDataStream.py 0 → 100644
View file @ bce74d9d
"""Create ingestion with data from stream"""
portal = context.getPortalObject()
request = portal.REQUEST
index = reference.index('-')
stream_reference = reference
ingestion_reference = reference[:index] + '.' + reference[index+1:len(reference)]
request.environ['REQUEST_METHOD'] = 'POST'
request.set('reference', ingestion_reference)
request.set('data_chunk', portal.data_stream_module.get(stream_reference).getData())
portal.portal_ingestion_policies.default.ingest()
return 'Data ingested'
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/portal_skins/erp5_wendelin_data_lake/Ingestion_ingestDataStream.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>reference</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Ingestion_ingestDataStream</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_01_wendelin_Tutorial_Howto_Create_Data_Product.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td>Init Ok</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>Create Data Product</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to create Data Product.</p>
<p>Data Product represents the type of data.&nbsp;</p>
<p>In this tutorial we will create a Data Product for&nbsp;incoming raw data from streaming hardware.&nbsp;</p>
</details>
</section>
<section>
<h1>Home</h1>
<img alt="" src="wendelin-new.Interface.home.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Go to your wendelin instance homepage and click on Modules.</p>
</details>
</section>
<section>
<h1>Data Product</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.1.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>On the modules page click on Data Product.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Add</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.2.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Add button.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Create Document</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.3.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Continue with clicking on Create Document.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Fill the Form</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.4.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Fill the form and click save.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Validate</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.5.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>After it is saved, click on Validate on the left side panel to validate the Data Product.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Confirm Valdation</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.7.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Proceed&nbsp;to confirm the&nbsp;validation.</p>
</details>
<p>&nbsp;</p>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Sample Environment Raw Data</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_quantity_unit_list_default:list"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>sample-environment-raw-data</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Stream</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[2]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Progress Indicator</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_use_list_default:list"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_01_wendelin_Tutorial_Howto_Create_Data_Product.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>contributor/person_module/functional_test_username</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-Tutorial.Howto.Create.Data.Product</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Create Data Product</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_01_wendelin_Tutorial_Howto_Create_Data_Product</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Create Data Product</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Create Data Product</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_02_wendelin_Tutorial_Howto_Create_Data_Acquisition.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td>Init Ok</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>Create Data Acquisition&nbsp;Unit</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to create Data Acquisition&nbsp;Unit.</p>
<p>Data Acquisition&nbsp;Unit represents the source from where the data comes.&nbsp;</p>
<p>In the scope of this Learning Track we use Olimex Environment Sensor to retrieve data, thus our&nbsp; Data Acquisition&nbsp;Unit will represent that sensor.&nbsp;</p>
<p>&nbsp;</p>
</details>
</section>
<section>
<h1>Home</h1>
<img alt="" src="wendelin-new.Interface.home.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Go to your wendelin instance homepage and click on Modules.</p>
</details>
</section>
<section>
<h1>Data Acquisition Unit</h1>
<img alt="" src="wendelin-new.Interface.modules.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>On the modules page click on Data Acquisition Units.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Add</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Acq.Unit.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Add button.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Create Document</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Acq.Unit.2.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Continue with clicking on Create Document.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Fill the Form</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Acq.Unit.3.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Fill the form and click save.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Validate</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Acq.Unit.4.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>After it is saved, click on Validate on left side panel to validate Data Acquisition Unit.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Confirm Valdation</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Acq.Unit.6.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Validate to confirm validation.</p>
</details>
<p>&nbsp;</p>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_acquisition_unit_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_acquisition_unit_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Olimex Environment Sensor</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>olimex-environment-sensor</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_02_wendelin_Tutorial_Howto_Create_Data_Acquisition.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>contributor/person_module/functional_test_username</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-Tutorial.Howto.Create.Data.Acquisition</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Create Data Acquisition Unit</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_02_wendelin_Tutorial_Howto_Create_Data_Acquisition</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Create Data Acquisition Unit</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Create Data Acquisition Unit</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_03_wendelin_Tutorial_Howto_Create_Data_Supply.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>Create Data Supply</li>
<li>Create Data Supply Lines</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to create&nbsp; Data Supply to prepare Wendelin to receive&nbsp;data sent by fluentd from IoT Gateway.</p>
<p>&nbsp;</p>
</details>
</section>
<section>
<h1>Data Supplies</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.1.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>On the modules page click on Data Supplies.</p>
<p>&nbsp;</p>
</details>
</section>
<section>
<h1>Add</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.2.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Add button to add new Data Supply.</p>
<p>&nbsp;</p>
</details>
</section>
<section>
<h1>Create Document</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.3.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Continue with clicking on Create Document.</p>
<p>&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_supply_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_supply_module')]</td>
<td></td>
</tr>
<!--<tr>
<td>waitForElementPresent</td>
<td>//tbody//tr[3]//td[1]//a</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//tbody//tr[3]//td[1]//a</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>-->
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Fill the Form</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.4.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>First fill in&nbsp;the <strong>Title </strong>and<strong>&nbsp; Reference&nbsp;</strong>for the Data Supply.&nbsp;</p>
<p>Pay attention on the <b>reference</b>, we name it simply&nbsp;<strong>sensor_1 </strong>as we have only 1 sensor that will be sending data to wendelin.&nbsp;</p>
<p>This reference will be used in later tutorials when we will configure fluentd.&nbsp;</p>
</details>
</section>
<section>
<h1>Fill the Form Cont.</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.5.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Imagine an example of&nbsp;a Sample Company that owns a Sample Factory where is a sensor that sends data to wendelin.&nbsp;</p>
<p>Then the source of the data is Sample Factory.&nbsp;</p>
<p><strong>Source -&nbsp;</strong>where data comes from.&nbsp;</p>
</details>
</section>
<section>
<h1>Fill the Form Cont.</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.6.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Continuing the example, the <strong>Source Section</strong> is Sample Company.</p>
<p><strong>Source Section - </strong>owner of data before the ingestion.</p>
</details>
</section>
<section>
<h1>Fill the Form Cont.</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.7.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p><strong>Destination - </strong>physical place where data is stored.</p>
<p><strong>Destination Section -&nbsp;</strong>owner of data after ingestion. Can be the same as the source.</p>
<p>&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Environment Sensor</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>sensor_1</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_version"]</td>
<td>001</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_source_title"]</td>
<td>Sample Factory</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_source_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[@name='Organisation']</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[@name='Organisation']</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_source_section_title"]</td>
<td>Sample Company</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_source_section_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[@name='Organisation']</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[@name='Organisation']</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_destination_title"]</td>
<td>Nexedi Data Center</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_destination_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[@name='Organisation']</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[@name='Organisation']</td>
<td></td>
</tr>
<!--<tr>
<td>type</td>
<td>//input[@id="field_my_effective_date"]</td>
<td>01/01/2020</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_expiration_date"]</td>
<td>01/01/2099</td>
</tr>-->
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_destination_section_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_destination_section_title"]</td>
<td>Sample Company</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_destination_section_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'organisation_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'organisation_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Save</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.8.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>After click on Save button to save the changes.&nbsp;&nbsp;</p>
</details>
</section>
<section>
<h1>Validate</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.9.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>After it is saved, click on Validate on left side panel to validate the Data Supply.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Confirm Valdation</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.11.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Validate to confirm validation.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Add Data Supply Line</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.1.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Add button to add Data Supply Line.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Create Document</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.2.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Continue with clicking on Create Document.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Fill the Form</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.3.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Title:&nbsp;<strong>Ingest Bucket Data</strong></p>
<p>Reference:&nbsp;<strong>ingestion_operation</strong></p>
<p>For&nbsp;Product or Service - data operation by default exists in wendelin. when you type in the field&nbsp;<strong>Ingest Data&nbsp;</strong>from drop down list choose the existing one.&nbsp;</p>
</details>
</section>
<section>
<h1>Fill the Form Cont.</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.4.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>For Item we choose Data Acquisition Unit that we created in the previous tutorial.&nbsp;</p>
</details>
</section>
<section>
<h1>Save</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.5.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>After click on Save button to save the changes.&nbsp;&nbsp;</p>
</details>
</section>
<section>
<h1>Back to Data Supply</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.6.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Data Supply Line: Ingest Bucket Data to go back and create another Data Supply line</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@name="submit"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Ingest Bucket Data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>ingestion_operation</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Ingest Data</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_aggregate_title_list"]</td>
<td>Olimex Environment Sensor</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_aggregate_title_list']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_acquisition_unit_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_acquisition_unit_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n, 'Data Supply Line')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Add Data Supply Line</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.1.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Click on Add button to add Data Supply Line.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Create Document</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.2.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Continue with clicking on Create Document.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Fill the Form</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.7.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Title:&nbsp;<strong>Sample Environment Data</strong></p>
<p>Reference:&nbsp;<strong>out_stream</strong></p>
<p>For&nbsp;Product or Service we choose Data Product that we created in the previous tutorial.</p>
</details>
</section>
<section>
<h1>Fill the Form Cont.</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.8.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>For <strong>Use</strong> from drop down list choose <strong>Big Data/Ingestion/Stream Ingestion</strong>.</p>
</details>
</section>
<section>
<h1>Save</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.9.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>After click on Save button to save the changes.&nbsp;&nbsp;</p>
</details>
</section>
<section>
<h1>Data Supply</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Supply.Line.10.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Data Supply with Data Supply Lines is ready.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@name="submit"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Sample Environment Data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>out_stream</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Sample Environment Raw Data</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_use"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n, 'Data Supply Line')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//input[@value="ingestion_operation"]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//input[@value="out_stream"]</td>
<td></td>
</tr>
<tr>
<td>open</td>
<td>${base_url_x}/portal_skins/erp5_wendelin_data_lake/Ingestion_ingestDataStream?reference=sensor_1-sample-environment-raw-data</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>60000</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_03_wendelin_Tutorial_Howto_Create_Data_Supply.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-Tutorial.Howto.Create.Data.Supply</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Create Data Supply</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_03_wendelin_Tutorial_Howto_Create_Data_Supply</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Create Data Supply</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Create Data Supply</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_04_wendelin_Tutorial_Check_Data_Arrival.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>See Data In Wendelin</li>
<li>Download Data</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to verify data arrival on wendelin and how to download the data.&nbsp;</p>
<p>Before doing this tutorial make sure you read and completed the following tutorials</p>
<ul>
<li><a href="wendelin-Tutorial.Setup.Sensor.IOTGateway">HowTo Setup Sensor and IOT Gateway</a></li>
<li><a href="https://slapos.nexedi.com/slapos-HowTo.Use.Slapos.Client">SlapOS HowTo Use SlapOS Client</a></li>
<li><a href="fluentd-HowTo.Setup.Fluentd">HowTo Supply Fluentd</a></li>
<li><a href="https://docs.fluentd.org/configuration">Fluentd Configuration File</a></li>
<li><a href="https://nexedi.erp5.net/web_page_module/10408/wendelin-Tutorial.Setup.Fluentd.on.IOTGateway">HowTo setup Fluentd on IOT Gateway</a></li>
<li><a href="https://nexedi.erp5.net/web_page_module/10408/wendelin-Tutorial.Setup.Fluentd.on.Sensor">HowTo setup Fluentd on Sensor</a></li>
</ul>
<p>&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Wendelin Dashboard</h1>
<img alt="" src="wendelin-Data.Arrival.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Open your Wendelin dashboard.&nbsp;</p>
<p>You see that new Data Stream and Data Ingestion were created.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Data Stream</h1>
<img alt="" src="wendelin-Data.Arrival.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>From your Wendelin dashboard click on Data Stream.&nbsp;</p>
<p>In Data Stream you can see the Total size (bytes) of the ingested data.</p>
<p>If you reload the page the size will change as new data arrives.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@data-gadget-scope="field_listbox_storage"]//div//div//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@data-gadget-scope="field_listbox_storage"]//div//div//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td>sensor_1-sample-environment-raw-data</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@data-gadget-scope="field_listbox_movement"]//div//div//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//div[@data-gadget-scope="field_listbox_storage"]//div//div//div[@class='document_table']//table//tbody//tr[1]//td[1]//a</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td>Validated</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Download Data</h1>
<img alt="" src="wendelin-Data.Arrival.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>You can see and download &nbsp;data using the <a href="https://curl.haxx.se/">curl</a> commad:&nbsp;</p>
<p><code>curl -k -u [user]:[password] https://[hostname]/erp5/data_stream_module/[id] -r [start]-[end] --output data.txt</code></p>
<p>As data can be very big you can define start and stop bytes to download a slice of data.&nbsp;</p>
</details>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_04_wendelin_Tutorial_Check_Data_Arrival.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-Tutorial.Check.Data.Arrival</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Check Data Arrival on Wendelin</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_04_wendelin_Tutorial_Check_Data_Arrival</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Check Data Arrival</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
<string>installation</string>
<string>setup</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Check Data Arrival</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_05_wendelin_Tutorial_Transform_Data.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>Create Data Operation</li>
<li>Create Data Product</li>
<li>Create Transformation Script</li>
<li>Create Data Transformation</li>
<li>See Data Analysis</li>
<li>See&nbsp;Data Array</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to transform raw data and write it in Data Arrays.</p>
<p>Before doing this tutorial make sure you read and completed the following tutorial and have data in your wendelin instance:</p>
<ul>
<li><a href="wendelin-Tutorial.Check.Data.Arrival">HowTo Check Data Arrival</a></li>
</ul>
<p>&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Open your Wendelin dashboard.&nbsp;</p>
<p>In Modules click on Data Operations Module</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Add</strong> to add a new Data Operation.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click <strong>Proceed</strong> to continue.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_operation_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_operation_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Fill the Form</h1>
<img alt="" src="wendelin-Data.Operation.4?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form to create Data Operation.</p>
<p><strong>Title&nbsp;</strong>-&nbsp; we name it&nbsp;<strong>Convert Raw Environment Data to Array</strong></p>
<p><strong>Reference&nbsp;</strong>&nbsp;-&nbsp;<strong>data-operation-convert-raw-data-to-array&nbsp;</strong></p>
<p><strong>Script ID&nbsp;</strong>-&nbsp;<strong>DataAnalysisLine_convertEnvironmentDataStreamToArray </strong>:&nbsp;this script will do all the magic. It doesn&#39;t exist yet, we will create it later in this tutorial.&nbsp;</p>
<p><strong><label for="field_my_aggregated_portal_type_list">Item Types&nbsp;</label></strong><label for="field_my_aggregated_portal_type_list">- <strong>Data Array&nbsp;</strong>: we choose Data Array as we will&nbsp;convert the raw data to Data Array</label></p>
</details>
</section>
<section class="screenshot">
<h1>Save</h1>
<img alt="" src="wendelin-Data.Operation.5?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click <strong>Save</strong> to save the changes.</p>
</details>
</section>
<section class="screenshot">
<h1>Validate</h1>
<img alt="" src="wendelin-Data.Operation.6?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Validate on left side panel to validate Data Operation.</p>
</details>
</section>
<section class="screenshot">
<h1>Confirm Validation</h1>
<img alt="" src="wendelin-Data.Operation.7?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click <strong>Proceed</strong> to confirm validation.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Convert Raw Environment Data to Array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>data-operation-convert-raw-data-to-array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_version"]</td>
<td>001</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_script_id"]</td>
<td>DataAnalysisLine_convertEnvironmentDataStreamToArray</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_quantity_unit_list_default:list"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Data Product</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.1.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Next we need to create a new Data Product which will the output Data Product of the Transformation.&nbsp;</p>
<p>&nbsp;</p>
</details>
</section>
<section>
<h1>Create Data Product</h1>
<img alt="" src="wendelin-Data.Transformation.17?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Create a new Data Product as described in <a href="wendelin-Tutorial.Howto.Create.Data.Product">HowTo Create Data Product</a> tutorial with following values</p>
<p>Title -&nbsp; <strong>Environment Raw Array</strong></p>
<p>Reference -&nbsp;<strong>environment-raw-array</strong></p>
<p>Item Types</p>
<ul>
<li><strong>Data Stream</strong></li>
<li><strong>Progress Indicator</strong></li>
<li><strong>Data Array</strong></li>
</ul>
<p>&nbsp;</p>
</details>
<p>&nbsp;</p>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Environment Raw Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_quantity_unit_list_default:list"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>environment-raw-array</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Stream</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[2]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Progress Indicator</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[3]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_use_list_default:list"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Portal Callables</h1>
<img alt="" src="wendelin-Data.Transformation.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After Data Product is created and validated, navigate to page called Portal Callables by clicking on&nbsp;<strong>Callables&nbsp;</strong> on the left side panel.</p>
<p>There we will create and store a python script that will transform raw data to arrays.&nbsp;</p>
<h1>&nbsp;</h1>
</details>
</section>
<section class="screenshot">
<h1>Add Transformation Script</h1>
<img alt="" src="wendelin-Data.Transformation.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Add button to add a new script.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Transformation Script Cont.</h1>
<img alt="" src="wendelin-Data.Transformation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Choose Python Script as Document Type and click on Create Document to create an empty python script.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Transformation.4?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Define <strong>ID</strong>, <strong>Title</strong> and <strong>Reference</strong> of your script.</p>
<p>We name it <strong>DataAnalysisLine_convertEnvironmentDataStreamToArray </strong> as we did in Data Operation.</p>
<p>And click <strong>Save</strong> to save the changes.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<!--<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>-->
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=portal_callables')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=portal_callables')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Python Script</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_id"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_id"]</td>
<td>DataAnalysisLine_convertEnvironmentDataStreamToArray</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>DataAnalysisLine_convertEnvironmentDataStreamToArray</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>DataAnalysisLine_convertEnvironmentDataStreamToArray</td>
</tr>
<tr>
<td>pause</td>
<td>15000</td>
<td></td>
</tr>
<!--<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>-->
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Parameters</h1>
<img alt="" src="wendelin-Data.Transformation.5?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Next we define the parameters we will give to our script.&nbsp;</p>
<p><strong>in_stream&nbsp;</strong>- the input Data Stream where raw data is stored</p>
<p><strong>out_array&nbsp;</strong>&nbsp;- the output Data Array where data will be stored.&nbsp;</p>
<p>At the end click <strong>Save</strong> to save the changes.</p>
</details>
</section>
<section class="screenshot">
<h1>Transformation Script</h1>
<img alt="" src="wendelin-Data.Transformation.6?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">The script we write in the textbox area at the bottom of the page.</details>
</section>
<section class="screenshot">
<h1>Transformation Script Cont.</h1>
<pre>
<code>
import pandas as pd
progress_indicator = in_stream[&quot;Progress Indicator&quot;]
in_data_stream = in_stream[&quot;Data Stream&quot;]
out_data_array = out_array[&quot;Data Array&quot;]
chunk_size = 20 * 10**6
start = progress_indicator.getIntOffsetIndex()
end = min(start+chunk_size, in_data_stream.getSize())
unpacked, end = in_data_stream.readMsgpackChunkList(start, end)
f = in_data_stream.extractDateTime
df = pd.DataFrame((dict(**o[1]) for o in unpacked), dtype=&quot;float64&quot;, index=(f(o[0]) for o in unpacked))
if df.shape[0] == 0:
&nbsp; return
df.index.name=&quot;date&quot;
ndarray = df.to_records(convert_datetime64=False)
zbigarray = out_data_array.getArray()
if zbigarray is None:
&nbsp; zbigarray = out_data_array.initArray(shape=(0,), dtype=ndarray.dtype.fields)
zbigarray.append(ndarray)
if end &gt; start:
&nbsp; progress_indicator.setIntOffsetIndex(end)
# tell caller to create new activity after processing if we did not reach end of stream
if end &lt; in_data_stream.getSize():
&nbsp; return 1</code><code>
</code>
</pre>
<details open="">
<p>This script will simply write raw data to Data Array.&nbsp; More sophisticated example like resampling data or data predictions are shown in following tutorials.&nbsp;</p>
<p>First we extract data from the Data stream and read a chunk of data at a time.&nbsp;</p>
<p>As data is in <a href="https://msgpack.org/">msgpack</a> format we unpack it using iterUnpack method of wendelin Data Stream which is based on&nbsp; Unpacker of msgpack.&nbsp;</p>
<p>Next we create a data frame using <a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html">pandas DataFrame</a>.&nbsp;</p>
<p>After we convert it to ndarray and append it to <a href="https://lab.nexedi.com/nexedi/wendelin.core">zbigarray</a>.</p>
<p>We use Progress Indicator to keep track of how much raw data we already processed.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_parameter_signature"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_parameter_signature"]</td>
<td>in_stream=None, out_array=None</td>
</tr>
<tr>
<td>type</td>
<td>//div[contains(@data-gadget-url,'gadget_editor.html')]//textarea</td>
<!--<td>//pre[@class=" CodeMirror-line " and @role="presentation"]</td>-->
<td>import pandas as pd
progress_indicator = in_stream[&quot;Progress Indicator&quot;]
in_data_stream = in_stream[&quot;Data Stream&quot;]
out_data_array = out_array[&quot;Data Array&quot;]
chunk_size = 20 * 10**6
start = progress_indicator.getIntOffsetIndex()
end = min(start+chunk_size, in_data_stream.getSize())
unpacked, end = in_data_stream.readMsgpackChunkList(start, end)
f = in_data_stream.extractDateTime
df = pd.DataFrame((dict(**o[1]) for o in unpacked), dtype=&quot;float64&quot;, index=(f(o[0]) for o in unpacked))
if df.shape[0] == 0:
&nbsp; return
df.index.name=&quot;date&quot;
ndarray = df.to_records(convert_datetime64=False)
zbigarray = out_data_array.getArray()
if zbigarray is None:
&nbsp; zbigarray = out_data_array.initArray(shape=(0,), dtype=ndarray.dtype.fields)
zbigarray.append(ndarray)
if end &gt; start:
&nbsp; progress_indicator.setIntOffsetIndex(end)
# tell caller to create new activity after processing if we did not reach end of stream
if end &lt; in_data_stream.getSize():
&nbsp; return 1
</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformations</h1>
<img alt="" src="wendelin-Data.Transformation.7?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After your transformation script is ready, it&#39;s time to create Data Transformation.</p>
<p>On Modules page click on Data Transformations.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Transformations</h1>
<img alt="" src="wendelin-Data.Transformation.8?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Add to add a new Data Transformation.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Data Transformation</h1>
<img alt="" src="wendelin-Data.Transformation.9?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Create Document to continue.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Transformation.10?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Chose descriptive title and reference. For example</p>
<p>Title : Convert Environment Raw Data</p>
<p>Reference : convert-environment-raw-data</p>
<p>For Initial Product we choose the Data Product we created in&nbsp;<a href="wendelin-Tutorial.Howto.Create.Data.Product">HowTo Create Data Product</a>&nbsp;tutorial.&nbsp;</p>
<p>At the end click Save to save the changes.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_transformation_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_transformation_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Convert Environment Raw Data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>convert-environment-raw-data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Sample Environment Raw Data</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation Lines</h1>
<img alt="" src="wendelin-Data.Transformation.11?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Add button to add a Data Transformation Line.</p>
<p>&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Create Operation Line</h1>
<img alt="" src="wendelin-Data.Transformation.12?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Choose <strong>Data Transformation Operation Line&nbsp;</strong>for Document Type and click on Create Document.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Transformation.13?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form.</p>
<p>For Data Operation put the name of the Data operation that we created at the beginning of this tutorial - <strong>Convert Raw Environment Data to Array</strong>.</p>
<p>At the end click on Save to save the changes.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Back To Data Transformation</h1>
<img alt="" src="wendelin-Data.Transformation.14?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Once needed fields are filled click Save to save the changes and go back to Data Transformations view by clicking on the upper panel.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Convert Raw Data to Array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>data_operation</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Convert Raw Environment Data to Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Convert</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Operation Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Operation Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation Lines</h1>
<img alt="" src="wendelin-Data.Transformation.15?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Add another Data Transformation Line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Transformation Line</h1>
<img alt="" src="wendelin-Data.Transformation.16?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Now for Document Type choose <strong>Data Transformation Resource Line</strong> and click on <strong>Create Document</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Transformation.18?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form.</p>
<p>For <strong>Data Product</strong> put the name of the Data Product that we created in <a href="wendelin-Tutorial.Howto.Create.Data.Product">HowTo Create Data Product</a>&nbsp;tutorial.</p>
<p>After filling in the Data Product name click Save to save the intermediate changes.&nbsp;</p>
<p>After saving a new field <strong>Item Types&nbsp;</strong>will appear.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Transformation.19?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Continue filling the form as shown on the screenshot</p>
<p>At the end don&#39;t forget to save the changes.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Raw Data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>in_stream</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Sample Environment Raw Data</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>-1</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Stream</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[2]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Progress Indicator</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Convert</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Add Output Line</h1>
<img alt="" src="wendelin-Data.Transformation.20?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Head back to Data Transformation and add one more Transformation line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Transformation Line</h1>
<img alt="" src="wendelin-Data.Transformation.16?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Once again&nbsp;for Document Type choose <strong>Data Transformation Resource Line</strong> and click on <strong>Create Document</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Transformation.21?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form.</p>
<p>For <strong>Data Product</strong> put the name of the Data Product that we created earlier in this tutorial - <strong>Environment Raw Array</strong>.</p>
<p>Once filling in the Data Product name click Save to save the intermediate changes.&nbsp;</p>
<p>After saving a new field <strong>Item Types&nbsp;</strong>will appear.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Transformation.22?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Continue filling the form as shown on the screenshot</p>
<p>At the end don&#39;t forget to save the changes.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Raw Data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>out_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Environment Raw Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Convert</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_use"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation</h1>
<img alt="" src="wendelin-Data.Transformation.23?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Head back to Data Transformation.</p>
<p>Now you can see that we have 3 Data Transformation Lines.</p>
<p>The first line defines what operation will be done.&nbsp;</p>
<p>The second line defines on what the operation will be done - the input.</p>
<p>And the third line defines the output.</p>
<p>Note the <strong>Quantity</strong> column of the Transformation Lines. Make sure the values are the same as shown on the screenshot.</p>
<p>The very&nbsp;last step is to validate the&nbsp;Data Transformation.&nbsp;</p>
<p>Click on Validate on the left side panel to validate the Data Transformation.</p>
</details>
</section>
<section class="screenshot">
<h1>Confirm Validation</h1>
<img alt="" src="wendelin-Data.Transformation.24?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Validate to confirm the Validation.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[2]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[3]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Analysis</h1>
<img alt="" src="wendelin-Data.Transformation.25?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Go to <strong>Data Analyses </strong>module<strong>.</strong></p>
</details>
</section>
<section class="screenshot">
<h1>Data Analysis</h1>
<img alt="" src="wendelin-Data.Transformation.26?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After few minutes in Data Analysis Module a new Data Analysis called&nbsp;<strong>Convert Environment Raw Data&nbsp;</strong>will appear.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<!--<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr>-->
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>350000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_analysis_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_analysis_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr//td[1]</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Convert Environment Raw Data</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Array Module</h1>
<img alt="" src="wendelin-Data.Transformation.27?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Go to Data Array Module to see the data.&nbsp;&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Transformation.28?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Here we can see the newly created Data Array called&nbsp;<strong>Convert Environment Raw Data.</strong></p>
<p>Click on it to navigate to array.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Transformation.29?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Preview</strong> to see the data.</p>
<p>Note it might take few minutes (&lt;10) until data appears.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Transformation.30?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Data is here!</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_array_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_array_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr//td[1]</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Convert Environment Raw Data</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//div[@class='document_table']//table//tbody//tr//td[1]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>350000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'preview')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'preview')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_05_wendelin_Tutorial_Transform_Data.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-Tutorial.Transform.Data</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Transform Data</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_05_wendelin_Tutorial_Transform_Data</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Transform Data</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
<string>installation</string>
<string>setup</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Transform Data</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_06_wendelin_Tutorial_Resample_Data.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>Create Data Operation</li>
<li>Configure Variation For Data Product</li>
<li>Create Data Product</li>
<li>Create Transformation Script</li>
<li>Create Data Transformation</li>
<li>See Data Analysis</li>
<li>See&nbsp;Data Array</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to create more&nbsp;sophisticated Data Transformation -&nbsp;resampling.&nbsp;</p>
<p>Before doing this tutorial make sure you read and completed the following tutorial and have data in your wendelin instance:</p>
<ul>
<li><a href="wendelin-Tutorial.Transform.Data">HowTo Transform&nbsp;Data</a>&nbsp;</li>
</ul>
<p>The general&nbsp;pipeline&nbsp;is exactly the same as in the previous tutorial.</p>
<p>&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Open your Wendelin dashboard.&nbsp;</p>
<p>In Modules click on Data Operations Module</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Add</strong> to add a new Data Operation.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click <strong>Proceed</strong> to continue.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill the Form</h1>
<img alt="" src="wendelin-Data.Resampling.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form to create Data Operation.</p>
<p><strong>Title&nbsp;</strong>-&nbsp; we name it&nbsp;<strong>Resample Raw Array All</strong></p>
<p><strong>Reference&nbsp;</strong>&nbsp;-&nbsp;<strong>data-operation-resample-raw-array-all&nbsp;</strong></p>
<p><strong>Script ID&nbsp;</strong>-&nbsp;<strong>DataAnalysisLine_resampleRawArrayAll</strong>:&nbsp;this script will do all the magic. It doesn&#39;t exist yet, we will create it later in this tutorial.&nbsp;</p>
<p>At the don&#39;t forget to Save the changes.</p>
</details>
</section>
<section class="screenshot">
<h1>Validate</h1>
<img alt="" src="wendelin-Data.Resampling.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Validate on left side panel to validate Data Operation.</p>
</details>
</section>
<section class="screenshot">
<h1>Confirm Validation</h1>
<img alt="" src="wendelin-Data.Resampling.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click <strong>Proceed</strong> to confirm validation.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_operation_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_operation_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resample Raw Array All</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>data-operation-resample-raw-array-all</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_version"]</td>
<td>001</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_script_id"]</td>
<td>DataAnalysisLine_resampleRawArrayAll</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_quantity_unit_list_default:list"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Data Product Variation</h1>
<img alt="" src="wendelin-Data.Resampling.System.Pref.Screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Before creating Data Product we want to add a variation of Data product called Resolution.&nbsp;</p>
<p>Navigate to page called Portal Preferences by clicking on&nbsp;<strong>Preferences&nbsp;</strong>on the left side panel.&nbsp;&nbsp;</p>
<p>It will redirect you to a page called&nbsp;<strong>Default Site Preferences.&nbsp;</strong></p>
<p>Click on the&nbsp;<strong>Preference: Default Site Preferences&nbsp;</strong>on the top of the page.&nbsp;</p>
<p>&nbsp;</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Data Product Variation</h1>
<img alt="" src="wendelin-Data.Resampling.4?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>From the list of preferences click on&nbsp;<b>Default System Preferences.</b></p>
<p>&nbsp;</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Wendelin Preferences</h1>
<img alt="" src="wendelin-Data.Resampling.5?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>On the left side panel click on&nbsp;<strong>Wendelin.</strong></p>
<p>&nbsp;</p>
</details>
</section>
<section>
<h1>Wendelin Preferences Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.6?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Then click on <strong>Editable</strong> on top panel to be able to make changes.</p>
<p>&nbsp;</p>
</details>
</section>
<section>
<h1>Wendelin Preferences Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.7?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Once the page becomes editable, a field called <strong>Data Product Individual Variation</strong> will appear.</p>
<p>From drop down list choose <strong>resolution</strong> and click on <strong>Save</strong> to save the changes.</p>
</details>
</section>
<section>
<h1>Data Product Variation Set</h1>
<img alt="" src="wendelin-Data.Resampling.8?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>New variation for Data Product called resolution is set.</p>
<p>Nest step is to create Data Product for the resampled data and see how we use this new variation.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=preference')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=preference')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Preference: Default Site Preference"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Preference: Default Site Preference"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[2]//td[1]//a</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//div[@class='document_table']//table//tbody//tr[2]//td[1]//a</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'wendelin_preference')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'wendelin_preference')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_my_preferred_data_product_individual_variation_base_category_list"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_preferred_data_product_individual_variation_base_category_list"]</td>
<td>resolution</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Data Product</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.1.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Now we need to create a new Data Product which will the output Data Product of the Transformation.</p>
</details>
</section>
<section>
<h1>Create Data Product</h1>
<img alt="" src="wendelin-Data.Resampling.9?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Create a new Data Product as described in <a href="wendelin-Tutorial.Howto.Create.Data.Product">HowTo Create Data Product</a> tutorial with following values</p>
<p>Title -&nbsp; <strong>Generic Interval Resampled Array</strong></p>
<p>Quantity Unit -&nbsp; <strong>Unit/Piece</strong></p>
<p>Reference -&nbsp;<strong>GENERIC-INTERVAL-RESAMPLED-ARRAY</strong></p>
<p>Item Types</p>
<ul>
<li><strong>Data Array</strong></li>
</ul>
<p>Use</p>
<ul>
<li><strong>Big Data/Ingestion/Stream Ingestion</strong></li>
</ul>
<p>Individual Variation Base Categories</p>
<ul>
<li><strong>resolution</strong></li>
</ul>
</details>
<p>&nbsp;</p>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Generic Interval Resampled Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_quantity_unit_list_default:list"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>GENERIC-INTERVAL-RESAMPLED-ARRAY</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_use_list_default:list"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_individual_variation_base_category_list"]</td>
<td>Resolution</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Create Data Product Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.10?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>After filling the form save the changes.</p>
<p>As this Data Product will represent our resampled data we will add here Data Product Individual Variations to define the resamplings.</p>
<p>For that, after saving the changes, click on Add button.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Create Data Product Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.11?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>For Document Type choose <strong>Product Individual Variation</strong> and click on Create Document.</p>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Create Data Product Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.12?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Fill in the title - <strong>60 S</strong> - this means resampling data with 60 seconds resolution.</p>
<p>Save the changes</p>
<p>Repeat this process 3 more times for</p>
<ul>
<li>300 S</li>
<li>3000 S</li>
<li>30000 S</li>
</ul>
</details>
<p>&nbsp;</p>
</section>
<section>
<h1>Create Data Product Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.13?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>It is ready!</p>
<p>Don&#39;t forget to Validate the Data Product.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Product Individual Variation</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>60 S</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_base_category"]</td>
<td>Resolution</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Product Individual Variation</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>300 S</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_base_category"]</td>
<td>Resolution</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Product Individual Variation</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>3000 S</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_base_category"]</td>
<td>Resolution</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Product Individual Variation</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>30000 S</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_base_category"]</td>
<td>Resolution</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Product Individual Variation:')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Portal Callables</h1>
<img alt="" src="wendelin-Data.Transformation.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After Data Product is created and validated, navigate to page called Portal Callables by adding&nbsp;<strong>/#/portal_callables</strong>&nbsp;at the end of the URL of your wendelin instance.&nbsp;</p>
<p>There we will create and store a python script that will resample data according to resolutions we defined.&nbsp;</p>
<h1>&nbsp;</h1>
</details>
</section>
<section class="screenshot">
<h1>Add Transformation Script</h1>
<img alt="" src="wendelin-Data.Transformation.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Add button to add a new script.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Transformation Script Cont.</h1>
<img alt="" src="wendelin-Data.Transformation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Choose Python Script as Document Type and click on Create Document to create an empty python script.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Resampling.14?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Define <strong>ID</strong>, <strong>Title</strong> and <strong>Reference</strong> of your script.</p>
<p>We name it <strong>DataAnalysisLine_resampleRawArrayAll</strong> as we did in Data Operation.</p>
<p>Next we define the parameters we will give to our script.&nbsp;</p>
<p><strong>in_array&nbsp;</strong>- the input dictionary that contains Data Array where raw data is stored after the previous transformation.</p>
<p><strong>out_array&nbsp;</strong>&nbsp;- the output Data Array where resampled data will be stored.&nbsp;</p>
<p>At the end click <strong>Save</strong> to save the changes.</p>
</details>
</section>
<section class="screenshot">
<h1>Transformation Script</h1>
<img alt="" src="wendelin-Data.Resampling.15?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">The script we write in the textbox area at the bottom of the page.</details>
</section>
<section class="screenshot">
<h1>Transformation Script Cont.</h1>
<pre>
<code>
import numpy as np
import pandas as pd
out_array_list = out_array
in_data_array = in_array[&quot;Data Array&quot;]
progress_indicator = in_array[&quot;Progress Indicator&quot;]
in_zbigarray = in_data_array.getArray()
if in_zbigarray is None:
return
if in_zbigarray.shape[0] == 0:
return
# first fill array with lowest resolution
default_data_array = out_array_list[0][&quot;Data Array&quot;]
default_resolution = out_array_list[0][&quot;resolution&quot;]
default_frequency = pd.to_timedelta(default_resolution)
default_zbigarray = default_data_array.getArray()
index = progress_indicator.getIntOffsetIndex()
# convert data to DataFrame
df = pd.DataFrame.from_records(in_zbigarray[index:].copy(), index=&#39;date&#39;)
# ignore data before start date of output array
if default_zbigarray is not None:
if default_zbigarray.shape[0] != 0:
df = df.loc[str(default_zbigarray[0][&#39;date&#39;]):]
if len(df) == 0:
return
# resample
df = df.resample(default_resolution).agg([&#39;min&#39;,&#39;mean&#39;,&#39;max&#39;]).fillna(0)
# rename columns from tuples like (&#39;x&#39;, &#39;min&#39;) to names like &#39;x_min&#39;
df.columns = [&#39;%s%s&#39; % (a, &#39;_%s&#39; % b if b else &#39;&#39;) for a, b in df.columns]
context.log(&quot;df.columns = &quot;, df.columns)
# save date vector for later
date_vector = df.index.values.copy()
context.log(&quot;date_vector = &quot;, date_vector)
# convert data back to ndarray
default_data = df.to_records(convert_datetime64=False)
# view as structured array
# set date to zero where all values are 0
mask_zero = (df==0).all(axis=1)
default_data[&#39;date&#39;][mask_zero] = 0
if default_zbigarray is None:
default_zbigarray = default_data_array.initArray(shape=(0,), dtype=default_data.dtype.fields)
if default_zbigarray.shape[0] == 0:
default_zbigarray.append(default_data)
else:
# calculate start and stop index of new data in output array
default_start_index = int((date_vector[0] - default_zbigarray[0][&#39;date&#39;]) / default_frequency)
default_stop_index = int((date_vector[-1] - default_zbigarray[0][&#39;date&#39;]) / default_frequency + 1)
# make sure data fits in
if default_stop_index &gt; default_zbigarray.shape[0]:
default_zbigarray.resize((default_stop_index,))
# fill holes in new data with values from old data
old_data = default_zbigarray[default_start_index:default_stop_index]
default_data[mask_zero ] = old_data[mask_zero]
# write new_data to zbigarray
default_zbigarray[default_start_index:default_stop_index] = default_data
# now use data in first resolution array for all other arrays
for out_array in out_array_list[1:]:
out_data_array = out_array[&quot;Data Array&quot;]
out_array_resolution = out_array[&quot;resolution&quot;]
out_zbigarray = out_data_array.getArray()
if out_zbigarray is None:
out_zbigarray = out_data_array.initArray(shape=(0,), dtype=default_data.dtype.fields)
if out_zbigarray.shape[0] == 0:
start_index = 0
else:
out_array_frequency = pd.to_timedelta(out_array_resolution)
new_stop_date = default_zbigarray[0][&#39;date&#39;] + default_zbigarray.shape[0] * default_frequency
old_stop_date = out_zbigarray[0][&#39;date&#39;] + out_zbigarray.shape[0] * out_array_frequency
start_date = old_stop_date - out_array_frequency
if old_stop_date &gt;= new_stop_date:
continue
# find row index in in_array from where to start resampling
start_index = int(max((start_date - default_zbigarray[0][&#39;date&#39;]) / default_frequency, 0))
# if we got data which has been already resampled, then we resample again and overwrite
start_index = min(start_index, default_start_index)
data = default_zbigarray[start_index:].copy()
# convert data to DataFrame and resample
df = pd.DataFrame.from_records(data, index=&#39;date&#39;)
# set our own date range index so that we can resample and keep 0-dates
resampling_start_date = default_zbigarray[0][&#39;date&#39;] + start_index * default_frequency
df.index = pd.date_range(start=resampling_start_date,
periods=data.shape[0],
freq=default_frequency)
df.index.name = &#39;date&#39;
# resample each column with appropriate aggregation method
aggregation_dict = {c: c.split(&#39;_&#39;)[-1] for c in df.columns}
df = df.resample(out_array_resolution).agg(aggregation_dict).fillna(0)
# save date vector for later
date_vector = df.index.values.copy()
# convert data back to ndarray
new_data = df.to_records(convert_datetime64=False)
# set date to zero where all values are 0
new_data[&#39;date&#39;][(df==0).all(axis=1)] = 0
if out_zbigarray.shape[0] == 0:
out_zbigarray.append(new_data)
else:
# calculate start and stop index of new data in output array
start_index = int((date_vector[0] - out_zbigarray[0][&#39;date&#39;]) / out_array_frequency)
stop_index = int((date_vector[-1] - out_zbigarray[0][&#39;date&#39;]) / out_array_frequency + 1)
# make sure data fits in
if stop_index &gt; out_zbigarray.shape[0]:
out_zbigarray.resize((stop_index,))
# write new_data to zbigarray
out_zbigarray[start_index:stop_index] = new_data
progress_indicator.setIntOffsetIndex(in_zbigarray.shape[0])
</code>
</pre>
<details open="">
<p>This script takes as an input data from Data Array and resamples it by given rules and writes it to new Data Arrays.</p>
<p>Don&#39;t forget to save the changes after editing the script.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=portal_callables')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=portal_callables')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Python Script</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_id"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_id"]</td>
<td>DataAnalysisLine_resampleRawArrayAll</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>DataAnalysisLine_resampleRawArrayAll</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>DataAnalysisLine_resampleRawArrayAll</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_parameter_signature"]</td>
<td>in_array={}, out_array=[]</td>
</tr>
<tr>
<td>pause</td>
<td>15000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//div[contains(@data-gadget-url,'gadget_editor.html')]//textarea</td>
<td>
import numpy as np
import pandas as pd
out_array_list = out_array
in_data_array = in_array[&quot;Data Array&quot;]
progress_indicator = in_array[&quot;Progress Indicator&quot;]
in_zbigarray = in_data_array.getArray()
if in_zbigarray is None:
&nbsp; return
if in_zbigarray.shape[0] == 0:
&nbsp; return
# first fill array with lowest resolution
default_data_array = out_array_list[0][&quot;Data Array&quot;]
default_resolution = out_array_list[0][&quot;resolution&quot;]
default_frequency = pd.to_timedelta(default_resolution)
default_zbigarray = default_data_array.getArray()
index = progress_indicator.getIntOffsetIndex()
# convert data to DataFrame
df = pd.DataFrame.from_records(in_zbigarray[index:].copy(), index=&#39;date&#39;)
# ignore data before start date of output array
if default_zbigarray is not None:
&nbsp; if default_zbigarray.shape[0] != 0:
&nbsp; &nbsp; df = df.loc[str(default_zbigarray[0][&#39;date&#39;]):]
if len(df) == 0:
&nbsp; return
# resample
df = df.resample(default_resolution).agg([&#39;min&#39;,&#39;mean&#39;,&#39;max&#39;]).fillna(0)
# rename columns from tuples like (&#39;x&#39;, &#39;min&#39;) to names like &#39;x_min&#39;
df.columns = [&#39;%s%s&#39; % (a, &#39;_%s&#39; % b if b else &#39;&#39;) for a, b in df.columns]
context.log(&quot;df.columns = &quot;, df.columns)
# save date vector for later
date_vector = df.index.values.copy()
context.log(&quot;date_vector = &quot;, date_vector)
# convert data back to ndarray
default_data = df.to_records(convert_datetime64=False)
# view as structured array
# set date to zero where all values are 0
mask_zero = (df==0).all(axis=1)
default_data[&#39;date&#39;][mask_zero] = 0
if default_zbigarray is None:
&nbsp; default_zbigarray = default_data_array.initArray(shape=(0,), dtype=default_data.dtype.fields)
if default_zbigarray.shape[0] == 0:
&nbsp; default_zbigarray.append(default_data)
else:
# calculate start and stop index of new data in output array
&nbsp; default_start_index = int((date_vector[0] - default_zbigarray[0][&#39;date&#39;]) / default_frequency)
&nbsp; default_stop_index = int((date_vector[-1] - default_zbigarray[0][&#39;date&#39;]) / default_frequency + 1)
# make sure data fits in
&nbsp; if default_stop_index &gt; default_zbigarray.shape[0]:
&nbsp; &nbsp; default_zbigarray.resize((default_stop_index,))
# fill holes in new data with values from old data
&nbsp; old_data = default_zbigarray[default_start_index:default_stop_index]
&nbsp; default_data[mask_zero ] = old_data[mask_zero]
# write new_data to zbigarray
&nbsp; default_zbigarray[default_start_index:default_stop_index] = default_data
# now use data in first resolution array for all other arrays
for out_array in out_array_list[1:]:
&nbsp; out_data_array = out_array[&quot;Data Array&quot;]
&nbsp; out_array_resolution = out_array[&quot;resolution&quot;]
&nbsp; out_zbigarray = out_data_array.getArray()
&nbsp; if out_zbigarray is None:
&nbsp; &nbsp; out_zbigarray = out_data_array.initArray(shape=(0,), dtype=default_data.dtype.fields)
&nbsp; if out_zbigarray.shape[0] == 0:
&nbsp; &nbsp; start_index = 0
&nbsp; else:
&nbsp; &nbsp; out_array_frequency = pd.to_timedelta(out_array_resolution)
&nbsp; &nbsp; new_stop_date = default_zbigarray[0][&#39;date&#39;] + default_zbigarray.shape[0] * default_frequency
&nbsp; &nbsp; old_stop_date = out_zbigarray[0][&#39;date&#39;] + out_zbigarray.shape[0] * out_array_frequency
&nbsp; &nbsp; start_date = old_stop_date - out_array_frequency
&nbsp; &nbsp; if old_stop_date &gt;= new_stop_date:
&nbsp; &nbsp; &nbsp; continue
# find row index in in_array from where to start resampling
&nbsp; &nbsp; start_index = int(max((start_date - default_zbigarray[0][&#39;date&#39;]) / default_frequency, 0))
# if we got data which has been already resampled, then we resample again and overwrite
&nbsp; &nbsp; start_index = min(start_index, default_start_index)
&nbsp; data = default_zbigarray[start_index:].copy()
# convert data to DataFrame and resample
&nbsp; df = pd.DataFrame.from_records(data, index=&#39;date&#39;)
# set our own date range index so that we can resample and keep 0-dates
&nbsp; resampling_start_date = default_zbigarray[0][&#39;date&#39;] + start_index * default_frequency
&nbsp; df.index = pd.date_range(start=resampling_start_date,
periods=data.shape[0],
freq=default_frequency)
&nbsp; df.index.name = &#39;date&#39;
# resample each column with appropriate aggregation method
&nbsp; aggregation_dict = {c: c.split(&#39;_&#39;)[-1] for c in df.columns}
&nbsp; df = df.resample(out_array_resolution).agg(aggregation_dict).fillna(0)
# save date vector for later
&nbsp; date_vector = df.index.values.copy()
# convert data back to ndarray
&nbsp; new_data = df.to_records(convert_datetime64=False)
# set date to zero where all values are 0
&nbsp; new_data[&#39;date&#39;][(df==0).all(axis=1)] = 0
&nbsp; if out_zbigarray.shape[0] == 0:
&nbsp; &nbsp; out_zbigarray.append(new_data)
&nbsp; else:
# calculate start and stop index of new data in output array
&nbsp; &nbsp; start_index = int((date_vector[0] - out_zbigarray[0][&#39;date&#39;]) / out_array_frequency)
&nbsp; &nbsp; stop_index = int((date_vector[-1] - out_zbigarray[0][&#39;date&#39;]) / out_array_frequency + 1)
# make sure data fits in
&nbsp; &nbsp; if stop_index &gt; out_zbigarray.shape[0]:
&nbsp; &nbsp; &nbsp; out_zbigarray.resize((stop_index,))
# write new_data to zbigarray
&nbsp; &nbsp; out_zbigarray[start_index:stop_index] = new_data
progress_indicator.setIntOffsetIndex(in_zbigarray.shape[0])
</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformations</h1>
<img alt="" src="wendelin-Data.Transformation.7?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After your transformation script is ready, it&#39;s time to create Data Transformation.</p>
<p>On Modules page click on Data Transformations.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Transformation</h1>
<img alt="" src="wendelin-Data.Resampling.16?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Add to add a new Data Transformation.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Data Transformation</h1>
<img alt="" src="wendelin-Data.Transformation.9?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Create Document to continue.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Resampling.17?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Chose descriptive title and reference. For example</p>
<p>Title : <strong>Resample Generic Interval Array</strong></p>
<p>Reference : <strong>generic-interval-resample</strong></p>
<p>For Initial Product we choose the Data Product we created in&nbsp;&nbsp;<a href="wendelin-Tutorial.Transform.Data">HowTo Transform&nbsp;Data</a>&nbsp;tutorial.</p>
<p>At the end click Save to save the changes</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_transformation_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_transformation_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resample Generic Interval Array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>generic-interval-resample</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Environment Raw Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation Lines</h1>
<img alt="" src="wendelin-Data.Resampling.18?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on add button to add a Data Transformation Line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Operation Line</h1>
<img alt="" src="wendelin-Data.Transformation.12?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Choose <strong>Data Transformation Operation Line&nbsp;</strong>for Document Type and click on Create Document.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Resampling.19?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form with followig values</p>
<p>Title: <strong>Resample Raw Array</strong></p>
<p>reference: <strong>data_operation</strong></p>
<p>Data Operation: put the name of the Data operation that we created at the beginning of this tutorial - <strong>Resample Raw Array All</strong>.</p>
<p>Trade Phase: <strong>Data/Subsample</strong>.</p>
<p>Quantity: <strong>1</strong></p>
<p>Quantity Unit: <strong>Unit/Piece</strong></p>
<p>At the end click on Save to save the changes.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Back To Data Transformation</h1>
<img alt="" src="wendelin-Data.Resampling.20?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Once needed fields are filled and saved go back to Data Transformations view by clicking on the upper panel.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resample Raw Array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>data_operation</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Resample Raw Array All</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Subsample</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Operation Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Operation Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation Lines</h1>
<img alt="" src="wendelin-Data.Resampling.21?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Add another Data Transformation Line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Transformation Line</h1>
<img alt="" src="wendelin-Data.Transformation.16?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Now for Document Type choose <strong>Data Transformation Resource Line</strong> and click on <strong>Create Document</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Resampling.22?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form with following values</p>
<p>Title: <strong>Raw Array</strong></p>
<p>Reference: <strong>in_array</strong></p>
<p>Quantity:&nbsp;<strong>-1</strong></p>
<p>Quantity Unit: <strong>Unit/Piece</strong></p>
<p>Data Product: put the name of the Data Product that we created in <a href="wendelin-Tutorial.Transform.Data">HowTo Transform Data</a>&nbsp;tutorial - <strong>Environment Raw Array</strong></p>
<p>After filling in the Data Product name click Save to save the intermediate changes.&nbsp;</p>
<p>After saving a new field <strong>Item Types&nbsp;</strong>will appear.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.23?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Continue filling the form as shown on the screenshot</p>
<p>At the end don&#39;t forget to save the changes.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Raw Array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>in_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Environment Raw Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>-1</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[2]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Progress Indicator</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Subsample</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Add Output Line</h1>
<img alt="" src="wendelin-Data.Resampling.24?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Head back to Data Transformation and add one more Transformation line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Transformation Line</h1>
<img alt="" src="wendelin-Data.Transformation.16?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Once again&nbsp;for Document Type choose <strong>Data Transformation Resource Line</strong> and click on <strong>Create Document</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Resampling.25?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form with following values</p>
<p>Title: <strong>Resampled Array (60 Seconds)</strong></p>
<p>Reference: <strong>out_array</strong></p>
<p>Data Product: put the name of the Data Product that we created earlier in this tutorial - <strong>Generic Interval Resampled Array</strong>.</p>
<p>Quantity: <strong>1</strong></p>
<p>Quantity Unit: <strong>Unit/Piece</strong></p>
<p>Then click Save to save the intermediate changes.&nbsp;</p>
<p>After saving a new field <strong>Item Types&nbsp;</strong>will appear.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Resampling.26?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Continue filling the form as shown on the screenshot</p>
<p>Item Types: <strong>Data Array</strong></p>
<p>Trade Phase: <strong>Data/Subsample</strong></p>
<p>Use: <strong>Big Data/Ingestion/Stream Ingestion</strong></p>
<p>Default Variation: <strong>Resolution/60 S</strong></p>
<p>At the end don&#39;t forget to save the changes.</p>
<p>&nbsp;</p>
<p><strong>Important:&nbsp;</strong>Create 3 more Data Transformation resource lines with&nbsp;300S, 3000S and 30000S resolutions.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resampled Array (60 Seconds)</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>out_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Generic Interval Resampled Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Subsample</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_use"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_category_list"]</td>
<td>Resolution/60 S</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resampled Array (300 Seconds)</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>out_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Generic Interval Resampled Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Subsample</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_use"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_category_list"]</td>
<td>Resolution/300 S</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resampled Array (3000 Seconds)</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>out_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Generic Interval Resampled Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Subsample</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_use"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_category_list"]</td>
<td>Resolution/3000 S</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resampled Array (30000 Seconds)</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>out_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Generic Interval Resampled Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Subsample</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_use"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_category_list"]</td>
<td>Resolution/30000 S</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation</h1>
<img alt="" src="wendelin-Data.Resampling.27?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Head back to Data Transformation.</p>
<p>Now you can see that we have 6 Data Transformation Lines.</p>
<p>The first line defines what operation will be done.&nbsp;</p>
<p>The second line defines on what the operation will be done - the input.</p>
<p>Lines 3 to 6 define the output with different resolutions.</p>
<p>The very&nbsp;last step is to validate the&nbsp;Data Transformation.&nbsp;</p>
<p>Click on Validate on the left side panel to validate the Data Transformation.</p>
</details>
</section>
<section class="screenshot">
<h1>Confirm Validation</h1>
<img alt="" src="wendelin-Data.Resampling.28?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Validate to confirm the Validation.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[2]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[3]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[4]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[5]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[6]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Analysis</h1>
<img alt="" src="wendelin-Data.Transformation.25?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Go to <strong>Data Analyses </strong>module<strong>.</strong></p>
</details>
</section>
<section class="screenshot">
<h1>Data Analysis</h1>
<img alt="" src="wendelin-Data.Resampling.29?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After few minutes in Data Analysis Module a new Data Analysis called&nbsp;<strong>Resample Generic Interval Array&nbsp;</strong>will appear.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>350000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_analysis_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_analysis_module')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr//td[1]</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Resample Generic Interval Array</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Array Module</h1>
<img alt="" src="wendelin-Data.Transformation.27?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Go to Data Array Module to see the data.&nbsp;&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Resampling.30?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Here we can see 4 newly created Data Arrays with four different resolutions.</p>
<p>Click on one of them to navigate to the array.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Resampling.31?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Preview</strong> to see the data.</p>
<p>Note it might take few minutes (&lt;10) until data appears.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Resampling.32?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Data is here!</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>350000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_array_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_array_module')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr//td[1]</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Resample Generic Interval Array</td>
<td></td>
</tr>
<tr>
<td>assertTextNotPresent</td>
<td>None</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_06_wendelin_Tutorial_Resample_Data.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-Tutorial.Resample.Data</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Resample Data</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_06_wendelin_Tutorial_Resample_Data</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Resample Data</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
<string>installation</string>
<string>setup</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Resample Data</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_07_wendelin_HowTo_Create_Notebook_with_Graphs.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>Create Notebook</li>
<li>Visualise&nbsp;Data</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to visualise data.</p>
<p>Before doing this tutorial make sure you read and completed the following tutorial and have data in your wendelin instance:</p>
<ul>
<li><a href="wendelin-Tutorial.Transform.Data">HowTo Transform&nbsp;Data</a>&nbsp;</li>
<li><a href="wendelin-Tutorial.Resample.Data">HowTo Resample Data</a></li>
</ul>
<p>&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Notebook Module</h1>
<img alt="" src="wendelin-Data.Visualisation.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>On Modules page click on <strong>Notebooks</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Notebook</h1>
<img alt="" src="wendelin-Data.Visualisation.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Add</strong> to add a new Notebook.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Notebook</h1>
<img alt="" src="wendelin-Data.Visualisation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Create Document</strong> to continue.</p>
</details>
</section>
<section class="screenshot">
<h1>Notebook</h1>
<img alt="" src="wendelin-Data.Visualisation.4?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill in the Title.</p>
<p>The visualisation code will be at the bottom text area.</p>
<p>Notebook uses the same technology as&nbsp;<a href="https://alpha.iodide.io/">Iodide</a>.&nbsp;</p>
<p>To learn more about programming in&nbsp;Iodide environment checkout&nbsp;<a href="https://alpha.iodide.io/">Iodide</a>&nbsp;homepage.</p>
</details>
</section>
<section class="screenshot">
<h1>Visualisation Script</h1>
<pre>
<code>
%% md
# Time Series
&lt;div class=&quot;plot_div&quot; id=&quot;plot_div&quot;/&gt;
%% fetch
js: jio.js
js: ndarray_bundle.js
js: wendelin.js
js: https://cdn.plot.ly/plotly-latest.min.js
%% js
hateoas_url = &quot;https://softinst133633.host.vifib.net/erp5/web_site_module/renderjs_runner/hateoas/&quot;;
jio = jIO.createJIO({
&nbsp; type: &quot;erp5&quot;,
&nbsp; url: hateoas_url,
&nbsp; default_view_reference: &quot;view&quot;
});
gadget = {
&nbsp; getSetting: function(property) {
&nbsp; &nbsp; return new RSVP.Queue()
&nbsp; &nbsp; &nbsp; .push(function () {
&nbsp; &nbsp; &nbsp; &nbsp; if (property == &quot;hateoas_url&quot;) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return hateoas_url;
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; return;
&nbsp; &nbsp; &nbsp; });
&nbsp; },
&nbsp; jio_getAttachment: function(id, url, parameter_dict) {
&nbsp; &nbsp; return jio.getAttachment(id, url, parameter_dict);
&nbsp; },
&nbsp; jio_get: function(id) {return jio.get(id);}
}
var graph = document.getElementById(&#39;plot_div&#39;),
&nbsp; graph_data = [],
&nbsp; label_list = [&quot;date&quot;, &quot;humidity_min&quot;,&quot;humidity_mean&quot;,&quot;humidity_max&quot;, &quot;pressure_min&quot;,&quot;pressure_mean&quot;,&quot;pressure_max&quot;, &quot;temperature_min&quot;,&quot;temperature_mean&quot;, &quot;temperature_max&quot;];
selectorOptions = {
&nbsp; &nbsp; &quot;buttons&quot;: [{
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;day&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 1,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;day&#39;
&nbsp; &nbsp; },/* {
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;day&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 7,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;week&#39;
&nbsp; &nbsp; },*/ {
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;month&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 1,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;month&#39;
&nbsp; &nbsp; }, {
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;year&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 1,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;year&#39;
&nbsp; &nbsp; }],
}
function update_graph(start_date, stop_date){
&nbsp; return getData(start_date, stop_date)
&nbsp; &nbsp; .push(function (data) {
&nbsp; &nbsp; &nbsp; return Plotly.react(graph,
&nbsp; &nbsp; &nbsp; &nbsp; data,
&nbsp; &nbsp; &nbsp; &nbsp; { &#39;title&#39; :&#39;Environmental Sensor Data&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;xaxis&#39;: {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;autorange&#39;: true,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;rangeselector&#39;: selectorOptions,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; );
&nbsp; &nbsp; });
}
function unpack(rows, key) {
&nbsp; return rows.map(function(row) { return row[key]; });
}
function getData(start_date, stop_date){
&nbsp; var graph_data=[],
&nbsp; &nbsp; ratio = screen.availWidth / 1024,
&nbsp; &nbsp; interval,
&nbsp; &nbsp; frequency,
&nbsp; &nbsp; start_index,
&nbsp; &nbsp; stop_index,
&nbsp; &nbsp; array_id;
&nbsp; &nbsp;if (stop_date === undefined) {
&nbsp; &nbsp; &nbsp;frequency = 30000;
&nbsp; &nbsp; &nbsp;start_index = 0;
&nbsp; &nbsp; &nbsp;stop_index = 0;
&nbsp; &nbsp;} else {
&nbsp; &nbsp; interval = stop_date - start_date;
&nbsp; &nbsp; if (interval &gt;= 1000 * 60 * 60 * 24 * 100 * ratio) {
&nbsp; &nbsp; &nbsp; frequency = 30000;
&nbsp; &nbsp; } else if (interval &gt;= 1000 * 60 * 60 * 24 * 28 * ratio) {
&nbsp; &nbsp; &nbsp; frequency = 3000;
&nbsp; &nbsp; } else if (interval &gt;= 1000 * 60 * 60 * 24 * 7 * ratio) {
&nbsp; &nbsp; &nbsp; frequency = 300;
&nbsp; &nbsp; } else {
&nbsp; &nbsp; &nbsp; frequency = 60;
&nbsp; &nbsp; }
&nbsp; }
&nbsp; return jio.allDocs({
&nbsp; &nbsp; query: &#39;portal_type:&quot;Data Analysis Line&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;title: &quot;Resampled Array (&#39; + frequency + &#39; Seconds)&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;resource_reference:&quot;GENERIC-INTERVAL-RESAMPLED-ARRAY&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;parent_source_title:&quot;Sample Factory&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;simulation_state:&quot;started&quot;&#39;
&nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; var data_analysis_line_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return jio.allDocs({
&nbsp; &nbsp; &nbsp; &nbsp; query: &#39;portal_type:&quot;Data Array&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;aggregate_related_relative_url:&quot;&#39; + data_analysis_line_id +&#39;&quot;&#39;
&nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, 0, 1);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_start_date = wendelin.convertFirstColToDate([[result.data[0]]])[0][0];
&nbsp; &nbsp; &nbsp; if (start_index === undefined) {
&nbsp; &nbsp; &nbsp; &nbsp; start_index = Math.max(0, Math.ceil((start_date - array_start_date) / (frequency*1000))),
&nbsp; &nbsp; &nbsp; &nbsp; stop_index = Math.ceil((stop_date - array_start_date) / (frequency*1000));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, start_index, stop_index);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result) {
&nbsp; &nbsp; &nbsp; for (i = 0; i &lt; label_list.length; i += 1) {
&nbsp; &nbsp; &nbsp; &nbsp; graph_data = graph_data.concat(nj.unpack(result.pick( null, label_list[i])));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return graph_data
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result){
&nbsp; &nbsp; &nbsp; var filtered_graph_data = [];
&nbsp; &nbsp; &nbsp; for (var i=0; i&lt;result.length; i++) {
&nbsp; &nbsp; &nbsp; &nbsp; if (result[i][0] != 0) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; filtered_graph_data.push(result[i]);
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.convertFirstColToDate(filtered_graph_data)
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (graph_data) {
&nbsp; &nbsp; &nbsp; var humidity_trace = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Humidity(%)&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, label_list.indexOf(&quot;date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, label_list.indexOf(&quot;humidity_mean&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#17BECF&#39;}
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; var pressure_trace = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Pressure(Pa)&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, label_list.indexOf(&quot;date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, label_list.indexOf(&quot;pressure_mean&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#7F7F7F&#39;}
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; var temperature_trace = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Temperature(C&deg;)&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, label_list.indexOf(&quot;date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, label_list.indexOf(&quot;temperature_mean&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#FF0000&#39;}
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; var data = [humidity_trace,pressure_trace,temperature_trace];
&nbsp; &nbsp; &nbsp; return data
&nbsp; &nbsp; });
}
function plot () {
&nbsp; return getData()
&nbsp; &nbsp; .push(function (data) {
&nbsp; &nbsp; &nbsp; Plotly.newPlot(
&nbsp; &nbsp; &nbsp; &nbsp; graph,
&nbsp; &nbsp; &nbsp; &nbsp; data,
&nbsp; &nbsp; &nbsp; &nbsp; { &#39;title&#39; :&#39;Environmental Sensor Data&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;xaxis&#39;: {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;autorange&#39;: true,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;rangeselector&#39;: selectorOptions,}
&nbsp; &nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; &nbsp; graph.on(&#39;plotly_relayout&#39;, function(eventdata) {
&nbsp; &nbsp; &nbsp; &nbsp; var start_date = new Date(eventdata[&quot;xaxis.range[0]&quot;]),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; stop_date = new Date(eventdata[&quot;xaxis.range[1]&quot;]);
&nbsp; &nbsp; &nbsp; &nbsp; return update_graph(start_date, stop_date);
&nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; });
}
plot();</code>
</pre>
<details open="">
<p>Copy/paste the code to your notebook and save the changes.&nbsp;</p>
<p><strong>Note:&nbsp;</strong>you will need to provide the URL of your instance in the script.&nbsp;</p>
<p>For that change the line :</p>
<pre>
<code>hateoas_url = &quot;https://softinstXXX.host.vifib.net/erp5/web_site_module/renderjs_runner/hateoas/&quot;;
</code></pre>
<p>The script will find automatically find the resampled arrays based on the Data Product&nbsp;and plot the data using&nbsp;<a href="https://plotly.com/">plotly library</a>.</p>
</details>
</section>
<section class="screenshot">
<h1>Notebook</h1>
<img alt="" src="wendelin-Data.Visualisation.5?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After writing the script and saving the changes click on the <strong>Editable</strong> checkbox to (uncheck it) on the left side panel to see the result.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Visualisation</h1>
<img alt="" src="wendelin-Data.Visualisation.6?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>If everything was done correctly you will see the plot of your data.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'key=notebook_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'key=notebook_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Environment Data Graph</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[contains(@class,'CodeMirror')]//div//textarea</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//div[contains(@class,'CodeMirror')]//div//textarea</td>
<td>%% md
# Time Series
&lt;div class=&quot;plot_div&quot; id=&quot;plot_div&quot;/&gt;
%% fetch
js: jio.js
js: ndarray_bundle.js
js: wendelin.js
js: https://cdn.plot.ly/plotly-latest.min.js
%% js
hateoas_url = &quot;${base_url_x}/web_site_module/renderjs_runner/hateoas/&quot;;
jio = jIO.createJIO({
&nbsp; type: &quot;erp5&quot;,
&nbsp; url: hateoas_url,
&nbsp; default_view_reference: &quot;view&quot;
});
gadget = {
&nbsp; getSetting: function(property) {
&nbsp; &nbsp; return new RSVP.Queue()
&nbsp; &nbsp; &nbsp; .push(function () {
&nbsp; &nbsp; &nbsp; &nbsp; if (property == &quot;hateoas_url&quot;) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return hateoas_url;
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; return;
&nbsp; &nbsp; &nbsp; });
&nbsp; },
&nbsp; jio_getAttachment: function(id, url, parameter_dict) {
&nbsp; &nbsp; return jio.getAttachment(id, url, parameter_dict);
&nbsp; },
&nbsp; jio_get: function(id) {return jio.get(id);}
}
var graph = document.getElementById(&#39;plot_div&#39;),
&nbsp; graph_data = [],
&nbsp; label_list = [&quot;date&quot;, &quot;humidity_min&quot;,&quot;humidity_mean&quot;,&quot;humidity_max&quot;, &quot;pressure_min&quot;,&quot;pressure_mean&quot;,&quot;pressure_max&quot;, &quot;temperature_min&quot;,&quot;temperature_mean&quot;, &quot;temperature_max&quot;];
selectorOptions = {
&nbsp; &nbsp; &quot;buttons&quot;: [{
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;day&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 1,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;day&#39;
&nbsp; &nbsp; },/* {
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;day&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 7,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;week&#39;
&nbsp; &nbsp; },*/ {
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;month&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 1,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;month&#39;
&nbsp; &nbsp; }, {
&nbsp; &nbsp; &nbsp; &nbsp; &quot;step&quot;: &#39;year&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;stepmode&quot;: &#39;backward&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;count&quot;: 1,
&nbsp; &nbsp; &nbsp; &nbsp; &quot;label&quot;: &#39;year&#39;
&nbsp; &nbsp; }],
}
function update_graph(start_date, stop_date){
&nbsp; return getData(start_date, stop_date)
&nbsp; &nbsp; .push(function (data) {
&nbsp; &nbsp; &nbsp; return Plotly.react(graph,
&nbsp; &nbsp; &nbsp; &nbsp; data,
&nbsp; &nbsp; &nbsp; &nbsp; { &#39;title&#39; :&#39;Environmental Sensor Data&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;xaxis&#39;: {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;autorange&#39;: true,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;rangeselector&#39;: selectorOptions,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; );
&nbsp; &nbsp; });
}
function unpack(rows, key) {
&nbsp; return rows.map(function(row) { return row[key]; });
}
function getData(start_date, stop_date){
&nbsp; var graph_data=[],
&nbsp; &nbsp; ratio = screen.availWidth / 1024,
&nbsp; &nbsp; interval,
&nbsp; &nbsp; frequency,
&nbsp; &nbsp; start_index,
&nbsp; &nbsp; stop_index,
&nbsp; &nbsp; array_id;
&nbsp; &nbsp;if (stop_date === undefined) {
&nbsp; &nbsp; &nbsp;frequency = 30000;
&nbsp; &nbsp; &nbsp;start_index = 0;
&nbsp; &nbsp; &nbsp;stop_index = 0;
&nbsp; &nbsp;} else {
&nbsp; &nbsp; interval = stop_date - start_date;
&nbsp; &nbsp; if (interval &gt;= 1000 * 60 * 60 * 24 * 100 * ratio) {
&nbsp; &nbsp; &nbsp; frequency = 30000;
&nbsp; &nbsp; } else if (interval &gt;= 1000 * 60 * 60 * 24 * 28 * ratio) {
&nbsp; &nbsp; &nbsp; frequency = 3000;
&nbsp; &nbsp; } else if (interval &gt;= 1000 * 60 * 60 * 24 * 7 * ratio) {
&nbsp; &nbsp; &nbsp; frequency = 300;
&nbsp; &nbsp; } else {
&nbsp; &nbsp; &nbsp; frequency = 60;
&nbsp; &nbsp; }
&nbsp; }
&nbsp; return jio.allDocs({
&nbsp; &nbsp; query: &#39;portal_type:&quot;Data Analysis Line&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;title: &quot;Resampled Array (&#39; + frequency + &#39; Seconds)&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;resource_reference:&quot;GENERIC-INTERVAL-RESAMPLED-ARRAY&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;parent_source_title:&quot;Sample Factory&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;simulation_state:&quot;started&quot;&#39;
&nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; var data_analysis_line_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return jio.allDocs({
&nbsp; &nbsp; &nbsp; &nbsp; query: &#39;portal_type:&quot;Data Array&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;aggregate_related_relative_url:&quot;&#39; + data_analysis_line_id +&#39;&quot;&#39;
&nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, 0, 1);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_start_date = wendelin.convertFirstColToDate([[result.data[0]]])[0][0];
&nbsp; &nbsp; &nbsp; if (start_index === undefined) {
&nbsp; &nbsp; &nbsp; &nbsp; start_index = Math.max(0, Math.ceil((start_date - array_start_date) / (frequency*1000))),
&nbsp; &nbsp; &nbsp; &nbsp; stop_index = Math.ceil((stop_date - array_start_date) / (frequency*1000));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, start_index, stop_index);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result) {
&nbsp; &nbsp; &nbsp; for (i = 0; i &lt; label_list.length; i += 1) {
&nbsp; &nbsp; &nbsp; &nbsp; graph_data = graph_data.concat(nj.unpack(result.pick( null, label_list[i])));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return graph_data
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result){
&nbsp; &nbsp; &nbsp; var filtered_graph_data = [];
&nbsp; &nbsp; &nbsp; for (var i=0; i&lt;result.length; i++) {
&nbsp; &nbsp; &nbsp; &nbsp; if (result[i][0] != 0) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; filtered_graph_data.push(result[i]);
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.convertFirstColToDate(filtered_graph_data)
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (graph_data) {
&nbsp; &nbsp; &nbsp; var humidity_trace = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Humidity(%)&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, label_list.indexOf(&quot;date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, label_list.indexOf(&quot;humidity_mean&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#17BECF&#39;}
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; var pressure_trace = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Pressure(Pa)&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, label_list.indexOf(&quot;date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, label_list.indexOf(&quot;pressure_mean&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#7F7F7F&#39;}
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; var temperature_trace = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Temperature(C&deg;)&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, label_list.indexOf(&quot;date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, label_list.indexOf(&quot;temperature_mean&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#FF0000&#39;}
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; var data = [humidity_trace,pressure_trace,temperature_trace];
&nbsp; &nbsp; &nbsp; return data
&nbsp; &nbsp; });
}
function plot () {
&nbsp; return getData()
&nbsp; &nbsp; .push(function (data) {
&nbsp; &nbsp; &nbsp; Plotly.newPlot(
&nbsp; &nbsp; &nbsp; &nbsp; graph,
&nbsp; &nbsp; &nbsp; &nbsp; data,
&nbsp; &nbsp; &nbsp; &nbsp; { &#39;title&#39; :&#39;Environmental Sensor Data&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;xaxis&#39;: {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;autorange&#39;: true,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;rangeselector&#39;: selectorOptions,}
&nbsp; &nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; &nbsp; graph.on(&#39;plotly_relayout&#39;, function(eventdata) {
&nbsp; &nbsp; &nbsp; &nbsp; var start_date = new Date(eventdata[&quot;xaxis.range[0]&quot;]),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; stop_date = new Date(eventdata[&quot;xaxis.range[1]&quot;]);
&nbsp; &nbsp; &nbsp; &nbsp; return update_graph(start_date, stop_date);
&nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; });
}
plot();
</td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//input[@id="editable" and @type="checkbox"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>20000</td>
<td></td>
</tr>
<tr>
<td>assertElementPresent</td>
<td>//div[@id="plot_div"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_07_wendelin_HowTo_Create_Notebook_with_Graphs.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-HowTo.Create.Notebook.with.Graphs</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Create Notebook with Graphs</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_07_wendelin_HowTo_Create_Notebook_with_Graphs</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Create Notebook with Graphs</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
<string>installation</string>
<string>setup</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Create Notebook with Graphs</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_08_wendelin_HowTo_Create_Basic_Prediction_with_scikit_learn.html 0 → 100644
View file @ bce74d9d
<section>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>store</td>
<td tal:content="python:here.getPortalObject().absolute_url()">http://example.com/erp5</td>
<td>base_url_x</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>
<h1>Agenda</h1>
<ul>
<li>Create Data Operation</li>
<li>Create Data Product</li>
<li>Create Transformation Script</li>
<li>Create Data Transformation</li>
<li>See Data Analysis</li>
<li>See&nbsp;Data Array</li>
<li>Plot Data</li>
</ul>
<details>
<p>&nbsp;</p>
<p>This tutorial shows an example of Machine Learning with Wendelin.&nbsp;</p>
<p>Based on data from previous tutorials we will predict air pressure from humidity using&nbsp;<a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html">Linear Regression by scikit-learn</a>&nbsp;and visualize results.&nbsp;</p>
<p>Before doing this tutorial make sure you read and completed the following tutorials&nbsp;</p>
<ul>
<li><a href="wendelin-Tutorial.Transform.Data">HowTo Transform&nbsp;Data</a>&nbsp;</li>
<li><a href="wendelin-Tutorial.Resample.Data">HowTo Resample Data</a></li>
<li><a href="wendelin-HowTo.Create.Notebook.with.Graphs">HowTo Create Notebook with Graphs</a></li>
</ul>
<p>&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Open your Wendelin dashboard.&nbsp;</p>
<p>In Modules click on Data Operations Module</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Operation</h1>
<img alt="" src="wendelin-Data.Prediction.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Add</strong> to add a new Data Operation.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Operation</h1>
<img alt="" src="wendelin-Data.Operation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click <strong>Proceed</strong> to continue.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill the Form</h1>
<img alt="" src="wendelin-Data.Prediction.2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form to create Data Operation.</p>
<p><strong>Title&nbsp;</strong>-&nbsp; we name it&nbsp;<strong>Predict Pressure</strong></p>
<p><strong>Reference&nbsp;</strong>&nbsp;-&nbsp;<strong>data-operation-predict-pressure</strong></p>
<p><strong>Script ID&nbsp;</strong>-&nbsp;<strong>DataAnalysisLine_predictPressureFromHumidity</strong>:&nbsp;this script will do all the magic. It doesn&#39;t exist yet, we will create it later in this tutorial.&nbsp;</p>
<p>At the don&#39;t forget to Save the changes.</p>
</details>
</section>
<section class="screenshot">
<h1>Validate</h1>
<img alt="" src="wendelin-Data.Prediction.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Validate on left side panel to validate Data Operation.</p>
</details>
</section>
<section class="screenshot">
<h1>Confirm Validation</h1>
<img alt="" src="wendelin-Data.Prediction.4?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click <strong>Validate</strong> to confirm validation.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url_x}/web_site_module/default_wendelin_front/</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_operation_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_operation_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Predict Pressure</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>data-operation-predict-pressure</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_version"]</td>
<td>001</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_script_id"]</td>
<td>DataAnalysisLine_predictPressureFromHumidity</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Acquisition Unit</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section>
<h1>Data Product</h1>
<img alt="" src="wendelin-new.Interface.add.Data.Product.1.screenshot?format=" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Now we need to create a new Data Product which will the output Data Product of the Transformation.</p>
</details>
</section>
<section>
<h1>Create Data Product</h1>
<img alt="" src="wendelin-Data.Prediction.5?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Create a new Data Product as described in <a href="wendelin-Tutorial.Howto.Create.Data.Product">HowTo Create Data Product</a> tutorial with following values</p>
<p>Title -&nbsp; <strong>Predicted Data Array</strong></p>
<p>Quantity Unit -&nbsp; <strong>Unit/Piece</strong></p>
<p>Reference -&nbsp;<strong>environment-predicted-array</strong></p>
<p>Item Types</p>
<ul>
<li><strong>Data Array</strong></li>
</ul>
<p>Use</p>
<ul>
<li><strong>Big Data/Ingestion/Stream Ingestion</strong></li>
</ul>
<p>At the end don&#39;t forget to save the changes and <strong>Validate</strong>.</p>
</details>
<p>&nbsp;</p>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_product_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Predicted Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_quantity_unit_list_default:list"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>environment-predicted-array</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="subfield_field_my_use_list_default:list"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Portal Callables</h1>
<img alt="" src="wendelin-Data.Prediction.6?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After Data Product is created and validated, navigate to page called Portal Callables by adding&nbsp;<strong>/#/portal_callables</strong>&nbsp;at the end of the URL of your wendelin instance.&nbsp;</p>
<p>There we will create and store a the prediction script.&nbsp;&nbsp;</p>
<h1>&nbsp;</h1>
</details>
</section>
<section class="screenshot">
<h1>Add Transformation Script</h1>
<img alt="" src="wendelin-Data.Prediction.7?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Add button to add a new script.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Transformation Script Cont.</h1>
<img alt="" src="wendelin-Data.Transformation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Choose Python Script as Document Type and click on Create Document to create an empty python script.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Prediction.8?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Define <strong>ID</strong>, <strong>Title</strong> and <strong>Reference</strong> of your script.</p>
<p>We name it <strong>DataAnalysisLine_predictPressureFromHumidity</strong> as we did in Data Operation at the beginning of this tutorial.</p>
<p>Next we define the parameters we will give to our script.&nbsp;</p>
<p><strong>in_array&nbsp;</strong>- the input dictionary that contains Data Array where raw data is stored after the previous transformation.</p>
<p><strong>out_array&nbsp;</strong>&nbsp;- the output Data Array where results of the prediction will be stored.&nbsp;</p>
<p>At the end click <strong>Save</strong> to save the changes.</p>
</details>
</section>
<section class="screenshot">
<h1>Transformation Script</h1>
<img alt="" src="wendelin-Data.Prediction.9?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>The script we write in the textbox area at the bottom of the page.</p>
</details>
</section>
<section class="screenshot">
<h1>Transformation Script Cont.</h1>
<pre>
<code>
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
def predict(zbigarray):
&nbsp; humidity = zbigarray[:][&quot;humidity_mean&quot;]
&nbsp; pressure = zbigarray[:][&quot;pressure_mean&quot;]
&nbsp; humidity = humidity[humidity !=0 ]
&nbsp; pressure = pressure[pressure !=0 ]
&nbsp; humid_train, humid_test, pressure_train, pressure_test = train_test_split(humidity.reshape(-1,1), pressure.reshape(-1,1), test_size=0.2, random_state=0)
&nbsp; regressor = LinearRegression()
&nbsp; regressor.fit(humid_train, pressure_train) #training the algorithm
&nbsp; pressure_pred = regressor.predict( zbigarray[:][&quot;humidity_mean&quot;].reshape(-1,1))
&nbsp; df = pd.DataFrame({&#39;Date&#39;: zbigarray[:][&quot;date&quot;],&#39;Actual&#39;:zbigarray[:][&quot;pressure_mean&quot;], &#39;Predicted&#39;: pressure_pred.flatten()})
&nbsp; return &nbsp;df
out_data_array = out_array[&quot;Data Array&quot;]
in_data_array = in_array[&quot;Data Array&quot;]
in_zbigarray = in_data_array.getArray()
if in_zbigarray is None:
&nbsp; return
if in_zbigarray.shape[0] == 0:
&nbsp; return
df = predict(in_zbigarray)
ndarray = df.to_records(convert_datetime64=False)
dtype = [(&#39;Date&#39;, &#39;&lt;M8[ns]&#39;),(&#39;Actual&#39;, &#39;&lt;f8&#39;), (&#39;Predicted&#39;, &#39;&lt;f8&#39;)]
zbigarray = out_data_array.getArray()
if zbigarray is None:
&nbsp; zbigarray = out_data_array.initArray(shape=(0,), dtype=dtype)
&nbsp; zbigarray.append(ndarray)
else:
&nbsp; start_index = len(zbigarray)
&nbsp; array_len = &nbsp;len(ndarray)
&nbsp; if start_index &lt; array_len:
&nbsp; &nbsp; zbigarray.append(ndarray[start_index:])
</code>
</pre>
<details open="">
<p>This script takes as an input data from resampled&nbsp;Data Array, splits data into training and tests set.&nbsp;</p>
<p>Then it learns on training set and predicts air pressure based on humidity.&nbsp;</p>
<p>In output array it stored the true and predicted values.&nbsp;</p>
<p>Don&#39;t forget to save the changes after editing the script.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=portal_callables')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=portal_callables')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Python Script</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_id"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_id"]</td>
<td>DataAnalysisLine_predictPressureFromHumidity</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>DataAnalysisLine_predictPressureFromHumidity</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>DataAnalysisLine_predictPressureFromHumidity</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_parameter_signature"]</td>
<td>in_array={}, out_array=[]</td>
</tr>
<tr>
<td>pause</td>
<td>15000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//div[contains(@data-gadget-url,'gadget_editor.html')]//textarea</td>
<td>import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
def predict(zbigarray):
&nbsp; humidity = zbigarray[:][&quot;humidity_mean&quot;]
&nbsp; pressure = zbigarray[:][&quot;pressure_mean&quot;]
&nbsp; humidity = humidity[humidity !=0 ]
&nbsp; pressure = pressure[pressure !=0 ]
&nbsp; humid_train, humid_test, pressure_train, pressure_test = train_test_split(humidity.reshape(-1,1), pressure.reshape(-1,1), test_size=0.2, random_state=0)
&nbsp; regressor = LinearRegression()
&nbsp; regressor.fit(humid_train, pressure_train) #training the algorithm
&nbsp; pressure_pred = regressor.predict( zbigarray[:][&quot;humidity_mean&quot;].reshape(-1,1))
&nbsp; df = pd.DataFrame({&#39;Date&#39;: zbigarray[:][&quot;date&quot;],&#39;Actual&#39;:zbigarray[:][&quot;pressure_mean&quot;], &#39;Predicted&#39;: pressure_pred.flatten()})
&nbsp; return &nbsp;df
out_data_array = out_array[&quot;Data Array&quot;]
in_data_array = in_array[&quot;Data Array&quot;]
in_zbigarray = in_data_array.getArray()
if in_zbigarray is None:
&nbsp; return
if in_zbigarray.shape[0] == 0:
&nbsp; return
df = predict(in_zbigarray)
ndarray = df.to_records(convert_datetime64=False)
dtype = [(&#39;Date&#39;, &#39;&lt;M8[ns]&#39;),(&#39;Actual&#39;, &#39;&lt;f8&#39;), (&#39;Predicted&#39;, &#39;&lt;f8&#39;)]
zbigarray = out_data_array.getArray()
if zbigarray is None:
&nbsp; zbigarray = out_data_array.initArray(shape=(0,), dtype=dtype)
&nbsp; zbigarray.append(ndarray)
else:
&nbsp; start_index = len(zbigarray)
&nbsp; array_len = &nbsp;len(ndarray)
&nbsp; if start_index &lt; array_len:
&nbsp; &nbsp; zbigarray.append(ndarray[start_index:])
</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformations</h1>
<img alt="" src="wendelin-Data.Transformation.7?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After your transformation script is ready, it&#39;s time to create Data Transformation.</p>
<p>On Modules page click on Data Transformations.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Data Transformation</h1>
<img alt="" src="wendelin-Data.Prediction.10?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Add to add a new Data Transformation.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Data Transformation</h1>
<img alt="" src="wendelin-Data.Transformation.9?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Create Document to continue.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Prediction.11?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Chose descriptive title and reference. For example</p>
<p>Title : <strong>Predict Pressure</strong></p>
<p>Reference : <strong>predict-environment-pressure-data</strong></p>
<p>For Initial Product we choose the Data Product we created in&nbsp;&nbsp;<a href="wendelin-Tutorial.Resample.Data">HowTo Resample Data</a>&nbsp;tutorial, as we want to do the prediction on the resampled data.</p>
<p>At the end click Save to save the changes</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Prediction.11_2?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After filling in the Initial Product and saving, values will appear in <strong>Initial Product Variation</strong> field.</p>
<p>Choose <strong>Resolution/60 S</strong> as we want to use 60 seconds resampled data.</p>
<p>At the end click Save to save the changes</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_transformation_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_transformation_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Predict Pressure</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>predict-environment-pressure-data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Generic Interval Resampled Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_my_variation_category_list"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_category_list"]</td>
<td>Resolution/60 S</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation Lines</h1>
<img alt="" src="wendelin-Data.Prediction.12?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on add button to add a Data Transformation Line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Operation Line</h1>
<img alt="" src="wendelin-Data.Transformation.12?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>&nbsp;</p>
<p>Choose <strong>Data Transformation Operation Line&nbsp;</strong>for Document Type and click on Create Document.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Prediction.13?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form with followig values</p>
<p>Title: <strong>Predict Pressure</strong></p>
<p>reference: <strong>data_operation</strong></p>
<p>Data Operation: put the name of the Data operation that we created at the beginning of this tutorial - <strong>Predict Pressure</strong></p>
<p>Trade Phase: <strong>Data/Convert</strong></p>
<p>Quantity: <strong>1</strong></p>
<p>Quantity Unit: <strong>Unit/Piece</strong></p>
<p>At the end click on Save to save the changes.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Prediction.14?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After saving the changes a new field <strong>Item Types</strong> will appear.</p>
<p>Choose <strong>Data Acquisition Unit</strong> for Item Types and save.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Predict Pressure</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>data_operation</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Predict Pressure</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_operation_module/')]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Convert</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_my_aggregated_portal_type_list"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Acquisition Unit</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Operation Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Operation Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Back To Data Transformation</h1>
<img alt="" src="wendelin-Data.Prediction.15?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Once needed fields are filled and saved go back to Data Transformations view by clicking on the upper panel.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Transformation Lines</h1>
<img alt="" src="wendelin-Data.Prediction.16?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Add another Data Transformation Line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Transformation Line</h1>
<img alt="" src="wendelin-Data.Prediction.17?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Now for Document Type choose <strong>Data Transformation Resource Line</strong> and click on <strong>Create Document</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Prediction.18?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form with following values</p>
<p>Title: <strong>Resampled Data</strong></p>
<p>Reference: <strong>in_array</strong></p>
<p>Data Product: put the name of the Data Product that we created in <a href="wendelin-Tutorial.Resample.Data">HowTo Resample Data</a>&nbsp;tutorial - <strong>Generic Interval Resampled Array</strong></p>
<p>After filling in the Data Product name click Save to save the intermediate changes.&nbsp;</p>
<p>After saving a new field <strong>Item Types&nbsp;</strong>will appear.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Prediction.19?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Continue filling the form as shown on the screenshot</p>
<p>Item Types: <strong>Data Array</strong></p>
<p>Trade Phase: <strong>Data/Convert</strong></p>
<p>Default Variation: <strong>Resolution/60 S</strong></p>
<p>Quantity: <strong>-1</strong></p>
<p>Quantity Unit: <strong>Unit/Piece</strong></p>
<p>At the end don&#39;t forget to save the changes.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Resampled Data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>in_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Generic Interval Resampled Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>-1</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Convert</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_variation_category_list"]</td>
<td>Resolution/60 S</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Add Output Line</h1>
<img alt="" src="wendelin-Data.Prediction.20?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Head back to Data Transformation and add one more Transformation line.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Transformation Line</h1>
<img alt="" src="wendelin-Data.Prediction.17?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Again for Document Type choose <strong>Data Transformation Resource Line</strong> and click on <strong>Create Document</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form</h1>
<img alt="" src="wendelin-Data.Prediction.21?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill the form with following values</p>
<p>Title: <strong>Predicted Data</strong></p>
<p>Reference: <strong>out_array</strong></p>
<p>Data Product: put the name of the Data Product that we created earlier in this tutorial - <strong>Predicted Data Array</strong>.</p>
<p>Then click Save to save the intermediate changes.&nbsp;</p>
<p>After saving a new field <strong>Item Types&nbsp;</strong>will appear.&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Fill The Form Cont.</h1>
<img alt="" src="wendelin-Data.Prediction.22?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Continue filling the form as shown on the screenshot</p>
<p>Item Types: <strong>Data Array</strong></p>
<p>Trade Phase: <strong>Data/Convert</strong></p>
<p>Use: <strong>Big Data/Ingestion/Stream Ingestion</strong></p>
<p>Quantity: <strong>1</strong></p>
<p>Quantity Unit: <strong>Unit/Piece</strong></p>
<p>At the end don&#39;t forget to save the changes.</p>
<p>&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//select[@id="field_your_select_action"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_your_select_action"]</td>
<td>Data Transformation Resource Line</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Predicted Data</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_reference"]</td>
<td>out_array</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_resource_title"]</td>
<td>Predicted Data Array</td>
</tr>
<tr>
<td>fireEvent</td>
<td>//input[@name='field_my_resource_title']</td>
<td>input</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//li[contains(@data-relative-url,'data_product_module/')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_quantity_unit"]</td>
<td>Unit/Piece</td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_quantity"]</td>
<td>1</td>
</tr>
<tr>
<td>select</td>
<td>//div[@data-gadget-scope="field_my_aggregated_portal_type_list"]//div//div//div[1]//div//select[@id="field_my_aggregated_portal_type_list"]</td>
<td>Data Array</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_trade_phase"]</td>
<td>Data/Convert</td>
</tr>
<tr>
<td>select</td>
<td>//select[@id="field_my_use"]</td>
<td>Big Data/Ingestion/Stream Ingestion</td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@data-i18n,'Data Transformation Resource Line')]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Transformation</h1>
<img alt="" src="wendelin-Data.Prediction.23?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Head back to Data Transformation.</p>
<p>Now you can see that we have 3 Data Transformation Lines.</p>
<p>The first line defines what operation will be done.&nbsp;</p>
<p>The second line defines on what the operation will be done - the input.</p>
<p>The last line defines the output.</p>
<p>The very&nbsp;last step is to validate the&nbsp;Data Transformation.&nbsp;</p>
<p>Click on Validate on the left side panel to validate the Data Transformation.</p>
</details>
</section>
<section class="screenshot">
<h1>Confirm Validation</h1>
<img alt="" src="wendelin-Data.Prediction.24?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on Validate to confirm the Validation.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[1]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[2]</td>
<td></td>
</tr>
<tr>
<td>verifyElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr[3]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'Dvalidate_action')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Analysis</h1>
<img alt="" src="wendelin-Data.Transformation.25?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Go to <strong>Data Analyses </strong>module<strong>.</strong></p>
</details>
</section>
<section class="screenshot">
<h1>Data Analysis</h1>
<img alt="" src="wendelin-Data.Prediction.25?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After few minutes in Data Analysis Module a new Data Analysis called&nbsp;<strong>Predict Pressure&nbsp;</strong>will appear.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>350000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_analysis_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_analysis_module')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr//td[1]</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Predict Pressure</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Data Array Module</h1>
<img alt="" src="wendelin-Data.Transformation.27?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Go to Data Array Module to see the data.&nbsp;&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Prediction.26?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Here we can see the newly created Data Arrays with prediction results - <strong>Predict Pressure</strong>.</p>
<p>Click on it to navigate to the array.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Prediction.27?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Preview</strong> to see the data.</p>
<p>Note it might take few minutes (&lt;10) until data appears.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Array</h1>
<img alt="" src="wendelin-Data.Prediction.28?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Data is here!</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>350000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'data_array_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'data_array_module')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[@class='document_table']//table//tbody//tr//td[1]</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Predict Pressure</td>
<td></td>
</tr>
<tr>
<td>assertTextNotPresent</td>
<td>None</td>
<td></td>
</tr>
<!--<tr>
<td>click</td>
<td>//div[@class='document_table']//table//tbody//tr//td[2]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>200000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'preview')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'preview')]</td>
<td></td>
</tr>-->
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Notebook Module</h1>
<img alt="" src="wendelin-Data.Visualisation.1?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>On Modules page click on <strong>Notebooks</strong>.</p>
</details>
</section>
<section class="screenshot">
<h1>Add Notebook</h1>
<img alt="" src="wendelin-Data.Prediction.29?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Add</strong> to add a new Notebook.</p>
</details>
</section>
<section class="screenshot">
<h1>Create Notebook</h1>
<img alt="" src="wendelin-Data.Visualisation.3?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Click on <strong>Create Document</strong> to continue.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'page=front')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[contains(@href,'key=notebook_module')]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[contains(@href,'key=notebook_module')]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//a[@data-i18n="Add"]</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@type="submit" and @data-i18n="Proceed"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
<section class="screenshot">
<h1>Notebook</h1>
<img alt="" src="wendelin-Data.Prediction.30?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Fill in the Title.</p>
<p>The visualisation code will be at the bottom text area.</p>
<p>Notebook uses the same technology as&nbsp;<a href="https://alpha.iodide.io/">Iodide</a>.&nbsp;</p>
<p>To learn more about programming in&nbsp;Iodide environment checkout&nbsp;<a href="https://alpha.iodide.io/">Iodide</a>&nbsp;homepage.</p>
</details>
</section>
<section class="screenshot">
<h1>Visualisation Script</h1>
<pre>
<code>
%% md
# Prediction
&lt;div class=&quot;prediction_plot_div&quot; id=&quot;prediction_plot_div&quot;/&gt;
%% fetch
js: jio.js
js: ndarray_bundle.js
js: wendelin.js
js: https://cdn.plot.ly/plotly-latest.min.js
%% js
hateoas_url = &quot;https://softinst133633.host.vifib.net/erp5/web_site_module/renderjs_runner/hateoas/&quot;;
jio = jIO.createJIO({
&nbsp; type: &quot;erp5&quot;,
&nbsp; url: hateoas_url,
&nbsp; default_view_reference: &quot;view&quot;
});
gadget = {
&nbsp; getSetting: function(property) {
&nbsp; &nbsp; return new RSVP.Queue()
&nbsp; &nbsp; &nbsp; .push(function () {
&nbsp; &nbsp; &nbsp; &nbsp; if (property == &quot;hateoas_url&quot;) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return hateoas_url;
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; return;
&nbsp; &nbsp; &nbsp; });
&nbsp; },
&nbsp; jio_getAttachment: function(id, url, parameter_dict) {
&nbsp; &nbsp; return jio.getAttachment(id, url, parameter_dict);
&nbsp; },
&nbsp; jio_get: function(id) {return jio.get(id);}
}
var prediction_label_list = [&quot;Date&quot;, &quot;Actual&quot;, &quot;Predicted&quot;];
var prediction_graph = document.getElementById(&#39;prediction_plot_div&#39;);
plot_prediction();
function plot_prediction() {
&nbsp; return getPredictionData()
&nbsp; &nbsp; .push(function (data) {
&nbsp; &nbsp; &nbsp; console.log(&quot;data&quot;)
&nbsp; &nbsp; &nbsp; console.log(data)
&nbsp; &nbsp; &nbsp; var layout = {barmode: &#39;stack&#39;,&#39;title&#39; :&#39;Predict Presure Based on Humidity&#39;};
&nbsp; &nbsp; &nbsp; Plotly.plot(prediction_graph,data, layout);
&nbsp; &nbsp; });
}
function getPredictionData(start_date, stop_date){
&nbsp; function unpack(rows, key) {
&nbsp; &nbsp; return rows.map(function(row) { return row[key]; });
&nbsp; }
&nbsp; array_id = &quot;data_array_module/90&quot;;
&nbsp; prediction_graph_data=[];
&nbsp; var start_index = 0;
&nbsp; var stop_index = undefined;
&nbsp; return jio.allDocs({
&nbsp; &nbsp; query: &#39;portal_type:&quot;Data Analysis Line&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;title: &quot;Predicted Data&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;resource_reference:&quot;environment-predicted-array&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;simulation_state:&quot;started&quot;&#39;
&nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; var data_analysis_line_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return jio.allDocs({
&nbsp; &nbsp; &nbsp; &nbsp; query: &#39;portal_type:&quot;Data Array&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;aggregate_related_relative_url:&quot;&#39; + data_analysis_line_id +&#39;&quot;&#39;
&nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, 0, 1);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_start_date = wendelin.convertFirstColToDate([[result.data[0]]])[0][0];
&nbsp; &nbsp; &nbsp; if (start_index === undefined) {
&nbsp; &nbsp; &nbsp; &nbsp; start_index = Math.max(0, Math.ceil((start_date - array_start_date) / (frequency*1000))),
&nbsp; &nbsp; &nbsp; &nbsp; stop_index = Math.ceil((stop_date - array_start_date) / (frequency*1000));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, start_index, stop_index);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result) {
&nbsp; &nbsp; &nbsp; for (i = 0; i &lt; prediction_label_list.length; i += 1) {
&nbsp; &nbsp; &nbsp; &nbsp; prediction_graph_data = prediction_graph_data.concat(nj.unpack(result.pick( null, prediction_label_list[i])));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return prediction_graph_data
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result){
&nbsp; &nbsp; &nbsp; var filtered_graph_data = [];
&nbsp; &nbsp; &nbsp; for (var i=0; i&lt;result.length; i++) {
&nbsp; &nbsp; &nbsp; &nbsp; if (result[i][0] != 0) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; filtered_graph_data.push(result[i]);
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.convertFirstColToDate(filtered_graph_data)
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (graph_data) {
&nbsp; &nbsp; &nbsp; var actual_data = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Real&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, prediction_label_list.indexOf(&quot;Date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, prediction_label_list.indexOf(&quot;Actual&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#17BECF&#39;}
&nbsp; &nbsp; &nbsp; };
&nbsp; &nbsp; &nbsp; var predicted_data = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Predicted&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, prediction_label_list.indexOf(&quot;Date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, prediction_label_list.indexOf(&quot;Predicted&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#7F7F7F&#39;}
&nbsp; &nbsp; &nbsp; };
&nbsp; &nbsp; &nbsp; var data = [actual_data,predicted_data];
&nbsp; &nbsp; &nbsp; return data
&nbsp; &nbsp; });
}</code>
</pre>
<details open="">
<p>Copy/paste the code to your notebook and save the changes.&nbsp;</p>
<p><strong>Note:&nbsp;</strong>you will need to provide the URL of your instance in the script.&nbsp;</p>
<p>For that change the line :</p>
<p>&nbsp;</p>
<pre>
<code>hateoas_url = &quot;https://softinstXXX.host.vifib.net/erp5/web_site_module/renderjs_runner/hateoas/&quot;;
</code></pre>
<p>&nbsp;</p>
<p>The script will find automatically&nbsp;the Data Array with prediction results based on the Data Product&nbsp;and plot the data using&nbsp;<a href="https://plotly.com/">plotly library</a>.</p>
</details>
</section>
<section class="screenshot">
<h1>Notebook</h1>
<img alt="" src="wendelin-Data.Prediction.31?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>After writing the script and saving the changes click on the <strong>Editable</strong> checkbox to (uncheck it) on the left side panel to see the result.</p>
</details>
</section>
<section class="screenshot">
<h1>Data Visualisation</h1>
<img alt="" src="wendelin-Data.Prediction.32?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>If everything was done correctly you will see the plot of your data.</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>waitForElementPresent</td>
<td>//input[@id="field_my_title"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@id="field_my_title"]</td>
<td>Predicted Data</td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//div[contains(@class,'CodeMirror')]//div//textarea</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//div[contains(@class,'CodeMirror')]//div//textarea</td>
<td>
%% md
# Prediction
&lt;div class=&quot;prediction_plot_div&quot; id=&quot;prediction_plot_div&quot;/&gt;
%% fetch
js: jio.js
js: ndarray_bundle.js
js: wendelin.js
js: https://cdn.plot.ly/plotly-latest.min.js
%% js
hateoas_url = &quot;${base_url_x}/web_site_module/renderjs_runner/hateoas/&quot;;
jio = jIO.createJIO({
&nbsp; type: &quot;erp5&quot;,
&nbsp; url: hateoas_url,
&nbsp; default_view_reference: &quot;view&quot;
});
gadget = {
&nbsp; getSetting: function(property) {
&nbsp; &nbsp; return new RSVP.Queue()
&nbsp; &nbsp; &nbsp; .push(function () {
&nbsp; &nbsp; &nbsp; &nbsp; if (property == &quot;hateoas_url&quot;) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return hateoas_url;
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; return;
&nbsp; &nbsp; &nbsp; });
&nbsp; },
&nbsp; jio_getAttachment: function(id, url, parameter_dict) {
&nbsp; &nbsp; return jio.getAttachment(id, url, parameter_dict);
&nbsp; },
&nbsp; jio_get: function(id) {return jio.get(id);}
}
var prediction_label_list = [&quot;Date&quot;, &quot;Actual&quot;, &quot;Predicted&quot;];
var prediction_graph = document.getElementById(&#39;prediction_plot_div&#39;);
plot_prediction();
function plot_prediction() {
&nbsp; return getPredictionData()
&nbsp; &nbsp; .push(function (data) {
&nbsp; &nbsp; &nbsp; console.log(&quot;data&quot;)
&nbsp; &nbsp; &nbsp; console.log(data)
&nbsp; &nbsp; &nbsp; var layout = {barmode: &#39;stack&#39;,&#39;title&#39; :&#39;Predict Presure Based on Humidity&#39;};
&nbsp; &nbsp; &nbsp; Plotly.plot(prediction_graph,data, layout);
&nbsp; &nbsp; });
}
function getPredictionData(start_date, stop_date){
&nbsp; function unpack(rows, key) {
&nbsp; &nbsp; return rows.map(function(row) { return row[key]; });
&nbsp; }
&nbsp; array_id = &quot;data_array_module/90&quot;;
&nbsp; prediction_graph_data=[];
&nbsp; var start_index = 0;
&nbsp; var stop_index = undefined;
&nbsp; return jio.allDocs({
&nbsp; &nbsp; query: &#39;portal_type:&quot;Data Analysis Line&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;title: &quot;Predicted Data&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;resource_reference:&quot;environment-predicted-array&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;simulation_state:&quot;started&quot;&#39;
&nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; var data_analysis_line_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return jio.allDocs({
&nbsp; &nbsp; &nbsp; &nbsp; query: &#39;portal_type:&quot;Data Array&quot; AND &#39; +
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&#39;aggregate_related_relative_url:&quot;&#39; + data_analysis_line_id +&#39;&quot;&#39;
&nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_id = result.data.rows[0].id;
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, 0, 1);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (result) {
&nbsp; &nbsp; &nbsp; array_start_date = wendelin.convertFirstColToDate([[result.data[0]]])[0][0];
&nbsp; &nbsp; &nbsp; if (start_index === undefined) {
&nbsp; &nbsp; &nbsp; &nbsp; start_index = Math.max(0, Math.ceil((start_date - array_start_date) / (frequency*1000))),
&nbsp; &nbsp; &nbsp; &nbsp; stop_index = Math.ceil((stop_date - array_start_date) / (frequency*1000));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.getArrayRawSlice(gadget, array_id, start_index, stop_index);
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result) {
&nbsp; &nbsp; &nbsp; for (i = 0; i &lt; prediction_label_list.length; i += 1) {
&nbsp; &nbsp; &nbsp; &nbsp; prediction_graph_data = prediction_graph_data.concat(nj.unpack(result.pick( null, prediction_label_list[i])));
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return prediction_graph_data
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function(result){
&nbsp; &nbsp; &nbsp; var filtered_graph_data = [];
&nbsp; &nbsp; &nbsp; for (var i=0; i&lt;result.length; i++) {
&nbsp; &nbsp; &nbsp; &nbsp; if (result[i][0] != 0) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; filtered_graph_data.push(result[i]);
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; return wendelin.convertFirstColToDate(filtered_graph_data)
&nbsp; &nbsp; })
&nbsp; &nbsp; .push(function (graph_data) {
&nbsp; &nbsp; &nbsp; var actual_data = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Real&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, prediction_label_list.indexOf(&quot;Date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, prediction_label_list.indexOf(&quot;Actual&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#17BECF&#39;}
&nbsp; &nbsp; &nbsp; };
&nbsp; &nbsp; &nbsp; var predicted_data = {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type: &quot;scattergl&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mode: &quot;markers&quot;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name: &#39;Predicted&#39;,
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; x: unpack(graph_data, prediction_label_list.indexOf(&quot;Date&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; y: unpack(graph_data, prediction_label_list.indexOf(&quot;Predicted&quot;)),
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; line: {color: &#39;#7F7F7F&#39;}
&nbsp; &nbsp; &nbsp; };
&nbsp; &nbsp; &nbsp; var data = [actual_data,predicted_data];
&nbsp; &nbsp; &nbsp; return data
&nbsp; &nbsp; });
}
</td>
</tr>
<tr>
<td>pause</td>
<td>5000</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//button[@name="submit" and @data-i18n="Save"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//button[@type="submit" and @class="success"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//input[@id="editable" and @type="checkbox"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>20000</td>
<td></td>
</tr>
<tr>
<td>assertElementPresent</td>
<td>//div[@id="prediction_plot_div"]</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_08_wendelin_HowTo_Create_Basic_Prediction_with_scikit_learn.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-Tutorial.Create.Basic.Prediction.scikit-learn</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Create Basic Prediction with scikit-learn</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_08_wendelin_HowTo_Create_Basic_Prediction_with_scikit_learn</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Create Basic Prediction</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Create Basic Prediction with scikit-learn</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_09_wendelin_HowTo_Enable_User_Regstration.html 0 → 100644
View file @ bce74d9d
<section>
<!--<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/init_test_environment"&gt;init&lt;/span&gt;</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/Zuite_setUpPersonTest</td>
<td></td>
</tr>
<tr>
<td>verifyTextPresent</td>
<td>Init Ok</td>
<td></td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/wait_for_activities"&gt; Wait for activities &lt;/span&gt;</td>
</tr>
<tr class="" style="opacity: 1;">
<td colspan="3">&lt;span metal:use-macro="container/Zuite_viewTestMacroLibrary/macros/login_as_functional_test_user"&gt;Login As Functional Test User&lt;/span&gt;</td>
</tr>
</tbody>
</table>
</test>-->
<h1>Agenda</h1>
<ul>
<li>Configure Mailhost</li>
</ul>
<details open="">
<p>&nbsp;</p>
<p>This tutorial teaches how to configure wendelin to allow user registration.</p>
<p><strong>Note</strong>: This feature is enabled only if wendelin configuration the&nbsp;<strong>Setup default data lake security model&nbsp;</strong>was checked.&nbsp;</p>
<p>&nbsp;</p>
</details>
</section>
<section class="screenshot">
<h1>Configure Mailhost</h1>
<img alt="" src="wendelin-configureMailhost.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Open <strong>https://YOUR_INSTANCE_URL/erp5/MailHost/manage_main</strong> to access the mail host configuration in the Zope interface (Wendelin is built on top of Zope).</p>
<p>Add a SMTP mailhost and <strong>login/password</strong> for the alias under which an email should be forwarded/sent.</p>
<p>Make sure the <strong>Port</strong> is also correctly set to your email provider&#39;s correct SMTP port.</p>
<p>Click <strong>Save</strong>&nbsp;<strong>Changes</strong> when done to continue.</p>
</details>
</section>
<section class="screenshot">
<h1>Register</h1>
<img alt="" src="wendelin-register.user.screenshot?format=" style="font-size: 13px;" title="" type="image/svg+xml" />
<details open="">
<p>Once the mailhost is configured users can register.</p>
<p>Open <strong>https://YOUR_INSTANCE_URL/erp5/web_site_module/default_wendelin_data_lake/</strong> and on left side panel click on <strong>Register.</strong></p>
<p><b>Note:&nbsp;</b>Pay attention on <b>/&nbsp;</b>(slash) at the end of the url, it is mandatory.&nbsp;</p>
<p><b>Note: </b>Registration works only if user is not logged in. Thus,&nbsp;before registering a new user make sure that you are not logged in with any user.&nbsp;</p>
<p>After filling the form and clicking on Register you will get an email to the specified email address with further instructions.&nbsp;</p>
</details>
<test>
<table style="display: none;" class="test">
<tbody>
<tr>
<td>openAndWait</td>
<td>${base_url}/logout</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/erp5/web_site_module/default_wendelin_data_lake/#/?page=register</td>
<td></td>
</tr>
<tr>
<td>waitForElementPresent</td>
<td>//input[@name="your_first_name"]</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="your_first_name"]</td>
<td>TestFirstName</td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="your_last_name"]</td>
<td>TestLastName</td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="your_email"]</td>
<td>test@example.com</td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="your_reference"]</td>
<td>TestReference</td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="your_password"]</td>
<td>TestPassword</td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="your_confirm_password"]</td>
<td>TestPassword</td>
</tr>
<tr>
<td>click</td>
<td>//input[@type="submit" and @value="Register"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>10000</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>Registration submitted.</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/login_form</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="__ac_name"]</td>
<td>functional_test_username</td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="__ac_password"]</td>
<td>secret</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>//input[@name="logged_in:method"]</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/erp5/view</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>20000</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/erp5/portal_activities/</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//table//tbody//tr[1]//td[2]//a</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>2000</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/erp5/event_module/</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//th[@class="listbox-table-header-cell"][4]//button[@type="submit" and @title="Ascending Display"]</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//td[@class="listbox-table-data-cell"]//a[contains(@href,'/event_module/')]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>10000</td>
<td></td>
</tr>
<tr>
<td>store</td>
<td>javascript{
link = String(selenium.browserbot.getCurrentWindow().document.getElementsByTagName("pre")[0].innerText).match("(>https.*<)")[0];
link.substring(1, link.length-1);
}</td>
<td>welcomelink</td>
</tr>
<tr>
<td>openAndWait</td>
<td>${welcomelink}</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>activated</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>40000</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/erp5/portal_activities/</td>
<td></td>
</tr>
<tr>
<td>click</td>
<td>//table//tbody//tr[1]//td[2]//a</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>2000</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/logout</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/login_form</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="__ac_name"]</td>
<td>TestReference</td>
</tr>
<tr>
<td>type</td>
<td>//input[@name="__ac_password"]</td>
<td>TestPassword</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>//input[@name="logged_in:method"]</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>3000</td>
<td></td>
</tr>
<tr>
<td>openAndWait</td>
<td>${base_url}/erp5/view</td>
<td></td>
</tr>
<tr>
<td>assertTextPresent</td>
<td>TestReference</td>
<td></td>
</tr>
</tbody>
</table>
</test>
</section>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/test_page_module/test_09_wendelin_HowTo_Enable_User_Regstration.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>contributor/person_module/functional_test_username</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>wendelin-HowTo.Enable.User.Regstration</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>HowTo Enable User Registration</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test_09_wendelin_HowTo_Enable_User_Regstration</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value> <string>HowTo Enable User Registration</string> </value>
</item>
<item>
<key> <string>subject</string> </key>
<value>
<tuple>
<string>Wendelin</string>
<string>installation</string>
<string>setup</string>
</tuple>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Wendelin - HowTo Enable User Registration</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value>
<none/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/web_page_module/rjs_gadget_ndarray_html.html 0 → 100644
View file @ bce74d9d
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, user-scalable=no" />
<title>Ndarray Gadget</title>
<!-- renderjs -->
<script src="rsvp.js" type="text/javascript"></script>
<script src="renderjs.js" type="text/javascript"></script>
<!-- custom script -->
<script src="ndarray_bundle.js" type="text/javascript"></script>
<script src="gadget_ndarray.js" type="text/javascript"></script>
</head>
<body>
</body>
</html>
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/web_page_module/rjs_gadget_ndarray_html.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Web Page" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>content_type</string> </key>
<value> <string>text/html</string> </value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>gadget_ndarray.html</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>rjs_gadget_ndarray_html</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value> <string>en</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Web Page</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Gadget Ndarray</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>001</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>document_publication_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
<item>
<key> <string>edit_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
<item>
<key> <string>processing_status_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAU=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>publish_alive</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846204.18</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>published_alive</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>edit</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>serial</string> </key>
<value> <string>951.55792.50694.62344</string> </value>
</item>
<item>
<key> <string>state</string> </key>
<value> <string>current</string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846140.49</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="5" aka="AAAAAAAAAAU=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>detect_converted_file</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>external_processing_state</string> </key>
<value> <string>converted</string> </value>
</item>
<item>
<key> <string>serial</string> </key>
<value> <string>0.0.0.0</string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846051.71</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/web_page_module/rjs_gadget_ndarray_js.js 0 → 100644
View file @ bce74d9d
/*global window, rJS, nj */
/*jslint indent: 2, maxerr: 3 */
(function (window, rJS, nj) {
"use strict";
// parse dict in string in python pprint.pfromat
function parserPythonPformatDict(s) {
s = s
.replace(/False/g, "false")
.replace(/True/g, "true")
.replace(/\(/g, "[")
.replace(/\)/g, "]")
.replace(/, }/g, "}")
.replace(/\'/g, '"')
return JSON.parse(s);
}
// Initial idea from https://github.com/BillMills/numpychuck/blob/master/helpers.js
// https://github.com/numpy/numpy/blob/master/doc/neps/npy-format.rst#format-specification-version-10
function unpackNPY(dv) {
return new RSVP.Queue()
.push(function () {
var HEADER_LEN = dv.getUint16(8, true),
offset = HEADER_LEN + 10,
dataLength = dv.byteLength - offset,
i,
header_string = '',
char,
header_dict,
value_length,
getValue,
shape = [],
array = [],
value;
for (i = 10; i <= offset; i ++) {
char = String.fromCharCode(dv.getUint8(i, true));
header_string += char;
if (char == '}') {
break;
}
}
header_dict = parserPythonPformatDict(header_string);
return unpackArray(dv, header_dict, offset);
});
}
// Initial idea from https://github.com/BillMills/numpychuck/blob/master/helpers.js
// https://github.com/numpy/numpy/blob/master/doc/neps/npy-format.rst#format-specification-version-10
function unpackArray(dv, header_dict, offset) {
var i,
array = [];
if (offset === undefined) {
offset = 0;
}
return new RSVP.Queue()
.push(function () {
if ((header_dict.descr === "<i8")
& (header_dict.fortran_order == false)) {
for (i = 0; i < dv.byteLength - offset - 1; i += 8) {
array.push(dv.getUint8(i + offset, true))
}
} else if (header_dict.descr == "<f8"
& (header_dict.fortran_order == false)) {
for (i = 0; i < dv.byteLength - offset - 1; i += 8) {
array.push(dv.getFloat64(i + offset, true))
}
} else {
throw("Unsupported dtype: " + header_dict.descr +
" with fortran_order = " + header_dict.fortran_order);
}
return nj.ndarray(array, header_dict.shape);
});
}
rJS(window)
.ready(function (gadget) {
})
.declareMethod('unpackNPY', function () {
return unpackNPY(arguments[0]);
})
.declareMethod('unpackArray', function () {
return unpackArray(arguments[0], arguments[1]);
})
}(window, rJS, nj));
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/web_page_module/rjs_gadget_ndarray_js.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Web Script" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>gadget_ndarray.js</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>rjs_gadget_ndarray_js</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value> <string>en</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Web Script</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Gadget Ndarray JS</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>001</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>document_publication_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
<item>
<key> <string>edit_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
<item>
<key> <string>processing_status_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAU=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>publish_alive</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846295.48</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>published_alive</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>edit</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>serial</string> </key>
<value> <string>951.55794.42149.14677</string> </value>
</item>
<item>
<key> <string>state</string> </key>
<value> <string>current</string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846268.46</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="5" aka="AAAAAAAAAAU=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>detect_converted_file</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>external_processing_state</string> </key>
<value> <string>converted</string> </value>
</item>
<item>
<key> <string>serial</string> </key>
<value> <string>0.0.0.0</string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846223.69</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/web_page_module/rjs_ndarray_bundle_js.js 0 → 100644
View file @ bce74d9d
This source diff could not be displayed because it is too large. You can view the blob instead.
bt5/erp5_wendelin_tutorial_ui_test/PathTemplateItem/web_page_module/rjs_ndarray_bundle_js.xml 0 → 100644
View file @ bce74d9d
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Web Script" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Add_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Change_local_roles_Permission</string> </key>
<value>
<tuple>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_Modify_portal_content_Permission</string> </key>
<value>
<tuple>
<string>Assignee</string>
<string>Assignor</string>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>_View_Permission</string> </key>
<value>
<tuple>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_md5</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>ndarray_bundle.js</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>rjs_ndarray_bundle_js</string> </value>
</item>
<item>
<key> <string>language</string> </key>
<value> <string>en</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Web Script</string> </value>
</item>
<item>
<key> <string>short_title</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Ndarray Bundle JS</string> </value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>001</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>document_publication_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
<item>
<key> <string>edit_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
<item>
<key> <string>processing_status_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAU=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>publish_alive</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846436.13</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>published_alive</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>edit</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>serial</string> </key>
<value> <string>951.55796.57580.45926</string> </value>
</item>
<item>
<key> <string>state</string> </key>
<value> <string>current</string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846390.31</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="5" aka="AAAAAAAAAAU=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>detect_converted_file</string> </value>
</item>
<item>
<key> <string>actor</string> </key>
<value> <string>zope</string> </value>
</item>
<item>
<key> <string>comment</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>error_message</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>external_processing_state</string> </key>
<value> <string>converted</string> </value>
</item>
<item>
<key> <string>serial</string> </key>
<value> <string>0.0.0.0</string> </value>
</item>
<item>
<key> <string>time</string> </key>
<value>
<object>
<klass>
<global name="DateTime" module="DateTime.DateTime"/>
</klass>
<tuple>
<none/>
</tuple>
<state>
<tuple>
<float>1465846347.86</float>
<string>UTC</string>
</tuple>
</state>
</object>
</value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin_tutorial_ui_test/bt/description 0 → 100644
View file @ bce74d9d
Includes gadgets used in the Wendelin tutorial.
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/bt/template_keep_last_workflow_history_only_path_list 0 → 100644
View file @ bce74d9d
web_page_module/rjs_gadget_ndarray_html
web_page_module/rjs_gadget_ndarray_js
web_page_module/rjs_ndarray_bundle_js
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/bt/template_keep_workflow_path_list 0 → 100644
View file @ bce74d9d
web_page_module/rjs_gadget_ndarray_html
web_page_module/rjs_gadget_ndarray_js
web_page_module/rjs_ndarray_bundle_js
test_page_module/*wendelin*
data_stream_module/*
portal_skins/erp5_wendelin_data_lake/Ingestion_*
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/bt/template_path_list 0 → 100644
View file @ bce74d9d
data_stream_module/*
portal_skins/erp5_wendelin_data_lake/Ingestion_*
test_page_module/*wendelin*
web_page_module/rjs_gadget_ndarray_html
web_page_module/rjs_gadget_ndarray_js
web_page_module/rjs_ndarray_bundle_js
\ No newline at end of file
bt5/erp5_wendelin_tutorial_ui_test/bt/title 0 → 100644
View file @ bce74d9d
erp5_wendelin_tutorial_ui_test
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7