Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
W
wendelin
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Nikola Balog
wendelin
Commits
02214c63
Commit
02214c63
authored
Jun 30, 2015
by
Ivan Tyagov
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Add a generic implementation of transforming appended data in a "close to real time" way.
parent
a81baf75
Changes
12
Show whitespace changes
Inline
Side-by-side
Showing
12 changed files
with
402 additions
and
1 deletion
+402
-1
bt5/erp5_wendelin/PortalTypeWorkflowChainTemplateItem/workflow_chain_type.xml
...rtalTypeWorkflowChainTemplateItem/workflow_chain_type.xml
+1
-1
bt5/erp5_wendelin/SkinTemplateItem/portal_skins/erp5_wendelin/DataStream_transformTail.xml
...m/portal_skins/erp5_wendelin/DataStream_transformTail.xml
+78
-0
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow.xml
...Item/portal_workflow/data_stream_interaction_workflow.xml
+46
-0
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/interactions.xml
...orkflow/data_stream_interaction_workflow/interactions.xml
+28
-0
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/interactions/DataStream_appendData.xml
...teraction_workflow/interactions/DataStream_appendData.xml
+100
-0
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/scripts.xml
...tal_workflow/data_stream_interaction_workflow/scripts.xml
+28
-0
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/scripts/DataStream_appendData.xml
...am_interaction_workflow/scripts/DataStream_appendData.xml
+74
-0
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/variables.xml
...l_workflow/data_stream_interaction_workflow/variables.xml
+22
-0
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/worklists.xml
...l_workflow/data_stream_interaction_workflow/worklists.xml
+22
-0
bt5/erp5_wendelin/bt/description
bt5/erp5_wendelin/bt/description
+1
-0
bt5/erp5_wendelin/bt/template_portal_type_workflow_chain_list
...erp5_wendelin/bt/template_portal_type_workflow_chain_list
+1
-0
bt5/erp5_wendelin/bt/template_workflow_id_list
bt5/erp5_wendelin/bt/template_workflow_id_list
+1
-0
No files found.
bt5/erp5_wendelin/PortalTypeWorkflowChainTemplateItem/workflow_chain_type.xml
View file @
02214c63
...
@@ -49,7 +49,7 @@
...
@@ -49,7 +49,7 @@
</chain>
</chain>
<chain>
<chain>
<type>
Data Stream
</type>
<type>
Data Stream
</type>
<workflow>
edit_workflow, validation_workflow
</workflow>
<workflow>
data_stream_interaction_workflow,
edit_workflow, validation_workflow
</workflow>
</chain>
</chain>
<chain>
<chain>
<type>
Data Supply
</type>
<type>
Data Supply
</type>
...
...
bt5/erp5_wendelin/SkinTemplateItem/portal_skins/erp5_wendelin/DataStream_transformTail.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"PythonScript"
module=
"Products.PythonScripts.PythonScript"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
Script_magic
</string>
</key>
<value>
<int>
3
</int>
</value>
</item>
<item>
<key>
<string>
_bind_names
</string>
</key>
<value>
<object>
<klass>
<global
name=
"NameAssignments"
module=
"Shared.DC.Scripts.Bindings"
/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key>
<string>
_asgns
</string>
</key>
<value>
<dictionary>
<item>
<key>
<string>
name_container
</string>
</key>
<value>
<string>
container
</string>
</value>
</item>
<item>
<key>
<string>
name_context
</string>
</key>
<value>
<string>
context
</string>
</value>
</item>
<item>
<key>
<string>
name_m_self
</string>
</key>
<value>
<string>
script
</string>
</value>
</item>
<item>
<key>
<string>
name_subpath
</string>
</key>
<value>
<string>
traverse_subpath
</string>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key>
<string>
_body
</string>
</key>
<value>
<string>
"""\n
Read tail of a Data Stream and aplly needed transformations.\n
This script is called every time we appendData to a Stream\n
using data_stream_interaction_workflow.\n
\n
The idea is to provide close to real time data transformations.\n
As transformation is quite specific we leave this script empty so developers\n
can hook in and add needed transformations.\n
\n
Interesting here ti to use OffsetIndex property sheet which allows to place\n
marker till where a Data Stream has been processed.\n
"""\n
</string>
</value>
</item>
<item>
<key>
<string>
_params
</string>
</key>
<value>
<string></string>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
DataStream_transformTail
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"InteractionWorkflowDefinition"
module=
"Products.ERP5.InteractionWorkflow"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
_objects
</string>
</key>
<value>
<tuple/>
</value>
</item>
<item>
<key>
<string>
creation_guard
</string>
</key>
<value>
<none/>
</value>
</item>
<item>
<key>
<string>
description
</string>
</key>
<value>
<string>
Data Stream interaction workflow
</string>
</value>
</item>
<item>
<key>
<string>
groups
</string>
</key>
<value>
<tuple/>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
data_stream_interaction_workflow
</string>
</value>
</item>
<item>
<key>
<string>
manager_bypass
</string>
</key>
<value>
<int>
0
</int>
</value>
</item>
<item>
<key>
<string>
title
</string>
</key>
<value>
<string>
Data Stream interaction workflow
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/interactions.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"Interaction"
module=
"Products.ERP5.Interaction"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
_mapping
</string>
</key>
<value>
<dictionary/>
</value>
</item>
<item>
<key>
<string>
_objects
</string>
</key>
<value>
<tuple/>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
interactions
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/interactions/DataStream_appendData.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"InteractionDefinition"
module=
"Products.ERP5.Interaction"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
actbox_category
</string>
</key>
<value>
<string>
workflow
</string>
</value>
</item>
<item>
<key>
<string>
actbox_name
</string>
</key>
<value>
<string></string>
</value>
</item>
<item>
<key>
<string>
actbox_url
</string>
</key>
<value>
<string></string>
</value>
</item>
<item>
<key>
<string>
activate_script_name
</string>
</key>
<value>
<tuple/>
</value>
</item>
<item>
<key>
<string>
after_script_name
</string>
</key>
<value>
<tuple/>
</value>
</item>
<item>
<key>
<string>
before_commit_script_name
</string>
</key>
<value>
<list>
<string>
DataStream_appendData
</string>
</list>
</value>
</item>
<item>
<key>
<string>
description
</string>
</key>
<value>
<string></string>
</value>
</item>
<item>
<key>
<string>
guard
</string>
</key>
<value>
<none/>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
DataStream_appendData
</string>
</value>
</item>
<item>
<key>
<string>
method_id
</string>
</key>
<value>
<list>
<string>
_appendData
</string>
</list>
</value>
</item>
<item>
<key>
<string>
once_per_transaction
</string>
</key>
<value>
<int>
0
</int>
</value>
</item>
<item>
<key>
<string>
portal_type_filter
</string>
</key>
<value>
<none/>
</value>
</item>
<item>
<key>
<string>
portal_type_group_filter
</string>
</key>
<value>
<none/>
</value>
</item>
<item>
<key>
<string>
script_name
</string>
</key>
<value>
<tuple/>
</value>
</item>
<item>
<key>
<string>
temporary_document_disallowed
</string>
</key>
<value>
<int>
1
</int>
</value>
</item>
<item>
<key>
<string>
title
</string>
</key>
<value>
<string></string>
</value>
</item>
<item>
<key>
<string>
trigger_type
</string>
</key>
<value>
<int>
2
</int>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/scripts.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"Scripts"
module=
"Products.DCWorkflow.Scripts"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
_mapping
</string>
</key>
<value>
<dictionary/>
</value>
</item>
<item>
<key>
<string>
_objects
</string>
</key>
<value>
<tuple/>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
scripts
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/scripts/DataStream_appendData.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"PythonScript"
module=
"Products.PythonScripts.PythonScript"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
Script_magic
</string>
</key>
<value>
<int>
3
</int>
</value>
</item>
<item>
<key>
<string>
_bind_names
</string>
</key>
<value>
<object>
<klass>
<global
name=
"NameAssignments"
module=
"Shared.DC.Scripts.Bindings"
/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key>
<string>
_asgns
</string>
</key>
<value>
<dictionary>
<item>
<key>
<string>
name_container
</string>
</key>
<value>
<string>
container
</string>
</value>
</item>
<item>
<key>
<string>
name_context
</string>
</key>
<value>
<string>
context
</string>
</value>
</item>
<item>
<key>
<string>
name_m_self
</string>
</key>
<value>
<string>
script
</string>
</value>
</item>
<item>
<key>
<string>
name_subpath
</string>
</key>
<value>
<string>
traverse_subpath
</string>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key>
<string>
_body
</string>
</key>
<value>
<string>
"""\n
Handle appended data chunks.\n
"""\n
data_stream = state_change[\'object\']\n
\n
# call you own script to handle newly appended data which \n
# is not processed yet\n
data_stream.DataStream_transformTail()\n
</string>
</value>
</item>
<item>
<key>
<string>
_params
</string>
</key>
<value>
<string>
state_change,**kw
</string>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
DataStream_appendData
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/variables.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"Variables"
module=
"Products.DCWorkflow.Variables"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
_mapping
</string>
</key>
<value>
<dictionary/>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
variables
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/WorkflowTemplateItem/portal_workflow/data_stream_interaction_workflow/worklists.xml
0 → 100644
View file @
02214c63
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"Worklists"
module=
"Products.DCWorkflow.Worklists"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
_mapping
</string>
</key>
<value>
<dictionary/>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
worklists
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_wendelin/bt/description
View file @
02214c63
Wendelin is a Big Data platform based on ERP5.
Wendelin is a Big Data platform based on ERP5.
More information can be found at http://www.wendelin.io/
More information can be found at http://www.wendelin.io/
\ No newline at end of file
bt5/erp5_wendelin/bt/template_portal_type_workflow_chain_list
View file @
02214c63
...
@@ -20,6 +20,7 @@ Data Release | edit_workflow
...
@@ -20,6 +20,7 @@ Data Release | edit_workflow
Data Release | validation_workflow
Data Release | validation_workflow
Data Set | edit_workflow
Data Set | edit_workflow
Data Set | validation_workflow
Data Set | validation_workflow
Data Stream | data_stream_interaction_workflow
Data Stream | edit_workflow
Data Stream | edit_workflow
Data Stream | validation_workflow
Data Stream | validation_workflow
Data Supply Line | edit_workflow
Data Supply Line | edit_workflow
...
...
bt5/erp5_wendelin/bt/template_workflow_id_list
0 → 100644
View file @
02214c63
data_stream_interaction_workflow
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment