Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
T
telecom
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Roque
telecom
Commits
88be7c22
Commit
88be7c22
authored
Aug 13, 2018
by
Roque Porchetto
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
erp5_wendelin_telecom_ingestion: revert process fif raw data and fix getDataStreamList performance
parent
541e9e80
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
27 additions
and
31 deletions
+27
-31
bt5/erp5_wendelin_telecom_ingestion/ExtensionTemplateItem/portal_components/extension.erp5.external_process_raw_data.py
...al_components/extension.erp5.external_process_raw_data.py
+8
-10
bt5/erp5_wendelin_telecom_ingestion/ExtensionTemplateItem/portal_components/extension.erp5.external_process_raw_data.xml
...l_components/extension.erp5.external_process_raw_data.xml
+7
-5
bt5/erp5_wendelin_telecom_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_telecom/getDataStreamList.py
...m/portal_skins/erp5_wendelin_telecom/getDataStreamList.py
+12
-16
No files found.
bt5/erp5_wendelin_telecom_ingestion/ExtensionTemplateItem/portal_components/extension.erp5.external_process_raw_data.py
View file @
88be7c22
...
...
@@ -70,17 +70,15 @@ def getMNEReportJSON(file_name, raw):
def
getRawData
(
file_name
):
raw
=
None
raw_functions
=
[
mne
.
io
.
read_raw_fif
,
mne
.
io
.
read_raw_edf
,
mne
.
io
.
read_events_eeglab
,
mne
.
io
.
read_annotations_eeglab
,
mne
.
io
.
read_raw_eeglab
,
mne
.
io
.
read_raw_artemis123
,
mne
.
io
.
read_raw_bti
,
mne
.
io
.
read_raw_cnt
,
mne
.
io
.
read_raw_ctf
,
mne
.
io
.
read_raw_brainvision
,
mne
.
io
.
read_raw_kit
,
mne
.
read_epochs_kit
,
mne
.
io
.
read_raw_nicolet
,
mne
.
io
.
read_raw_eeglab
,
mne
.
io
.
read_raw_brainvision
,
mne
.
io
.
read_raw_egi
,
mne
.
io
.
read_info
,
mne
.
io
.
read_epochs_eeglab
]
for
function
in
raw_functions
:
try
:
raw
=
mne
.
io
.
read_raw_fif
(
file_name
,
preload
=
False
,
verbose
=
None
)
except
:
pass
if
raw
is
None
:
try
:
raw
=
function
(
file_name
)
print
(
"Function %s could read file %s "
%
(
str
(
function
),
file_name
))
break
except
Exception
as
e
:
print
(
"Exception reading raw data with function %s: %s"
%
(
str
(
function
),
str
(
e
)))
raw
=
mne
.
io
.
read_raw_edf
(
file_name
,
preload
=
False
,
verbose
=
None
)
except
:
pass
if
raw
is
None
:
raise
StandardError
(
"the file does not contain raw data."
)
return
raw
...
...
bt5/erp5_wendelin_telecom_ingestion/ExtensionTemplateItem/portal_components/extension.erp5.external_process_raw_data.xml
View file @
88be7c22
...
...
@@ -52,11 +52,13 @@
<key>
<string>
text_content_warning_message
</string>
</key>
<value>
<tuple>
<string>
W:100, 8: Unused variable \'times\' (unused-variable)
</string>
<string>
W:137, 6: No exception type(s) specified (bare-except)
</string>
<string>
W:173, 8: Unreachable code (unreachable)
</string>
<string>
W:192, 11: Using type() instead of isinstance() for a typecheck. (unidiomatic-typecheck)
</string>
<string>
W:196, 10: No exception type(s) specified (bare-except)
</string>
<string>
W: 75, 2: No exception type(s) specified (bare-except)
</string>
<string>
W: 80, 4: No exception type(s) specified (bare-except)
</string>
<string>
W: 98, 8: Unused variable \'times\' (unused-variable)
</string>
<string>
W:135, 6: No exception type(s) specified (bare-except)
</string>
<string>
W:171, 8: Unreachable code (unreachable)
</string>
<string>
W:190, 11: Using type() instead of isinstance() for a typecheck. (unidiomatic-typecheck)
</string>
<string>
W:194, 10: No exception type(s) specified (bare-except)
</string>
</tuple>
</value>
</item>
...
...
bt5/erp5_wendelin_telecom_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_telecom/getDataStreamList.py
View file @
88be7c22
...
...
@@ -18,25 +18,21 @@ data_set = portal.data_set_module.get(data_set_reference)
if
data_set
is
None
:
return
[]
states_query
=
ComplexQuery
(
Query
(
simulation_state
=
'Delivered'
),
Query
(
simulation_state
=
'Stopped'
),
Query
(
simulation_state
=
'Started'
),
logical_operator
=
"OR"
)
query_dict
=
{
"
query"
:
states_query
,
"
portal_type"
:
"Data Ingestion Line
"
,
"
resource_portal_type"
:
"Data Product"
}
"
portal_type"
:
"Data Stream"
,
"
reference"
:
data_set
.
getReference
()
+
"/%
"
,
"
validation_state"
:
'validated'
}
data_stream_list
=
[]
for
line
in
portal_catalog
(
**
query_dict
):
i
f
line
.
getAggregateDeviceConfiguration
()
==
"data_set_module/"
+
str
(
data_set
.
getId
()):
for
item
in
line
.
getAggregateSet
():
if
"data_stream_module/"
in
item
:
data_stream
=
context
.
restrictedTraverse
(
item
)
if
data_stream
.
getValidationState
==
"invalidated"
or
data_stream
.
getReference
().
endswith
(
"_invalid"
)
:
break
data_stream_list
.
append
([
item
,
data_stream
.
getReference
()])
break
for
stream
in
portal_catalog
(
**
query_dict
):
i
ng_dict
=
{
"portal_type"
:
"Data Ingestion"
,
"reference"
:
stream
.
getReference
()}
ingestions
=
portal_catalog
(
**
ing_dict
)
if
len
(
ingestions
)
>
0
:
ingestion
=
ingestions
[
0
]
if
ingestion
.
getSimulationState
()
in
[
'started'
,
'stopped'
,
'delivered'
]:
data_stream_list
.
append
([
"data_stream_module/"
+
stream
.
getId
(),
stream
.
getReference
()])
return
{
"status_code"
:
0
,
"result"
:
data_stream_list
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment