Skip to content

S
slapos.core
Commit ea940ce8 authored by Rafael Monnerat's avatar Rafael Monnerat
Browse files
Options

slapos_cloud: Introduce SlapOSComputeNodeMixin 

   Move away part of SlapTool code into a Mixin specific to fill and get Computer Information cache
parent 2372fb34
Expand all Show whitespace changes
Inline Side-by-side
Showing with 534 additions and 299 deletions
master/bt5/slapos_cloud/MixinTemplateItem/portal_components/mixin.erp5.SlapOSComputeNodeMixin.xml 0 → 100644
View file @ ea940ce8
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Mixin Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>SlapOSComputeNodeMixin</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>mixin.erp5.SlapOSComputeNodeMixin</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Mixin Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>error_message</string> </key>
<value>
<list>
<persistent> <string encoding="base64">AAAAAAAAAAU=</string> </persistent>
</list>
</value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="5" aka="AAAAAAAAAAU=">
<pickle>
<global name="Message" module="Products.ERP5Type.Message"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>default</string> </key>
<value> <string>Class ${reference} must be defined</string> </value>
</item>
<item>
<key> <string>domain</string> </key>
<value> <string>erp5_ui</string> </value>
</item>
<item>
<key> <string>mapping</string> </key>
<value>
<dictionary>
<item>
<key> <string>reference</string> </key>
<value> <string>SlapOSComputeNodeCacheMixin</string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>message</string> </key>
<value> <string>Class ${reference} must be defined</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
master/bt5/slapos_cloud/PortalTypeTypeMixinTemplateItem/type_mixin.xml
View file @ ea940ce8
<type_mixin> <type_mixin>
<portal_type id="Compute Node"> <portal_type id="Compute Node">
<item>SlapOSCacheMixin</item> <item>SlapOSCacheMixin</item>
<item>SlapOSComputeNodeMixin</item>
</portal_type> </portal_type>
<portal_type id="Compute Partition"> <portal_type id="Compute Partition">
<item>SlapOSCacheMixin</item> <item>SlapOSCacheMixin</item>
......
master/bt5/slapos_cloud/bt/template_mixin_id_list
View file @ ea940ce8
mixin.erp5.SlapOSCacheMixin mixin.erp5.SlapOSCacheMixin
mixin.erp5.SlapOSComputeNodeMixin
\ No newline at end of file
master/bt5/slapos_cloud/bt/template_path_list
View file @ ea940ce8
...@@ -19,6 +19,8 @@ portal_alarms/slapos_stop_collect_instance ...@@ -19,6 +19,8 @@ portal_alarms/slapos_stop_collect_instance
portal_alarms/slapos_update_compute_node_capacity_scope portal_alarms/slapos_update_compute_node_capacity_scope
portal_caches/access_status_data_cache_factory portal_caches/access_status_data_cache_factory
portal_caches/access_status_data_cache_factory/volatile_cache_plugin portal_caches/access_status_data_cache_factory/volatile_cache_plugin
portal_caches/compute_node_information_cache_factory
portal_caches/compute_node_information_cache_factory/persistent_cache_plugin
portal_caches/last_stored_data_cache_factory portal_caches/last_stored_data_cache_factory
portal_caches/last_stored_data_cache_factory/volatile_cache_plugin portal_caches/last_stored_data_cache_factory/volatile_cache_plugin
product_module/compute_node product_module/compute_node
......
master/bt5/slapos_cloud/bt/template_portal_type_type_mixin_list
View file @ ea940ce8
Compute Node | SlapOSCacheMixin Compute Node | SlapOSCacheMixin
Compute Node | SlapOSComputeNodeMixin
Compute Partition | SlapOSCacheMixin Compute Partition | SlapOSCacheMixin
Person | SlapOSCacheMixin Person | SlapOSCacheMixin
\ No newline at end of file
master/bt5/slapos_slap_tool/ExtensionTemplateItem/portal_components/extension.erp5.SlapOSSlapTool.py
View file @ ea940ce8
...@@ -17,30 +17,29 @@ def getComputeNodeReferenceAndUserId(item): ...@@ -17,30 +17,29 @@ def getComputeNodeReferenceAndUserId(item):
compute_node = partition.getParentValue() compute_node = partition.getParentValue()
if compute_node is not None and compute_node.getValidationState() == 'validated': if compute_node is not None and compute_node.getValidationState() == 'validated':
return compute_node.getReference(), compute_node.getUserId() return compute_node, compute_node.getReference(), compute_node.getUserId()
return None, None return None, None, None
def Item_activateFillComputeNodeInformationCache(state_change): def Item_activateFillComputeNodeInformationCache(state_change):
item = state_change['object'] item = state_change['object']
portal = item.getPortalObject() portal = item.getPortalObject()
compute_node_reference, user_id = getComputeNodeReferenceAndUserId(item) compute_node, compute_node_reference, user_id = getComputeNodeReferenceAndUserId(item)
if compute_node_reference is None: if compute_node is None:
return None return None
if user_id is None: if user_id is None:
return None return None
user = portal.acl_users.getUserById(user_id) user = portal.acl_users.getUserById(user_id)
if user is None: if user is None:
raise ValueError("User %s not found" % user_id) raise ValueError("User %s not found" % user_id)
sm = getSecurityManager() sm = getSecurityManager()
try: try:
newSecurityManager(None, user) newSecurityManager(None, user)
portal.portal_slap._activateFillComputeNodeInformationCache( compute_node._activateFillComputeNodeInformationCache(
compute_node_reference, compute_node_reference) compute_node_reference)
finally: finally:
setSecurityManager(sm) setSecurityManager(sm)
......
master/bt5/slapos_slap_tool/TestTemplateItem/portal_components/test.erp5.testSlapOSSlapTool.py
View file @ ea940ce8
...@@ -78,16 +78,16 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin): ...@@ -78,16 +78,16 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin):
self.tic() self.tic()
self.login(self.compute_node_user_id) self.login(self.compute_node_user_id)
self.portal_slap.getFullComputerInformation(self.compute_node_id)
# First access. # First access.
# Cache has been filled by interaction workflow # Cache has been filled by interaction workflow
# (luckily, it seems the cache is filled after everything is indexed) # (luckily, it seems the cache is filled after everything is indexed)
response = self.portal_slap.getFullComputerInformation(self.compute_node_id) response = self.portal_slap.getFullComputerInformation(self.compute_node_id)
self.commit() self.commit()
first_etag = self.portal_slap._calculateRefreshEtag() first_etag = self.compute_node._calculateRefreshEtag()
first_body_fingerprint = hashData( first_body_fingerprint = hashData(
self.portal_slap._getCacheComputeNodeInformation(self.compute_node_id, self.compute_node._getCacheComputeNodeInformation(self.compute_node_id)
self.compute_node_id)
) )
self.assertEqual(200, response.status) self.assertEqual(200, response.status)
self.assertTrue('last-modified' not in response.headers) self.assertTrue('last-modified' not in response.headers)
...@@ -120,10 +120,9 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin): ...@@ -120,10 +120,9 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin):
self.commit() self.commit()
self.assertEqual(200, response.status) self.assertEqual(200, response.status)
self.assertTrue('last-modified' not in response.headers) self.assertTrue('last-modified' not in response.headers)
second_etag = self.portal_slap._calculateRefreshEtag() second_etag = self.compute_node._calculateRefreshEtag()
second_body_fingerprint = hashData( second_body_fingerprint = hashData(
self.portal_slap._getCacheComputeNodeInformation(self.compute_node_id, self.compute_node._getCacheComputeNodeInformation(self.compute_node_id)
self.compute_node_id)
) )
self.assertNotEqual(first_etag, second_etag) self.assertNotEqual(first_etag, second_etag)
# The indexation timestamp does not impact the response body # The indexation timestamp does not impact the response body
...@@ -154,10 +153,9 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin): ...@@ -154,10 +153,9 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin):
# Check that the result is stable, as the indexation timestamp is not changed yet # Check that the result is stable, as the indexation timestamp is not changed yet
current_activity_count = len(self.portal.portal_activities.getMessageList()) current_activity_count = len(self.portal.portal_activities.getMessageList())
# Edition does not impact the etag # Edition does not impact the etag
self.assertEqual(second_etag, self.portal_slap._calculateRefreshEtag()) self.assertEqual(second_etag, self.compute_node._calculateRefreshEtag())
third_body_fingerprint = hashData( third_body_fingerprint = hashData(
self.portal_slap._getCacheComputeNodeInformation(self.compute_node_id, self.compute_node._getCacheComputeNodeInformation(self.compute_node_id)
self.compute_node_id)
) )
# The edition impacts the response body # The edition impacts the response body
self.assertNotEqual(first_body_fingerprint, third_body_fingerprint) self.assertNotEqual(first_body_fingerprint, third_body_fingerprint)
...@@ -177,7 +175,7 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin): ...@@ -177,7 +175,7 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin):
self.commit() self.commit()
self.assertEqual(200, response.status) self.assertEqual(200, response.status)
self.assertTrue('last-modified' not in response.headers) self.assertTrue('last-modified' not in response.headers)
third_etag = self.portal_slap._calculateRefreshEtag() third_etag = self.compute_node._calculateRefreshEtag()
self.assertNotEqual(second_etag, third_etag) self.assertNotEqual(second_etag, third_etag)
self.assertEqual(third_etag, response.headers.get('etag')) self.assertEqual(third_etag, response.headers.get('etag'))
self.assertEqual(third_body_fingerprint, hashData(response.body)) self.assertEqual(third_body_fingerprint, hashData(response.body))
...@@ -192,12 +190,11 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin): ...@@ -192,12 +190,11 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin):
# Check that the result is stable, as the indexation timestamp is not changed yet # Check that the result is stable, as the indexation timestamp is not changed yet
current_activity_count = len(self.portal.portal_activities.getMessageList()) current_activity_count = len(self.portal.portal_activities.getMessageList())
# Edition does not impact the etag # Edition does not impact the etag
self.assertEqual(third_etag, self.portal_slap._calculateRefreshEtag()) self.assertEqual(third_etag, self.compute_node._calculateRefreshEtag())
# The edition does not impact the response body yet, as the aggregate relation # The edition does not impact the response body yet, as the aggregate relation
# is not yet unindex # is not yet unindex
self.assertEqual(third_body_fingerprint, hashData( self.assertEqual(third_body_fingerprint, hashData(
self.portal_slap._getCacheComputeNodeInformation(self.compute_node_id, self.compute_node._getCacheComputeNodeInformation(self.compute_node_id)
self.compute_node_id)
)) ))
response = self.portal_slap.getFullComputerInformation(self.compute_node_id) response = self.portal_slap.getFullComputerInformation(self.compute_node_id)
self.commit() self.commit()
...@@ -217,10 +214,9 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin): ...@@ -217,10 +214,9 @@ class TestSlapOSSlapToolgetFullComputerInformation(TestSlapOSSlapToolMixin):
self.commit() self.commit()
self.assertEqual(200, response.status) self.assertEqual(200, response.status)
self.assertTrue('last-modified' not in response.headers) self.assertTrue('last-modified' not in response.headers)
fourth_etag = self.portal_slap._calculateRefreshEtag() fourth_etag = self.compute_node._calculateRefreshEtag()
fourth_body_fingerprint = hashData( fourth_body_fingerprint = hashData(
self.portal_slap._getCacheComputeNodeInformation(self.compute_node_id, self.compute_node._getCacheComputeNodeInformation(self.compute_node_id)
self.compute_node_id)
) )
self.assertNotEqual(third_etag, fourth_etag) self.assertNotEqual(third_etag, fourth_etag)
# The indexation timestamp does not impact the response body # The indexation timestamp does not impact the response body
...@@ -733,43 +729,9 @@ class TestSlapOSSlapToolComputeNodeAccess(TestSlapOSSlapToolMixin): ...@@ -733,43 +729,9 @@ class TestSlapOSSlapToolComputeNodeAccess(TestSlapOSSlapToolMixin):
response = self.portal_slap.computerBang(self.compute_node_id, response = self.portal_slap.computerBang(self.compute_node_id,
error_log) error_log)
self.assertEqual('None', response) self.assertEqual('None', response)
created_at = rfc1123_date(DateTime()) # We do not assert getComputerStatus on this test, since
since = created_at # the change of the timestamp is part of reportComputeNodeBang
response = self.portal_slap.getComputerStatus(self.compute_node_id)
# check returned XML
xml_fp = StringIO.StringIO()
xml.dom.ext.PrettyPrint(xml.dom.ext.reader.Sax.FromXml(response.body),
stream=xml_fp)
xml_fp.seek(0)
got_xml = xml_fp.read()
expected_xml = """\
<?xml version='1.0' encoding='UTF-8'?>
<marshal>
<dictionary id='i2'>
<string>created_at</string>
<string>%(created_at)s</string>
<string>no_data_since_15_minutes</string>
<int>0</int>
<string>no_data_since_5_minutes</string>
<int>0</int>
<string>since</string>
<string>%(since)s</string>
<string>state</string>
<string/>
<string>text</string>
<string>#error bang</string>
<string>user</string>
<string>%(compute_node_id)s</string>
</dictionary>
</marshal>
""" % dict(
created_at=created_at,
since=since,
compute_node_id=self.compute_node_id,
)
self.assertEqual(expected_xml, got_xml,
'\n'.join([q for q in difflib.unified_diff(expected_xml.split('\n'), got_xml.split('\n'))]))
self.assertComputeNodeBangSimulator((), {'comment': error_log}) self.assertComputeNodeBangSimulator((), {'comment': error_log})
finally: finally:
if os.path.exists(self.compute_node_bang_simulator): if os.path.exists(self.compute_node_bang_simulator):
...@@ -2635,43 +2597,9 @@ class TestSlapOSSlapToolPersonAccess(TestSlapOSSlapToolMixin): ...@@ -2635,43 +2597,9 @@ class TestSlapOSSlapToolPersonAccess(TestSlapOSSlapToolMixin):
response = self.portal_slap.computerBang(self.compute_node_id, response = self.portal_slap.computerBang(self.compute_node_id,
error_log) error_log)
self.assertEqual('None', response) self.assertEqual('None', response)
created_at = rfc1123_date(DateTime()) # We do not assert getComputerStatus on this test, since
since = created_at # the change of the timestamp is part of reportComputeNodeBang
response = self.portal_slap.getComputerStatus(self.compute_node_id)
# check returned XML
xml_fp = StringIO.StringIO()
xml.dom.ext.PrettyPrint(xml.dom.ext.reader.Sax.FromXml(response.body),
stream=xml_fp)
xml_fp.seek(0)
got_xml = xml_fp.read()
expected_xml = """\
<?xml version='1.0' encoding='UTF-8'?>
<marshal>
<dictionary id='i2'>
<string>created_at</string>
<string>%(created_at)s</string>
<string>no_data_since_15_minutes</string>
<int>0</int>
<string>no_data_since_5_minutes</string>
<int>0</int>
<string>since</string>
<string>%(since)s</string>
<string>state</string>
<string/>
<string>text</string>
<string>#error bang</string>
<string>user</string>
<string>%(person_reference)s</string>
</dictionary>
</marshal>
""" % dict(
created_at=created_at,
since=since,
person_reference=self.person_reference,
)
self.assertEqual(expected_xml, got_xml,
'\n'.join([q for q in difflib.unified_diff(expected_xml.split('\n'), got_xml.split('\n'))]))
self.assertComputeNodeBangSimulator((), {'comment': error_log}) self.assertComputeNodeBangSimulator((), {'comment': error_log})
finally: finally:
if os.path.exists(self.compute_node_bang_simulator): if os.path.exists(self.compute_node_bang_simulator):
......
master/bt5/slapos_slap_tool/bt/template_keep_path_list
View file @ ea940ce8
portal_caches/last_stored_data_cache_factory portal_caches/last_stored_data_cache_factory
portal_caches/last_stored_data_cache_factory/volatile_cache_plugin portal_caches/last_stored_data_cache_factory/volatile_cache_plugin
portal_caches/compute_node_information_cache_factory
portal_caches/compute_node_information_cache_factory/persistent_cache_plugin
\ No newline at end of file
master/bt5/slapos_slap_tool/bt/template_path_list
View file @ ea940ce8
portal_caches/compute_node_information_cache_factory
portal_caches/compute_node_information_cache_factory/persistent_cache_plugin
portal_caches/slap_cache_factory portal_caches/slap_cache_factory
portal_caches/slap_cache_factory/persistent_cache_plugin portal_caches/slap_cache_factory/persistent_cache_plugin
web_site_module/slapos_hateoas web_site_module/slapos_hateoas
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7