Add StoredSequence to run tests from an initial stored sequence
When using Sequences to run tests, we usually have multiple tests that look like this :
def test_01(self):
sequence_list = SequenceList()
sequence_string = self.base_sequence_string # Initialize the tests with a shared base sequence
# Add more steps uniq to this test
sequence_string += """
stepXXX
stepXXY
...
stepZZZ
"""
sequence_list.addSequenceString(sequence_string)
sequence_list.play(self)
The idea is to run only once the steps defined in the sequence self.base_sequence_string
, save the stte, and restore it for each state based on self.base_sequence_string
. The state of ERP5 is stored in a Trash Bin in portal_trash. The state is composed of : all the objects contained in ERP5 and the content of Sequence._dict.
The code would become something like this :
# Register the base sequence_string
registerSequenceString("base", """
step1
step2
...
"""
def test_01(self):
sequence_list = SequenceList()
# Use a StoredSequence, and inform it to bootstrap
# from the base sequence_string
sequence = StoredSequence(self, "base")
# Define the custom step sequence for this test
sequence.setSequenceString("""
stepXXX
stepXXY
...
stepZZZ
""")
sequence_list.addSequence(sequence)
sequence_list.play(self)
I did a little benchmark using real tests from a customer project. The test has 22 steps in the base sequence, and 10 custom steps. I ran the 3 following cases 5 times, excluded the 2 most extrem values each time, and calculated the average time of the 3 remaining times.
- running the test with a Sequence, as usual : 29.642s
- running the test with a StoredSequence, but the iniatial state is not stored in portal_trash yet : 31.007s
- running the test with a StoredSequence, restoring the initial state from portal_trash : 20.878s
So we see an overhead of 4% when we need to store the state, compared to running the test "as usual". This overhead happens only once. Then the test runs 30% faster.
Of course this benchmark is really case dependant, as the more shared states, the more time could be saved. But it gives an idea.
Another advantage of the StoredSequence
is that it is very easy to adapt existing code. There is no need to write any new logic, to split test cases, and can be adopted progressively.