- 17 Jul, 2024 2 commits
-
-
Kirill Smelkov authoredThis protects us from ZODB and other third-party code from using global PRNG and thus affecting its state and gen_testdata.py output. Till now we were lucky that there was no such third-party user and so the output of gen_testdata.py remains exactly the same as before hereby patch. But better be robust and avoid even potential possibility of such situations. Using private PRNG will also help later when running gen_testdata.py via Python3 : in py3 they changed behaviour of builtin random and emitted numbers differ from the same random when run under py2. By using our own PRNG we will be able to level-out that and use essentially the same PRNG on both py2 and py3.
-
Kirill Smelkov authored
Previsouly we were generating e.g. 1.fs starting over existing 1.fs and due to that when FileStoraga was opening it, it was e.g. scanning existing 1.fs.index, thus doing some time() calls and so when the database was started to be generted the state of the timer was different compared to the situation if the database did not existed before gen_testdata.py run. In other words the result of gen_testdata.py run was different for the cases when it run after `rm testdata/1*` and without that rm. -> Make gen_testdata result to be stable irregardless of the start state when it runs by explicitly removing test databases fitst. As diff to *.zpickledis.ok shows, resulting changes are only in transaction IDs.
-
- 16 Jul, 2024 1 commit
-
-
Kirill Smelkov authored
test/gen_testdata: Make sure that "with zext" and "without zext" databases differ only in transaction extension Previously diff in between 1.zdump.zpickledis.ok and 1_!zext.zdump.zpickledis.ok indicated difference in many data records as demonstrated in the appendix. That was due to that !zext variant, while skipping generating transaction extension, contrary to zext variant, did not consumed PRNG output, and so when going further PRNG state for zext and !zext variants was diverging. -> Fix it via consuming PRNG output in !zext case exactly the same way as in zext case, but throwing out ext4subj output. After the patch the diff in between `1.zdump.zpickledis.ok 1_\!zext.zdump.zpickledis.ok` shows only differences in between txn.extension fields: --- a/1.zdump.zpickledis.ok +++ b/1_!zext.zdump.zpickledis.ok @@ -1,19 +1,7 @@ txn 0285cbac75555580 "p" user "user0.12" description "step 0.12" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie8' - 17: U SHORT_BINSTRING '2MHMU' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (f)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 +extension "" obj 0000000000000001 34 sha1:9fae782fe3e33273b520c954925704856ca90dc6 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' ... Appendix. `1.zdump.zpickledis.ok 1_\!zext.zdump.zpickledis.ok` before this patch diff --git a/1.zdump.zpickledis.ok b/1_!zext.zdump.zpickledis.ok index 9385444..f80531d 100644 --- a/1.zdump.zpickledis.ok +++ b/1_!zext.zdump.zpickledis.ok @@ -1,114 +1,49 @@ -txn 0285cbac75555580 "p" -user "user0.12" -description "step 0.12" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie8' - 17: U SHORT_BINSTRING '2MHMU' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (f)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000001 34 sha1:9fae782fe3e33273b520c954925704856ca90dc6 - 0: \x80 PROTO 2 - 2: c GLOBAL '__main__ Object' - 19: q BINPUT 1 - 21: . STOP - highest protocol among opcodes = 2 - 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'f0.12' - 31: q BINPUT 2 - 33: . STOP - highest protocol among opcodes = 2 - - txn 0285cbac8369d066 "p" user "user0.15" description "step 0.15" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieY' - 17: U SHORT_BINSTRING 'EDZ10' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:5dbf26359289e5a25f9755f832ea91cc8ee76686 +extension "" +obj 0000000000000003 34 sha1:e616d80f35ea7aaaecdbc65e1e5838c25f4bf07a 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e0.15' + 24: U SHORT_BINSTRING 'b0.15' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 -txn 0285cbac962fc999 "p" -user "user0.19" -description "step 0.19" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieF' - 17: U SHORT_BINSTRING 'OUC9L' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (a)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000005 34 sha1:9e0a8797eb8f60498563703b1961126fba35be9b +txn 0285cbac9f92c633 "p" +user "user0.21" +description "step 0.21" +extension "" +obj 0000000000000001 34 sha1:025d34759b434419251b5a429f3ad78ce3c2fd98 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'a0.19' + 24: U SHORT_BINSTRING 'f0.21' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 -txn 0285cbac9f92c633 "p" -user "user0.21" -description "step 0.21" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie8' - 17: U SHORT_BINSTRING '0QC1A' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 34 sha1:009174c1f01142f24495fa13738749ff528fd82c +txn 0285cbaca4444480 "p" +user "user0.22" +description "step 0.22" +extension "" +obj 0000000000000006 34 sha1:0734767857e69ab8bfd0007d4ca3f1a7fd69b677 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd0.21' + 24: U SHORT_BINSTRING 'g0.22' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -117,27 +52,15 @@ obj 0000000000000002 34 sha1:009174c1f01142f24495fa13738749ff528fd82c txn 0285cbacb70a3db3 "p" user "root0.1\nYour\nMagesty " description "undo 0.1\nmore detailed description\n\nzzz ...\t" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieV' - 17: U SHORT_BINSTRING 'VHBGT' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (undo AoXLrK2nQRk=)' - 69: u SETITEMS (MARK at 5) - 70: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 34 sha1:484358413b2746e8a05b1e3173051abedd28e1fa +extension "" +obj 0000000000000004 34 sha1:fe21381adc01e51965c3278c6debf52ebd63068c 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g0.11' + 24: U SHORT_BINSTRING 'd0.11' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -146,27 +69,15 @@ obj 0000000000000003 34 sha1:484358413b2746e8a05b1e3173051abedd28e1fa txn 0285cbacbbbbbc00 "p" user "user" description "cyclic reference" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieE' - 17: U SHORT_BINSTRING 'ZM3QZ' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (cycle)' - 57: u SETITEMS (MARK at 5) - 58: . STOP - highest protocol among opcodes = 2 -obj 0000000000000007 41 sha1:7108c96ccb9cbeaab1164d533174c300e51309f9 +extension "" +obj 0000000000000002 41 sha1:1e2e3ac81badec749c2082a08d205c06c6bb5119 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' + 24: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' 34: q BINPUT 2 36: h BINGET 1 38: \x86 TUPLE2 @@ -177,9 +88,9 @@ obj 0000000000000007 41 sha1:7108c96ccb9cbeaab1164d533174c300e51309f9 txn 0285cbacc06d3a4c "p" user "" -description "predelete 4" +description "predelete 5" extension "" -obj 0000000000000000 213 sha1:a6e70638fafd4619841032356ae93a4da7b539c5 +obj 0000000000000000 194 sha1:4a9e413a4ff168a6a7f84be32b9168364048fd6d 0: \x80 PROTO 2 2: c GLOBAL 'persistent.mapping PersistentMapping' 40: q BINPUT 1 @@ -193,62 +104,56 @@ obj 0000000000000000 213 sha1:a6e70638fafd4619841032356ae93a4da7b539c5 56: } EMPTY_DICT 57: q BINPUT 4 59: ( MARK - 60: U SHORT_BINSTRING 'a' - 63: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x05' + 60: U SHORT_BINSTRING 'c' + 63: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' 73: q BINPUT 5 75: c GLOBAL '__main__ Object' 92: q BINPUT 6 94: \x86 TUPLE2 95: Q BINPERSID - 96: U SHORT_BINSTRING 'c' - 99: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' + 96: U SHORT_BINSTRING 'b' + 99: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x03' 109: q BINPUT 7 111: h BINGET 6 113: \x86 TUPLE2 114: Q BINPERSID - 115: U SHORT_BINSTRING 'b' - 118: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x08' + 115: U SHORT_BINSTRING 'e' + 118: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' 128: q BINPUT 8 130: h BINGET 6 132: \x86 TUPLE2 133: Q BINPERSID - 134: U SHORT_BINSTRING 'e' - 137: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x06' + 134: U SHORT_BINSTRING 'd' + 137: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x04' 147: q BINPUT 9 149: h BINGET 6 151: \x86 TUPLE2 152: Q BINPERSID - 153: U SHORT_BINSTRING 'd' - 156: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' + 153: U SHORT_BINSTRING 'g' + 156: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x06' 166: q BINPUT 10 168: h BINGET 6 170: \x86 TUPLE2 171: Q BINPERSID - 172: U SHORT_BINSTRING 'g' - 175: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x03' + 172: U SHORT_BINSTRING 'f' + 175: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x01' 185: q BINPUT 11 187: h BINGET 6 189: \x86 TUPLE2 190: Q BINPERSID - 191: U SHORT_BINSTRING 'f' - 194: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x01' - 204: q BINPUT 12 - 206: h BINGET 6 - 208: \x86 TUPLE2 - 209: Q BINPERSID - 210: u SETITEMS (MARK at 59) - 211: s SETITEM - 212: . STOP + 191: u SETITEMS (MARK at 59) + 192: s SETITEM + 193: . STOP highest protocol among opcodes = 2 -obj 0000000000000008 32 sha1:936674657cf846998d27356363832827fa612092 +obj 0000000000000007 32 sha1:b9d61167d2388ac320b1bc2940b43e7a8bad46ac 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'b0*' + 24: U SHORT_BINSTRING 'e0*' 29: q BINPUT 2 31: . STOP highest protocol among opcodes = 2 @@ -257,20 +162,8 @@ obj 0000000000000008 32 sha1:936674657cf846998d27356363832827fa612092 txn 0285cbad06d3a0cc " " user "user1.0" description "step 1.0" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieE' - 17: U SHORT_BINSTRING 'VAZ3U' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 33 sha1:f18e991b87b63cf1f9486d74d70020ff8d573eec +extension "" +obj 0000000000000007 33 sha1:f18e991b87b63cf1f9486d74d70020ff8d573eec 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -286,27 +179,15 @@ obj 0000000000000006 33 sha1:f18e991b87b63cf1f9486d74d70020ff8d573eec txn 0285cbad0b851f19 " " user "user1.1" description "step 1.1" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieN' - 17: U SHORT_BINSTRING 'GSV4I' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (b)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000008 33 sha1:c37e1d2350c8fc4e18cdbc53b432dba50e5196ba +extension "" +obj 0000000000000006 33 sha1:bf4f1be3d63bca5c7dccf98f7660f419597a0f3d 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'b1.1' + 24: U SHORT_BINSTRING 'g1.1' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -315,20 +196,8 @@ obj 0000000000000008 33 sha1:c37e1d2350c8fc4e18cdbc53b432dba50e5196ba txn 0285cbad10369d66 " " user "user1.2" description "step 1.2" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieY' - 17: U SHORT_BINSTRING 'A01OK' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 33 sha1:28e9880fc0f50a9fea5c4a9e861adc1fe44c9f5c +extension "" +obj 0000000000000006 33 sha1:28e9880fc0f50a9fea5c4a9e861adc1fe44c9f5c 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -344,27 +213,15 @@ obj 0000000000000003 33 sha1:28e9880fc0f50a9fea5c4a9e861adc1fe44c9f5c txn 0285cbad14e81bb3 " " user "user1.3" description "step 1.3" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-generator' - 19: q BINPUT 2 - 21: U SHORT_BINSTRING 'zodb/py2 (g)' - 35: U SHORT_BINSTRING 'x-cookieW' - 46: U SHORT_BINSTRING '1QPNP' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 33 sha1:5fb466e36ea6f847b73ad7976def8ad60e00e766 +extension "" +obj 0000000000000004 33 sha1:e8796affc44d563a09a2913907f91467a34498fc 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g1.3' + 24: U SHORT_BINSTRING 'd1.3' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -373,27 +230,15 @@ obj 0000000000000003 33 sha1:5fb466e36ea6f847b73ad7976def8ad60e00e766 txn 0285cbad19999a00 " " user "user1.4" description "step 1.4" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieC' - 17: U SHORT_BINSTRING 'J7L05' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (c)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000007 33 sha1:90b0ffa657df9de708913a2cbbd454126fd9de15 +extension "" +obj 0000000000000006 33 sha1:15a38343610d25057cd88521669671350eb0c0a8 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'c1.4' + 24: U SHORT_BINSTRING 'g1.4' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -402,27 +247,15 @@ obj 0000000000000007 33 sha1:90b0ffa657df9de708913a2cbbd454126fd9de15 txn 0285cbad1e4b184c " " user "user1.5" description "step 1.5" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieA' - 17: U SHORT_BINSTRING 'CM15Z' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (f)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000001 33 sha1:70b0a88b7652b82b82539800484dc7788277f32a +extension "" +obj 0000000000000003 33 sha1:0adf3b787e6814b6cc073bd51a42c3f18615a674 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'f1.5' + 24: U SHORT_BINSTRING 'b1.5' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -431,27 +264,15 @@ obj 0000000000000001 33 sha1:70b0a88b7652b82b82539800484dc7788277f32a txn 0285cbad22fc9699 " " user "user1.6" description "step 1.6" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieI' - 17: U SHORT_BINSTRING 'AH816' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 33 sha1:e9018b0bc67c9de08becf1f1fe1a548ed263fb29 +extension "" +obj 0000000000000001 33 sha1:4d811fcaaa707127f5f6f49dc368ffd2ebae1018 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd1.6' + 24: U SHORT_BINSTRING 'f1.6' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -460,27 +281,15 @@ obj 0000000000000002 33 sha1:e9018b0bc67c9de08becf1f1fe1a548ed263fb29 txn 0285cbad27ae14e6 " " user "user1.7" description "step 1.7" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieU' - 17: U SHORT_BINSTRING 'BE3WH' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (c)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000007 33 sha1:0cd3f4b725517a5371429e2ef2f56ea54fe405fb +extension "" +obj 0000000000000004 33 sha1:4c3e1f41eebd1f56c0fef1f609d64e061a137f5a 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'c1.7' + 24: U SHORT_BINSTRING 'd1.7' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -489,27 +298,77 @@ obj 0000000000000007 33 sha1:0cd3f4b725517a5371429e2ef2f56ea54fe405fb txn 0285cbad2c5f9333 " " user "user1.8" description "step 1.8" -extension +extension "" +obj 0000000000000000 213 sha1:8ccd62e915379b196be62bdb500c4019dfb7825f 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-generator' - 19: q BINPUT 2 - 21: U SHORT_BINSTRING 'zodb/py2 (c)' - 35: U SHORT_BINSTRING 'x-cookieW' - 46: U SHORT_BINSTRING 'HPFAQ' - 53: u SETITEMS (MARK at 5) - 54: . STOP + 2: c GLOBAL 'persistent.mapping PersistentMapping' + 40: q BINPUT 1 + 42: . STOP + highest protocol among opcodes = 2 + 43: \x80 PROTO 2 + 45: } EMPTY_DICT + 46: q BINPUT 2 + 48: U SHORT_BINSTRING 'data' + 54: q BINPUT 3 + 56: } EMPTY_DICT + 57: q BINPUT 4 + 59: ( MARK + 60: U SHORT_BINSTRING 'a' + 63: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x08' + 73: q BINPUT 5 + 75: c GLOBAL '__main__ Object' + 92: q BINPUT 6 + 94: \x86 TUPLE2 + 95: Q BINPERSID + 96: U SHORT_BINSTRING 'c' + 99: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' + 109: q BINPUT 7 + 111: h BINGET 6 + 113: \x86 TUPLE2 + 114: Q BINPERSID + 115: U SHORT_BINSTRING 'b' + 118: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x03' + 128: q BINPUT 8 + 130: h BINGET 6 + 132: \x86 TUPLE2 + 133: Q BINPERSID + 134: U SHORT_BINSTRING 'e' + 137: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' + 147: q BINPUT 9 + 149: h BINGET 6 + 151: \x86 TUPLE2 + 152: Q BINPERSID + 153: U SHORT_BINSTRING 'd' + 156: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x04' + 166: q BINPUT 10 + 168: h BINGET 6 + 170: \x86 TUPLE2 + 171: Q BINPERSID + 172: U SHORT_BINSTRING 'g' + 175: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x06' + 185: q BINPUT 11 + 187: h BINGET 6 + 189: \x86 TUPLE2 + 190: Q BINPERSID + 191: U SHORT_BINSTRING 'f' + 194: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x01' + 204: q BINPUT 12 + 206: h BINGET 6 + 208: \x86 TUPLE2 + 209: Q BINPERSID + 210: u SETITEMS (MARK at 59) + 211: s SETITEM + 212: . STOP highest protocol among opcodes = 2 -obj 0000000000000007 33 sha1:13e366fb1d15d36a62099e6b835638407718229f + +obj 0000000000000008 33 sha1:7cffb67651540355d8467852ec6cabcb48d964aa 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'c1.8' + 24: U SHORT_BINSTRING 'a1.8' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -518,27 +377,15 @@ obj 0000000000000007 33 sha1:13e366fb1d15d36a62099e6b835638407718229f txn 0285cbad31111180 " " user "user1.9" description "step 1.9" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieQ' - 17: U SHORT_BINSTRING 'DZM23' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 33 sha1:3ac37991a56061c7407cc093ee2a71eef4379131 +extension "" +obj 0000000000000001 33 sha1:c32c4831eabc3a97810c4f925465c6093e4fb716 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e1.9' + 24: U SHORT_BINSTRING 'f1.9' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -547,27 +394,15 @@ obj 0000000000000006 33 sha1:3ac37991a56061c7407cc093ee2a71eef4379131 txn 0285cbad35c28fcc " " user "user1.10" description "step 1.10" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieO' - 17: U SHORT_BINSTRING 'EIGHL' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (a)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000005 34 sha1:407cc5710a22f6c387df45aab613aa3673b221c6 +extension "" +obj 0000000000000002 34 sha1:07fe4a745e127abf54f451d41fba5db7bd10ac9d 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'a1.10' + 24: U SHORT_BINSTRING 'c1.10' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -576,27 +411,15 @@ obj 0000000000000005 34 sha1:407cc5710a22f6c387df45aab613aa3673b221c6 txn 0285cbad3a740e19 " " user "user1.11" description "step 1.11" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie2' - 17: U SHORT_BINSTRING 'Z9RFC' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (c)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000007 34 sha1:88ab1add11652101077535c03b04e83fe4ddb88b +extension "" +obj 0000000000000001 34 sha1:2f0fc397cd9bdceb59cfb7e40b823ae3a81e2f7c 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'c1.11' + 24: U SHORT_BINSTRING 'f1.11' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -605,20 +428,8 @@ obj 0000000000000007 34 sha1:88ab1add11652101077535c03b04e83fe4ddb88b txn 0285cbad3f258c66 " " user "user1.12" description "step 1.12" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie7' - 17: U SHORT_BINSTRING 'WGO4E' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:4808aef147f8ded08ffaae2ce04265506385e7f7 +extension "" +obj 0000000000000007 34 sha1:4808aef147f8ded08ffaae2ce04265506385e7f7 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -634,27 +445,15 @@ obj 0000000000000006 34 sha1:4808aef147f8ded08ffaae2ce04265506385e7f7 txn 0285cbad43d70ab3 " " user "user1.13" description "step 1.13" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie5' - 17: U SHORT_BINSTRING '757DJ' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 34 sha1:a9f47880096587b359fa7ea6a0fd213e800a24a4 +extension "" +obj 0000000000000008 34 sha1:656d65291d6e34225fe2c8213c2241b390e7487a 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g1.13' + 24: U SHORT_BINSTRING 'a1.13' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -663,27 +462,15 @@ obj 0000000000000003 34 sha1:a9f47880096587b359fa7ea6a0fd213e800a24a4 txn 0285cbad48888900 " " user "user1.14" description "step 1.14" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieX' - 17: U SHORT_BINSTRING '5EOVH' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 34 sha1:e5b3820378e102a61be2b5998ded3182c106c7db +extension "" +obj 0000000000000004 34 sha1:87a243d2e9bdd70a84129119894b7d25479f4abc 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g1.14' + 24: U SHORT_BINSTRING 'd1.14' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -692,27 +479,15 @@ obj 0000000000000003 34 sha1:e5b3820378e102a61be2b5998ded3182c106c7db txn 0285cbad4d3a074c " " user "user1.15" description "step 1.15" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieC' - 17: U SHORT_BINSTRING 'HO7L7' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 34 sha1:f136bac1befa0fbd1ebd50218f8d9afe00b9b0a5 +extension "" +obj 0000000000000006 34 sha1:2e9ebc331bf53c095e9573de64a9b31f2b340b2f 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd1.15' + 24: U SHORT_BINSTRING 'g1.15' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -721,27 +496,15 @@ obj 0000000000000002 34 sha1:f136bac1befa0fbd1ebd50218f8d9afe00b9b0a5 txn 0285cbad51eb8599 " " user "user1.16" description "step 1.16" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieU' - 17: U SHORT_BINSTRING 'T159S' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 34 sha1:778621ce5c5e97b65343b1ab0cc1a3ce5702fbc8 +extension "" +obj 0000000000000003 34 sha1:86adb5f06801883ba2be7776f68bd8ab46b34dd8 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g1.16' + 24: U SHORT_BINSTRING 'b1.16' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -750,27 +513,15 @@ obj 0000000000000003 34 sha1:778621ce5c5e97b65343b1ab0cc1a3ce5702fbc8 txn 0285cbad569d03e6 " " user "user1.17" description "step 1.17" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie8' - 17: U SHORT_BINSTRING 'T23V1' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (f)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000001 34 sha1:eb6d2d192f3d8fe47a1b2e8119d390ed580c3fb4 +extension "" +obj 0000000000000002 34 sha1:033ab1c2cfacb42e1db63ec9fac8427c700cc3cb 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'f1.17' + 24: U SHORT_BINSTRING 'c1.17' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -779,27 +530,15 @@ obj 0000000000000001 34 sha1:eb6d2d192f3d8fe47a1b2e8119d390ed580c3fb4 txn 0285cbad5b4e8233 " " user "user1.18" description "step 1.18" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieY' - 17: U SHORT_BINSTRING 'UB55N' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (a)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000005 34 sha1:921682b323eb62f109d052bc1dfd4ffe0dbf79db +extension "" +obj 0000000000000006 34 sha1:11f0228fea479a8dacdda2351f2720d005c04b06 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'a1.18' + 24: U SHORT_BINSTRING 'g1.18' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -808,27 +547,15 @@ obj 0000000000000005 34 sha1:921682b323eb62f109d052bc1dfd4ffe0dbf79db txn 0285cbad60000080 " " user "user1.19" description "step 1.19" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieZ' - 17: U SHORT_BINSTRING 'IKOSR' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 34 sha1:b0b31eb0b48548119153628eb3c6711d959e9f9b +extension "" +obj 0000000000000003 34 sha1:6f5ef8a80a64540885791a6b1e2d1a352ee91d1a 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g1.19' + 24: U SHORT_BINSTRING 'b1.19' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -837,27 +564,15 @@ obj 0000000000000003 34 sha1:b0b31eb0b48548119153628eb3c6711d959e9f9b txn 0285cbad64b17ecc " " user "user1.20" description "step 1.20" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieS' - 17: U SHORT_BINSTRING '7JLTH' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000003 34 sha1:6bfd3298d0bea74cfa3d1f01bb722e958e3f1520 +extension "" +obj 0000000000000004 34 sha1:1c032191dab251d941e739b3703f42ed7e43b246 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g1.20' + 24: U SHORT_BINSTRING 'd1.20' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -866,27 +581,15 @@ obj 0000000000000003 34 sha1:6bfd3298d0bea74cfa3d1f01bb722e958e3f1520 txn 0285cbad6962fd19 " " user "user1.21" description "step 1.21" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieI' - 17: U SHORT_BINSTRING 'USN06' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:0d4b4837500e84b190ea2f92b16ab8ec0c486db5 +extension "" +obj 0000000000000003 34 sha1:d90118a669a7a9dbe6b779bcc61110df0cc97fd1 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e1.21' + 24: U SHORT_BINSTRING 'b1.21' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -895,27 +598,15 @@ obj 0000000000000006 34 sha1:0d4b4837500e84b190ea2f92b16ab8ec0c486db5 txn 0285cbad6e147b66 " " user "user1.22" description "step 1.22" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie2' - 17: U SHORT_BINSTRING 'UXAET' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (a)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000005 34 sha1:eacbd02d0d78eece9d784da7f3fd0b2738ca6ce0 +extension "" +obj 0000000000000006 34 sha1:cd7d6434b0aa8290342b60c85d7378cda0d19734 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'a1.22' + 24: U SHORT_BINSTRING 'g1.22' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -924,27 +615,15 @@ obj 0000000000000005 34 sha1:eacbd02d0d78eece9d784da7f3fd0b2738ca6ce0 txn 0285cbad72c5f9b3 " " user "user1.23" description "step 1.23" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieH' - 17: U SHORT_BINSTRING 'AT11F' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (a)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000005 34 sha1:1b443228d5a434ddd9616ecb5aa90672c0ce9ba2 +extension "" +obj 0000000000000001 34 sha1:230c0adc97facea312cd09aeb825f9a6c953e029 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'a1.23' + 24: U SHORT_BINSTRING 'f1.23' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -953,27 +632,15 @@ obj 0000000000000005 34 sha1:1b443228d5a434ddd9616ecb5aa90672c0ce9ba2 txn 0285cbad77777800 " " user "user1.24" description "step 1.24" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieD' - 17: U SHORT_BINSTRING 'O5ZEM' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (f)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000001 34 sha1:b35a1826cc6cb71b9ddff043f0e8f88b4d90281f +extension "" +obj 0000000000000004 34 sha1:ec728760e33cbafcd21496354cbe752995dc02c4 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'f1.24' + 24: U SHORT_BINSTRING 'd1.24' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -982,63 +649,27 @@ obj 0000000000000001 34 sha1:b35a1826cc6cb71b9ddff043f0e8f88b4d90281f txn 0285cbad7c28f64c " " user "root1.0\nYour\nMagesty " description "undo 1.0\nmore detailed description\n\nzzz ...\t" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie3' - 17: U SHORT_BINSTRING 'G51MM' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (undo AoXLrXLF+bM=)' - 69: u SETITEMS (MARK at 5) - 70: . STOP - highest protocol among opcodes = 2 -obj 0000000000000005 from 0285cbad6e147b66 +extension "" +obj 0000000000000001 from 0285cbad3a740e19 txn 0285cbad80da7499 " " user "root1.1\nYour\nMagesty " description "undo 1.1\nmore detailed description\n\nzzz ...\t\t" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieL' - 17: U SHORT_BINSTRING 'CDRHV' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (undo AoXLrXd3eAA=)' - 69: u SETITEMS (MARK at 5) - 70: . STOP - highest protocol among opcodes = 2 -obj 0000000000000001 from 0285cbad569d03e6 +extension "" +obj 0000000000000004 from 0285cbad64b17ecc txn 0285cbad858bf2e6 " " user "user" description "cyclic reference" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie4' - 17: U SHORT_BINSTRING 'C4OMS' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (cycle)' - 57: u SETITEMS (MARK at 5) - 58: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 41 sha1:1e2e3ac81badec749c2082a08d205c06c6bb5119 +extension "" +obj 0000000000000008 41 sha1:ecd78bff79e5e10c04d6a14c8277d3fd5baba3a2 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' + 24: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x08' 34: q BINPUT 2 36: h BINGET 1 38: \x86 TUPLE2 @@ -1051,7 +682,7 @@ txn 0285cbad8a3d7133 " " user "" description "predelete 3" extension "" -obj 0000000000000000 213 sha1:e278899979bad10d72962170790ea2a2f5865567 +obj 0000000000000000 213 sha1:e8e6e66cf82c7cbff72efb7bed28826296e51a01 0: \x80 PROTO 2 2: c GLOBAL 'persistent.mapping PersistentMapping' 40: q BINPUT 1 @@ -1066,38 +697,38 @@ obj 0000000000000000 213 sha1:e278899979bad10d72962170790ea2a2f5865567 57: q BINPUT 4 59: ( MARK 60: U SHORT_BINSTRING 'a' - 63: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x05' + 63: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x08' 73: q BINPUT 5 75: c GLOBAL '__main__ Object' 92: q BINPUT 6 94: \x86 TUPLE2 95: Q BINPERSID 96: U SHORT_BINSTRING 'c' - 99: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' + 99: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' 109: q BINPUT 7 111: h BINGET 6 113: \x86 TUPLE2 114: Q BINPERSID 115: U SHORT_BINSTRING 'b' - 118: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x08' + 118: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\t' 128: q BINPUT 8 130: h BINGET 6 132: \x86 TUPLE2 133: Q BINPERSID 134: U SHORT_BINSTRING 'e' - 137: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x06' + 137: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' 147: q BINPUT 9 149: h BINGET 6 151: \x86 TUPLE2 152: Q BINPERSID 153: U SHORT_BINSTRING 'd' - 156: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' + 156: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x04' 166: q BINPUT 10 168: h BINGET 6 170: \x86 TUPLE2 171: Q BINPERSID 172: U SHORT_BINSTRING 'g' - 175: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\t' + 175: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x06' 185: q BINPUT 11 187: h BINGET 6 189: \x86 TUPLE2 @@ -1113,14 +744,14 @@ obj 0000000000000000 213 sha1:e278899979bad10d72962170790ea2a2f5865567 212: . STOP highest protocol among opcodes = 2 -obj 0000000000000009 32 sha1:38aaea7061f311a5ff41e144ea56df2c8f66435c +obj 0000000000000009 32 sha1:94f4c80027cdf77affb7dd9f26b634975b4619e4 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g1*' + 24: U SHORT_BINSTRING 'b1*' 29: q BINPUT 2 31: . STOP highest protocol among opcodes = 2 @@ -1129,45 +760,21 @@ obj 0000000000000009 32 sha1:38aaea7061f311a5ff41e144ea56df2c8f66435c txn 0285cbad8eeeef80 " " user "root1\nYour\nRoyal\nMagesty' \x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" description "delete 1\nalpha beta gamma'delta\"lambda\n\nqqq ..." -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieS' - 17: U SHORT_BINSTRING 'XVOTI' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (delete 3)' - 60: u SETITEMS (MARK at 5) - 61: . STOP - highest protocol among opcodes = 2 +extension "" obj 0000000000000003 delete txn 0285cbadcbf25966 " " user "user2.0" description "step 2.0" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie1' - 17: U SHORT_BINSTRING 'GRGS2' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (f)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000001 33 sha1:7b5599bdbf192e2d33e2597b52c8a72a751ddd13 +extension "" +obj 0000000000000004 33 sha1:aff1248f33b29d65fe0b038d7dc11db007e2690d 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'f2.0' + 24: U SHORT_BINSTRING 'd2.0' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1176,27 +783,15 @@ obj 0000000000000001 33 sha1:7b5599bdbf192e2d33e2597b52c8a72a751ddd13 txn 0285cbadd0a3d7b3 " " user "user2.1" description "step 2.1" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie3' - 17: U SHORT_BINSTRING 'WYNK7' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 33 sha1:50b0cf792f2fdc3fbfc5f47f148784924350e31c +extension "" +obj 0000000000000006 33 sha1:2a2f21f6f3480aef6afd1398bec7f6bffa43109c 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd2.1' + 24: U SHORT_BINSTRING 'g2.1' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1205,27 +800,15 @@ obj 0000000000000002 33 sha1:50b0cf792f2fdc3fbfc5f47f148784924350e31c txn 0285cbadd5555600 " " user "user2.2" description "step 2.2" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieF' - 17: U SHORT_BINSTRING 'SPFA4' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000009 33 sha1:71f1255a9751e0f223f079552e182cf26b27c0a6 +extension "" +obj 0000000000000008 33 sha1:07e79c0eb72eea524bbfbce530b0671a4f90f483 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g2.2' + 24: U SHORT_BINSTRING 'a2.2' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1234,27 +817,15 @@ obj 0000000000000009 33 sha1:71f1255a9751e0f223f079552e182cf26b27c0a6 txn 0285cbadda06d44c " " user "user2.3" description "step 2.3" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie1' - 17: U SHORT_BINSTRING 'XE3RQ' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000009 33 sha1:e6236b8f0a4a2201c0c2375a8f360905108eff2d +extension "" +obj 0000000000000004 33 sha1:9985268077ca849a615e15ed1b37634875749695 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g2.3' + 24: U SHORT_BINSTRING 'd2.3' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1263,20 +834,8 @@ obj 0000000000000009 33 sha1:e6236b8f0a4a2201c0c2375a8f360905108eff2d txn 0285cbaddeb85299 " " user "user2.4" description "step 2.4" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie2' - 17: U SHORT_BINSTRING '1XYQ2' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 33 sha1:55a37439e3dc66552d680833af332b8be83bfc2a +extension "" +obj 0000000000000007 33 sha1:55a37439e3dc66552d680833af332b8be83bfc2a 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -1292,27 +851,15 @@ obj 0000000000000006 33 sha1:55a37439e3dc66552d680833af332b8be83bfc2a txn 0285cbade369d0e6 " " user "user2.5" description "step 2.5" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieW' - 17: U SHORT_BINSTRING 'C0ZT2' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (a)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000005 33 sha1:6beb5d1ca083744ff1e8a13c0bee70c2df54a05c +extension "" +obj 0000000000000004 33 sha1:2e472bec634f703fc24e5192114b0c61364d9273 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'a2.5' + 24: U SHORT_BINSTRING 'd2.5' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1321,27 +868,15 @@ obj 0000000000000005 33 sha1:6beb5d1ca083744ff1e8a13c0bee70c2df54a05c txn 0285cbade81b4f33 " " user "user2.6" description "step 2.6" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie0' - 17: U SHORT_BINSTRING 'OX40D' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (b)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000008 33 sha1:2a3221e27ac8fbf15ab75b38a9a65e727d237355 +extension "" +obj 0000000000000001 33 sha1:e0148787fb3ffe52ef3ba85ad82b6b8afa47e20f 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'b2.6' + 24: U SHORT_BINSTRING 'f2.6' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1350,27 +885,15 @@ obj 0000000000000008 33 sha1:2a3221e27ac8fbf15ab75b38a9a65e727d237355 txn 0285cbadeccccd80 " " user "user2.7" description "step 2.7" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieY' - 17: U SHORT_BINSTRING '64CY4' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000009 33 sha1:408fddc8c7255c5e2ed94c239ac57c211ab94b6d +extension "" +obj 0000000000000004 33 sha1:9472357211e402a8dfcda8b72f7ef98c2b8d8b20 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g2.7' + 24: U SHORT_BINSTRING 'd2.7' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1379,27 +902,15 @@ obj 0000000000000009 33 sha1:408fddc8c7255c5e2ed94c239ac57c211ab94b6d txn 0285cbadf17e4bcc " " user "user2.8" description "step 2.8" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieZ' - 17: U SHORT_BINSTRING 'AXYM6' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 33 sha1:72eda0cdae0addbec9472e28b5a9a91ecdf41bbf +extension "" +obj 0000000000000006 33 sha1:d57709ac596de41959e36e1dd6902a853213267e 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd2.8' + 24: U SHORT_BINSTRING 'g2.8' 30: q BINPUT 2 32: . STOP highest protocol among opcodes = 2 @@ -1408,20 +919,8 @@ obj 0000000000000002 33 sha1:72eda0cdae0addbec9472e28b5a9a91ecdf41bbf txn 0285cbadf62fca19 " " user "user2.9" description "step 2.9" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie8' - 17: U SHORT_BINSTRING '5WYSB' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 33 sha1:f1d87ba386f57291ecc925020a05df06caedb278 +extension "" +obj 0000000000000007 33 sha1:f1d87ba386f57291ecc925020a05df06caedb278 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -1437,27 +936,15 @@ obj 0000000000000006 33 sha1:f1d87ba386f57291ecc925020a05df06caedb278 txn 0285cbadfae14866 " " user "user2.10" description "step 2.10" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie2' - 17: U SHORT_BINSTRING 'F1402' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (c)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000007 34 sha1:6696fa0434fadb645687c74c8561f0f55fce1fd6 +extension "" +obj 0000000000000007 34 sha1:5118b5ada1ab6ce6852eef392212ea9300ac6f06 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'c2.10' + 24: U SHORT_BINSTRING 'e2.10' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1466,20 +953,8 @@ obj 0000000000000007 34 sha1:6696fa0434fadb645687c74c8561f0f55fce1fd6 txn 0285cbadff92c6b3 " " user "user2.11" description "step 2.11" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieG' - 17: U SHORT_BINSTRING 'Q5FM3' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 34 sha1:14e8b27b3bc8bf5b4d86ca162cc1c912a29a1c05 +extension "" +obj 0000000000000004 34 sha1:14e8b27b3bc8bf5b4d86ca162cc1c912a29a1c05 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -1495,27 +970,15 @@ obj 0000000000000002 34 sha1:14e8b27b3bc8bf5b4d86ca162cc1c912a29a1c05 txn 0285cbae04444500 " " user "user2.12" description "step 2.12" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie7' - 17: U SHORT_BINSTRING '2EFQB' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (g)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000009 34 sha1:db269b90a0df33aa7d411c672f93fb7d86bbdb87 +extension "" +obj 0000000000000007 34 sha1:60c6854b7737f5bbc7569c8b6a694910ed370ccc 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'g2.12' + 24: U SHORT_BINSTRING 'e2.12' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1524,27 +987,15 @@ obj 0000000000000009 34 sha1:db269b90a0df33aa7d411c672f93fb7d86bbdb87 txn 0285cbae08f5c34c " " user "user2.13" description "step 2.13" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieS' - 17: U SHORT_BINSTRING 'YS9KO' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 34 sha1:4b28f9e97ff4f61f3dfba30b9f7aceb4913215ce +extension "" +obj 0000000000000002 34 sha1:5d89801d288c5238f605784e7c71b2d61598d362 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd2.13' + 24: U SHORT_BINSTRING 'c2.13' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1553,27 +1004,15 @@ obj 0000000000000002 34 sha1:4b28f9e97ff4f61f3dfba30b9f7aceb4913215ce txn 0285cbae0da74199 " " user "user2.14" description "step 2.14" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie5' - 17: U SHORT_BINSTRING 'RF8GX' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (c)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000007 34 sha1:c3eabecf360015b4b7555abc3a0dc0cea77fe7ed +extension "" +obj 0000000000000007 34 sha1:0a27d685d77313b1a2b7c3478888a98798847077 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'c2.14' + 24: U SHORT_BINSTRING 'e2.14' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1582,27 +1021,15 @@ obj 0000000000000007 34 sha1:c3eabecf360015b4b7555abc3a0dc0cea77fe7ed txn 0285cbae1258bfe6 " " user "user2.15" description "step 2.15" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie0' - 17: U SHORT_BINSTRING 'H70PM' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:25ffaeb6090581d72bce075a765d3cf4198af90f +extension "" +obj 0000000000000002 34 sha1:ae3975f91e13b4ec60e6810006be027149642926 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e2.15' + 24: U SHORT_BINSTRING 'c2.15' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1611,27 +1038,15 @@ obj 0000000000000006 34 sha1:25ffaeb6090581d72bce075a765d3cf4198af90f txn 0285cbae170a3e33 " " user "user2.16" description "step 2.16" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-generator' - 19: q BINPUT 2 - 21: U SHORT_BINSTRING 'zodb/py2 (f)' - 35: U SHORT_BINSTRING 'x-cookieO' - 46: U SHORT_BINSTRING 'MJ5PG' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000001 34 sha1:e934438dede49d14ee2d1d2afa8fa18774547764 +extension "" +obj 0000000000000009 34 sha1:0f253572ebd02bdd419b287ac75113bc8213ef83 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'f2.16' + 24: U SHORT_BINSTRING 'b2.16' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1640,20 +1055,8 @@ obj 0000000000000001 34 sha1:e934438dede49d14ee2d1d2afa8fa18774547764 txn 0285cbae1bbbbc80 " " user "user2.17" description "step 2.17" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie1' - 17: U SHORT_BINSTRING 'RAZ4V' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (b)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000008 34 sha1:9961f82b3f01204f80efbb3b62a2b98d9d3202fa +extension "" +obj 0000000000000009 34 sha1:9961f82b3f01204f80efbb3b62a2b98d9d3202fa 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -1669,27 +1072,15 @@ obj 0000000000000008 34 sha1:9961f82b3f01204f80efbb3b62a2b98d9d3202fa txn 0285cbae206d3acc " " user "user2.18" description "step 2.18" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieR' - 17: U SHORT_BINSTRING 'KE39A' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 34 sha1:5294513e1e00c7de532f4f90068b2147bf87f673 +extension "" +obj 0000000000000007 34 sha1:00cce52a596916011d8935c4e53dcb04510af2d9 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd2.18' + 24: U SHORT_BINSTRING 'e2.18' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1698,20 +1089,8 @@ obj 0000000000000002 34 sha1:5294513e1e00c7de532f4f90068b2147bf87f673 txn 0285cbae251eb919 " " user "user2.19" description "step 2.19" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie8' - 17: U SHORT_BINSTRING '1SBCJ' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:523ec17c6d74016e3464d52bb7c7b7baa4b82a20 +extension "" +obj 0000000000000007 34 sha1:523ec17c6d74016e3464d52bb7c7b7baa4b82a20 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 @@ -1727,27 +1106,15 @@ obj 0000000000000006 34 sha1:523ec17c6d74016e3464d52bb7c7b7baa4b82a20 txn 0285cbae29d03766 " " user "user2.20" description "step 2.20" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieJ' - 17: U SHORT_BINSTRING 'EAIKM' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (b)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000008 34 sha1:14b17f0e944432782cb270205b2e96948d112619 +extension "" +obj 0000000000000004 34 sha1:9cf3a2031b4e9af93e7fd40e58dbceedc753b7d6 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'b2.20' + 24: U SHORT_BINSTRING 'd2.20' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1756,27 +1123,15 @@ obj 0000000000000008 34 sha1:14b17f0e944432782cb270205b2e96948d112619 txn 0285cbae2e81b5b3 " " user "user2.21" description "step 2.21" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieM' - 17: U SHORT_BINSTRING 'ESSAD' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:27744ea516240e0d00be75f26af9698f842bdda5 +extension "" +obj 0000000000000008 34 sha1:330a069412b6cfb00a43da001b600b7f6b2380e6 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e2.21' + 24: U SHORT_BINSTRING 'a2.21' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1785,27 +1140,15 @@ obj 0000000000000006 34 sha1:27744ea516240e0d00be75f26af9698f842bdda5 txn 0285cbae33333400 " " user "user2.22" description "step 2.22" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieH' - 17: U SHORT_BINSTRING 'DL5OC' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:a3f303ddd4f2fb9369f6fbbb38ae8030f7f8188d +extension "" +obj 0000000000000001 34 sha1:bf2d0d8f4cd81e1592001eb3c265a22894cdb51b 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e2.22' + 24: U SHORT_BINSTRING 'f2.22' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1814,27 +1157,15 @@ obj 0000000000000006 34 sha1:a3f303ddd4f2fb9369f6fbbb38ae8030f7f8188d txn 0285cbae37e4b24c " " user "user2.23" description "step 2.23" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieQ' - 17: U SHORT_BINSTRING 'PBN2A' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (d)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 34 sha1:343bed4b31f4fe69a92ee51d28fd7b9cfd6ecb8b +extension "" +obj 0000000000000006 34 sha1:c47e0f1415e14f41016809ac3a4323745ad79170 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'd2.23' + 24: U SHORT_BINSTRING 'g2.23' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 @@ -1843,84 +1174,36 @@ obj 0000000000000002 34 sha1:343bed4b31f4fe69a92ee51d28fd7b9cfd6ecb8b txn 0285cbae3c963099 " " user "user2.24" description "step 2.24" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookie2' - 17: U SHORT_BINSTRING '0GV0I' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (e)' - 53: u SETITEMS (MARK at 5) - 54: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 34 sha1:8804c60dc27b2e2f6908e2a099a5c5d4b5abc843 +extension "" +obj 0000000000000006 34 sha1:2030a0ccc56595b70aac033e70c899060dbfb0f9 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e2.24' + 24: U SHORT_BINSTRING 'g2.24' 31: q BINPUT 2 33: . STOP highest protocol among opcodes = 2 -txn 0285cbae4147aee6 " " +txn 0285cbae45f92d33 " " user "root2.0\nYour\nMagesty " description "undo 2.0\nmore detailed description\n\nzzz ...\t\t" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieM' - 17: U SHORT_BINSTRING 'OQO01' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (undo AoXLrjfkskw=)' - 69: u SETITEMS (MARK at 5) - 70: . STOP - highest protocol among opcodes = 2 -obj 0000000000000002 from 0285cbae206d3acc +extension "" +obj 0000000000000001 from 0285cbade81b4f33 -txn 0285cbae45f92d33 " " +txn 0285cbae4aaaab80 " " user "root2.1\nYour\nMagesty " description "undo 2.1\nmore detailed description\n\nzzz ...\t\t\t" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieA' - 17: U SHORT_BINSTRING 'VPQ8R' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (undo AoXLrjyWMJk=)' - 69: u SETITEMS (MARK at 5) - 70: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 from 0285cbae33333400 +extension "" +obj 0000000000000006 from 0285cbae37e4b24c -txn 0285cbae4aaaab80 " " +txn 0285cbae4f5c29cc " " user "user" description "cyclic reference" -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-generator' - 19: q BINPUT 2 - 21: U SHORT_BINSTRING 'zodb/py2 (cycle)' - 39: U SHORT_BINSTRING 'x-cookieG' - 50: U SHORT_BINSTRING 'B6FWF' - 57: u SETITEMS (MARK at 5) - 58: . STOP - highest protocol among opcodes = 2 +extension "" obj 0000000000000006 41 sha1:863d327e4b795efff7dff75bb73c0d20ea3981aa 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' @@ -1937,11 +1220,11 @@ obj 0000000000000006 41 sha1:863d327e4b795efff7dff75bb73c0d20ea3981aa highest protocol among opcodes = 2 -txn 0285cbae4f5c29cc " " +txn 0285cbae540da819 " " user "" -description "predelete 6" +description "predelete 1" extension "" -obj 0000000000000000 213 sha1:b44d53e1b6cc465c4ab3ba2a3384a80fbba4eb8a +obj 0000000000000000 213 sha1:ae6936a4e7c9f3f748d8ecf28186e259897651dd 0: \x80 PROTO 2 2: c GLOBAL 'persistent.mapping PersistentMapping' 40: q BINPUT 1 @@ -1956,44 +1239,44 @@ obj 0000000000000000 213 sha1:b44d53e1b6cc465c4ab3ba2a3384a80fbba4eb8a 57: q BINPUT 4 59: ( MARK 60: U SHORT_BINSTRING 'a' - 63: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x05' + 63: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x08' 73: q BINPUT 5 75: c GLOBAL '__main__ Object' 92: q BINPUT 6 94: \x86 TUPLE2 95: Q BINPERSID 96: U SHORT_BINSTRING 'c' - 99: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' + 99: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' 109: q BINPUT 7 111: h BINGET 6 113: \x86 TUPLE2 114: Q BINPERSID 115: U SHORT_BINSTRING 'b' - 118: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x08' + 118: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\t' 128: q BINPUT 8 130: h BINGET 6 132: \x86 TUPLE2 133: Q BINPERSID 134: U SHORT_BINSTRING 'e' - 137: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\n' + 137: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' 147: q BINPUT 9 149: h BINGET 6 151: \x86 TUPLE2 152: Q BINPERSID 153: U SHORT_BINSTRING 'd' - 156: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x02' + 156: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x04' 166: q BINPUT 10 168: h BINGET 6 170: \x86 TUPLE2 171: Q BINPERSID 172: U SHORT_BINSTRING 'g' - 175: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\t' + 175: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x06' 185: q BINPUT 11 187: h BINGET 6 189: \x86 TUPLE2 190: Q BINPERSID 191: U SHORT_BINSTRING 'f' - 194: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x01' + 194: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\n' 204: q BINPUT 12 206: h BINGET 6 208: \x86 TUPLE2 @@ -2003,34 +1286,22 @@ obj 0000000000000000 213 sha1:b44d53e1b6cc465c4ab3ba2a3384a80fbba4eb8a 212: . STOP highest protocol among opcodes = 2 -obj 000000000000000a 32 sha1:35a18f6ce20260014618957689b770b74d7b3c78 +obj 000000000000000a 32 sha1:da5c7f574b5e6a64d0d58314a6939ef761266d41 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 - 24: U SHORT_BINSTRING 'e2*' + 24: U SHORT_BINSTRING 'f2*' 29: q BINPUT 2 31: . STOP highest protocol among opcodes = 2 -txn 0285cbae540da819 " " +txn 0285cbae58bf2666 " " user "root2\nYour\nRoyal\nMagesty' \x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" description "delete 2\nalpha beta gamma'delta\"lambda\n\nqqq ..." -extension - 0: \x80 PROTO 2 - 2: } EMPTY_DICT - 3: q BINPUT 1 - 5: ( MARK - 6: U SHORT_BINSTRING 'x-cookieT' - 17: U SHORT_BINSTRING '4WFSD' - 24: U SHORT_BINSTRING 'x-generator' - 37: q BINPUT 2 - 39: U SHORT_BINSTRING 'zodb/py2 (delete 6)' - 60: u SETITEMS (MARK at 5) - 61: . STOP - highest protocol among opcodes = 2 -obj 0000000000000006 delete +extension "" +obj 0000000000000001 delete
-
- 26 Jun, 2024 1 commit
-
-
Kirill Smelkov authored
I've tried to run `zodb dump --pretty=zpickledis` on wendelin.core test data in WCFS(*) and hit the following failure: (z-dev) kirr@deca:~/src/wendelin/wendelin.core/wcfs/internal/zdata/testdata$ zodb dump --pretty=zpickledis zblk.fs ... obj 0000000000000005 685 sha1:865171b709f575b355afd2cc9e1f32b9781c6510 Traceback (most recent call last): File "/home/kirr/src/wendelin/venv/z-dev/bin/zodb", line 11, in <module> load_entry_point('zodbtools', 'console_scripts', 'zodb')() File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 129, in main return command_module.main(argv) File "<decorator-gen-3>", line 2, in main File "/home/kirr/src/wendelin/venv/z-dev/lib/python2.7/site-packages/golang/__init__.py", line 103, in _ return f(*argv, **kw) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbdump.py", line 341, in main zodbdump(stor, tidmin, tidmax, hashonly, pretty) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbdump.py", line 167, in zodbdump pickletools.dis(dataf, disf) # state File "/usr/lib/python2.7/pickletools.py", line 2005, in dis raise ValueError(errormsg) ValueError: memo key 1 has never been stored into The problem turned out to be due to that state part of zpickle is referring to another object with the same class as already saved in class part of zpickle, so that class was being referred to via GET matching corresponding PUT done in the class part, but our zpickledis handler did not shared the memo in between those two parts and so the GET became unmatched. In more details the problem is illustrated by the following zpickle that corresponds to Object.value referring to the same Object. The first part of zpickle contains class part and refers to __main__.Object global with putting it into memo[1]. The second part of zpickle contains state part and refers to that object by `(Object, 7) PERSID` where Object is retrieved via memo[1] GET: obj 0000000000000007 41 sha1:7108c96ccb9cbeaab1164d533174c300e51309f9 0: \x80 PROTO 2 2: c GLOBAL '__main__ Object' 19: q BINPUT 1 <-- NOTE 21: . STOP highest protocol among opcodes = 2 22: \x80 PROTO 2 24: U SHORT_BINSTRING '\x00\x00\x00\x00\x00\x00\x00\x07' 34: q BINPUT 2 36: h BINGET 1 <-- NOTE 38: \x86 TUPLE2 39: Q BINPERSID 40: . STOP highest protocol among opcodes = 2 To handle such zpickles well we need to share the memo when dumping class and state disassemblies similarly to how ZODB does in its ObjectWriter._dump: https://github.com/zopefoundation/ZODB/blob/5.8.1-0-g72cebe6bc/src/ZODB/serialize.py#L436-L443 Pickletools.dis has explicit support for using shared memo - originally added in https://github.com/python/cpython/commit/62235e701e37 and likely motivated by ZODB use-case. (*) https://lab.nexedi.com/nexedi/wendelin.core/-/blob/07087ec8/wcfs/internal/zdata/testdata/zblk.fs generated by wendelin.core@2c152d41 /reviewed-by @jerome /reviewed-on !28
-
- 16 Feb, 2024 1 commit
-
-
Jérome Perrin authored
Co-authored-by:
Kirill Smelkov <kirr@nexedi.com>
-
- 01 Sep, 2023 3 commits
-
-
Jérome Perrin authored
even though the interface of IStorageRestorable.tpc_begin does not have a "status" argument, it is described in the notes below that the actual implementation uses it: https://github.com/zopefoundation/ZODB/blob/0632974d/src/ZODB/interfaces.py#L950-L956 This is used by FileStorage: https://github.com/zopefoundation/ZODB/blob/0632974d/src/ZODB/FileStorage/format.py#L30-L39 and the storage methods seem to accept this argument: https://github.com/zopefoundation/ZODB/blob/0632974d/src/ZODB/BaseStorage.py#L182 https://github.com/zopefoundation/ZEO/blob/e5637818/src/ZEO/ClientStorage.py#L888 https://lab.nexedi.com/nexedi/neoppod/blob/fd87e153/neo/client/app.py#L473 Propagating the status fixes some cases where restoring commits did not recreate a storage that is byte-to-byte equivalent. This happened with a FileStorage that was packed and contained transactions with "p" status. Co-authored-by:
Kirill Smelkov <kirr@nexedi.com>
Reviewed-on: nexedi/zodbtools!24 -
Kirill Smelkov authored
Until now we were generating only regular transactions with " " status and this does not cover e.g. restore case when it needs to replicate packed transaction: instead of recreating it bit-to-bit exactly as original with "p" status, restore recreates it with " " status, breaking restore promise. Adjusting testdata this way exposes that bug in restore: ======================================== FAILURES ======================================== ________________________________________ test_zodbrestore[!zext] ________________________________________ tmpdir = local('/tmp/pytest-of-kirr/pytest-17/test_zodbrestore__zext_0'), zext = <function _ at 0x7fd6b7a03750> @func def test_zodbrestore(tmpdir, zext): zkind = '_!zext' if zext.disabled else '' # restore from testdata/1.zdump.ok and verify it gives result that is # bit-to-bit identical to testdata/1.fs tdata = dirname(__file__) + "/testdata" @func def _(): zdump = open("%s/1%s.zdump.raw.ok" % (tdata, zkind), 'rb') defer(zdump.close) stor = storageFromURL('%s/2.fs' % tmpdir) defer(stor.close) zodbrestore(stor, zdump) _() zfs1 = readfile(fs1_testdata_py23(tmpdir, "%s/1%s.fs" % (tdata, zkind))) zfs2 = readfile("%s/2.fs" % tmpdir) > assert zfs1 == zfs2 E assert 'FS21\x02\x85...0\x00\x00\xb2' == 'FS21\x02\x85\...0\x00\x00\xb2' E Skipping 49 identical leading characters in diff, use -v to show E Skipping 22871 identical trailing characters in diff, use -v to show E - 0\x00\x00tp\x00\x08\x00\t\x00\x00user0.15step 0.15\x00\x00\x00\x00\x00\x00\x00\x03\x02\x85\xcb\xac\x83i\xd0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00"\x80\x02c__main__ E ? ^ E + 0\x00\x00t \x00\x08\x00\t\x00\x00user0.15step 0.15\x00\x00\x00\x00\x00\x00\x00\x03\x02\x85\xcb\xac\x83i\xd0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00... E E ...Full output truncated (39 lines hidden), use '-vv' to show test_restore.py:53: AssertionError Having "p" transactions in the testdata will also make sure that all tools should handle such transactions well. The problem of restore not handling "p" status properly was reported by Jérome at !24. In the next patch we will fix that problem. /reviewed-by @jerome /reviewed-on !24 -
Kirill Smelkov authored
In 80559a94 ("zodbdump: support --pretty option with a format to show pickles disassembly") we added support for zodbdump --pretty and adjusted files in testdata/ to be named like 1.zdump.{raw,zpickledis}.ok instead of just 1.zdump.ok. However, that renaming and generation of 1.zdump.zpickledis.ok, it seems, were done by hand, because rerunning gen_testdata.py still regenerates old 1.zdump.ok. It seems that during !22 I missed that gen_testdata.py was not updated. -> Fix it. Running gen_testdata.py with py2 and ZODB 5.8.1 regenerates *.fs and *.ok files in testdata/ in exactly the same state they were. /reviewed-by @jerome /reviewed-on !24
-
- 08 Sep, 2022 1 commit
-
-
Kirill Smelkov authored
Penultimate patch needs `bstr` from pygolang to work ok (see kirr/pygolang@c9648c44), but it won't hurt if we merge this without waiting for pygolang bits because without bstr zodbtools continues to work ok on py2, and it will be py3 mode which will not work fully ok. Previous discussions and py3 porting attempts: - !8 (comment 73726) - !12 - conversation from !13 (comment 81553) to !13 (comment 81874) - !19 (comment 129023) - kirr/zodbtools@42799cf6 (comment 166403) /reviewed-by @jerome /reviewed-on !23
-
- 07 Sep, 2022 7 commits
-
-
Kirill Smelkov authored
Empty-range test added in b4824ad5 (analyze: fix ZeroDivisionErrors when report is empty) intended to use 0xffffffffffffffff TID, but used just 'ffffffffffffffff' string instead. It was passing on py2 partly by luck, but on py3 it fails because tidmin type is mismatched: _______________________________ test_zodbanalyze _______________________________ tmpdir = local('/tmp/pytest-of-kirr/pytest-30/test_zodbanalyze0') capsys = <_pytest.capture.CaptureFixture object at 0x7f7bb3f9a4f0> def test_zodbanalyze(tmpdir, capsys): ... # empty range report( > analyze( tfs1, use_dbm=False, delta_fs=False, tidmin="ffffffffffffffff", tidmax=None, ), csv=False, ) zodbtools/test/test_analyze.py:68: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../venv/py3.venv/lib/python3.9/site-packages/decorator.py:232: in fun return caller(func, *(extras + args), **kw) ../../../tools/go/pygolang/golang/__init__.py:103: in _ return f(*argv, **kw) zodbtools/zodbanalyze.py:181: in analyze fsi = fs.iterator(tidmin, tidmax) ../ZODB/src/ZODB/FileStorage/FileStorage.py:1381: in iterator return FileIterator(self._file_name, start, stop) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ZODB.FileStorage.FileStorage.FileIterator object at 0x7f7bb348c6d0> filename = '/tmp/pytest-of-kirr/pytest-30/test_zodbanalyze0/1.fs' start = 'ffffffffffffffff', stop = None, pos = 4 def __init__(self, filename, start=None, stop=None, pos=4): assert isinstance(filename, STRING_TYPES) file = open(filename, 'rb') self._file = file self._file_name = filename if file.read(4) != packed_version: raise FileStorageFormatError(file.name) file.seek(0, 2) self._file_size = file.tell() if (pos < 4) or pos > self._file_size: raise ValueError("Given position is greater than the file size", pos, self._file_size) self._pos = pos > assert start is None or isinstance(start, bytes) E AssertionError ../ZODB/src/ZODB/FileStorage/FileStorage.py:1816: AssertionError ------------------------------ Captured log call ------------------------------- ERROR ZODB.FileStorage:FileStorage.py:480 loading index UnicodeDecodeError: 'ascii' codec can't decode byte 0xb7 in position 25: ordinal not in range(128) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/kirr/src/wendelin/z/ZODB/src/ZODB/FileStorage/FileStorage.py", line 478, in _restore_index info = fsIndex.load(index_name) File "/home/kirr/src/wendelin/z/ZODB/src/ZODB/fsIndex.py", line 138, in load v = unpickler.load() SystemError: <built-in method read of _io.BufferedReader object at 0x7f7bb3df03b0> returned a result with an error set ERROR ZODB.FileStorage:FileStorage.py:480 loading index UnicodeDecodeError: 'ascii' codec can't decode byte 0xb7 in position 25: ordinal not in range(128) ... -> Fix it by preparing tidmin in the test a 8-bytes binary properly.
-
Kirill Smelkov authored
e.g. for ObjectData .hashfunc: In many contexts we need that .hashfunc to be like string, e.g. for accessing hashRegistry by keys. In many other contexts - e.g. when zodbdump input it parsed or emitted, it is more handy to handle it like raw bytes. If we let .hashfunc to be of type str - it breaks the second mode. If of type bytes - it breaks the first mode. And also in many places it is hard to constantly encode/decode str and bytes, especially in the places where an object is sometimes used in strings context, and sometimes in binary context. -> Fix it all in one go by using bytestring type from pygolang, which provides both unicode string and binary semantics simultaneously. This needs bstr from pygolang (see kirr/pygolang@c9648c44), but even if pygolang comes without bstr, with this patch zodbtools continues to work ok on py2 - it will be just py3 mode that won't work. The list of test failures before this patch is provided below: _______________________________ test_zodbanalyze _______________________________ tmpdir = local('/tmp/pytest-of-kirr/pytest-22/test_zodbanalyze0') capsys = <_pytest.capture.CaptureFixture object at 0x7f3de6835c70> def test_zodbanalyze(tmpdir, capsys): tfs1 = fs1_testdata_py23(tmpdir, os.path.join(os.path.dirname(__file__), "testdata", "1.fs")) for use_dbm in (False, True): > report( analyze( tfs1, use_dbm=use_dbm, delta_fs=False, tidmin=None, tidmax=None, ), csv=False, ) zodbtools/test/test_analyze.py:30: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ rep = <zodbtools.zodbanalyze.Report object at 0x7f3de5e16b20>, csv = False def report(rep, csv=False): ... print (fmtp % (t_display, rep.TYPEMAP[t], rep.TYPESIZE[t], pct, rep.TYPESIZE[t] * 1.0 / rep.TYPEMAP[t], > rep.COIDSMAP[t], rep.CBYTESMAP[t], rep.FOIDSMAP.get(t, 0), rep.FBYTESMAP.get(t, 0))) E KeyError: b'persistent.mapping.PersistentMapping' zodbtools/zodbanalyze.py:147: KeyError ____________________________ test_zodbcommit[!zext] ____________________________ zext = <function zext.<locals>._ at 0x7f3deb5c3e50> @func def test_zodbcommit(zext): tmpd = mkdtemp('', 'zodbcommit.') defer(lambda: rmtree(tmpd)) stor = storageFromURL('%s/2.fs' % tmpd) defer(stor.close) head = stor.lastTransaction() # commit some transactions via zodbcommit and verify if storage dump gives # what is expected. t1 = Transaction(z64, ' ', b'user name', b'description ...', zext(dumps({'a': 'b'}, _protocol)), [ ObjectData(p64(1), b'data1', 'sha1', sha1(b'data1')), ObjectData(p64(2), b'data2', 'sha1', sha1(b'data2'))]) t1.tid = zodbcommit(stor, head, t1) t2 = Transaction(z64, ' ', b'user2', b'desc2', b'', [ ObjectDelete(p64(2))]) t2.tid = zodbcommit(stor, t1.tid, t2) buf = BytesIO() zodbdump(stor, p64(u64(head)+1), None, out=buf) dumped = buf.getvalue() > assert dumped == b''.join([_.zdump() for _ in (t1, t2)]) zodbtools/test/test_commit.py:61: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ zodbtools/test/test_commit.py:61: in <listcomp> assert dumped == b''.join([_.zdump() for _ in (t1, t2)]) zodbtools/zodbdump.py:521: in zdump z += obj.zdump() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <zodbtools.zodbdump.ObjectData object at 0x7f3de5d26d90> def zdump(self): data = self.data hashonly = isinstance(data, HashOnly) if hashonly: size = data.size else: size = len(data) > z = b'obj %s %d %s:%s' % (ashex(self.oid), size, self.hashfunc, ashex(self.hash_)) E TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'str' zodbtools/zodbdump.py:569: TypeError _______________________________ test_dumpreader ________________________________ def test_dumpreader(): in_ = b"""\ txn 0123456789abcdef " " user "my name" description "o la-la..." extension "zzz123 def" obj 0000000000000001 delete obj 0000000000000002 from 0123456789abcdee obj 0000000000000003 54 adler32:01234567 - obj 0000000000000004 4 sha1:9865d483bc5a94f2e30056fc256ed3066af54d04 ZZZZ obj 0000000000000005 9 crc32:52fdeac5 ABC DEF! txn 0123456789abcdf0 " " user "author2" description "zzz" extension "qqq" """ r = DumpReader(BytesIO(in_)) > t1 = r.readtxn() zodbtools/test/test_dump.py:78: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ zodbtools/zodbdump.py:443: in readtxn self._badline('unknown hash function %s' % qq(hashfunc)) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <zodbtools.zodbdump.DumpReader object at 0x7f3de5d69cd0> msg = 'unknown hash function "adler32"' def _badline(self, msg): > raise RuntimeError("%s+%d: invalid line: %s (%s)" % (_ioname(self._r), self.lineno, msg, qq(self._line))) E RuntimeError: +7: invalid line: unknown hash function "adler32" ("obj 0000000000000003 54 adler32:01234567 -") zodbtools/zodbdump.py:382: RuntimeError ___________________________ test_zodbrestore[!zext] ____________________________ tmpdir = local('/tmp/pytest-of-kirr/pytest-22/test_zodbrestore__zext_0') zext = <function zext.<locals>._ at 0x7f3de5d6ddc0> @func def test_zodbrestore(tmpdir, zext): zkind = '_!zext' if zext.disabled else '' # restore from testdata/1.zdump.ok and verify it gives result that is # bit-to-bit identical to testdata/1.fs tdata = dirname(__file__) + "/testdata" @func def _(): zdump = open("%s/1%s.zdump.raw.ok" % (tdata, zkind), 'rb') defer(zdump.close) stor = storageFromURL('%s/2.fs' % tmpdir) defer(stor.close) zodbrestore(stor, zdump) > _() zodbtools/test/test_restore.py:49: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../venv/py3.venv/lib/python3.9/site-packages/decorator.py:232: in fun return caller(func, *(extras + args), **kw) ../../../tools/go/pygolang/golang/__init__.py:103: in _ return f(*argv, **kw) zodbtools/test/test_restore.py:48: in _ zodbrestore(stor, zdump) zodbtools/zodbrestore.py:39: in zodbrestore txn = zr.readtxn() zodbtools/zodbdump.py:443: in readtxn self._badline('unknown hash function %s' % qq(hashfunc)) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <zodbtools.zodbdump.DumpReader object at 0x7f3de5d79e20> msg = 'unknown hash function "sha1"' def _badline(self, msg): > raise RuntimeError("%s+%d: invalid line: %s (%s)" % (_ioname(self._r), self.lineno, msg, qq(self._line))) E RuntimeError: /home/kirr/src/wendelin/z/zodbtools/zodbtools/test/testdata/1_!zext.zdump.raw.ok+5: invalid line: unknown hash function "sha1" ("obj 0000000000000000 61 sha1:664e6de0f153d8eaeda638d616a320c6e3c5feb1") zodbtools/zodbdump.py:382: RuntimeError
-
Kirill Smelkov authored
Zodbcommit reads input in zodbdump format from stdin and then uses zodbdump.DumpReader to parser that input. The parser works on binary data. However zodbcommit, was preparing that input data mixing bytes and strings, which is failing on py3: (py3.venv) kirr@deca:~/src/wendelin/z/zodbtools$ zodb commit 1.fs 00 Ignoring index for /home/kirr/src/wendelin/z/zodbtools/1.fs aaa Traceback (most recent call last): File "/home/kirr/src/wendelin/venv/py3.venv/bin/zodb", line 33, in <module> sys.exit(load_entry_point('zodbtools', 'console_scripts', 'zodb')()) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 129, in main return command_module.main(argv) File "/home/kirr/src/wendelin/venv/py3.venv/lib/python3.9/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), **kw) File "/home/kirr/src/tools/go/pygolang/golang/__init__.py", line 103, in _ return f(*argv, **kw) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 222, in main zin += sys.stdin.read() TypeError: can't concat str to bytes -> Fix it by reading stdin in binary mode. No test currently as zodbcommit.main is not covered by tests (hopefully yet). -
Kirill Smelkov authored
pickletools.dis, which is used to handle --pretty=zpickledis (*), expects output stream be text-like, not binary. We were passing a binary stream to it. As the result pickle disassembly was failing on py3: _______________________ test_zodbdump[!zext-zpickledis] ________________________ tmpdir = local('/tmp/pytest-of-kirr/pytest-11/test_zodbdump__zext_zpickledis0') zext = <function zext.<locals>._ at 0x7f538b508670>, pretty = 'zpickledis' @mark.parametrize('pretty', ('raw', 'zpickledis')) def test_zodbdump(tmpdir, zext, pretty): tdir = dirname(__file__) zkind = '_!zext' if zext.disabled else '' tfs1 = fs1_testdata_py23(tmpdir, '%s/testdata/1%s.fs' % (tdir, zkind)) stor = FileStorage(tfs1, read_only=True) with open('%s/testdata/1%s.zdump.%s.ok' % (tdir, zkind, pretty), 'rb') as f: dumpok = f.read() out = BytesIO() > zodbdump(stor, None, None, pretty=pretty, out=out) zodbtools/test/test_dump.py:48: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ zodbtools/zodbdump.py:165: in zodbdump pickletools.dis(dataf, disf) # class _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ pickle = <_io.BytesIO object at 0x7f538b577130> out = <_io.BytesIO object at 0x7f538b49f8b0>, memo = {}, indentlevel = 4 annotate = 0 def dis(pickle, out=None, memo=None, indentlevel=4, annotate=0): """Produce a symbolic disassembly of a pickle...""" ... for opcode, arg, pos in genops(pickle): if pos is not None: > print("%5d:" % pos, end=' ', file=out) E TypeError: a bytes-like object is required, not 'str' /usr/lib/python3.9/pickletools.py:2450: TypeError -> Fix it by letting pickletools.dis to emit its output to StringIO instead of BytesIO. (*) see 80559a94 "zodbdump: support --pretty option with a format to show pickles disassembly" -
Kirill Smelkov authored
FileStorage/py2 uses `FS21` magic in file header, whereas FileStorage/py3 uses `FS30` magic: https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/_compat.py#L39 https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/_compat.py#L74 And if, upon opening the database, file magic does not match to what ZODB expects, open is rejected: https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/FileStorage/FileStorage.py#L88 https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/FileStorage/FileStorage.py#L1625-L1630 This is done with the idea for a database, that was written from Python2, to be rejected to be opened from Python3 and vice-versa because strings/bytes semantics changed in between py23. As the result, many zodbtools tests currently fail on py3 when they try to access prepared FileStorage database in testdata, because that database was originally prepared on py2. Here is, for example, how test_zodbdump fails: ___________________________ test_zodbdump[zext-raw] ____________________________ zext = <function zext.<locals>._ at 0x7f28530bf9d0>, pretty = 'raw' @mark.parametrize('pretty', ('raw', 'zpickledis')) def test_zodbdump(zext, pretty): tdir = dirname(__file__) zkind = '_!zext' if zext.disabled else '' > stor = FileStorage('%s/testdata/1%s.fs' % (tdir, zkind), read_only=True) zodbtools/test/test_dump.py:41: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ZODB/src/ZODB/FileStorage/FileStorage.py:315: in __init__ self._pos, self._oid, tid = read_index( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ file = <_io.BufferedReader name='/home/kirr/src/wendelin/z/zodbtools/zodbtools/test/testdata/1.fs'> name = '/home/kirr/src/wendelin/z/zodbtools/zodbtools/test/testdata/1.fs' index = <ZODB.fsIndex.fsIndex object at 0x7f2852fee2b0>, tindex = {} stop = b'\xff\xff\xff\xff\xff\xff\xff\xff' ltid = b'\x00\x00\x00\x00\x00\x00\x00\x00', start = 4 maxoid = b'\x00\x00\x00\x00\x00\x00\x00\x00', recover = 0, read_only = True def read_index(file, name, index, tindex, stop=b'\377'*8, ltid=z64, start=4, maxoid=z64, recover=0, read_only=0): """Scan the file storage and update the index.""" ... if file_size: if file_size < start: raise FileStorageFormatError(file.name) seek(0) if read(4) != packed_version: > raise FileStorageFormatError(name) E ZODB.FileStorage.FileStorage.FileStorageFormatError: /home/kirr/src/wendelin/z/zodbtools/zodbtools/test/testdata/1.fs ../ZODB/src/ZODB/FileStorage/FileStorage.py:1630: FileStorageFormatError Since zodbtools primarily work on raw data without decoding stored pickles, unlike Zope or ERP5, it should not be a problem for zodbtools to work on py3 with the database that was prepared on py2. -> Adjust all tests to use FileStorage data generated on the fly based on original files in testdata/ but with FileStorage header being rewritten to match current python. -
Kirill Smelkov authored
A counterpart to readfile - to write a file instead of reading it. We will need this function in the next patch.
-
Kirill Smelkov authored
Soon we will need to use it not only from test_restore.py
-
- 29 Mar, 2022 1 commit
-
-
Jérome Perrin authored
Showing pickle disassembly can sometimes be useful to analyse details of the pickle content. We realized that in some data structures used in ERP5 the same string was saved multiple times in the same pickle and by using the exact same string (ie. for which `s1 is s2` is True), the pickle will have the string only once and pickles are a bit smaller. For more reference, the context was nexedi/erp5!1560 (comment 154825) This introduces a new --pretty option that we will be able to extend later with more output formats. Co-authored-by:
Kirill Smelkov <kirr@nexedi.com>
Reviewed-on: nexedi/zodbtools!22
-
- 01 Apr, 2021 1 commit
-
-
Jérome Perrin authored
@kirr wrote (nexedi/zodbtools!19 (comment 129442)) For the reference - contrary to ZODB5, restore tests on ZODB4 are currently [broken](https://nexedijs.erp5.net/#/test_result_module/20210317-B3AC205A/2). Restored file is not bit-to-bit identical to the original. The problem is that on commit/restore, we need to save user/description/extension. For extension `zodbdump.Transaction` provides .extension_bytes, which ZODB5 uses to save its raw copy. However ZODB4 goes through `.extension` and pickles it: https://lab.nexedi.com/nexedi/zodbtools/blob/129afa67/zodbtools/zodbdump.py#L425-453 https://github.com/zopefoundation/ZODB/blob/4/src/ZODB/BaseStorage.py#L220-L240 This leads to unpickle-repickle round-trip and different extension being committed on restore: ```diff diff --git a/1zdump b/2zdump index 5033bc1..a3a32aa 100644 --- a/1zdump +++ b/2zdump @@ -10,7 +10,7 @@ q^A. txn 0285cbac3d0369e6 " " user "user0.0" description "step 0.0" -extension "\x80\x02}q\x01(U\tx-cookieSU\x05RF9IEU\vx-generatorq\x02U\fzodb/py2 (f)u." +extension "}q\x01(U\tx-cookieSU\x05RF9IEU\vx-generatorU\fzodb/py2 (f)u." obj 0000000000000000 98 sha1:eba252d1984f975ecb636bc1b3a89c953dd20527 ... ``` What might save us is to somehow in Transaction.extension returns a dict-subclass object that is somehow pickled to the exact bytes remembered when it was created. However, after briefly checking, I could not find a mechanism to do so yet... @jerome wrote (nexedi/zodbtools!19 (comment 129479)) @kirr we already have pytest fixtures to test differently depending on whether the ZODB version has support for extension_bytes, so what about using it in the test and testing restoring the extension bytes version of the dump only for ZODB5 ? @kirr wrote (nexedi/zodbtools!19 (comment 129482)) @jerome, yes we have this, but I believe we should actually fix zodbrestore to be reliable whatever ZODB is used. For ZODB5 it works. For ZODB4-wc2 we can adjust ZODB code to use extension_bytes similarly to how ZODB5 does. But unpatched ZODB4 is currently out of luck. As it was decided that Nexedi will use both ZODB4 and ZODB4-wc2, I think we should fix zodbrestore to work on all those versions to be reliable. /cc @tomo @kirr: -> No universal ZODB4 fix for now (this would require to monkey patch ZODB in several places), so mark "restore with extension" test as xfail similarly to how we already do for "dump with extension" test. This brings -ZODB4 and -ZODB4-wc2 tests back to PASS state. Even though on ZODB4 extension is restored not bit-to-bit exactly, it is restored to be the same dictionary equal to what was used to produce the dump. Not ideal, but still not loosing the information in practice. One more reason to switch to ZODB5...
-
- 16 Mar, 2021 2 commits
-
-
Kirill Smelkov authored
In the previous patch we taught object copy handler to report more details, but it was still incomplete - the error was missing details about which operation was run - commit, or restore of particular transaction. Noting that it can be also noted that other errors reported from that function lack such context. -> So fix it universally, at least for zodbcommit for now: set top-level runctx to topic of what we are doing, and use that runctx when generating errors. Runctx describes what we are running, and could be also later used for logging and tracing. That's why it is called runctx instead of just errctx for "error context". TODO currently it is only exceptions that we explicitly raise which get the context. If an exception is raised by something that we call - the context won't be added. It would be good to later rework error handling and append such context for any raised error. Defer and https://lab.nexedi.com/kirr/go123/blob/863c4602/xerr/__init__.py has something preliminary for this. The particular error when restoring a missing object copy becomes ValueError: /tmp/demo002868462/δ0285cbac75555580/δ.fs: restore 0285cbacb70a3db3 @0285cbacb258bf66: object 0000000000000003: copy from @0285cbac70a3d733: no data instead of older ValueError: /tmp/demo358030847/δ0285cbac75555580/δ.fs: object 0000000000000003: copy from @0285cbac70a3d733: no data /reviewed-by @jerome /reviewed-on nexedi/zodbtools!20
-
Kirill Smelkov authored
When zodbdump input says to copy an object, we first load that object. However if object does not exist loadBefore raises POSKeyError, and when object at copied-from revision was deleted loadBefore returns None. -> Handle that explicitly to provide failure details to the user, so that instead of cryptic === RUN TestLoad/δstart=0285cbac75555580 Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 133, in <module> main() File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 129, in main return command_module.main(argv) File "<decorator-gen-6>", line 2, in main File "/home/kirr/src/tools/go/pygolang/golang/__init__.py", line 103, in _ return f(*argv, **kw) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbrestore.py", line 94, in main zodbrestore(stor, asbinstream(sys.stdin), _) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbrestore.py", line 43, in zodbrestore zodbcommit(stor, at, txn) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 122, in zodbcommit _() File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 91, in _ data, _, _ = stor.loadBefore(obj.oid, p64(u64(obj.copy_from)+1)) TypeError: 'NoneType' object is not iterable xtesting.go:483: /tmp/demo009767458/δ0285cbac75555580/δ.fs: zpyrestore: exit status 1 it fails with something more understandable: === RUN TestLoad/δstart=0285cbac75555580 Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 133, in <module> main() File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodb.py", line 129, in main return command_module.main(argv) File "<decorator-gen-6>", line 2, in main File "/home/kirr/src/tools/go/pygolang/golang/__init__.py", line 103, in _ return f(*argv, **kw) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbrestore.py", line 94, in main zodbrestore(stor, asbinstream(sys.stdin), _) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbrestore.py", line 43, in zodbrestore zodbcommit(stor, at, txn) File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 129, in zodbcommit _() File "/home/kirr/src/wendelin/z/zodbtools/zodbtools/zodbcommit.py", line 97, in _ (stor.getName(), ashex(obj.oid), ashex(obj.copy_from))) ValueError: /tmp/demo358030847/δ0285cbac75555580/δ.fs: object 0000000000000003: copy from @0285cbac70a3d733: no data xtesting.go:483: /tmp/demo358030847/δ0285cbac75555580/δ.fs: zpyrestore: exit status 1 For the implementation it would be easier to use loadAt (https://github.com/zopefoundation/ZODB/pull/323), but we don't have that yet. /reviewed-by @jerome /reviewed-on nexedi/zodbtools!20
-
- 15 Mar, 2021 4 commits
-
-
Kirill Smelkov authored
Suggessted by @jerome here: nexedi/zodbtools!19 (comment 129181) Co-authored-with: Jérome Perrin <jerome@nexedi.com> /reviewed-on nexedi/zodbtools!19
-
Kirill Smelkov authored
Zodbrestore is long-coming counterpart to zodbdump. Implementation is internally based on reworked zodbcommit. For FileStorage restored database is verified via test to be bit-to-bit identical to the original. For NEO it won't be exactly the case, as NEO does not implement IStorageRestoreable: there is only tpc_begin(tid=...) but no restore(). /helped-by @jerome /reviewed-on nexedi/zodbtools!19
-
Kirill Smelkov authored
This current serial will not be needed on new codepaths to be added to zodbcommit in the next patch. -> Move the computation to function to trigger it only from places where knowing current serial is actually needed. /reviewed-by @jerome /reviewed-on nexedi/zodbtools!19
-
Kirill Smelkov authored
Two-phase commit protocol assumes that after tpc_begin, it will be either successful tpc_vote + tpc_finish, or tpc_abort. We were not calling tpc_abort on an error, potentially leaving storage in "commit is in progress" state on an error. /reviewed-by @jerome /reviewed-on nexedi/zodbtools!19
-
- 10 Mar, 2021 2 commits
-
-
Kirill Smelkov authored
Nexedi stack is dropping support for that old ZODB version - see e.g. - nexedi/slapos@70d05199 - nexedi/neoppod@3a8f6f03 - nexedi/wendelin.core@0802da2b Regarding test/gen_testdata.py: even though ZODB4 uses zodbpickle, and so should be able to load pickles encoded with protocol 3 even on python2, in practice it does not work so well: ZODB4 tests fail if I set --- a/src/ZODB/_compat.py +++ b/src/ZODB/_compat.py @@ -34,7 +34,7 @@ HIGHEST_PROTOCOL = cPickle.HIGHEST_PROTOCOL IMPORT_MAPPING = {} NAME_MAPPING = {} - _protocol = 1 + _protocol = 3 FILESTORAGE_MAGIC = b"FS21" else: # Python 3.x: can't use stdlib's pickle because -> so continue to preserve protocol < 3 when generating the test database for compatibility - now with ZODB4/py2. /reviewed-by @jerome /reviewed-on nexedi/zodbtools!18
-
Kirill Smelkov authored
The patch that provides raw-extension functionality was merged into ZODB 5.6: https://github.com/zopefoundation/ZODB/commit/2f8cc67a3ba3 So when testing with ZODB5 >= 5.6 the tests will excercise code path that uses txn.extension_bytes, and when testing with ZODB4 the tests will excercise code path that work-arounds lack of txn.extension_bytes. /reviewed-by @jerome /reviewed-on nexedi/zodbtools!18
-
- 02 Nov, 2020 1 commit
-
-
Kirill Smelkov authored
Nxdtest[1] is tox-like tool to run tests under Nexedi testing infrastructure. [1] https://lab.nexedi.com/nexedi/nxdtest /reviewed-on nexedi/zodbtools!17
-
- 30 Apr, 2020 1 commit
-
-
Kirill Smelkov authored
Flushing changes from yet another attempt. Still not completely there yet, but closer. Reviewed-by: @jerome Reviewed-on: nexedi/zodbtools!16
-
- 29 Apr, 2020 6 commits
-
-
Kirill Smelkov authored
ashex gives bytes, whereas reference_tid was str.
-
Kirill Smelkov authored
The sequence cannot be randomly accessed, e.g. In [5]: d = {1:2} In [6]: kv = d.keys() In [7]: kv Out[7]: dict_keys([1]) In [8]: kv[0] --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-8-643f90e1910b> in <module>() ----> 1 kv[0] TypeError: 'dict_keys' object is not subscriptable -> Use list(dict.keys()) in places where we need random access. -
Kirill Smelkov authored
Otherwise it breaks with str on py3: In [1]: from io import BytesIO In [2]: BytesIO("abc") --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-2-52a130edd46d> in <module>() ----> 1 BytesIO("abc") TypeError: a bytes-like object is required, not 'str' -
Kirill Smelkov authored
Zodbdump format is text-binary and is saved into files opened in binary mode. -> We have to emit bytes - not strings - into it, since otherwise on Python3 it would break. This needs qq support from pygolang[1] to be able to use qq with both string and bytestring format, e.g. for "hello %s" % qq(name), and b"hello %s" % qq(name) to give the same output irregardless of whether name is str or bytes. [1] nexedi/pygolang!1
-
Kirill Smelkov authored
Zodbdump format is already described as semi text-binary in top-level zodbdump.py documentation. However zdump() docstring was referring to it as "text". Fix it and use binary to handle places where zdump is loaded/saved.
-
Kirill Smelkov authored
%r has different output for strings and bytes on python3: In [1]: a = 'hello' In [2]: b = b'hello' In [3]: repr(a) Out[3]: "'hello'" In [4]: repr(b) Out[4]: "b'hello'" -> Use qq whose output is stable irregardless of whether input is string or bytes.
-
- 13 Mar, 2020 1 commit
-
-
Kirill Smelkov authored
zodbinfo: Provide "head" as command to query DB head; Turn "last_tid" into deprecated alias for head Similarly to go version: kirr/neo@151d8b79.
-
- 14 Feb, 2020 1 commit
-
-
Kirill Smelkov authored
Starting with upcoming ZODB 5.5.2 ZODB tries to preserve `extension_bytes` transaction metadata property in the raw form as it was stored on disk in the database: https://github.com/zopefoundation/ZODB/commit/2f8cc67a However now when running test/gen_testdata.py with ZODB with that patch (and gen_testdata.py refuses to work if it detects that ZODB does not properly supports .extension_bytes property because we want it to be present in the generated test database [1,2]) it now breaks: $ ./gen_testdata.py Traceback (most recent call last): File "./gen_testdata.py", line 230, in <module> main() File "./gen_testdata.py", line 224, in main gen_testdb("%s.fs" % dbname, zext=zext) File "./gen_testdata.py", line 194, in gen_testdb stor.tpc_begin(txn) File "/home/kirr/src/wendelin/z/ZODB/src/ZODB/BaseStorage.py", line 193, in tpc_begin ext = transaction.extension_bytes AttributeError: 'Transaction' object has no attribute 'extension_bytes' The breakage is because, as specified in ZODB interfaces[3,4], storage requires ZODB.IStorageTransactionMetaData, not transaction.ITransaction instance gen_testdata.py was using. The script used to work before just by luck. The fix is to convert transaction instance into storage transaction metadata object for the place where we talk to storage at raw level. HOWEVER, when checking regenerated database and its dump I noticed: ZODB >= 5.4.0 uses pickle protocol 3 on both python2 and python3 https://github.com/zopefoundation/ZODB/commit/12ee41c4 In other words it saves e.g. OID of an object as pickle binary, which decodes as bytes on py3 and zodbpickle.binary on py2 when decoding via zodbpickle. However it will result in *DecodeError* when decoding on py2 with standard pickle module. The latter means that ZODB3 will _fail_ to load data from test database, because ZODB3 - contrary to ZODB4 and ZODB5 - uses std pickle module, not zodbpickle. We still care about ZODB3 and in particular it is included into zodbtools test matrix: https://lab.nexedi.com/nexedi/zodbtools/blob/7bc0385e/tox.ini#L9-14 so we cannot break it. -> Temporarily patch ZODB at runtime to make sure it emits data with older protocol and without using zodbpickle.binary for oid, so that generated test database could be loaded on ZODB3 as well. gen_testdata.py now works with latest ZODB, but produces exactly the same bit-to-bit output as before. [1] https://lab.nexedi.com/nexedi/zodbtools/blob/7bc0385e/zodbtools/test/gen_testdata.py#L215-217 [2] https://lab.nexedi.com/nexedi/zodbtools/blob/7bc0385e/zodbtools/test/testutil.py#L31-63 [3] https://github.com/zopefoundation/ZODB/blob/5.5.1-35-gb5895a5c2/src/ZODB/interfaces.py#L815-L818 [4] https://github.com/zopefoundation/ZODB/blob/5.5.1-35-gb5895a5c2/src/ZODB/interfaces.py#L538-L575 /reviewed-on !15
-
- 09 Jul, 2019 1 commit
-
-
Kirill Smelkov authored
-> Use .[test] to refer to them. https://stackoverflow.com/a/41398850/9456786 /reviewed-by @jerome /reviewed-on nexedi/zodbtools!14
-
- 03 Jun, 2019 1 commit
-
-
Kirill Smelkov authored
@jerome, I was trying to make zodbtools work with Python3 and along that road picked some bits of your work from nexedi/zodbtools!12. At present the migration to Python3 is not complete, and even though now I have the answer to how handle strings in both python2/3 in compatible and reasonable way (I can share details if you are interested), I have to put that work on hold for some time and use https://pypi.org/project/pep3134 directly in wcfs tests, since getting all string details right, even after figuring on how to do it, will take time. Anyway the bits presented here should be ready for master and could be merged now. Could you please have a look? Thanks beforehand, Kirill /reviewed-on nexedi/zodbtools!13
-
- 24 May, 2019 2 commits
-
-
Kirill Smelkov authored
Zodbdump format is mixed text+binary so dumping to unicode stdout won't work. Based on patch by Jérome Perrin.
-
Kirill Smelkov authored
Because on Py3: def test_dumpreader(): in_ = b"""\ txn 0123456789abcdef " " user "my name" description "o la-la..." extension "zzz123 def" obj 0000000000000001 delete obj 0000000000000002 from 0123456789abcdee obj 0000000000000003 54 adler32:01234567 - obj 0000000000000004 4 sha1:9865d483bc5a94f2e30056fc256ed3066af54d04 ZZZZ obj 0000000000000005 9 crc32:52fdeac5 ABC DEF! txn 0123456789abcdf0 " " user "author2" description "zzz" extension "qqq" """ r = DumpReader(BytesIO(in_)) t1 = r.readtxn() assert isinstance(t1, Transaction) > assert t1.tid == '0123456789abcdef'.decode('hex') E AttributeError: 'str' object has no attribute 'decode' test/test_dump.py:77: AttributeError Based on patch by Jérome Perrin.
-
