- 01 Nov, 2020 2 commits
-
-
Kirill Smelkov authored
Corresponds to NEO/py commit 3efbbfe3 ("master: automatically discard feeding cells that get out-of-date").
-
Kirill Smelkov authored
NEO 1.10 * tag 'v1.10': (55 commits) Release version 1.10 Maximize resiliency by taking into account the topology of storage nodes storage: also commit updated cell TID at each replicated chunk of 'obj' records storage: skip useless work when unlocking transactions qa: flush logs at the end of each test when -L is not used qa: add a log in case that a mysterious bug happens again storage: clarify log about data deletion of discarded cells debug: new example to run the profiler for 1 minute mysql: fix replication of big oids (> 16M) tests/cluster: speedup waiting a bit protocol: update packet docstrings Bump protocol version protocol: a single byte is more than enough to encode enums protocol: small cleanup in packet registration Optimize resumption of replication by starting from a greater TID importer: update comment about a workaround for ZODB3 Micro-optimization of p64/u64 qa: add a log in testBackupNodeLost for easier debugging Document that the bug when checking replicas may also cause the master to crash storage: stop logging 'Abort TXN' for txn that have been locked storage: split _migrate2() for reusable _alterTable() qa: new testStorageUpgrade qa: update testStorageUpgrade data for what is not automatically upgraded qa: original data for the future testStorageUpgrade sqlite: fix indexes of upgraded db importer: fix NameError when recovering during tpc_finish fixup! importer: fetch and process the data to import in a separate process Serialize empty transaction extension with an empty string client: fix partial import from a source storage qa: give a title to subprocesses of functional tests importer: give a title to the 'import' and 'writeback' subprocesses importer: fetch and process the data to import in a separate process importer: new option to write back new transactions to the source database importer: log when the transaction index for FileStorage DB is built importer: open imported zodb in read-only whenever possible fixup! mysql: fix remaining places where a server disconnection was not catched fixup! storage: speed up replication by sending bigger network packets mysql: do not full-scan for duplicates of big oids if deduplication is disabled mysql: fix remaining places where a server disconnection was not catched fixup! Add support for custom compression levels importer: reenable compression by default qa: review testImporter qa: remove a few uses of 'chr' Fix a few issues with ZODB5 importer: small code cleanup in speedupFileStorageTxnLookup patch importer: do not trigger speedupFileStorageTxnLookup uselessly Add support for custom compression levels setup: update MANIFEST.in importer: do not checksum data twice client: store uncompressed if compressed size is equal fixup! master: automatically discard feeding cells that get out-of-date master: automatically discard feeding cells that get out-of-date qa: remove useless indentation in testSafeTweak bench: new option to mesure ZEO perfs in matrix test bench: reduce number of partitions in matrix test storage: fix replication of creation undone
-
- 30 Oct, 2020 1 commit
-
-
Kirill Smelkov authored
-
- 27 Oct, 2020 9 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 26 Oct, 2020 6 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 21 Oct, 2020 2 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
For example NEOCluster needs to shutdown gracefully, else there are processes left for e.g. storage nodes and they dump somthing as below on the terminal after tests completion: === RUN TestLoad 2020/10/21 14:33:00 zodb: FIXME: open ../zodb/storage/fs1/testdata/1.fs: raw cache is not ready for invalidations -> NoCache forced === RUN TestLoad/py I: runneo.py: /tmp/neo445013868/1: Started master(s): 127.0.0.1:24661 WARNING: This is not the recommended way to import data to NEO: you should use the Importer backend instead. NEO also does not implement IStorageRestoreable interface, which means that undo information is not preserved when using this tool: conflict resolution could happen when undoing an old transaction. Migrating from ../zodb/storage/fs1/testdata/1.fs to 127.0.0.1:24661 Migration done in 0.19877 --- PASS: TestLoad (0.75s) --- PASS: TestLoad/py (0.74s) PASS ok lab.nexedi.com/kirr/neo/go/neo 0.749s (neo) (z-dev) (g.env) kirr@deco:~/src/neo/src/lab.nexedi.com/kirr/neo/go/neo$ Traceback (most recent call last): File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/tests/functional/__init__.py", line 182, in start getattr(neo.scripts, command).main() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/scripts/neostorage.py", line 66, in main app.run() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 147, in run self._run() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 178, in _run self.doOperation() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 266, in doOperation poll() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 87, in _poll self.em.poll(1) File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 155, in poll self._poll(blocking) File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 253, in _poll timeout_object.onTimeout() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 259, in onTimeout on_timeout() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/manager.py", line 207, in _deferredCommit self.commit() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/manager.py", line 193, in commit self._commit() File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/sqlite.py", line 90, in _commit retry_if_locked(self.conn.commit) File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/sqlite.py", line 45, in retry_if_locked return f(*args) OperationalError: disk I/O error
-
- 20 Oct, 2020 2 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 19 Oct, 2020 2 commits
-
-
Kirill Smelkov authored
This fixes === RUN TestLoad/py xtesting.go:297: load 0285cbacc06d3a4c:0000000000000007: returned err unexpected: have: <nil> want: neo://1@127.0.0.1:27145: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c xtesting.go:297: load 0285cbacc06d3a4c:0000000000000007: returned tid unexpected: 0285cbacc06d3a4c ; want: 0285cbacc06d3a4c xtesting.go:297: load 0285cbacc06d3a4c:0000000000000007: returned buf != nil xtesting.go:297: load 0285cbad858bf2e6:0000000000000006: returned err unexpected: have: <nil> want: neo://1@127.0.0.1:27145: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6 xtesting.go:297: load 0285cbad858bf2e6:0000000000000006: returned tid unexpected: 0285cbad858bf2e6 ; want: 0285cbad858bf2e6 xtesting.go:297: load 0285cbad858bf2e6:0000000000000006: returned buf != nil xtesting.go:315: load 7fffffffffffffff:0000000000000006: returned err unexpected: have: <nil> want: neo://1@127.0.0.1:27145: load 7fffffffffffffff:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6 xtesting.go:315: load 7fffffffffffffff:0000000000000006: returned tid unexpected: 0285cbad858bf2e6 ; want: 0285cbad858bf2e6 xtesting.go:315: load 7fffffffffffffff:0000000000000006: returned buf != nil xtesting.go:315: load 7fffffffffffffff:0000000000000007: returned err unexpected: have: <nil> want: neo://1@127.0.0.1:27145: load 7fffffffffffffff:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c xtesting.go:315: load 7fffffffffffffff:0000000000000007: returned tid unexpected: 0285cbacc06d3a4c ; want: 0285cbacc06d3a4c xtesting.go:315: load 7fffffffffffffff:0000000000000007: returned buf != nil
-
Kirill Smelkov authored
NEO/py sends all zeros for empty data. Without this fix tests fail: --- FAIL: TestLoad/py (0.74s) xtesting.go:297: load 0285cbacc06d3a4c:0000000000000007: returned err unexpected: have: neo://1@127.0.0.1:25910: load 0285cbacc06d3a4c:0000000000000007: data corrupt: checksum mismatch want: neo://1@127.0.0.1:25910: load 0285cbacc06d3a4c:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c xtesting.go:297: load 0285cbad858bf2e6:0000000000000006: returned err unexpected: have: neo://1@127.0.0.1:25910: load 0285cbad858bf2e6:0000000000000006: data corrupt: checksum mismatch want: neo://1@127.0.0.1:25910: load 0285cbad858bf2e6:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6 xtesting.go:315: load 7fffffffffffffff:0000000000000006: returned err unexpected: have: neo://1@127.0.0.1:25910: load 7fffffffffffffff:0000000000000006: data corrupt: checksum mismatch want: neo://1@127.0.0.1:25910: load 7fffffffffffffff:0000000000000006: 0000000000000006: object was deleted @0285cbad858bf2e6 xtesting.go:315: load 7fffffffffffffff:0000000000000007: returned err unexpected: have: neo://1@127.0.0.1:25910: load 7fffffffffffffff:0000000000000007: data corrupt: checksum mismatch want: neo://1@127.0.0.1:25910: load 7fffffffffffffff:0000000000000007: 0000000000000007: object was deleted @0285cbacc06d3a4c
-
- 18 Oct, 2020 16 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-