An error occurred fetching the project authors.
- 10 Jul, 2019 7 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
The latter state is the only valid state in wcfs mode when pagefault handler sees already loaded page in fileh->pagemap.
-
Kirill Smelkov authored
-
- 09 Jul, 2019 6 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 08 Jul, 2019 1 commit
-
-
Kirill Smelkov authored
Else on-heap allocated RAM object is leaked. Fixes e.g. the following error on ASAN: Direct leak of 56 byte(s) in 1 object(s) allocated from: #0 0x7fc9ef390518 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9518) #1 0x555ca792f309 in zalloc include/wendelin/utils.h:67 #2 0x555ca7935f9a in ram_limited_new bigfile/tests/../../t/t_utils.c:35 #3 0x555ca793a0ba in test_file_access_synthetic bigfile/tests/test_virtmem.c:292 #4 0x555ca7967bc4 in main bigfile/tests/test_virtmem.c:1121 #5 0x7fc9ef0e909a in __libc_start_main ../csu/libc-start.c:308
-
- 07 Jul, 2019 1 commit
-
-
Kirill Smelkov authored
-
- 24 Oct, 2017 1 commit
-
-
Kirill Smelkov authored
Relicense to GPLv3+ with wide exception for all Free Software / Open Source projects + Business options. Nexedi stack is licensed under Free Software licenses with various exceptions that cover three business cases: - Free Software - Proprietary Software - Rebranding As long as one intends to develop Free Software based on Nexedi stack, no license cost is involved. Developing proprietary software based on Nexedi stack may require a proprietary exception license. Rebranding Nexedi stack is prohibited unless rebranding license is acquired. Through this licensing approach, Nexedi expects to encourage Free Software development without restrictions and at the same time create a framework for proprietary software to contribute to the long term sustainability of the Nexedi stack. Please see https://www.nexedi.com/licensing for details, rationale and options.
-
- 10 Jan, 2017 1 commit
-
-
Kirill Smelkov authored
This allows writeout code not to scan whole pagemap to find dirty pages to write out, which should be faster. But more importantly iterating whole pagemap on writeout would become unsafe, when in upcoming patch storeblk() will be called with virt_lock released: because there pagemap could be modified e.g. due to processing other read accesses. So maintain fileh->dirty_pages list and use it when we need to go through dirtied pages. Updates: #6
-
- 09 Jan, 2017 1 commit
-
-
Kirill Smelkov authored
Currently fileh_dirty_writeout() writes page via storeblk() in order - - those with lower ->f_pgoffset are stored first. This happens because current fileh_dirty_writeout() iterates whole pagemap to find dirty pages and pagemap iteration is ordered by f_pgoffset. In upcoming patch we'll rework writeout code not to iterate through whole pagemap, but only through dirty pages. However the property that pages are emitted in canonical order is useful, so let's make sure via tests this will stay preserved: In mkdirty2() we modify pages in 2, 0 order, but the latter code checks (via storeblk_trace()) they were actually stored in 0, 2 order.
-
- 15 Dec, 2015 1 commit
-
-
Kirill Smelkov authored
loadblk() calls are potentially slow and external code that serve the cal can take other locks in addition to virtmem lock taken by virtmem subsystem. If that "other locks" are also taken before external code calls e.g. fileh_invalidate_page() in different codepath a deadlock can happen, e.g. T1 T2 page-access invalidation-from-server received V -> loadblk Z <- ClientStorage.invalidateTransaction() Z -> zeo.load V <- fileh_invalidate_page The solution to avoid deadlock is to call loadblk() with virtmem lock released and upon loadblk() completion recheck virtmem data structures carefully. To make that happen: - new page state is introduces: PAGE_LOADING (file content loading is in progress) - virtmem releases virt_lock before calling loadblk() when serving pagefault - because loading is now done with virtmem lock released, now: 1. After loading completes we need to recheck fileh/vma data structures The recheck is done in full - vma_on_pagefault() just asks its driver (see VM_RETRY and VM_HANDLED codes) to retry handling the fault completely. This should work as the freshly loaded page was just inserted into fileh->pagemap and should be found there in the cache on next lookup. On the other hand this also works correctly, if there was concurrent change - e.g. vma was unmapped while we were loading the data - in that case the fault will be also processed correctly - but loaded data will stay in fileh->pagemap (and if not used will be evicted as not-needed eventually by RAM reclaim). 2. Similar to retrying mechanism is used for cases when two threads concurrently access the same page and would both try to load corresponding block - only one thread issues the actual loadblk() and another waits for load to complete with polling and VM_RETRY. 3. To correctly invalidate loading-in-progress pages another new page state is introduced: PAGE_LOADING_INVALIDATED (file content loading was in progress while request to invalidate the page came in) which fileh_invalidate_page() uses to propagate invalidation message to loadblk() caller. 4. Blocks loading can now happen in parallel with other block loading and other virtmem operations - e.g. invalidation. For such cases tests are added to test_thread.py 5. virtmem lock now becomes just regular lock, instead of being previously recursive. For virtmem lock to be recursive was needed for cases, when code under loadblk() could trigger other virtmem calls, e.g. due to GC and calling another VMA dtor that would want to lock virtmem, but virtmem lock was already held. This is no longer needed. 6. To catch double faults we now cannot use just on static variable in_on_pagefault. That variable thus becomes thread-local. 7. Old test in test_thread to "test that access vs access don't overlap" no longer holds true - and is thus removed. /cc @Tyagov, @klaus
-
- 17 Aug, 2015 1 commit
-
-
Kirill Smelkov authored
FileH is a handle representing snapshot of a file. If, for a pgoffset, fileh already has loaded page, but we know the content of the file has changed externally after loading has been done, we need to propagate to fileh that such-and-such page should be invalidated (and reloaded on next access). This patch introduces fileh_invalidate_page(fileh, pgoffset) to do just that. In the next patch we'll use this facility to propagate invalidations of ZBlk ZODB objects to virtmem subsystem. NOTE Since invalidation removes "dirtiness" from a page state, several subsequent invalidations can make a fileh completely non-dirty (invalidating all dirty page). Previously fileh->dirty was just a one bit, so we needed to improve how we track dirtiness. One way would be to have a dirty list for fileh pages and operate on that. This has advantage to even optimize dirty pages processing like fileh_dirty_writeout() where we currently scan through all fileh pages just to write only PAGE_DIRTY ones. Another simpler way is to make fileh->dirty a counter and maintain that. Since we are going to move virtmem subsystem back into the kernel, here, a simpler less-intrusive approach is used.
-
- 03 Apr, 2015 1 commit
-
-
Kirill Smelkov authored
Does similar things to what kernel does - users can mmap file parts into address space and access them read/write. The manager will be getting invoked by hardware/OS kernel for cases when there is no page loaded for read, or when a previousle read-only page is being written to. Additionally to features provided in kernel, it support to be used to store back changes in transactional way (see fileh_dirty_writeout()) and potentially use huge pages for mappings (though this is currently TODO)
-