- 03 Jan, 2021 1 commit
-
-
Kirill Smelkov authored
Rename the interface that specifies methods to be implemented by trace receivers. Reason is that in the next patch we'll need to expose "Tracer" as the tracing networker itself.
-
- 29 Dec, 2020 3 commits
-
-
Kirill Smelkov authored
Generate g for ~ Go 1.16beta1 (go1.16beta1-34-g4fd9455882) Compared to Go1.15 there are only cosmetic changes: --- a/zruntime_g_go1.15.go +++ b/zruntime_g_go1.16.go @@ -1,6 +1,6 @@ // Code generated by g_typedef; DO NOT EDIT. -// +build go1.15,!go1.16 +// +build go1.16,!go1.17 package xruntime @@ -144,7 +144,7 @@ type gobuf struct { ctxt unsafe.Pointer ret uintreg lr uintptr - bp uintptr // for GOEXPERIMENT=framepointer + bp uintptr // for framepointer-enabled architectures } type funcval struct { fn uintptr @@ -158,6 +158,8 @@ type timer struct { // Timer wakes up at when, and then at when+period, ... (period > 0 only) // each time calling f(arg, now) in the timer goroutine, so f must be // a well-behaved function and not block. + // + // when must be positive on an active timer. when int64 period int64 f func(interface{}, uintptr) Regenerated files stay without changes for Go1.15 and previous releases.
-
Kirill Smelkov authored
When a virtnet network - e.g. lonet - is explicitly used in tests, the open and close sequence is easy to implement: network = lonet.Join(...) defer network.Close() hostα = network.NewHost("α") defer hostα.Close() hostβ = network.NewHost("β") defer hostβ.Close() ... However in other scenarios - for example when the code wants to call xnet.Networker factory that creates a networker out of several potential choices according to given parameters, only interface for network access-point (e.g. Host in lonet/virtnet case) is returned. The network itself (virtnet.SubNetwork) is not returned to the user. As it is this makes it impossible to close the network and release its resources. Instead of adding complexity to such "create-networker" functions to e.g. change Close of returned networker to also close subnetwork, or return both networker and a separate close func, let's teach virtnet to close SubNetwork automatically when last host of this subnetwork is closed. Make it opt-in since this behaviour is not universally wanted. This way a factory that is asked to connect to network as e.g. a node on lonet, could do the following: func neonet.Join(...) xnet.Networker { ... // join lonet "<net>" as host "<node>" network = lonet.Join("<net>") host = network.NewHost("<node>") network.AutoClose() // host.Close will close network return host } and network will be closed on host.Close() call. -> Add SubNetwork.AutoClose to schedule Close to be called after last host on the subnetwork is closed. -> Mirror this change in virtnet part of lonet.py
-
Kirill Smelkov authored
Even though there is no need to manually release resources for NetPlain and NetTLS, for other networkers - for example lonet - releasing resources is needed. In general programs cannot always use lonet explicitly, as there might be a factory function that creates a Networker according to given parameters. -> Adjust Networker interface to include general Close. -> Adjust NetPlain to follow added "interrupt-on-close" semantic for uniformity.
-
- 17 Dec, 2020 3 commits
-
-
Kirill Smelkov authored
Similarly to what we do in package xio (see 7ad867a3 "xio: Add Reader/Writer/... interfaces - io analogs that add support for contexts"), add utilities to bind xnet.Listener to context (giving net.Listener), and, on the other hand, adapt uncontextified net.Listener to xnet.Listener that handles cancellation. This utilities are needed because a lot of third-party code wants to work with net.Listener interface only. Contrary to xio, we can make it working reliably in both ways (because Accept already buffers accepting connections starting at OS level).
-
Kirill Smelkov authored
In the previous patch xnet.Networker was changed: Listen now accepts ctx and returns xnet.Listener instead of net.Listener. -> Adapt the code all around to that.
-
Kirill Smelkov authored
We already handle cancellation in Dial, but Accepting was out of luck until now. This makes it more difficult for clients to implement and wrap acceptors where they need to handle cancellations. This also makes it possible for a test or program to get stuck in Accept loop if it is not careful enough to manually handle ctx cancel around Accept calls. -> Fix it in one place - here, in xnet - so that users are offloaded from all this and can just call Accept(ctx) and rely on underlying implementation to handle ctx cancel. This patch: - introduces xnet.Listener interface, which is like net.Listener, but Accept goes with ctx argument. - changes Networker.Listen signature to return xnet.Listener instead of net.Listener. While we are here - changing it - also add ctx argument to Listen call itself. - Adds listenerCtx - which, given net.Listener, provides xnet.Listener by wrapping some logic around original. - Adapts NetPlain, NetTLS and NetTrace to provide updated interface. We'll fix up everything in other packages to match/use updated interface in the next patch.
-
- 01 Dec, 2020 1 commit
-
-
Kirill Smelkov authored
See nexedi/pygolang@21756bd3.
-
- 18 Oct, 2020 1 commit
-
-
Kirill Smelkov authored
Go1.14.10 and Go1.15.3 changed internal structure of g: https://github.com/golang/go/commit/878da0bf881b (go1.14) https://github.com/golang/go/commit/bf79f91d3dc4 (go1.15)
-
- 16 Sep, 2020 1 commit
-
-
Kirill Smelkov authored
A shorthand for when exception context is just formatted string. Simplifies func doSomething(path string) { defer exc.Context(func() interface{} { return fmt.Sprintf("doing something %s", path) })() ... to func doSomething(path string) { defer exc.Contextf("doing something %s", path) ...
-
- 15 Sep, 2020 1 commit
-
-
Kirill Smelkov authored
`go mod init` + build/test that fetched in dependencies.
-
- 27 May, 2020 4 commits
-
-
Kirill Smelkov authored
It was address-of error (the type) instead of &err.
-
Kirill Smelkov authored
https://github.com/golang/go/issues/36503 https://github.com/golang/go/issues/36448 They are there for 6 months already without being considered for real and so the progress is very unlikely, but still...
-
Kirill Smelkov authored
Generate g for ~ Go 1.15beta1 (go1.14beta1-1729-gc0e8e405c0) Compared to Go1.14 there are no changes. Regenerated files stay without changes for Go1.14 and previous releases.
-
Kirill Smelkov authored
-
- 03 Apr, 2020 1 commit
-
-
Kirill Smelkov authored
Provide race.Enabled for programs to know whether they were built with race detector or not. Code originates from https://lab.nexedi.com/kirr/wendelin.core/tree/25c3184d/wcfs/internal/race See also: https://github.com/golang/go/issues/36477.
-
- 16 Jan, 2020 1 commit
-
-
Kirill Smelkov authored
So that Go1.13 and golang.org/x/xerrors understand the wrapping: https://blog.golang.org/go1.13-errors Preserve Cause for compatibility with github.com/pkg/errors.
-
- 13 Jan, 2020 1 commit
-
-
Kirill Smelkov authored
Add xsync.WorkGroup that is similar to https://godoc.org/golang.org/x/sync/errgroup, but amends it with a bit better (imho) design where work context is explicitly passed into worker (see also [1]). This mirrors sync.WorkGroup that we already have at Pygolang side [2]. [1] https://github.com/golang/go/issues/34510 [2] https://pypi.org/project/pygolang/#concurrency
-
- 26 Dec, 2019 6 commits
-
-
Kirill Smelkov authored
I need this to support sysread(/head/watch) cancellation in WCFS filesystem [1,2,3]. [1] wendelin.core@b17aeb8c [2] wendelin.core@f05271b1 [3] wendelin.core@5ba816da
-
Kirill Smelkov authored
Rework Pipe to create (xio.Reader, xio.Writer) instead of (io.Reader, io.Writer) and teach xio.Pipe{Reader,Writer} to accept ctx argument and handle cancellation. I need this to support sysread(/head/watch) cancellation in WCFS filesystem [1,2,3]. See also [4]. [1] wendelin.core@b17aeb8c [2] wendelin.core@f05271b1 [3] wendelin.core@5ba816da [4] https://github.com/golang/go/issues/20280
-
Kirill Smelkov authored
Just make it compile by trivially adapting the code to refer thing from io package. Change license reference to the place where Go license lives in go123 (LICENSE-go, added in ad78da1b "xflag: Add counting flag from go.git internals").
-
Kirill Smelkov authored
This is pristine copy of pipe.go + tests from Go go1.14beta1-24-g075c20cea8 . We'll adapt copied code to xio and add cancellation support in further commits.
-
Kirill Smelkov authored
Done toprovide uniform API in xio package. Compensate for API change in places where xio.CountReader was used.
-
Kirill Smelkov authored
In many cases IO needs to be able to be canceled. For example in WCFS filesystem I need to cancel handling sysread(/head/watch) when FUSE INTERRUPT request comes in [1,2,3]. The READ handler for /head/watch inside WCFS is interally implemented via io.Pipe which does not provide read/write cancellattion besides "destructive" close. Standard Go answer for cancellations is via contexts. So as a first step let's add corresponding interfaces - xio.Reader, xio.Writer etc - that are io analogs that add support for contexts. For compatibility with legacy code that work with only io.X (e.g. only with io.Reader), in spirit of [4], add BindCtx which binds xio.X instance with context and converts it into io.X. Add WithCtx - corresponding inverse operation that converts io.X back into xio.X and for general io.X adds cancellation handling on a best-effort basis. [1] wendelin.core@b17aeb8c [2] wendelin.core@f05271b1 [3] wendelin.core@5ba816da [4] https://github.com/golang/go/issues/20280 [5] https://github.com/golang/go/issues/16522
-
- 20 Dec, 2019 1 commit
-
-
Kirill Smelkov authored
Generate g for ~ Go 1.14beta1 (go1.14beta1-10-g5c6f42773c) Compared to Go1.13 there are several changed to g, _defer and timer related to: - non-cooperative preemption (https://golang.org/issue/10958, https://golang.org/issue/24543); - inlined defers (https://golang.org/issue/14939, https://golang.org/issue/34481); - timers rework to be integrated with network poller (https://golang.org/issue/6239, https://golang.org/issue/27707): Regenerated files stay without changes for Go1.13 and previous releases. ---- 8< ---- diff --git a/zruntime_g_go1.13.go b/zruntime_g_go1.14.go index 76851fd..19cebae 100644 --- a/zruntime_g_go1.13.go +++ b/zruntime_g_go1.14.go @@ -1,6 +1,6 @@ // Code generated by g_typedef; DO NOT EDIT. -// +build go1.13,!go1.14 +// +build go1.14,!go1.15 package xruntime @@ -32,12 +32,25 @@ type g struct { schedlink guintptr waitsince int64 // approx time when the g become blocked waitreason waitReason // if status==Gwaiting + preempt bool // preemption signal, duplicates stackguard0 = stackpreempt + preemptStop bool // transition to _Gpreempted on preemption; otherwise, just deschedule + preemptShrink bool // shrink stack at synchronous safe point + + // asyncSafePoint is set if g is stopped at an asynchronous + // safe point. This means there are frames on the stack + // without precise pointer information. + asyncSafePoint bool + paniconfault bool // panic (instead of crash) on unexpected fault address - preemptscan bool // preempted g does scan for gc gcscandone bool // g has scanned stack; protected by _Gscan bit in status - gcscanvalid bool // false at start of gc cycle, true if G has not run since last scan; TODO: remove? throwsplit bool // must not split stack + // activeStackChans indicates that there are unlocked channels + // pointing into this goroutine's stack. If true, stack + // copying needs to acquire channel locks to protect these + // areas of the stack. + activeStackChans bool + raceignore int8 // ignore race detection events sysblocktraced bool // StartTrace has emitted EvGoInSyscall about this goroutine sysexitticks int64 // cputicks when syscall has returned (for tracing) @@ -76,18 +89,37 @@ type _panic struct { argp unsafe.Pointer // pointer to arguments of deferred call run during panic; cannot move - known to liblink arg interface{} // argument to panic link *_panic // link to earlier panic + pc uintptr // where to return to in runtime if this panic is bypassed + sp unsafe.Pointer // where to return to in runtime if this panic is bypassed recovered bool // whether this panic is over aborted bool // the panic was aborted + goexit bool } type _defer struct { siz int32 // includes both arguments and results started bool heap bool + // openDefer indicates that this _defer is for a frame with open-coded + // defers. We have only one defer record for the entire frame (which may + // currently have 0, 1, or more defers active). + openDefer bool sp uintptr // sp at time of defer - pc uintptr - fn *funcval + pc uintptr // pc at time of defer + fn *funcval // can be nil for open-coded defers _panic *_panic // panic that is running defer link *_defer + + // If openDefer is true, the fields below record values about the stack + // frame and associated function that has the open-coded defer(s). sp + // above will be the sp for the frame, and pc will be address of the + // deferreturn call in the function. + fd unsafe.Pointer // funcdata for the function associated with the frame + varp uintptr // value of varp for the stack frame + // framepc is the current pc associated with the stack frame. Together, + // with sp above (which is the sp associated with the stack frame), + // framepc/sp can be used as pc/sp pair to continue a stack trace via + // gentraceback(). + framepc uintptr } type gobuf struct { // The offsets of sp, pc, and g are known to (hard-coded in) libmach. @@ -114,8 +146,10 @@ type funcval struct { fn uintptr } type timer struct { - tb *timersBucket // the bucket the timer lives in - i int // heap index + // If this timer is on a heap, which P's heap it is on. + // puintptr rather than *p to match uintptr in the versions + // of this struct defined in other packages. + pp puintptr // Timer wakes up at when, and then at when+period, ... (period > 0 only) // each time calling f(arg, now) in the timer goroutine, so f must be @@ -125,6 +159,12 @@ type timer struct { f func(interface{}, uintptr) arg interface{} seq uintptr + + // What to set the when field to in timerModifiedXX status. + nextwhen int64 + + // The status field holds one of the values below. + status uint32 } type guintptr uintptr type puintptr uintptr
-
- 28 Jul, 2019 1 commit
-
-
Kirill Smelkov authored
Generate g for ~ Go 1.13beta2 (go1.13beta1-91-ga25c2878c7). Compared to Go1.12 _defer gains `heap bool` member. No other shanges.
-
- 19 May, 2019 2 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Pygolang does it. However my comment on Go issue tracker about this topic got no feedback at all: https://github.com/golang/go/issues/30694#issuecomment-483820400
-
- 16 Apr, 2019 1 commit
-
-
Kirill Smelkov authored
As of Go1.12 the check for whether particular trace event is enabled or not is inlined into caller. For example: ---- 8< ---- (connection.go) //trace:event traceMsgSendPre(l *NodeLink, connId uint32, msg proto.Msg) func (link *NodeLink) sendMsg(connId uint32, msg proto.Msg) error { traceMsgSendPre(link, connId, msg) buf := msgPack(connId, msg) return link.sendPkt(buf) } ---- 8< ---- (ztrace.go) // traceevent: traceMsgSendPre(l *NodeLink, connId uint32, msg proto.Msg) type _t_traceMsgSendPre struct { tracing.Probe probefunc func(l *NodeLink, connId uint32, msg proto.Msg) } var _traceMsgSendPre *_t_traceMsgSendPre func traceMsgSendPre(l *NodeLink, connId uint32, msg proto.Msg) { if _traceMsgSendPre != nil { _traceMsgSendPre_run(l, connId, msg) } } func _traceMsgSendPre_run(l *NodeLink, connId uint32, msg proto.Msg) { for p := _traceMsgSendPre; p != nil; p = (*_t_traceMsgSendPre)(unsafe.Pointer(p.Next())) { p.probefunc(l, connId, msg) } } ---- 8< ---- (connection.o) TEXT lab·nexedi·com∕kirr∕neo∕go∕neo∕neonet·(*NodeLink).sendMsg(SB), ABIInternal, $40-48 // connection.go:1368 // MOVQ (TLS), CX (stack growth prologue) // CMPQ SP, 16(CX) // JLS 177 // SUBQ $40, SP // MOVQ BP, 32(SP) (BP save) // LEAQ 32(SP), BP (BP init) // FUNCDATA $0, gclocals·dbae954985b577993f0eafab1347dd21(SB) (args) FUNCDATA $1, gclocals·f6bd6b3389b872033d462029172c8612(SB) (locals) FUNCDATA $3, gclocals·47f0f3578d0f16ef296c57af2564832c(SB) PCDATA $2, $0 // connection.go:1369 PCDATA $0, $0 XCHGL AX, AX CMPQ lab·nexedi·com∕kirr∕neo∕go∕neo∕neonet·_traceMsgSendPre(SB), $0 <-- inlined traceMsgSendPre JNE pc130 // ztrace.go:50 <-- pc44: MOVL connId+56(SP), AX // connection.go:1371 <-- normal path MOVL AX, (SP) MOVQ msg+64(SP), AX MOVQ AX, 8(SP) PCDATA $2, $1 PCDATA $0, $1 MOVQ msg+72(SP), AX PCDATA $2, $0 MOVQ AX, 16(SP) CALL lab·nexedi·com∕kirr∕neo∕go∕neo∕neonet·msgPack(SB) PCDATA $2, $1 MOVQ 24(SP), AX PCDATA $2, $2 // connection.go:1372 PCDATA $0, $2 MOVQ link+48(SP), CX PCDATA $2, $1 MOVQ CX, (SP) PCDATA $2, $0 MOVQ AX, 8(SP) CALL lab·nexedi·com∕kirr∕neo∕go∕neo∕neonet·(*NodeLink).sendPkt(SB) PCDATA $2, $1 MOVQ 24(SP), AX MOVQ 16(SP), CX PCDATA $0, $3 MOVQ CX, _r2+80(SP) PCDATA $2, $0 MOVQ AX, _r2+88(SP) // MOVQ 32(SP), BP (BP restore) // ADDQ $40, SP (SP restore) RET pc130: <-- slow path (tracepoint attached) PCDATA $2, $1 // ztrace.go:51 PCDATA $0, $0 MOVQ link+48(SP), AX PCDATA $2, $0 MOVQ AX, (SP) MOVL connId+56(SP), CX MOVL CX, 8(SP) MOVQ msg+64(SP), DX MOVQ DX, 16(SP) PCDATA $2, $3 MOVQ msg+72(SP), BX PCDATA $2, $0 MOVQ BX, 24(SP) CALL lab·nexedi·com∕kirr∕neo∕go∕neo∕neonet·_traceMsgSendPre_run(SB) JMP pc44 // NOP (stack growth) // PCDATA $0, $-1 // connection.go:1368 // PCDATA $2, $-1 // CALL runtime.morestack_noctxt(SB) // JMP 0
-
- 12 Apr, 2019 1 commit
-
-
Kirill Smelkov authored
This continues b46436bd (tracing/runtime: Try to add support for Go1.12): Move `+build ...` constraint in assembly files to the top, as suggested by Russ Cox: https://github.com/golang/go/issues/31410#issuecomment-482599134 Then go stops complaining about it. The build constraint had no effect previously, but it was not caught because getg is used only on race builds and I was testing on amd64 only, which is practically almost the only ISA supported by race detector as of Go1.11 .
-
- 11 Apr, 2019 2 commits
-
-
Kirill Smelkov authored
Generate g for Go 1.12; no changes here compared to Go 1.11, however go1.12 build build fails because of +build in .s file: # lab.nexedi.com/kirr/go123/tracing/internal/xruntime ./runtime_g_amd64.s:3:1: +build comment must appear before package clause and be followed by a blank line Build problem reported upstream: https://github.com/golang/go/issues/31410
-
Kirill Smelkov authored
Pygolang, starting from 0.0.0.dev8, deprecates @method(cls) in favour of @func(cls): pygolang@942ee900
-
- 24 Jan, 2019 1 commit
-
-
Kirill Smelkov authored
- sqliteutil moved to -> sqlitex (https://github.com/crawshaw/sqlite/commit/e68ccf033b) - Poll moved from sqlite to sqlitex (https://github.com/crawshaw/sqlite/commit/1967107395) - Poll now takes full context, not only done channel (https://github.com/crawshaw/sqlite/commit/93b88b3c5c)
-
- 17 Jan, 2019 1 commit
-
-
Kirill Smelkov authored
-
- 16 Jan, 2019 1 commit
-
-
Kirill Smelkov authored
To -> pygolang@32a21d5b
-
- 19 Nov, 2018 1 commit
-
-
Kirill Smelkov authored
Pygolang, starting from 0.0.0.dev5 provides working defer.
-
- 06 Jul, 2018 1 commit
-
-
Kirill Smelkov authored
-
- 03 Jul, 2018 1 commit
-
-
Kirill Smelkov authored
We will hopefully need it in the future to test external python code similarly to when testing in-process go code with the help of tracepoints. See module description for details (tracetest is not yet done). There is no yet Go client API for interacting with such-traced python program. For the record: manually inspecting traced python process could be done via e.g. socat EXEC:"./pyruntraced 3 neo.trace.py -- ../../../t/backup-play/N1-writer",fdin=3,fdout=3 TCP:localhost:8888 and telnetting to localhost:8888 from another xterm.
-
- 20 Jun, 2018 1 commit
-
-
- 19 Jun, 2018 1 commit
-
-
Kirill Smelkov authored
This patch brings lonet implementation in Python with the idea that Go and Python programs could interoperate via lonet network and thus mixed Go/Python application cluster could be tested. Implementation quality is lower compared to Go version, but still it should be more or less ok.
-