How Go detects struct copies with sync.noCopy

# How Go detects struct copies with sync.noCopy Source: [https://func25.dev/posts/go-sync-nocopy/](https://func25.dev/posts/go-sync-nocopy/) If you have read the source code of the`sync`package, you may have noticed that several structs contain an unusual field of type`noCopy`, such as`sync\.Mutex`,`sync\.Once`, and`sync\.Map`: go ``` type Mutex struct { _ noCopy ... } type Once struct { _ noCopy ... } type Map struct { _ noCopy ... } ``` `noCopy`is a special marker for types that must not be copied after their first use\. But the marker itself is only an empty struct with two empty methods: go ``` type noCopy struct{} func (*noCopy) Lock() {} func (*noCopy) Unlock() {} ``` Despite the method names, there is no lock and nothing gets unlocked\. But nothing here stops us from copying the value\. This post explains what can break after a copy, why`noCopy`needs these two methods, and how to add the same marker to your own types\. ## 1\. What noCopy does and does not do The`noCopy`marker does not add any special rule to the Go compiler\. You can still copy a`sync\.Map`after it has been used: go ``` var a sync.Map a.Store("k", 1) b := a // copying a sync.Map ``` The assignment copies all the fields from`a`into`b`, exactly as it would for any other struct value\. The code still passes`go build`because the compiler gives no special meaning to the name`noCopy`or to its`Lock`and`Unlock`methods\. It turns out that the warning comes from a separate tool called`go vet`\. This static\-analysis command is included with Go and reports suspicious code that the compiler still accepts\. When`go vet`checks the same assignment, it reports: ``` assignment copies lock value to b: sync.Map contains sync.noCopy ``` The phrase`copies lock value`comes from the purpose of the`copylocks`checker in`go vet`\. It was created to report copies of values that contain a lock, such as`sync\.Mutex`, because copying a lock after it has been used also copies its internal state\. The behavior is easy to reproduce with a regular struct that contains`sync\.Mutex`: go ``` type Counter struct { mu sync.Mutex value int } func main() { var a Counter b := a _ = b.value } ``` ``` $ go vet ./... main.go:12:7: assignment copies lock value to b: example.com/nocopy-repro.Counter contains sync.Mutex ``` `Counter`contains an actual mutex, so copying the outer struct also copies the mutex state\. ## 2\. How go vet and noCopy work So`go vet`produces this warning through its`copylocks`checker and this checker does not search for a field named`noCopy`\. It uses the following rule while inspecting the copied type and its fields: go ``` if types.Implements(types.NewPointer(typ), lockerType) && !types.Implements(typ, lockerType) { return []string{typ.String()} } ``` The checker starts with the type being copied \(e\.g\.,`Counter`\) and asks whether its pointer implements`sync\.Locker`while its value does not\. If the type is a struct and the answer is no, the checker applies the same rule to the type of every field, including fields inside nested structs\. For`Counter`, this search finds`sync\.Mutex`in the`mu`field\. COPYLOCKS RULE\*T implements sync\.LockerT does not implement sync\.LockerCountersync\.Onceno matchcheck field muno matchcheck field \_ noCopysync\.MutexRULE MATCHESnoCopyRULE MATCHESThe checker applies the same rule to the outer type and its field types\.For`sync\.Once`, it finds`noCopy`in the`\_ noCopy`field\. This recursive search is why the warning can report that an outer struct contains a lock or`noCopy`\. That is why the`noCopy`marker’s methods are named`Lock`and`Unlock`\. As mentioned, the rule above checks whether a type implements`sync\.Locker`\. This interface contains exactly two methods: go ``` type Locker interface { Lock() Unlock() } type noCopy struct{} func (*noCopy) Lock() {} func (*noCopy) Unlock() {} ``` These names make the pointer`\*noCopy`implement`sync\.Locker`, while the value`noCopy`does not\. `noCopy`was added to the standard library in 2016 by[Aliaksandr Valialkin](https://github.com/golang/go/commit/c81a3532fea42df33dea54497dfaa96873c2d976), CTO of[VictoriaMetrics](https://github.com/VictoriaMetrics), based on a pattern proposed by[Russ Cox](https://github.com/golang/go/issues/8005#issuecomment-190753527)\. These types were already unsafe to copy\. The change gave`go vet`a way to report copies it had previously missed\. The 2016 implementation had only`Lock`\. In[2018](https://github.com/golang/go/commit/c2eba53e7f80df21d51285879d51ab81bcfbf6bc),`noCopy`gained`Unlock`when`copylocks`switched to checking`sync\.Locker`\. But you may notice that some structs in the`sync`package already contained a`sync\.Mutex`, such as`sync\.Map`: go ``` type Map struct { mu Mutex ... } ``` If`go vet`could already find the mutex, why does`sync\.Map`also need`\_ noCopy`? `sync\.Map`did not need`\_ noCopy`merely to trigger a warning\.`copylocks`could already reach its`mu Mutex`field\. The explicit markers in`sync\.Map`,`sync\.Once`, and`sync\.Mutex`have three purposes\. First,`\_ noCopy`gives each struct an explicit marker, so`copylocks`does not depend on how that struct is implemented\. The checker can find`\_ noCopy`directly instead of finding an internal mutex or relying on the outer type’s`Lock`and`Unlock`methods\. Second, the warning describes the copying restriction more clearly\. The old warning for`sync\.Map`named the mutex found inside it: ``` assignment copies lock value to b: sync.Map contains sync.Mutex ``` The current warning names`noCopy`instead\. The name directly indicates that the outer type must not be copied: ``` assignment copies lock value to b: sync.Map contains sync.noCopy ``` Third, the marker fixes a false negative involving`sync\.Mutex`\. A false negative means the checker should report a copy but does not\. The checker recognized`sync\.Mutex`through its`Lock`and`Unlock`methods, but a new named type does not inherit those methods: go ``` type LocalMutex sync.Mutex ``` Before the explicit marker was added to`sync\.Mutex`,`copylocks`could miss a copied`LocalMutex`\. Its underlying struct now includes the`\_ noCopy`field, so the recursive search can find the marker even though`LocalMutex`does not have the methods of`sync\.Mutex`\. Note Interestingly,`sync\.RWMutex`is the only exported struct in the`sync`package that must not be copied after first use but does not contain a`noCopy`field\. But`go vet`still catches a direct copy because`\*RWMutex`implements`sync\.Locker`while`RWMutex`does not\. What about a new defined type that loses those methods, just like`LocalMutex`? go ``` type LocalRWMutex sync.RWMutex ``` Its source provides another path for the checker: go ``` type RWMutex struct { w Mutex writerSem uint32 readerSem uint32 readerCount atomic.Int32 readerWait atomic.Int32 } ``` The recursive search reaches the highlighted`w`field, so`copylocks`still reports`LocalRWMutex contains sync\.Mutex`\. The old`LocalMutex`had only integer fields after it lost the methods, which is why that type needed the explicit marker to fix its false negative\. But it’s an exception from a consistency POV\. Now that we understand how the marker works, your package can define its own version\.`sync\.noCopy`is unexported, so code outside the`sync`package cannot use it directly: go ``` type noCopy struct{} func (*noCopy) Lock() {} func (*noCopy) Unlock() {} type Session struct { _ noCopy id string closed bool } ``` The marker still cannot stop the compiler from copying a value\.`go test`runs several vet checks by default, but`copylocks`is not one of them\. A default`go test \./\.\.\.`run therefore says nothing about this warning: sh ``` $ go test ./... ok example 0.4s $ go vet ./... lib.go:7:7: assignment copies lock value to b: example.Session contains example.noCopy ``` Many projects run`go vet`directly, either as a developer command or in CI, so`copylocks`can report copied values\. If you prefer to run the check through`go test`, enable the`copylocks`checker explicitly, either by itself or as part of all vet checks: sh ``` go test -vet=copylocks ./... # Or run all vet checks. go test -vet=all ./... ``` Both commands run`vet`on the package source and its test source files before running the tests\. ## 3\. What breaks when you ignore it There is no single failure behind this warning\. Different types can behave differently after they are copied\. ### A WaitGroup that never finishes go ``` func finish(w sync.WaitGroup) { // w is a copy w.Done() } var wg sync.WaitGroup wg.Add(1) finish(wg) wg.Wait() // never returns ``` The call to`finish\(wg\)`copies the fields of`wg`into the parameter`w`, including the counter value of 1\.`w\.Done\(\)`changes only the copied counter from 1 to 0\. original wgcounter1Wait blockscopy inside the callcounter0Done changes copycopiedDone changes the copy while the original keeps waiting\.The original counter stays at 1, so`wg\.Wait\(\)`has no reason to return\. This is the classic case, and it is the reason many developers first meet the warning\. go ``` var a sync.Map a.Store("x", 1) b := a b.Store("fromB", 1) _, seen := a.Load("fromB") // true ``` The first`Store`initializes`a`and gives it a root pointer\. The assignment to`b`copies that pointer, so`a`and`b`both access the same trie\. If we later call`Store`on`b`, the new entry may also be visible through`a`, even though`a`and`b`are separate variables\. > The shared storage of`a`and`b`in this example comes from the root pointer in the current`sync\.Map`implementation\. Since that copy happens after the map’s first use, its behavior is not guaranteed, and a future implementation may fail differently\. We will discuss the current implementation in a dedicated`sync\.Map`article soon\. ### Two copied variables initialize separate tries The behavior changes when we make the copy before the first`Store`: go ``` var c sync.Map d := c // copied before either is used c.Store("fromC", 1) d.Store("fromD", 1) _, cSeesD := c.Load("fromD") // false _, dSeesC := d.Load("fromC") // false ``` Because the copy happens before the first use,`c`has no root pointer to copy into`d`, a detail of the current`sync\.Map`implementation\. Both variables start empty, and each one initializes separate map storage when`Store`is called\. This is why neither variable sees the entry added through the other\. copied after first usecopied before first useabone storage"fromB"both write herecdstorage"fromC"storage"fromD"neither sees the otherThe same copy produces sharing or separation depending on when you do it\.Go’s documented[`sync\.Map`contract](https://pkg.go.dev/sync#Map)allows this copy: ``` "A Map must not be copied after first use." ``` But`copylocks`still reports it as the checker does not track whether the map has already been used\. Even so, copying a`sync\.Map`is usually a bad idea since the code then relies on order\-dependent behavior\. ## 4\. Field position controls struct size `noCopy`has size zero, but a trailing`noCopy`field can still increase the total size of its containing struct because Go may add padding after it\. A first\-field position avoids that extra padding: go ``` type plain struct { n int64 } type first struct { _ noCopy n int64 } type last struct { n int64 _ noCopy } ``` Using`unsafe\.Sizeof`on a 64\-bit platform such as amd64 or arm64 gives: ``` plain 8 bytes noCopy first 8 bytes noCopy last 16 bytes ``` In`first`,`noCopy`has size zero and starts at offset 0\. The`int64`can also start at offset 0 and spans bytes 0 through 7, so the whole struct uses 8 bytes\. noCopy firstnoCopy \+ n \(offset 0\)n \(int64\)bytes 0 to 7offset 0offset 88 bytes totalThe zero\-size field and int64 both start at offset 0\.In`last`, the`int64`spans bytes 0 through 7, which puts`noCopy`at offset 8\. A struct with a size of 8 bytes ends at that same offset, so the address of`noCopy`would sit outside the struct’s allocated memory\. Go adds 1 byte after the field, then rounds the total size up to the struct’s alignment\. On amd64 and arm64, the alignment is 8 bytes, so the total size increases from 8 to 16 bytes\. On 386 and arm, the alignment is 4 bytes, so the same struct has a total size of 12 bytes\. noCopy lastWITHOUT PADDINGn \(int64\)bytes 0 to 7noCopy \(offset 8\)outside allocated bytesn \(offset 0\)offset 8WITH PADDINGn \(int64\)bytes 0 to 7paddingbytes 8 to 15noCopy \(offset 8\)n \(offset 0\)offset 8offset 1616 bytes totalOn a 64\-bit platform, a trailing zero\-size field causes padding after offset 8\.The standard library avoids this size increase by putting`noCopy`before fields with a non\-zero size\. In`sync\.Map`,`sync\.Once`,`sync\.Pool`, and`sync\.WaitGroup`, it is the first field\. > `atomic\.Pointer\[T\]`has another zero\-size field before`noCopy`: go ``` type Pointer[T any] struct { _ [0]*T _ noCopy v unsafe.Pointer } ``` Both fields above`v`have zero size\. The pointer that stores the state still comes after`noCopy`\. ## Source references - [`src/sync/cond\.go`](https://go.dev/src/sync/cond.go) - [`src/sync/map\.go`](https://go.dev/src/sync/map.go) - [`src/sync/rwmutex\.go`](https://go.dev/src/sync/rwmutex.go) - [`src/internal/sync/hashtriemap\.go`](https://go.dev/src/internal/sync/hashtriemap.go) - [the 2016 change that introduced`noCopy`](https://github.com/golang/go/commit/c81a3532fea42df33dea54497dfaa96873c2d976) - [the 2018 change that added`Unlock`](https://github.com/golang/go/commit/c2eba53e7f80df21d51285879d51ab81bcfbf6bc) - [compiler struct\-size calculation](https://go.dev/src/cmd/compile/internal/types/size.go) - [the`copylocks`analyzer](https://cs.opensource.google/go/x/tools/+/master:go/analysis/passes/copylock/copylock.go) - [`go vet`documentation](https://pkg.go.dev/cmd/vet)