Loading
prevent concurrent map race in RecordEvent by using DeepCopy
This MR fixes a fatal error: concurrent map read and map write crash that occurred because of 2 goroutines accessing the same map simultaneously.
Tests done:
Confirmed the race using a -race against a live cluster
go build -race -o sylvactl-race ./cmd./sylvactl-race watch --record /tmp/events.yaml Kustomization/sylva-system/management-sylva-units
watching resources on namespace: sylva-system
==================
WARNING: DATA RACE
Write at 0x00c0009d6960 by goroutine 39:
runtime.mapdelete_faststr()
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.1.linux-amd64/src/internal/runtime/maps/runtime_faststr.go:402 +0x0
k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.RemoveNestedField()
/root/go/pkg/mod/k8s.io/apimachinery@v0.36.1/pkg/apis/meta/v1/unstructured/helpers.go:323 +0x1f9
gitlab.com/sylva-projects/sylva-elements/sylvactl/internal/recorder.(*recorder).RecordEvent()
/home/ubuntu/sylvactl/internal/recorder/event_recorder.go:80 +0x9f
gitlab.com/sylva-projects/sylva-elements/sylvactl/internal/informer.(*TreeInformer).eventHandler()
/home/ubuntu/sylvactl/internal/informer/flux_informer.go:294 +0x21e
gitlab.com/sylva-projects/sylva-elements/sylvactl/internal/informer.NewInformer.gowrap1()
/home/ubuntu/sylvactl/internal/informer/flux_informer.go:190 +0x2e
Previous read at 0x00c0009d6960 by goroutine 40:
runtime.mapaccess2_faststr()
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.1.linux-amd64/src/internal/runtime/maps/runtime_faststr.go:161 +0x0
k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.NestedFieldNoCopy()
/root/go/pkg/mod/k8s.io/apimachinery@v0.36.1/pkg/apis/meta/v1/unstructured/helpers.go:62 +0x10f
k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.NestedString()
/root/go/pkg/mod/k8s.io/apimachinery@v0.36.1/pkg/apis/meta/v1/unstructured/helpers.go:76 +0x6b
k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.getNestedString()
/root/go/pkg/mod/k8s.io/apimachinery@v0.36.1/pkg/apis/meta/v1/unstructured/helpers.go:327 +0x44
k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.(*Unstructured).GetResourceVersion()
/root/go/pkg/mod/k8s.io/apimachinery@v0.36.1/pkg/apis/meta/v1/unstructured/unstructured.go:287 +0x96
k8s.io/client-go/tools/cache.(*sharedIndexInformer).OnUpdate()
/root/go/pkg/mod/k8s.io/client-go@v0.36.1/tools/cache/shared_informer.go:980 +0xc9
k8s.io/client-go/tools/cache.processDeltas()
/root/go/pkg/mod/k8s.io/client-go@v0.36.1/tools/cache/controller.go:644 +0x32b
k8s.io/client-go/tools/cache.(*sharedIndexInformer).handleDeltas()Both goroutines accessed address 0x00c0009d6960 , confirming they shared the same object.
After applying the fix, the previous race disappeared. The race error now shown is because of something else
./sylvactl-race watch --record /tmp/events.yaml Kustomization/sylva-system/management-sylva-units
watching resources on namespace: sylva-system
==================evnull - Resource is ready
WARNING: DATA RACEylva-capi-cluster - Resource is ready
Write at 0x00c000113218 by goroutine 39:Resource is ready
gitlab.com/sylva-projects/sylva-elements/sylvactl/internal/informer.(*fluxObject).SetMissingDependencies()
/home/ubuntu/sylvactl/internal/informer/flux_adapters.go:82 +0xe4
gitlab.com/sylva-projects/sylva-elements/sylvactl/internal/informer.(*node).SetMissingDependencies()
<autogenerated>:1 +0x5ec-openstack-images - Resource is ready
gitlab.com/sylva-projects/sylva-elements/sylvactl/internal/informer.(*objectMap).updateDependencies()
/home/ubuntu/sylvactl/internal/informer/dag.go:376 +0xcd4
gitlab.com/sylva-projects/sylva-elements/sylvactl/internal/informer.(*objectMap).updateMissingDependencies()
/home/ubuntu/sylvactl/internal/informer/dag.go:329 +0x22crelated to #100
Edited by Priya Goyal