Reflection is very useful in Golang. Without reflection, data encoding functionality like JSON, Gob and XML
won’t be possible. Even a very crucial package like fmt uses Golang’s reflection to do its job. Outside the
standard library, people also often write their own utility function using reflection to perform various tasks
like deep copy, check equality between two objects, calling a method based on their name and
pretty print golang’s objects.
Despite their benefit, using reflection is not always the best option due to its performance overhead. Writing a method to deep copy a struct manually often yields a better performance compared to using reflection. The code below shows the performance difference of deep copying a Golang’s struct using manual implementation and reflection:
1package main
2
3import (
4 "github.com/mohae/deepcopy"
5 "testing"
6)
7
8type Example struct {
9 A string
10 B int
11 C []int
12}
13
14func (e Example) DeepCopyManual() Example {
15 valueC := make([]int, len(e.C))
16 for i, val := range e.C {
17 valueC[i] = val
18 }
19 return Example{
20 A: e.A,
21 B: e.B,
22 C: valueC,
23 }
24}
25
26func (e Example) DeepCopyReflection() Example {
27 return deepcopy.Copy(e).(Example)
28}
29
30func BenchmarkManualDeepCopy(b *testing.B) {
31 example := Example{
32 A: "just some string",
33 B: 10,
34 C: []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10},
35 }
36 for range b.N {
37 example.DeepCopyManual()
38 }
39}
40
41func BenchmarkReflectionDeepCopy(b *testing.B) {
42 example := Example{
43 A: "just some string",
44 B: 10,
45 C: []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10},
46 }
47 for range b.N {
48 example.DeepCopyReflection()
49 }
50}
Here is the result of running benchmark test of above code:
1❯ go test ./reflect_test.go -bench=. -v
2goos: darwin
3goarch: arm64
4BenchmarkManualDeepCopy
5BenchmarkManualDeepCopy-8 53719882 22.03 ns/op
6BenchmarkReflectionDeepCopy
7BenchmarkReflectionDeepCopy-8 2248674 534.6 ns/op
8PASS
9ok command-line-arguments 3.973s
As we can see above, performing deep copy using reflection is about 24x slower compared to the manual implementation. This performance overhead might not be a big deal when the reflection doesn’t executed in hot path in the system and only affecting small percentage of the overall system performance. But, in some cases, the overhead might be significant to the overall system’s performance. When this happen, implementing the deep copy manually might makes more sense.
Now, the problem with manual implementation is that it is tedious and prone to error. But, I can argue that most of the time, it’s not that tedious and most of the work will only happen once. When we change the struct definition by adding, removing or changing a field, we only need to update a small portion of the deep copy code. The more dangerous problem is about its correctness. When someone change your struct definition, how do you make sure that they also change the deep copy implementation? If they change your struct’s definition but forgot to change the deep copy implementation, nothing will happen even though it’s wrong. It might not even affecting the production environment for some time. You only see a wrong result or even worse you might even running into problem in your data integrity and it will be very hard to debug.
Now, if you think about it, this is what automated testing are used for. It prevents code changes that affect incorrectness to be committed to our codebase. Whenever someone make changes, those changes can only be merged to our main branch if all of the tests are passed. We can use this concept to protect our deep copy implementation. Now, what we need to do is to write test for our deep copy implementation so that when someone implementing the deep copy incorrectly, the test will fail.
Here is the sweet part: it is ok to use reflection in your tests. Why? because your tests is not the performance bottleneck of your application. You only run your tests occasionally, maybe when you push your changes, or you want to merge your branch to the main branch. You can think that your tests QPS (query per second) is less than 1. Compared to your actual application, it’s nothing.
Now, consider the code below:
1package main
2
3import "testing"
4
5type Example struct {
6 A string
7 B int
8 C []int
9}
10
11func (e Example) DeepCopy() Example {
12 valueC := make([]int, len(e.C))
13 for i, val := range e.C {
14 valueC[i] = val
15 }
16 return Example{
17 A: e.A,
18 B: e.B,
19 C: valueC,
20 }
21}
22
23func TestDeepCopy(t *testing.T) {
24 t.Run("Example", func(t *testing.T) {
25 testDeepCopyImplementation(t, func(value Example) Example {
26 return value.DeepCopy()
27 })
28 })
29}
30
31func testDeepCopyImplementation[T any](t *testing.T, f func(T) T) {
32 // The implementation of this function is left
33 // for the reader for exercise.
34 //
35 // Note that, you can use reflection in this
36 // function to make it work for any type (or at
37 // least for most of the types).
38 // Here are few hints:
39 // 1. You can use deep equal to check if the
40 // resulting struct is copied equally. The
41 // deep equal can be implemented using
42 // reflection.
43 // 2. You can modify the first struct to check
44 // if the modification of the first struct
45 // doesn't affect the copied struct.
46 // 3. Use can create a random struct instance
47 // using reflection.
48 // 4. Types like channel and function are not
49 // meant to be deep copied. You can fail
50 // the test when the user use these types.
51 // 5. There are special type that can't be
52 // deep copied as easily like time.Time.
53 // Make sure your test fail by default
54 // if you don't handle them.
55}
Now using above code, you will have the performance of manually-implemented deep copy with correctness safety. When you run the test, nothing will happen since your implementation is correct:
1❯ go test ./reflect_test.go -v
2=== RUN TestDeepCopy
3=== RUN TestDeepCopy/Example
4--- PASS: TestDeepCopy (0.00s)
5 --- PASS: TestDeepCopy/Example (0.00s)
Whenever someone added a new field in your Example struct but forgot to update the deep copy implementation,
their changes won’t pass the TestDeepEqualTest. Consider the code changes below:
1type Example struct {
2 A string
3 B int
4 C []int
5 // This new field is added:
6 D int
7}
8
9func (e Example) DeepCopy() Example {
10 valueC := make([]int, len(e.C))
11 for i, val := range e.C {
12 valueC[i] = val
13 }
14 return Example{
15 A: e.A,
16 B: e.B,
17 C: valueC,
18 // The field D is not deep copied here.
19 }
20}
When you re-run the tests, it will fail:
1❯ go test ./reflect_test.go -v
2=== RUN TestDeepCopy
3=== RUN TestDeepCopy/Example
4 reflect_test.go:86:
5 Error Trace: /private/tmp/reflect_test.go:86
6 /private/tmp/reflect_test.go:64
7 /private/tmp/reflect_test.go:34
8 Error: Not equal:
9 expected: main.Example{A:"a", B:1, C:[]int{1}, D:1}
10 actual : main.Example{A:"a", B:1, C:[]int{1}, D:0}
11
12 Diff:
13 --- Expected
14 +++ Actual
15 @@ -6,3 +6,3 @@
16 },
17 - D: (int) 1
18 + D: (int) 0
19 }
20 Test: TestDeepCopy/Example
21--- FAIL: TestDeepCopy (0.00s)
22 --- FAIL: TestDeepCopy/Example (0.00s)
23FAIL
24FAIL command-line-arguments 0.506s
25FAIL
You might get a different error message depending on how you implemented your testDeepCopyImplementation function.
In addition to that technique, if you still think that doing manual implementation is tedious, you can always use code generator tool to write your implmentation. But, be aware that the problem still exists. If someone change the struct definition but forgot to run the code generator tool, you will have an incorrect result without any error or warning. This technique will still needed to prevent that.