From bc1d71ff64b01ac912cb0fa28c0f6dd5904fcd56 Mon Sep 17 00:00:00 2001 From: thepudds Date: Thu, 15 Jun 2023 15:09:11 -0400 Subject: [PATCH] cmd/compile: add a 'Tips' section to README to help new contributors This CL adds a new 'Tips' section to the cmd/compile README. The primary intent is to help new-ish contributors. It includes some basics on getting started, testing changes, viewing coverage, juggling different compiler versions, some links to additional tools, and so on. Updates #30074 Change-Id: I393bf1137db9d2bb851f7e254b08455273ccad8c Reviewed-on: https://go-review.googlesource.com/c/go/+/503895 TryBot-Result: Gopher Robot Reviewed-by: Matthew Dempsky Run-TryBot: t hepudds Reviewed-by: Keith Randall Reviewed-by: Keith Randall --- src/cmd/compile/README.md | 170 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 170 insertions(+) diff --git a/src/cmd/compile/README.md b/src/cmd/compile/README.md index 5cac4076bd..9b99a1b105 100644 --- a/src/cmd/compile/README.md +++ b/src/cmd/compile/README.md @@ -140,7 +140,177 @@ a series of obj.Prog instructions. These are passed to the assembler final object file. The object file will also contain reflect data, export data, and debugging information. +### 8. Tips + +#### Getting Started + +* If you have never contributed to the compiler before, a simple way to begin + can be adding a log statement or `panic("here")` to get some + initial insight into whatever you are investigating. + +* The compiler itself provides logging, debugging and visualization capabilities, + such as: + ``` + $ go build -gcflags=-m=2 # print optimization info, including inlining, escape analysis + $ go build -gcflags=-d=ssa/check_bce/debug # print bounds check info + $ go build -gcflags=-W # print internal parse tree after type checking + $ GOSSAFUNC=Foo go build # generate ssa.html file for func Foo + $ go build -gcflags=-S # print assembly + $ go tool compile -bench=out.txt x.go # print timing of compiler phases + ``` + + Some flags alter the compiler behavior, such as: + ``` + $ go tool compile -h file.go # panic on first compile error encountered + $ go build -gcflags=-d=checkptr=2 # enable additional unsafe pointer checking + ``` + + There are many additional flags. Some descriptions are available via: + ``` + $ go tool compile -h # compiler flags, e.g., go build -gcflags='-m=1 -l' + $ go tool compile -d help # debug flags, e.g., go build -gcflags=-d=checkptr=2 + $ go tool compile -d ssa/help # ssa flags, e.g., go build -gcflags=-d=ssa/prove/debug=2 + ``` + + There are some additional details about `-gcflags` and the differences between `go build` + vs. `go tool compile` in a [section below](#-gcflags-and-go-build-vs-go-tool-compile). + +* In general, when investigating a problem in the compiler you usually want to + start with the simplest possible reproduction and understand exactly what is + happening with it. + +#### Testing your changes + +* Be sure to read the [Quickly testing your changes](https://go.dev/doc/contribute#quick_test) + section of the Go Contribution Guide. + +* Some tests live within the cmd/compile packages and can be run by `go test ./...` or similar, + but many cmd/compile tests are in the top-level + [test](https://github.com/golang/go/tree/master/test) directory: + + ``` + $ go test cmd/internal/testdir # all tests in 'test' dir + $ go test cmd/internal/testdir -run='Test/escape.*.go' # test specific files in 'test' dir + ``` + For details, see the [testdir README](https://github.com/golang/go/tree/master/test#readme). + The `errorCheck` method in [testdir_test.go](https://github.com/golang/go/blob/master/src/cmd/internal/testdir/testdir_test.go) + is helpful for a description of the `ERROR` comments used in many of those tests. + + In addition, the `go/types` package from the standard library and `cmd/compile/internal/types2` + have shared tests in `src/internal/types/testdata`, and both type checkers + should be checked if anything changes there. + +* The new [application-based coverage profiling](https://go.dev/testing/coverage/) can be used + with the compiler, such as: + + ``` + $ go install -cover -coverpkg=cmd/compile/... cmd/compile # build compiler with coverage instrumentation + $ mkdir /tmp/coverdir # pick location for coverage data + $ GOCOVERDIR=/tmp/coverdir go test [...] # use compiler, saving coverage data + $ go tool covdata textfmt -i=/tmp/coverdir -o coverage.out # convert to traditional coverage format + $ go tool cover -html coverage.out # view coverage via traditional tools + ``` + +#### Juggling compiler versions + +* Many of the compiler tests use the version of the `go` command found in your PATH and + its corresponding `compile` binary. + +* If you are in a branch and your PATH includes `/bin`, + doing `go install cmd/compile` will build the compiler using the code from your + branch and install it to the proper location so that subsequent `go` commands + like `go build` or `go test ./...` will exercise your freshly built compiler. + +* [toolstash](https://pkg.go.dev/golang.org/x/tools/cmd/toolstash) provides a way + to save, run, and restore a known good copy of the Go toolchain. For example, it can be + a good practice to initially build your branch, save that version of + the toolchain, then restore the known good version of the tools to compile + your work-in-progress version of the compiler. + + Sample set up steps: + ``` + $ go install golang.org/x/tools/cmd/toolstash@latest + $ git clone https://go.googlesource.com/go + $ cd go + $ git checkout -b mybranch + $ ./src/all.bash # build and confirm good starting point + $ export PATH=$PWD/bin:$PATH + $ toolstash save # save current tools + ``` + After that, your edit/compile/test cycle can be similar to: + ``` + <... make edits to cmd/compile source ...> + $ toolstash restore && go install cmd/compile # restore known good tools to build compiler + <... 'go build', 'go test', etc. ...> # use freshly built compiler + ``` + +* toolstash also allows comparing the installed vs. stashed copy of + the compiler, such as if you expect equivalent behavior after a refactor. + For example, to check that your changed compiler produces identical object files to + the stashed compiler while building the standard library: + ``` + $ toolstash restore && go install cmd/compile # build latest compiler + $ go build -toolexec "toolstash -cmp" -a -v std # compare latest vs. saved compiler + ``` + +* If versions appear to get out of sync (for example, with errors like + `linked object header mismatch` with version strings like + `devel go1.21-db3f952b1f`), you might need to do + `toolstash restore && go install cmd/...` to update all the tools under cmd. + +#### Additional helpful tools + +* [compilebench](https://pkg.go.dev/golang.org/x/tools/cmd/compilebench) benchmarks + the speed of the compiler. + +* [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat) is the standard tool + for reporting performance changes resulting from compiler modifications, + including whether any improvements are statistically significant: + ``` + $ go test -bench=SomeBenchmarks -count=20 > new.txt # use new compiler + $ toolstash restore # restore old compiler + $ go test -bench=SomeBenchmarks -count=20 > old.txt # use old compiler + $ benchstat old.txt new.txt # compare old vs. new + ``` + +* [bent](https://pkg.go.dev/golang.org/x/benchmarks/cmd/bent) facilitates running a + large set of benchmarks from various community Go projects inside a Docker container. + +* [perflock](https://github.com/aclements/perflock) helps obtain more consistent + benchmark results, including by manipulating CPU frequency scaling settings on Linux. + +* [view-annotated-file](https://github.com/loov/view-annotated-file) (from the community) + overlays inlining, bounds check, and escape info back onto the source code. + +* [godbolt.org](https://go.godbolt.org) is widely used to examine + and share assembly output from many compilers, including the Go compiler. It can also + [compare](https://go.godbolt.org/z/5Gs1G4bKG) assembly for different versions of + a function or across Go compiler versions, which can be helpful for investigations and + bug reports. + +#### -gcflags and 'go build' vs. 'go tool compile' + +* `-gcflags` is a go command [build flag](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies). + `go build -gcflags=` passes the supplied `` to the underlying + `compile` invocation(s) while still doing everything that the `go build` command + normally does (e.g., handling the build cache, modules, and so on). In contrast, + `go tool compile ` asks the `go` command to invoke `compile ` a single time + without involving the standard `go build` machinery. In some cases, it can be helpful to have + fewer moving parts by doing `go tool compile `, such as if you have a + small standalone source file that can be compiled without any assistance from `go build`. + In other cases, it is more convenient to pass `-gcflags` to a build command like + `go build`, `go test`, or `go install`. + +* `-gcflags` by default applies to the packages named on the command line, but can + use package patterns such as `-gcflags='all=-m=1 -l'`, or multiple package patterns such as + `-gcflags='all=-m=1' -gcflags='fmt=-m=2'`. For details, see the + [cmd/go documentation](https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependencies). + ### Further reading To dig deeper into how the SSA package works, including its passes and rules, head to [cmd/compile/internal/ssa/README.md](internal/ssa/README.md). + +Finally, if something in this README or the SSA README is unclear +or if you have an idea for an improvement, feel free to leave a comment in +[issue 30074](https://go.dev/issue/30074). -- 2.50.0