From 73b8ccb0b0cd298da681df17ce1af7ac89c41798 Mon Sep 17 00:00:00 2001 From: Andrew Gerrand Date: Wed, 7 Mar 2012 07:40:21 +1100 Subject: [PATCH] doc: add "The go command" article based on Russ' mail This is a philosophical document. We can soup it up at a later stage, but for now it's nice to have a URL to point to that isn't a mailing list post. R=golang-dev, r CC=golang-dev https://golang.org/cl/5676061 --- doc/articles/go_command.html | 265 +++++++++++++++++++++++++++++++++++ doc/docs.html | 1 + 2 files changed, 266 insertions(+) create mode 100644 doc/articles/go_command.html diff --git a/doc/articles/go_command.html b/doc/articles/go_command.html new file mode 100644 index 0000000000..343fcbc6da --- /dev/null +++ b/doc/articles/go_command.html @@ -0,0 +1,265 @@ + + +

The Go distribution includes a command, named +"go", that +automates the downloading, building, installation, and testing of Go packages +and commands. This document talks about why we wrote a new command, what it +is, what it's not, and how to use it.

+ +

Motivation

+ +

You might have seen early Go talks in which Rob Pike jokes that the idea +for Go arose while waiting for a large Google server to compile. That +really was the motivation for Go: to build a language that worked well +for building the large software that Google writes and runs. It was +clear from the start that such a language must provide a way to +express dependencies between code libraries clearly, hence the package +grouping and the explicit import blocks. It was also clear from the +start that you might want arbitrary syntax for describing the code +being imported; this is why import paths are string literals.

+ +

An explicit goal for Go from the beginning was to be able to build Go +code using only the information found in the source itself, not +needing to write a makefile or one of the many modern replacements for +makefiles. If Go needed a configuration file to explain how to build +your program, then Go would have failed.

+ +

At first, there was no Go compiler, and the initial development +focused on building one and then building libraries for it. For +expedience, we postponed the automation of building Go code by using +make and writing makefiles. When compiling a single package involved +multiple invocations of the Go compiler, we even used a program to +write the makefiles for us. You can find it if you dig through the +repository history.

+ +

The purpose of the new go command is our return to this ideal, that Go +programs should compile without configuration or additional effort on +the part of the developer beyond writing the necessary import +statements.

+ +

Configuration versus convention

+ +

The way to achieve the simplicity of a configuration-free system is to +establish conventions. The system works only to the extent that the convention +is followed. When we first launched Go, many people published packages that +had to be installed in certain places, under certain names, using certain build +tools, in order to be used. That's understandable: that's the way it works in +most other languages. Over the last few years we consistently reminded people +about the old goinstall command +(now replaced by go get) +and its conventions: first, that the import path is derived in a known way from +the URL of the source code; second, that that the place to store the sources in +the local file system is derived in a known way from the import path; third, +that each directory in a source tree corresponds to a single package; and +fourth, that the package is built using only information in the source code. +Today, the vast majority of packages follow these conventions. +The Go ecosystem is simpler and more powerful for it.

+ +

We received many requests to allow a makefile in a package directory to +provide just a little extra configuration beyond what's in the source code. +But that would have introduced new rules. Because we did not accede to such +requests, we were able to write the go command and eliminate our use of make +or any other build system.

+ +

It is important to understand that the go command is not a general +build tool. It cannot be configured and it does not attempt to build +anything but Go packages. These are important simplifying +assumptions: they simplify not only the implementation but also, more +important, the use of the tool itself.

+ +

Go's conventions

+ +

The go command requires that code adheres to a few key, +well-established conventions.

+ +

First, the import path is derived in an known way from the URL of the +source code. For Bitbucket, GitHub, Google Code, and Launchpad, the +root directory of the repository is identified by the repository's +main URL, without the http:// prefix. Subdirectories are named by +adding to that path. For example, the supplemental networking +libraries for Go are obtained by running

+ +
+hg clone http://code.google.com/p/go.net
+
+ +

and thus the import path for the root directory of that repository is +"code.google.com/p/go.net". The websocket package is stored in a +subdirectory, so its import path is +"code.google.com/p/go.net/websocket".

+ +

These paths are on the long side, but in exchange we get an +automatically managed name space for import paths and the ability for +a tool like the go command to look at an unfamiliar import path and +deduce where to obtain the source code.

+ +

Second, the place to store sources in the local file system is derived +in a known way from the import path. Specifically, the first choice +is $GOPATH/src/<import-path>. If $GOPATH is +unset, the go command will fall back to storing source code alongside the +standard Go packages, in $GOROOT/src/pkg/<import-path>. +If $GOPATH is set to a list of paths, the go command tries +<dir>/src/<import-path> for each of the directories in +that list.

+ +

Each of those trees contains, by convention, a top-level directory named +"bin", for holding compiled executables, and a top-level directory +named "pkg", for holding compiled packages that can be imported, +and the "src" directory, for holding package source files. +Imposing this structure lets us keep each of these directory trees +self-contained: the compiled form and the sources are always near each +other.

+ +

These naming conventions also let us work in the reverse direction, +from a directory name to its import path. This mapping is important +for many of the go command's subcommands, as we'll see below.

+ +

Third, each directory in a source tree corresponds to a single +package. By restricting a directory to a single package, we don't have +to create hybrid import paths that specify first the directory and +then the package within that directory. Also, most file management +tools and UIs work on directories as fundamental units. Tying the +fundamental Go unit—the package—to file system structure means +that file system tools become Go package tools. Copying, moving, or +deleting a package corresponds to copying, moving, or deleting a +directory.

+ +

Fourth, each package is built using only the information present in +the source files. This makes it much more likely that the tool will +be able to adapt to changing build environments and conditions. For +example, if we allowed extra configuration such as compiler flags or +command line recipes, then that configuration would need to be updated +each time the build tools changed; it would also be inherently tied +to the use of a specific tool chain.

+ +

Getting started with the go command

+ +

Finally, a quick tour of how to use the go command, to supplement +the information in How to Write Go Code, +which you might want to read first. Assuming you want +to keep your source code separate from the Go distribution source +tree, the first step is to set $GOPATH, the one piece of global +configuration that the go command needs. The $GOPATH can be a +list of directories, but by far the most common usage should be to set it to a +single directory. In particular, you do not need a separate entry in +$GOPATH for each of your projects. One $GOPATH can +support many projects.

+ +

Here’s an example. Let’s say we decide to keep our Go code in the directory +$HOME/mygo. We need to create that directory and set +$GOPATH accordingly.

+ +
+$ mkdir $HOME/mygo
+$ export GOPATH=$HOME/mygo
+$
+
+ +

Into this directory, we now add some source code. Suppose we want to use +the indexing library from the codesearch project along with a left-leaning +red-black tree. We can install both with the "go get" +subcommand:

+ +
+$ go get code.google.com/p/codesearch/index
+$ go get github.com/petar/GoLLRB/llrb
+$
+
+ +

Both of these projects are now downloaded and installed into our +$GOPATH directory. The one tree now contains the two directories +src/code.google.com/p/codesearch/index/ and +src/github.com/petar/GoLLRB/llrb/, along with the compiled +packages (in pkg/) for those libraries and their dependencies.

+ +

Because we used version control systems (Mercurial and Git) to check +out the sources, the source tree also contains the other files in the +corresponding repositories, such as related packages. The "go list" +subcommand lists the import paths corresponding to its arguments, and +the pattern "./..." means start in the current directory +("./") and find all packages below that directory +("..."):

+ +
+$ go list ./...
+code.google.com/p/codesearch/cmd/cgrep
+code.google.com/p/codesearch/cmd/cindex
+code.google.com/p/codesearch/cmd/csearch
+code.google.com/p/codesearch/index
+code.google.com/p/codesearch/regexp
+code.google.com/p/codesearch/sparse
+github.com/petar/GoLLRB/example
+github.com/petar/GoLLRB/llrb
+$
+
+ +

We can also test those packages:

+ +
+$ go test ./...
+?       code.google.com/p/codesearch/cmd/cgrep   [no test files]
+?       code.google.com/p/codesearch/cmd/cindex  [no test files]
+?       code.google.com/p/codesearch/cmd/csearch [no test files]
+ok      code.google.com/p/codesearch/index       0.239s
+ok      code.google.com/p/codesearch/regexp      0.021s
+?       code.google.com/p/codesearch/sparse      [no test files]
+?       github.com/petar/GoLLRB/example          [no test files]
+ok      github.com/petar/GoLLRB/llrb             0.231s
+$
+
+ +

If a go subcommand is invoked with no paths listed, it operates on the +current directory:

+ +
+$ cd $GOPATH/src/code.google.com/p/codesearch/regexp
+$ go list
+code.google.com/p/codesearch/regexp
+$ go test -v
+=== RUN TestNstateEnc
+--- PASS: TestNstateEnc (0.00 seconds)
+=== RUN TestMatch
+--- PASS: TestMatch (0.01 seconds)
+=== RUN TestGrep
+--- PASS: TestGrep (0.00 seconds)
+PASS
+ok      code.google.com/p/codesearch/regexp     0.021s
+$ go install
+$
+
+ +

That "go install" subcommand installs the latest copy of the +package into the pkg directory. Because the go command can analyze the +dependency graph, "go install" also installs any packages that +this package imports but that are out of date, recursively.

+ +

Notice that "go install" was able to determine the name of the +import path for the package in the current directory, because of the convention +for directory naming. It would be a little more convenient if we could pick +the name of the directory where we kept source code, and we probably wouldn't +pick such a long name, but that ability would require additional configuration +and complexity in the tool. Typing an extra directory name or two is a small +price to pay for the increased simplicity and power.

+ +

As the example shows, it’s fine to work with packages from many different +projects at once within a single $GOPATH root directory.

+ +

Limitations

+ +

As mentioned above, the go command is not a general-purpose build +tool. In particular, it does not have any facility for generating Go +source files during a build. Instead, if you want to use a tool like +yacc or the protocol buffer compiler, you will need to write a +makefile (or a configuration file for the build tool of your choice) +to generate the Go files and then check those generated source files +into your repository. This is more work for you, the package author, +but it is significantly less work for your users, who can use +"go get" without needing to obtain and build +any additional tools.

+ +

More information

+ +

For more information, read How to Write Go Code +and see the go command documentation.

diff --git a/doc/docs.html b/doc/docs.html index f48eb0a08f..973299bdd6 100644 --- a/doc/docs.html +++ b/doc/docs.html @@ -112,6 +112,7 @@ Guided tours of Go programs.

Tools