From: Jay Conrod Date: Fri, 6 Dec 2019 23:24:29 +0000 (-0500) Subject: doc: add section on go.mod file syntax X-Git-Tag: go1.14~9^2~23 X-Git-Url: http://www.git.cypherpunks.su/?a=commitdiff_plain;h=153a9e80332ca165d9d00149e9d8b9747d486c18;p=gostls13.git doc: add section on go.mod file syntax Updates #33637 Change-Id: I265e4fda863b871a3ce0ca7b6c926081dadbf5a6 Reviewed-on: https://go-review.googlesource.com/c/go/+/210799 Reviewed-by: Bryan C. Mills --- diff --git a/doc/modules.md b/doc/modules.md index 42ae25c4ef..4d582c597d 100644 --- a/doc/modules.md +++ b/doc/modules.md @@ -136,7 +136,7 @@ Many Go projects released versions at `v2` or higher without using a major version suffix before migrating to modules (perhaps before modules were even introduced). These versions are annotated with a `+incompatible` build tag (for example, `v2.0.0+incompatible`). See [Compatibility with non-module -repositories](#compatibility-with-non-module-repositories) for more information. +repositories](#non-module-compat) for more information. ### Resolving a package to a module @@ -201,23 +201,390 @@ the following requests: * Request for latest version of `golang.org` After a suitable module has been found, the `go` command will add a new -requirement with the new module's path and version to the main module's `go.mod` -file. This ensures that when the same package is loaded in the future, the same -module will be used at the same version. If the resolved package is not imported -by a package in the main module, the new requirement will have an `// indirect` -comment. +[requirement](#go.mod-require) with the new module's path and version to the +main module's `go.mod` file. This ensures that when the same package is loaded +in the future, the same module will be used at the same version. If the resolved +package is not imported by a package in the main module, the new requirement +will have an `// indirect` comment. ## `go.mod` files - -### `go.mod` file format +A module is defined by a UTF-8 encoded text file named `go.mod` in its root +directory. The `go.mod` file is line-oriented. Each line holds a single +directive, made up of a keyword followed by arguments. For example: + +``` +module example.com/my/thing + +go 1.12 + +require example.com/other/thing v1.0.2 +require example.com/new/thing/v2 v2.3.4 +exclude example.com/old/thing v1.2.3 +replace example.com/bad/thing v1.4.5 => example.com/good/thing v1.4.5 +``` + +The leading keyword can be factored out of adjacent lines to create a block, +like in Go imports. + +``` +require ( + example.com/new/thing/v2 v2.3.4 + example.com/old/thing v1.2.3 +) +``` + +The `go.mod` file is designed to be human readable and machine writable. The +`go` command provides several subcommands that change `go.mod` files. For +example, [`go get`](#go-get) can upgrade or downgrade specific dependencies. +Commands that load the module graph will [automatically update](#go.mod-updates) +`go.mod` when needed. [`go mod edit`](#go-mod-tidy) can perform low-level edits. +The +[`golang.org/x/mod/modfile`](https://pkg.go.dev/golang.org/x/mod/modfile?tab=doc) +package can be used by Go programs to make the same changes programmatically. + + +### Lexical elements + +When a `go.mod` file is parsed, its content is broken into a sequence of tokens. +There are several kinds of tokens: whitespace, comments, punctuation, +keywords, identifiers, and strings. + +*White space* consists of spaces (U+0020), tabs (U+0009), carriage returns +(U+000D), and newlines (U+000A). White space characters other than newlines have +no effect except to separate tokens that would otherwise be combined. Newlines +are significant tokens. + +*Comments* start with `//` and run to the end of a line. `/* */` comments are +not allowed. + +*Punctuation* tokens include `(`, `)`, and `=>`. + +*Keywords* distinguish different kinds of directives in a `go.mod` file. Allowed +keywords are `module`, `go`, `require`, `replace`, and `exclude`. + +*Identifiers* are sequences of non-whitespace characters, such as module paths +or semantic versions. + +*Strings* are quoted sequences of characters. There are two kinds of strings: +interpreted strings beginning and ending with quotation marks (`"`, U+0022) and +raw strings beginning and ending with grave accents (<, +U+0060). Interpreted strings may contain escape sequences consisting of a +backslash (`\`, U+005C) followed by another character. An escaped quotation +mark (`\"`) does not terminate an interpreted string. The unquoted value +of an interpreted string is the sequence of characters between quotation +marks with each escape sequence replaced by the character following the +backslash (for example, `\"` is replaced by `"`, `\n` is replaced by `n`). +In contrast, the unquoted value of a raw string is simply the sequence of +characters between grave accents; backslashes have no special meaning within +raw strings. + +Identifiers and strings are interchangeable in the `go.mod` grammar. + + +### Module paths and versions + +Most identifiers and strings in a `go.mod` file are either module paths or +versions. + +A module path must satisfy the following requirements: + +* The path must consist of one or more path elements separated by slashes + (`/`, U+002F). It must not begin or end with a slash. +* Each path element is a non-empty string made of up ASCII letters, ASCII + digits, and limited ASCII punctuation (`+`, `-`, `.`, `_`, and `~`). +* A path element may not begin or end with a dot (`.`, U+002E). +* The element prefix up to the first dot must not be a reserved file name on + Windows, regardless of case (`CON`, `com1`, `NuL`, and so on). + +If the module path appears in a `require` directive and is not replaced, or +if the module paths appears on the right side of a `replace` directive, +the `go` command may need to download modules with that path, and some +additional requirements must be satisfied. + +* The leading path element (up to the first slash, if any), by convention a + domain name, must contain only lower-case ASCII letters, ASCII digits, dots + (`.`, U+002E), and dashes (`-`, U+002D); it must contain at least one dot and + cannot start with a dash. +* For a final path element of the form `/vN` where `N` looks numeric (ASCII + digits and dots), `N` must not begin with a leading zero, must not be `/v1`, + and must not contain any dots. + * For paths beginning with `gopkg.in/`, this requirement is replaced by a + requirement that the path follow the [gopkg.in](https://gopkg.in) service's + conventions. + +Versions in `go.mod` files may be [canonical](#glos-canonical-version) or +non-canonical. + +A canonical version starts with the letter `v`, followed by a semantic version +following the [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html) +specification. See [Versions](#versions) for more information. + +Most other identifiers and strings may be used as non-canonical versions, though +there are some restrictions to avoid problems with file systems, repositories, +and [module proxies](#glos-module-proxy). Non-canonical versions are only +allowed in the main module's `go.mod` file. The `go` command will attempt to +replace each non-canonical version with an equivalent canonical version when it +automatically [updates](#go.mod-updates) the `go.mod` file. + +In places where a module path is associated with a verison (as in `require`, +`replace`, and `exclude` directives), the final path element must be consistent +with the version. See [Major version suffixes](#major-version-suffixes). + + +### Grammar + +`go.mod` syntax is specified below using Extended Backus-Naur Form (EBNF). +See the [Notation section in the Go Language Specificiation](/ref/spec#Notation) +for details on EBNF syntax. + +``` +GoMod = { Directive } . +Directive = ModuleDirective | + GoDirective | + RequireDirective | + ExcludeDirective | + ReplaceDirective . +``` + +Newlines, identifiers, and strings are denoted with `newline`, `ident`, and +`string`, respectively. + +Module paths and versions are denoted with `ModulePath` and `Version`. + +``` +ModulePath = ident | string . /* see restrictions above */ +Version = ident | string . /* see restrictions above */ +``` + + +### `module` directive + +A `module` directive defines the main module's [path](#glos-module-path). A +`go.mod` file must contain exactly one `module` directive. + +``` +ModuleDirective = "module" ( ModulePath | "(" newline ModulePath newline ")" newline . +``` + +Example: + +``` +module golang.org/x/net +``` + + +### `go` directive + +A `go` directive sets the expected language version for the module. The +version must be a valid Go release version: a positive integer followed by a dot +and a non-negative integer (for example, `1.9`, `1.14`). + +The language version determines which language features are available when +compiling packages in the module. Language features present in that version +will be available for use. Language features removed in earlier versions, +or added in later versions, will not be available. The language version does not +affect build tags, which are determined by the Go release being used. + +The language version is also used to enable features in the `go` command. For +example, automatic [vendoring](#vendoring) may be enabled with a `go` version of +`1.14` or higher. + +A `go.mod` file may contain at most one `go` directive. Most commands will add a +`go` directive with the current Go version if one is not present. + +``` +GoDirective = "go" GoVersion newline . +GoVersion = string | ident . /* valid release version; see above */ +``` + +Example: + +``` +go 1.14 +``` + + +### `require` directive + +A `require` directive declares a minimum required version of a given module +dependency. For each required module version, the `go` command loads the +`go.mod` file for that version and incorporates the requirements from that +file. Once all requirements have been loaded, the `go` command resolves them +using [minimal version selection (MVS)](#minimal-version-selection) to produce +the [build list](#glos-build-list). + +The `go` command automatically adds `// indirect` comments for some +requirements. An `// indirect` comment indicates that no package from the +required module is directly imported by any package in the main module. +The `go` command adds an indirect requirement when the selected version of a +module is higher than what is already implied (transitively) by the main +module's other dependencies. That may occur because of an explicit upgrade +(`go get -u`), removal of some other dependency that previously imposed the +requirement (`go mod tidy`), or a dependency that imports a package without +a corresponding requirement in its own `go.mod` file (such as a dependency +that lacks a `go.mod` file altogether). + +``` +RequireDirective = "require" ( RequireSpec | "(" newline { RequireSpec } ")" newline ) . +RequireSpec = ModulePath Version newline . +``` + +Example: + +``` +require golang.org/x/net v1.2.3 + +require ( + golang.org/x/crypto v1.4.5 // indirect + golang.org/x/text v1.6.7 +) +``` + + +### `exclude` directive + +An `exclude` directive prevents a module version from being loaded by the `go` +command. If an excluded version is referenced by a `require` directive in a +`go.mod` file, the `go` command will list available versions for the module (as +shown with `go list -m -versions`) and will load the next higher non-excluded +version instead. Both release and pre-release versions are considered for this +purpose, but pseudo-versions are not. If there are no higher versions, +the `go` command will report an error. Note that this [may +change](https://golang.org/issue/36465) in Go 1.15. + + + +`exclude` directives only apply in the main module's `go.mod` file and are +ignored in other modules. See [Minimal version +selection](#minimal-version-selection) for details. + +``` +ExcludeDirective = "exclude" ( ExcludeSpec | "(" newline { ExcludeSpec } ")" ) . +ExcludeSpec = ModulePath Version newline . +``` + +Example: + +``` +exclude golang.org/x/net v1.2.3 + +exclude ( + golang.org/x/crypto v1.4.5 + golang.org/x/text v1.6.7 +) +``` + + +### `replace` directive + +A `replace` directive replaces the contents of a specific version of a module, +or all versions of a module, with contents found elsewhere. The replacement +may be specified with either another module path and version, or a +platform-specific file path. + +If a version is present on the left side of the arrow (`=>`), only that specific +version of the module is replaced; other versions will be accessed normally. +If the left version is omitted, all versions of the module are replaced. + +If the path on the right side of the arrow is an absolute or relative path +(beginning with `./` or `../`), it is interpreted as the local file path to the +replacement module root directory, which must contain a `go.mod` file. The +replacement version must be omitted in this case. + +If the path on the right side is not a local path, it must be a valid module +path. In this case, a version is required. The same module version must not +also appear in the build list. + +Regardless of whether a replacement is specified with a local path or module +path, if the replacement module has a `go.mod` file, its `module` directive +must match the module path it replaces. + +`replace` directives only apply in the main module's `go.mod` file +and are ignored in other modules. See [Minimal version +selection](#minimal-version-selection) for details. + +``` +ReplaceDirective = "replace" ( ReplaceSpec | "(" newline { ReplaceSpec } ")" newline ")" ) . +ReplaceSpec = ModulePath [ Version ] "=>" FilePath newline + | ModulePath [ Version ] "=>" ModulePath Version newline . +FilePath = /* platform-specific relative or absolute file path */ +``` + +Example: + +``` +replace golang.org/x/net v1.2.3 => example.com/fork/net v1.4.5 + +replace ( + golang.org/x/net v1.2.3 => example.com/fork/net v1.4.5 + golang.org/x/net => example.com/fork/net v1.4.5 + golang.org/x/net v1.2.3 => ./fork/net + golang.org/x/net => ./fork/net +) +``` + + +### Automatic updates + +The `go` command automatically updates `go.mod` when it uses the module graph if +some information is missing or `go.mod` doesn't accurately reflect reality. For +example, consider this `go.mod` file: + +``` +module example.com/M + +require ( + example.com/A v1 + example.com/B v1.0.0 + example.com/C v1.0.0 + example.com/D v1.2.3 + example.com/E dev +) + +exclude example.com/D v1.2.3 +``` + +The update rewrites non-canonical version identifiers to +[canonical](#glos-canonical-version) semver form, so `example.com/A`'s `v1` +becomes `v1.0.0`, and `example.com/E`'s `dev` becomes the pseudo-version for the +latest commit on the `dev` branch, perhaps `v0.0.0-20180523231146-b3f5c0f6e5f1`. + +The update modifies requirements to respect exclusions, so the requirement on +the excluded `example.com/D v1.2.3` is updated to use the next available version +of `example.com/D`, perhaps `v1.2.4` or `v1.3.0`. + +The update removes redundant or misleading requirements. For example, if +`example.com/A v1.0.0` itself requires `example.com/B v1.2.0` and `example.com/C +v1.0.0`, then `go.mod`'s requirement of `example.com/B v1.0.0` is misleading +(superseded by `example.com/A`'s need for `v1.2.0`), and its requirement of +`example.com/C v1.0.0` is redundant (implied by `example.com/A`'s need for the +same version), so both will be removed. If the main module contains packages +that directly import packages from `example.com/B` or `example.com/C`, then the +requirements will be kept but updated to the actual versions being used. + +Finally, the update reformats the `go.mod` in a canonical formatting, so +that future mechanical changes will result in minimal diffs. The `go` command +will not update `go.mod` if only formatting changes are needed. + +Because the module graph defines the meaning of import statements, any commands +that load packages also use and therefore update `go.mod`, including `go build`, +`go get`, `go install`, `go list`, `go test`, `go mod graph`, `go mod tidy`, and +`go mod why`. + +The `-mod=readonly` flag prevents commands from automatically updating +`go.mod`. However, if a command needs to perform an action that would +update to `go.mod`, it will report an error. For example, if +`go build` is asked to build a package not provided by any module in the build +list, `go build` will report an error instead of looking up the module and +updating requirements in `go.mod`. -### Minimal version selection (MVS) +## Minimal version selection (MVS) -### Compatibility with non-module repositories +## Compatibility with non-module repositories ## Module-aware build commands @@ -234,6 +601,9 @@ comment. ### Vendoring + +### `go get` + ### `go mod download` @@ -599,7 +969,7 @@ for future `go` command invocations. ## Environment variables - + ## Glossary