From: Russ Cox Date: Thu, 28 Feb 2013 18:44:29 +0000 (-0800) Subject: cmd/cgo: extend implementation comment X-Git-Tag: go1.1rc2~778 X-Git-Url: http://www.git.cypherpunks.su/?a=commitdiff_plain;h=b89000bcabe16644d877e8d1251a3e6a784153d9;p=gostls13.git cmd/cgo: extend implementation comment This is the plan for how to make host linking work with the rest of the system. There are two complications: 1. It is a goal to preserve the property that pure Go programs (even ones importing "net") can be compiled without needing gcc, so that a Go toolchain download works out of the box. This forces the support for two linking modes: with and without gcc. 2. It is a goal to allow users with old copies of SWIG to continue to use those copies. This forces the support for "internal only" packages. Perhaps it is reasonable to require a new SWIG. I don't know. R=iant CC=golang-dev https://golang.org/cl/7433043 --- diff --git a/src/cmd/cgo/doc.go b/src/cmd/cgo/doc.go index 6dac10a096..4738304858 100644 --- a/src/cmd/cgo/doc.go +++ b/src/cmd/cgo/doc.go @@ -396,4 +396,230 @@ and libcgo_thread_start to a gcc-compiled function that can be used to create a new thread, in place of the runtime's usual direct system calls. +[NOTE: From here down is planned but not yet implemented.] + +Internal and External Linking + +The text above describes "internal" linking, in which 6l parses and +links host object files (ELF, Mach-O, PE, and so on) into the final +executable itself. Keeping 6l simple means we cannot possibly +implement the full semantics of the host linker, so the kinds of +objects that can be linked directly into the binary is limited (other +code can only be used as a dynamic library). On the other hand, when +using internal linking, 6l can generate Go binaries by itself. + +In order to allow linking arbitrary object files without requiring +dynamic libraries, cgo will soon support an "external" linking mode +too. In external linking mode, 6l does not process any host object +files. Instead, it collects all the Go code and writes a single go.o +object file containing it. Then it invokes the host linker (usually +gcc) to combine the go.o object file and any supporting non-Go code +into a final executable. External linking avoids the dynamic library +requirement but introduces a requirement that the host linker be +present to create such a binary. + +Most builds both compile source code and invoke the linker to create a +binary. When cgo is involved, the compile step already requires gcc, so +it is not problematic for the link step to require gcc too. + +An important exception is builds using a pre-compiled copy of the +standard library. In particular, package net uses cgo on most systems, +and we want to preserve the ability to compile pure Go code that +imports net without requiring gcc to be present at link time. (In this +case, the dynamic library requirement is less significant, because the +only library involved is libc.so, which can usually be assumed +present.) + +This conflict between functionality and the gcc requirement means we +must support both internal and external linking, depending on the +circumstances: if net is the only cgo-using package, then internal +linking is probably fine, but if other packages are involved, so that there +are dependencies on libraries beyond libc, external linking is likely +to work better. The compilation of a package records the relevant +information to support both linking modes, leaving the decision +to be made when linking the final binary. + +Linking Directives + +In either linking mode, package-specific directives must be passed +through to 6l. These are communicated by writing #pragma directives +in a C source file compiled by 6c. The directives are copied into the .6 object file +and then processed by the linker. + +The directives are: + +#pragma cgo_dynamic_import [ [""]] + + In internal linking mode, allow an unresolved reference to + , assuming it will be resolved by a dynamic library + symbol. The optional specifies the symbol's name and + possibly version in the dynamic library, and the optional "" + names the specific library where the symbol should be found. + + In the , # or @ can be used to introduce a symbol version. + + Examples: + #pragma cgo_dynamic_import puts + #pragma cgo_dynamic_import puts puts#GLIBC_2.2.5 + #pragma cgo_dynamic_import puts puts#GLIBC_2.2.5 "libc.so.6" + + A side effect of the cgo_dynamic_import directive with a + library is to make the final binary depend on that dynamic + library. To get the dependency without importing any specific + symbols, use _ for local and remote. + + Example: + #pragma cgo_dynamic_import _ _ "libc.so.6" + + For compatibility with current versions of SWIG, + #pragma dynimport is an alias for #pragma cgo_dynamic_import. + +#pragma cgo_dynamic_linker "" + + In internal linking mode, use "" as the dynamic linker + in the final binary. This directive is only needed from one + package when constructing a binary; by convention it is + supplied by runtime/cgo. + + Example: + #pragma cgo_dynamic_linker "/lib/ld-linux.so.2" + +#pragma cgo_export + + In both internal and external linking modes, put the Go symbol + named into the program's exported symbol table as + , so that C code can refer to it by that name. This + mechanism makes it possible for C code to call back into Go or + to share Go's data. + + For compatibility with current versions of SWIG, + #pragma dynexport is an alias for #pragma cgo_export. + +#pragma cgo_static_import + + In external linking mode, allow unresolved references to + in the go.o object file prepared for the host linker, + under the assumption that will be supplied by the + other object files that will be linked with go.o. + + Example: + #pragma cgo_static_import puts_wrapper + +#pragma cgo_ldflag "" + + In external linking mode, invoke the host linker (usually gcc) + with "" as a command-line argument following the .o files. + Note that the arguments are for "gcc", not "ld". + + Example: + #pragma cgo_ldflag "-lpthread" + #pragma cgo_ldflag "-L/usr/local/sqlite3/lib" + +A package compiled with cgo will include directives for both +internal and external linking; the linker will select the appropriate +subset for the chosen linking mode. + +Example + +As a simple example, consider a package that uses cgo to call C.sin. +The following code will be generated by cgo: + + // compiled by 6g + + type _Ctype_double float64 + func _Cfunc_sin(_Ctype_double) _Ctype_double + + // compiled by 6c + + #pragma cgo_dynamic_import sin sin#GLIBC_2.2.5 "libm.so.6" + #pragma cgo_dynamic_linker "/lib/ld-linux.so.2" + + #pragma cgo_static_import _cgo_gcc_Cfunc_sin + #pragma cgo_ldflag "-lm" + + void _cgo_gcc_Cfunc_sin(void*); + + void + ·_Cfunc_sin(struct{uint8 x[16];}p) + { + runtime·cgocall(_cgo_gcc_Cfunc_sin, &p); + } + + // compiled by gcc, into foo.cgo2.o + + void + _cgo_gcc_Cfunc_sin(void *v) + { + struct { + double p0; + double r; + } __attribute__((__packed__)) *a = v; + a->r = sin(a->p0); + } + +What happens at link time depends on whether the final binary is linked +using the internal or external mode. If other packages are compiled in +"external only" mode, then the final link will be an external one. +Otherwise the link will be an internal one. + +The directives in the 6c-compiled file are used according to the kind +of final link used. + +In internal mode, 6l itself processes all the host object files, in +particular foo.cgo2.o. To do so, it uses the cgo_dynamic_import and +cgo_dynamic_linker directives to learn that the otherwise undefined +reference to sin in foo.cgo2.o should be rewritten to refer to the +symbol sin with version GLIBC_2.2.5 from the dynamic library +"libm.so.6", and the binary should request "/lib/ld-linux.so.2" as its +runtime dynamic linker. + +In external mode, 6l does not process any host object files, in +particular foo.cgo2.o. It links together the 6g- and 6c-generated +object files, along with any other Go code, into a go.o file. While +doing that, 6l will discover that there is no definition for +_cgo_gcc_Cfunc_sin, referred to by the 6c-compiled source file. This +is okay, because 6l also processes the cgo_static_import directive and +knows that _cgo_gcc_Cfunc_sin is expected to be supplied by a host +object file, so 6l does not treat the missing symbol as an error when +creating go.o. Indeed, the definition for _cgo_gcc_Cfunc_sin will be +provided to the host linker by foo2.cgo.o, which in turn will need the +symbol 'sin'. 6l also processes the cgo_ldflag directives, so that it +knows that the eventual host link command must include the -lm +argument, so that the host linker will be able to find 'sin' in the +math library. + +6l Command Line Interface + +The go command and any other Go-aware build systems invoke 6l +to link a collection of packages into a single binary. By default, 6l will +present the same interface it does today: + + 6l main.a + +produces a file named 6.out, even if 6l does so by invoking the host +linker in external linking mode. + +By default, 6l will decide the linking mode as follows: if the only +packages using cgo are those on a whitelist of standard library +packages (net, os/user, runtime/cgo), 6l will use internal linking +mode. Otherwise, there are non-standard cgo packages involved, and 6l +will use external linking mode. The first rule means that a build of +the godoc binary, which uses net but no other cgo, can run without +needing gcc available. The second rule means that a build of a +cgo-wrapped library like sqlite3 can generate a standalone executable +instead of needing to refer to a dynamic library. The specific choice +can be overridden using a command line flag: 6l -cgolink=internal or +6l -cgolink=external. + +In an external link, 6l will create a temporary directory, write any +host object files found in package archives to that directory (renamed +to avoid conflicts), write the go.o file to that directory, and invoke +the host linker. The default value for the host linker is $CC, split +into fields, or else "gcc". The specific host linker command line can +be overridden using a command line flag: 6l -hostld='gcc -ggdb' + +These defaults mean that Go-aware build systems can ignore the linking +changes and keep running plain '6l' and get reasonable results, but +they can also control the linking details if desired. + */