Russ Cox [Mon, 7 Mar 2011 20:10:01 +0000 (15:10 -0500)]
gc: unsafe.Pointer is not a pointer
Change unsafe.Pointer to be its own kind of
type, instead of making it equivalent to *any.
The change complicates import and export
but avoids the need to find all the places that
operate on pointers but should not operate on
unsafe.Pointer.
Robert Griesemer [Mon, 7 Mar 2011 19:01:23 +0000 (11:01 -0800)]
go/ast, go/parser: populate identifier scopes at parse time
The parser populates all scopes for a given file (except
type-related scopes for structs, interfaces, and methods
of types) at parse time.
A new parser flag, DeclarationErrors, enables error messages
related to declaration errors (as far as it is possible to
provide them).
The resulting AST has all (non-field, non-method) identifiers
resolved that can be resolved w/o doing imports or parsing
missing package files.
The ast.File node contains the (partially complete)
package scope and a list of unresolved global identifiers.
All type-specific data structures have been removed from the AST.
The existing typechecker is functional but needs to be adjusted
(simplified) accordingly. Utility functions to resolve all
identifiers for a package (after handling imports and parsing
all package files) are missing.
Unrelated changes:
- Rename typechecker/testdata files to that they are not considered
by gofmt.
- Minor cleanups/simplifications.
Parses all .go files in src and misc without declaration errors.
Runs all tests. Changes do not affect gofmt output.
Gustavo Niemeyer [Mon, 7 Mar 2011 17:53:39 +0000 (12:53 -0500)]
goinstall: handle .c files with gc when cgo isn't used
As a data point, this enables goinstall to handle the standard
syscall package almost unchanged (there's one file with the _bsd
extension, and a .c file which isn't supposed to be compiled in).
Russ Cox [Mon, 7 Mar 2011 15:37:42 +0000 (10:37 -0500)]
runtime: scheduler, cgo reorganization
* Change use of m->g0 stack (aka scheduler stack).
* Provide runtime.mcall(f) to invoke f() on m->g0 stack.
* Replace scheduler loop entry with runtime.mcall(schedule).
Runtime.mcall eliminates the need for fake scheduler states that
exist just to run a bit of code on the m->g0 stack
(Grecovery, Gstackalloc).
The elimination of the scheduler as a loop that stops and
starts using gosave and gogo fixes a bad interaction with the
way cgo uses the m->g0 stack. Cgo runs external (gcc-compiled)
C functions on that stack, and then when calling back into Go,
it sets m->g0->sched.sp below the added call frames, so that
other uses of m->g0's stack will not interfere with those frames.
Unfortunately, gogo (longjmp) back to the scheduler loop at
this point would end up running scheduler with the lower
sp, which no longer points at a valid stack frame for
a call to scheduler. If scheduler then wrote any function call
arguments or local variables to where it expected the stack
frame to be, it would overwrite other data on the stack.
I realized this possibility while debugging a problem with
calling complex Go code in a Go -> C -> Go cgo callback.
This wasn't the bug I was looking for, it turns out, but I believe
it is a real bug nonetheless. Switching to runtime.mcall, which
only adds new frames to the stack and never jumps into
functions running in existing ones, fixes this bug.
* Move cgo-related code out of proc.c into cgocall.c.
* Add very large comment describing cgo call sequences.
* Simpilify, regularize cgo function implementations and names.
* Add test suite as misc/cgo/test.
Now the Go -> C path calls cgocall, which calls asmcgocall,
and the C -> Go path calls cgocallback, which calls cgocallbackg.
The shuffling, which affects mainly the callback case, moves
most of the callback implementation to cgocallback running
on the m->curg stack (not the m->g0 scheduler stack) and
only while accounted for with $GOMAXPROCS (between calls
to exitsyscall and entersyscall).
The previous callback code did not block in startcgocallback's
approximation to exitsyscall, so if, say, the garbage collector
were running, it would still barge in and start doing things
like call malloc. Similarly endcgocallback's approximation of
entersyscall did not call matchmg to kick off new OS threads
when necessary, which caused the bug in issue 1560.
Gustavo Niemeyer [Sun, 6 Mar 2011 23:05:57 +0000 (18:05 -0500)]
cgo: fix dwarf type parsing
The recursive algorithm used to parse types in cgo
has a bug related to building the C type representation.
As an example, when the recursion starts at a type *T,
the C type representation won't be known until type T
itself is parsed. But then, it is possible that type T
references the type **T internally. The latter
representation is built based on the one of *T, which
started the recursion, so it won't attempt to parse it
again, and will instead use the current representation
value for *T, which is still empty at this point.
This problem was fixed by introducing a simple TypeRepr
type which builds the string representation lazily,
analogous to how the Go type information is built within
the same algorithm. This way, even if a type
representation is still unknown at some level in the
recursion, representations dependant on it can still
be created correctly.
Gustavo Niemeyer [Sun, 6 Mar 2011 22:33:23 +0000 (17:33 -0500)]
path/filepath: new OS-specific path support
The path package now contains only functions which
deal with slashed paths, sensible for any OS when dealing
with network paths or URLs. OS-specific functionality
has been moved into the new path/filepath package.
This also includes fixes for godoc, goinstall and other
packages which were mixing slashed and OS-specific paths.
Russ Cox [Sun, 6 Mar 2011 22:04:24 +0000 (17:04 -0500)]
build: remove old cgo files
Cgo changed to write these files into _obj, but some
trees may still have the old ones in the source directory.
They need to be removed during make clean so that
a subsequent build will use the ones in _obj.
Devon H. O'Dell [Sun, 6 Mar 2011 19:57:05 +0000 (14:57 -0500)]
syscall: work around FreeBSD execve kernel bug
FreeBSD's execve implementation has an integer underflow in a bounds test which
causes it to erroneously think the argument list is too long when argv[0] is
longer than interpreter + path.
Rob Pike [Sun, 6 Mar 2011 00:05:35 +0000 (16:05 -0800)]
gob: don't send type information about unexported fields.
A change a while back stop sending data for unexported fields
but due to an oversight the type info was being sent also. It's
inconsequential but wrong to do that.
Brad Fitzpatrick [Sat, 5 Mar 2011 21:51:35 +0000 (13:51 -0800)]
httptest: introduce TempServer, clean up tests
This also breaks fs_test into two parts
as the range tests test http's private httpRange
and I had to change the fs_test package from
"http" to "http_test" to use httptest which otherwise
has a cyclic depedency back on http.
Aside: we should start exposing the Range
stuff in the future.
Russ Cox [Sat, 5 Mar 2011 19:35:03 +0000 (14:35 -0500)]
compress/flate: fix test
The test was checking for a buffer to be empty but
actually racing with the background goroutine that
was emptying it. Left a comment so that the check
is not reintroduced later.
Russ Cox [Sat, 5 Mar 2011 19:24:44 +0000 (14:24 -0500)]
cgo: use correct frame size for 0 arguments
Passing a frame size of 1 was causing the cgo callback
to push 1 byte of arguments onto the stack, making
the stack pointer misaligned, which had the effect of
hiding all the pointers on the stack from the garbage
collector.
SWIG only wraps calls to C++ virtual methods, so it
always has at least 1 argument, so SWIG does not need
to be fixed too.
Rob Pike [Fri, 4 Mar 2011 22:18:52 +0000 (14:18 -0800)]
gob: enable the GobEncoder and GobDecoder interfaces.
These allow data items to control their own representation.
For now, the implementation requires that the value passed
to Encode and Decode must be exactly the type of the
methods' receiver; it cannot be, for instance, T if the receiver
is of type *T. This will be fixed in a later CL.
Rob Pike [Fri, 4 Mar 2011 20:25:18 +0000 (12:25 -0800)]
gob: beginning of support for GobEncoder/GobDecoder interfaces.
This allows a data item that can marshal itself to be transmitted by its
own encoding, enabling some types to be handled that cannot be
normally, plus providing a way to use gobs on data with unexported
fields.
In this CL, the necessary methods are protected by leading _, so only
package gob can use the facilities (in its tests, of course); this
code is not ready for real use yet. I could be talked into enabling
it for experimentation, though. The main drawback is that the
methods must be implemented by the actual type passed through,
not by an indirection from it. For instance, if *T implements
GobEncoder, you must send a *T, not a T. This will be addressed
in due course.
Also there is improved commentary and a couple of unrelated
minor bug fixes.
Brad Fitzpatrick [Thu, 3 Mar 2011 20:22:13 +0000 (12:22 -0800)]
http: allow handlers to send non-chunked responses
Currently all http handlers reply to HTTP/1.1 requests with
chunked responses. This patch allows handlers to opt-out of
that behavior by pre-declaring their Content-Length (which is
then enforced) and unsetting their Transfer-Encoding or
setting it to the "identity" encoding.
Russ Cox [Thu, 3 Mar 2011 19:51:49 +0000 (14:51 -0500)]
io/ioutil: add TempDir
It's a little confusing that os.TempDir and ioutil.TempDir have
different meanings. I don't know what to change the names to,
if anything. At least they also have different signatures.
Roger Peppe [Thu, 3 Mar 2011 18:43:29 +0000 (10:43 -0800)]
fmt: make recursive scan more efficient.
Detect when scan is being called recursively and
re-use the same scan state.
On my machine, for a recursion-heavy benchmark, this
results in 44x speed up. This does impose a 4% penalty
on the non-recursive case, which can be removed by
heap-allocating the saved state, at 40% performance penalty
on the recursive case. Either way is fine with me.
Adam Langley [Thu, 3 Mar 2011 14:13:06 +0000 (09:13 -0500)]
bzip2: speed up decompression.
This borrows a trick from the bzip2 source and effects a decent speed
up when decompressing highly compressed sources. Rather than unshuffle
the BTW block when performing the IBTW, a linked-list is threaded
through the array, in place. This improves cache hit rates.
Gustavo Niemeyer [Wed, 2 Mar 2011 21:18:17 +0000 (16:18 -0500)]
gc: fix init of packages named main
This change removes the special case which existed
for handling the initalization of the main package,
so that other modules named 'main' get properly
initialized when imported.
Note that gotest of main packages will break in most
cases without this.
Roger Peppe [Wed, 2 Mar 2011 20:04:08 +0000 (15:04 -0500)]
goinstall: protect against malicious filenames.
It was possible to make package run arbitrary
commands when installing if its filenames contained
make metacharacters.
Roger Peppe [Wed, 2 Mar 2011 19:22:33 +0000 (14:22 -0500)]
cgo: put temporary source files in _obj.
Fixes #1572.
Initially I tried changing things so all object
files get put in _obj, but it's too much - everything
needs changing. Perhaps some other time.
Roger Peppe [Wed, 2 Mar 2011 18:54:23 +0000 (10:54 -0800)]
fmt: allow recursive calls to Fscan etc.
Add a new Read method to ScanState so that it
satisfies the io.Reader interface; rename
Getrune and Ungetrune to ReadRune and UnreadRune.
Make sure ReadRune does not read past width restrictions;
remove now-unnecessary Width method from ScanState.
Also make the documentation a little clearer as to
how ReadRune and UnreadRune are used.
Rob Pike [Tue, 1 Mar 2011 21:54:22 +0000 (13:54 -0800)]
docs: make "runtime" a word only as a name for the package.
Computer people have an agglutinating streak that I like to resist.
As a time of execution: run time.
As an adjective: run-time.
As a noun: run-time support/code/library.