Marcel van Lohuizen [Mon, 24 Dec 2012 15:53:25 +0000 (16:53 +0100)]
exp/norm: changed API of Iter.
Motivations:
- Simpler UI. Previous API proved a bit awkward for practical purposes.
- Iter is often used in cases where one want to be able to bail out early.
The old implementaton had too much look-ahead to be efficient.
Disadvantages:
- ASCII performance is bad. This is unavoidable for tiny iterations.
Example is included to show how to work around this.
Description:
Iter now iterates per boundary/segment. It returns a slice of bytes that
either points to the input bytes, the internal decomposition strings,
or the small internal buffer that each iterator has. In many cases, copying
bytes is avoided.
The method Seek was added to support jumping around the input without
having to reinitialize.
Details:
- Table adjustments: some decompositions exist of multiple segments.
Decompositions that are of this type are now marked so that Iter can
handle them separately.
- The old iterator had a different next function for different normal forms
that was assigned to a function pointer called by Next.
The new iterator uses this mechanism to switch between different modes
for handling different type of input as well. This greatly improves
performance for Hangul and ASCII. It is also used for multi-segment
decompositions.
- input is now a struct of sting and []byte, instead of an interface.
This simplifies optimizing the ASCII case.
Marcel van Lohuizen [Mon, 24 Dec 2012 15:42:29 +0000 (16:42 +0100)]
exp/locale/collate: include composed characters into the table. This eliminates
the need to decompose characters for the majority of cases. This considerably
speeds up collation while increasing the table size minimally.
To detect non-normalized strings, rather than relying on exp/norm, the table
now includes CCC information. The inclusion of this information does not
increase table size.
DETAILS
- Raw collation elements are now a struct that includes the CCC, rather
than a slice of ints.
- Builder now ensures that NFD and NFC counterparts are included in the table.
This also fixes a bug for Korean which is responsible for most of the growth
of the table size.
- As there is no more normalization step, code should now handle both strings
and byte slices as input. Introduced source type to facilitate this.
NOTES
- This change does not handle normalization correctly entirely for contractions.
This causes a few failures with the regtest. table_test.go contains a few
uncommented tests that can be enabled once this is fixed. The easiest is to
fix this once we have the new norm.Iter.
- Removed a test cases in table_test that covers cases that are now guaranteed
to not exist.
Dmitriy Vyukov [Mon, 24 Dec 2012 11:33:32 +0000 (15:33 +0400)]
runtime/race: make test driver print compilation errors
Currently it silently "succeeds" saying that it run 0 tests
if there are compilations errors.
With this change it fails and outputs the compilation error.
Russ Cox [Sat, 22 Dec 2012 22:23:33 +0000 (17:23 -0500)]
cmd/gc: add diagnostic for var, type, const named init
Before this CL, defining the variable worked fine, but then when
the implicit package-level init func was created, that caused a
name collision and a confusing error about the redeclaration.
Also add a test for issue 3705 (func init() needs body).
Russ Cox [Sat, 22 Dec 2012 20:06:28 +0000 (15:06 -0500)]
runtime: ignore failure from madvise
When we release memory to the OS, if the OS doesn't want us
to release it (for example, because the program executed
mlockall(MCL_FUTURE)), madvise will fail. Ignore the failure
instead of crashing.
Rémy Oudompheng [Sat, 22 Dec 2012 18:16:31 +0000 (19:16 +0100)]
cmd/gc: fix wrong interaction between inlining and embedded builtins.
The patch makes the compile user an ordinary package-local
symbol for the name of embedded fields of builtin type.
This is incompatible with the fix delivered for issue 2687
(revision 3c060add43fb) but fixes it in a different way, because
the explicit symbol on the field makes the typechecker able to
find it in lookdot.
remove zerostack compiler experiment; will do at link time instead
««« original CL description
cmd/gc: add GOEXPERIMENT=zerostack to clear stack on function entry
This is expensive but it might be useful in cases where
people are suffering from false positives during garbage
collection and are willing to trade the CPU time for getting
rid of the false positives.
On the other hand it only eliminates false positives caused
by other function calls, not false positives caused by dead
temporaries stored in the current function call.
The 5g/6g/8g changes were pulled out of the history, from
the last time we needed to do this (to work around a goto bug).
The code in go.h, lex.c, pgen.c is new but tiny.
Erik St. Martin [Sat, 22 Dec 2012 16:14:56 +0000 (11:14 -0500)]
regexp: fix index panic in Replace
When using subexpressions ($1) as replacements, when they either don't exist or values weren't found causes a panic.
This patch ensures that the match location isn't -1, to prevent out of bounds errors.
Fixes #3816.
Chris Jones [Sat, 22 Dec 2012 15:00:36 +0000 (10:00 -0500)]
encoding/xml: Marshal ",any" fields
Fixes #3559.
This makes Marshal handle fields marked ",any" instead of ignoring
them. That makes Marshal more symmetrical with Unmarshal, which seems
to have been a design goal.
Note some test cases were changed, because this patch changes
marshalling behavior. I think the previous behavior was buggy, but
there's still a backward-compatibility question to consider.
Dmitriy Vyukov [Fri, 21 Dec 2012 15:12:00 +0000 (19:12 +0400)]
runtime/race: update linux runtime to r170876.
This disables checks for limited address space
and unlimited stack. They are not required for Go.
Fixes #4577.
Rémy Oudompheng [Thu, 20 Dec 2012 22:27:28 +0000 (23:27 +0100)]
cmd/gc: remove an incorrect assertion in escape analysis.
A fatal error used to happen when escassign-ing a multiple
function return to a single node. However, the situation
naturally appears when using "go f(g())" or "defer f(g())",
because g() is escassign-ed to sink.
Joel Sing [Thu, 20 Dec 2012 14:43:19 +0000 (01:43 +1100)]
cgo: enable cgo on openbsd
Enable cgo on OpenBSD.
The OpenBSD ld.so(1) does not currently support PT_TLS sections. Work
around this by fixing up the TCB that has been provided by librthread
and reallocating a TCB with additional space for TLS. Also provide a
wrapper for pthread_create, allowing zeroed TLS to be allocated for
threads created externally to Go.
Joel Sing [Thu, 20 Dec 2012 14:27:50 +0000 (01:27 +1100)]
cmd/[568]l: do not generate PT_TLS on openbsd
The OpenBSD ld.so(1) does not currently support PT_TLS and refuses
to load ELF binaries that contain PT_TLS sections. Do not emit PT_TLS
sections - we will handle this appropriately in runtime/cgo instead.
Robert Griesemer [Mon, 17 Dec 2012 19:35:59 +0000 (11:35 -0800)]
exp/types: completed typechecking of parameter passing
Details:
- fixed variadic parameter passing and calls of the form f(g())
- fixed implementation of ^x for unsigned constants x
- fixed assignability of untyped booleans
- resolved a few TODOs, various minor fixes
- enabled many more tests (only 6 std packages don't typecheck)
Russ Cox [Mon, 17 Dec 2012 19:32:26 +0000 (14:32 -0500)]
cmd/gc: add GOEXPERIMENT=zerostack to clear stack on function entry
This is expensive but it might be useful in cases where
people are suffering from false positives during garbage
collection and are willing to trade the CPU time for getting
rid of the false positives.
On the other hand it only eliminates false positives caused
by other function calls, not false positives caused by dead
temporaries stored in the current function call.
The 5g/6g/8g changes were pulled out of the history, from
the last time we needed to do this (to work around a goto bug).
The code in go.h, lex.c, pgen.c is new but tiny.
Shenghou Ma [Mon, 17 Dec 2012 17:26:48 +0000 (01:26 +0800)]
fmt, encoding/gob: fix misuse of Read
reader.Read() can return both 0,nil and len(buf),err.
To be safe, we use io.ReadFull instead of doing reader.Read directly.
Anthony Martin [Mon, 17 Dec 2012 16:33:51 +0000 (08:33 -0800)]
syscall: lazily populate the environment cache on Plan 9
This decreases the amount of system calls during the
first call to Getenv. Calling Environ will still read
in all environment variables and populate the cache.
Dmitriy Vyukov [Mon, 17 Dec 2012 08:55:41 +0000 (12:55 +0400)]
cmd/gc: racewalk: fix compiler crash
The code:
func main() {
v := make([]int64, 10)
i := 1
_ = v[(i*4)/3]
}
crashes compiler with:
Program received signal SIGSEGV, Segmentation fault.
0x000000000043c274 in walkexpr (np=0x7fffffffc9b8, init=0x0) at src/cmd/gc/walk.c:587
587 *init = concat(*init, n->ninit);
(gdb) bt
#0 0x000000000043c274 in walkexpr (np=0x7fffffffc9b8, init=0x0) at src/cmd/gc/walk.c:587
#1 0x0000000000432d15 in copyexpr (n=0x7ffff7f69a48, t=<optimized out>, init=0x0) at src/cmd/gc/subr.c:2020
#2 0x000000000043f281 in walkdiv (init=0x0, np=0x7fffffffca70) at src/cmd/gc/walk.c:2901
#3 walkexpr (np=0x7ffff7f69760, init=0x0) at src/cmd/gc/walk.c:956
#4 0x000000000043d801 in walkexpr (np=0x7ffff7f69bc0, init=0x0) at src/cmd/gc/walk.c:988
#5 0x000000000043cc9b in walkexpr (np=0x7ffff7f69d38, init=0x0) at src/cmd/gc/walk.c:1068
#6 0x000000000043c50b in walkexpr (np=0x7ffff7f69f50, init=0x0) at src/cmd/gc/walk.c:879
#7 0x000000000043c50b in walkexpr (np=0x7ffff7f6a0c8, init=0x0) at src/cmd/gc/walk.c:879
#8 0x0000000000440a53 in walkexprlist (l=0x7ffff7f6a0c8, init=0x0) at src/cmd/gc/walk.c:357
#9 0x000000000043d0bf in walkexpr (np=0x7fffffffd318, init=0x0) at src/cmd/gc/walk.c:566
#10 0x00000000004402bf in vmkcall (fn=<optimized out>, t=0x0, init=0x0, va=0x7fffffffd368) at src/cmd/gc/walk.c:2275
#11 0x000000000044059a in mkcall (name=<optimized out>, t=0x0, init=0x0) at src/cmd/gc/walk.c:2287
#12 0x000000000042862b in callinstr (np=0x7fffffffd4c8, init=0x7fffffffd568, wr=0, skip=<optimized out>) at src/cmd/gc/racewalk.c:478
#13 0x00000000004288b7 in racewalknode (np=0x7ffff7f68108, init=0x7fffffffd568, wr=0, skip=0) at src/cmd/gc/racewalk.c:287
#14 0x0000000000428781 in racewalknode (np=0x7ffff7f65840, init=0x7fffffffd568, wr=0, skip=0) at src/cmd/gc/racewalk.c:302
#15 0x0000000000428abd in racewalklist (l=0x7ffff7f65840, init=0x0) at src/cmd/gc/racewalk.c:97
#16 0x0000000000428d0b in racewalk (fn=0x7ffff7f5f010) at src/cmd/gc/racewalk.c:63
#17 0x0000000000402b9c in compile (fn=0x7ffff7f5f010) at src/cmd/6g/../gc/pgen.c:67
#18 0x0000000000419f86 in funccompile (n=0x7ffff7f5f010, isclosure=0) at src/cmd/gc/dcl.c:1414
#19 0x0000000000424161 in p9main (argc=<optimized out>, argv=<optimized out>) at src/cmd/gc/lex.c:431
#20 0x0000000000401739 in main (argc=<optimized out>, argv=<optimized out>) at src/lib9/main.c:35
Jan Ziak [Mon, 17 Dec 2012 00:32:12 +0000 (19:32 -0500)]
runtime: struct Obj in mgc0.c and buffers in scanblock()
Details:
- This CL is the conceptual skeleton of code found in CL 6114046
- The garbage collector uses struct Obj to specify memory blocks
- scanblock() is putting found memory blocks into an intermediate buffer
(xbuf) before adding/flushing them to the main work buffer (wbuf)
- The main loop in scanblock() is replaced with a skeleton code that
in the future will be able to recognize the type of objects and
thus will improve the garbage collector's precision.
For now, all objects are simply sequences of pointers so
the precision of the garbage collector remains unchanged.
- The code plugs .gcdata and .gcbss sections into the garbage collector.
scanblock() in this CL is unable to make any use of this.
Mikio Hara [Sun, 16 Dec 2012 02:51:47 +0000 (11:51 +0900)]
net: change ListenUnixgram signature to return UnixConn instead of UDPConn
This CL breaks Go 1 API compatibility but it doesn't matter because
previous ListenUnixgram doesn't work in any use cases, oops.
The public API change is:
-pkg net, func ListenUnixgram(string, *UnixAddr) (*UDPConn, error)
+pkg net, func ListenUnixgram(string, *UnixAddr) (*UnixConn, error)
Fixes #3875.
R=rsc, golang-dev, dave
CC=golang-dev
https://golang.org/cl/6937059
Alex Brainman [Fri, 14 Dec 2012 06:33:59 +0000 (17:33 +1100)]
cmd/go: handle os signals
Ignore signals during "go run" and wait for running child
process to exit. Stop executing further tests during "go test",
wait for running tests to exit and report error exit code.
Ian Lance Taylor [Thu, 13 Dec 2012 23:11:31 +0000 (15:11 -0800)]
test: s/float/float32/
I just committed a patch to gccgo that notices that float was
never defined, causing an additional unmatched error message.
Rename the type to avoid that message.
Nick Craig-Wood [Thu, 13 Dec 2012 21:02:39 +0000 (13:02 -0800)]
os: Improve the accuracy of os.Chtimes
I've been writing some code which involves syncing files (like
rsync) and it became apparent that under Linux I could read
modification times (os.Lstat) with nanosecond precision but
only write them with microsecond precision. This difference
in precision is rather annoying when trying to discover
whether files need syncing or not!
I've patched syscall and os to increases the accuracy of of
os.Chtimes for Linux and Windows. This involved exposing the
utimensat system call under Linux and a bit of extra code
under Windows. I decided not to expose the "at" bit of the
system call as it is impossible to replicate under Windows, so
the patch adds syscall.Utimens() to all architectures along
with a ImplementsUtimens flag.
If the utimensat syscall isn't available (utimensat was added
to Linux in 2.6.22, Released, 8 July 2007) then it silently
falls back to the microsecond accuracy version it uses now.
The improved accuracy for Windows should be good for all
versions of Windows.
Unfortunately Darwin doesn't seem to have a utimensat system
call that I could find so I couldn't implement it there. The
BSDs do, but since they share their syscall implementation
with Darwin I couldn't figure out how to define a syscall for
*BSD and not Darwin. I've left this as a TODO in the code.
In the process I implemented the missing methods for Timespec
under Windows which I needed which just happened to round out
the Timespec API for all platforms!