Commit Graph

18349 Commits

Author SHA1 Message Date
Dmitriy Vyukov
d5a36cd6bb runtime: zero 2-word memory blocks in-place
Currently for 2-word blocks we set the flag to clear the flag. Makes no sense.
In particular on 32-bits we call memclr always.

R=golang-codereviews, dave, iant
CC=golang-codereviews, khr, rsc
https://golang.org/cl/41170044
2014-01-21 10:53:51 +04:00
Dmitriy Vyukov
b039abfc3e runtime: fix specials deadlock
The deadlock is between span->specialLock and proflock:

goroutine 11 [running]:
runtime.MProf_Free(0x7fa272d26508, 0xc210054180, 0xc0)
        src/pkg/runtime/mprof.goc:220 +0x27
runtime.freespecial(0x7fa272d1e088, 0xc210054180, 0xc0)
        src/pkg/runtime/mheap.c:691 +0x6a
runtime.freeallspecials(0x7fa272d1af50, 0xc210054180, 0xc0)
        src/pkg/runtime/mheap.c:717 +0xb5
runtime.free(0xc210054180)
        src/pkg/runtime/malloc.goc:190 +0xfd
selectgo(0x7fa272a5ef58)
        src/pkg/runtime/chan.c:1136 +0x2d8
runtime.selectgo(0xc210054180)
        src/pkg/runtime/chan.c:840 +0x12
runtime_test.func·058()
        src/pkg/runtime/proc_test.go:146 +0xb4
runtime.goexit()
        src/pkg/runtime/proc.c:1405
created by runtime_test.TestTimerFairness
        src/pkg/runtime/proc_test.go:152 +0xd1

goroutine 12 [running]:
addspecial(0xc2100540c0, 0x7fa272d1e0a0)
        src/pkg/runtime/mheap.c:569 +0x88
runtime.setprofilebucket(0xc2100540c0, 0x7fa272d26508)
        src/pkg/runtime/mheap.c:668 +0x73
runtime.MProf_Malloc(0xc2100540c0, 0xc0, 0x0)
        src/pkg/runtime/mprof.goc:212 +0x16b
runtime.mallocgc(0xc0, 0x0, 0xc200000000)
        src/pkg/runtime/malloc.goc:142 +0x239
runtime.mal(0xbc)
        src/pkg/runtime/malloc.goc:703 +0x38
newselect(0x2, 0x7fa272a5cf60)
        src/pkg/runtime/chan.c:632 +0x53
runtime.newselect(0xc200000002, 0xc21005f000)
        src/pkg/runtime/chan.c:615 +0x28
runtime_test.func·058()
        src/pkg/runtime/proc_test.go:146 +0x37
runtime.goexit()
        src/pkg/runtime/proc.c:1405
created by runtime_test.TestTimerFairness
        src/pkg/runtime/proc_test.go:152 +0xd1

Fixes #7099.

R=golang-codereviews, khr
CC=golang-codereviews
https://golang.org/cl/53120043
2014-01-21 10:48:37 +04:00
Dmitriy Vyukov
d76a1e593c runtime: fix test on windows
The test prints an excessive \n when /dev/null is not present.

R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/54890043
2014-01-21 10:44:08 +04:00
Dmitriy Vyukov
bfd3c223f9 net: fix data race in test
Fixes #7157.

R=alex.brainman, bradfitz
CC=golang-codereviews
https://golang.org/cl/54880043
2014-01-21 10:35:27 +04:00
Dmitriy Vyukov
90eca36a23 runtime: ensure fair scheduling during frequent GCs
What was happenning is as follows:
Each writer goroutine always triggers GC during its scheduling quntum.
After GC goroutines are shuffled so that the timer goroutine is always second in the queue.
This repeats infinitely, causing timer goroutine starvation.
Fixes #7126.

R=golang-codereviews, shanemhansen, khr, khr
CC=golang-codereviews
https://golang.org/cl/53080043
2014-01-21 10:24:42 +04:00
Brad Fitzpatrick
cfb9cf0f06 expvar: sort maps, fix race
It's pretty distracting to use expvar with the output of both
the top-level map and map values jumping around randomly.

Also fixes a potential race where multiple clients trying to
increment a map int or float key at the same time could lose
updates.

R=golang-codereviews, couchmoney
CC=golang-codereviews
https://golang.org/cl/54320043
2014-01-20 09:59:23 -08:00
Brad Fitzpatrick
6592aeb8f3 net/http, net/http/httputil: make chunked reader alloc test more robust
Use testing.AllocsPerRun now that it exists, instead of doing it by hand.

Fixes #6076

R=golang-codereviews, alex.brainman
CC=golang-codereviews
https://golang.org/cl/53810043
2014-01-19 10:02:10 -08:00
Michael Gehring
081e2d0153 syscall: add syscall.Termios on netbsd
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/54290043
2014-01-19 09:57:02 -08:00
Mikio Hara
b51e15780f syscall: fix typo
R=r
CC=golang-codereviews
https://golang.org/cl/54040043
2014-01-18 16:02:59 +09:00
Keith Randall
6c9f198c9a runtime: print stack trace when "panic during panic"
Fixes bug 7145

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53970043
2014-01-17 18:47:40 -08:00
Mikio Hara
985893acff syscall: fix build
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/54000043
2014-01-18 11:22:32 +09:00
Mikio Hara
3baa98f26d syscall: make getrlimit, setrlimit tests work on unix variants
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53690043
2014-01-18 08:34:31 +09:00
Keith Randall
6f6a9445c9 runtime, cmd/gc: Get rid of vararg channel calls.
Vararg C calls present a problem for the GC because the
argument types are not derivable from the signature.  Remove
them by passing pointers to channel elements instead of the
channel elements directly.

R=golang-codereviews, gobot, rsc, dvyukov
CC=golang-codereviews
https://golang.org/cl/53430043
2014-01-17 14:48:45 -08:00
Russ Cox
98178b345a runtime: fix TestLFStackStress
Fixes #7138.

R=r, bradfitz, dave
CC=dvyukov, golang-codereviews
https://golang.org/cl/53910043
2014-01-17 17:42:24 -05:00
Russ Cox
5c9585953f undo CL 45930043 / c22889382a17
The compiler change is an ugly hack.
We can do better.

««« original CL description
syscall: mark arguments to Syscall as noescape
Heap arguments to "async" syscalls will break when/if we have moving GC anyway.
With this change is must not break until moving GC, because a user must
reference the object in Go to preserve liveness. Otherwise the code is broken already.
Reduces number of leaked params from 125 to 36 on linux.

R=golang-codereviews, mikioh.mikioh, bradfitz
CC=cshapiro, golang-codereviews, khr, rsc
https://golang.org/cl/45930043
»»»

R=golang-codereviews, r
CC=bradfitz, dvyukov, golang-codereviews
https://golang.org/cl/53870043
2014-01-17 16:58:14 -05:00
Rob Pike
451667a67f syscall: allocate 64 bits of "basep" for Getdirentries
Recent crashes on 386 Darwin appear to be caused by this system call
smashing the stack. Phenomenology shows that allocating more data
here addresses the probem.
The guess is that since the actual system call is getdirentries64, 64 is
what we should allocate.

Should fix the darwin/386 build.

R=rsc
CC=golang-codereviews
https://golang.org/cl/53840043
2014-01-17 13:19:00 -08:00
David Symonds
1e67453d93 doc: replace "private" with "unexported" in Effective Go.
R=r
CC=golang-codereviews
https://golang.org/cl/53720043
2014-01-17 09:06:39 -08:00
David Symonds
23e72645dd regexp: remove unnecessary sentence in doc comment.
R=rsc, iant
CC=golang-codereviews
https://golang.org/cl/53190046
2014-01-17 09:06:28 -08:00
Adam Langley
77e6ceb75a A+C: Maxim Khitrov (individual CLA)
Generated by addca.

R=gobot
CC=golang-codereviews
https://golang.org/cl/53730043
2014-01-17 11:26:42 -05:00
Dmitriy Vyukov
fc37eba149 syscall: mark arguments to Syscall as noescape
Heap arguments to "async" syscalls will break when/if we have moving GC anyway.
With this change is must not break until moving GC, because a user must
reference the object in Go to preserve liveness. Otherwise the code is broken already.
Reduces number of leaked params from 125 to 36 on linux.

R=golang-codereviews, mikioh.mikioh, bradfitz
CC=cshapiro, golang-codereviews, khr, rsc
https://golang.org/cl/45930043
2014-01-17 20:18:37 +04:00
Luke Curley
701982f173 crypto/cipher: improved cbc performance
decrypt: reduced the number of copy calls from 2n to 1.
encrypt: reduced the number of copy calls from n to 1.

Encryption is straight-forward: use dst instead of tmp when
xoring the block with the iv.

Decryption now loops backwards through the blocks abusing the
fact that the previous block's ciphertext (src) is the iv. This
means we don't need to copy the iv every time, in addition to
using dst instead of tmp like encryption.

R=golang-codereviews, agl, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/50900043
2014-01-17 11:07:04 -05:00
Adam Langley
a75875fc08 A+C: Luke Curley (individual CLA)
Generated by addca.

R=gobot
CC=golang-codereviews
https://golang.org/cl/53710043
2014-01-17 11:01:03 -05:00
Aram Hăvărneanu
a46b434931 runtime: add support for GOOS=solaris
R=alex.brainman, dave, jsing, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/35990043
2014-01-17 17:58:10 +13:00
Rob Pike
f8225bdb35 net/rpc: fix inconsistency in documentation of Service.Register
Falsely claimed an old, no longer true condition that the first argument
must be a pointer.
Fixes #6697

R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/53480043
2014-01-16 15:57:32 -08:00
Dave Cheney
8bc32785b9 net: skip TestDualStackTCPListener in short mode
Update #5001

This test is flakey on linux servers and fails otherwise good builds. Mikio has some proposals to fix the test, but they require additional plumbing.

In the meantime, disable this test in -short mode so it will run during the full net test suite, but not during builder ci.

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53410043
2014-01-17 09:49:38 +11:00
Rob Pike
f8cd243669 time: break parse and formatting tests into a separate source file
No changes, just rearrangement. The tests were in need of a little
housekeeping.

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53400043
2014-01-16 14:30:01 -08:00
Brad Fitzpatrick
055b588e55 syscall: add Flock_t on Linux
Matches Darwin and the BSDs. This means leveldb-go, kv,
Camlistore, etc can stop defining these structs on Linux by
hand.

Update #7059

R=golang-codereviews, dave, iant
CC=golang-codereviews
https://golang.org/cl/53350043
2014-01-16 14:08:32 -08:00
Keith Randall
873aaa59b7 reflect: Remove imprecise techniques from channel/select operations.
Reflect used to communicate to the runtime using interface words,
which is bad for precise GC because sometimes iwords hold a pointer
and sometimes they don't.  This change rewrites channel and select
operations to always pass pointers to the runtime.

reflect.Select gets somewhat more expensive, as we now do an allocation
per receive case instead of one allocation whose size is the max of
all the received types.  This seems unavoidable to get preciseness
(unless we move the allocation into selectgo, which is a much bigger
change).

Fixes #6490

R=golang-codereviews, dvyukov, rsc
CC=golang-codereviews
https://golang.org/cl/52900043
2014-01-16 13:35:29 -08:00
Brad Fitzpatrick
36477291cc net/http: don't allow Content-Type or body on 204 and 1xx
Status codes 204, 304, and 1xx don't allow bodies. We already
had a function for this, but we were hard-coding just 304
(StatusNotModified) in a few places.  Use the function
instead, and flesh out tests for all codes.

Fixes #6685

R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/53290044
2014-01-16 11:43:52 -08:00
Kamil Kisiel
18d644111e net/smtp: add examples
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/8274046
2014-01-16 10:49:58 -08:00
Brad Fitzpatrick
4deead7645 net/http: cache transport environment lookup
Apparently this is expensive on Windows.

Fixes #7020

R=golang-codereviews, alex.brainman, mattn.jp, dvyukov
CC=golang-codereviews
https://golang.org/cl/52840043
2014-01-16 10:25:45 -08:00
Rob Pike
fc908a0298 fmt: fix bug printing large zero-padded hexadecimal
We forgot to include the width of "0x" when computing the crossover
from internal buffer to allocated buffer.
Also add a helper function to the test for formatting large zero-padded
test strings.

Fixes #6777.

R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/50820043
2014-01-16 09:48:23 -08:00
Russ Cox
ca9975a45e cmd/gc: handle non-escaping address-taken variables better
This CL makes the bitmaps a little more precise about variables
that have their address taken but for which the address does not
escape to the heap, so that the variables are kept in the stack frame
rather than allocated on the heap.

The code before this CL handled these variables by treating every
return statement as using every such variable and depending on
liveness analysis to essentially treat the variable as live during the
entire function. That approach has false positives and (worse) false
negatives. That is, it's both sloppy and buggy:

        func f(b1, b2 bool) {	// x live here! (sloppy)
                if b2 {
                        print(0) // x live here! (sloppy)
                        return
                }
                var z **int
                x := new(int)
                *x = 42
                z = &x
                print(**z) // x live here (conservative)
                if b2 {
                        print(1) // x live here (conservative)
                        return
                }
                for {
                        print(**z) // x not live here (buggy)
                }
        }

The first two liveness annotations (marked sloppy) are clearly
wrong: x cannot be live if it has not yet been declared.

The last liveness annotation (marked buggy) is also wrong:
x is live here as *z, but because there is no return statement
reachable from this point in the code, the analysis treats x as dead.

This CL changes the liveness calculation to mark such variables
live exactly at points in the code reachable from the variable
declaration. This keeps the conservative decisions but fixes
the sloppy and buggy ones.

The CL also detects ambiguously live variables, those that are
being marked live but may not actually have been initialized,
such as in this example:

        func f(b1 bool) {
                var z **int
                if b1 {
                        x := new(int)
                        *x = 42
                        z = &x
                } else {
                        y := new(int)
                        *y = 54
                        z = &y
                }
                print(**z) // x, y live here (conservative)
        }

Since the print statement is reachable from the declaration of x,
x must conservatively be marked live. The same goes for y.
Although both x and y are marked live at the print statement,
clearly only one of them has been initialized. They are both
"ambiguously live".

These ambiguously live variables cause problems for garbage
collection: the collector cannot ignore them but also cannot
depend on them to be initialized to valid pointer values.

Ambiguously live variables do not come up too often in real code,
but recent changes to the way map and interface runtime functions
are invoked has created a large number of ambiguously live
compiler-generated temporary variables. The next CL will adjust
the analysis to understand these temporaries better, to make
ambiguously live variables fairly rare.

Once ambiguously live variables are rare enough, another CL will
introduce code at the beginning of a function to zero those
slots on the stack. At that point the garbage collector and the
stack copying routines will be able to depend on the guarantee that
if a slot is marked as live in a liveness bitmap, it is initialized.

R=khr
CC=golang-codereviews, iant
https://golang.org/cl/51810043
2014-01-16 10:32:30 -05:00
Russ Cox
fbfb9430dc cmd/gc: fix race build
Missed this case in CL 51010045.

TBR=khr
CC=golang-codereviews
https://golang.org/cl/53200043
2014-01-16 10:11:06 -05:00
Dmitriy Vyukov
c0b9e6218c runtime: output how long goroutines are blocked
Example of output:

goroutine 4 [sleep for 3 min]:
time.Sleep(0x34630b8a000)
        src/pkg/runtime/time.goc:31 +0x31
main.func·002()
        block.go:16 +0x2c
created by main.main
        block.go:17 +0x33

Full program and output are here:
http://play.golang.org/p/NEZdADI3Td

Fixes #6809.

R=golang-codereviews, khr, kamil.kisiel, bradfitz, rsc
CC=golang-codereviews
https://golang.org/cl/50420043
2014-01-16 12:54:46 +04:00
Dmitriy Vyukov
4722b1cbd3 runtime: use lock-free ring for work queues
Use lock-free fixed-size ring for work queues
instead of an unbounded mutex-protected array.
The ring has single producer and multiple consumers.
If the ring overflows, work is put onto global queue.

benchmark              old ns/op    new ns/op    delta
BenchmarkMatmult               7            5  -18.12%
BenchmarkMatmult-4             2            2  -18.98%
BenchmarkMatmult-16            1            0  -12.84%

BenchmarkCreateGoroutines                     105           88  -16.10%
BenchmarkCreateGoroutines-4                   376          219  -41.76%
BenchmarkCreateGoroutines-16                  241          174  -27.80%
BenchmarkCreateGoroutinesParallel             103           87  -14.66%
BenchmarkCreateGoroutinesParallel-4           169          143  -15.38%
BenchmarkCreateGoroutinesParallel-16          158          151   -4.43%

R=golang-codereviews, rsc
CC=ddetlefs, devon.odell, golang-codereviews
https://golang.org/cl/46170044
2014-01-16 12:17:00 +04:00
Keith Randall
2af7a26f1e reflect: add precise GC info for Call argument frame.
Give proper types to the argument/return areas
allocated for reflect calls.  Avoid use of iword to
manipulate receivers, which may or may not be pointers.

Update #6490

R=rsc
CC=golang-codereviews
https://golang.org/cl/52110044
2014-01-15 13:56:59 -08:00
Brad Fitzpatrick
35710eecd6 net/http: add disabled test for Body Read/Close lock granularity
Update #7121

R=golang-codereviews, gobot, dsymonds
CC=golang-codereviews
https://golang.org/cl/51750044
2014-01-15 13:12:32 -08:00
Robert Griesemer
fc80ce8194 go/scanner: report too short escape sequences
Generally improve error messages for escape sequences.

R=adonovan
CC=golang-codereviews
https://golang.org/cl/49430046
2014-01-15 09:50:55 -08:00
Rob Pike
fdbf3d901b cmd/pack: rewrite in Go
Replace the pack command, a C program, with a clean reimplementation in Go.
It does not need to reproduce the full feature set and it is no longer used by
the build chain, but has a role in looking inside archives created by the build
chain directly.

Since it's not in C, it is no longer build by dist, so remove it from cmd/dist and
make it a "tool" in cmd/go terminology.

Fixes #2705

R=rsc, dave, minux.ma, josharian
CC=golang-codereviews
https://golang.org/cl/52310044
2014-01-15 09:13:52 -08:00
Dmitriy Vyukov
b3a3afc9b7 runtime: fix data race in GC
Fixes #5139.
Update #7065.

R=golang-codereviews, bradfitz, minux.ma
CC=golang-codereviews
https://golang.org/cl/52090045
2014-01-15 19:38:08 +04:00
Brad Fitzpatrick
89c9d6b7f8 net/http: return UnexpectedEOF instead of EOF on truncated resposne
Fixes #6564

R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/52420043
2014-01-14 19:08:40 -08:00
Shenghou Ma
0db71338ed runtime/debug: force GC after setting of GCPercent to make it effective.
See also discussion in CL 51010045.

R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/52230043
2014-01-14 19:23:36 -05:00
Rob Pike
71377d3cd3 cmd/ld: document the -w flag, which disables DWARF generation
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/52360044
2014-01-14 15:34:27 -08:00
Rob Pike
227fe5f64e spec: tighten the wording around . imports
Make it clear that if you do a . import, you cannot use a qualified identifier.

R=gri
CC=golang-codereviews
https://golang.org/cl/52390043
2014-01-14 15:16:01 -08:00
Rob Pike
591265fcb4 reflect: better document the tri-state for TryRecv
R=rsc, iant
CC=golang-codereviews
https://golang.org/cl/52360043
2014-01-14 15:04:16 -08:00
Keith Randall
8454e2c287 runtime: Change size of map iter offset so 32-bit version compiles cleanly.
R=golang-codereviews, minux.ma
CC=golang-codereviews
https://golang.org/cl/52310043
2014-01-14 13:46:22 -08:00
Michael Kelly
26cc10289f net/http: escape contents of the directory indexes generated by FileServer
Previously, filenames containing special characters could:
      1) Escape the <a> tag, with a file called something like: ">foo
      2) Break the links in the index by prematurely ending the path portion
      of the url, with a file called: foo?bar

      In order to avoid a forbidden dependency on the html package, I'm
      using htmlReplacer from net/http/server.go, which is equivalent to
      html.EscapeString.

      This change also expands fakeFile.Readdir to better emulate
os.File.Readdir.

R=golang-codereviews, rsc, gobot, bradfitz, josharian, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/37440043
2014-01-14 12:55:12 -08:00
Josh Bleecher Snyder
3be4d95731 runtime: change map iteration randomization to use intra-bucket offset
Map iteration previously started from a random bucket, but walked each
bucket from the beginning. Now, iteration always starts from the first
bucket and walks each bucket starting at a random offset. For
performance, the random offset is selected at the start of iteration
and reused for each bucket.

Iteration over a map with 8 or fewer elements--a single bucket--will
now be non-deterministic. There will now be only 8 different possible
map iterations.

Significant benchmark changes, on my OS X laptop (rough but consistent):

benchmark                              old ns/op     new ns/op     delta
BenchmarkMapIter                       128           121           -5.47%
BenchmarkMapIterEmpty                  4.26          4.45          +4.46%
BenchmarkNewEmptyMap                   114           111           -2.63%

Fixes #6719.

R=khr, bradfitz
CC=golang-codereviews
https://golang.org/cl/47370043
2014-01-14 12:54:05 -08:00
Brad Fitzpatrick
1e2b13355f undo CL 47560044 / 40a37153a550
Still work to do. See http://golang.org/issue/7125

««« original CL description
net/http/cookiejar: document format of domain in PublicSuffix

Document what values a PublicSuffixList must accept as
a domain in a call to PublicSuffix.

R=bradfitz, nigeltao
CC=golang-codereviews
https://golang.org/cl/47560044

»»»

R=golang-codereviews, minux.ma
CC=golang-codereviews
https://golang.org/cl/51770044
2014-01-14 12:53:21 -08:00