Commit Graph

46441 Commits

Author SHA1 Message Date
Joel Sing
dab3e5affe runtime: switch runtime to libc for openbsd/amd64
Use libc rather than performing direct system calls for the runtime on
openbsd/amd64.

Updates #36435

Change-Id: Ib708009c3743f56a3fd6cb3bc731451e4a398849
Reviewed-on: https://go-review.googlesource.com/c/go/+/270379
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-01-22 18:58:08 +00:00
Michael Matloob
a1b53d85da cmd/go: add documentation for test and xtest fields output by go list
The TestEmbedPatterns, TestEmbedFiles, XTestEmbedPatterns, and
XTestEmbedFiles fields were left out of golang.org/cl/282195 which was
supposed to document the embed fields available in the go list
output. Add documentation for them in this CL.

Fixes #43081

Change-Id: Ifc256c476daec7c0f0e2c41f86b82f958b3e2b1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/284258
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-22 18:51:44 +00:00
Joel Sing
b268b60774 runtime: remove pthread_kill/pthread_self for openbsd
We're now using getthrid() and thrkill() instead.

Updates #36435

Change-Id: I1c6bcfb9b46d149e0a2a10e936a244576489a88e
Reviewed-on: https://go-review.googlesource.com/c/go/+/285692
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-01-22 18:23:05 +00:00
Ikko Ashimine
ec4051763d runtime: fix typo in mgcscavenge.go
recieved -> received

Change-Id: I84336170e179832604e1311ea9263af36f9ce15a
GitHub-Last-Rev: a6068c1d2b
GitHub-Pull-Request: golang/go#43845
Reviewed-on: https://go-review.googlesource.com/c/go/+/285675
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2021-01-22 16:43:04 +00:00
Damien Neil
7ece3a7b17 net/http: fix flaky TestDisableKeepAliveUpgrade
This test hijacks a connection. It was reading from the net.Conn
returned by Hijack, not the bufio.ReadWriter, causing flaky failures
when a read-ahead byte was held in the read buffer.

Fixes #43073.

Change-Id: Ic3e7f704fba9635fd851cb3c0c0c74e312b75f6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/285596
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Anmol Sethi <nhooyr@gmail.com>
2021-01-22 16:23:59 +00:00
Brad Fitzpatrick
50cba0506f time: clarify Timer.Reset behavior on AfterFunc Timers
Fixes #28100

Change-Id: I37d4d7badf455e4ecf982d4fc7cb070052de2e45
Reviewed-on: https://go-review.googlesource.com/c/go/+/285632
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-01-22 14:50:17 +00:00
Ian Lance Taylor
cf10e69f17 doc/go1.16: mention net/http.Transport.GetProxyConnectHeader
For #40700
For #41048

Change-Id: Ida6bcaaf5edaa2bba9ba2b8e02ec9959481f8302
Reviewed-on: https://go-review.googlesource.com/c/go/+/285594
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-01-22 06:35:43 +00:00
Ian Lance Taylor
ec1b945265 doc/go1.16: mention path/filepath.WalkDir
For #40700
For #42027

Change-Id: Ifb73050dfdab21784fa52d758ad9c408e6489684
Reviewed-on: https://go-review.googlesource.com/c/go/+/285595
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-01-22 06:35:01 +00:00
Ian Lance Taylor
11def3d40b doc/go1.16: mention syscall.AllThreadsSyscall
For #1435
For #40700

Change-Id: I01d277617ab511c90b9663fc89e418402e5ee2be
Reviewed-on: https://go-review.googlesource.com/c/go/+/285597
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-01-22 06:33:04 +00:00
Ian Lance Taylor
07b0235609 doc/go1.16: add notes about package-specific fs.FS changes
For #40700
For #41190

Change-Id: I964d6856d5cad62c859d0f3a7afdd349a8ad87cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/285093
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-01-21 23:50:47 +00:00
Austin Clements
e2b4f1fea5 doc/go1.16: minor formatting fix
Change-Id: Ie924a15da0d9a35089d1d4b3d6c4a07b93bdf270
Reviewed-on: https://go-review.googlesource.com/c/go/+/285393
Trust: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-01-21 20:47:45 +00:00
Ian Lance Taylor
9f43a9e07b doc/go1.16: mention new debug/elf constants
For #40700

Change-Id: If105d2f043539bb0893f577a984f14ee3e7ca753
Reviewed-on: https://go-review.googlesource.com/c/go/+/285212
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-01-21 20:32:52 +00:00
Roland Shoemaker
3c2f11ba5b cmd/go: overwrite program name with full path
If the program path is resolved, replace the first argument of the
exec.Cmd, which is the bare program name with the resolved path.

Change-Id: I92cf5e6f4bb7c8fef9b59f5eab963f4e75b90d07
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/957908
Reviewed-by: Katie Hockman <katiehockman@google.com>
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284784
Trust: Roland Shoemaker <roland@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
2021-01-21 19:15:21 +00:00
Roland Shoemaker
953d1feca9 all: introduce and use internal/execabs
Introduces a wrapper around os/exec, internal/execabs, for use in
all commands. This wrapper prevents exec.LookPath and exec.Command from
running executables in the current directory.

All imports of os/exec in non-test files in cmd/ are replaced with
imports of internal/execabs.

This issue was reported by RyotaK.

Fixes CVE-2021-3115
Fixes #43783

Change-Id: I0423451a6e27ec1e1d6f3fe929ab1ef69145c08f
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/955304
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Katie Hockman <katiehockman@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284783
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
2021-01-21 19:10:18 +00:00
Russ Cox
b186e4d70d cmd/go: add test case for cgo CC setting
Change-Id: Ied986053a64447c5eac6369f6c9b69ed3d3f94d9
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/949415
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284782
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
2021-01-21 19:09:28 +00:00
Russ Cox
5a8a2265fb cmd/cgo: report exec errors a bit more clearly
Change-Id: I0e6bebf0e2e6efdef4be880e0c6c7451b938924b
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/949417
Reviewed-by: Katie Hockman <katiehockman@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284781
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
2021-01-21 19:08:52 +00:00
Russ Cox
46e2e2e9d9 cmd/go: pass resolved CC, GCCGO to cgo
This makes sure the go command and cgo agree about
exactly which compiler is being used.

This issue was reported by RyotaK.

Fixes CVE-2021-3115
Fixes #43783

Change-Id: If171c5c8b2523efb5ea2d957e5ad1380a038149c
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/949416
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284780
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
2021-01-21 19:08:04 +00:00
Joel Sing
3d40895e36 runtime: switch openbsd/arm64 to pthreads
This switches openbsd/arm64 to thread creation via pthreads, rather than doing
direct system calls.

Update #36435

Change-Id: I7cf60fa954f92628e05f15d2732833a2fbdccdb9
Reviewed-on: https://go-review.googlesource.com/c/go/+/250182
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-01-21 18:54:12 +00:00
Filippo Valsorda
d95ca91380 crypto/elliptic: fix P-224 field reduction
This patch fixes two independent bugs in p224Contract, the function that
performs the final complete reduction in the P-224 field. Incorrect
outputs due to these bugs were observable from a high-level
P224().ScalarMult() call.

The first bug was in the calculation of out3GT. That mask was supposed
to be all ones if the third limb of the value is greater than the third
limb of P (out[3] > 0xffff000). Instead, it was also set if they are
equal. That meant that if the third limb was equal, the value was always
considered greater than or equal to P, even when the three bottom limbs
were all zero. There is exactly one affected value, P - 1, which would
trigger the subtraction by P even if it's lower than P already.

The second bug was more easily hit, and is the one that caused the known
high-level incorrect output: after the conditional subtraction by P, a
potential underflow of the lowest limb was not handled. Any values that
trigger the subtraction by P (values between P and 2^224-1, and P - 1
due to the bug above) but have a zero lowest limb would produce invalid
outputs. Those conditions apply to the intermediate representation
before the subtraction, so they are hard to trace to precise inputs.

This patch also adds a test suite for the P-224 field arithmetic,
including a custom fuzzer that automatically explores potential edge
cases by combining limb values that have various meanings in the code.
contractMatchesBigInt in TestP224Contract finds the second bug in less
than a second without being tailored to it, and could eventually find
the first one too by combining 0, (1 << 28) - 1, and the difference of
(1 << 28) and (1 << 12).

The incorrect P224().ScalarMult() output was found by the
elliptic-curve-differential-fuzzer project running on OSS-Fuzz and
reported by Philippe Antoine (Catena cyber).

Fixes CVE-2021-3114
Fixes #43786

Change-Id: I50176602d544de3da854270d66a293bcaca57ad7
Reviewed-on: https://go-review.googlesource.com/c/go/+/284779
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-01-21 18:52:19 +00:00
Baokun Lee
d7e71c01ad [dev.regabi] cmd/compile: replace ir.Name map with ir.NameSet for dwarf
Same as CL 284897, but for dwarf.

Passes toolstash -cmp.

Fixes #43819

Change-Id: Icbe43aa2e3cb96e6a6c318523c643247da8e4c74
Reviewed-on: https://go-review.googlesource.com/c/go/+/284899
Run-TryBot: Baokun Lee <bk@golangcn.org>
Trust: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-01-21 09:56:27 +00:00
Baokun Lee
5248f59a22 [dev.regabi] cmd/compile: replace ir.Name map with ir.NameSet for SSA
Same as CL 284897, but for SSA.

Passes toolstash -cmp.

Updates #43819

Change-Id: I3c500ad635a3192d95d16fdc36f154ba3ea5df69
Reviewed-on: https://go-review.googlesource.com/c/go/+/284898
Run-TryBot: Baokun Lee <bk@golangcn.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Baokun Lee <bk@golangcn.org>
2021-01-21 09:56:12 +00:00
Baokun Lee
970d8b6cb2 [dev.regabi] cmd/compile: replace ir.Name map with ir.NameSet in inlining
As CL 282212 mentioned, we should clean all map[*ir.Name]bool with
ir.NameSet.

Passes toolstash -cmp.

Updates #43819

Change-Id: I1ce5d2055f88539f807dc021cd8e3941b425bc4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/284897
Run-TryBot: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Baokun Lee <bk@golangcn.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-01-21 09:55:49 +00:00
Cuong Manh Le
68a4664475 [dev.regabi] cmd/compile: remove tempAssigns in walkCall1
Passes toolstash -cmp.

Change-Id: I588c663324443e02b901cda461b999ff192e150c
Reviewed-on: https://go-review.googlesource.com/c/go/+/284896
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-01-21 06:40:06 +00:00
Cuong Manh Le
fd9a391cdd [dev.regabi] cmd/compile: remove CallExpr.Rargs
Instead, push the temps assignments to init. This does not pass
toolstash, since when before this, the temps were evaluated after
function callee, now we evaluate them before.

Change-Id: Icb9cb10e036925b56c1ef3eec468416a11f4932f
Reviewed-on: https://go-review.googlesource.com/c/go/+/284894
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-01-21 06:38:25 +00:00
Cuong Manh Le
19a6db6b63 [dev.regabi] cmd/compile: make sure mkcall* passed non-nil init
So next CL can pass temporaries assignments for function arguments in to
init instead of CallExpr.Rargs.

Passes toolstash -cmp.

Change-Id: I2c3cb6a63e8bf9d0418052b39c1db58050f71305
Reviewed-on: https://go-review.googlesource.com/c/go/+/284893
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-01-21 06:38:18 +00:00
Baokun Lee
9f036844db [dev.regabi] cmd/compile: use ir.DoChildren directly in inlining
Passes toolstash -cmp.

Change-Id: Ie35e8163fa0e61ed9e1b259929c8cbe82ee5301e
Reviewed-on: https://go-review.googlesource.com/c/go/+/282212
Run-TryBot: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Baokun Lee <bk@golangcn.org>
2021-01-21 05:44:57 +00:00
Cuong Manh Le
213c3905e9 [dev.regabi] cmd/compile: use node walked flag to prevent double walk for walkSelect
Same as CL 283733, but for walkSelect.

Passes toolstash -cmp.

Change-Id: I3ecb8d6eafd395379191c15fc58c95f75809fec9
Reviewed-on: https://go-review.googlesource.com/c/go/+/284895
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-01-21 05:22:22 +00:00
Dan Scales
1760d736f6 [dev.regabi] cmd/compile: exporting, importing, and inlining functions with OCLOSURE
I have exporting, importing, and inlining of functions with closures
working in all cases (issue #28727). all.bash runs successfully without
errors.

Approach:
  - Write out the Func type, Dcls, ClosureVars, and Body when exporting
    an OCLOSURE.

  - When importing an OCLOSURE, read in the type, dcls, closure vars,
    and body, and then do roughly equivalent code to (*noder).funcLit

  - During inlining of a closure within inlined function, create new
    nodes for all params and local variables (including closure
    variables), so they can have a new Curfn and some other field
    values. Must substitute not only on the Nbody of the closure, but
    also the Type, Cvars, and Dcl fields.

Fixes #28727

Change-Id: I4da1e2567c3fa31a5121afbe82dc4e5ee32b3170
Reviewed-on: https://go-review.googlesource.com/c/go/+/283112
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Dan Scales <danscales@google.com>
2021-01-20 22:53:32 +00:00
Dmitri Shuralyov
ecf4ebf100 cmd/internal/moddeps: check content of all modules in GOROOT
Expand the scope of the TestAllDependenciesVendored test to check
that all modules in GOROOT are tidy, that packages are vendored,
the vendor content matches the upstream copy exactly, and that
bundled packages are re-generated (using x/tools/cmd/bundle at
the version selected in cmd module; this is deterministic and
guaranteed to be updated over time).

This is done in a conceptually simple way:

1.	Make a temporary copy of the entire GOROOT tree (except .git),
	one that is safe to modify.
2.	Run a list of high-level commands, the same commands we expect
	Go developers should be able to run in a normal complete GOROOT
	tree to make it clean and tidy.
3.	Diff the end result with the original GOROOT tree being tested
	to catch any unexpected differences.

The current set of commands that are run require the cmd/go command,
and a functional compiler itself (because re-generating the syscall
package involves a directive like //go:generate go run [...]). As a
result, copying a large majority of the GOROOT tree is a requirement.

Instead of looking for the few files or directories that can we can
get away not copying (e.g., the testdata directories aren't strictly
needed at this time), we opt not to optimize and just do the simple
copy. This is motivated by these reasons:

•	We end up having a complete, normal GOROOT tree, one that happens
	to be located at another path. There's a very high likelihood that
	module management/code generation commands, both the ones we run
	today and any additional ones that we might want to add in the
	future, will result in correct results even as the Go project
	evolves over time.

•	Having a completely stand-alone copy of the GOROOT tree without
	symlinks minimizes the risk of some of the module management/code
	generation commands, either now or in the future, from modifying
	the user's original GOROOT tree, something that should not happen
	during test execution. Overlays achieved with symlinks work well
	when we can guarantee only new files are added, but that isn't
	the case here.

•	Copying the entire GOROOT (without .git), takes around 5 seconds
	on a fairly modern computer with an SSD. The most we can save is
	a couple of seconds.

(We make some minor exceptions: the GOROOT/.git directory isn't copied,
and GOROOT/{bin,pkg} are deemed safe to share and thus symlink instead
of copying. If these optimizations cease to be viable to make, we'll
need to remove them.)

Since this functionality is fairly expensive to execute and requires
network access, it runs only when the test is executed without -short
flag. The previous behavior of the TestAllDependenciesVendored test is
kept in -short test mode. all.bash runs package tests with -short flag,
so its behavior is unchanged. The expectation is that the new test will
run on some of the longtest builders to catch problems. Users can invoke
the test manually 'go test cmd/internal/moddeps' (and it's run as part
of 'go test cmd', again, only when -short flag isn't provided).

On a 2017 MacBook Pro, a successful long test takes under 15 seconds,
which should be within scope of all long tests that are selected by
'go test std cmd'. We may further adjust when and where the test runs
by default based on our experience.

Fixes #36852.
Fixes #41409.
Fixes #43687.
Updates #43440.

Change-Id: I9eb85205fec7ec62e3f867831a0a82e3c767f618
Reviewed-on: https://go-review.googlesource.com/c/go/+/283643
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-01-20 22:16:54 +00:00
David Chase
92cb157cf3 [dev.regabi] cmd/compile: late expansion of return values
By-hand rebase of earlier CL, because that was easier than
letting git try to figure things out.

This will naively insert self-moves; in the case that these
involve memory, the expander detects these and removes them
and their vardefs.

Change-Id: Icf72575eb7ae4a186b0de462bc8cf0bedc84d3e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/279519
Trust: David Chase <drchase@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2021-01-20 19:57:48 +00:00
Ian Lance Taylor
d2d155d1ae runtime: don't adjust timer pp field in timerWaiting status
Before this CL, the following sequence was possible:

* GC scavenger starts and sets up scavenge.timer
* GC calls readyForScavenger, but sysmon is sleeping
* program calls runtime.GOMAXPROCS to shrink number of processors
* procresize destroys a P, the one that scavenge.timer is on
* (*pp).destroy calls moveTimers, which gets to the scavenger timer
* scavenger timer is timerWaiting, and moveTimers clears t.pp
* sysmon wakes up and calls wakeScavenger
* wakeScavengers calls stopTimer on scavenger.timer, still timerWaiting
* stopTimer calls deltimer which loads t.pp, which is still nil
* stopTimer tries to increment deletedTimers on nil t.pp, and crashes

The point of vulnerability is the time that t.pp is set to nil by
moveTimers and the time that t.pp is set to non-nil by moveTimers,
which is a few instructions at most. So it's not likely and in
particular is quite unlikely on x86. But with a more relaxed memory
model the area of vulnerability can be somewhat larger. This appears
to tbe the cause of two builder failures in a few months on linux-mips.

This CL fixes the problem by making moveTimers change the status from
timerWaiting to timerMoving while t.pp is clear. That will cause
deltimer to wait until the status is back to timerWaiting, at which
point t.pp has been set again.

Fixes #43712

Change-Id: I66838319ecfbf15be66c1fac88d9bd40e2295852
Reviewed-on: https://go-review.googlesource.com/c/go/+/284775
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-01-20 19:55:44 +00:00
Michael Matloob
803d18fc6c cmd/go: set Incomplete field on go list output if no files match embed
If no files match the embed pattern, the Error field will be set on
the package output by go list. Also set the Incomplete field for
consistency.

Fixes #43727

Change-Id: I5b4bb2a03a751269641a9bc4ef1d0fa0e37d46aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/284257
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-20 17:38:12 +00:00
Michael Matloob
6e243ce71d cmd/go: have go mod vendor copy embedded files in subdirs
If a package vendored with go mod vendor depends on embedded
files contained in subdirectories, copy them into the the
corresponding place in the module's vendor tree. (Embeds in
parent directories are disallowed by the embed pattern rules, and
embeds in the same directory are copied because go mod vendor
already copies the non-go files in the package's own directory).

Export the vendor pattern expansion code in internal/load so
internal/modcmd's vendor code can use it.

Fixes #43077

Change-Id: I61edb344d73df590574a6498ffb6069e8d72a147
Reviewed-on: https://go-review.googlesource.com/c/go/+/283641
Trust: Michael Matloob <matloob@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-01-20 17:37:54 +00:00
Jay Conrod
be28e5abc5 cmd/go: fix mod_get_fallback test
Fixes #43795

Change-Id: I3d791d0ac9ce0b523c78c649aaf5e339a7f63b76
Reviewed-on: https://go-review.googlesource.com/c/go/+/284797
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-01-20 14:59:52 +00:00
Joel Sing
928bda4f4a runtime: convert openbsd/amd64 locking to libc
Switch openbsd/amd64 to locking via libc, rather than performing direct
system calls.

Update #36435

Change-Id: I5e92bd70ce557b78ff385577088a9775cc468ea9
Reviewed-on: https://go-review.googlesource.com/c/go/+/270378
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-01-20 09:06:08 +00:00
Constantin Konstantinidis
824f2d635c cmd/go: allow go fmt to complete when embedded file is missing
Fixes #43273

Change-Id: I75fe2e608cb43c048e3c2a22fe7fbb6eb779504a
Reviewed-on: https://go-review.googlesource.com/c/go/+/280452
Trust: Jay Conrod <jayconrod@google.com>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-01-19 20:58:35 +00:00
Russ Cox
0575e35e50 cmd/compile: require 'go 1.16' go.mod line for //go:embed
This will produce better errors when earlier versions of
Go compile code using //go:embed. (The import will cause
a compilation error but then the go command will add to
the output that the Go toolchain in use looks too old
and maybe that's the problem.)

This CL also adds a test for disallowing embed of a var inside a func.
It's a bit too difficult to rebase down into that CL.

The build system configuration check is delayed in order to
make it possible to use errorcheck for these tests.

Change-Id: I12ece4ff2d8d53380b63f54866e8f3497657d54c
Reviewed-on: https://go-review.googlesource.com/c/go/+/282718
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-19 20:07:52 +00:00
Cuong Manh Le
9423d50d53 [dev.regabi] cmd/compile: use '%q' for printing rune values less than 128
Fixes #43762

Change-Id: I51734c9b4ee2366a5dae53b2d27b363f4d5fe6c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/284592
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-01-19 18:27:50 +00:00
Cherry Zhang
ccb2e90688 cmd/link: exit before Asmb2 if error
If there are already errors emitted, don't run the Asmb2 pass
and just exit. At the point of Asmb2 relocations are already
resolved and errors should have been reported, if any. Asmb2 is
unlikely to emit additional useful users errors. Instead, the
invalid input may cause inconsistencies and crash the linker, or
it may emit some internal errors which are more confusing than
helpful. Exit on error before Asmb2.

Fixes #43748.

Change-Id: Icf6e27f2eef5b6259e921ec0e64bebad5dd805f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/284576
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-01-19 17:33:33 +00:00
Jay Conrod
ca5774a5a5 embed: treat uninitialized FS as empty
As described in the FS documentation.

This prevents http.FS and other clients from panicking when the
go:embed directive is missing.

For #43682
Related #43698

Change-Id: Iecf26d229a099e55d24670c3119cd6c6d17ecc6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/283852
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-19 16:31:09 +00:00
Joel Sing
d047c91a6c cmd/link,runtime: switch openbsd/amd64 to pthreads
This switches openbsd/amd64 to thread creation via pthreads, rather than doing
direct system calls.

Update #36435

Change-Id: I1105d5c392aa3e4c445d99c8cb80b927712e3529
Reviewed-on: https://go-review.googlesource.com/c/go/+/250180
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-01-19 12:49:13 +00:00
Joel Sing
61debffd97 runtime: factor out usesLibcall
Rather than inline lists of GOOS values, factor out the code that checks
if a runtime makes system calls via libcall.

Change-Id: Ib19d7e63a2b4b8314f1841c0ff26e1b3a16b4b22
Reviewed-on: https://go-review.googlesource.com/c/go/+/259239
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
2021-01-19 12:48:38 +00:00
Joel Sing
9fed39d281 runtime: factor out mStackIsSystemAllocated
Rather than repeat long lists of GOOS values, factor out the code that checks
if a runtime starts on a system allocated stack. Note that this adds aix to
one case, which appears to have been previously missed.

Change-Id: I5cecb0bb47dd79cde8d723e5a42ba541e43cbfff
Reviewed-on: https://go-review.googlesource.com/c/go/+/250179
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
2021-01-19 12:07:43 +00:00
Matthew Dempsky
a2f825c542 [dev.regabi] cmd/compile: directly create go.map and go.track symbols
These symbols are implementation details and don't correspond to Go
source symbols, so directly create them as linker symbols and get rid
of their pseudo packages.

Passes toolstash -cmp w/ -gcflags=all=-abiwrap.

Change-Id: I2e97374c21f3e909f6d350f15e7a5ed3574cadf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/284372
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Matthew Dempsky <mdempsky@google.com>
2021-01-19 03:02:52 +00:00
Matthew Dempsky
4a4212c0e5 [dev.regabi] cmd/compile: refactor Linksym creation
Currently there's a lot of logic within package types for creating
Linksyms. This CL pulls it out into base, where it can be more easily
reused by other compiler code that shouldn't need to depend on package
types.

Package base probably isn't the best place for this, but it's
convenient because it's a package that types already depends on. It's
also where the Ctxt object lives, which these functions depend upon.

Passes toolstash -cmp w/ -gcflags=all=-abiwrap.

Change-Id: I50d8b7e4596955205036969eab24d7dab053b363
Reviewed-on: https://go-review.googlesource.com/c/go/+/284231
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
2021-01-19 02:41:42 +00:00
Matthew Dempsky
4f5c603c0f [dev.regabi] cmd/compile: cleanup callTargetLSym
Now that TailCallStmt carries an *ir.Name instead of a *types.Sym,
callTargetLSym can be similarly updated to take the target function as
an *ir.Name.

This inches us closer towards being able to move Linksym and other
properties from *types.Sym to *ir.Name, where they belong.

Passes toolstash -cmp w/ -gcflags=all=-abiwrap.

Change-Id: I091da290751970eba8ed0438f66d6cca88b665a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/284228
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-01-19 02:41:30 +00:00
Jason A. Donenfeld
dbab079835 runtime: free Windows event handles after last lock is dropped
Calls to lock may need to use global members of mOS that also need to be
cleaned up before the thread exits. Before this commit, these resources
would leak. Moving them to be cleaned up in unminit, however, would race
with gstack on unix. So this creates a new helper, mdestroy, to release
resources that must be destroyed only after locks are no longer
required. We also move highResTimer lifetime to the same semantics,
since it doesn't help to constantly acquire and release the timer object
during dropm.

Updates #43720.

Change-Id: Ib3f598f3fda1b2bbcb608099616fa4f85bc1c289
Reviewed-on: https://go-review.googlesource.com/c/go/+/284137
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2021-01-18 18:16:12 +00:00
Jason A. Donenfeld
5a8fbb0d2d os: do not close syscall.Stdin in TestReadStdin
By calling NewConsoleFile on syscall.Stdin, we wind up closing it when
the function returns, which causes errors when all the tests are run in
a loop. To fix this, we instead create a duplicate handle of stdin.

Fixes #43720.

Change-Id: Ie6426e6306c7e1e39601794f4ff48bbf2fe67502
Reviewed-on: https://go-review.googlesource.com/c/go/+/284140
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2021-01-18 17:21:53 +00:00
Matthew Dempsky
422f38fb6c [dev.regabi] cmd/compile: move stack objects to liveness
Calculating and emitting stack objects are essentially part of
liveness analysis, so move the code from ssagen to liveness. Allows
unexporting liveness.ShouldTrack.

Passes toolstash -cmp.

Change-Id: I88b5b2e75b8dfb46b8b03a2fa09a9236865cbf3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/284413
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-01-18 06:33:56 +00:00
Matthew Dempsky
6113db0bb4 [dev.regabi] cmd/compile: convert OPANIC argument to interface{} during typecheck
Currently, typecheck leaves arguments to OPANIC as their original
type. This CL changes it to insert implicit OCONVIFACE operations to
convert arguments to `interface{}` like how any other function call
would be handled.

No immediate benefits, other than getting to remove a tiny bit of
special-case logic in order.go's handling of OPANICs. Instead, the
generic code path for handling OCONVIFACE is used, if necessary.
Longer term, this should be marginally helpful for #43753, as it
reduces the number of cases where we need values to be addressable for
runtime calls.

However, this does require adding some hacks to appease existing
tests:

1. We need yet another kludge in inline budgeting, to ensure that
reflect.flag.mustBe stays inlinable for cmd/compile/internal/test's
TestIntendedInlining.

2. Since the OCONVIFACE expressions are now being introduced during
typecheck, they're now visible to escape analysis. So expressions like
"panic(1)" are now seen as "panic(interface{}(1))", and escape
analysis warns that the "interface{}(1)" escapes to the heap. These
have always escaped to heap, just now we're accurately reporting about
it.

(Also, unfortunately fmt.go hides implicit conversions by default in
diagnostics messages, so instead of reporting "interface{}(1) escapes
to heap", it actually reports "1 escapes to heap", which is
confusing. However, this confusing messaging also isn't new.)

Change-Id: Icedf60e1d2e464e219441b8d1233a313770272af
Reviewed-on: https://go-review.googlesource.com/c/go/+/284412
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Matthew Dempsky <mdempsky@google.com>
2021-01-18 05:55:08 +00:00