This CL adds late expanded memequal(x, const, sz) inlining for 2, 4, 8
bytes size. This PoC is using the same method as CL 248404.
This optimization fires about 100 times in Go compiler (1675 occurrences
reduced to 1574, so -6%).
Also, added unit-tests to codegen/comparisions.go file.
Updates #37275
Change-Id: Ia52808d573cb706d1da8166c5746ede26f46c5da
Reviewed-on: https://go-review.googlesource.com/c/go/+/328291
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Trust: David Chase <drchase@google.com>
This adds a rule to avoid the zero extension after an AtomicLoad8
or AtomicLoad32 since the atomic load has already filled it with
zeros. This eliminates an instruction in a high use block in findObject
and the AtomicLoad8 appears many times within runtime.
Change-Id: I7e684bf73d3812110bd371e05b1aa44fa235fc9b
Reviewed-on: https://go-review.googlesource.com/c/go/+/354029
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
RFC 2616 permits multiple digits in the major and minor numbers of an
HTTP version:
https://datatracker.ietf.org/doc/html/rfc2616#section-3.1
RFC 7230 obsoletes 2616 and tightens the specification to permit only a
single digit in the major and minor number:
https://datatracker.ietf.org/doc/html/rfc7230#section-2.6
Use the stricter definition.
Also fix a bug which caused version numbers with a leading "+" to
be accepted (e.g., "HTTP/1.+1".)
Fixes#46587.
Change-Id: Ic5923bb858e5ac402cfde486fba2c075e221553d
Reviewed-on: https://go-review.googlesource.com/c/go/+/325874
Trust: Damien Neil <dneil@google.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change-Id: Ied57fb6e71d56618d46aeb36a37a709e08b4346e
Reviewed-on: https://go-review.googlesource.com/c/go/+/354136
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
The slowest thing that can happen in funcdata is a cache miss
on moduledata.gofunc. Move that memory load earlier.
Also, for better ergonomics when working on this code,
do more calculations as uintptrs.
name old time/op new time/op delta
StackCopyWithStkobj-8 10.5ms ± 5% 9.9ms ± 4% -6.03% (p=0.000 n=15+15)
Change-Id: I590f4449725983c7f8d274c4ac7ed384d9018d85
Reviewed-on: https://go-review.googlesource.com/c/go/+/354134
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
funcspdelta should be inlined: It is a tiny wrapper around another func.
The sanity check prevents that. Condition the sanity check on debugPcln.
While we're here, make the sanity check throw when it fails.
Change-Id: Iec022b8463b13a8e5a6d8479e7ddcb68909d6fe0
Reviewed-on: https://go-review.googlesource.com/c/go/+/354133
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This CL enables intrinsic support to emit the following prefetch
instructions for PPC64 platform that are already emitted on other
platforms
1. Prefetch - prefetches data from memory address to cache;
2. PrefetchStreamed - prefetches data from memory address, with a
hint that this data is being streamed.
Benchmarks picked from go/test/bench/garbage
Parameters tested with:
GOMAXPROCS=8
tree2 -heapsize=1000000000 -cpus=8
tree -n=18
parser
peano
Performance results with this change on POWER9
name old time/op new time/op delta
Tree2-8 75.3ms ± 2% 65.0ms ± 6% -13.61% (p=0.003 n=5+7)
Tree-8 576ms ± 2% 576ms ± 1% ~ (p=0.756 n=11+10)
Parser-8 3.60s ± 2% 3.59s ± 1% ~ (p=0.818 n=6+6)
Peano-8 84.8ms ± 1% 84.6ms ± 1% ~ (p=0.180 n=6+6)
Results on POWER8 and POWER10 are similar
Change-Id: If4ac95a85aaa7b2266014e1f8fb7cd7440cbf906
Reviewed-on: https://go-review.googlesource.com/c/go/+/353730
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
In the last CL I missed the fact that except for the final value the
code already unwraps the argument.
For #31103
Change-Id: Ic9099aeb50c6b3322fc14a90ac8026c1d8cb1698
Reviewed-on: https://go-review.googlesource.com/c/go/+/354091
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
In the case of a nointerface method on an instantiated type, we still
have to call methodWrapper, because methodWrapper generates the actual
generic method on the type as well. Currently, we don't call
methodWrapper, so the method on the instantiated type never gets filled
in.
Adjusted the code to still call methodWrapper, but not use the result,
in the case of a nointerface method on an instantiated type.
Change-Id: I34bca58de2861aa772be04eb8dd7695c5b7f3a77
Reviewed-on: https://go-review.googlesource.com/c/go/+/353369
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Dan Scales <danscales@google.com>
Add a simple test with an exported generic function that does
recover/defer, to test that recover/defer are exported/imported
properly (and a generic function with recover/defer works fine).
Change-Id: Idc3af101cbb78fc96bf945f1f5eab2740dd8994b
Reviewed-on: https://go-review.googlesource.com/c/go/+/353883
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Dan Scales <danscales@google.com>
This test hasn't passed since CL 349997, but the failure was not
detected because the Go project's builders do not have a 'fossil'
binary installed (#48802).
For #43684
Change-Id: I25544574ab48f4f146ae3795e541179e78815758
Reviewed-on: https://go-review.googlesource.com/c/go/+/354149
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
This reverts commit golang.org/cl/352953.
Reason for revert: unsafe.Slice is considerably slower.
Part of this is extra safety checks (good), but most of it
is the function call overhead. We should consider open-coding it (#48798).
Impact of this change:
name old time/op new time/op delta
StackCopyWithStkobj-8 12.1ms ± 5% 11.6ms ± 3% -4.03% (p=0.009 n=10+8)
Change-Id: Ib2448e3edac25afd8fb55ffbea073b8b11521bde
Reviewed-on: https://go-review.googlesource.com/c/go/+/354090
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This is a clean port of CL 354070 from types2 to go/types.
Change-Id: I44de1b8e6c0177e2a33e7f36a82465dc520c35aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/354092
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
For now, the constraint's underlying type set must be a single
type that is sliceable.
Change-Id: I08b6a2e88fe35e8238a95b3f40dc969689021a0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/354070
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In case of amd64 the compiler issues checks if extensions are
available on a platform. With GOAMD64 microarchitecture levels
provided, some of the checks could be eliminated.
Change-Id: If15c178bcae273b2ce7d3673415cb8849292e087
Reviewed-on: https://go-review.googlesource.com/c/go/+/352010
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
roundsd and FMA (vfmadd231sd).
Change-Id: I2d91332667e577bd9bb903ac58904f62b8454128
Reviewed-on: https://go-review.googlesource.com/c/go/+/354069
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This is a clean port of CL 353831 from go/types to types2.
For #47916.
Change-Id: I2c2b9c7bbcd416fb21f3032c55a06406bad9334a
Reviewed-on: https://go-review.googlesource.com/c/go/+/353934
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
When compiling with GOAMD64=v1, clobber all the >v1 instructions
with faulting instructions. Run the binary with the corresponding
feature flags off. We shouldn't try to execute any of the clobbered
instructions.
Change-Id: I295acaf9fd0eafd037192aa6f933365c794cc76e
Reviewed-on: https://go-review.googlesource.com/c/go/+/352831
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Keith Randall <khr@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This is like CL 304432 and CL 307229, for XCOFF.
With this, GOEXPERIMENT=regabi works on AIX/PPC64.
Change-Id: I8cf00681df5c93f397913febd78f38099d91e7c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/353972
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Every function has associated numbered extra funcdata to another symbol.
Prior to this change, a funcdata pointer was stored as a relocation.
This change alters this to be an offset relative to go.func.* or go.funcrel.*.
This reduces the number of relocations on darwin/arm64 by about 40%.
It also shrinks externally linked binaries. On darwin/arm64:
size before after Δ %
addr2line 3788498 3699730 -88768 -2.343%
api 5100018 4951074 -148944 -2.920%
asm 4855234 4744274 -110960 -2.285%
buildid 2500162 2419986 -80176 -3.207%
cgo 4338258 4218306 -119952 -2.765%
compile 22764418 22132226 -632192 -2.777%
cover 4583186 4432770 -150416 -3.282%
dist 3200962 3094626 -106336 -3.322%
doc 3680402 3583602 -96800 -2.630%
fix 3114914 3023922 -90992 -2.921%
link 6308578 6154786 -153792 -2.438%
nm 3754338 3665826 -88512 -2.358%
objdump 4124738 4015234 -109504 -2.655%
pack 2232626 2155010 -77616 -3.476%
pprof 13497474 13044066 -453408 -3.359%
test2json 2483810 2402146 -81664 -3.288%
trace 10108898 9748802 -360096 -3.562%
vet 6884322 6681314 -203008 -2.949%
total 107320836 104167700 -3153136 -2.938%
relocs before after Δ %
addr2line 33357 25563 -7794 -23.365%
api 31589 18409 -13180 -41.723%
asm 27825 18904 -8921 -32.061%
buildid 15603 9513 -6090 -39.031%
cgo 27809 17103 -10706 -38.498%
compile 114769 64829 -49940 -43.513%
cover 32932 19462 -13470 -40.902%
dist 18797 10796 -8001 -42.565%
doc 22891 13503 -9388 -41.012%
fix 19700 11465 -8235 -41.802%
link 37324 23198 -14126 -37.847%
nm 33226 25480 -7746 -23.313%
objdump 35237 26610 -8627 -24.483%
pack 13535 7951 -5584 -41.256%
pprof 97986 63961 -34025 -34.724%
test2json 15113 8735 -6378 -42.202%
trace 66786 39636 -27150 -40.652%
vet 43328 25971 -17357 -40.060%
total 687806 431088 -256718 -37.324%
It should also incrementally speed up binary launching
and may reduce linker memory use.
This is another step towards removing relocations so
that pages that were previously dirtied by the loader may remain clean,
which will offer memory savings useful in constrained environments like iOS.
Removing the relocations in .stkobj symbols will allow some simplifications.
There will be no references into go.funcrel.*,
so we will no longer need to use the bottom bit to distinguish offset bases.
Change-Id: I83d34c1701d6f3f515b9905941477d522441019d
Reviewed-on: https://go-review.googlesource.com/c/go/+/352110
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This change moves all symbols referred to by FUNCDATA
into go.func.* and go.funcrel.*.
Surprisingly (because it inhibits some content-addressability),
it shrinks binaries by a little bit, about 0.1%.
This paves the way for a subsequent change to change
FUNCDATA relocations to offsets.
Change-Id: I70e487205073699f442192b0791cc92da5663057
Reviewed-on: https://go-review.googlesource.com/c/go/+/352189
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
In testing the register ABI changes I found that the benchmarks
for strhash and memhash degraded unless I marked them as
ABIInternal. This fixes that.
Change-Id: I9c7a04eaa6a66b888877f43454c51277c07e638a
Reviewed-on: https://go-review.googlesource.com/c/go/+/353832
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
CL 269999 added support for getrandom on Dragonfly.
CL 299134 added support for getrandom on Solaris.
CL 302489 added support for getentropy on macOS.
Update the godoc for Reader accordingly.
Change-Id: Ice39e5e62f052f21b664db6abbfd97f03944586e
Reviewed-on: https://go-review.googlesource.com/c/go/+/353190
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
After CL 353871, darwin/arm64 now do concurrent build, so enable the
test for it.
Updates #48490
Change-Id: I29336f6fc7d7d2f463d8ad2a620534bd7f048d2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/353949
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
We don't need cmd/cgo for building go_bootstrap or toolchain2, so skip
building it as part of toolchain1.
This allows cmd/cgo to assume a current go/ast; e.g., that
ast.IndexListExpr exists (needed for next CL).
Change-Id: I642bba780bf273e6ea9c6e7c5d5d7ccfe86bf462
Reviewed-on: https://go-review.googlesource.com/c/go/+/353884
Trust: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime.gcbits symbols are pointer masks, which are just bytes.
Change-Id: I6e86359451c7da69da435e1928e55712dd904047
Reviewed-on: https://go-review.googlesource.com/c/go/+/353571
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Use sort.Search instead of open-coding the binary search.
This makes the code a lot easier to work on.
As a bonus, it speeds it up.
name old time/op new time/op delta
115/LineToPC-8 57.4µs ± 5% 59.2µs ± 8% +3.19% (p=0.003 n=15+13)
115/PCToLine-8 255ns ± 1% 192ns ± 3% -24.63% (p=0.000 n=15+15)
Change-Id: I41da18bfb0e745c40d24e5b96e50dfdd0c3b79f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/353879
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change-Id: I827a9702dfa01b712b88331668434f8db94df249
Reviewed-on: https://go-review.googlesource.com/c/go/+/353569
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
runtime.no_pointers_stackmap is an odd beast.
It is defined in a Go file, populated by assembly,
used by the GC, and its address is magic used
by async pre-emption to ascertain whether a
routine was implemented in assembly.
A subsequent change will force all GC data into the go.func.* linker symbol.
runtime.no_pointers_stackmap is GC data, so it must go there.
Yet it also needs to go into rodata, for the runtime address trick.
This change eliminates it entirely.
Replace the runtime address check with the newly introduced asm funcflag.
Handle the assembly macro as magic, similarly to our handling of go_args_stackmap.
This allows the no_pointers_stackmap to be identical in all ways
to other gclocals stackmaps, including content-addressability.
Change-Id: Id2f20a262cfab0719beb88e6342984ec4b196268
Reviewed-on: https://go-review.googlesource.com/c/go/+/353672
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
It'll be used in second place in a subsequent change.
No functional changes.
Change-Id: I58dd12d7dde45b36995d031fc7fbb27d6eaf48d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/353670
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Trust: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
There's no good way to ascertain at runtime whether
a function was implemented in assembly.
The existing workaround doesn't play nicely
with some upcoming linker changes.
This change introduces an explicit marker for routines
implemented in assembly.
This change doesn't use the new bit anywhere,
it only introduces it.
Change-Id: I4051dc0afc15b260724a04b9d18aeeb94911bb29
Reviewed-on: https://go-review.googlesource.com/c/go/+/353671
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Also move the logic of setting arginfo symbols content-addressable
to the place of symbol creation.
Change-Id: Ia5c3d77b1cec988c42c84d573170120948575c07
Reviewed-on: https://go-review.googlesource.com/c/go/+/353830
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
This change preserves the observable invariant that for an *ast.CallExpr
'call', Info.Types[call.Fun] is the signature being called.
Updates #47916
Change-Id: I3e97c712a7ee33a4f29e8cf4c18dc7c946b66cc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/353831
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Go exchanges siginfo and sigevent structures with the kernel. They
contain unions, but Go's use is limited to the first few fields. Pad out
the rest so the size Go sees is the same as what the Linux kernel sees.
This is a follow-up to CL 342052 which added the sigevent struct without
padding. It updates the siginfo struct as well so there are no bad
examples in the defs_linux_*.go files.
Change-Id: Id991d4a57826677dd7e6cc30ad113fa3b321cddf
Reviewed-on: https://go-review.googlesource.com/c/go/+/353136
Run-TryBot: Rhys Hiltner <rhys@justin.tv>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
This change updates the log lines to clarify that the printed
interesting count is only for newly discovered cache entries, and prints
the total cache size. It only prints information about interesting
entries when coverageEnabled is true.
Fixes#48669
Change-Id: I2045afc204764c1842d323e8ae42016fb21b6fb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/353172
Trust: Michael Knyszek <mknyszek@google.com>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
