Commit Graph

833 Commits

Author SHA1 Message Date
Ilija Tovilo
379e913e1a
Fix run-tests.php differ calculateCommonSubsequence for EXPECTF
calculateCommonSubsequence should not contain regexes.

Fixes GH-13083
Closes GH-13084
2024-01-08 13:36:29 +01:00
Ilija Tovilo
ed927365c2
Merge branch 'PHP-8.2' into PHP-8.3
* PHP-8.2:
  Retry tests on deadlock
2023-11-16 16:02:58 +01:00
Ilija Tovilo
58712e0698
Retry tests on deadlock
Closes GH-12693
2023-11-16 16:02:20 +01:00
Ilija Tovilo
082219ba0e
Merge branch 'PHP-8.2' into PHP-8.3
* PHP-8.2:
  Automatically mark tests as flaky
2023-11-15 13:22:35 +01:00
Ilija Tovilo
6d59640980
Automatically mark tests as flaky
Marking all of these tests as flaky is annoying, so attempt to recognize them
automatically.

Closes GH-12638
2023-11-15 13:21:58 +01:00
Ilija Tovilo
8f4738f41d
Merge branch 'PHP-8.2' into PHP-8.3
* PHP-8.2:
  Increase run-tests.php timeout for asan
2023-08-30 23:10:24 +02:00
Ilija Tovilo
936538d8d7
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  Increase run-tests.php timeout for asan
2023-08-30 23:10:10 +02:00
Ilija Tovilo
f4a6a6d096
Increase run-tests.php timeout for asan
Closes GH-12087
2023-08-30 23:09:33 +02:00
Ilija Tovilo
b20ff2f9f5
Merge branch 'PHP-8.2'
* PHP-8.2:
  Make unrepeatable tests retriable
2023-08-29 13:59:26 +02:00
Ilija Tovilo
d5f7ffb914
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  Make unrepeatable tests retriable
2023-08-29 13:58:53 +02:00
Ilija Tovilo
f2c16b7ba3
Make unrepeatable tests retriable
This was an error in thinking on my side. The reason these tests are
unrepeatable is because --repeat executes the same request in the same process,
and does not run the CLEAN section in between runs. This is not the case when
retrying tests.

We could potentially make CLEAN tests repeatable by including the CLEAN section
in the tested script. This does however not work for all tests (e.g. tests that
set open_basedir).

Closes GH-12072
2023-08-29 13:56:13 +02:00
Peter Kokot
e199104a35
Fix passing null to parameter of type string (#12014)
This fixes builds without cgi or phpdbg:
  ./configure --disable-cgi --disable-phpdbg
  make
  ./sapi/cli/php run-tests.php

Otherwise, deprecation warnings (since PHP-8.1) are emitted:
  Deprecated: escapeshellarg(): Passing null to parameter #1 ($arg) of
  type string is deprecated in run-tests.php...
2023-08-22 02:51:31 +01:00
Ilija Tovilo
d397fec129
Skip xleak tests on asan (#11610)
Fixes GH-11077
2023-07-07 09:24:05 +02:00
Ilija Tovilo
b0a2727749
Merge branch 'PHP-8.2'
* PHP-8.2:
  Implement flaky test section
2023-07-06 09:46:15 +02:00
Ilija Tovilo
4e3d9b26f9
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  Implement flaky test section
2023-07-06 09:46:03 +02:00
Ilija Tovilo
849fdcae7d
Implement flaky test section
This re-uses the already used for automatic retesting. That's certainly better
than XFAIL.

Closes GH-11325
2023-07-06 09:45:35 +02:00
Ilija Tovilo
2d883a6ff0
Merge branch 'PHP-8.2'
* PHP-8.2:
  [skip ci] Add more patterns to run-tests.php retry list
2023-05-27 19:22:07 +02:00
Ilija Tovilo
f735543bc9
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  [skip ci] Add more patterns to run-tests.php retry list
2023-05-27 19:21:55 +02:00
Ilija Tovilo
f249958cd3
[skip ci] Add more patterns to run-tests.php retry list
CURL: 404: Page Not Found
IMAP: Can't create a temporary mailbox: [ALREADYEXISTS] Mailbox already exists
Sockets: socket_bind(): Unable to bind address [98]: Address already in use
2023-05-27 19:21:19 +02:00
Ilija Tovilo
4e6025a641
Merge branch 'PHP-8.2'
* PHP-8.2:
  Add retry mechanism in run-tests.php
2023-05-03 13:51:31 +02:00
Ilija Tovilo
81b2d2ee45
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  Add retry mechanism in run-tests.php
2023-05-03 13:50:04 +02:00
Ilija Tovilo
11597d18d6
Add retry mechanism in run-tests.php
We have lots of spurious failures in CI, many of them with the "all" CONFLICT.
We're limiting the retrying to specific error messages. In the future we may
also provide a FLAKY section to retry specific tests.

Closes GH-10892
2023-05-03 13:47:13 +02:00
Ilija Tovilo
fc88f155be
Add zend_alloc XLEAK support
In the future we may want to use a different exit code to warn for tests that
didn't leak.

Closes GH-10999
2023-04-03 12:55:26 +02:00
Ilija Tovilo
50f58c8923
Add ASAN XLEAK support
Only disable LSAN instead of skipping the test. This way we can still detect
memory issues which is arguably more important anyway.

Closes GH-10996
2023-04-03 08:02:19 +02:00
Ilija Tovilo
ed0b773cb9
[skip ci] Avoid infinite loop in differ
TBH I don't understand why this can happen here but not in sebastian/diff. I'll
have to take a closer look.
2023-04-01 15:44:46 +02:00
Ilija Tovilo
b9f8b696c4
Fix one more differ direct comparison (through in_array) 2023-03-27 17:39:52 +02:00
Ilija Tovilo
c58c2666a1
Fix direct comparison in run-tests.php differ 2023-03-23 16:07:59 +01:00
Ilija Tovilo
d854492655
[skip ci] Fix var_dump statement in run-tests.php 2023-03-23 11:23:17 +01:00
Ilija Tovilo
ac961bbb1d
Implement better diff for run-tests.php
Borrow sebastianbergmann/diff with
MemoryEfficientLongestCommonSubsequenceCalculator

Fixes GH-10806
Closes GH-10875
2023-03-22 20:02:41 +01:00
Ilija Tovilo
5ef46020b6
Merge branch 'PHP-8.2'
* PHP-8.2:
  Another attempt to fix MSAN nightly on master
2023-03-15 23:38:22 +01:00
Ilija Tovilo
6343975f34
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  Another attempt to fix MSAN nightly on master
2023-03-15 23:38:09 +01:00
Ilija Tovilo
471105abd7
Another attempt to fix MSAN nightly on master 2023-03-15 23:37:36 +01:00
Ilija Tovilo
a5da519d59
Merge branch 'PHP-8.2'
* PHP-8.2:
  use_tls=0 on MSAN
2023-03-14 17:43:12 +01:00
Ilija Tovilo
ca7c7e9c05
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  use_tls=0 on MSAN
2023-03-14 17:43:02 +01:00
Ilija Tovilo
63525ee600
use_tls=0 on MSAN
Attempt to fix MSAN failure in CI

Closes GH-10851
2023-03-14 17:42:23 +01:00
Ilija Tovilo
c0ee8e833c
Merge branch 'PHP-8.2'
* PHP-8.2:
  [skip ci] Fix diff cmd env name in run-tests.php
2023-03-10 01:10:55 +01:00
Ilija Tovilo
d3c9585204
[skip ci] Fix diff cmd env name in run-tests.php 2023-03-10 01:10:25 +01:00
Ilija Tovilo
486fb80947
[skip ci] Remove accidental run-tests.php line 2023-03-06 15:05:11 +01:00
Ilija Tovilo
9944f58d3f
Merge branch 'PHP-8.2'
* PHP-8.2:
  Fix GH-10709: UAF in recursive AST evaluation
2023-03-06 15:03:19 +01:00
Danack
3e6d49e042
Fix warning in run-tests when PHP compiled without generating phpdbg support. (#10745) 2023-03-03 01:50:02 +00:00
Niels Dossche
dcc3255b18
Fix GH-10489: run-tests.php does not escape path when building cmd (#10560)
Multiple tests had to be changed to escape the arguments in shell
commands. Some tests are skipped because they behave differently with
spaces in the path versus without. One notable example of this is the
hashbang test which does not work because spaces in hashbangs paths are
not supported in Linux.

Co-authored-by: Michael Voříšek <mvorisek@mvorisek.cz>
2023-02-25 14:02:06 +00:00
Arnaud Le Blanc
9788244a42 Merge branch 'PHP-8.2'
* PHP-8.2:
  Fix run-tests.php hanging when a worker process dies without notice (#9931)
2023-01-13 18:07:58 +01:00
Arnaud Le Blanc
61048f984a Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  Fix run-tests.php hanging when a worker process dies without notice (#9931)
2023-01-13 18:07:45 +01:00
Arnaud Le Blanc
8c0698f66b
Fix run-tests.php hanging when a worker process dies without notice (#9931)
run-tests.php with `-jN` can hang if a parallel worker dies without notice. This
can happen due to fatal errors in the worker, or if the worker is killed.

- run-tests.php (main process)
  \_ run-tests.php (worker #0) // main process hangs if this one crashes
     \_ test-001.php (test-001.phpt)
2023-01-13 17:43:17 +01:00
Ilija Tovilo
86af42be1b
Merge branch 'PHP-8.2'
* PHP-8.2:
  Migrate i386 to GitHub actions
2022-11-03 14:40:35 +01:00
Ilija Tovilo
b08901b63f
Merge branch 'PHP-8.1' into PHP-8.2
* PHP-8.1:
  Migrate i386 to GitHub actions
2022-11-03 14:37:59 +01:00
Ilija Tovilo
ed1e703716
Merge branch 'PHP-8.0' into PHP-8.1
* PHP-8.0:
  Migrate i386 to GitHub actions
2022-11-03 14:37:17 +01:00
Ilija Tovilo
75970077b0
Migrate i386 to GitHub actions
Closes GH-9856
2022-11-03 14:34:54 +01:00
Tyson Andre
d498908ec4
Make run-tests.php check for tcp fwrite edge cases (#9242)
When the recipient is busy or the payload is large, fwrite can block
or return a value smaller than the length of the stream.

workers in run-tests.php communicates over tcp sockets with the manager.

https://cirrus-ci.com/task/5315675320221696?logs=tests#L130
showed notices for fwrite/unserialize

This is a similar approach to the approach used in
https://github.com/phan/phan/blob/v5/src/Phan/LanguageServer/ProtocolStreamWriter.php
for the tcp language server writing.
2022-10-07 20:02:22 -04:00
Arnaud Le Blanc
722ed598d5 Do not require opcache.preload_user in cli SAPIs 2022-10-07 15:48:05 +02:00