Commit Graph

633 Commits

Author SHA1 Message Date
Christoph M. Becker
790b0b109a Fix handling of succeeding XFAIL tests
Due to a typo in commit e1151c9[1], succeeding XFAIL test without an
EXPECTF or EXPECTREGEX section have been reported reported as passed
instead of warned.

[1] <http://git.php.net/?p=php-src.git;a=commit;h=e1151c9549cf5a7235c1609019f79bbb1bbcdbea>
2019-08-05 11:38:47 +02:00
Levi Morrison
d758691b9b Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  Remove .post files only for passing tests
2019-07-25 09:50:47 -06:00
Levi Morrison
c624a7fe43 Merge branch 'PHP-7.2' into PHP-7.3
* PHP-7.2:
  Remove .post files only for passing tests
2019-07-25 09:49:18 -06:00
Levi Morrison
5649267b25 Remove .post files only for passing tests
This allows the sh script for failing tests with --POST-- to work
2019-07-25 09:48:39 -06:00
George Peter Banyard
6d6d954d0d Cleanup of remaining E_STRICT in tests 2019-07-23 11:27:23 +02:00
Peter Kokot
b1643e166d Fix failed tests for phar
- when $status is boolean, E_NOTICE appears and tests fail
- opcache is never enabled on these tests anyway.
- Add opcache.revalidate_freq=0 to testing script

  Instead of checking this in the tests files it can be used directly in
  the main testing script. Also this fixes failed tests.

- Use opcache_invalidate() instead of sleep() delay in some tests. Some
  delays are still used since the opcache_invalidate seems to be buggy
  and not working as expected.

Closes GH-4392
2019-07-21 00:49:35 +02:00
rjhdby
d574df63dc Deprecate alternative array access syntax
RFC: https://wiki.php.net/rfc/deprecate_curly_braces_array_access
2019-07-19 10:06:10 +02:00
Christoph M. Becker
55d34f7023 Remove remaining lightnings from run-tests.php
The classic Windows console only supports the Unicode BMP, so it can't
handle the LIGHTNING MOOD character (U+1F5F2).  It also seems that
common fonts do not support the LIGHTNING (U+2607) character, so a
general removal is appropriate.

However, Windows is already "ready for the future"[1], but PHP for
Windows is not (yet).

[1] <https://devblogs.microsoft.com/commandline/windows-command-line-unicode-and-utf-8-output-text-buffer/>
2019-07-18 12:15:25 +02:00
Nikita Popov
562591f5eb Remove lightning
Windows is not quite ready for the future.
2019-07-18 11:16:24 +02:00
Christoph M. Becker
e2ed7e6716 Allow multiple cache instances per user/host on Windows
Formerly, there was at most a single OPcache instance per user and the
so called system ID (which is determined from the PHP version).
Sometimes multiple OPcaches might be desired, though, particularly for
unrelated CLI scripts, which may even be necessary (e.g. for our test
suite in parallel mode).

We therefore introduce a new INI directive `opcache.cache_id` which
allows to configure independent OPcache instances for the same user.

We also use `GetUserNameW()` instead of `php_win32_get_username()`,
because the latter retrieves the user name encoded in the
`default_charset`, which can obviously yield different results for
different charsets, leading to OPcache "incompatibilities".  Slightly
worse, some characters may not even be encodeable in the
`default_charset` and would be replaced by question marks, which could
result in different users sharing the same OPcache.

We also refactor, and re-use existing APIs to avoid duplicated code.
2019-07-17 19:51:07 +02:00
Peter Kokot
f5e4771824 Remove checks for always enabled spl and pcre
Closes GH-4397
2019-07-13 14:47:37 +02:00
Nikita Popov
c02dfddda3 Include test name in JUnit testcase name
Azure does not show the classname attribute in any convenient way,
and the file name is the part we usually care about.
2019-07-11 12:46:56 +02:00
Nikita Popov
3fa9f9cfae Add CONFLICTS to recognized sections
It does nothing on 7.3, but this makes it easier to write tests in
3rd party extensions.
2019-07-10 13:09:06 +02:00
Nikita Popov
54dd762f59 Set up asan+ubsan scheduled build on azure
Also adds an --asan flag to run-tests.php to setup all the necessary
environment variables. Some tests are marked as skipped because they
are incompatible with asan or too slow.

I'm basing this on the DEBUG_ZTS build, which seems to give us the
most mileage.
2019-06-28 15:00:54 +02:00
Nikita Popov
b195412bf3 run-tests: Don't pass --INI-- settings to --SKIPIF--
If we're testing ini settings that cause startup failures, we'll
never get to that SKIPIF block...

Also change settings2param to return a value instead of modifying
(to a different type no less!) in place.
2019-06-28 09:17:09 +02:00
Nikita Popov
74a7d00846 run-tests: Don't die unnecessarily
die/exit leak memory, don't use them if we don't need to.
2019-06-25 14:28:58 +02:00
Nikita Popov
3a36e68c49 Don't force USE_ZEND_ALLOC=0 in run-tests.php
USE_ZEND_ALLOC=0 is useful not just for valgrind ... we should not
force USE_ZEND_ALLOC=1 if the user explicitly disabled it.
2019-06-25 10:25:44 +02:00
Nikita Popov
da275e6214 Merge branch 'PHP-7.3' into PHP-7.4 2019-06-13 12:41:01 +02:00
Nikita Popov
ee137a7033 Merge branch 'PHP-7.2' into PHP-7.3 2019-06-13 12:38:57 +02:00
Nikita Popov
f8a68fd935 Add test for bug #78106
Also add PHP_TEST_EXTRA_ARGS environment variable, which allows
to pass on -c, -d etc flags provided by run-tests.php. Otherwise
we won't get the built-in server to run with opcache.
2019-06-13 12:35:29 +02:00
Joe Watkins
bed8ae1d0f
junit testcase classname is used as filename on azure 2019-06-05 19:27:50 +02:00
Joe Watkins
22213de179
only clear when showing in parallel runner 2019-06-01 06:13:30 +02:00
Nikita Popov
4ef7f70987 Fix NO_INTERACTION handling in parallel test runner
This printing shouldn't depend on NO_INTERACTION, the variable that
controls this is SHOW_ONLY_GROUPS.
2019-05-31 14:03:22 +02:00
Nikita Popov
dd2bf44869 Add junit support for parallel test runner 2019-05-31 12:38:06 +02:00
Nikita Popov
137747bdaf run-tests: Remove redundant 'suites' key
We don't have nestead testsuites or similar, so drop this confusing
key.
2019-05-31 12:04:23 +02:00
Joe Watkins
a3879208c5
have parallel test runner respect NO_INTERACTION 2019-05-30 07:46:31 +02:00
Nikita Popov
24e22eb31b Run sapi tests when running all tests 2019-05-29 17:07:12 +02:00
Craig Duncan
72c330f1e6 Restore the execute bit for run-tests.php
It looks like this was accidentally removed in 03f15f705a
2019-05-28 21:51:26 +01:00
Joe Watkins
979e7d0fee
fix notice in run-tests 2019-05-23 13:09:18 +02:00
Joe Watkins
e1151c9549
run-tests improvements:
Add XLEAK section, this is made necessary by the support
  for valgrind tools other than memcheck.
  XLEAK is to a leaking test what XFAIL is to a failing test.
  Tidy up summary a little to exclude Expected Fail and Expected Leak
  when there are no expected fail/leaks
2019-05-23 10:55:25 +02:00
Joe Watkins
536532c572
add support for valgrind tools to run-tests.php 2019-05-22 20:20:13 +02:00
Christoph M. Becker
ec10e191c4 Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  Add {TMP} placeholder for PHPT INI sections
2019-05-08 19:26:02 +02:00
Christoph M. Becker
236522529c Merge branch 'PHP-7.2' into PHP-7.3
* PHP-7.2:
  Add {TMP} placeholder for PHPT INI sections
2019-05-08 19:22:21 +02:00
Christoph M. Becker
3d0a0f801b Add {TMP} placeholder for PHPT INI sections
Several tests use `/tmp` in the `--INI--` section, but this is not
portable.  We therefore introduce the `{TMP}` placeholder which
evaluates to the system's temporary directory using
`sys_get_temp_dir()`.

We also remove the doubtful `strpos()` optimization.
2019-05-08 19:21:38 +02:00
Peter Kokot
434388a610 Add minimum required PHP version to run-tests.php
This integrates a simple minimum PHP version info in the tool and to
also define the minimum version that the tool syntax should support.
2019-03-26 22:31:01 +01:00
Nikita Popov
071ffa2e79 Don't use parallel testing with -j1 2019-03-25 14:15:26 +01:00
Nikita Popov
3c61972451 Drop lightning for concurrent worker count
This looked rather awkward even on terminals that have full unicode
support.
2019-03-25 14:13:10 +01:00
Tyson Andre
936356bf1e Make run-tests.php compatible with php 7.0-7.2
Motivation:
As an extension author, I want to speed up running tests in php <=7.3,
both locally and in CI (e.g. with valgrind).
This can be done by manually copying php 7.4's run-tests.php script
to replace the one generated by `phpize`

- list() doesn't work in php 7.0
- negative string offset doesn't work in php 7.2

If run-tests.php can be copied from php-src without any manual patches,
that would be the easiest.

Related to #2822 - I didn't see any discussion for/against compatibility
with older php versions
2019-03-25 13:58:22 +01:00
Nikita Popov
152e5393e5 Add special "all" conflict
If a test conflicts with "all", then no other tests may be run in
parallel. This is needed for windows_mb_path tests, which rely on
the console codepage, which is shared across all parallel workers.

Also add support for comments in the CONFLICTS section/file.
2019-02-21 11:04:43 +01:00
Nikita Popov
967fa51db0 Don't use uniqid for temporary POST data files
uniqid() is not necessarily unique when multiple testing processes
run in parallel.
2019-02-20 16:01:15 +01:00
Nikita Popov
e5304bb647 Add a --shuffle mode to run-tests
Makes it easier to find paralellization issues. Of course only works
if the tests aren't order-dependent...
2019-02-20 15:59:13 +01:00
Nikita Popov
57fef27521 Use PHP_BINARY by default in run-tests.php
You pretty much always want to run with -P, make it the default.
2019-02-20 12:38:40 +01:00
Nikita Popov
c0e15a3b7f Implement fine-grained conflict handling
Tests can specify conflict keys, either in --CONFLICTS-- or
a per-directory CONFLICTS file. Non-conflicting tests may be run
in parallel.
2019-02-20 11:20:40 +01:00
Dmitry Stogov
555fb294ec Don't preload tests without --FILE-- section 2019-02-20 00:06:57 +03:00
Dmitry Stogov
b2073253bf Better new-line handling 2019-02-19 19:22:42 +03:00
Nikita Popov
fa915614fe Make parallel run-tests compatible with IO capture tests
Inherit the main stdin/stdout/stderr streams into the worker processes.
We don't talk to workers via these pipes, so there shouldn't be a
problem doing this.
2019-02-19 15:53:15 +01:00
Dmitry Stogov
5cde3afda7 Better line handling 2019-02-19 17:16:48 +03:00
Dmitry Stogov
bf51b081f5 Filter preloading warning messages 2019-02-19 17:09:07 +03:00
Andrea Faulds
5ee81e0b9d run-tests.php: fix typo 2019-02-18 20:32:16 +01:00
Andrea Faulds
39792f5bb7 run-tests.php: experimental parallel testing support 2019-02-18 14:58:53 +01:00