Commit Graph

681 Commits

Author SHA1 Message Date
Paul Dragoonis
5d7ff25311 Removing HTML Functionality from run-tests.php
As discussed on GH-5632, the HTML functionality does not appear
to be in active use. For HTML rendering of test results, it is
suggested to instead use the JUnit integration, in combination
with your favorite JUnit viewer.

Closes GH-5705.
2020-06-19 09:59:38 +02:00
Nikita Popov
c260613c6f Optimize parallel test runner startup
Don't connect to each worker individually: First spawn all processes
and then establish connections in any order.

This avoids a ~1s wait when running on many cores.
2020-06-16 11:39:03 +02:00
Gerard Roche
d1dde094f2 run-tests: refactor 2020-05-22 09:58:04 +02:00
Gerard Roche
2ef88f517d run-tests: echo message and exit if no tests found
The test runner currently defaults to running the entire test suite if
no selected tests can be found. This can be unexpected.

For example the ext/mysqlnd/ directory has no tests, if you specify that
directory when testing the entire test suite will be run.

    run-tests.php [options] ext/mysqlnd/

Closes GH-5605.
2020-05-22 09:37:46 +02:00
Nikita Popov
0374cf08c7 Add --file-cache-prime/use options to run-tests
--file-cache-prime populates the file cache,
--file-cache-use uses the file cache.

And fix a number of tests to run under file cache or disabled
timestamp validation.
2020-05-20 15:29:32 +02:00
Nikita Popov
ec1e8540d8 Slightly deduplicate run-tests code 2020-05-20 11:58:42 +02:00
Derick Rethans
3707d75f4b Merge branch 'PHP-7.4' 2020-05-18 13:51:47 +01:00
Derick Rethans
d2797505cd Put Xdebug 3 into 'off' mode for run-tests.php 2020-05-18 13:51:40 +01:00
Gerard Roche
ecc0a87ff2 run-tests: extract usage message
Put the usage message near the top of the script,
into a separate function.

Closes GH-5558.
2020-05-13 09:54:55 +02:00
Gerard Roche
10edee7f03 run-tests: cs fixes (cleanup)
I used php-cs-fixer to do the cs fixes. The configuration I used is
posted below. The reason I disabled some of the rules is because they
create too much noise and would make it difficult to review. But please
feel free to close this PR and run the php-cs-fixer yourself.

    <?php

    $config = PhpCsFixer\Config::create();
    $config->setRiskyAllowed(false);
    $config->setRules([
        '@PSR2' => true,
        '@Symfony' => true,
        'array_syntax' => false,
        'binary_operator_spaces' => false,
        'blank_line_before_statement' => false,
        'concat_space' => false,
        'increment_style' => false,
        'phpdoc_align' => false,
        'single_quote' => false,
        'trailing_comma_in_multiline_array' => false,
        'unary_operator_spaces' => false,
        'yoda_style' => false,
    ]);

    $finder = PhpCsFixer\Finder::create();
    $finder->in(getcwd());

    $finder->exclude('Zend');
    $finder->exclude('build');
    $finder->exclude('ext');
    $finder->exclude('pear');
    $finder->exclude('sapi');
    $finder->exclude('scripts');
    $finder->exclude('win32');

    $config->setFinder($finder);

    return $config;

Closes GH-5557.
2020-05-13 09:52:44 +02:00
Gerard Roche
16f23cd159 run-tests: remove use of FILE_BINARY constant
The FILE_BINARY (and FILE_TEXT) constants are not really valid or useful
constants. It looks like they were added in 5.2.7 and have "no effect,
and are only available for forward compatibility."

See: https://www.php.net/manual/en/filesystem.constants.php

The default value of the file_put_contents() flags parameter is 0 and
FILE_BINARY is set to 0, so removing it doesn't change functionality.

P.S. Maybe those constants should be deprecated or removed in 8.0.

Closes GH-5556.
2020-05-12 16:59:25 +02:00
Christoph M. Becker
8cb237345a Revert "Show eventual output of clean sections"
This reverts commit 5eb4ab07f2.
The temporary hack has served its purpose.
2020-04-30 09:51:10 +02:00
Christoph M. Becker
daaaf2b4ff Merge branch 'PHP-7.4'
* PHP-7.4:
  Pass worker ID to clean scripts
2020-04-29 13:33:32 +02:00
Christoph M. Becker
9a98569efe Pass worker ID to clean scripts
On Windows, reusing/sharing of OPcache instances with different
configuration is not necessarily supported, so we have to make that it
does not happen for the clean scripts, by using `$orig_ini_settings`
instead of `$clean_params`.
2020-04-29 13:30:28 +02:00
Christoph M. Becker
5eb4ab07f2 Show eventual output of clean sections
This is a hack to investigate why mysqli_insert_packet_overflow.phpt
intermittently fails on AppVeyor, and will be reverted in due time.
See <https://github.com/php/php-src/pull/5447#issuecomment-620508790>.
2020-04-28 16:08:17 +02:00
Flávio Heleno
1baa58317f Fixed run-tests.php for PHP 7.2
Flexible heredoc syntax is only available since PHP 7.3.

Closes GH-5444.
2020-04-27 11:55:40 +02:00
Benjamin Eberlei
ec88bbc4c6 Enhance test failure SH script to allow gdb, valgrind, rr as alternative modes. 2020-04-18 16:14:00 +02:00
Nikita Popov
fd5dc55759 Force short_open_tag=0 in run-tests.php
Make sure we don't accidentially add tests depending on short tags.
2020-04-15 11:27:03 +02:00
Nikita Popov
a939805641 Use serialize_precision for var_dump()
var_dump() is debugging functionality, so it should print
floating-point numbers accurately. We do this by switching
to serialize_precision, which (by default) will print with
as much precision as necessary to preserve the exact value
of the float.

This also affects debug_zval_dump().

Closes GH-5172.
2020-02-25 09:51:32 +01:00
Christoph M. Becker
7d6384d1f7 Merge branch 'PHP-7.4'
* PHP-7.4:
  Create a new console for each test worker on Windows
2020-02-17 18:54:32 +01:00
Christoph M. Becker
e1de11d467 Create a new console for each test worker on Windows
The primary motivation to have each test worker running its own console
is to allow the windows_mb_path tests to run in parallel.  A nice side
effect is that this also prevents changing the code page of the
tester's console window (which can even cause its font to be changed).

To be able to do so, we introduce the `create_new_console` option for
`proc_open()`, which might occasionally be useful for other purposes
than testing.
2020-02-17 18:52:45 +01:00
Nikita Popov
169805777c Merge branch 'PHP-7.4'
* PHP-7.4:
  Apply tidy formatting
2020-02-03 13:42:08 +01:00
Nikita Popov
58b17906f5 Apply tidy formatting
Mostly reindent PHP scripts to spaces.
2020-02-03 13:41:31 +01:00
Nikita Popov
98a8ab1389 Merge branch 'PHP-7.4'
* PHP-7.4:
  Add WHITESPACE_SENSITIVE run-tests section
2020-02-03 10:08:20 +01:00
Nikita Popov
27fb0b28dd Add WHITESPACE_SENSITIVE run-tests section
This is used to indicate that the test should not be changed by
automated formatting changes.
2020-02-03 10:08:15 +01:00
Nikita Popov
1d0247db7d Merge branch 'PHP-7.4'
* PHP-7.4:
  Don't use CRLF when generating diffs
2020-01-24 12:28:54 +01:00
Nikita Popov
c6cf354a9a Don't use CRLF when generating diffs 2020-01-24 12:28:37 +01:00
Tyson Andre
0c696577eb Use smaller batch size in run-tests.php when appropriate
- When valgrind is used, communication overhead is relatively small,
  so just use a batch size of 1.
- If this is running a small enough number of tests,
  reduce the batch size to give batches to more workers.
  (Previously, if there were 90 tests and -j8,
  only 3 of 8 workers would get a batch of size 32 or less.
  After this change, the batch size is 12 or less)

Closes GH-5098
2020-01-21 20:07:08 -05:00
Tyson Andre
549f55fc3f Don't start unnecessary processes with run-tests.php -j
If there's only 2 files to test, then only start 2 workers instead of N.

If there's only 1 file, then avoid parallelism entirely.
A separate option such as `--force-parallel` could be added
if this turns out to be something developers would want to do
when debugging test failures.
2020-01-21 20:05:06 -05:00
Nikita Popov
85ea04b747 Merge branch 'PHP-7.4'
* PHP-7.4:
  Avoid some @count() suppressions in run-tests.php
2020-01-21 18:50:14 +01:00
Nikita Popov
59b3ab827a Avoid some @count() suppressions in run-tests.php 2020-01-21 18:49:59 +01:00
Christoph M. Becker
718fc4cefd Use IS_WINDOWS 2020-01-09 12:04:28 +01:00
Christoph M. Becker
10c420f84a Replace fakemail with minimal PHP script 2020-01-09 12:04:28 +01:00
Christoph M. Becker
64633044c5 Unify mail related tests for *nix and Windows
Currently mail related tests are split for *nix and Windows (if there
are even Windows versions).  The basic difference is that the *nix
variants set the INI directive sendmail_path to just write the email to
disk, while the Windows tests use ext/imap.  The latter tests are way
more verbose, and such duplicated tests are generally a pain point.
Furthermore, the Windows tests are much slower, and could not be run
without ext/imap being available.

We therefore introduce a small fakemail application, which basically
works like `tee <path> >/dev/null`, and which will be shipped with the
Windows tests packs.  fakemail.exe would also need to be added to the
PHP binary SDK, so these tests could be run during developments.

To cater to the remaining differences, we also introduce support for
`{MAIL:<path>}` placeholders in the INI sections to run-tests.php.  How
to use this can be seen in mail_basic.phpt, which is currently the only
modified test case, because these tests are yet supposed to fail on
Windows, due to the missing fakemail.exe in the PHP SDK.
2020-01-09 12:04:28 +01:00
Nikita Popov
725c2a999d Merge branch 'PHP-7.4'
* PHP-7.4:
  Set zend.exception_ignore_args=0 in run-tests.php
2019-12-10 16:16:03 +01:00
Nikita Popov
e87dc32569 Set zend.exception_ignore_args=0 in run-tests.php
Any existing tests are going to be written under this assumption...
2019-12-10 16:15:26 +01:00
Fabien Villepinte
4d529757f8 Merge branch 'PHP-7.4' 2019-11-08 20:36:55 +01:00
Fabien Villepinte
9c4b989c75 Make test runner runnable without arguments
The default PHP executable was not set when no args were provided.

Closes GH-4894.
2019-11-08 20:33:19 +01:00
Fabien Villepinte
26579db403 Define IS_WINDOWS in the test runner
Avoid code duplication

Closes GH-4866.
2019-10-28 13:14:50 +01:00
Nikita Popov
c1da10b77b Set display_startup_errors=0 when running skipif
Now that we're displaying errors in skipif, suppress this particular
category. Otherwise we get warnings in the SKIPIF of tests with
EXTENSION where the extension does not exist, and there's no way
to suppress them.
2019-10-10 10:43:21 +02:00
Christoph M. Becker
e54d2efdda Merge branch 'PHP-7.4'
* PHP-7.4:
  Bail out if JUNIT is not enabled
2019-10-07 13:22:47 +02:00
Christoph M. Becker
8cdab9b217 Bail out if JUNIT is not enabled
Otherwise we would try to access an array element of `false`, which
issues a notice as of PHP 7.4.0.  This would happen, for instance, for
bug63447_001.phpt if CGI is not available.
2019-10-07 13:22:11 +02:00
Christoph M. Becker
d37454d9f0 Merge branch 'PHP-7.4'
* PHP-7.4:
  Delete temporary .post files of run-tests.php
2019-10-07 12:18:27 +02:00
Christoph M. Becker
53077217e2 Delete temporary .post files of run-tests.php
Commit 5649267[1] changed run-tests.php to only delete .post files for
passing tests.  However, that code wouldn't be exercised at all, since
`run_test()` already returned, so we move the deletion upwards.

[1]
<http://git.php.net/?p=php-src.git;a=commit;h=5649267b257c78e46a934434c0bff894e0b5b694>
2019-10-07 12:17:41 +02:00
Nikita Popov
7d90f4864d Require a space ofter run-tests "warn"
We don't want to treat a PHP "Warning: " as a run-tests warn.
2019-10-01 17:36:42 +02:00
Fabien Villepinte
e49593a7c3 Mark test as borked when skipif produces invalid output
SKIPIF section should either output nothing, or start with a
supported keyword like "skip". Mark all tests that don't do so as
BORKED.
2019-09-30 17:52:43 +02:00
Gabriel Caruso
5d6e923d46
Remove mention of PHP major version in Copyright headers
Closes GH-4732.
2019-09-25 14:51:43 +02:00
Nikita Popov
279f745a60 Fix reporting of redirect test failures in junit
These were being reported under the wrong name ... which means that
they don't show up as failures on azure :(
2019-09-07 22:47:50 +02:00
Christoph M. Becker
790b0b109a Fix handling of succeeding XFAIL tests
Due to a typo in commit e1151c9[1], succeeding XFAIL test without an
EXPECTF or EXPECTREGEX section have been reported reported as passed
instead of warned.

[1] <http://git.php.net/?p=php-src.git;a=commit;h=e1151c9549cf5a7235c1609019f79bbb1bbcdbea>
2019-08-05 11:38:47 +02:00
Levi Morrison
d758691b9b Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  Remove .post files only for passing tests
2019-07-25 09:50:47 -06:00