We're starting to see a mix between uses of zend_bool and bool.
Replace all usages with the standard bool type everywhere.
Of course, zend_bool is retained as an alias.
In the case of a stream with no filters, php_stream_fill_read_buffer
only reads stream->chunk_size into the read buffer. If the stream has
filters attached, it could unnecessarily buffer a large amount of data.
With this change, php_stream_fill_read_buffer only proceeds until either
the requested size or stream->chunk_size is available in the read buffer.
Co-authored-by: Christoph M. Becker <cmbecker69@gmx.de>
Closes GH-6444.
This allows reusing an existing zend_string inside a memory stream
without reallocating. For non-readonly streams, the string will
only get separated on write.
Reading from a stream may return greater than zero, but nonetheless the
stream's EOF flag may have been set. We have to cater to this
condition by setting the close flag for filters.
We also have to cater to that change in the zlib.inflate filter:
If `inflate()` is called with flush mode `Z_FINISH`, but the output
buffer is not large enough to inflate all available data, it fails with
`Z_BUF_ERROR`. However, `Z_BUF_ERROR` is not fatal; in fact, the zlib
manual states: "If deflate returns with Z_OK or Z_BUF_ERROR, this
function must be called again with Z_FINISH and more output space
(updated avail_out) but no more input data, until it returns with
Z_STREAM_END or an error." Hence, we do so.
Closes GH-6001.
mysqlnd currently sets error_reporting=0 to suppress errors while
writing to streams. Unfortunately these errors are still visible
to userland error handlers, which is a source of confusion.
See for example https://bugs.php.net/bug.php?id=80412.
Instead add a stream flag that suppresses the emission of
read/write errors in the first place, and set it in mysqlnd.
I think it might be useful to have this option for userland as
well in the future, but for now this is just an internal
mechanism.
Closes GH-6458.
See GCC bug 69602: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69602
which emits the warning for (errno == EWOULDBLOCK || errno == EAGAIN)
which is the correct way of handling errors as the value of EWOULDBLOCK
and EAGAIN is implementation defined.
Therefore introduce a new macro function PHP_IS_TRANSIENT_ERROR()
which handles the case when EWOULDBLOCK and EAGAIN are identical.
Thanks to @twose for the idea.
We have to log errors in `stream_opener` callbacks to the wrapper's
error log, because otherwise we may pick up an unrelated `errno` or a
most generic message.
Closes GH-6187.
If restoring of any not registered built-in wrapper is requested, the
function is supposed to fail with a warning, so we have to check this
condition first.
Furthermore, to be able to detect whether a built-in wrapper has been
changed, it is not sufficient to check whether *any* userland wrapper
has been registered, but rather whether the specific wrapper has been
modified.
Closes GH-6183.
Passing `NULL` as `lpFileSizeHigh` to `GetFileSize()` gives wrong
results for files larger than 0xFFFFFFFF bytes. We fix this by using
`GetFileSizeEx()`, and let the mapping fail, if the file size is too
large for the architecture.
Closes GH-5319.
Instead of attempting to map large files into memory at once, we map
chunks of at most `PHP_STREAM_MMAP_MAX` bytes, and repeat that until we
hit the point where `php_stream_seek()` fails (see bug 54902), and copy
the rest of the file by reading and writing small chunks.
We also fix the mapping behavior for zero bytes on Windows, which did
not error (as with `mmap()`), but would have mapped the remaining file.
A recent commit[1] which fixed a memory leak introduced a regression
regarding the formerly liberal handling of IP addresses to bind to. We
fix this by reverting that commit, and fix the memory leak where it
actually occurs. In other words, this fix is less intrusive than the
former fix.
[1] <http://git.php.net/?p=php-src.git;a=commit;h=0b8c83f5936581942715d14883cdebddc18bad30>
Closes GH-6104.
This is actually about three distinct issues:
* If an empty string is passed as $address to `stream_socket_sendto()`,
the `sa` is not initialized, so we must not pass it as `addr` to
`php_stream_xport_sendto()`.
* On POSIX, `recvfrom()` truncates messages which are too long to fit
into the specified buffer (unless `MSG_PEEK` is given), discards the
excessive bytes, and returns the buffer length. On Windows, the same
happens, but `recvfrom()` returns `SOCKET_ERROR` with the error code
`WSAEMSGSIZE`. We have to catch this for best POSIX compatibility.
* In `php_network_parse_network_address_with_port()`, we have to zero
`in6` (not only its alias `sa`) to properly support IPv6.
Co-Authored-By: Nikita Popov <nikita.ppv@googlemail.com>
Don't report EAGAIN/EWOULDBLOCK as errors for fwrite on
non-blocking socket streams. This matches behavior for fread,
as well as behavior for plain file streams.
Closes GH-5026.
We add the `is_seekable` member to `php_stdio_stream_data`, and prefer
that over `is_pipe`, since the latter is simply a misnomer. We keep
`is_pipe` for now for Windows only, though, because we need special
support for pipes there. We also fix the misaligned bitfield which
formerly took 33 bit.
First, the limitation already doesn't trigger if you copy the whole
file (i.e. use copy() or stream_copy_to_stream() and don't specify
a length). This happens because length will be 0 at the time of the
check and only later calculated based on the file size. This means
that we're already completely blowing the length limit for what is
likely the most common case, and it doesn't seem like anyone complained
about that.
Second, the premise of the code comment ("to avoid runaway swapping")
seems incorrect to me. Because this performs a file-backed non-private
mmap, no swap backing is needed for the mapping. Concerns over "memory
usage" are also misplaced, as this is a virtual mapping.
We're currently splitting up large writes into 8K size chunks, which
adversely affects I/O performance in some cases. Splitting up writes
doesn't make a lot of sense, as we already must have a backing buffer,
so there is no memory/performance tradeoff to be made here.
This change disables the write chunking at the stream layer, but
retains the current retry loop for partial writes. In particular
network writes will typically only write part of the data for large
writes, so we need to keep the retry loop to preserve backwards
compatibility.
If issues due to this change turn up, chunking should be reintroduced
at lower levels where it is needed to avoid issues for specific streams,
rather than unnecessarily enforcing it for all streams.
This makes the stream opening actually fail, and avoids assertion
failures when we tokenize with EG(exception) set.
Also avoid throwing an additional warning after an exception has
already been thrown.
stream_get-line repeatedly calls php_stream_fill_read_buffer until
enough data is accumulated in buffer. However, when stream contains
filters attached to it, then each call to fill buffer essentially
resets buffer read/write pointers and new data is written over old.
This causes stream_get_line to skip parts of data from stream
This patch fixes such behavior, so fill buffer call will append.
There are two related changes here:
1. Also check for S_ISCHR/FILE_TYPE_CHAR when checking for pipes, so
that we detect ttys as well, which are also not seekable.
2. Always set position=-1 (i.e. ftell will return false) when a pipe
is detected. Previously position=0 was sometimes used, depending on
whether we're on Windows/Linux and whether the FD or FILE codepath
was used.
The php_stream_read() and php_stream_write() functions now return
an ssize_t value, with negative results indicating failure. Functions
like fread() and fwrite() will return false in that case.
As a special case, EWOULDBLOCK and EAGAIN on non-blocking streams
should not be regarded as error conditions, and be reported as
successful zero-length reads/writes instead. The handling of EINTR
remains unclear and is internally inconsistent (e.g. some code-paths
will automatically retry on EINTR, while some won't).
I'm landing this now to make sure the stream wrapper ops API changes
make it into 7.4 -- however, if the user-facing changes turn out to
be problematic we have the option of clamping negative returns to
zero in php_stream_read() and php_stream_write() to restore the
old behavior in a relatively non-intrusive manner.
By adding a flag to avoid forced fstat for includes. The two fstats
will happen back to back and we don't care about a possible
invalidation.
I was hoping to move this higher up in the stack and make the
ISREG check somewhere in fsizer of fixup, but this doesn't really
seem to be possible. E.g. an FP stdin handle will not be a regular
file but of course needs to be allowed. Additionally custom stream
wrappers may not implement this functionality.
Streams will be freed in an unpredictable order during shutdown.
Ignore explicit calls to php_stream_close() entirely to avoid
use-after-free -- instead let the stream resource destructor
deal with it. We have to account for a few special cases:
* Enclosed streams should be freed, as the resource destructor
will forward to the enclosing stream.
* Stream cookies also directly free streams, because we delegate
to the cookie destruction if one exists.
* Mysqlnd also directly frees streams, because it explicitly
removes stream resources (because mysqlnd!)
RFC: https://wiki.php.net/rfc/tostring_exceptions
And convert some object to string conversion related recoverable
fatal errors into Error exceptions.
Improve exception safety of internal code performing string
conversions.
readdir_r() is deprecated in modern glibc versions. readdir() is
thread safe in practice, as long as there are no concurrent accesses
on the *same* directory stream.
In order to rename safer, we do the following:
- set umask to 077 (unfortunately, not TS, so excluding ZTS)
- chown() first, to set proper group before allowing group access
- chmod() after, even if chown() fails
This patch removes the so called local variables defined per
file basis for certain editors to properly show tab width, and
similar settings. These are mainly used by Vim and Emacs editors
yet with recent changes the once working definitions don't work
anymore in Vim without custom plugins or additional configuration.
Neither are these settings synced across the PHP code base.
A simpler and better approach is EditorConfig and fixing code
using some code style fixing tools in the future instead.
This patch also removes the so called modelines for Vim. Modelines
allow Vim editor specifically to set some editor configuration such as
syntax highlighting, indentation style and tab width to be set in the
first line or the last 5 lines per file basis. Since the php test
files have syntax highlighting already set in most editors properly and
EditorConfig takes care of the indentation settings, this patch removes
these as well for the Vim 6.0 and newer versions.
With the removal of local variables for certain editors such as
Emacs and Vim, the footer is also probably not needed anymore when
creating extensions using ext_skel.php script.
Additionally, Vim modelines for setting php syntax and some editor
settings has been removed from some *.phpt files. All these are
mostly not relevant for phpt files neither work properly in the
middle of the file.
This patch adds missing newlines, trims multiple redundant final
newlines into a single one, and trims redundant leading newlines.
According to POSIX, a line is a sequence of zero or more non-' <newline>'
characters plus a terminating '<newline>' character. [1] Files should
normally have at least one final newline character.
C89 [2] and later standards [3] mention a final newline:
"A source file that is not empty shall end in a new-line character,
which shall not be immediately preceded by a backslash character."
Although it is not mandatory for all files to have a final newline
fixed, a more consistent and homogeneous approach brings less of commit
differences issues and a better development experience in certain text
editors and IDEs.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206
[2] https://port70.net/~nsz/c/c89/c89-draft.html#2.1.1.2
[3] https://port70.net/~nsz/c/c99/n1256.html#5.1.1.2
This patch adds missing newlines, trims multiple redundant final
newlines into a single one, and trims redundant leading newlines.
According to POSIX, a line is a sequence of zero or more non-' <newline>'
characters plus a terminating '<newline>' character. [1] Files should
normally have at least one final newline character.
C89 [2] and later standards [3] mention a final newline:
"A source file that is not empty shall end in a new-line character,
which shall not be immediately preceded by a backslash character."
Although it is not mandatory for all files to have a final newline
fixed, a more consistent and homogeneous approach brings less of commit
differences issues and a better development experience in certain text
editors and IDEs.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206
[2] https://port70.net/~nsz/c/c89/c89-draft.html#2.1.1.2
[3] https://port70.net/~nsz/c/c99/n1256.html#5.1.1.2
This patch adds missing newlines, trims multiple redundant final
newlines into a single one, and trims redundant leading newlines.
According to POSIX, a line is a sequence of zero or more non-' <newline>'
characters plus a terminating '<newline>' character. [1] Files should
normally have at least one final newline character.
C89 [2] and later standards [3] mention a final newline:
"A source file that is not empty shall end in a new-line character,
which shall not be immediately preceded by a backslash character."
Although it is not mandatory for all files to have a final newline
fixed, a more consistent and homogeneous approach brings less of commit
differences issues and a better development experience in certain text
editors and IDEs.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206
[2] https://port70.net/~nsz/c/c89/c89-draft.html#2.1.1.2
[3] https://port70.net/~nsz/c/c99/n1256.html#5.1.1.2
This patch adds missing newlines, trims multiple redundant final
newlines into a single one, and trims redundant leading newlines.
According to POSIX, a line is a sequence of zero or more non-' <newline>'
characters plus a terminating '<newline>' character. [1] Files should
normally have at least one final newline character.
C89 [2] and later standards [3] mention a final newline:
"A source file that is not empty shall end in a new-line character,
which shall not be immediately preceded by a backslash character."
Although it is not mandatory for all files to have a final newline
fixed, a more consistent and homogeneous approach brings less of commit
differences issues and a better development experience in certain text
editors and IDEs.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206
[2] https://port70.net/~nsz/c/c89/c89-draft.html#2.1.1.2
[3] https://port70.net/~nsz/c/c99/n1256.html#5.1.1.2
Since Autoconf 2.50+ macro AC_STRUCT_ST_BLOCKS defines the new
HAVE_STRUCT_STAT_ST_BLOCKS symbol and has deprecated the previous
HAVE_ST_BLOCKS.
PHP 5.3 required Autoconf 2.13 (released in 1999) or newer, since PHP
5.4 the autoconf 2.59 (released in 2003) or newer was required, and
since PHP 7.2, autoconf 2.64 (released in 2008) or newer is required.
Autoconf 2.50 released in 2001 has made several macros obsolete. Instead
of the AC_STRUCT_ST_BLKSIZE and AC_STRUCT_ST_RDEV the new
AC_CHECK_MEMBERS should be used.
When checking for the presence of stat struct members st_blkzize and
st_rdev the new AC_CHECK_MEMBERS macro defines new constants
HAVE_STRUCT_STAT_ST_BLKSIZE and HAVE_STRUCT_STAT_ST_RDEV.
Old constants HAVE_ST_BLKSIZE and HAVE_ST_RDEV need to be replaced
respectively in PHP code (this patch) and in PHP extenstions if they use
them.
PHP 5.4 to 7.1 require Autoconf 2.59+ version, PHP 7.2 and above require
2.64+ version, and the PHP 7.2 phpize script requires 2.59+ version which
are all greater than above mentioned 2.50 version.
The $Id$ keywords were used in Subversion where they can be substituted
with filename, last revision number change, last changed date, and last
user who changed it.
In Git this functionality is different and can be done with Git attribute
ident. These need to be defined manually for each file in the
.gitattributes file and are afterwards replaced with 40-character
hexadecimal blob object name which is based only on the particular file
contents.
This patch simplifies handling of $Id$ keywords by removing them since
they are not used anymore.
5060fc23 attempted to fix#68948 by treating all non-uri streams
as non-blocking, however php://fd/* streams (which includes stdin)
may block if the other end of the IPC isn't finished.
This represents a partial revert to the pre RC6 state,
but includes an escape hatch for php://memory and php://temp
streams which are local to the current process.
This also restores stream_set_chunk_size test to previous state.
* master:
Use per-request heap instead of system one
Extend zend_register_class_alias_ex() with additional argument to allow creating persistent or per-request aliases
Makrk persistent resources and references with GC_PERSISTENT flag
Prohibit direct update of GC_REFCOUNT(), GC_SET_REFCOUNT(), GC_ADDREF() and GC_DELREF() shoukf be instead.
Added mactros to validate reference-counting (disabled for now).
These macros are going to be used to eliminate race-condintions during reference-counting on data shared between threads.
This patch however does not drop support for the BeOS compatible variant, Haiku, see Github PR #2697 which is currently a WiP
I intentionally left out some fragments for BeOS in the build system for that seems to be bundles
Having `int` there is no real profit in the size or speed, while unsigned
improves security and overall integration. ZPP supplied strings can
be then accepted directly and structs can be still handled with smaller
unsigned types for size reasons, which is safe. Yet some related places
are to go.
basic move tsrm_realpath_r to size_t
fix conditions and sync with affected places
touch ocurrences of php_sys_readlink usage
follow up on phar path handling
remove duplicated check
move zend_resolve_path and related pieces to size_t
touch yet resolve path related places
remove cast
missing pieces
missing piece
yet cleanups for php_sys_readlink for ssize_t
fix wrong return
While the case in bug #74429 is not documented and is only worky due to
an implementation bug, the strength seems to breach some real world
apps. Given this patch doesn't impact the initial security fix for
bug #74216, it is reasonable to let the apps keep working. As mentioned
in the ticket, this behavior is a subject to change in future versions
and should not be abused.
