php-src/Zend
Arnaud Le Blanc 4df3dd7679
Reduce memory allocated by var_export, json_encode, serialize, and other (#8902)
smart_str uses an over-allocated string to optimize for append operations. Functions that use smart_str tend to return the over-allocated string directly. This results in unnecessary memory usage, especially for small strings.

The overhead can be up to 231 bytes for strings smaller than that, and 4095 for other strings. This can be avoided for strings smaller than `4096 - zend_string header size - 1` by reallocating the string.

This change introduces `smart_str_trim_to_size()`, and calls it in `smart_str_extract()`. Functions that use `smart_str` are updated to use `smart_str_extract()`.

Fixes GH-8896
2022-07-08 14:47:46 +02:00
..
asm Add IBT support for fiber 2022-05-25 13:42:31 -05:00
Optimizer Add support for Disjoint Normal Form (DNF) types (#8725) 2022-07-08 11:30:23 +01:00
tests Add support for Disjoint Normal Form (DNF) types (#8725) 2022-07-08 11:30:23 +01:00
bench.php Apply tidy formatting 2020-02-03 13:41:31 +01:00
LICENSE
Makefile.frag Fix race condition in automatic zend_vm_gen.php invocation 2021-07-16 10:03:00 +02:00
micro_bench.php Apply tidy formatting 2020-02-03 13:41:31 +01:00
README.md [ci skip] Update README.md on ZE description 2022-02-08 10:38:33 +01:00
zend_alloc_sizes.h
zend_alloc.c Specify unit in out of memory error (#8820) 2022-06-21 12:37:38 +01:00
zend_alloc.h Determine value of ZEND_MM_* during config and fix sign conversion (#6981) 2022-04-01 15:43:42 +01:00
zend_API.c Add support for Disjoint Normal Form (DNF) types (#8725) 2022-07-08 11:30:23 +01:00
zend_API.h Fix enum preloading again (#8859) 2022-06-25 14:55:46 +02:00
zend_arena.h Replace zend_bool uses with bool 2021-01-15 12:33:06 +01:00
zend_ast.c Allow arbitrary const expressions in backed enums 2022-06-12 22:56:05 +02:00
zend_ast.h Merge concatenated literal strings while compiling. (#7948) 2022-01-16 13:31:58 -05:00
zend_atomic.c Make vm_interrupt and timed_out atomic (#8327) 2022-06-01 09:43:25 -06:00
zend_atomic.h Fix zend_atomic_bool_exchange_ex() in HAVE_NO_ATOMICS case (#8801) 2022-06-30 11:54:47 +08:00
zend_attributes_arginfo.h Allow for arbitrary (class) attributes in stubs 2022-07-05 18:23:05 +02:00
zend_attributes.c Allow for arbitrary (class) attributes in stubs 2022-07-05 18:23:05 +02:00
zend_attributes.h Allow for arbitrary (class) attributes in stubs 2022-07-05 18:23:05 +02:00
zend_attributes.stub.php Allow for arbitrary (class) attributes in stubs 2022-07-05 18:23:05 +02:00
zend_bitset.h Update http->https in license (#6945) 2021-05-06 12:16:35 +02:00
zend_build.h
zend_builtin_functions_arginfo.h Allow for arbitrary (class) attributes in stubs 2022-07-05 18:23:05 +02:00
zend_builtin_functions.c Support the #[\AllowDynamicProperties] attribute in stubs (#8776) 2022-06-21 09:28:57 +02:00
zend_builtin_functions.h Improve type declarations for Zend APIs 2020-08-28 15:41:27 +02:00
zend_builtin_functions.stub.php Support the #[\AllowDynamicProperties] attribute in stubs (#8776) 2022-06-21 09:28:57 +02:00
zend_closures_arginfo.h Add ZEND_ACC_NOT_SERIALIZABLE flag 2021-07-19 15:59:11 +02:00
zend_closures.c Convert return type of various object handlers from int to zend_result (#8755) 2022-06-26 01:00:19 +02:00
zend_closures.h Add first-class callables 2021-07-14 14:37:25 +02:00
zend_closures.stub.php Add ZEND_ACC_NOT_SERIALIZABLE flag 2021-07-19 15:59:11 +02:00
zend_compile.c Add support for Disjoint Normal Form (DNF) types (#8725) 2022-07-08 11:30:23 +01:00
zend_compile.h Fix GH-8661: Nullsafe in coalesce triggers undefined variable warning 2022-06-12 21:52:14 +02:00
zend_config.w32.h Remove "inline" defines on Windows (#7071) 2021-05-31 10:05:18 +02:00
zend_constants_arginfo.h Indent with TAB in .h files generated by gen_stub 2022-06-13 08:55:54 -04:00
zend_constants.c Add support for sensitive parameters in stubs 2022-06-04 18:15:05 +02:00
zend_constants.h Fix memory leak 2022-05-11 11:43:01 +03:00
zend_constants.stub.php Add support for class constants in stubs (#7434) 2022-05-22 22:27:23 +02:00
zend_cpuinfo.c Merge branch 'PHP-8.0' into PHP-8.1 2021-12-23 16:21:28 +01:00
zend_cpuinfo.h fix [-Wstrict-prototypes] buid warnings 2022-02-24 15:44:09 +01:00
zend_default_classes.c Implement Fibers 2021-04-26 11:07:06 -05:00
zend_dtrace.c fix dtrace observer callback prototype 2021-06-04 16:03:09 +02:00
zend_dtrace.d
zend_dtrace.h fix dtrace observer callback prototype 2021-06-04 16:03:09 +02:00
zend_enum_arginfo.h Implement enums 2021-03-17 19:08:03 +01:00
zend_enum.c Fix enum preloading again (#8859) 2022-06-25 14:55:46 +02:00
zend_enum.h Allow arbitrary const expressions in backed enums 2022-06-12 22:56:05 +02:00
zend_enum.stub.php Implement enums 2021-03-17 19:08:03 +01:00
zend_errors.h
zend_exceptions_arginfo.h Add some stub related todo comments 2022-05-25 13:24:11 +02:00
zend_exceptions.c Add support for class constants in stubs (#7434) 2022-05-22 22:27:23 +02:00
zend_exceptions.h Allow creating Graceful/UnwindExit and use when destroying a fiber (#7174) 2021-06-28 15:23:34 -05:00
zend_exceptions.stub.php Add some stub related todo comments 2022-05-25 13:24:11 +02:00
zend_execute_API.c Fix leak of backed_enum_table with preloading 2022-06-23 19:17:44 +02:00
zend_execute.c Add support for Disjoint Normal Form (DNF) types (#8725) 2022-07-08 11:30:23 +01:00
zend_execute.h Merge branch 'PHP-8.1' 2022-07-01 12:20:32 +02:00
zend_extensions.c Merge branch 'PHP-8.1' 2022-03-01 14:55:09 +01:00
zend_extensions.h Merge branch 'PHP-8.1' 2022-03-01 14:55:09 +01:00
zend_fibers_arginfo.h Add ZEND_ACC_NOT_SERIALIZABLE flag 2021-07-19 15:59:11 +02:00
zend_fibers.c Zend, ext/opcache: use PR_SET_VMA_ANON_NAME (Linux 5.17) (#8234) 2022-06-20 12:27:01 +01:00
zend_fibers.h Extend resource reservation to Fibers (#7292) 2021-07-21 20:19:20 +02:00
zend_fibers.stub.php Add ZEND_ACC_NOT_SERIALIZABLE flag 2021-07-19 15:59:11 +02:00
zend_float.c Revert "Fix #79595: zend_init_fpu() alters FPU precision" 2020-05-22 16:57:14 +02:00
zend_float.h Fix #79595: zend_init_fpu() alters FPU precision 2020-05-26 17:19:54 +02:00
zend_gc.c Avoid possible [-Wstrict-prototypes] build warnings 2022-02-24 16:14:47 +01:00
zend_gc.h Don't perform recursive get_gc call 2021-06-09 11:15:59 +02:00
zend_gdb.c Merge branch 'PHP-8.0' into PHP-8.1 2021-12-21 07:19:58 +01:00
zend_gdb.h Improve type declarations for Zend APIs 2020-08-28 15:41:27 +02:00
zend_generators_arginfo.h Add ZEND_ACC_NOT_SERIALIZABLE flag 2021-07-19 15:59:11 +02:00
zend_generators.c Merge branch 'PHP-8.1' 2022-04-04 14:25:42 +02:00
zend_generators.h Replace zend_bool uses with bool 2021-01-15 12:33:06 +01:00
zend_generators.stub.php Add ZEND_ACC_NOT_SERIALIZABLE flag 2021-07-19 15:59:11 +02:00
zend_globals_macros.h
zend_globals.h Make vm_interrupt and timed_out atomic (#8327) 2022-06-01 09:43:25 -06:00
zend_hash.c Add some const qualifiers in zend_string/hash (#8304) 2022-04-20 15:56:51 +01:00
zend_hash.h Add some const qualifiers in zend_string/hash (#8304) 2022-04-20 15:56:51 +01:00
zend_highlight.c Introduce pseudo-keyword ZEND_FALLTHROUGH 2021-04-07 00:46:29 +01:00
zend_highlight.h Accept zend_string in highlight_string API 2020-10-22 14:35:40 +02:00
zend_inheritance.c Add support for Disjoint Normal Form (DNF) types (#8725) 2022-07-08 11:30:23 +01:00
zend_inheritance.h Fix delayed early binding with optimization 2021-09-29 18:00:20 +02:00
zend_ini_parser.y Change Zend Stream API to use zend_string* instead of char*. 2021-03-16 20:31:36 +03:00
zend_ini_scanner.h Improve type declarations for Zend APIs 2020-08-28 15:41:27 +02:00
zend_ini_scanner.l Fix/improve handling of escaping in ini parser 2021-08-30 16:59:22 +02:00
zend_ini.c Fix type (#8814) 2022-06-17 15:23:21 +02:00
zend_ini.h Fix type (#8814) 2022-06-17 15:23:21 +02:00
zend_interfaces_arginfo.h Drop serial denier functions 2021-07-20 13:11:11 +02:00
zend_interfaces.c Improve error message class type 2022-03-20 20:54:36 +01:00
zend_interfaces.h Fix GH-7958: Nested CallbackFilterIterator is leaking memory 2022-02-21 12:39:07 +01:00
zend_interfaces.stub.php Drop serial denier functions 2021-07-20 13:11:11 +02:00
zend_istdiostream.h
zend_iterators.c Add get_gc handle for object iterators 2020-07-01 15:17:22 +02:00
zend_iterators.h Cache method overrides of ArrayAccess in zend_class_entry 2021-12-04 11:35:38 -05:00
zend_language_parser.y Add support for Disjoint Normal Form (DNF) types (#8725) 2022-07-08 11:30:23 +01:00
zend_language_scanner.h Fix parsing of semi-reserved tokens at offset > 4 GB 2021-01-25 14:37:36 +01:00
zend_language_scanner.l Avoid non-immutable map_ptr indirection 2021-10-14 12:16:18 +03:00
zend_list.c Use more compact representation for packed arrays. 2021-11-03 15:18:26 +03:00
zend_list.h Improve type declarations for Zend APIs 2020-08-28 15:41:27 +02:00
zend_llist.c Change Zend Stream API to use zend_string* instead of char*. 2021-03-16 20:31:36 +03:00
zend_llist.h Fix MSVC level 1 (severe) warnings 2020-06-05 11:17:05 +02:00
zend_long.h Replace php_stdint.h header with standard headers (#8613) 2022-05-29 11:20:56 +01:00
zend_map_ptr.h ext/opcache: C++ compatibility 2022-03-20 20:38:23 +01:00
zend_mmap.h Zend, ext/opcache: use PR_SET_VMA_ANON_NAME (Linux 5.17) (#8234) 2022-06-20 12:27:01 +01:00
zend_modules.h Prepare for PHP 8.2 2021-08-31 19:13:49 +02:00
zend_multibyte.c Improve type declarations for Zend APIs 2020-08-28 15:41:27 +02:00
zend_multibyte.h Improve type declarations for Zend APIs 2020-08-28 15:41:27 +02:00
zend_multiply.h Improve type declarations for Zend APIs 2020-08-28 15:41:27 +02:00
zend_object_handlers.c Merge branch 'PHP-8.1' 2022-07-01 12:20:32 +02:00
zend_object_handlers.h Convert return type of various object handlers from int to zend_result (#8755) 2022-06-26 01:00:19 +02:00
zend_objects_API.c Add API to prevent Fiber switch in select contexts 2021-06-14 14:19:00 -05:00
zend_objects_API.h We don't have to clear zend_object structure, it's initialized by zend_object_std_init() anyway. 2021-04-02 16:11:34 +03:00
zend_objects.c Use more compact representation for packed arrays. 2021-11-03 15:18:26 +03:00
zend_objects.h
zend_observer.c Merge branch 'PHP-8.1' 2022-03-01 14:55:09 +01:00
zend_observer.h Merge branch 'PHP-8.0' into PHP-8.1 2022-03-01 14:54:09 +01:00
zend_opcode.c Fix enum preloading again (#8859) 2022-06-25 14:55:46 +02:00
zend_operators.c Deprecate zend_atol() / add zend_ini_parse_quantity() (#7951) 2022-06-17 14:12:53 +02:00
zend_operators.h Deprecate zend_atol() / add zend_ini_parse_quantity() (#7951) 2022-06-17 14:12:53 +02:00
zend_portability.h Add ZEND_THREEWAY_COMPARE() macro to fix casting underflowed unsigned to signed (#8220) 2022-06-08 13:24:18 +01:00
zend_ptr_stack.c Replace zend_bool uses with bool 2021-01-15 12:33:06 +01:00
zend_ptr_stack.h Use safe_*erealloc* flavor in few places to mitigate possible overflows. 2022-07-04 14:41:04 +01:00
zend_range_check.h
zend_signal.c Use memrchr() when available 2021-11-24 16:13:34 +03:00
zend_signal.h Update http->https in license (#6945) 2021-05-06 12:16:35 +02:00
zend_smart_str_public.h Reduce memory allocated by var_export, json_encode, serialize, and other (#8902) 2022-07-08 14:47:46 +02:00
zend_smart_str.c Merge branch 'PHP-8.0' into PHP-8.1 2021-09-16 16:30:01 +02:00
zend_smart_str.h Reduce memory allocated by var_export, json_encode, serialize, and other (#8902) 2022-07-08 14:47:46 +02:00
zend_smart_string_public.h Update http->https in license (#6945) 2021-05-06 12:16:35 +02:00
zend_smart_string.h Update http->https in license (#6945) 2021-05-06 12:16:35 +02:00
zend_sort.c Remove zend_qsort 2020-05-12 16:57:53 +02:00
zend_sort.h Remove zend_qsort 2020-05-12 16:57:53 +02:00
zend_stack.c Replace zend_bool uses with bool 2021-01-15 12:33:06 +01:00
zend_stack.h Replace zend_bool uses with bool 2021-01-15 12:33:06 +01:00
zend_stream.c Fix some MSAN complaints under Clang (#8553) 2022-05-13 23:30:20 +01:00
zend_stream.h Change Zend Stream API to use zend_string* instead of char*. 2021-03-16 20:31:36 +03:00
zend_string.c Merge branch 'PHP-8.1' 2022-05-07 16:48:23 +01:00
zend_string.h Add true as a type (#8326) 2022-06-12 23:28:19 +01:00
zend_strtod_int.h Drop unnecessary stdint and inttypes header checks 2020-04-22 20:18:19 +02:00
zend_strtod.c Move php_gcvt to zend_gcvt 2021-08-02 14:51:46 +02:00
zend_strtod.h Move php_gcvt to zend_gcvt 2021-08-02 14:51:46 +02:00
zend_system_id.c Update http->https in license (#6945) 2021-05-06 12:16:35 +02:00
zend_system_id.h Update http->https in license (#6945) 2021-05-06 12:16:35 +02:00
zend_type_info.h Convert iterable into an internal alias for Traversable|array (#7309) 2022-06-07 13:35:34 +01:00
zend_types.h Convert iterable into an internal alias for Traversable|array (#7309) 2022-06-07 13:35:34 +01:00
zend_variables.c Fixed bug #81514 2021-10-08 10:31:24 +02:00
zend_variables.h Drop various unused macros/APIs 2020-08-26 12:59:43 +02:00
zend_virtual_cwd.c Initialise zend_stat_t to fix MSAN build 2022-05-22 16:06:27 +01:00
zend_virtual_cwd.h Update http->https in license (#6945) 2021-05-06 12:16:35 +02:00
zend_vm_def.h Don't shortcut empty oparray executions if zend_execute_ex has been overridden 2022-06-22 12:28:36 +01:00
zend_vm_execute.h Don't shortcut empty oparray executions if zend_execute_ex has been overridden 2022-06-22 12:28:36 +01:00
zend_vm_execute.skl Cleanup observer API and add JIT support 2020-09-18 12:55:58 +03:00
zend_vm_gen.php Collapse more generated files in PRs (#8754) 2022-06-12 10:56:35 +02:00
zend_vm_handlers.h Merge branch 'PHP-8.1' 2022-02-01 16:26:29 +03:00
zend_vm_opcodes.c Merge branch 'PHP-8.1' 2022-02-01 16:26:29 +03:00
zend_vm_opcodes.h Remove JMPZNZ opcode 2022-01-10 22:07:10 +01:00
zend_vm_trace_handlers.h Use more compact representation for packed arrays. 2021-11-03 15:18:26 +03:00
zend_vm_trace_lines.h A helper to trace executed source lines 2020-06-10 00:13:39 +03:00
zend_vm_trace_map.h Use more compact representation for packed arrays. 2021-11-03 15:18:26 +03:00
zend_vm.h Add function exposing HAVE_GCC_GLOBAL_REGS (#8359) 2022-06-08 12:32:30 +01:00
zend_weakrefs_arginfo.h Not serializable flag permeation 2021-07-20 12:28:35 +02:00
zend_weakrefs.c Convert return type of various object handlers from int to zend_result (#8755) 2022-06-26 01:00:19 +02:00
zend_weakrefs.h Improve performance of WeakReference/WeakMap. Avoid hash collisions on pointers. (#7690) 2021-11-27 19:52:30 -05:00
zend_weakrefs.stub.php Not serializable flag permeation 2021-07-20 12:28:35 +02:00
zend.c Reduce memory allocated by var_export, json_encode, serialize, and other (#8902) 2022-07-08 14:47:46 +02:00
zend.h Fix enum preloading again (#8859) 2022-06-25 14:55:46 +02:00
Zend.m4 Determine value of ZEND_MM_* during config and fix sign conversion (#6981) 2022-04-01 15:43:42 +01:00

Zend Engine

Zend memory manager

General

The goal of the new memory manager (available since PHP 5.2) is to reduce memory allocation overhead and speedup memory management.

Debugging

Normal:

sapi/cli/php -r 'leak();'

Zend MM disabled:

USE_ZEND_ALLOC=0 valgrind --leak-check=full sapi/cli/php -r 'leak();'

Shared extensions

Since PHP 5.3.11 it is possible to prevent shared extensions from unloading so that valgrind can correctly track the memory leaks in shared extensions. For this there is the ZEND_DONT_UNLOAD_MODULES environment variable. If set, then DL_UNLOAD() is skipped during the shutdown of shared extensions.

ZEND_VM

ZEND_VM architecture allows specializing opcode handlers according to op_type fields and using different execution methods (call threading, switch threading and direct threading). As a result ZE2 got more than 20% speedup on raw PHP code execution (with specialized executor and direct threading execution method). As in most PHP applications raw execution speed isn't the limiting factor but system calls and database calls are, your mileage with this patch will vary.

Most parts of the old zend_execute.c go into zend_vm_def.h. Here you can find opcode handlers and helpers. The typical opcode handler template looks like this:

ZEND_VM_HANDLER(<OPCODE-NUMBER>, <OPCODE>, <OP1_TYPES>, <OP2_TYPES>)
{
    <HANDLER'S CODE>
}

<OPCODE-NUMBER> is a opcode number (0, 1, ...) <OPCODE> is an opcode name (ZEN_NOP, ZEND_ADD, :) <OP1_TYPES> and <OP2_TYPES> are masks for allowed operand op_types. Specializer will generate code only for defined combination of types. You can use any combination of the following op_types UNUSED, CONST, VAR, TMP and CV also you can use ANY mask to disable specialization according operand's op_type. <HANDLER'S CODE> is a handler's code itself. For most handlers it stills the same as in old zend_execute.c, but now it uses macros to access opcode operands and some internal executor data.

You can see the conformity of new macros to old code in the following list:

EXECUTE_DATA
    execute_data
ZEND_VM_DISPATCH_TO_HANDLER(<OP>)
    return <OP>_helper(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU)
ZEND_VM_DISPATCH_TO_HELPER(<NAME>)
    return <NAME>(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU)
ZEND_VM_DISPATCH_TO_HELPER_EX(<NAME>,<PARAM>,<VAL>)
    return <NAME>(<VAL>, ZEND_OPCODE_HANDLER_ARGS_PASSTHRU)
ZEND_VM_CONTINUE()
    return 0
ZEND_VM_NEXT_OPCODE()
    NEXT_OPCODE()
ZEND_VM_SET_OPCODE(<TARGET>
    SET_OPCODE(<TARGET>
ZEND_VM_INC_OPCODE()
    INC_OPCOD()
ZEND_VM_RETURN_FROM_EXECUTE_LOOP()
    RETURN_FROM_EXECUTE_LOOP()
ZEND_VM_C_LABEL(<LABEL>):
    <LABEL>:
ZEND_VM_C_GOTO(<LABEL>)
    goto <LABEL>
OP<X>_TYPE
    opline->op<X>.op_type
GET_OP<X>_ZVAL_PTR(<TYPE>)
    get_zval_ptr(&opline->op<X>, EX(Ts), &free_op<X>, <TYPE>)
GET_OP<X>_ZVAL_PTR_PTR(<TYPE>)
    get_zval_ptr_ptr(&opline->op<X>, EX(Ts), &free_op<X>, <TYPE>)
GET_OP<X>_OBJ_ZVAL_PTR(<TYPE>)
    get_obj_zval_ptr(&opline->op<X>, EX(Ts), &free_op<X>, <TYPE>)
GET_OP<X>_OBJ_ZVAL_PTR_PTR(<TYPE>)
    get_obj_zval_ptr_ptr(&opline->op<X>, EX(Ts), &free_op<X>, <TYPE>)
IS_OP<X>_TMP_FREE()
    IS_TMP_FREE(free_op<X>)
FREE_OP<X>()
    FREE_OP(free_op<X>)
FREE_OP<X>_IF_VAR()
    FREE_VAR(free_op<X>)
FREE_OP<X>_VAR_PTR()
    FREE_VAR_PTR(free_op<X>)

Executor's helpers can be defined without parameters or with one parameter. This is done with the following constructs:

ZEND_VM_HELPER(<HELPER-NAME>, <OP1_TYPES>, <OP2_TYPES>)
{
    <HELPER'S CODE>
}

ZEND_VM_HELPER_EX(<HELPER-NAME>, <OP1_TYPES>, <OP2_TYPES>, <PARAM_SPEC>)
{
    <HELPER'S CODE>
}

The executors code is generated by the PHP script zend_vm_gen.php. It uses zend_vm_def.h and zend_vm_execute.skl as input and produces zend_vm_opcodes.h and zend_vm_execute.h. The first file is a list of opcode definitions. It is included from zend_compile.h. The second one is an executor code itself. It is included from zend_execute.c.

zend_vm_gen.php can produce different kind of executors. You can select a different opcode threading model using --with-vm-kind=CALL|SWITCH|GOTO|HYBRID. You can disable opcode specialization using --without-specializer. At last you can debug the executor using the original zend_vm_def.h or the generated zend_vm_execute.h file. Debugging with the original file requires the --with-lines option. By default, Zend Engine uses the following command to generate the executor:

# Default VM kind is HYBRID
php zend_vm_gen.php --with-vm-kind=HYBRID