New implementation of hashtables introduced a compaction step which is
triggered when a hashtable is full but it contains at least one deleted
bucket. Therefore there is a possibility that a cleverly crafted code can
trigger this compaction step (which takes time proportional to the size of
hashtabe) by executing constatnt number of operations. When the hashtable
is full, deletion and subsequent addition or single element triggers a
table compaction and these two steps can be repeated ad infinitum. This
might be avenue for a DOS attack.
This patch allows compaction to be performed only if the hashtable contains
at least 1/32 deleted elements, otherwise the hashtable is doubled in size.
Linear amount of work caused by compaction is amortized over multiple
malicious additions and deletions.
Removed HashTable->arHash (reduced memory consumption). Now hash slots may be accessed using HT_HASH() macro.
Hash slotas are allocated together with Buckets (before them) and lay in reverse order from HashTable->arData base address (see comments in Zend/zend_types.h)
Indexes in hash table and conflict resolution chains (Z_NEXT) may be stored as indeces or offsets in bytes, depending on system (32 or 64-bit).
HashTable data filelds are reordered to keep the most useful for zend_hash_find() data in the same CPU cache line.
Now each HashTable is also zend_array, so it's refcounted and may be a subject for Copy on Write
zend_array_dup() was changed to allocate and return HashTable, instead of taking preallocated HashTable as argument.
Squashed commit of the following:
commit 1e41295097
Author: Dmitry Stogov <dmitry@zend.com>
Date: Sat Jan 31 07:28:58 2015 +0300
Generalize HashTableIterator API to allows its usage without involvement of HashTable.nInternalPonter
commit 5406f21b11
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 18:08:43 2015 +0300
Reduced alghorithms complexity
commit b37f1d58d2
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 18:08:30 2015 +0300
Fixed test name
commit fb2d079645
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 18:08:05 2015 +0300
API cleanup
commit 08302c0d6d
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 14:20:46 2015 +0300
Make array_splice() to preserve foreach hash position
commit cc4b7be41e
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 12:24:31 2015 +0300
Make internal function, operation on array passed by reference, to preserve foreach hash position
commit 5aa9712b0a
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 09:49:35 2015 +0300
Implement consistent behavior for foreach by value over plain object
commit 4c5b385ff5
Author: Dmitry Stogov <dmitry@zend.com>
Date: Fri Jan 30 07:56:37 2015 +0300
More careful iterators update.
commit 721fc9e80d
Author: Dmitry Stogov <dmitry@zend.com>
Date: Thu Jan 29 21:43:28 2015 +0300
Added new test
commit 15a23b1218
Author: Dmitry Stogov <dmitry@zend.com>
Date: Thu Jan 29 21:05:02 2015 +0300
Reimplement iteration magic with HashTableIterators (see https://wiki.php.net/rfc/php7_foreach#implementation_details)
commit 10a3260b1f
Author: Dmitry Stogov <dmitry@zend.com>
Date: Thu Jan 29 21:04:44 2015 +0300
New test
commit eef80c5837
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 16:52:21 2015 +0300
Fixed foreach by reference iteration over constant array
commit 61e7391873
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 14:59:54 2015 +0300
Fixed temporary variable re-allocation pass
commit 92e90c09f0
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 12:44:57 2015 +0300
Fixed operand destruction in case of exceptions in iterator
commit dd2a36a207
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 10:02:34 2015 +0300
Use GET_OP1_ZVAL_PTR_DEREF() (IS_TMP_VAR and IS_CONST can't be IS_REFERENCE)
commit 4638f7b914
Author: Dmitry Stogov <dmitry@zend.com>
Date: Wed Jan 28 07:43:28 2015 +0300
Change "foreach" statement behavior (this is just a PoC yet)
- "foreach by value" don't relay on internal array/object pointer and doesnt perform array duplication. It just locks it incrementing reference counter. If the original array is modified by some code, the copy on write is performed and "foreach" still work with the old copy.
- it makes no difference if array given to "foreach by value" is reference itself
- "foreach by reference" still use internal array/object pointer and should work similar to PHP-5. (This id not completely implemented)
Make nTableMask to be 0 for packed arrays.
Remove checks fo HASH_FLAG_PACKED in zend_hash_find/zend_hash_del and family (string keys are resolved through uninitialized_bucket).
Change HashTable layout for better locality.
* Drop unnecessary checks in non-inlined part of numeric key
detection
* Drop zend_hash_apply_deleter in favor of zend_hash_del_el.
Also move block/unblock interruption macros in there.
nNumUsed should always be greater or equal to nNumOfElements so original condition is never true and arrays are always doubled in size and compaction is never triggered
most of the calls are with a size <= 8, so don't even bother any
bitshifts. For the bigger sizes, use an intrinsic (should be extended
with gcc ones though).
This implements the original functionality of php_splice, but
as an in-place operation, thus avoiding copying the HT.
This is much faster (~10x) if the splice removes a small portion
of the array and doesn't insert many elements.
The implementation differs from the original in array.c in that it
rehashes the hashtable in the same loop. This is approximately two
times faster (not counting the rare case of a purely associative
array).
The interruption handling was normalized to wrap the entire deletion
operation (instead of leaving out the destructor call and/or the
deallocation) per the recommendation of somebody (don't remember who)
familiar with zend signals.
There is an always-inlined variant of the function for use in
the main deletion function, to ensure there is no performance
impact.
Also remove lies in the comment above the function. apply_deleter
is not safe against modification, if the modification involves
removing the next bucket.
Also modified the implementation of zend_hash_graceful_destroy()
to be fully graceful (the reverse variant already was).
