clones/php-src
1
0
Fork 0
mirror of https://github.com/php/php-src.git synced 2024-10-01 06:46:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
ce9f52adcd
php-src/ext/standard/tests/strings/htmlentities20.phpt

200 lines
4.0 KiB
Plaintext
Raw Normal View History

- Completed rewrite of html.c. Except for determine_charset, almost nothing remains. - Fixed bug on determine_charset that was preventing correct detection in combination with internal mbstring encoding "none", "pass" or "auto". - Added profiles for entity encode/decode for HTMl 4.01, XHTML 1.0, XML 1.0 and HTML 5. Added the constants ENT_HTML401, ENT_XML1, ENT_XHTML and ENT_HTML5. - htmlentities()/htmlspecialchars(), when told not to double encode, verify the correctness of the existenting entities more thoroughly. It is checked whether the numerical entity represents a valid unicode code point (number is between 0 and 0x10FFFF). If using the flag ENT_DISALLOWED, it is also checked whether that numerical entity is valid in selected document. In HTML 4.01, all the numerical entities that represent a Unicode code point (< U+10FFFFFF) are valid, but that's not the case with other document types. If the entity is not valid, & is encoded to &amp;. For named entities, the check is also more thorough. While before the only check would be to determine if the entity was constituted by alphanumeric characters, now it is checked whether that entity is necessarily defined for the target document type. Otherwise, & is encoded to &amp;. - For html_entity_decode(), only valid numerical and named entities (as defined above for htmlentities()/htmlspecialchars() + !double_encode) are decoded. But there is in this case one additional check. Entities that represent non-SGML or otherwise invalid characters are not decoded. Note that, in HTML5, U+000D is a valid literal character, but the entity &#x0D is not valid and is therefore not decoded. - The hash tables lazily created for decoding in html_entity_decode() that were added recently were substituted by static hash tables. Instead of 1 hash table per encoding, there's only one hash table per document type defined in terms of unicode code points. This means that for charsets other than UTF-8 and ISO-8859-1, a conversion to unicode code points is necessary before decoding. - On the encoding side, the ad hoc ranges of entities of the translation tables, which mapped (in general) non-unicode code points to HTML entities were replaced by three-stage tables for HTML 4 and HTML 5. This mapping tables are defined only in terms of unicode code points, so a conversion is necessary for charsets other than UTF-8 and ISO-8859-1. Even so, the multi-stage table is much faster than the previous method, by a factor of 5; the conversion to unicode is a small penalty because it's just a simple table lookup. XML 1.0/htmlspecialchars() uses a simple table instead of a three-stage table. - Added the flag ENT_SUBSTITUTE, which makes htmlentities()/htmlspecialchars() replace the invalid multibyte sequences with U+FFFD (UTF-8) or &#FFFD; (other encodings). - Added the flag ENT_DISALLOWED. Implements FR #52860. Characters that cannot appear literally are replaced by U+FFFD (UTF-8) or &#FFFD; (otherwise). An alternative implementation would be to encode those characters into numerical entities, but that would only work in HTML 4.01 due to limitations on the values of numerical entities in other document types. See also the effects on htmlentities()/htmlspecialchars() with !double_encode above.
2010-10-24 15:01:02 +00:00
--TEST--
htmlentities() / htmlspecialchars() ENT_DISALLOWED
--FILE--
<?php
function codepoint_to_utf8($k) {
if ($k < 0x80) {
$retval = pack('C', $k);
} else if ($k < 0x800) {
$retval = pack('C2',
0xc0 | ($k >> 6),
0x80 | ($k & 0x3f));
} else if ($k < 0x10000) {
$retval = pack('C3',
0xe0 | ($k >> 12),
0x80 | (($k >> 6) & 0x3f),
0x80 | ($k & 0x3f));
} else {
$retval = pack('C4',
0xf0 | ($k >> 18),
0x80 | (($k >> 12) & 0x3f),
0x80 | (($k >> 6) & 0x3f),
0x80 | ($k & 0x3f));
}
return $retval;
}
$tests = array(
0x00, //C0
0x01,
0x09,
0x0A,
0x0B,
0x0C,
0x0D,
0x0E,
0x1F,
0x20, //allowed always
0x7F, //DEL
0x80, //C1
0x9F,
0xA0, //allowed always
0xD7FF, //surrogates
0xD800,
0xDFFF,
0xE000, //allowed always
0xFFFE, //nonchar
0xFFFF,
0xFDCF, //allowed always
0xFDD0, //nonchar
0xFDEF,
0xFDF0, //allowed always
0x2FFFE, //nonchar
0x2FFFF,
);
$tests2 = array_map('codepoint_to_utf8', $tests);
$subchr = codepoint_to_utf8(0xFFFD);
function test($flag) {
global $tests, $tests2;
$i = -1;
foreach ($tests2 as $test) {
$i++;
$a = htmlentities($test, $flag | ENT_DISALLOWED, "UTF-8");
$b = htmlspecialchars($test, $flag | ENT_DISALLOWED, "UTF-8");
if ($a == "" && $b == "") { echo sprintf("%05X", $tests[$i]), ": INVALID SEQUENCE\n"; continue; }
echo sprintf("%05X", $tests[$i]), ": ", bin2hex($a), " ", bin2hex($b), "\n";
}
}
echo "*** Testing HTML 4.01 ***\n";
test(ENT_HTML401);
echo "\n*** Testing XHTML 1.0 ***\n";
test(ENT_XHTML);
echo "\n*** Testing HTML 5 ***\n";
test(ENT_HTML5);
echo "\n*** Testing XML 1.0 ***\n";
test(ENT_XML1);
?>
--EXPECT--
*** Testing HTML 4.01 ***
00000: efbfbd efbfbd
00001: efbfbd efbfbd
00009: 09 09
0000A: 0a 0a
0000B: efbfbd efbfbd
0000C: efbfbd efbfbd
0000D: 0d 0d
0000E: efbfbd efbfbd
0001F: efbfbd efbfbd
00020: 20 20
0007F: efbfbd efbfbd
00080: efbfbd efbfbd
0009F: efbfbd efbfbd
000A0: 266e6273703b c2a0
0D7FF: ed9fbf ed9fbf
0D800: INVALID SEQUENCE
0DFFF: INVALID SEQUENCE
0E000: ee8080 ee8080
0FFFE: efbfbe efbfbe
0FFFF: efbfbf efbfbf
0FDCF: efb78f efb78f
0FDD0: efb790 efb790
0FDEF: efb7af efb7af
0FDF0: efb7b0 efb7b0
2FFFE: f0afbfbe f0afbfbe
2FFFF: f0afbfbf f0afbfbf
*** Testing XHTML 1.0 ***
00000: efbfbd efbfbd
00001: efbfbd efbfbd
00009: 09 09
0000A: 0a 0a
0000B: efbfbd efbfbd
0000C: efbfbd efbfbd
0000D: 0d 0d
0000E: efbfbd efbfbd
0001F: efbfbd efbfbd
00020: 20 20
0007F: 7f 7f
00080: c280 c280
0009F: c29f c29f
000A0: 266e6273703b c2a0
0D7FF: ed9fbf ed9fbf
0D800: INVALID SEQUENCE
0DFFF: INVALID SEQUENCE
0E000: ee8080 ee8080
0FFFE: efbfbd efbfbd
0FFFF: efbfbd efbfbd
0FDCF: efb78f efb78f
0FDD0: efb790 efb790
0FDEF: efb7af efb7af
0FDF0: efb7b0 efb7b0
2FFFE: f0afbfbe f0afbfbe
2FFFF: f0afbfbf f0afbfbf
*** Testing HTML 5 ***
00000: efbfbd efbfbd
00001: efbfbd efbfbd
00009: 265461623b 09
0000A: 264e65774c696e653b 0a
0000B: efbfbd efbfbd
0000C: 0c 0c
0000D: 0d 0d
0000E: efbfbd efbfbd
0001F: efbfbd efbfbd
00020: 20 20
0007F: efbfbd efbfbd
00080: efbfbd efbfbd
0009F: efbfbd efbfbd
000A0: 266e6273703b c2a0
0D7FF: ed9fbf ed9fbf
0D800: INVALID SEQUENCE
0DFFF: INVALID SEQUENCE
0E000: ee8080 ee8080
0FFFE: efbfbd efbfbd
0FFFF: efbfbd efbfbd
0FDCF: efb78f efb78f
0FDD0: efbfbd efbfbd
0FDEF: efbfbd efbfbd
0FDF0: efb7b0 efb7b0
2FFFE: efbfbd efbfbd
2FFFF: efbfbd efbfbd
*** Testing XML 1.0 ***
00000: efbfbd efbfbd
00001: efbfbd efbfbd
00009: 09 09
0000A: 0a 0a
0000B: efbfbd efbfbd
0000C: efbfbd efbfbd
0000D: 0d 0d
0000E: efbfbd efbfbd
0001F: efbfbd efbfbd
00020: 20 20
0007F: 7f 7f
00080: c280 c280
0009F: c29f c29f
000A0: c2a0 c2a0
0D7FF: ed9fbf ed9fbf
0D800: INVALID SEQUENCE
0DFFF: INVALID SEQUENCE
0E000: ee8080 ee8080
0FFFE: efbfbd efbfbd
0FFFF: efbfbd efbfbd
0FDCF: efb78f efb78f
0FDD0: efb790 efb790
0FDEF: efb7af efb7af
0FDF0: efb7b0 efb7b0
2FFFE: f0afbfbe f0afbfbe
2FFFF: f0afbfbf f0afbfbf
Reference in New Issue Copy Permalink