UtfNormalTest2.php File Reference

Other tests for the unicode normalization module. More...

Go to the source code of this file.

Enumerations
enum	COMMENT
enum	SEPARATOR
Functions
	getRow ($f)
	my_assert ($file, $line, $code)
	normalize_form_c ($c)
	The following section will be used for testing different normalization methods.
	normalize_form_c_php ($c)
	This set of functions is only useful if youve added a param to the following functions to force pure PHP usage.
	normalize_form_d ($c)
	normalize_form_d_php ($c)
	normalize_form_kc ($c)
	normalize_form_kc_php ($c)
	normalize_form_kd ($c)
	normalize_form_kd_php ($c)
	unichr ($c)
	unistr ($c)
Variables
	$count = 0
	$f = fopen($file, "r")
if(PHP_SAPI!= 'cli')	$file = "NormalizationTest.txt"
	$lineNo = 0
	if ($f!==false)

---

Detailed Description

Other tests for the unicode normalization module.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. http://www.gnu.org/copyleft/gpl.html

Definition in file UtfNormalTest2.php.

---

Enumeration Type Documentation

enum COMMENT

Definition at line 33 of file UtfNormalTest2.php.

enum SEPARATOR

Definition at line 36 of file UtfNormalTest2.php.

---

Function Documentation

getRow

(

$

f

)

Definition at line 231 of file UtfNormalTest2.php.

References $f, $ret, array(), as, and unistr().

my_assert	(	$	file,
		$	line,
		$	code
	)

Definition at line 91 of file UtfNormalTest2.php.

References $lineNo, and global.

normalize_form_c

(

$

c

)

The following section will be used for testing different normalization methods.

• Pure PHP ~ no assertion errors ~ 6.25 minutes

• php_utfnormal.so or intl extension: both are wrappers around libicu so we list the version of libicu when making the comparison

• libicu Ubuntu 3.8.1-3ubuntu1.1 php 5.2.6-3ubuntu4.5 ~ 2200 assertion errors ~ 5 seconds ~ output: http://paste2.org/p/921566

• libicu Ubuntu 4.2.1-3 php 5.3.2-1ubuntu4.2 ~ 1384 assertion errors ~ 15 seconds ~ output: http://paste2.org/p/921435

• libicu Debian 4.4.1-5 php 5.3.2-1ubuntu4.2 ~ no assertion errors ~ 13 seconds

• Tests comparing pure PHP output with libicu output were added later and slow down the runtime.

Definition at line 69 of file UtfNormalTest2.php.

References UtfNormal\toNFC().

normalize_form_c_php

(

$

c

)

This set of functions is only useful if youve added a param to the following functions to force pure PHP usage.

I decided not to commit that code since might produce a slowdown in the UTF normalization code just for the sake of these tests. -- hexmode

Returns:

string

Definition at line 81 of file UtfNormalTest2.php.

References UtfNormal\toNFC().

normalize_form_d

(

$

c

)

Definition at line 70 of file UtfNormalTest2.php.

References UtfNormal\toNFD().

normalize_form_d_php

(

$

c

)

Definition at line 82 of file UtfNormalTest2.php.

References UtfNormal\toNFD().

normalize_form_kc

(

$

c

)

Definition at line 71 of file UtfNormalTest2.php.

References UtfNormal\toNFKC().

normalize_form_kc_php

(

$

c

)

Definition at line 83 of file UtfNormalTest2.php.

References UtfNormal\toNFKC().

normalize_form_kd

(

$

c

)

Definition at line 72 of file UtfNormalTest2.php.

References UtfNormal\toNFKD().

normalize_form_kd_php

(

$

c

)

Definition at line 84 of file UtfNormalTest2.php.

References UtfNormal\toNFKD().

unichr

(

$

c

)

Definition at line 210 of file UtfNormalTest2.php.

unistr

(

$

c

)

Definition at line 227 of file UtfNormalTest2.php.

Referenced by getRow().

---

Variable Documentation

$count = 0

Definition at line 96 of file UtfNormalTest2.php.

Referenced by ApiQueryORM\addResults(), Benchmarker\bench(), ImageBuilder\buildTable(), Profiler\calltreeCount(), CheckStorage\check(), UserDupes\checkDupes(), LanguageKsh\convertPlural(), SpecialWatchlist\countItems(), Sanitizer\decodeCharReferencesAndNormalize(), PopulateRevisionLength\doDBUpdates(), FixExtLinksProtocolRelative\doDBUpdates(), PopulateParentId\doDBUpdates(), DeleteArchivedFilesImplementation\doDelete(), DeleteArchivedRevisionsImplementation\doDelete(), DeleteOldRevisions\doDelete(), SpecialRecentChanges\doHeader(), PopulateRevisionSha1\doSha1LegacyUpdates(), PopulateRevisionSha1\doSha1Updates(), MovePageForm\doSubmit(), ApiQueryTags\doTag(), MysqlUpdater\doTemplatelinksUpdate(), HTMLCacheUpdate\doUpdate(), DatabasePostgres\estimateRowCount(), StorageTypeStats\execute(), CheckBadRedirects\execute(), ApiQueryLangLinks\execute(), ApiQueryDeletedrevs\execute(), ApiQueryExternalLinks\execute(), ApiQueryIWLinks\execute(), MigrateUserGroup\execute(), ApiQueryFilearchive\execute(), CleanupSpam\execute(), DeleteOrphanedRevisions\execute(), NukePage\execute(), RemoveUnusedAccounts\execute(), ApiQueryBlocks\execute(), ApiQueryContributions\execute(), ApiQueryLogEvents\execute(), ApiQueryPagePropNames\execute(), RefreshFileHeaders\execute(), RenameDbPrefix\execute(), ShowJobs\execute(), ApiQueryImageInfo\execute(), ApiQueryAllUsers\execute(), RunJobs\execute(), NukeNS\execute(), CopyFileBackend\execute(), UpdateCollation\execute(), ApiQueryRevisions\execute(), nextJobDB\executeReadyPeriodicTasks(), lessc\expandParentSelectors(), CdbWriter_PHP\finish(), MostcategoriesPage\formatResult(), MostinterwikisPage\formatResult(), CategoryPager\formatRow(), CreditsAction\getContributors(), InfoAction\getContributors(), User\getEditCount(), DeleteLogFormatter\getMessageParameters(), SpecialEditWatchlist\getNormalForm(), BacklinkCache\getNumLinks(), WantedPagesPage\getQueryInfo(), RandomImageGenerator\getRandomWordPairs(), DatabaseOracle\indexExists(), ImageBuilder\init(), RemoveUnusedAccounts\isInactiveAccount(), Fallback\mb_substr(), SpecialMergeHistory\merge(), Title\moveSubpages(), moveToExternal(), lessc\multiplySelectors(), SpecialSpecialpages\outputPageList(), CheckLanguageCLI\outputText(), CheckLanguageCLI\outputWiki(), PurgeChangedPages\pageableSortedRows(), CoreParserFunctions\pagesincategory(), PPFuzzTester\pickEntryPoint(), PurgeRedundantText(), ApiQueryContinueTestBase\query(), EnhancedChangesList\recentChangesBlockGroup(), DatabasePostgres\relationExists(), lessc_parser\removeComments(), DbTestPreviewer\report(), ApiQueryImages\run(), ApiQueryProtectedTitles\run(), ApiQueryIWBacklinks\run(), ApiQueryLangBacklinks\run(), ApiQueryWatchlistRaw\run(), ApiQueryExtLinksUsage\run(), ApiQueryCategories\run(), ApiQueryDuplicateFiles\run(), ApiQueryCategoryMembers\run(), ApiQueryAllCategories\run(), ApiQueryPagesWithProp\run(), ApiQueryWatchlist\run(), ApiQuerySearch\run(), ApiQueryAllPages\run(), ApiQueryQueryPage\run(), ApiQueryAllImages\run(), ApiQueryLinks\run(), ApiQueryAllLinks\run(), ApiQueryRandom\run(), ApiQueryRecentChanges\run(), ApiQueryRandom\runQuery(), UserrightsPage\showEditUserGroupsForm(), MovePageForm\showSubpages(), SevenZipStream\stream_read(), UploadSourceAdapter\stream_read(), ApiEditPageTest\testEditAppend(), ApiEditPageTest\testEditConflict(), ApiEditPageTest\testEditConflict_bug41990(), ApiEditPageTest\testEditConflict_redirect(), SiteTest\testSetAndRemovePath(), GenericArrayObjectTest\testUnset(), lessc_parser\throwError(), userOptions\USAGER(), Exif\validate(), and wfIncrStats().

$f = fopen($file, "r")

Definition at line 38 of file UtfNormalTest2.php.

Referenced by LinkCache\addLinkObj(), CheckSyntax\buildFileList(), MimeMagic\doGuessMimeType(), CheckSyntax\execute(), ForeignAPIRepo\fileExistsBatch(), findAuxFile(), JSParser\FunctionDefinition(), HtmlFormatterTest\getHtmlData(), BmpHandler\getImageSize(), MimeMagic\getMediaType(), getRow(), XCFHandler\getXCFMetaData(), JSParser\parse(), lessc\reduce(), FileRepoTest\testFileRepoConstructionWithRequiredOptions(), TextContentTest\testGetParserOutput(), IPTest\testInvalidIPs(), GlobalTest\testReadOnlySet(), IPTest\testValidIPs(), and wfBacktrace().

if (PHP_SAPI!= 'cli') $file = "NormalizationTest.txt"

Definition at line 30 of file UtfNormalTest2.php.

Referenced by ExtensionsTestSuite\__construct(), DumpDBZip2Output\__construct(), ImportStreamSource\__construct(), CheckSyntax\addDirectoryContent(), ImageBuilder\addMissingImage(), lessc\addParsedFile(), CheckSyntax\buildFileList(), GIFHandler\canAnimateThumbnail(), ImageHandler\canRender(), TiffHandler\canRender(), AutoLoaderTest\checkAutoLoadConf(), CheckSyntax\checkEvilToken(), CheckSyntax\checkFileWithCli(), CheckSyntax\checkFileWithParsekit(), CheckSyntax\checkForMistakes(), CheckSyntax\checkRegex(), Sqlite\checkSqlSyntax(), UploadBase\checkXMLEncodingMissmatch(), ApiDelete\deleteFile(), UploadFromUrlTest\deleteFile(), ApiTestCaseUpload\deleteFileByTitle(), UploadFromUrlTestSuite\deleteFiles(), MimeMagic\detectMimeType(), UploadBase\detectScript(), UploadBase\detectVirus(), PopulateImageSha1\doDBUpdates(), MimeMagic\doGuessMimeType(), TestFileOpPerformance\doPerfTest(), RevDel_FileList\doPostCommitUpdates(), MovePageForm\doSubmit(), DjVuImage\dump(), DjVuImage\dumpForm(), ResourceLoaderLESSFunctions\embed(), ResourceLoaderLESSFunctions\embeddable(), CSSMin\encodeImageAsDataURI(), ApiQueryStashImageInfo\execute(), CheckLess\execute(), CheckImages\execute(), CountMessages\execute(), MwSql\execute(), ApiQueryFilearchive\execute(), UpdateSpecialPages\execute(), UploadStashCleanup\execute(), PatchSql\execute(), EraseArchivedFile\execute(), RefreshFileHeaders\execute(), DeleteBatch\execute(), UpdateSearchIndex\execute(), MoveBatch\execute(), ApiImageRotate\execute(), RefreshImageMetadata\execute(), UpdateMediaWiki\execute(), ForeignAPIRepo\fileExistsBatch(), CoreParserFunctions\filepath(), UploadDumper\filterItem(), findAuxFile(), LocalRepo\findBySha1s(), findFiles(), lessc\findImport(), SvgHandler\formatMetadata(), ImageListPager\formatValue(), GenerateNormalizerData\generateArabic(), GenerateCollationData\generateFirstChars(), FindHooks\getBadHooksFromFile(), FindHooks\getBadHooksFromPath(), MWDebug\getCallerDescription(), ImageHandler\getDimensionsString(), getFileCommentFromSourceWiki(), WebInstaller_Document\getFileContents(), MWDebug\getFilesIncluded(), getFileUserFromSourceWiki(), MediaHandler\getGeneralLongDesc(), MediaHandler\getGeneralShortDesc(), FindHooks\getHooksFromFile(), FindHooks\getHooksFromPath(), UploadBase\getImageInfo(), SvgHandler\getImageSize(), DjVuImage\getInfo(), ApiQueryImageInfo\getInfo(), CSSMin\getLocalFileReferences(), MWExceptionHandler\getLogMessage(), ImageHandler\getLongDesc(), SvgHandler\getLongDesc(), MediaHandler\getLongDesc(), CSSMin\getMimeType(), DjVuImage\getMultiPageInfo(), CountMessages\getNumMessages(), DjVuImage\getPageInfo(), SpecialFilepath\getRedirect(), ExifBitmapHandler\getRotation(), ResourceLoaderFileModule\getScriptURLsForDebug(), ImageHandler\getShortDesc(), MediaHandler\getShortDesc(), Skin\getSkinNames(), ResourceLoaderFileModule\getStyleURLsForDebug(), ForeignAPIFile\getThumbnails(), MimeMagic\guessMimeType(), ImageBuilder\imageCallback(), BackupReader\importFromFile(), BackupReader\importFromStdin(), MWCryptRand\initialRandomState(), SvgHandler\isAnimatedImage(), CheckSyntax\isSuitableFile(), Title\isValidMoveTarget(), RecompressTracked\logToFile(), Linker\makeImageLink(), Linker\makeImageLink2(), Linker\makeMediaLinkFile(), Linker\makeThumbLink2(), Linker\makeThumbLinkObj(), Title\moveTo(), BitmapHandler\mustRender(), ForeignDBFile\newFromRow(), OldLocalFile\newFromRow(), ImageBuilder\oldimageCallback(), UploadDumper\outputItem(), SpecialUploadStash\outputLocallyScaledThumb(), SpecialUploadStash\outputRemoteScaledThumb(), SpecialUploadStash\outputThumbFromStash(), SpecialUploadStash\parseKey(), Linker\processResponsiveImages(), ResourcesTest\provideResourceFiles(), XMPTest\provideXMPParse(), PruneFileCache\prune_directory(), ForeignAPIFile\purgeThumbnails(), DjVuImage\readChunk(), CSSMin\remap(), CheckStorage\restoreText(), JpegHandler\rotate(), BitmapHandler\rotate(), ApiQueryDuplicateFiles\run(), ApiQueryAllImages\run(), SpecialUndelete\showFileConfirmationForm(), SpecialUploadStash\showUploads(), DjVuImage\skipChunk(), UploadBase\stashFile(), UploadStashTest\testBug29408(), BitmapScalingTest\testImageArea(), ExifRotationTest\testMetadata(), ExifRotationTest\testMetadataNoAutoRotate(), BitmapScalingTest\testNormaliseParams(), ExifRotationTest\testRotationRendering(), ExifRotationTest\testRotationRenderingNoAutoRotate(), BitmapScalingTest\testTooBigImage(), BitmapScalingTest\testTooBigMustRenderImage(), JpegMetadataExtractorTest\testUtf8Comment(), LocalFileTest\testWfLocalFile(), Title\validateFileMoveOperation(), XmlTypeCheck\validateFromFile(), wfBacktrace(), wfErrorLog(), wfExtractThumbParams(), and wfGetPrecompiledData().

$lineNo = 0

Definition at line 97 of file UtfNormalTest2.php.

Referenced by my_assert().

if($f!==false)

Definition at line 98 of file UtfNormalTest2.php.