Performs fuzz-style testing of MediaWiki's parser and forms. More...

Detailed Description

Performs fuzz-style testing of MediaWiki's parser and forms.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. http://www.gnu.org/copyleft/gpl.html

Author:: Nick Jenkins ( http://nickj.org/ ).

Started: 18 May 2006.

Description: Performs fuzz-style testing of MediaWiki's parser and forms.

How:

Generate lots of nasty wiki text.
Ask the Parser to render that wiki text to HTML, or ask MediaWiki's forms to deal with that wiki text.
Check MediaWiki's output for problems.
Repeat.

Why:

To help find bugs.
To help find security issues, or potential security issues.

What type of problems are being checked for:

Unclosed tags.
Errors or interesting warnings from Tidy.
PHP errors / warnings / notices.
MediaWiki internal errors.
Very slow responses.
No response from apache.
Optionally checking for malformed HTML using the W3C validator.

Background: Many of the wikiFuzz class methods are a modified PHP port, of a "shameless" Python port, of LCAMTUF'S MANGELME:

Video: There's an XviD video discussing this fuzz tester. You can get it from: http://files.nickj.org/MediaWiki/Fuzz-Testing-MediaWiki-xvid.avi

Requirements: To run this, you will need:

Command-line PHP5, with PHP-curl enabled (not all installations have this enabled - try "apt-get install php5-curl" if you're on Debian to install).
the Tidy standalone executable. ("apt-get install tidy").

Optional:

If you want to run the curl scripts, you'll need standalone curl installed ("apt-get install curl")
For viewing the W3C validator output on a command line, the "html2text" program may be useful ("apt-get install html2text")

Saving tests and test results: Any of the fuzz tests which find problems are saved for later review. In order to help track down problems, tests are saved in a number of different formats. The default filename extensions and their meanings are:

".test.php" : PHP script that reproduces just that one problem using PHP-Curl.
".curl.sh" : Shell script that reproduces that problem using standalone curl.
".data.bin" : The serialized PHP data so that this script can re-run the test.
".info.txt" : A human-readable text file with details of the field contents.

Wiki configuration for testing: You should make some additions to LocalSettings.php in order to catch the most errors. Note this configuration is for **TESTING PURPOSES ONLY**, and is IN NO WAY, SHAPE, OR FORM suitable for deployment on a hostile network. That said, personally I find these additions to be the most helpful for testing purposes:

--------- Start --------- Everyone can do everything. Very useful for testing, yet useless for deployment. $wgGroupPermissions['*']['autoconfirmed'] = true; $wgGroupPermissions['*']['block'] = true; $wgGroupPermissions['*']['bot'] = true; $wgGroupPermissions['*']['delete'] = true; $wgGroupPermissions['*']['deletedhistory'] = true; $wgGroupPermissions['*']['deleterevision'] = true; $wgGroupPermissions['*']['editinterface'] = true; $wgGroupPermissions['*']['hiderevision'] = true; $wgGroupPermissions['*']['import'] = true; $wgGroupPermissions['*']['importupload'] = true; $wgGroupPermissions['*']['minoredit'] = true; $wgGroupPermissions['*']['move'] = true; $wgGroupPermissions['*']['patrol'] = true; $wgGroupPermissions['*']['protect'] = true; $wgGroupPermissions['*']['proxyunbannable'] = true; $wgGroupPermissions['*']['renameuser'] = true; $wgGroupPermissions['*']['reupload'] = true; $wgGroupPermissions['*']['reupload-shared'] = true; $wgGroupPermissions['*']['rollback'] = true; $wgGroupPermissions['*']['siteadmin'] = true; $wgGroupPermissions['*']['unwatchedpages'] = true; $wgGroupPermissions['*']['upload'] = true; $wgGroupPermissions['*']['userrights'] = true; $wgGroupPermissions['*']['renameuser'] = true; $wgGroupPermissions['*']['makebot'] = true; $wgGroupPermissions['*']['makesysop'] = true;

Enable weird and wonderful options: Increase default error reporting level. error_reporting (E_ALL); // At a later date could be increased to E_ALL | E_STRICT $wgEnableUploads = true; // enable uploads. $wgDBerrorLog = "/root/mediawiki-db-error-log.txt"; // log DB errors, replace with suitable path. $wgShowSQLErrors = true; // Show SQL errors (instead of saying the query was hidden). $wgShowExceptionDetails = true; // want backtraces. $wgEnableAPI = true; // enable API. $wgEnableWriteAPI = true; // enable API.

Install & enable Parser Hook extensions to increase code coverage. E.g.: require_once "extensions/ParserFunctions/ParserFunctions.php"; require_once "extensions/Cite/Cite.php"; require_once "extensions/inputbox/inputbox.php"; require_once "extensions/Sort/Sort.php"; require_once "extensions/wikihiero/wikihiero.php"; require_once "extensions/CharInsert/CharInsert.php"; require_once "extensions/FixedImage/FixedImage.php";

Install & enable Special Page extensions to increase code coverage. E.g.: require_once "extensions/Cite/SpecialCite.php"; require_once "extensions/Renameuser/SpecialRenameuser.php"; --------- End ---------

If you want to try E_STRICT error logging, add this to the above: --------- Start --------- error_reporting (E_ALL | E_STRICT); set_error_handler( 'error_handler' ); function error_handler ($type, $message, $file=__FILE__, $line=__LINE__) { if ($message == "var: Deprecated. Please use the public/private/protected modifiers") return; print " \nStrict Standards: Type: $type: $message in $file on line $line \n"; } --------- End ---------

Also add/change this in LocalSettings.php: --------- Start --------- $wgEnableProfileInfo = true; $wgDBserver = "localhost"; // replace with DB server hostname --------- End ---------

Usage: Run with "php fuzz-tester.php". To see the various command-line options, run "php fuzz-tester.php --help". To stop the script, press Ctrl-C.

Console output:

If requested, first any previously failed tests will be rerun.
Then new tests will be generated and run. Any tests that fail will be saved, and a brief message about why they failed will be printed on the console.
The console will show the number of tests run, time run, number of tests failed, number of tests being done per minute, and the name of the current test.

TODO: Some known things that could improve this script:

Logging in with cookie jar storage needed for some tests (as there are some pages that cannot be tested without being logged in, and which are currently untested - e.g. Special:Emailuser, Special:Preferences, adding to Watchist).
Testing of Timeline extension (I cannot test as ploticus has/had issues on my architecture).

Definition in file fuzz-tester.php.