==================== SciPy Course Outline ==================== :author: Dave Kuhlman :address: dkuhlman@rexx.com http://www.rexx.com/~dkuhlman :revision: 1.0a :date: June 23, 2006 :copyright: Copyright (c) 2005 Dave Kuhlman. All Rights Reserved. This software is subject to the provisions of the MIT License http://www.opensource.org/licenses/mit-license.php. :abstract: This document provides an outline for a course on NumPy/SciPy. PyTables and Matplotlib are also discussed. .. sectnum:: .. contents:: What is SciPy? ============== SciPy is both (1) a way to handle large arrays of numerical data in Python and (2) a way to apply scientific, statistical, and mathematical operations to those arrays of data. When combined with a package such as PyTables, if is also capable of storing and retrieving large arrays of data in an efficient way. Since much of it's calculations are done in C extension modules, SciPy can be quite fast. Resources and Help ================== You can find help here: - `SciPy `_ - Plotting -- `Matplotlib `_ Installation ============ See the instructions at the SciPy Web site: - `Installing SciPy `_. - `SciPy Download `_. On UNIX/Linux ------------- Below, we look at two ways to install SciPy: - Method 1: Install from the distribution bundles. - Method 2: Install from SVN (Subversion: version control system). If you have installed previous versions, it is recommended that you remove the old versions from your Python ``site-packages`` directory first. Method 1 -- Use this version for the newest distribution. Note that the version numbers of the latest versions may have changed by the time you read this. The new SciPy is composed of two projects: (1) numpy and (2) scipy. 1. Download and install NumPy. A link to NumPy is available at: http://www.scipy.org/ After extracting it, use the following to build and install it:: $ cd numpy-?.?.? $ python setup.py build $ python setup.py install # as root 2. Download and install SciPy. SciPy is available at: http://new.scipy.org/Wiki/Download After extracting it, use the following to build and install it:: $ cd scipy-?.?.? $ python setup.py build $ python setup.py install # as root Method 2 -- You can also install from SVN: 1. Install NumPy:: $ svn co http://svn.scipy.org/svn/numpy/trunk numpy $ cd numpy $ rm -rf build $ python setup.py build $ sudo python setup.py install # or without sudo as root 2. Install SciPy:: $ svn co http://svn.scipy.org/svn/scipy/trunk scipy $ cd scipy $ rm -rf build $ python setup.py build $ sudo python setup.py install # or without sudo as root On MS Windows ------------- There are binary installers for MS Windows. You can find them at: http://www.scipy.org/Download. Help on SciPy ============= Suggestion: Use IPython for your interactive Python shell. IPython comes with a SciPy profile. You can run it with:: $ ipython -p scipy Doing so automatically loads SciPy. In IPython, get help on SciPy modules, classes, functions, and methods with the ``help`` built-in function. Or, with IPython, use the ``?`` operator and the ``pdoc`` magic command. Examples:: In [6]:help(scipy.io.read_array) ... In [7]:scipy.io.read_array? ... In [8]: %pdoc stats.norm.pdf Probability density function at x of the given RV. ... To see the contents of modules, use ``dir()``. Example:: In [24]:dir(scipy.io) Or, if you have loaded SciPy by doing ``ipython -p scipy``, you can do:: In [9]: dir(io) Arrays and Array Operations =========================== A good introduction to arrays is at `Numerical Python `_. In particular, in that document, see `5. Array Basics `_ and `8. Array Functions `_. Arrays are simple. An example:: $ ipython In [1]:import scipy In [2]:a1 = scipy.array([1, 2, 3, 4,]) In [3]:a2 = scipy.array([4, 3, 2, 1,]) In [4]:print a1 [1 2 3 4] In [5]:a3 = a1 * a2 In [6]:print a3 [4 6 6 4] o o o In [41]: a1 = scipy.zeros((4,5)) In [42]: print a1 [[0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0] [0 0 0 0 0]] In [43]: a2 = scipy.empty((4,5)) In [44]: print a2 [[-1209828888 -1209828888 14 3 24] [ 24 6 6 6 6] [ 6 6 6 139519736 64] [ 9 139519712 11 12 139519680]] In [45]: a3 = scipy.zeros((4,5), dtype='f') In [46]: print a3 [[ 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0.]] To index into multi-dimension arrays, use either of the following:: In [37]:a2 = zeros((4,3),dtype='f') In [38]:a2 Out[38]:NumPy array, format: long [[ 0. 0. 0.] [ 0. 0. 0.] [ 0. 0. 0.] [ 0. 0. 0.]] In [39]:a2[3,0] = 5. In [40]:a2[2][1] = 6. In [41]:a2 Out[41]:NumPy array, format: long [[ 0. 0. 0.] [ 0. 0. 0.] [ 0. 6. 0.] [ 5. 0. 0.]] But, indexing into a complex array seems a little counter intuitive:: In [31]: aa = zeros((5,4), dtype=complex64) In [32]: aa Out[32]: array([[ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j]], dtype=complex64) In [33]: aa.real[0,0] = 1.0 In [34]: aa.imag[0,0] = 2.0 In [35]: aa Out[35]: array([[ 1.+2.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j], [ 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j]], dtype=complex64) Note that we use this:: aa.real[0,0] = 1.0 aa.imag[0,0] = 2.0 and *not* this:: aa[0,0].real = 1.0 # wrong aa[0,0].imag = 2.0 # wrong Package ``base`` has array helper functions. Examples:: import scipy def test(): a1 = scipy.arange(5, 10) print a1 a2 = scipy.zeros((4,5), dtype='f') print a2 test() Prints the following:: [5 6 7 8 9] [[ 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0.]] For help, use something like the following:: help(scipy) help(scipy.zeros) Or in ``IPython``:: scipy? scipy.zeros? You can also "reshape" and transpose arrays:: In [47]: a1 = arange(12) In [48]: a1 Out[48]: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) In [49]: a2 = a1.reshape(3,4) In [50]: a2 Out[50]: array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]]) In [51]: a3 = a2.transpose() In [52]: a3 Out[52]: array([[ 0, 4, 8], [ 1, 5, 9], [ 2, 6, 10], [ 3, 7, 11]]) And, you can get the "shape" of an array:: In [53]: a1.shape Out[53]: (12,) In [54]: a2.shape Out[54]: (3, 4) In [55]: a3.shape Out[55]: (4, 3) You can "vectorize" a function. Doing so turns a function that takes a scalar as an argument into one when can process a vector. For example:: In [9]:def t(x): ....: return x + 3 In [10]:a1 = scipy.zeros((5,4)) In [11]:a1 Out[11]:NumPy array, format: long [[0 0 0 0] [0 0 0 0] [0 0 0 0] [0 0 0 0] [0 0 0 0]] In [12]:s = scipy.vectorize(t) In [13]:a2 = s(a1) In [14]:a2 Out[14]:NumPy array, format: long [[3 3 3 3] [3 3 3 3] [3 3 3 3] [3 3 3 3] [3 3 3 3]] The array interface and array protocol -------------------------------------- The array interface is a specification for a developer who wishes to implement a replacement for the implementation of arrays, e.g. those used in scipy. The array protocol is the way in which, for example, a scipy user uses arrays. It includes such things as: - The ability to select elements in an array, for example, with ``a1[3]`` or ``a1[3,4]``. - The ability to select slices of an array, for example, with ``a1[1:3]``. - The ability to convert arrays without copying (see `Converting arrays`_, below). - The iterator protocol -- The ability to iterate over items in an array. - Can respond to request for its length, for example ``len(my_array)``. You should be aware of the difference between (1) ``a1[3,4]`` and (2) ``a1[3][4]``. Both work. However, the second results in two calls to the ``__getitem__`` method. Converting arrays ----------------- At times you may need to convert an array from one type to another, for example from a numpy array to a scipy array or the reverse. The array protocol will help. In particular, the ``asarray()`` function can convert an array without copying. Examples:: In [8]: import numpy In [9]: import scipy In [10]: a1 = zeros((4,6)) In [11]: type(a1) Out[11]: In [12]: a2 = numpy.asarray(a1) In [13]: type(a2) Out[13]: In [14]: a3 = numpy.zeros((3,5)) In [15]: type(a3) Out[15]: In [16]: a4 = scipy.asarray(a3) In [17]: type(a4) Out[17]: Input and Output ================ SciPy Basic I/O --------------- SciPy has its own input/output capabilities. They are in module ``scipy.io``. Here is a simple example:: import scipy def test_io(): scipyArray1 = scipy.array([[1.0,2.0],[3.0,4.0],[5.0,6.0]]) outFile = file('tmpdata1.txt', 'w') scipy.io.write_array(outFile, scipyArray1) outFile.close() inFile = file('tmpdata1.txt', 'r') scipyArray2 = scipy.io.read_array(inFile) print 'type(scipyArray2):', type(scipyArray2) print 'scipyArray2:\n', scipyArray2 test_io() prints the following:: type(scipyArray2): scipyArray2: [[ 1. 2.] [ 3. 4.] [ 5. 6.]] ``io.write_array()`` and ``io.read_array()`` become very slow when applied to large arrays. PyTables scales much better. PyTables and HDF5 ----------------- PyTables writes and reads HDF5 files. It supports the ability to save and retrieve SciPy arrays into HDF5 files. Multiple arrays and separate data sets can be organized in nested groups (analogous to folders or directories). You can learn more about PyTables at `PyTables -- Hierarchical Datasets in Python `_. Installing PyTables ~~~~~~~~~~~~~~~~~~~ Obtain PyTables from `PyTables -- Hierarchical Datasets in Python `_. For MS Windows, there are binary executable installers. For Linux, install PyTables with something like the following (depending on the version):: $ tar xvzf orig/pytables-1.3.2.tar.gz $ cd pytables-1.3.2/ $ python setup.py build_ext --inplace $ sudo python setup.py install When installing from source, there are possible problems with Pyrex (possibly in combination with Python 2.4). If you try installing PyTables before these problems are fixed and get errors while building and installing, take a look at the fixes suggested in the following messages: - http://mail.python.org/pipermail/distutils-sig/2005-November/005300.html - http://lists.copyleft.no/pipermail/pyrex/2005-July/001256.html Using PyTables ~~~~~~~~~~~~~~ There is extensive documentation in the PyTables source distribution. See: ``pytables-?.?.?/doc/html/usersguide.html``. The source distribution also contains a number of examples. See: ``pytables-?.?.?/examples``. You can also find user documentation at the PyTables Web site. See `PyTables User's Guide `_: http://www.pytables.org/docs/manual/. Of particular interest are: - Chapter 3: Tutorials - Chapter 4: Library Reference From PyTables 1.3 on, PyTables supports NumPy (and hence SciPy) arrays right out of the box in Array objects. So, if you write a NumPy array, you will get a NumPy array back, and the same goes for Numeric and numarray arrays. In other objects (EArray, VLArray or Table) you can make use of the 'flavor' parameter in constructors to tell PyTables: "Hey, every time that I read from this object, please, return me an (rec)array with the appropriate flavor". Of course, PyTables will try hard to avoid doing data copies in conversions (i.e. the array protocol is used whenever possible). For versions of PyTables prior to 1.3, PyTables can save and read only numarray arrays. You can still use PyTables with SciPy, but for versions of PyTables prior to 1.3, an array conversion is needed. If you are using a recent version of SciPy and numarray, then you will be able to do this conversion *without* copying, using the array protocol. Converting a Scipy array to a numarray array:: numarray_array = numarray.asarray(scipy_array) And, converting a numarray array to a SciPy array:: scipy_array = scipy.asarray(numarray_array) If you insist on using older versions, a simple method is to convert a SciPy array to a Python list. For example:: In [17]:data1 = s.array([[1.0,2.0],[3.0,4.0],[5.0,6.0]]) In [18]:list1 = data1.to In [18]:list1 = data1.tolist() In [19]:print list1 [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]] However, conversion from numarray arrays to SciPy arrays is simple. This example:: import scipy import numarray def test(): scipyArray = scipy.array([[1.0,2.0],[3.0,4.0],[5.0,6.0]]) list1 = scipyArray.tolist() print 'list1:', list1 numarrayArray = numarray.array([[1.0,2.0],[3.0,4.0],[5.0,6.0]]) print 'numarrayArray:\n', numarrayArray scipyArray2 = scipy.array(numarrayArray) print 'type(scipyArray2):', type(scipyArray2) print 'scipyArray2:\n', scipyArray2 test() prints the following:: list1: [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]] numarrayArray: [[ 1. 2.] [ 3. 4.] [ 5. 6.]] type(scipyArray2): scipyArray2: [[ 1. 2.] [ 3. 4.] [ 5. 6.]] Here is an example that uses sufficiently recent versions of PyTables and SciPy to write and read arrays:: #!/usr/bin/env python import sys import getopt import scipy import tables Filename = 'testpytables2.h5' Dataset1 = [[1, 2, 3, 4],[5, 6, 7, 8],[9, 10, 11, 12]] Dataset2 = [[1.,2., 2.1],[3.,4.,4.1],[5.,6.,6.1]] def test1(): """Write out several sample data sets. """ filename = Filename print "Creating file:", filename #filter = tables.Filters() h5file = tables.openFile(filename, mode = "w", title = "PyTables test file", # filters=filter ) print '=' * 30 print h5file print '=' * 30 root = h5file.createGroup(h5file.root, "Datasets", "Test data sets") datasets = h5file.createGroup(root, "Phase1", "Test data sets") scipy_array = scipy.array(Dataset1) h5file.createArray(datasets, 'dataset1', scipy_array, "Test data set #1") scipy_array = scipy.array(Dataset2) h5file.createArray(datasets, 'dataset2', scipy_array, "Test data set #2") scipy_array = scipy.zeros((100,100)) h5file.createArray(datasets, 'dataset3', scipy_array, "Test data set #3") h5file.close() # # Read in and display the data sets. # def test2(): filename = Filename h5file = tables.openFile(filename, 'r') dataset1Obj = h5file.getNode('/Datasets/Phase1', 'dataset1') dataset2Obj = h5file.getNode('/Datasets/Phase1', 'dataset2') print repr(dataset1Obj) print repr(dataset2Obj) dataset1Array = dataset1Obj.read() dataset2Array = dataset2Obj.read() print 'type(dataset1Array):', type(dataset1Array) print 'type(dataset2Array):', type(dataset2Array) print 'array1:\n', dataset1Array print 'array2:\n', dataset2Array # print several slices of our array. print 'slice [0]:', dataset1Array[0] print 'slice [0:2]:',dataset1Array[0:2] print 'slice [1, 0:4:2]:',dataset1Array[1, 0:4:2] h5file.close() USAGE_TEXT = """ Usage: python test_pytables1.py [options] Options: -h, --help Display this help message. -t n, --test=n Test number: 1: Write file 2: Read file Example: python test_pytables1.py -t 1 python test_pytables1.py -t 2 """ def usage(): print USAGE_TEXT sys.exit(-1) def main(): args = sys.argv[1:] try: opts, args = getopt.getopt(args, 'ht:', ['help', 'test=', ]) except: usage() testno = 0 for opt, val in opts: if opt in ('-h', '--help'): usage() elif opt in ('-t', '--test'): testno = int(val) if len(args) != 0: usage() if testno == 1: test1() elif testno == 2: test2() else: usage() if __name__ == '__main__': main() Run the above by typing the following at the command line:: $ python test_pytables2.py -t 1 $ python test_pytables2.py -t 2 Notes: - We use ``h5file.createGroup()`` to create a group in the HDF5 file and then to create another group nested inside that one. A group is the equivalent of a folder or directory. PyTables supports nested groups in HDF5 files. - To write an array, we use ``h5file.createArray()``. - To retrieve an array, we use ``getNode()`` followed by ``node.read()``. - Notice, also, that we can read slices of an array directly from disk using the array subscription and slicing notation. See function ``test2``. - You may find both ``h5dump`` and ``h5ls`` (from ``hdf5-tools``) helpful for displaying the nested data structures:: $ h5dump -n testpytables2.h5 HDF5 "testpytables2.h5" { FILE_CONTENTS { group /Datasets group /Datasets/Phase1 dataset /Datasets/Phase1/dataset1 dataset /Datasets/Phase1/dataset2 dataset /Datasets/Phase1/dataset3 } } See `NCSA HDF5 Tools `_. - Other examples are provided in the PyTables distribution and in the PyTables tutorial. Plotting and Graphics ===================== We'll be learning the use of matplotlib. You can find matplotlib and information about it here: http://matplotlib.sourceforge.net/. There are `Examples (zip) `_. Most of the code that follows is based on those examples. Extensive documentation can be found at the matplotlib Web site, including: - A tutorial -- `Matplotlib / pylab - matlab style python plotting (plots, graphs, charts) `_ - A user guide (PDF) -- Go to http://matplotlib.sourceforge.net/ and click on "User's Guide". - Instructions on using matplotlib interactively -- `Using matplotlib interactively `_. - A FAQ -- `matplotlib FAQ `_. - A cookbook -- `matplotlib CookBook `_. Configuration ------------- See the user guide and also see the comments in the (default) ``matplotlibrc``. Some notes: - Configuration can be set in ``~/.matplotlib/matplotlibrc``. - During runtime, configuration options are acessible in the dictionary ``pylab.rcParams``. - If you have installed the new scipy/numpy combination, then you will want to change your configuration (in ``~/.matplotlib/matplotlibrc``) to use it:: numerix : numpy # Numeric or numarray - If you wish to use matplotlib interactively (for example in IPython, the following may be helpful:: backend : TkAgg # the default backend interactive : True # see http://matplotlib.sourceforge.net/interactive.html - With IPython, you may want to start up with the following:: ipython -pylab -p scipy This has the effect of doing:: from scipy import * from pylab import * and also sets some options that make showing, drawing, and updating graphs more automatic and convenient. A simple plot ------------- This code is based on ``simple_plot.py`` in `Examples (zip) `_. :: #!/usr/bin/env python """ dave_simple_plot.py """ import sys import pylab as pl # epydoc -- specify the input format. # __docformat__ = "restructuredtext en" def simple(funcName): """Create a simple plot and save the plot to a .png file @param funcName: The name of a function, e.g. sin, cos, tan, ... """ t = pl.arange(0.0, 1.0+0.01, 0.01) funcStr = 'pl.%s(2*2*pl.pi*t)' % (funcName,) s = eval(funcStr) pl.plot(t, s) pl.xlabel('time (s)') pl.ylabel('voltage (mV)') pl.title('About as simple as it gets, folks') pl.grid(True) pl.savefig('simple_plot') pl.show() def usage(): print 'Usage: python dave_simple_plot.py ' print 'Examples:' print ' python dave_simple_plot.py sin' print ' python dave_simple_plot.py cos' print ' python dave_simple_plot.py tan' sys.exit(-1) def main(): args = sys.argv[1:] if len(args) != 1: usage() simple(args[0]) if __name__ == '__main__': main() Notes: - ``pl.plot()`` creates a plot to be displayed. - ``pl.xlabel()``, ``pl.ylabel()``, and ``pl.title()`` add annotations to our plot. - ``pl.grid(True)`` adds a grid. - ``pl.savefig()`` saves the figure. The file name extension determines the type of file generated. On my machine, ``.png``, ``.svg``, and ``.ps`` produce Portable Network Graphics (PNG), Scalable Vector Graphics (SVG), and PostScript files, respectively. - ``pl.show()`` shows (makes visible) all the plots we have created. This is typically the last line of your script. Note that in the shell (for example, ipython) when interactive mode is set in your matplotlibrc, ``show()`` is not needed. Interactive use --------------- Configuration -- For interactive use, you can set the following in your ``matplotlibrc``: - backend: TkAgg - interactive: True Or, for interactive use, ``ipython`` is a recommended shell. It has a matplotlib mode. Start it with:: $ ipython -pylab Several notes on interactive mode and IPython: - (On my installation at least) setting ``interactive: True`` in matplotlabrc is *not* needed for IPython run with the ``-pylab`` flag. Apparently, IPython sets up interactive use for you. - The ``interactive: True`` flag in matplotlabrc also affects scripts run from the command line. On my system, those scripts do not exit. A sample session (after starting ``ipython -pylab``:: 1: t = arange(0.0, 1.0+0.01, 0.01) 2: s = tan(2 * 2 * pi * t) 3: plot(t,s) A sample session with ``ipython`` and *without* ``-pylab``: 1: import pylab as pl 2: t = pl.arange(0.0, 1.0+0.01, 0.01) 3: s = pl.tan(2 * 2 * pl.pi * t) 4: pl.plot(t,s) Using IPython -- Start IPython with one of the following:: $ ipython -pylab or:: $ ipython -pylab -p scipy Some convenient and useful commands while using matplotlib interactively: - ``clf()`` -- Clear the current figure. - ``cla()`` -- Clear the current axes. - ``draw()`` -- Redraw the current figure. - ``close()`` -- Close the current figure. ``close(num)`` -- Close figure number num. ``close(h)`` -- Close figure whose handle is h. ``close('all')`` -- Close all figure windows. Learn the following in order to create and control your plots: - ``figure(num)`` -- Create or activate figure number num. - ``subplot()`` -- Create a sub-plot within the current figure. - ``plot()`` Learn the following in order to annotate your plots: - ``xlabel(s)`` -- Add a label s to the x axis. - ``ylabel(s)`` -- Add a label s to the y axis. - ``title(s)`` -- Add a title s to the axes. - ``text(x, y, s)`` Add text s to the axes at x, y in data coords. - ``figtext(x, y, s)`` -- Add text to the figure at x, y in relative 0-1 figure coords. Note that these annotation functions return a matplotlib Text object. You can use this object to get and set properties of the text. Example:: 91: cla() 92: plot([1,2,3]) 93: t = xlabel('increasing temp') 94: t.set_weight('bold') 95: t.set_color('b') 96: draw() Notes: - The call to ``draw()`` may be needed in interactive mode (e.g. when in IPython) in order to update or refresh the drawing window. - Use ``dir()`` and ``help()`` (or "?" in IPython) to learn what methods are supported by the matplotlib Text object (or any other objects), what they do, what parameters they take, etc. Also, see the matplotlib user guide. Embedding matplotlib in a GUI application ----------------------------------------- You can display your matplotlib plots inside a GUI application written in Tk, WxPython, ... There are examples in the ``examples`` directory of the matplotlib source distribution. These examples are also available at the matplotlib Web site. Go to `matplotlib `_, then click on "Examples (zip)". The example files for embedding are named:: embedding_in_gtk.py embedding_in_tk.py embedding_in_wx3.py embedding_in_gtk2.py embedding_in_tk2.py embedding_in_wx4.py embedding_in_gtk3.py embedding_in_wx.py embedding_in_qt.py embedding_in_wx2.py In general, you can create your plot, possibly testing it interactively in IPython, then use one of the examples for embedding the plot in the GUI tool of your choice. Mathematical, Statistic, and Scientific Capabilities ==================================================== This section lists and gives brief descriptions of the contents of scipy. Much of the following documentation was generated from within IPython. I used either (1) the ``help(obj)`` built-in or (2) IPython's ``?`` operator to view documentation on a module, class, etc, and then, where necessary, used the "s" command from within ``less`` (my pager) to save to a file. I've also done some light editing to reformat this for reST (reStructuredText), which is the format for the source of this document. For more on reST see: `Docutils: Documentation Utilities `_. scipy (top-level) ----------------- SciPy: A scientific computing package for Python Available subpackages: - __core_config__ - __scipy_config__ - __svn_version__ - _import_tools - core_version - fft, fft2, fftn -- discrete Fourier transform. Also see fftshift, fftfreq. - `fftpack`_ (package) - `integrate`_ (package) - `interpolate`_ (package) - `io`_ (package) - `lib`_ (package) - `linalg`_ (package) - old__init__ - `optimize`_ (package) - scipy_version - `sparse`_ (package) - `special`_ (package) - `stats`_ (package) - `test`_ (package) - `utils`_ (package) base ---------- ``scipy`` provides functions for defining a multi-dimensional array and useful procedures for Numerical computation. Use the following to get a list of the members of ``scipy``:: >>> import scipy >>> dir(scipy) Functions: - array - NumPy Array construction - zeros - Return an array of all zeros - empty - Return an unitialized array - shape - Return shape of sequence or array - rank - Return number of dimensions - size - Return number of elements in entire array or a certain dimension - fromstring - Construct array from (byte) string - take - Select sub-arrays using sequence of indices - put - Set sub-arrays using sequence of 1-D indices - putmask - Set portion of arrays using a mask - reshape - Return array with new shape - repeat - Repeat elements of array - choose - Construct new array from indexed array tuple - cross_correlate - Correlate two 1-d arrays - searchsorted - Search for element in 1-d array - sum - Total sum over a specified dimension - average - Average, possibly weighted, over axis or array. - cumsum - Cumulative sum over a specified dimension - product - Total product over a specified dimension - cumproduct - Cumulative product over a specified dimension - alltrue - Logical and over an entire axis - sometrue - Logical or over an entire axis - allclose - Tests if sequences are essentially equal More Functions: - arrayrange (arange) - Return regularly spaced array - asarray - Guarantee NumPy array - sarray - Guarantee a NumPy array that keeps precision - convolve - Convolve two 1-d arrays - swapaxes - Exchange axes - concatenate - Join arrays together - transpose - Permute axes - sort - Sort elements of array - argsort - Indices of sorted array - argmax - Index of largest value - argmin - Index of smallest value - innerproduct - Innerproduct of two arrays - dot - Dot product (matrix multiplication) - outerproduct - Outerproduct of two arrays - resize - Return array with arbitrary new shape - indices - Tuple of indices - fromfunction - Construct array from universal function - diagonal - Return diagonal array - trace - Trace of array - dump - Dump array to file object (pickle) - dumps - Return pickled string representing data - load - Return array stored in file object - loads - Return array from pickled string - ravel - Return array as 1-D - nonzero - Indices of nonzero elements for 1-D array - shape - Shape of array - where - Construct array from binary result - compress - Elements of array where condition is true - clip - Clip array between two values - ones - Array of all ones - identity - 2-D identity array (matrix) (Universal) Math Functions: - absolute - add - arccos - arccosh - arcsin - arcsinh - arctan - arctan2 - arctanh - around - bitwise_and - bitwise_or - bitwise_xor - ceil - conjugate - cos - cosh - divide - divide_safe - equal - exp - fabs - floor - fmod - greater - greater_equal - hypot - invert - left_shift - less - less_equal - log - log10 - logical_and - logical_not - logical_or - logical_xor - maximum - minimum - multiply - negative - not_equal - power - right_shift - sign - sin - sinh - sqrt - subtract - tan - tanh Basic functions used by several sub-packages and useful to have in the main name-space Type handling: - iscomplexobj -- Test for complex object, scalar result - isrealobj -- Test for real object, scalar result - iscomplex -- Test for complex elements, array result - isreal -- Test for real elements, array result - imag -- Imaginary part - real -- Real part - real_if_close -- Turns complex number with tiny imaginary part to real - isneginf -- Tests for negative infinity - isposinf -- Tests for positive infinity - isnan -- Tests for nans - isinf -- Tests for infinity - isfinite -- Tests for finite numbers - isscalar -- True if argument is a scalar - nan_to_num -- Replaces NaN's with 0 and infinities with large numbers - cast -- Dictionary of functions to force cast to each type - common_type -- Determine the 'minimum common type code' for a group of arrays - mintypecode -- Return minimal allowed common typecode. Index tricks: - mgrid -- Method which allows easy construction of N-d 'mesh-grids' - r\_ -- Append and construct arrays: turns slice objects into ranges and concatenates them, for 2d arrays appends rows. - index_exp -- Konrad Hinsen's index_expression class instance which can be useful for building complicated slicing syntax. Useful functions: - select -- Extension of where to multiple conditions and choices - extract -- Extract 1d array from flattened array according to mask - insert -- Insert 1d array of values into Nd array according to mask - linspace -- Evenly spaced samples in linear space - logspace -- Evenly spaced samples in logarithmic space - fix -- Round x to nearest integer towards zero - mod -- Modulo mod(x,y) = x % y except keeps sign of y - amax -- Array maximum along axis - amin -- Array minimum along axis - ptp -- Array max-min along axis - cumsum -- Cumulative sum along axis - prod -- Product of elements along axis - cumprod -- Cumluative product along axis - diff -- Discrete differences along axis - angle -- Returns angle of complex argument - unwrap -- Unwrap phase along given axis (1-d algorithm) - sort_complex -- Sort a complex-array (based on real, then imaginary) - trim_zeros -- trim the leading and trailing zeros from 1D array. - vectorize -- a class that wraps a Python function taking scalar arguments into a generalized function which can handle arrays of arguments using the broadcast rules of numerix Python. - alter_numeric -- enhance numeric array behavior - restore_numeric -- restore alterations done by alter_numeric Shape manipulation: - squeeze -- Return a with length-one dimensions removed. - atleast_1d -- Force arrays to be > 1D - atleast_2d -- Force arrays to be > 2D - atleast_3d -- Force arrays to be > 3D - vstack -- Stack arrays vertically (row on row) - hstack -- Stack arrays horizontally (column on column) - column_stack -- Stack 1D arrays as columns into 2D array - dstack -- Stack arrays depthwise (along third dimension) - split -- Divide array into a list of sub-arrays - hsplit -- Split into columns - vsplit -- Split into rows - dsplit -- Split along third dimension Matrix (2d array) manipluations: - fliplr -- 2D array with columns flipped - flipud -- 2D array with rows flipped - rot90 -- Rotate a 2D array a multiple of 90 degrees - eye -- Return a 2D array with ones down a given diagonal - diag -- Construct a 2D array from a vector, or return a given diagonal from a 2D array. - mat -- Construct a Matrix - bmat -- Build a Matrix from blocks Polynomials: - poly1d -- A one-dimensional polynomial class - poly -- Return polynomial coefficients from roots - roots -- Find roots of polynomial given coefficients - polyint -- Integrate polynomial - polyder -- Differentiate polynomial - polyadd -- Add polynomials - polysub -- Substract polynomials - polymul -- Multiply polynomials - polydiv -- Divide polynomials - polyval -- Evaluate polynomial at given argument Import tricks: - ppimport -- Postpone module import until trying to use it - ppimport_attr -- Postpone module import until trying to use its attribute - ppresolve -- Import postponed module and return it. Machine arithmetics: - machar_single -- MachAr instance storing the parameters of system single precision floating point arithmetics - machar_double -- MachAr instance storing the parameters of system double precision floating point arithmetics Threading tricks: - ParallelExec -- Execute commands in parallel thread. fftpack ------- Discrete Fourier Transform algorithms. Fast Fourier Transforms: - fft --- FFT of arbitrary type periodic sequences - ifft --- Inverse of fft - fftn --- Multi-dimensional FFT - ifftn --- Inverse of fftn - fft2 --- Two-dimensional FFT - ifft2 --- Inverse of fft2 - rfft --- FFT of real periodic sequences - irfft --- Inverse of rfft Differential and pseudo-differential operators: - diff --- Differentiation and integration of periodic sequences - tilbert --- Tilbert transform: cs_diff(x,h,h) - itilbert --- Inverse Tilbert transform: sc_diff(x,h,h) - hilbert --- Hilbert transform: cs_diff(x,inf,inf) - ihilbert --- Inverse Hilbert transform: sc_diff(x,inf,inf) - cs_diff --- cosh/sinh pseudo-derivative of periodic sequences - sc_diff --- sinh/cosh pseudo-derivative of periodic sequences - ss_diff --- sinh/sinh pseudo-derivative of periodic sequences - cc_diff --- cosh/cosh pseudo-derivative of periodic sequences - shift --- Shift periodic sequences Helper functions: - fftshift --- Shift zero-frequency component to center of spectrum - ifftshift --- Inverse of freqshift - dftfreq --- DFT sample frequencies - rfftfreq --- DFT sample frequencies (specific to rfft,irfft) Extension modules: - _fftpack --- Provides functions zfft, drfft, zrfft, zfftnd, destroy_*_cache - convolve --- Provides functions convolve, convolve_z, init_convolution_kernel, destroy_convolve_cache integrate --------- Integration routines. Methods for Integrating Functions given function object: - quad -- General purpose integration. - dblquad -- General purpose double integration. - tplquad -- General purpose triple integration. - fixed_quad -- Integrate func(x) using Gaussian quadrature of order n. - quadrature -- Integrate with given tolerance using Gaussian quadrature. - romberg -- Integrate func using Romberg integration. Methods for Integrating Functions given fixed samples. - trapz -- Use trapezoidal rule to compute integral from samples. - cumtrapz -- Use trapezoidal rule to cumulatively compute integral. - simps -- Use Simpson's rule to compute integral from samples. - romb -- Use Romberg Integration to compute integral from (2**k + 1) evenly-spaced samples. See the special module's orthogonal polynomials (special) for Gaussian quadrature roots and weights for other weighting factors and regions. Interface to numerical integrators of ODE systems: - odeint -- General integration of ordinary differential equations. - ode -- Integrate ODE using vode routine. interpolate ----------- Interpolation Tools. Wrappers around FITPACK functions: - splrep -- find smoothing spline given (x,y) points on curve. - splprep -- find smoothing spline given parametrically defined curve. - splev -- evaluate the spline or its derivatives. - splint -- compute definite integral of a spline. - sproot -- find the roots of a cubic spline. - spalde -- compute all derivatives of a spline at given points. - bisplrep -- find bivariate smoothing spline representation. - bisplev -- evaluate bivariate smoothing spline. Interpolation class: - interp1d -- Create a class whose instances can linearly interpolate to compute unknown values. io -- Data input and output. Classes: - fopen -- a class for easily reading and writing binary data. Functions: - read_array -- reading ascii streams into Numeric arrays - write_array -- write an array to an ascii stream - loadmat -- read a MATLAB style mat file (version 4 and 5) - savemat -- write a MATLAB (version <= 4) style mat file - fread -- low-level reading - fwrite -- low-level writing - bswap -- in-place byte-swapping - packbits -- Pack a binary array of 1's and 0's into an array of bytes - unpackbits -- Unpack an array packed by packbits. - save --- simple storing of Python dictionary into module that can then be imported and the data accessed as attributes of the module. - mminfo -- query matrix info from Matrix Market formatted file - mmread -- read matrix from Matrix Market formatted file - mmwrite -- write matrix to Matrix Market formatted file lib --- - lapack (package) linalg ------ Linear algebra routines. Linear Algebra Basics: - inv --- Find the inverse of a square matrix - solve --- Solve a linear system of equations - solve_banded --- Solve a linear system of equations with a banded matrix - det --- Find the determinant of a square matrix - norm --- matrix and vector norm - lstsq --- Solve linear least-squares problem - pinv --- Pseudo-inverse (Moore-Penrose) using lstsq - pinv2 --- Pseudo-inverse using svd Eigenvalues and Decompositions: - eig --- Find the eigenvalues and vectors of a square matrix - eigvals --- Find the eigenvalues of a square matrix - lu --- LU decomposition of a matrix - lu_factor --- LU decomposition returning unordered matrix and pivots - lu_solve --- solve Ax=b using back substitution with output of lu_factor - svd --- Singular value decomposition of a matrix - svdvals --- Singular values of a matrix - diagsvd --- construct matrix of singular values from output of svd - orth --- construct orthonormal basis for range of A using svd - cholesky --- Cholesky decomposition of a matrix - cho_factor --- Cholesky decomposition for use in solving linear system - cho_solve --- Solve previously factored linear system - qr --- QR decomposition of a matrix - schur --- Schur decomposition of a matrix - rsf2csf --- Real to complex schur form - hessenberg --- Hessenberg form of a matrix matrix Functions: - expm --- matrix exponential using Pade approx. - expm2 --- matrix exponential using Eigenvalue decomp. - expm3 --- matrix exponential using Taylor-series expansion - logm --- matrix logarithm - cosm --- matrix cosine - sinm --- matrix sine - tanm --- matrix tangent - coshm --- matrix hyperbolic cosine - sinhm --- matrix hyperbolic sine - tanhm --- matrix hyperbolic tangent - signm --- matrix sign - sqrtm --- matrix square root - funm --- Evaluating an arbitrary matrix function. Iterative linear systems solutions - cg --- Conjugate gradient (symmetric systems only) - cgs --- Conjugate gradient squared - qmr --- Quasi-minimal residual - gmres --- Generalized minimal residual - bicg --- Bi-conjugate gradient - bicgstab --- Bi-conjugate gradient stabilized Package contents: - _flinalg - _iterative - atlas_version - basic - blas - calc_lwork - cblas - clapack - decomp - fblas - flapack - flinalg - info - interface_gen - iterative - lapack - linalg_version - matfuncs - setup - setup_atlas_version optimize -------- Optimization Tools -- A collection of general-purpose optimization routines. - fmin -- Nelder-Mead Simplex algorithm (uses only function calls) - fmin_powell -- Powell's (modified) level set method (uses only function calls) - fmin_cg -- Non-linear (Polak-Rubiere) conjugate gradient algorithm (can use function and gradient). - fmin_bfgs -- Quasi-Newton method (can use function and gradient) - fmin_ncg -- Line-search Newton Conjugate Gradient (can use function, gradient and hessian). - leastsq -- Minimize the sum of squares of M equations in N unknowns given a starting estimate. Constrained Optimizers (multivariate): - fmin_l_bfgs_b -- Zhu, Byrd, and Nocedal's L-BFGS-B constrained optimizer (if you use this please quote their papers -- see help) - fmin_tnc -- Truncated Newton Code originally written by Stephen Nash and adapted to C by Jean-Sebastien Roy. - fmin_cobyla -- Contrained Optimization BY Linear Approximation Global Optimizers - anneal -- Simulated Annealing - brute -- Brute Force searching Optimizer Scalar function minimizers - fminbound -- Bounded minimization of a scalar function. - brent -- 1-D function minimization using Brent method. - golden -- 1-D function minimization using Golden Section method - bracket -- Bracket a minimum (given two starting points) Also a collection of general_purpose root-finding routines. - fsolve -- Non-linear multi-variable equation solver. Scalar function solvers - brentq -- quadratic interpolation Brent method - brenth -- Brent method (modified by Harris with hyperbolic extrapolation) - ridder -- Ridder's method - bisect -- Bisection method - newton -- Secant method or Newton's method - fixed_point -- Single-variable fixed-point solver. Utility Functions - line_search -- Return a step that satisfies the strong Wolfe conditions. - check_grad -- Check the supplied derivative using finite difference techniques. Package contents: - _cobyla - _lbfgsb - _minpack - _zeros - anneal - cobyla - common_routines - info - lbfgsb - linesearch - minpack - minpack2 - moduleTNC - optimize - setup - tnc - zeros sparse ------ Sparse matrix support. Package contents: - sparse - sparsetools Functions: arange(...) arange(start, stop=None, step=1, dtype=intp) Just like range() except it returns an array whose type can be specified by the keyword argument typecode. array(...) array(object, dtype=None, copy=1, fortran=0, subok=0) will return a new array formed from the given object type given. Object can anything with an __array__ method, or any object exposing the array interface, or any (nested) sequence. If no type is given, then the type will be determined as the minimum type required to hold the objects in the sequence. If copy is zero and sequence is already an array with the right type, a reference will be returned. If the sequence is an array, type can be used only to upcast the array. For downcasting use .astype(t) method. If subok is true, then subclasses of the array may be returned. Otherwise, a base-class ndarray is returned arrayrange = arange(...) arange(start, stop=None, step=1, dtype=intp) Just like range() except it returns an array whose type can be specified by the keyword argument typecode. bincount(...) bincount(...) can_cast(...) can_cast_safely(from=d1, to=d2) returns True if data type d1 can be cast to data type d2 without losing precision. concatenate(...) concatenate((a1,a2,...),axis=None). digitize(...) digitize(...) dot(...) dot(a,v) returns matrix-multiplication between a and b. The product-sum is over the last dimension of a and the second-to-last dimension of b. dump(...) dump(obj, file, protocol=0) -- Write an object in pickle format to the given file. See the Pickler docstring for the meaning of optional argument proto. dumps(...) dumps(obj, protocol=0) -- Return a string containing an object in pickle format. See the Pickler docstring for the meaning of optional argument proto. empty(...) empty((d1,...,dn),dtype=intp,fortran=0) will return a new array of shape (d1,...,dn) and given type with all its entries uninitialized. This can be faster than zeros. fastCopyAndTranspose = _fastCopyAndTranspose(...) _fastCopyAndTranspose(a) frombuffer(...) frombuffer(buffer=, dtype=intp, count=-1, swap=0) Returns a 1-d array of data type dtype from buffer. The buffer argument must be an object that exposes the buffer interface. If count is -1 then the entire buffer is used, otherwise, count is the size of the output. If the buffer has data that is out not in machine byte-order, than set swap=1. The data will not be byteswapped, but the array will manage it in future operations. fromfile(...) fromfile(file=, dtype=intp, count=-1, sep='') Return an array of the given data type from a (text or binary) file. The file argument can be an open file or a string with the name of a file to read from. If count==-1, then the entire file is read, otherwise count is the number of items of the given type read in. If sep is '' then read a binary file, otherwise it gives the separator between elements in a text file. **WARNING:** This function should be used sparingly, as it is not a robust method of persistence. But it can be useful to read in simply-formatted or binary data quickly. frompyfunc(...) frompyfunc(func, nin, nout) take an arbitrary python function that takes nin objects as input and returns nout objects and return a universal function (ufunc). This ufunc always returns PyObject arrays fromstring(...) fromstring(string, dtype=intp, count=-1, swap=False) returns a new 1d array initialized from the raw binary data in string. If count is positive, the new array will have count elements, otherwise it's size is determined by the size of string. inner(...) inner(a,b) returns the dot product of two arrays, which has shape a.shape[:-1] + b.shape[:-1] with elements computed by the product of the elements from the last dimensions of a and b. innerproduct = inner(...) inner(a,b) returns the dot product of two arrays, which has shape a.shape[:-1] + b.shape[:-1] with elements computed by the product of the elements from the last dimensions of a and b. loads(...) loads(string) -- Load a pickle from the given string matrixmultiply = dot(...) dot(a,v) returns matrix-multiplication between a and b. The product-sum is over the last dimension of a and the second-to-last dimension of b. register_dtype(...) register_dtype(a) registers a new type object -- gives it a typenum set_numeric_ops(...) set_numeric_ops(op=func, ...) sets some or all of the number methods for all array objects. Don't forget ``**dict`` can be used as the argument list. Returns the functions that were replaced -- can be stored and set later. set_string_function(...) set_string_function(f, repr=1) sets the python function f to be the function used to obtain a pretty printable string version of a array whenever a array is printed. f(M) should expect a array argument M, and should return a string consisting of the desired representation of M for printing. where(...) where(condition, | x, y) is shaped like condition and has elements of x and y where condition is respectively true or false. If x or y are not given, then it is equivalent to nonzero(condition). zeros(...) zeros((d1,...,dn),dtype=intp,fortran=0) will return a new array of shape (d1,...,dn) and type typecode with all it's entries initialized to zero. special ------- Special Functions. Airy Functions: - airy -- Airy functions and their derivatives. - airye -- Exponentially scaled Airy functions - ai_zeros -- \*\*Zeros of Airy functions Ai(x) and Ai'(x) - bi_zeros -- \*\*Zeros of Airy functions Bi(x) and Bi'(x) Elliptic Functions and Integrals: - ellipj -- Jacobian elliptic functions - ellipk -- Complete elliptic integral of the first kind. - ellipkinc -- Incomplete elliptic integral of the first kind. - ellipe -- Complete elliptic integral of the second kind. - ellipeinc -- Incomplete elliptic integral of the second kind. Bessel Functions: - jn -- Bessel function of integer order and real argument. - jv -- Bessel function of real-valued order and complex argument. - jve -- Exponentially scaled Bessel function. - yn -- Bessel function of second kind (integer order). - yv -- Bessel function of the second kind (real-valued order). - yve -- Exponentially scaled Bessel function of the second kind. - kn -- Modified Bessel function of the third kind (integer order). - kv -- Modified Bessel function of the third kind (real order). - kve -- Exponentially scaled modified Bessel function of the third kind. - iv -- Modified Bessel function. - ive -- Exponentially scaled modified Bessel function. - hankel1 -- Hankel function of the first kind. - hankel1e -- Exponentially scaled Hankel function of the first kind. - hankel2 -- Hankel function of the second kind. - hankel2e -- Exponentially scaled Hankel function of the second kind. - lmbda -- \*\*Sequence of lambda functions with arbitrary order v. Zeros of Bessel Functions: - jnjnp_zeros -- \*\*Zeros of integer-order Bessel functions and derivatives sorted in order. - jnyn_zeros -- \*\*Zeros of integer-order Bessel functions and derivatives as separate arrays. - jn_zeros -- \*\*Zeros of Jn(x) - jnp_zeros -- \*\*Zeros of Jn'(x) - yn_zeros -- \*\*Zeros of Yn(x) - ynp_zeros -- \*\*Zeros of Yn'(x) - y0_zeros -- \*\*Complex zeros: Y0(z0)=0 and values of Y0'(z0) - y1_zeros -- \*\*Complex zeros: Y1(z1)=0 and values of Y1'(z1) - y1p_zeros -- \*\*Complex zeros of Y1'(z1')=0 and values of Y1(z1') Faster versions of common Bessel Functions: - j0 -- Bessel function of order 0. - j1 -- Bessel function of order 1. - y0 -- Bessel function of second kind of order 0. - y1 -- Bessel function of second kind of order 1. - i0 -- Modified Bessel function of order 0. - i0e -- Exponentially scaled modified Bessel function of order 0. - i1 -- Modified Bessel function of order 1. - i1e -- Exponentially scaled modified Bessel function of order 1. - k0 -- Modified Bessel function of the third kind of order 0. - k0e -- Exponentially scaled modified Bessel function of the third kind of order 0. - k1 -- Modified Bessel function of the third kind of order 1. - k1e -- Exponentially scaled modified Bessel function of the third kind of order 1. Integrals of Bessel Functions: - itj0y0 -- Basic integrals of j0 and y0 from 0 to x. - it2j0y0 -- Integrals of (1-j0(t))/t from 0 to x and y0(t)/t from x to inf. - iti0k0 -- Basic integrals of i0 and k0 from 0 to x. - it2i0k0 -- Integrals of (i0(t)-1)/t from 0 to x and k0(t)/t from x to inf. - besselpoly -- Integral of a bessel function: Jv(2*a*x) * x**lambda from x=0 to 1. Derivatives of Bessel Functions: - jvp -- Nth derivative of Jv(v,z) - yvp -- Nth derivative of Yv(v,z) - kvp -- Nth derivative of Kv(v,z) - ivp -- Nth derivative of Iv(v,z) - h1vp -- Nth derivative of H1v(v,z) - h2vp -- Nth derivative of H2v(v,z) Spherical Bessel Functions: - sph_jn -- \*\*Sequence of spherical Bessel functions, jn(z) - sph_yn -- \*\*Sequence of spherical Bessel functions, yn(z) - sph_jnyn -- \*\*Sequence of spherical Bessel functions, jn(z) and yn(z) - sph_in -- \*\*Sequence of spherical Bessel functions, in(z) - sph_kn -- \*\*Sequence of spherical Bessel functions, kn(z) - sph_inkn -- \*\*Sequence of spherical Bessel functions, in(z) and kn(z) Ricatti-Bessel Functions: - riccati_jn -- \*\*Sequence of Ricatti-Bessel functions of first kind. - riccati_yn -- \*\*Sequence of Ricatti-Bessel functions of second kind. Struve Functions: - struve -- Struve function --- Hv(x) - modstruve -- Modified struve function --- Lv(x) - itstruve0 -- Integral of H0(t) from 0 to x - it2struve0 -- Integral of H0(t)/t from x to Inf. - itmodstruve0 -- Integral of L0(t) from 0 to x. Raw Statistical Functions (Friendly versions in scipy.stats): - pdf. - bdtrc -- Sum of terms k+1 through n of the binomial pdf. - bdtri -- Inverse of bdtr - btdtr -- Integral from 0 to x of beta pdf. - btdtri -- Quantiles of beta distribution - fdtr -- Integral from 0 to x of F pdf. - fdtrc -- Integral from x to infinity under F pdf. - fdtri -- Inverse of fdtrc - gdtr -- Integral from 0 to x of gamma pdf. - gdtrc -- Integral from x to infinity under gamma pdf. - gdtri -- Quantiles of gamma distribution - nbdtr -- Sum of terms 0 through k of the negative binomial pdf. - nbdtrc -- Sum of terms k+1 to infinity under negative binomial pdf. - nbdtri -- Inverse of nbdtr - pdtr -- Sum of terms 0 through k of the Poisson pdf. - pdtrc -- Sum of terms k+1 to infinity of the Poisson pdf. - pdtri -- Inverse of pdtr - stdtr -- Integral from -infinity to t of the Student-t pdf. - stdtri -- Inverse of stdtr (quantiles) - chdtr -- Integral from 0 to x of the Chi-square pdf. - chdtrc -- Integral from x to infnity of Chi-square pdf. - chdtri -- Inverse of chdtrc. - ndtr -- Integral from -infinity to x of standard normal pdf - ndtri -- Inverse of ndtr (quantiles) - smirnov -- Kolmogorov-Smirnov complementary CDF for one-sided test statistic (Dn+ or Dn-) - smirnovi -- Inverse of smirnov. - kolmogorov -- The complementary CDF of the (scaled) two-sided test statistic (Kn*) valid for large n. - kolmogi -- Inverse of kolmogorov - tklmbda -- Tukey-Lambda CDF Gamma and Related Functions: - gamma -- Gamma function. - gammaln -- Log of the absolute value of the gamma function. - gammainc -- Incomplete gamma integral. - gammaincinv -- Inverse of gammainc. - gammaincc -- Complemented incomplete gamma integral. - gammainccinv -- Inverse of gammaincc. - beta -- Beta function. - betaln -- Log of the absolute value of the beta function. - betainc -- Incomplete beta integral. - betaincinv -- Inverse of betainc. - betaincinva -- Inverse (in first argument, a) of betainc - betaincinvb -- Inverse (in first argument, b) of betainc - psi(digamma) -- Logarithmic derivative of the gamma function. - rgamma -- One divided by the gamma function. - polygamma -- Nth derivative of psi function. Error Function and Fresnel Integrals: - erf -- Error function. - erfc -- Complemented error function (1- erf(x)) - erfinv -- Inverse of error function - erfcinv -- Inverse of erfc - erf_zeros -- \*\*Complex zeros of erf(z) - fresnel -- Fresnel sine and cosine integrals. - fresnel_zeros -- Complex zeros of both Fresnel integrals - fresnelc_zeros -- \*\*Complex zeros of fresnel cosine integrals - fresnels_zeros -- \*\*Complex zeros of fresnel sine integrals - modfresnelp -- Modified Fresnel integrals F\_+(x) and K\_+(x) - modfresnelm -- Modified Fresnel integrals F\_-(x) and K\_-(x) Legendre Functions: - lpn -- \*\*Legendre Functions (polynomials) of the first kind - lqn -- \*\*Legendre Functions of the second kind. - lpmn -- \*\*Associated Legendre Function of the first kind. - lqmn -- \*\*Associated Legendre Function of the second kind. - lpmv -- Associated Legendre Function of arbitrary non-negative degree v. - sph_harm -- Spherical Harmonics (complex-valued) Y^m_n(theta,phi) Orthogonal polynomials --- 15 types \*\* These functions all return a polynomial class which can then be evaluated: vals = chebyt(n)(x). This class also has an attribute 'weights' which return the roots, weights, and total weights for the appropriate form of Gaussian quadrature. These are returned in an n x 3 array with roots in the first column, weights in the second column, and total weights in the final column. - legendre -- \*\*Legendre polynomial P_n(x) (lpn -- for function). - chebyt -- \*\*Chebyshev polynomial T_n(x) - chebyu -- \*\*Chebyshev polynomial U_n(x) - chebyc -- \*\*Chebyshev polynomial C_n(x) - chebys -- \*\*Chebyshev polynomial S_n(x) - jacobi -- \*\*Jacobi polynomial P^(alpha,beta)_n(x) - laguerre -- \*\*Laguerre polynomial, L_n(x) - genlaguerre -- \*\*Generalized (Associated) Laguerre polynomial, L^alpha_n(x) - hermite -- \*\*Hermite polynomial H_n(x) - hermitenorm -- \*\*Normalized Hermite polynomial, He_n(x) - gegenbauer -- \*\*Gegenbauer (Ultraspherical) polynomials, C^(alpha)_n(x) - sh_legendre -- \*\*shifted Legendre polynomial, P*_n(x) - sh_chebyt -- \*\*shifted Chebyshev polynomial, T*_n(x) - sh_chebyu -- \*\*shifted Chebyshev polynomial, U*_n(x) - sh_jacobi -- \*\*shifted Jacobi polynomial, J*_n(x) = G^(p,q)_n(x) HyperGeometric Functions: - hyp2f1 -- Gauss hypergeometric function (2F1) - hyp1f1 -- Confluent hypergeometric function (1F1) - hyperu -- Confluent hypergeometric function (U) - hyp0f1 -- Confluent hypergeometric limit function (0F1) - hyp2f0 -- Hypergeometric function (2F0) - hyp1f2 -- Hypergeometric function (1F2) - hyp3f0 -- Hypergeometric function (3F0) Parabolic Cylinder Functions: - pbdv -- Parabolic cylinder function Dv(x) and derivative. - pbvv -- Parabolic cylinder function Vv(x) and derivative. - pbwa -- Parabolic cylinder function W(a,x) and derivative. - pbdv_seq -- \*\*Sequence of parabolic cylinder functions Dv(x) - pbvv_seq -- \*\*Sequence of parabolic cylinder functions Vv(x) - pbdn_seq -- \*\*Sequence of parabolic cylinder functions Dn(z), complex z mathieu and Related Functions (and derivatives): - mathieu_a -- Characteristic values for even solution (ce_m) - mathieu_b -- Characteristic values for odd solution (se_m) - mathieu_even_coef -- \*\*sequence of expansion coefficients for even solution - mathieu_odd_coef -- \*\*sequence of expansion coefficients for odd solution \*\* All the following return both function and first derivative \*\* - mathieu_cem -- Even mathieu function - mathieu_sem -- Odd mathieu function - mathieu_modcem1 -- Even modified mathieu function of the first kind - mathieu_modcem2 -- Even modified mathieu function of the second kind - mathieu_modsem1 -- Odd modified mathieu function of the first kind - mathieu_modsem2 -- Odd modified mathieu function of the second kind Spheroidal Wave Functions: - pro_ang1 -- Prolate spheroidal angular function of the first kind - pro_rad1 -- Prolate spheroidal radial function of the first kind - pro_rad2 -- Prolate spheroidal radial function of the second kind - obl_ang1 -- Oblate spheroidal angluar function of the first kind - obl_rad1 -- Oblate spheroidal radial function of the first kind - obl_rad2 -- Oblate spheroidal radial function of the second kind - pro_cv -- Compute characteristic value for prolate functions - obl_cv -- Compute characteristic value for oblate functions - pro_cv_seq -- Compute sequence of prolate characteristic values - obl_cv_seq -- Compute sequence of oblate characteristic values \*\* The following functions require pre-computed characteristic values \*\* - pro_ang1_cv -- Prolate spheroidal angular function of the first kind - pro_rad1_cv -- Prolate spheroidal radial function of the first kind - pro_rad2_cv -- Prolate spheroidal radial function of the second kind - obl_ang1_cv -- Oblate spheroidal angluar function of the first kind - obl_rad1_cv -- Oblate spheroidal radial function of the first kind - obl_rad2_cv -- Oblate spheroidal radial function of the second kind Kelvin Functions: - kelvin -- All Kelvin functions (order 0) and derivatives. - kelvin_zeros -- \*\*Zeros of All Kelvin functions (order 0) and derivatives - ber -- Kelvin function ber x - bei -- Kelvin function bei x - berp -- Derivative of Kelvin function ber x - beip -- Derivative of Kelvin function bei x - ker -- Kelvin function ker x - kei -- Kelvin function kei x - kerp -- Derivative of Kelvin function ker x - keip -- Derivative of Kelvin function kei x - ber_zeros -- \*\*Zeros of Kelvin function bei x - bei_zeros -- \*\*Zeros of Kelvin function ber x - berp_zeros -- \*\*Zeros of derivative of Kelvin function ber x - beip_zeros -- \*\*Zeros of derivative of Kelvin function bei x - ker_zeros -- \*\*Zeros of Kelvin function kei x - kei_zeros -- \*\*Zeros of Kelvin function ker x - kerp_zeros -- \*\*Zeros of derivative of Kelvin function ker x - keip_zeros -- \*\*Zeros of derivative of Kelvin function kei x Other Special Functions: - expn -- Exponential integral. - exp1 -- Exponential integral of order 1 (for complex argument) - expi -- Another exponential integral -- Ei(x) - wofz -- Fadeeva function. - dawsn -- Dawson's integral. - shichi -- Hyperbolic sine and cosine integrals. - sici -- Integral of the sinc and "cosinc" functions. - spence -- Dilogarithm integral. - zeta -- Riemann zeta function of two arguments. - zetac -- 1.0 - standard Riemann zeta function. Convenience Functions: - cbrt -- Cube root. - exp10 -- 10 raised to the x power. - exp2 -- 2 raised to the x power. - radian -- radian angle given degrees, minutes, and seconds. - cosdg -- cosine of the angle given in degrees. - sindg -- sine of the angle given in degrees. - tandg -- tangent of the angle given in degrees. - cotdg -- cotangent of the angle given in degrees. - log1p -- log(1+x) - expm1 -- exp(x)-1 - cosm1 -- cos(x)-1 - round -- round the argument to the nearest integer. If argument ends in 0.5 exactly, pick the nearest even integer. \*\* in the description indicates a function which is not a universal function and does not follow broadcasting and automatic array-looping rules. Error handling: Errors are handled by returning nans, or other appropriate values. Some of the special function routines will print an error message when an error occurs. By default this printing is disabled. To enable such messages use errprint(1) To disable such messages use errprint(0). Example:: >>> print scipy.special.bdtr(-1,10,0.3) >>> scipy.special.errprint(1) >>> print scipy.special.bdtr(-1,10,0.3) stats ----- Statistical functions. This module contains a large number of probability distributions as well as a growing library of statistical functions. Each included distribution is an instance of the class rv_continous. For each given name the following methods are available. See docstring for rv_continuous for more information. - rvs -- random variates with the distribution - pdf -- probability density function - cdf -- cummulative distribution function - sf -- survival function (1.0 - cdf) - ppf -- percent-point function (inverse of cdf) - isf -- inverse survival function - stats -- mean, variance, and optionally skew and kurtosis Calling the instance as a function returns a frozen pdf whose shape, location, and scale parameters are fixed. For example, to generate a single normally distributed random variable, use something like the following:: $ ipython o o o In [1]: from scipy import stats In [2]: stats.norm.rvs(size=10, loc=5.0) Out[2]: array([ 4.45700017, 4.39348877, 5.82171326, 3.05493492, 4.77358828, 4.86479922, 5.42006364, 2.59309408, 4.01344497, 6.1543075 ]) o o o In [4]: stats.norm.rvs(size=10, loc=5.0, scale=1) Out[4]: array([ 4.04022461, 3.76628997, 3.49915895, 4.38231034, 4.53075502, 3.37048989, 4.39382196, 3.65657395, 5.79550509, 4.57862224]) In [5]: stats.norm.rvs(size=10, loc=5.0, scale=2) Out[5]: array([ 4.60439161, 3.21791066, 4.1434995 , 2.70335034, 8.23381385, 7.85801707, 5.07002064, 4.66661538, 2.97583978, 5.77055363]) In [6]: stats.norm.rvs(size=10, loc=5.0, scale=5) Out[6]: array([ 4.50706583, 5.62037197, 5.04515902, 2.6058127 , 1.84169023, 15.28502793, 0.87783722, 6.73873743, 12.52279616, 7.53976885]) o o o The distributions available with the above methods are: Continuous (Total == 81 distributions): - norm -- Normal (Gaussian) - alpha -- Alpha - anglit -- Anglit - arcsine -- Arcsine - beta -- Beta - betaprime -- Beta Prime - bradford -- Bradford - burr -- Burr - fisk -- Fisk - cauchy -- Cauchy - chi -- Chi - chi2 -- Chi-squared - cosine -- Cosine - dgamma -- Double Gamma - dweibull -- Double Weibull - erlang -- Erlang - expon -- Exponential - exponweib -- Exponentiated Weibull - exponpow -- Exponential Power - fatiguelife -- Fatigue Life (Birnbaum-Sanders) - foldcauchy -- Folded Cauchy - f -- F (Snecdor F) - foldnorm -- Folded Normal - frechet_r -- Frechet Right Sided, Extreme Value Type II (Extreme LB) or weibull_min - frechet_l -- Frechet Left Sided, Weibull_max - genlogistic -- Generalized Logistic - genpareto -- Generalized Pareto - genexpon -- Generalized Exponential - genextreme -- Generalized Extreme Value - gausshyper -- Gauss Hypergeometric - gamma -- Gamma - gengamma -- Generalized gamma - genhalflogistic -- Generalized Half Logistic - gompertz -- Gompertz (Truncated Gumbel) - gumbel_r -- Right Sided Gumbel, Log-Weibull, Fisher-Tippett, Extreme Value Type I - gumbel_l -- Left Sided Gumbel, etc. - halfcauchy -- Half Cauchy - halflogistic -- Half Logistic - halfnorm -- Half Normal - hypsecant -- Hyperbolic Secant - invgamma -- Inverse Gamma - invnorm -- Inverse Normal - invweibull -- Inverse Weibull - johnsonsb -- Johnson SB - johnsonsu -- Johnson SU - laplace -- Laplace - logistic -- Logistic - loggamma -- Log-Gamma - loglaplace -- Log-Laplace (Log Double Exponential) - lognorm -- Log-Normal - gilbrat -- Gilbrat - lomax -- Lomax (Pareto of the second kind) - maxwell -- Maxwell - mielke -- Mielke's Beta-Kappa - nakagami -- Nakagami - ncx2 -- Non-central chi-squared - ncf -- Non-central F - t -- Student's T - nct -- Non-central Student's T - pareto -- Pareto - powerlaw -- Power-function - powerlognorm -- Power log normal - powernorm -- Power normal - rdist -- R distribution - reciprocal -- Reciprocal - rayleigh -- Rayleigh - rice -- Rice - recipinvgauss -- Reciprocal Inverse Gaussian - semicircular -- Semicircular - triang -- Triangular - truncexpon -- Truncated Exponential - truncnorm -- Truncated Normal - tukeylambda -- Tukey-Lambda - uniform -- Uniform - von_mises -- Von-Mises (Circular) - wald -- Wald - weibull_min -- Minimum Weibull (see Frechet) - weibull_max -- Maximum Weibull (see Frechet) - wrapcauchy -- Wrapped Cauchy - ksone -- Kolmogorov-Smirnov one-sided (no stats) - kstwobign -- Kolmogorov-Smirnov two-sided test for Large N (no stats) Discrete (Total == 10 distributions): - binom -- Binomial - bernoulli -- Bernoulli - nbinom -- Negative Binomial - geom -- Geometric - hypergeom -- Hypergeometric - logser -- Logarithmic (Log-Series, Series) - poisson -- Poisson - planck -- Planck (Discrete Exponential) - boltzmann -- Boltzmann (Truncated Discrete Exponential) - randint -- Discrete Uniform - zipf -- Zipf - dlaplace -- Discrete Laplacian Statistical Functions (adapted from Gary Strangman): - gmean - hmean - mean - cmedian - median - mode - tmean - tvar - tmin - tmax - tstd - tsem - moment - variation - skew - kurtosis - describe - skewtest - kurtosistest - normaltest - itemfreq - scoreatpercentile - percentileofscore - histogram2 - histogram - cumfreq - relfreq - obrientransform - samplevar - samplestd - signaltonoise - bayesmvs - var - std - stderr - sem - z - zs - zmap - threshold - trimboth - trim1 - cov - corrcoef - foneway - paired - pearsonr - spearmanr - pointbiserialr - kendalltau - linregress - ttest1samp - ttestind - ttestrel - kstest - chisquare - ks2samp - meanwhitneyu - tiecorrect - ranksums - wilcoxon - kruskal - friedmanchisquare - ansari - bartlett - levene - shapiro - anderson - binomtest - fligner - mood - oneway - glm - anova Doc strings are available for many of the above functions. For example, from within IPython, use the ``pdoc`` "magic command":: $ ipython -p scipy In [1]: %pdoc scipy.stats.gmean Calculates the geometric mean of the values in the passed array. That is: n-th root of (x1 * x2 * ... * xn). If a is 1D, a single value is returned. If a is multi-dimensional, the geometric mean along the dimension specified is calculated. The returned array has one less dimension than a. dimension defaults to the last dimension of the array. This means that, for a two dimensional array, the default is to calculate the geometric mean of each row. In [2]: pdoc scipy.stats.anova o o o If you omit the ``-p scipy`` flag to ``ipython``, then you will need to do ``import scipy.stats``. As an additional example, we consider the gamma function. Let us first look at the documentation (and doc-string):: In [2]: ?stats.gamma o o o gamma.rvs(a,loc=0,scale=1) - random variates o o o Gamma distribution For ``a = integer``, this is the Erlang distribution, and for ``a=1`` it is the exponential distribution:: gamma.pdf(x,a) = x**(a-1)*exp(-x)/gamma(a) for x >= 0, a > 0. These last lines explain the meaning of the parameter ``a`` (mentioned in the line gamma.rvs). To generate multiple gamma distributed random variates, use:: In [6]: stats.gamma.rvs(2,size=10) Out[6]: array([ 2.12111063, 1.91618176, 0.86085755, 0.27087561, 0.21773439, 3.14291742, 1.58128949, 0.82045958, 4.64099272, 6.66068163]) Note that the first argument is the parameter ``a``, and the second is the ``size``. Reversing these two function arguments results in an error. Plot-tests: - probplot - ppccmax - ppccplot Once again, in IPython, you can obtain information about each of the above. For example, use:: In [1]: %pdoc stats.probplot Return (osm, osr){,(scale,loc,r)} where (osm, osr) are order statistic medians and ordered response data respectively so that plot(osm, osr) is a probability plot. If fit==1, then do a regression fit and compute the slope (scale), intercept (loc), and correlation coefficient (r), of the best straight line through the points. If fit==0, only (osm, osr) is returned. sparams is a tuple of shape parameter arguments for the distribution. utils ----- Package contents: - common -- Common functions. - helpmod -- Functions for obtaining help about scipy. - info -- A function for obtaining information about scipy objects etc. - limits -- A module containing machine limits for Float32 and Float64. - pilutil -- Functions for PIL images. - ppimport -- Postpone module import to future. test ---- test(self, level=1, verbosity=1) Run Scipy module test suite with level and verbosity.