tt_core Namespace Reference

Module caffe2.python.tt_core. More...

Functions
def	init_tt_cores (inp_sizes, out_sizes, tt_ranks, seed=1234)
def	matrix_to_tt (W, inp_sizes, out_sizes, tt_ranks)
def	tt_svd (W, sizes, tt_ranks)
def	fc_net_to_tt_net (net)

Detailed Description

Module caffe2.python.tt_core.

Function Documentation

init_tt_cores()

def tt_core.init_tt_cores	(		inp_sizes,
			out_sizes,
			tt_ranks,
			seed = 1234
	)

Initialize randomized orthogonalized TT-cores.

This method should be used when a TT-layer will trained from scratch. The
sizes of each of the cores are specified by the inp_sizes and out_sizes, and
the respective tt_ranks will dictate the ranks of each of the cores. Note
that a larger set of tt_ranks will result in slower computation but will
result in more accurate approximations. The size of the ith core is:

    tt_ranks[i] * inp_sizes[i] * out_sizes[i] * tt_ranks[i + 1].

Note that the following relationships of lengths of each input is expected:

    len(inp_sizes) == len(out_sizes) == len(tt_ranks) - 1.

Args:
    inp_sizes: list of the input dimensions of the respective cores
    out_sizes: list of the output dimensions of the respective cores
    tt_ranks: list of the ranks of the respective cores
    seed: integer to seed the random number generator

Returns:
    cores: One-dimensional list of cores concatentated along an axis

Definition at line 21 of file tt_core.py.

matrix_to_tt()

def tt_core.matrix_to_tt	(		W,
			inp_sizes,
			out_sizes,
			tt_ranks
	)

Convert a matrix into the TT-format.

This method will consume an a 2D weight matrix such as those used in fully
connected layers in a neural network and will compute the TT-decomposition
of the weight matrix and return the TT-cores of the resulting computation.
This method should be used when converting a trained, fully connected layer,
into a TT-layer for increased speed and decreased parameter size. The size
of the ith core is:

    tt_ranks[i] * inp_sizes[i] * out_sizes[i] * tt_ranks[i + 1].

Note that the following relationships of lengths of each input is expected:

    len(inp_sizes) == len(out_sizes) == len(tt_ranks) - 1.

We also require that np.prod(inp_sizes) == W.shape[0] and that
np.prod(out_sizes) == W.shape[1].

Args:
    W: two-dimensional weight matrix numpy array representing a fully
       connected layer to be converted to TT-format; note that the weight
       matrix is transposed before decomposed because we want to emulate the
       X * W^T operation that the FC layer performs.
    inp_sizes: list of the input dimensions of the respective cores
    out_sizes: list of the output dimensions of the respective cores
    tt_ranks: list of the ranks of the respective cores

Returns:
    new_cores: One-dimensional list of cores concatentated along an axis

Definition at line 100 of file tt_core.py.

tt_svd()

def tt_core.tt_svd	(		W,
			sizes,
			tt_ranks
	)

Helper method for the matrix_to_tt() method performing the TT-SVD
decomposition.

Uses the TT-decomposition algorithm to convert a matrix to TT-format using
multiple reduced SVD operations.

Args:
    W: two-dimensional weight matrix representing a fully connected layer to
       be converted to TT-format preprocessed by the matrix_to_tt() method.
    sizes: list of the dimensions of each of the cores
    tt_ranks: list of the ranks of the respective cores

Returns:
    cores: One-dimensional list of cores concatentated along an axis

Definition at line 193 of file tt_core.py.