OpFromGraph¶
This page describes theano.OpFromGraph
, an Op that allows to
encapsulate a Theano graph in an op.
This can be used to encapsulate some functionality in one block. It is useful to scale Theano compilation for regular bigger graphs when we reuse that encapsulated fonctionality with different inputs many times. Due to this encapsulation, it can make Theano compilation phase faster for graphs with many nodes.
Using this for small graphs is not recommended as it disables optimizations between what is inside the encapsulation and outside of it.
-
class
theano.compile.builders.
OpFromGraph
(inputs, outputs, inline=False, grad_overrides='default', rop_overrides='default', name=None, **kwargs)[source]¶ This creates an
Op
from inputs and outputs lists of variables. The signature is similar totheano.function
and the resultingOp
‘s perform will do the same operation as:orig_function(inputs, outputs, **kwargs)
Currently does not support
updates
orgivens
argument.Parameters: - inputs (list of
Variable
) – - outputs (list of
Variable
) – - inline (bool, optional) –
Defaults to
False
True
: Cause the Op’s original graph being used during compilation, the Op will not be visible in the compiled graph but rather its internal graph.False
: will use a pre-compiled function inside. - grad_overrides (single or list of {'default', OpFromGraph, callable, Variable with special type}, optional) –
Defaults to
'default'
.'default'
: Do not override, use default grad() resultOpFromGraph instance : Override with another OpFromGraph, should accept inputs as the same order and types of “inputs” and “output_grads” arguments as one would specify in grad() method.
callable : similar to OpFromGraph instance, must return list of
Variable
.- Variable :
NullType() instance
: Treat as non-differentiableDisconnectedType() instance
: Treat as disconnected gradient, numerically gives zero
list: Each OpFromGraph/callable must return a single
Variable
. Each list element corresponds to gradient of a specific input, length of list must be equal to number of inputs. - rop_overrides (single or list of {'default', OpFromGraph, callable, Variable with special type}, optional) –
Defaults to
default
.'default'
: Do not override, use default R_op() resultOpFromGraph instance : Override with another OpFromGraph, should accept inputs as the same order and types of “inputs” and “output_grads” arguments as one would specify in grad() method.
callable : similar to OpFromGraph instance, must return list of
Variable
.- Variable :
NullType() instance
: Treat as non-differentiableDisconnectedType() instance
: Treat as zero since DisconnectedType is not yet supported in R_op
list: Each OpFromGraph/callable must return a single
Variable
. Each list element corresponds to a specific output of R_op, length of list must be equal to number of outputs. - name (string, optional) – A name for debugging purposes
- **kwargs (optional) – Check
orig_function
for more arguments, only works when not inline.
Notes
- We support shared variables in the inner graph. This is automatic and invisible to the user. They can be as input to the node or in the inner graph.
- We support unused inputs. This is needed for the grad.
- We support nested OpFromGraph.
inline=True
will cause better runtime optimization at the cost of compilation time. Currently only works withfast_compile
orfast_run
mode.- It’s recommanded to provide pure functions (no side effects like setting global variable) as callable(s). The callable(s) supplied for overriding gradient/rop will be called only once at the first call to grad/R_op, and will be converted to OpFromGraph instances.
Examples
Example 1:
from theano import function, OpFromGraph, tensor x, y, z = tensor.scalars('xyz') e = x + y * z op = OpFromGraph([x, y, z], [e]) # op behaves like a normal theano op e2 = op(x, y, z) + op(z, y, x) fn = function([x, y, z], [e2])
Example 2 with shared variable:
import numpy as np import theano from theano import config, function, OpFromGraph, tensor x, y, z = tensor.scalars('xyz') s = theano.shared(np.random.rand(2, 2).astype(config.floatX)) e = x + y * z + s op = OpFromGraph([x, y, z], [e]) # op behaves like a normal theano op e2 = op(x, y, z) + op(z, y, x) fn = function([x, y, z], [e2])
Example 3 override gradient
from theano import function, OpFromGraph, tensor, grad x, y, z = tensor.scalars('xyz') e = x + y * z def rescale_dy(inps, grads): x, y, z = inps g, = grads return z*2 op = OpFromGraph( [x, y, z], [e], grad_overrides=['default', rescale_dy, 'default'] e2 = op(x, y, z) dx, dy, dz = grad(e2, [x, y, z]) fn = function([x, y, z], [dx, dy, dz]) # the gradient wrt y is now doubled fn(2., 3., 4.) # [1., 8., 3.]
- inputs (list of