com.rapidminer.operator.performance
Class UserBasedPerformanceEvaluator

java.lang.Object
  com.rapidminer.tools.AbstractObservable<Operator>
      com.rapidminer.operator.Operator
          com.rapidminer.operator.performance.AbstractPerformanceEvaluator
              com.rapidminer.operator.performance.UserBasedPerformanceEvaluator

All Implemented Interfaces:

ConfigurationListener, PreviewListener, ResourceConsumer, CapabilityProvider, ParameterHandler, LoggingHandler, Observable<Operator>

public class UserBasedPerformanceEvaluator
extends AbstractPerformanceEvaluator

This performance evaluator operator should be used for regression tasks, i.e. in cases where the label attribute has a numerical value type. The operator expects a test ExampleSet as input, whose elements have both true and predicted labels, and delivers as output a list of performance values according to a list of performance criteria that it calculates. If an input performance vector was already given, this is used for keeping the performance values.

Additional user-defined implementations of PerformanceCriterion can be specified by using the parameter list additional_performance_criteria. Each key/value pair in this list must specify a fully qualified classname (as the key), and a string parameter (as value) that is passed to the constructor. Please make sure that the class files are in the classpath (this is the case if the implementations are supplied by a plugin) and that they implement a one-argument constructor taking a string parameter. It must also be ensured that these classes extend MeasuredPerformance since the PerformanceEvaluator operator will only support these criteria. Please note that only the first three user defined criteria can be used as logging value with names "user1", ... , "user3".

The resulting performance vectors are usually compared with a standard performance comparator which only compares the fitness values of the main criterion. Other implementations than this simple comparator can be specified using the parameter comparator_class. This may for instance be useful if you want to compare performance vectors according to the weighted sum of the individual criteria. In order to implement your own comparator, simply subclass PerformanceComparator. Please note that for true multi-objective optimization usually another selection scheme is used instead of simply replacing the performance comparator.

Author:

Ingo Mierswa

Field Summary

static java.lang.String	PARAMETER_ADDITIONAL_PERFORMANCE_CRITERIA The parameter name for "List of classes that implement com.rapidminer..operator.performance.PerformanceCriterion.
static java.lang.String[]	USER_CRITERIA_NAMES The names of allowed user criteria.

Fields inherited from class com.rapidminer.operator.performance.AbstractPerformanceEvaluator

PARAMETER_COMPARATOR_CLASS, PARAMETER_MAIN_CRITERION, PARAMETER_SKIP_UNDEFINED_LABELS

Fields inherited from interface com.rapidminer.operator.learner.CapabilityProvider

PROPERTY_RAPIDMINER_GENERAL_CAPABILITIES_WARN

Constructor Summary

UserBasedPerformanceEvaluator(OperatorDescription description)

Method Summary

protected boolean	canEvaluate(int valueType) Returns true iff this operator can evaluate labels of the given value type.
protected void	checkCompatibility(ExampleSet exampleSet) Does nothing.
protected double[]	getClassWeights(Attribute label) Returns null.
java.util.List<PerformanceCriterion>	getCriteria() Delivers the list of criteria which is able for this operator.
java.util.List<ParameterType>	getParameterTypes() Returns a list of ParameterTypes describing the parameters of this operator.
protected boolean	showCriteriaParameter() Returns false.
boolean	supportsCapability(OperatorCapability capability) Checks for Learner capabilities.

Methods inherited from class com.rapidminer.operator.performance.AbstractPerformanceEvaluator

checkProperties, doWork, doWork, evaluate, evaluate, init, shouldAutoConnect, showComparatorParameter, showSkipNaNLabelsParameter

Methods inherited from class com.rapidminer.operator.Operator

acceptsInput, addError, addError, addValue, addWarning, apply, apply, assumePreconditionsSatisfied, checkAll, checkAllExcludingMetaData, checkDeprecations, checkForStop, checkIO, clear, clearErrorList, cloneOperator, collectErrors, createExperimentTree, createExperimentTree, createFromXML, createFromXML, createFromXML, createMarkedExperimentTree, createMarkedProcessTree, createProcessTree, createProcessTree, disconnectPorts, execute, fireUpdate, freeMemory, getAddOnlyAdditionalOutput, getApplyCount, getCompatibilityLevel, getDeliveredOutputClasses, getDeprecationInfo, getDesiredInputClasses, getDOMRepresentation, getEncoding, getErrorList, getExecutionUnit, getExperiment, getIncompatibleVersionChanges, getInput, getInput, getInput, getInputClasses, getInputDescription, getInputPorts, getIODescription, getLog, getLogger, getName, getNumberOfBreakpoints, getOperatorClassName, getOperatorDescription, getOutputClasses, getOutputPorts, getParameter, getParameterAsBoolean, getParameterAsChar, getParameterAsColor, getParameterAsDouble, getParameterAsFile, getParameterAsFile, getParameterAsInputStream, getParameterAsInt, getParameterAsMatrix, getParameterAsRepositoryLocation, getParameterAsString, getParameterHandler, getParameterList, getParameters, getParameterTupel, getParameterType, getParent, getPortOwner, getProcess, getResourceConsumptionEstimator, getRoot, getStartTime, getTransformer, getUserDescription, getValue, getValues, getXML, getXML, getXML, hasBreakpoint, hasBreakpoint, hasInput, inApplyLoop, isDebugMode, isDirty, isEnabled, isExpanded, isParallel, isParameterSet, isRunning, log, log, logError, logNote, logWarning, lookupOperator, makeDirty, makeDirtyOnUpdate, notifyRenaming, performAdditionalChecks, preAutoWire, processFinished, processStarts, producesOutput, propagateDirtyness, register, registerOperator, remove, removeAndKeepConnections, rename, resume, setBreakpoint, setCompatibilityLevel, setEnabled, setEnclosingProcess, setExpanded, setInput, setListParameter, setPairParameter, setParameter, setParameters, setUserDescription, shouldAutoConnect, shouldStopStandaloneExecution, toString, transformMetaData, unregisterOperator, updateExecutionOrder, walk, writeXML, writeXML

Methods inherited from class com.rapidminer.tools.AbstractObservable

addObserver, addObserverAsFirst, fireUpdate, removeObserver

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PARAMETER_ADDITIONAL_PERFORMANCE_CRITERIA

public static final java.lang.String PARAMETER_ADDITIONAL_PERFORMANCE_CRITERIA

The parameter name for "List of classes that implement com.rapidminer..operator.performance.PerformanceCriterion."

See Also:

Constant Field Values

USER_CRITERIA_NAMES

public static final java.lang.String[] USER_CRITERIA_NAMES

The names of allowed user criteria. These are necessary for plotting purposes and the definition of the main criterion.

Constructor Detail

UserBasedPerformanceEvaluator

public UserBasedPerformanceEvaluator(OperatorDescription description)

Method Detail

checkCompatibility

protected void checkCompatibility(ExampleSet exampleSet)
                           throws OperatorException

Does nothing.

Specified by:

checkCompatibility in class AbstractPerformanceEvaluator

Throws:

OperatorException

getClassWeights

protected double[] getClassWeights(Attribute label)
                            throws UndefinedParameterError

Returns null.

Specified by:

getClassWeights in class AbstractPerformanceEvaluator

Throws:

UndefinedParameterError

showCriteriaParameter

protected boolean showCriteriaParameter()

Returns false.

Overrides:

showCriteriaParameter in class AbstractPerformanceEvaluator

getCriteria

public java.util.List<PerformanceCriterion> getCriteria()

Description copied from class: AbstractPerformanceEvaluator

Delivers the list of criteria which is able for this operator. Please note that all criteria in the list must be freshly instantiated since no copy is created in different runs of this operator. This is important in order to not mess up the results. This method must not return null but should return an empty list in this case. ATTENTION! This method is called during the creation of parameters. Do not try to get a parameter value inside this method, since this will create an infinite loop!

Specified by:

getCriteria in class AbstractPerformanceEvaluator

getParameterTypes

public java.util.List<ParameterType> getParameterTypes()

Description copied from class: Operator

Returns a list of ParameterTypes describing the parameters of this operator. The default implementation returns an empty list if no input objects can be retained and special parameters for those input objects which can be prevented from being consumed. ATTENTION! This will create new parameterTypes. For calling already existing parameter types use getParameters().getParameterTypes();

Specified by:

getParameterTypes in interface ParameterHandler

Overrides:

getParameterTypes in class AbstractPerformanceEvaluator

canEvaluate

protected boolean canEvaluate(int valueType)

Description copied from class: AbstractPerformanceEvaluator

Returns true iff this operator can evaluate labels of the given value type.

Specified by:

canEvaluate in class AbstractPerformanceEvaluator

See Also:

Ontology.ATTRIBUTE_VALUE_TYPE

supportsCapability

public boolean supportsCapability(OperatorCapability capability)

Description copied from interface: CapabilityProvider

Checks for Learner capabilities. Should return true if the given capability is supported.

Specified by:

supportsCapability in interface CapabilityProvider

Overrides:

supportsCapability in class AbstractPerformanceEvaluator