com.rapidminer.operator.features.selection
Class RemoveCorrelatedFeatures

java.lang.Object
  com.rapidminer.tools.AbstractObservable<Operator>
      com.rapidminer.operator.Operator
          com.rapidminer.operator.AbstractExampleSetProcessing
              com.rapidminer.operator.features.AbstractFeatureProcessing
                  com.rapidminer.operator.features.selection.AbstractFeatureSelection
                      com.rapidminer.operator.features.selection.RemoveCorrelatedFeatures

All Implemented Interfaces:

ConfigurationListener, PreviewListener, ResourceConsumer, ParameterHandler, LoggingHandler, Observable<Operator>

public class RemoveCorrelatedFeatures
extends AbstractFeatureSelection

Removes (un-) correlated features due to the selected filter relation. The procedure is quadratic in number of attributes. In order to get more stable results, the original, random, and reverse order of attributes is available.

Please note that this operator might fail in some cases when the attributes should be filtered out, e.g. it might not be able to remove for example all negative correlated features. The reason for this behaviour seems to be that for the complete m x m - matrix of correlations (for m attributes) the correlations will not be recalculated and hence not checked if one of the attributes of the current pair was already marked for removal. That means for three attributes a1, a2, and a3 that it might be that a2 was already ruled out by the negative correlation with a1 and is now not able to rule out a3 any longer.

The used correlation function is the Pearson correlation.

Author:

Daniel Hakenjos, Ingo Mierswa

Field Summary

static java.lang.String	PARAMETER_ATTRIBUTE_ORDER The parameter name for "The algorithm takes this attribute order to calculate correlation and filter.
static java.lang.String	PARAMETER_CORRELATION The parameter name for "Use this correlation for the filter relation.
static java.lang.String	PARAMETER_FILTER_RELATION The parameter name for "Removes one of two features if their correlation fulfill this relation.
static java.lang.String	PARAMETER_USE_ABSOLUTE_CORRELATION The parameter name for "Indicates if the absolute value of the correlations should be used for comparison.
static OperatorVersion	VERSION_DETERMINISTIC_RANDOM_NUMBERS

Constructor Summary

RemoveCorrelatedFeatures(OperatorDescription description)

Method Summary

ExampleSet	apply(ExampleSet exampleSet) Delegate for the apply method.
OperatorVersion[]	getIncompatibleVersionChanges() Returns the versions of an operator after which its behavior incompatibly changed in ascending order.
java.util.List<ParameterType>	getParameterTypes() Returns a list of ParameterTypes describing the parameters of this operator.

Methods inherited from class com.rapidminer.operator.features.selection.AbstractFeatureSelection

modifyMetaData, writesIntoExistingData

Methods inherited from class com.rapidminer.operator.AbstractExampleSetProcessing

doWork, getExampleSetInputPort, getExampleSetOutputPort, getInputPort, getRequiredMetaData, shouldAutoConnect

Methods inherited from class com.rapidminer.operator.Operator

acceptsInput, addError, addError, addValue, addWarning, apply, apply, assumePreconditionsSatisfied, checkAll, checkAllExcludingMetaData, checkDeprecations, checkForStop, checkIO, checkProperties, clear, clearErrorList, cloneOperator, collectErrors, createExperimentTree, createExperimentTree, createFromXML, createFromXML, createFromXML, createMarkedExperimentTree, createMarkedProcessTree, createProcessTree, createProcessTree, disconnectPorts, execute, fireUpdate, freeMemory, getAddOnlyAdditionalOutput, getApplyCount, getCompatibilityLevel, getDeliveredOutputClasses, getDeprecationInfo, getDesiredInputClasses, getDOMRepresentation, getEncoding, getErrorList, getExecutionUnit, getExperiment, getInput, getInput, getInput, getInputClasses, getInputDescription, getInputPorts, getIODescription, getLog, getLogger, getName, getNumberOfBreakpoints, getOperatorClassName, getOperatorDescription, getOutputClasses, getOutputPorts, getParameter, getParameterAsBoolean, getParameterAsChar, getParameterAsColor, getParameterAsDouble, getParameterAsFile, getParameterAsFile, getParameterAsInputStream, getParameterAsInt, getParameterAsMatrix, getParameterAsRepositoryLocation, getParameterAsString, getParameterHandler, getParameterList, getParameters, getParameterTupel, getParameterType, getParent, getPortOwner, getProcess, getResourceConsumptionEstimator, getRoot, getStartTime, getTransformer, getUserDescription, getValue, getValues, getXML, getXML, getXML, hasBreakpoint, hasBreakpoint, hasInput, inApplyLoop, isDebugMode, isDirty, isEnabled, isExpanded, isParallel, isParameterSet, isRunning, log, log, logError, logNote, logWarning, lookupOperator, makeDirty, makeDirtyOnUpdate, notifyRenaming, performAdditionalChecks, preAutoWire, processFinished, processStarts, producesOutput, propagateDirtyness, register, registerOperator, remove, removeAndKeepConnections, rename, resume, setBreakpoint, setCompatibilityLevel, setEnabled, setEnclosingProcess, setExpanded, setInput, setListParameter, setPairParameter, setParameter, setParameters, setUserDescription, shouldAutoConnect, shouldStopStandaloneExecution, toString, transformMetaData, unregisterOperator, updateExecutionOrder, walk, writeXML, writeXML

Methods inherited from class com.rapidminer.tools.AbstractObservable

addObserver, addObserverAsFirst, fireUpdate, removeObserver

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PARAMETER_CORRELATION

public static final java.lang.String PARAMETER_CORRELATION

The parameter name for "Use this correlation for the filter relation."

See Also:

Constant Field Values

PARAMETER_FILTER_RELATION

public static final java.lang.String PARAMETER_FILTER_RELATION

The parameter name for "Removes one of two features if their correlation fulfill this relation."

See Also:

Constant Field Values

PARAMETER_ATTRIBUTE_ORDER

public static final java.lang.String PARAMETER_ATTRIBUTE_ORDER

The parameter name for "The algorithm takes this attribute order to calculate correlation and filter."

See Also:

Constant Field Values

PARAMETER_USE_ABSOLUTE_CORRELATION

public static final java.lang.String PARAMETER_USE_ABSOLUTE_CORRELATION

The parameter name for "Indicates if the absolute value of the correlations should be used for comparison."

See Also:

Constant Field Values

VERSION_DETERMINISTIC_RANDOM_NUMBERS

public static final OperatorVersion VERSION_DETERMINISTIC_RANDOM_NUMBERS

Constructor Detail

RemoveCorrelatedFeatures

public RemoveCorrelatedFeatures(OperatorDescription description)

Method Detail

apply

public ExampleSet apply(ExampleSet exampleSet)
                 throws OperatorException

Description copied from class: AbstractExampleSetProcessing

Delegate for the apply method. The given ExampleSet is already a clone of the input example set so that changing this examples set does not affect the original one. Subclasses should avoid cloning again unnecessarily.

Specified by:

apply in class AbstractExampleSetProcessing

Throws:

OperatorException

getParameterTypes

public java.util.List<ParameterType> getParameterTypes()

Description copied from class: Operator

Returns a list of ParameterTypes describing the parameters of this operator. The default implementation returns an empty list if no input objects can be retained and special parameters for those input objects which can be prevented from being consumed. ATTENTION! This will create new parameterTypes. For calling already existing parameter types use getParameters().getParameterTypes();

Specified by:

getParameterTypes in interface ParameterHandler

Overrides:

getParameterTypes in class Operator

getIncompatibleVersionChanges

public OperatorVersion[] getIncompatibleVersionChanges()

Description copied from class: Operator

Returns the versions of an operator after which its behavior incompatibly changed in ascending order. Only the versions after which the new behaviour was introduced are returned. See comment of OperatorVersion for details.

Overrides:

getIncompatibleVersionChanges in class Operator