Random subspace method

Random subspace method (or attribute bagging) is an ensemble classifier that consists of several classifiers each operating in a subspace of the original feature space, and outputs the class based on the outputs of these individual classifiers. Random subspace method has been used for decision trees (random decision forests), linear classifiers, support vector machines, nearest neighbours and other types of classifiers. This method is also applicable to one-class classifiers.

The algorithm is an attractive choice for classification problems where the number of features is much larger than the number of training objects, such as fMRI data or gene expression data.

In machine learning and statistics, feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables, predictors) for use in model construction. Feature selection techniques are used for three reasons:

1. simplification of models to make them easier to interpret by researchers/users,[1]
2. shorter training times,
3. enhanced generalization by reducing overfitting[2](formally, reduction of variance[1])

The central premise when using a feature selection technique is that the data contains many features that are either redundant or irrelevant, and can thus be removed without incurring much loss of information.[2] Redundant or irrelevant features are two distinct notions, since one relevant feature may be redundant in the presence of another relevant feature with which it is strongly correlated.[3]