Perform the stepwise regression algorithm to determine which subset of variables appear to be significant predictors. More...
#include <mbl_stepwise_regression.h>
Public Member Functions | |
| mbl_stepwise_regression (const vnl_matrix< double > &x, const vnl_vector< double > &y) | |
| Constructor, note you must supply the data references. | |
| void | operator() () |
| Run the algorithm as determined by mode. | |
| const vcl_set< unsigned > | basis () const |
| return the basis variables. | |
| void | set_mode (step_mode mode) |
| Set the mode to forwards or backwards. | |
| const vnl_vector< double > & | weights () const |
| Return the regression coefficients + constant (final term). | |
Private Types | |
| enum | step_mode { eFORWARDS, eBACKWARDS } |
| Forwards or backwards stepwise (add or remove variables). More... | |
Private Member Functions | |
| bool | add_variable (bool forceAdd=false) |
| Add a new variable and return if significant. | |
| bool | remove_variable () |
| Remove a variable that makes no significant difference to RSS. | |
| double | f_ratio (double rssExtended, double rssBase, unsigned q) |
| bool | test_significance (double rssExtended, double rssBase, double fthresh) |
| Evaluate F-ratio to test if change in sum of squares was significant. | |
| void | do_forward_stepwise_regression () |
| Step forward through basis complement, adding in new significant variables and removing those that cease to be significant. | |
| void | do_backward_stepwise_regression () |
| Step back and remove all insignificant variables, then try and step forward again. | |
Private Attributes | |
| const vnl_matrix< double > & | x_ |
| The data matrix of x values (predictor variables). | |
| const vnl_vector< double > & | y_ |
| Vector of dependent y values. | |
| unsigned | num_examples_ |
| number of training examples (i.e. number of rows in x_). | |
| unsigned | num_vars_ |
| dimensionality - (i.e. number of columns in x_). | |
| vnl_matrix< double > | XtX_ |
| [x,-1]'[x,-1]. | |
| vnl_vector< double > | XtY_ |
| [x,-1]'[y,-1]. | |
| vcl_set< unsigned > | basis_ |
| Basis (i.e. all significant variables). | |
| vcl_set< unsigned > | basis_complement_ |
| All non-basis variables. | |
| vnl_vector< double > | weights_ |
| The regression coefficients + constant (final term). | |
| double | rss_ |
| The residual sum of squares. | |
| double | FthreshAdd_ |
| F-ratio significance threshold for adding a variable. | |
| double | FthreshRemove_ |
| F-ratio significance threshold for removing a variable. | |
| step_mode | mode_ |
| forwards or backwards mode. | |
Perform the stepwise regression algorithm to determine which subset of variables appear to be significant predictors.
Definition at line 15 of file mbl_stepwise_regression.h.
enum mbl_stepwise_regression::step_mode [private] |
Forwards or backwards stepwise (add or remove variables).
Definition at line 18 of file mbl_stepwise_regression.h.
| mbl_stepwise_regression::mbl_stepwise_regression | ( | const vnl_matrix< double > & | x, |
| const vnl_vector< double > & | y | ||
| ) |
Constructor, note you must supply the data references.
These must remain in scope during algorithm execution The data matrix of x values (predictor variables) is arranged as: Each training example is a row, each x-variable dimension corresponds to a column
Definition at line 20 of file mbl_stepwise_regression.cxx.
| bool mbl_stepwise_regression::add_variable | ( | bool | forceAdd = false | ) | [private] |
Add a new variable and return if significant.
Only added if it makes a significant reduction in RSS, unless forceAdd is set Always adds the variable making most difference to RSS
Definition at line 146 of file mbl_stepwise_regression.cxx.
| const vcl_set<unsigned > mbl_stepwise_regression::basis | ( | ) | const [inline] |
return the basis variables.
I.e. those determined to be significantly correlated with y in stepwise search
Definition at line 90 of file mbl_stepwise_regression.h.
| void mbl_stepwise_regression::do_backward_stepwise_regression | ( | ) | [private] |
Step back and remove all insignificant variables, then try and step forward again.
Definition at line 134 of file mbl_stepwise_regression.cxx.
| void mbl_stepwise_regression::do_forward_stepwise_regression | ( | ) | [private] |
Step forward through basis complement, adding in new significant variables and removing those that cease to be significant.
Definition at line 122 of file mbl_stepwise_regression.cxx.
| double mbl_stepwise_regression::f_ratio | ( | double | rssExtended, |
| double | rssBase, | ||
| unsigned | q | ||
| ) | [inline, private] |
Definition at line 59 of file mbl_stepwise_regression.h.
| void mbl_stepwise_regression::operator() | ( | ) |
Run the algorithm as determined by mode.
Definition at line 85 of file mbl_stepwise_regression.cxx.
| bool mbl_stepwise_regression::remove_variable | ( | ) | [private] |
Remove a variable that makes no significant difference to RSS.
Will remove the one that causes least change to RSS
Definition at line 193 of file mbl_stepwise_regression.cxx.
| void mbl_stepwise_regression::set_mode | ( | step_mode | mode | ) | [inline] |
Set the mode to forwards or backwards.
Note backwards can take a long compute time in a space of high dimension
Definition at line 94 of file mbl_stepwise_regression.h.
| bool mbl_stepwise_regression::test_significance | ( | double | rssExtended, |
| double | rssBase, | ||
| double | fthresh | ||
| ) | [inline, private] |
Evaluate F-ratio to test if change in sum of squares was significant.
Definition at line 67 of file mbl_stepwise_regression.h.
| const vnl_vector<double >& mbl_stepwise_regression::weights | ( | ) | const [inline] |
Return the regression coefficients + constant (final term).
Definition at line 97 of file mbl_stepwise_regression.h.
vcl_set<unsigned> mbl_stepwise_regression::basis_ [private] |
Basis (i.e. all significant variables).
Definition at line 33 of file mbl_stepwise_regression.h.
vcl_set<unsigned> mbl_stepwise_regression::basis_complement_ [private] |
All non-basis variables.
Definition at line 35 of file mbl_stepwise_regression.h.
double mbl_stepwise_regression::FthreshAdd_ [private] |
F-ratio significance threshold for adding a variable.
Definition at line 42 of file mbl_stepwise_regression.h.
double mbl_stepwise_regression::FthreshRemove_ [private] |
F-ratio significance threshold for removing a variable.
Note must be less than addition threshold or infinite cycling will occur
Definition at line 45 of file mbl_stepwise_regression.h.
step_mode mbl_stepwise_regression::mode_ [private] |
forwards or backwards mode.
Definition at line 47 of file mbl_stepwise_regression.h.
unsigned mbl_stepwise_regression::num_examples_ [private] |
number of training examples (i.e. number of rows in x_).
Definition at line 25 of file mbl_stepwise_regression.h.
unsigned mbl_stepwise_regression::num_vars_ [private] |
dimensionality - (i.e. number of columns in x_).
Definition at line 27 of file mbl_stepwise_regression.h.
double mbl_stepwise_regression::rss_ [private] |
The residual sum of squares.
Definition at line 40 of file mbl_stepwise_regression.h.
vnl_vector<double> mbl_stepwise_regression::weights_ [private] |
The regression coefficients + constant (final term).
Definition at line 37 of file mbl_stepwise_regression.h.
const vnl_matrix<double>& mbl_stepwise_regression::x_ [private] |
The data matrix of x values (predictor variables).
Each training example is a row, each x-variable dimension corresponds to a column
Definition at line 21 of file mbl_stepwise_regression.h.
vnl_matrix<double> mbl_stepwise_regression::XtX_ [private] |
[x,-1]'[x,-1].
Definition at line 29 of file mbl_stepwise_regression.h.
vnl_vector<double> mbl_stepwise_regression::XtY_ [private] |
[x,-1]'[y,-1].
Definition at line 31 of file mbl_stepwise_regression.h.
const vnl_vector<double>& mbl_stepwise_regression::y_ [private] |
Vector of dependent y values.
Definition at line 23 of file mbl_stepwise_regression.h.
1.7.5.1