A sequential minimal optimisation for certain restricted quadratic problems. More...

#include <clsfy_smo_1.h>

Inheritance diagram for clsfy_smo_1_rbf:

Public Member Functions
double	gamma () const
	0.5 sigma^-2, where sigma is the width of the Gaussian kernel.
void	set_gamma (double gamma)
	Control sigma, the width of the Gaussian kernel.
	clsfy_smo_1_rbf ()
virtual int	calc ()
	Run the optimisation.
void	set_data (const mbl_data_wrapper< vnl_vector< double > > &data, const vcl_vector< int > &targets)
	Takes a copy of the data wrapper, but not the data.
double	C () const
void	set_C (double C)
	Set the upper bound on the Lagrange multipliers.
const vnl_vector< double > &	lagrange_mults () const
	Get the optimised parameters.
void	set_lagrange_mults (const vnl_vector< double > &lagrange_mults)
	Set the initial values of the parameters to be optimised.
double	bias ()
	Bias term in function evaluation.
virtual void	reseed (unsigned long seed)
	Reseeds the internal random number generator.
const double &	tolerance () const
	amount by which a sample can violate the KKT conditions.
void	set_tolerance (double tolerance)
	Set the amount by which a sample can violate the KKT conditions.
double	eps () const
	tolerance on several equalities.
void	set_eps (double eps)
	Set the tolerance on several equalities.
virtual double	error_rate ()
double	error ()
	error rate on the training data.
Protected Member Functions
virtual double	kernel (int i1, int i2)
	Calculate the kernel for data items i1 and i2.
virtual int	take_step (int i1, int i2, double E1)
	Attempt to jointly optimise Lagrange multipliers i1, and i2.
virtual int	examine_example (int i1)
	Attempt to optimise sample i1.
const vnl_vector< double > &	data_point (unsigned long l)
	Access the data points.
virtual double	learned_func (int k)
	Calculate the classifier function learnt so far for data item k.
Protected Attributes
double	gamma_
double	C_
	upper bound on Lagrange multipliers.
double	error_
	Error rate on the training data.
mbl_data_wrapper< vnl_vector < double > > *	data_
	An iterator on the data;.
vnl_vector< double >	alph_
	The parameters to be optimised.
double	tolerance_
	Amount by which a sample can violate the KKT conditions.
double	eps_
	Tolerance on several equalities.
double	b_
	Bias.
vcl_vector< double >	error_cache_
	Cache KKT error values for unbound multipliers.
vcl_vector< int >	target_
	Target values y_i.
vcl_vector< double >	precomputed_self_dot_product_
	The norm of each training vector is useful to know quickly.
vnl_random	rng_

Detailed Description

A sequential minimal optimisation for certain restricted quadratic problems.

This finds the optimal point on a quadratic function constrained by inequality bounds on each parameter, and a single equality constraint. The is the class of problems associated with Support Vector Machines. Uses RBF kernel operator.

Definition at line 61 of file clsfy_smo_1.h.

Constructor & Destructor Documentation

clsfy_smo_1_rbf::clsfy_smo_1_rbf ( )

Definition at line 96 of file clsfy_smo_1.cxx.

Member Function Documentation

double clsfy_smo_base::bias ( ) [inherited]

Bias term in function evaluation.

For SVMs this would be the value to be subtracted from sum of kernel functions to get 0 as class boundary.

Definition at line 68 of file clsfy_smo_base.cxx.

double clsfy_smo_1_lin::C ( ) const [inherited]

Definition at line 64 of file clsfy_smo_1.cxx.

int clsfy_smo_1_rbf::calc ( ) [virtual]

Run the optimisation.

Reimplemented from clsfy_smo_1_lin.

Definition at line 320 of file clsfy_smo_1.cxx.

const vnl_vector< double > & clsfy_smo_base::data_point ( unsigned long l ) [protected, inherited]

Access the data points.

Definition at line 29 of file clsfy_smo_base.cxx.

double clsfy_smo_base::eps ( ) const [inherited]

tolerance on several equalities.

Tolerance on several equalities.

Including testing if a Lagrange multiplier is at one of the bounds.

Definition at line 109 of file clsfy_smo_base.cxx.

double clsfy_smo_base::error ( ) [inherited]

error rate on the training data.

Definition at line 21 of file clsfy_smo_base.cxx.

double clsfy_smo_base::error_rate ( ) [virtual, inherited]

Definition at line 141 of file clsfy_smo_base.cxx.

int clsfy_smo_1_lin::examine_example ( int i1 ) [protected, virtual, inherited]

Attempt to optimise sample i1.

This attempts to find another value i2, in order to jointly optimise both.

Implements clsfy_smo_base.

Definition at line 243 of file clsfy_smo_1.cxx.

double clsfy_smo_1_rbf::gamma ( ) const

0.5 sigma^-2, where sigma is the width of the Gaussian kernel.

-0.5 sigma^-2, where sigma is the width of the Gaussian kernel.

Definition at line 80 of file clsfy_smo_1.cxx.

double clsfy_smo_1_rbf::kernel	(	int	i1,
		int	i2
	)		`[protected, virtual]`

Calculate the kernel for data items i1 and i2.

Reimplemented from clsfy_smo_1_lin.

Definition at line 36 of file clsfy_smo_1.cxx.

const vnl_vector< double > & clsfy_smo_base::lagrange_mults ( ) const [inherited]

Get the optimised parameters.

Definition at line 51 of file clsfy_smo_base.cxx.

double clsfy_smo_base::learned_func ( int k ) [protected, virtual, inherited]

Calculate the classifier function learnt so far for data item k.

Definition at line 37 of file clsfy_smo_base.cxx.

void clsfy_smo_base::reseed ( unsigned long seed ) [virtual, inherited]

Reseeds the internal random number generator.

To achieve quasi-random initialisation use;

 #include <vcl_ctime.h>
 ..
 sampler.reseed(vcl_time(0));

Definition at line 82 of file clsfy_smo_base.cxx.

void clsfy_smo_1_lin::set_C ( double C ) [inherited]

Set the upper bound on the Lagrange multipliers.

Definition at line 71 of file clsfy_smo_1.cxx.

void clsfy_smo_1_lin::set_data	(	const mbl_data_wrapper< vnl_vector< double > > &	data,
		const vcl_vector< int > &	targets
	)		`[inherited]`

Takes a copy of the data wrapper, but not the data.

Be careful not to destroy the underlying data while using this object.

Definition at line 49 of file clsfy_smo_1.cxx.

void clsfy_smo_base::set_eps ( double eps ) [inherited]

Set the tolerance on several equalities.

Including testing if a Lagrange multiplier is at one of the bounds. Default value is 0.001

Including testing if a Lagrange multiplier is at one of the bounds. Default value is 0.001;

Definition at line 119 of file clsfy_smo_base.cxx.

void clsfy_smo_1_rbf::set_gamma ( double gamma )

Control sigma, the width of the Gaussian kernel.

gamma = -0.5 sigma^-2

gamma = 0.5 sigma^-2

Definition at line 89 of file clsfy_smo_1.cxx.

void clsfy_smo_base::set_lagrange_mults ( const vnl_vector< double > & lagrange_mults ) [inherited]

Set the initial values of the parameters to be optimised.

The caller is responsible for ensuring that the initial values fulfill the constraints;

Definition at line 61 of file clsfy_smo_base.cxx.

void clsfy_smo_base::set_tolerance ( double tolerance ) [inherited]

Set the amount by which a sample can violate the KKT conditions.

Default value is 0.001

Definition at line 99 of file clsfy_smo_base.cxx.

int clsfy_smo_1_lin::take_step	(	int	i1,
		int	i2,
		double	E1
	)		`[protected, virtual, inherited]`

Attempt to jointly optimise Lagrange multipliers i1, and i2.

Parameters:

i1	first Lagrange multiplier.
i2	second Lagrange multiplier.
E1	The amount by which i1 violates KKT conditions.

new values of alpha_1, alpha_2

Implements clsfy_smo_base.

Definition at line 110 of file clsfy_smo_1.cxx.

const double & clsfy_smo_base::tolerance ( ) const [inherited]

amount by which a sample can violate the KKT conditions.

Amount by which a sample can violate the KKT conditions.

Definition at line 90 of file clsfy_smo_base.cxx.

Member Data Documentation

vnl_vector<double> clsfy_smo_base::alph_ [protected, inherited]

The parameters to be optimised.

Definition at line 35 of file clsfy_smo_base.h.

double clsfy_smo_base::b_ [protected, inherited]

Bias.

Invariant:: a_i unbound and optimal => KKT_i holds

Definition at line 46 of file clsfy_smo_base.h.

double clsfy_smo_1_lin::C_ [protected, inherited]

upper bound on Lagrange multipliers.

Definition at line 22 of file clsfy_smo_1.h.

mbl_data_wrapper<vnl_vector<double> >* clsfy_smo_base::data_ [protected, inherited]

An iterator on the data;.

Definition at line 32 of file clsfy_smo_base.h.

double clsfy_smo_base::eps_ [protected, inherited]

Tolerance on several equalities.

Including testing if a Lagrange multiplier is at one of the bounds.

Definition at line 42 of file clsfy_smo_base.h.

double clsfy_smo_base::error_ [protected, inherited]

Error rate on the training data.

Definition at line 29 of file clsfy_smo_base.h.

vcl_vector<double> clsfy_smo_base::error_cache_ [protected, inherited]

Cache KKT error values for unbound multipliers.

Invariant:: a_i unbound => E_i = f(x_i) - y1

Definition at line 50 of file clsfy_smo_base.h.

double clsfy_smo_1_rbf::gamma_ [protected]

-0.5 sigma^-2, where sigma is the width of the Gaussian kernel.

Definition at line 66 of file clsfy_smo_1.h.

vcl_vector<double> clsfy_smo_base::precomputed_self_dot_product_ [protected, inherited]

The norm of each training vector is useful to know quickly.

Definition at line 56 of file clsfy_smo_base.h.

vnl_random clsfy_smo_base::rng_ [protected, inherited]

Definition at line 58 of file clsfy_smo_base.h.

vcl_vector<int> clsfy_smo_base::target_ [protected, inherited]

Target values y_i.

Definition at line 53 of file clsfy_smo_base.h.

double clsfy_smo_base::tolerance_ [protected, inherited]

Amount by which a sample can violate the KKT conditions.

Definition at line 38 of file clsfy_smo_base.h.

The documentation for this class was generated from the following files:

contrib/mul/clsfy/clsfy_smo_1.h
contrib/mul/clsfy/clsfy_smo_1.cxx

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation