This class implements a huber regularizer. More...

#include <regularizers_imp.h>

Inheritance diagram for dismec::objective::HuberRegularizer:

Public Member Functions
	HuberRegularizer (real_t epsilon, real_t scale=1.0, bool ignore_bias=false)
	Constructor for a Huber regularizer objective. More...

real_t	point_wise_value (real_t x) const

real_t	point_wise_grad (real_t x) const

real_t	point_wise_quad (real_t x) const

Public Member Functions inherited from dismec::objective::PointWiseRegularizer< HuberRegularizer >
	PointWiseRegularizer (real_t scale=1, bool ignore_bias=false)

long	num_variables () const noexcept final
	The pointwise regularizer can act on arbitrarily sized vectors, so `num_variables() == -1`. More...

real_t	value_unchecked (const HashVector &location) override

void	hessian_times_direction_unchecked (const HashVector &location, const DenseRealVector &direction, Eigen::Ref< DenseRealVector > target) override

void	gradient_unchecked (const HashVector &location, Eigen::Ref< DenseRealVector > target) override

void	gradient_at_zero_unchecked (Eigen::Ref< DenseRealVector > target) override

void	diag_preconditioner_unchecked (const HashVector &location, Eigen::Ref< DenseRealVector > target) override

void	project_to_line_unchecked (const HashVector &location, const DenseRealVector &direction) override

real_t	lookup_on_line (real_t a) override
	Looks up the value of the objective on the line defined by the last call to `project_to_line()`. More...

bool	dont_regularize_bias () const

real_t	scale () const
	Returns the common scale factor for the entire regularizer. More...

Public Member Functions inherited from dismec::objective::Objective
	Objective ()

virtual	~Objective () noexcept=default

real_t	value (const HashVector &location)
	Evaluate the objective at the given `location`. More...

void	diag_preconditioner (const HashVector &location, Eigen::Ref< DenseRealVector > target)
	Get precondition to be used in CG optimization. More...

void	project_to_line (const HashVector &location, const DenseRealVector &direction)
	creates a function g such that `g(a) = objective(location + a * direction)` Use `lookup_on_line()` to evaluate `g`. More...

virtual void	declare_vector_on_last_line (const HashVector &location, real_t t)
	State that the given vector corresponds to a certain position on the line of the last line search. More...

void	gradient_at_zero (Eigen::Ref< DenseRealVector > target)
	Gets the gradient for location zero. More...

void	gradient (const HashVector &location, Eigen::Ref< DenseRealVector > target)
	Evaluate the gradient at `location`. More...

void	hessian_times_direction (const HashVector &location, const DenseRealVector &direction, Eigen::Ref< DenseRealVector > target)
	Calculates the product of the Hessian matrix at `location` with `direction`. More...

void	gradient_and_pre_conditioner (const HashVector &location, Eigen::Ref< DenseRealVector > gradient, Eigen::Ref< DenseRealVector > pre)
	Combines the calculation of gradient and pre-conditioner, which may be more efficient in some cases. More...

Public Member Functions inherited from dismec::stats::Tracked
	Tracked ()
	Default constructor, creates the internal `stats::StatisticsCollection`. More...

void	register_stat (const std::string &name, std::unique_ptr< Statistics > stat)
	Registers a tracker for the statistics `name`. More...

std::shared_ptr< StatisticsCollection >	get_stats () const
	Gets an ownership-sharing reference to the `StatisticsCollection`. More...

Private Attributes
real_t	m_Epsilon {1}

Additional Inherited Members
Protected Member Functions inherited from dismec::stats::Tracked
	~Tracked ()
	Non-virtual destructor. Declared protected, so we don't accidentally try to do a polymorphic delete. More...

template<class T >
void	record (stat_id_t stat, T &&value)
	Record statistics. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

void	declare_stat (stat_id_t index, StatisticMetaData meta)
	Declares a new statistics. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

void	declare_tag (tag_id_t index, std::string name)
	Declares a new tag. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

template<class... Args>
void	set_tag (tag_id_t tag, long value)
	Set value of tag. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

template<class... Args>
auto	make_timer (stat_id_t id, Args... args)
	Creates a new `ScopeTimer` using `stats::record_scope_time`. More...

Detailed Description

This class implements a huber regularizer.

The regularizer acts pointwise on each weight by applying a Huber function. This function is smooth, and consists of a parabola around zero and linear parts for larger values. These linear parts are potentially problematic when doing second order optimization, as the Hessian would be zero there. For that reason, the hessian_times_direction() function does not use the actual Hessian of the function, but a quadratic upper bound on the function. The details are discussed in section 2.1 of the pdf.

By choosing the switching point between quadratic and linear expression to be very small, this function can be used as a continuously differentiable approximation to L1 regularization.

Definition at line 70 of file regularizers_imp.h.

Constructor & Destructor Documentation

◆ HuberRegularizer()

HuberRegularizer::HuberRegularizer	(	real_t	epsilon,
		real_t	scale = `1.0`,
		bool	ignore_bias = `false`
	)

explicit

Constructor for a Huber regularizer objective.

Parameters

epsilon The cutoff point between quadratic and linear behaviour.

Exceptions

invalid_argument,if epsilon < 0.

Definition at line 60 of file regularizers_imp.cpp.

References m_Epsilon, and THROW_EXCEPTION.

Member Function Documentation

◆ point_wise_grad()

real_t dismec::objective::HuberRegularizer::point_wise_grad ( real_t x ) const

Definition at line 73 of file regularizers_imp.cpp.

◆ point_wise_quad()

real_t dismec::objective::HuberRegularizer::point_wise_quad ( real_t x ) const

Definition at line 83 of file regularizers_imp.cpp.

◆ point_wise_value()

real_t dismec::objective::HuberRegularizer::point_wise_value ( real_t x ) const

Definition at line 67 of file regularizers_imp.cpp.

Member Data Documentation

◆ m_Epsilon

real_t dismec::objective::HuberRegularizer::m_Epsilon {1}

private

Definition at line 83 of file regularizers_imp.h.

Referenced by HuberRegularizer().

The documentation for this class was generated from the following files:

src/objective/regularizers_imp.h
src/objective/regularizers_imp.cpp

Public Member Functions

Private Attributes

Additional Inherited Members