Class that models an optimization objective. More...

#include <objective.h>

Inheritance diagram for dismec::objective::Objective:

Public Member Functions
	Objective ()

virtual	~Objective () noexcept=default

real_t	value (const HashVector &location)
	Evaluate the objective at the given `location`. More...

virtual long	num_variables () const noexcept=0

void	diag_preconditioner (const HashVector &location, Eigen::Ref< DenseRealVector > target)
	Get precondition to be used in CG optimization. More...

void	project_to_line (const HashVector &location, const DenseRealVector &direction)
	creates a function g such that `g(a) = objective(location + a * direction)` Use `lookup_on_line()` to evaluate `g`. More...

virtual real_t	lookup_on_line (real_t position)=0
	Looks up the value of the objective on the line defined by the last call to `project_to_line()`. More...

virtual void	declare_vector_on_last_line (const HashVector &location, real_t t)
	State that the given vector corresponds to a certain position on the line of the last line search. More...

void	gradient_at_zero (Eigen::Ref< DenseRealVector > target)
	Gets the gradient for location zero. More...

void	gradient (const HashVector &location, Eigen::Ref< DenseRealVector > target)
	Evaluate the gradient at `location`. More...

void	hessian_times_direction (const HashVector &location, const DenseRealVector &direction, Eigen::Ref< DenseRealVector > target)
	Calculates the product of the Hessian matrix at `location` with `direction`. More...

void	gradient_and_pre_conditioner (const HashVector &location, Eigen::Ref< DenseRealVector > gradient, Eigen::Ref< DenseRealVector > pre)
	Combines the calculation of gradient and pre-conditioner, which may be more efficient in some cases. More...

Public Member Functions inherited from dismec::stats::Tracked
	Tracked ()
	Default constructor, creates the internal `stats::StatisticsCollection`. More...

void	register_stat (const std::string &name, std::unique_ptr< Statistics > stat)
	Registers a tracker for the statistics `name`. More...

std::shared_ptr< StatisticsCollection >	get_stats () const
	Gets an ownership-sharing reference to the `StatisticsCollection`. More...

Private Member Functions
virtual real_t	value_unchecked (const HashVector &location)=0

virtual void	gradient_unchecked (const HashVector &location, Eigen::Ref< DenseRealVector > target)=0

virtual void	hessian_times_direction_unchecked (const HashVector &location, const DenseRealVector &direction, Eigen::Ref< DenseRealVector > target)=0

virtual void	gradient_at_zero_unchecked (Eigen::Ref< DenseRealVector > target)

virtual void	gradient_and_pre_conditioner_unchecked (const HashVector &location, Eigen::Ref< DenseRealVector > gradient, Eigen::Ref< DenseRealVector > pre)

virtual void	diag_preconditioner_unchecked (const HashVector &location, Eigen::Ref< DenseRealVector > target)

virtual void	project_to_line_unchecked (const HashVector &location, const DenseRealVector &direction)=0

Additional Inherited Members
Protected Member Functions inherited from dismec::stats::Tracked
	~Tracked ()
	Non-virtual destructor. Declared protected, so we don't accidentally try to do a polymorphic delete. More...

template<class T >
void	record (stat_id_t stat, T &&value)
	Record statistics. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

void	declare_stat (stat_id_t index, StatisticMetaData meta)
	Declares a new statistics. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

void	declare_tag (tag_id_t index, std::string name)
	Declares a new tag. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

template<class... Args>
void	set_tag (tag_id_t tag, long value)
	Set value of tag. This function just forwards all its arguments to the internal `StatisticsCollection`. More...

template<class... Args>
auto	make_timer (stat_id_t id, Args... args)
	Creates a new `ScopeTimer` using `stats::record_scope_time`. More...

Detailed Description

Class that models an optimization objective.

Currently, we expect objectives to be continuous with Lipschitz-continuous derivative. The following operations need to be implemented by an objective:

num_variables(): Gets the number of variables this objective expects for its location parameters.
value(): Evaluates the objective at the given location.
gradient(): Calculates the gradient at the given location.
hessian_times_direction(): Calculates the product of the Hessian at the given location with the supplied direction vector.

Additionally, the functions diag_preconditioner() and gradient_at_zero() do have default implementations, but are expected to be implementable more usefully (for the preconditioner) or more efficiently (for the gradient at zero) in the derived classes.

These methods take the location parameter using a HashVector type. This is so that it is possible to cache certain computations inside the objective, i.e. if multiple hessian_times_direction() calculations are required for the same location but different direction. We do not use the same Objective instance from multiple threads, so the internal caching does not need to take any synchronization into account. This is also indicated by the fact that the calculation function, which do not change the visible state of the objective, are not marked as const. Note that, though it may be unlikely that we ask for the value() (or gradient or hessian_times_direction()) of the same location twice (then the caller should do the caching), caching can be used to extract computations that are common between value(), gradient(), hessian_times_direction(), and project_to_line().

Definition at line 41 of file objective.h.

Constructor & Destructor Documentation

◆ Objective()

Objective::Objective ( )

Definition at line 24 of file objective.cpp.

References dismec::stats::Tracked::declare_stat(), anonymous_namespace{objective.cpp}::STAT_PERF_GRAD_AND_PRE, anonymous_namespace{objective.cpp}::STAT_PERF_GRAD_AT_ZERO, anonymous_namespace{objective.cpp}::STAT_PERF_GRADIENT, anonymous_namespace{objective.cpp}::STAT_PERF_HESSIAN, anonymous_namespace{objective.cpp}::STAT_PERF_PRECONDITIONER, anonymous_namespace{objective.cpp}::STAT_PERF_PROJ_TO_LINE, and anonymous_namespace{objective.cpp}::STAT_PERF_VALUE.

◆ ~Objective()

virtual dismec::objective::Objective::~Objective ( )

virtualdefaultnoexcept

Member Function Documentation

◆ declare_vector_on_last_line()

virtual void dismec::objective::Objective::declare_vector_on_last_line	(	const HashVector &	location,
		real_t	t
	)

inlinevirtual

State that the given vector corresponds to a certain position on the line of the last line search.

This function is a pure optimization hint. It is used in the following scenario: If several computations need the product of weight vector w and feature matrix X, then we can compute this product only once and use a cached value for all later invocations. This can be done by comparing the vector hashes. However, as soon as a vector is modified, these hashes are invalidated. To do an efficient line search over ‘w’ = w + t d, we also cache the value ofX d, so thatX w' = X w + t X d. This function then declares that the vector given inlocationcorresponds tow + t d, wherewandd are the arguments passed to the last call ofproject_to_line()`.

Todo:: improve this interface, together with project_to_line, to be less error prone!

Reimplemented in dismec::objective::LinearClassifierBase, and dismec::objective::DenseAndSparseLinearBase.

Definition at line 100 of file objective.h.

◆ diag_preconditioner()

void Objective::diag_preconditioner	(	const HashVector &	location,
		Eigen::Ref< DenseRealVector >	target
	)

Get precondition to be used in CG optimization.

Calculates the diagonal of a preconditioning matrix for the Hessian and places it in target. The default implementation returns a unit vector, i.e. does not result in any preconditioning.

Parameters

location	The location where the corresponding hessian would be calculated.
target	Pre-allocated vector where the diagonal entries of the Hessian will be placed.

Definition at line 43 of file objective.cpp.

References ALWAYS_ASSERT_EQUAL, diag_preconditioner_unchecked(), dismec::stats::Tracked::make_timer(), num_variables(), and anonymous_namespace{objective.cpp}::STAT_PERF_PRECONDITIONER.

Referenced by TEST_CASE(), and anonymous_namespace{generic_linear.cpp}::test_equivalence().

◆ diag_preconditioner_unchecked()

void Objective::diag_preconditioner_unchecked	(	const HashVector &	location,
		Eigen::Ref< DenseRealVector >	target
	)

privatevirtual

The function that does the actual computation. This is called in diag_preconditioner() after the arguments have been validated. The default implementation returns ones.

Reimplemented in dismec::objective::PointWiseRegularizer< CRTP >, dismec::objective::PointWiseRegularizer< HuberRegularizer >, dismec::objective::PointWiseRegularizer< ElasticNetRegularizer >, dismec::objective::PointWiseRegularizer< SquaredNormRegularizer >, dismec::objective::LinearClassifierImpBase< Derived >, dismec::objective::LinearClassifierImpBase< Regularized_SquaredHingeSVC >, dismec::objective::GenericLinearClassifier, and dismec::objective::DenseAndSparseLinearBase.

Definition at line 54 of file objective.cpp.

Referenced by diag_preconditioner(), and gradient_and_pre_conditioner_unchecked().

◆ gradient()

void Objective::gradient	(	const HashVector &	location,
		Eigen::Ref< DenseRealVector >	target
	)

Evaluate the gradient at location.

The result will be placed in target. The location is passed as a HashVector to allow caching some calculations (See detail description of Objective).

Parameters

location	The location (=value of the variables) at which the gradient should be computed. This vector needs to have `location->size() == num_variables()`, unless `num_variables() == -1`.
target	Reference to a vector in which the result will be placed. Needs to have `target.size() == location->size()`.

Definition at line 96 of file objective.cpp.

References ALWAYS_ASSERT_EQUAL, gradient_unchecked(), dismec::stats::Tracked::make_timer(), num_variables(), and anonymous_namespace{objective.cpp}::STAT_PERF_GRADIENT.

Referenced by gradient_and_pre_conditioner(), dismec::objective::Regularized_SquaredHingeSVC::gradient_and_pre_conditioner_imp(), dismec::objective::Regularized_SquaredHingeSVC::gradient_and_pre_conditioner_tpl(), gradient_and_pre_conditioner_unchecked(), dismec::objective::DenseAndSparseLinearBase::gradient_and_pre_conditioner_unchecked(), dismec::objective::LinearClassifierImpBase< Derived >::gradient_and_pre_conditioner_unchecked(), dismec::objective::DenseAndSparseMargin< MarginFunction, SparseRegFunction, DenseRegFunction >::regularization_gradient(), dismec::objective::DenseAndSparseMargin< MarginFunction, SparseRegFunction, DenseRegFunction >::regularization_gradient_at_zero(), TEST_CASE(), anonymous_namespace{generic_linear.cpp}::test_equivalence(), and anonymous_namespace{regularizers_imp.cpp}::verify_bias().

◆ gradient_and_pre_conditioner()

void Objective::gradient_and_pre_conditioner	(	const HashVector &	location,
		Eigen::Ref< DenseRealVector >	gradient,
		Eigen::Ref< DenseRealVector >	pre
	)

Combines the calculation of gradient and pre-conditioner, which may be more efficient in some cases.

See gradient() and get_diag_preconditioner() for details. The default implementation just calls these two functions.

Parameters

location	Value of the weights for which to get gradient and pre-conditioner.
gradient	Reference to a vector in which the gradient will be placed.
pre	Reference to a vector in which the pre-conditioning will be placed.

Definition at line 58 of file objective.cpp.

References ALWAYS_ASSERT_EQUAL, gradient(), gradient_and_pre_conditioner_unchecked(), dismec::stats::Tracked::make_timer(), num_variables(), and anonymous_namespace{objective.cpp}::STAT_PERF_GRAD_AND_PRE.

Referenced by anonymous_namespace{generic_linear.cpp}::test_equivalence().

◆ gradient_and_pre_conditioner_unchecked()

void Objective::gradient_and_pre_conditioner_unchecked	(	const HashVector &	location,
		Eigen::Ref< DenseRealVector >	gradient,
		Eigen::Ref< DenseRealVector >	pre
	)

privatevirtual

The function that does the actual computation. This is called in gradient_and_pre_conditioner() after the arguments have been validated. The default implementation sucessively calls gradient() and diag_preconditioner().

Reimplemented in dismec::objective::LinearClassifierImpBase< Derived >, dismec::objective::LinearClassifierImpBase< Regularized_SquaredHingeSVC >, dismec::objective::GenericLinearClassifier, and dismec::objective::DenseAndSparseLinearBase.

Definition at line 74 of file objective.cpp.

References diag_preconditioner_unchecked(), gradient(), and gradient_unchecked().

Referenced by gradient_and_pre_conditioner().

◆ gradient_at_zero()

void Objective::gradient_at_zero ( Eigen::Ref< DenseRealVector > target )

Gets the gradient for location zero.

This operation can sometimes be implemented to be much faster than the actual gradient. Therefore, we provide the option that Objective classes overwrite this function. Note that, if this function is not implemented in a derived class, the default implementation may be rather slow, since it dynamically creates the corresponding zero vector.

Definition at line 82 of file objective.cpp.

References ALWAYS_ASSERT_EQUAL, gradient_at_zero_unchecked(), dismec::stats::Tracked::make_timer(), num_variables(), and anonymous_namespace{objective.cpp}::STAT_PERF_GRAD_AT_ZERO.

Referenced by TEST_CASE(), and anonymous_namespace{generic_linear.cpp}::test_equivalence().

◆ gradient_at_zero_unchecked()

void Objective::gradient_at_zero_unchecked ( Eigen::Ref< DenseRealVector > target )

privatevirtual

The function that does the actual computation. This is called in gradient_at_zero() after the argument has been validated. The default implementation is rather inefficient and creates a new temporary zero vector.

Reimplemented in dismec::objective::PointWiseRegularizer< CRTP >, dismec::objective::PointWiseRegularizer< HuberRegularizer >, dismec::objective::PointWiseRegularizer< ElasticNetRegularizer >, dismec::objective::PointWiseRegularizer< SquaredNormRegularizer >, dismec::objective::LinearClassifierImpBase< Derived >, dismec::objective::LinearClassifierImpBase< Regularized_SquaredHingeSVC >, dismec::objective::GenericLinearClassifier, and dismec::objective::DenseAndSparseLinearBase.

Definition at line 90 of file objective.cpp.

References gradient_unchecked().

Referenced by gradient_at_zero().

◆ gradient_unchecked()

virtual void dismec::objective::Objective::gradient_unchecked	(	const HashVector &	location,
		Eigen::Ref< DenseRealVector >	target
	)

privatepure virtual

The function that does the actual gradient computation. This is called in gradient() after the arguments have been validated.

Implemented in anonymous_namespace{minimizer.cpp}::MockObjective, dismec::objective::PointWiseRegularizer< CRTP >, dismec::objective::PointWiseRegularizer< HuberRegularizer >, dismec::objective::PointWiseRegularizer< ElasticNetRegularizer >, dismec::objective::PointWiseRegularizer< SquaredNormRegularizer >, dismec::objective::LinearClassifierImpBase< Derived >, dismec::objective::LinearClassifierImpBase< Regularized_SquaredHingeSVC >, dismec::objective::GenericLinearClassifier, and dismec::objective::DenseAndSparseLinearBase.

Referenced by gradient(), gradient_and_pre_conditioner_unchecked(), and gradient_at_zero_unchecked().

◆ hessian_times_direction()

void Objective::hessian_times_direction	(	const HashVector &	location,
		const DenseRealVector &	direction,
		Eigen::Ref< DenseRealVector >	target
	)

Calculates the product of the Hessian matrix at location with direction.

The result will be placed in target. The location is passed as a HashVector to allow caching some calculations (See detail description of Objective). We currently don't support any caching for direction, as this parameter is expected to change for each invocation.

Parameters

location	Where should the Hessian be calculated.
direction	Vector to multiply with the Hessian.
target	Reference to a buffer where the resulting vector will be placed.

Definition at line 107 of file objective.cpp.

References ALWAYS_ASSERT_EQUAL, hessian_times_direction_unchecked(), dismec::stats::Tracked::make_timer(), num_variables(), and anonymous_namespace{objective.cpp}::STAT_PERF_HESSIAN.

Referenced by TEST_CASE(), anonymous_namespace{generic_linear.cpp}::test_equivalence(), and anonymous_namespace{regularizers_imp.cpp}::verify_bias().

◆ hessian_times_direction_unchecked()

virtual void dismec::objective::Objective::hessian_times_direction_unchecked	(	const HashVector &	location,
		const DenseRealVector &	direction,
		Eigen::Ref< DenseRealVector >	target
	)

privatepure virtual

The function that does the actual computation. This is called in hessian_times_direction() after the arguments have been validated.

Implemented in anonymous_namespace{minimizer.cpp}::MockObjective, dismec::objective::PointWiseRegularizer< CRTP >, dismec::objective::PointWiseRegularizer< HuberRegularizer >, dismec::objective::PointWiseRegularizer< ElasticNetRegularizer >, dismec::objective::PointWiseRegularizer< SquaredNormRegularizer >, dismec::objective::LinearClassifierImpBase< Derived >, dismec::objective::LinearClassifierImpBase< Regularized_SquaredHingeSVC >, dismec::objective::GenericLinearClassifier, and dismec::objective::DenseAndSparseLinearBase.

Referenced by hessian_times_direction().

◆ lookup_on_line()

virtual real_t dismec::objective::Objective::lookup_on_line ( real_t position )

pure virtual

Looks up the value of the objective on the line defined by the last call to project_to_line().

Parameters

position The location where the objective is calculated.

Returns: The value of objective(location + position * direction), where location and direction are the vectors passed to the last call of project_to_line().

Attention: This function may use results cached project_to_line(), so it has to be called after a call to that function. A new call to project_to_line() will change the line which is evaluated.

Referenced by anonymous_namespace{regularizers_imp.cpp}::verify_line_search().

◆ num_variables()

virtual long dismec::objective::Objective::num_variables ( ) const

pure virtualnoexcept

Gets the number of variables this objective expects. May return -1 if the objective is agnostic to the number of variables, e.g. for regularizers.

Implemented in anonymous_namespace{minimizer.cpp}::MockObjective, dismec::objective::LinearClassifierBase, dismec::objective::DenseAndSparseLinearBase, dismec::objective::PointWiseRegularizer< CRTP >, dismec::objective::PointWiseRegularizer< HuberRegularizer >, dismec::objective::PointWiseRegularizer< ElasticNetRegularizer >, and dismec::objective::PointWiseRegularizer< SquaredNormRegularizer >.

Referenced by diag_preconditioner(), gradient(), gradient_and_pre_conditioner(), gradient_at_zero(), hessian_times_direction(), project_to_line(), and value().

◆ project_to_line()

void Objective::project_to_line	(	const HashVector &	location,
		const DenseRealVector &	direction
	)

creates a function g such that g(a) = objective(location + a * direction) Use lookup_on_line() to evaluate g.

This prepares the evaluation of the objective along a line. The purpose is that in many cases, this allows for much faster computations, since we can cache certain results like the product of features and direction vector.

Definition at line 124 of file objective.cpp.

References ALWAYS_ASSERT_EQUAL, dismec::stats::Tracked::make_timer(), num_variables(), project_to_line_unchecked(), and anonymous_namespace{objective.cpp}::STAT_PERF_PROJ_TO_LINE.

Referenced by anonymous_namespace{regularizers_imp.cpp}::verify_line_search().

◆ project_to_line_unchecked()

virtual void dismec::objective::Objective::project_to_line_unchecked	(	const HashVector &	location,
		const DenseRealVector &	direction
	)

privatepure virtual

The function that does the actual computation. This is called in project_to_line() after the arguments have been validated.

Referenced by project_to_line().

◆ value()

real_t Objective::value ( const HashVector & location )

Evaluate the objective at the given location.

The location is passed as a HashVector to allow caching some calculations (see detail description of Objective).

Parameters

location The location (=value of the variables) at which the objective should be evaluated. This vector needs to have location->size() == num_variables().

Returns: The objective's value.

Definition at line 35 of file objective.cpp.

References ALWAYS_ASSERT_EQUAL, dismec::stats::Tracked::make_timer(), num_variables(), anonymous_namespace{objective.cpp}::STAT_PERF_VALUE, and value_unchecked().

Referenced by TEST_CASE(), anonymous_namespace{generic_linear.cpp}::test_equivalence(), anonymous_namespace{regularizers_imp.cpp}::verify_bias(), and anonymous_namespace{regularizers_imp.cpp}::verify_line_search().

◆ value_unchecked()

virtual real_t dismec::objective::Objective::value_unchecked ( const HashVector & location )

privatepure virtual

The function that does the actual value computation. This is called in value() after the argument has been validated.

Referenced by value().

The documentation for this class was generated from the following files:

src/objective/objective.h
src/objective/objective.cpp

Public Member Functions

Private Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ Objective()

◆ ~Objective()

Member Function Documentation

◆ declare_vector_on_last_line()

◆ diag_preconditioner()

◆ diag_preconditioner_unchecked()

◆ gradient()

◆ gradient_and_pre_conditioner()

◆ gradient_and_pre_conditioner_unchecked()

◆ gradient_at_zero()

◆ gradient_at_zero_unchecked()

◆ gradient_unchecked()

◆ hessian_times_direction()

◆ hessian_times_direction_unchecked()

◆ lookup_on_line()

◆ num_variables()

◆ project_to_line()

◆ project_to_line_unchecked()

◆ value()

◆ value_unchecked()