DiSMEC++
Test List
Member TEST_CASE ("minimizer base class minimize")
These tests verify that the Minimizer base class minimize performs the correct parameter checking and result passing. These are:
  • verify the correct size of the initial value
  • time the duration for the minimization
  • pass the result through unchanged
Member TEST_CASE ("get dense weights errors")
We check the that the following result in an exception when getting the label weights:
  • supplying a target vector of non-matching size
  • accessing an invalid label
Member TEST_CASE ("set dense weights errors")
We check the that the following result in an exception when setting the label weights:
  • supplying a target vector of non-matching size
  • accessing an invalid label
Member TEST_CASE ("get/set dense weights round-trip")
This test verifies that setting and getting weights for labels round-trips.
Member TEST_CASE ("predict_scores checks")
This test verifies that Model::predict_scores() throws errors if the matrices have wrong shapes
Member TEST_CASE ("partial model")
This test verifies the partial model interface part of the Model base class. We use DenseModel as the instantiation. This doubles as a test for the DenseModel constructors.
Member TEST_CASE ("DenseModel ctor consistency")
This test verifies that the constructor of DenseModel throws an error if the model specifications are impossible.
Member TEST_CASE ("precision_at_k")
Validate precision at k by using a few examples. Note that this test validates that the calculations are correct, but not that the code will also work when called from multiple threads.
Member TEST_CASE ("abandonment_at_k")
Validate abandonment at k by using a few examples. Note that this test validates that the calculations are correct, but not that the code will also work when called from multiple threads.
Member TEST_CASE ("read into buffers bounds checks")
This test verifies that read_into_buffers performs correct bounds checking for its features, labels and examples. We check that both overflow (for examples, labels, features) and underflow (for labels, features) are detected, and that these take into account whether indexing is one-based or zero-based
Member TEST_CASE ("HashVector content")
This checks that construction and assignment result in the correct content in the vector.
Member TEST_CASE ("HashVector versioning")
This checks that non-mutation operations leave the hash value as is, but mutation operations change the value.
Member TEST_CASE ("HashVector uniqueness")
This checks that two different HashVector objects always have different hash.
Member TEST_CASE ("uninitialized VectorHash")
This checks that the uninitialized VectorHash compares unequal to the hash value of a vector.
Member TEST_CASE ("HyperParameterBase")
This test case verifies that direct hyperparamters work as expected for classes derived from HyperParameterBase. We check that setting and getting correctly read/write the parameters, and that type mismatches and name mismatches throw errors.
Member TEST_CASE ("nested hyper parameter object")
This test verifies that registering a sub-object results in correct getters and setters for the sub-object values.
Member TEST_CASE ("wrong subtype causes error")
This verifies that supplying a ptr-to-member to a different class than this results in an exception.
Member TEST_CASE ("HyperParameters")
Here we test the HyperParameters objects. We check
  • throw on getting unknown name
  • read/write of values
  • applying to HyperParameterBase object with corresponding type and name checks.
Member TEST_CASE ("HyperParameters nested")
This test ensures that the HyperParameters::apply function can cope with nested objects.
Member TEST_CASE ("read_sparse_prediction check")
This test case verifies that (certain) erroneous prediction files are rejected.
Member TEST_CASE ("read dense vector from text")
This checks that reading a dense vector from text works.
Member TEST_CASE ("parse sparse vector")
This test checks that XMC feature parsing of valid lines gives the correct result. We check robustness to
  • leading spaces
  • space vs tab
  • scientific notation
  • space after separator
  • space after last feature
Member TEST_CASE ("parse sparse vector errors")
This test checks that XMC sparse vector parsing of invalid lines results in an error. We check the following formatting errors:
  • non-integer index
  • not-a-number
  • missing value with and without space/with and without colon :
  • wrong separator
  • space in front of colon.
  • missing feature index
Member TEST_CASE ("binary dump/load")
This test case checks that binary dump and load round-trip data
Member TEST_CASE ("parse valid header")
Checks that valid XMC headers are parsed correctly
Member TEST_CASE ("parse invalid header")
Check that invalid XMC headers are causing an exception. The headers are invalid if either the number of data does not match, of if any of the supplied counts are non-positive.
Member TEST_CASE ("partial model writer verifier")
This test case checks that PartialModelSaver verifies that the partial models it receives are compatible. This includes checking that the number of total labels/features match. To check that the error condition we want to check for is really the problem, we first try the add_model call with an invalid partial model, and secondly call it with a corrected version. If the corrected version also fails, we know that our check did not work, or that the first call did throw but still added the partial model to its model list. We assume that label overlap tests are correctly performed; this is tested in the unit test for insert_sub_file.
Member TEST_CASE ("save_load_sparse_predictions")
Checks in an example that sparse predictions writes as expected. It also checks that the reading back gives the same result.
Member TEST_CASE ("save_sparse_predictions checking")
The test case checks that save_sparse_predictions verifies the compatibility of value and index matrices.
Member TEST_CASE ("check write dense vector")
This checks that vector writing works. It in particular verifies the special cases of 0 and 1-element vectors
Member TEST_CASE ("save_dense_predictions")
Checks in an example that dense predictions writes as expected.
Member TEST_CASE ("xmc round trip")
This test case verifies that reading and writing XMC files round-trips.
Member TEST_CASE ("save/load weights as plain text")
In this test we check that weight matrices are saved correctly in dense and sparse txt formats. This also verifies that the partial model settings are used correctly, i.e. that we do not try to save weight vectors that are not actually present in the partial model. We then check that loading from the saved strings reproduces the original weights.
Member TEST_CASE ("parse valid header")
Checks that valid XMC headers are parsed correctly
Member TEST_CASE ("parse invalid header")
Check that invalid XMC headers are causing an exception. The headers are invalid if either the number of data does not match, of if any of the supplied counts are non-positive.
Member TEST_CASE ("count features")
Checks that the number of features are counted correctly. Since feature counting is only a very simple approximation, that assumes correctly formatted input data, the only thing we test here is that 1) the counts are correct and 2) empty lines and comments are skipped as they are supposed to.
Member TEST_CASE ("parse labels errors")
This test checks that XMC label parsing of incorrectly formatted lines results in errors. Note that some formatting problems (such as a space before a comma: 5 ,1 10:3.0) are perfectly legal from a label parsing point of view, but will result in an error when the subsequent feature parsing happens. We currently check the following error conditions:
  • trailing comma
  • not-a-number
  • not an integer
  • wrong separator between labels
Member TEST_CASE ("parse labels")
This test checks that XMC label parsing of correctly formatted lines gives the desired results. We check that whitespace is robust to space/tab, and that the empty labels situation is correctly recognised.