[PIPE2D-31] Produce summary of test coverage Created: 18/Jun/16  Updated: 31/Aug/16  Resolved: 31/Aug/16

Status: Done
Project: DRP 2-D Pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: rhl Assignee: aritter
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
is blocked by PIPE2D-32 Create tests for SpectrumSet Done
Sprint: 2014-13, 2014-14, 2014-15, 2014-16
Reviewers: rhl

 Description   

Please go over all the tests and summarise what they tell us.



 Comments   
Comment by aritter [ 01/Jul/16 ]

There currently are unit tests for FiberTrace/FiberTraceSet and Spectrum/SpectrumSet.
The testing method is the Black Box Method, meaning that the tester only knows the interface and tests the functions provided by the interface, without knowledge of what functions are called inside the tested function.
Tested are:

  • the constructors: - test that an objected is created
  • test that the private variables are set according to the constructor,
    e.g. if one creates a FiberTrace of width 5 and height 100 pix, the
    image, mask, and variance arrays should be 5x100 pix, all
    "_is...Set" (_isTraceSet, _isProfileSet,
    _isFiberTraceProfileFittingControlSet) variables are false
  • get/set methods: - test that we can get and set private variables using these methods,
    and if set the private variable is actually changed
  • other methods provided by the class: - test that they do what they are supposed to do
  • free functions manipulating objects (defined in the header file of the class):
  • test that they do what they are supposed to do e.g.
    + PTR(FiberTraceSet<ImageT, MaskT, VarianceT>) math::findAndTraceApertures:
  • test that the method returns a shared pointer to a FiberTraceSet containing the
    expected number of FiberTraces with a height within the expected limits
    + SpectrumSet FiberTraceSet::extractAllTracesFromProfile()

Sets: - test "add...", "get...", "set...", "erase"

Tests will never be complete as it would be nearly impossible to really test ALL possible input parameter combinations, but at least I am very confident that the tested functions (and functions they call) are doing the right thing. Another test is to execute the python command line tasks "reduceArc" and "reduceArcRefSpec". Both tasks find and trace the FiberTraces in a Flat, extract them, calculate the profile, "optimally" extract the spectra from an Arc, compare them to a line list, and wavelength calibrate the Arc spectra (so called "Incremental Testing"). The only difference between the 2 tasks is that one ("reduceArc") creates the line list from the wavelengths of the lines and the simulated map wavelength<->pixel. The other task ("reduceArcRefSpec") uses a previously identified reference Spectrum to stretch and cross-correlate it with respect to the spectra to be identified, and creates an approximate line list from that.
Tested are the butler (deliver the Flat belonging to this Arc), FindAndTraceAperturesTask (python task), CreateFlatFiberTraceProfileTask (python), ExtractSpectraTask (python), math::createLineList (C++, calculates the pixel position for the line list from the simulated map pixel<=>wavelength), math::stretchAndCrossCorrelateSpec (C++, does what it says plus creates the line list for each individual Spectrum), Spectrum.identify (C++, uses the line list to identify and fit the lines, plus fit the dispersion function), write SpArm file (Python + C++, still place-holders in most HDUs).

I guess (more) discussion tomorrow...

Comment by aritter [ 01/Jul/16 ]

Darn, forgot to mention something more (had it in, cut it out to put it somewhere else, got distracted,...). For the SpectrumSet we also test that the length of the Spectrum is equal to the height of the FiberTrace, the RMS of the lines used for fitting the dispersion (I have independently confirmed that the user-defined number of randomly selected lines kept out of the fitting process deliver a similar RMS, but Spectrum currently only stores the RMS of the lines used for fitting), that the wavelength solution is monotonic, and that the wavelength range is within certain limits.

Comment by rhl [ 13/Aug/16 ]

Please transfer anything useful into the tests (no need to document things like "check the setters/getters"). Then OK to merge and close

Comment by aritter [ 31/Aug/16 ]

Nothing to merge, this was a summary of test coverage only.

Generated at Sat Feb 10 15:47:32 JST 2024 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.