[PIPE2D-678] Add visit and date restrictions for generateReductionSpec Created: 10/Dec/20 Updated: 14/Jan/21 Resolved: 14/Jan/21 |
|
| Status: | Done |
| Project: | DRP 2-D Pipeline |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Story | Priority: | Normal |
| Reporter: | price | Assignee: | sogo.mineo |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Sprint: | 2DDRP-2021 A | ||||||||
| Reviewers: | price | ||||||||
| Description |
|
generateReductionSpec.py currently extracts reduction specs for the entire opdb. This will not scale, so we should add options to restrict the list of visits by a visit and/or date range. |
| Comments |
| Comment by sogo.mineo [ 12/Jan/21 ] |
|
I wonder how to express a time span in command lines. First, I think that the end of a time span should be exclusive. `Sps_exposure.time_exp_start` has microsecond accuracy. If the end of a time span is inclusive, it is difficult to iterate over dates without losing unfortunate records in the gaps at midnights. I mean, if a user wants to do this:
somecommand.py --daterange=2021-01-01T00:00:00..2021-01-01T23:59:59
somecommand.py --daterange=2021-01-02T00:00:00..2021-01-02T23:59:59
somecommand.py --daterange=2021-01-03T00:00:00..2021-01-03T23:59:59
:
Then, a record at 2021-01-01T23:59:59.999999 will be lost. I don't know how well SQLite3 works, but I know some poor programs that occasionally output 2021-01-01T23:59:60.000000 by rounding 2021-01-01T23:59:59.9999996 (This is not related to leap seconds). So, it may be useless for the user to write
somecommand.py --daterange=2021-01-01T00:00:00..2021-01-01T23:59:59.999999
somecommand.py --daterange=2021-01-02T00:00:00..2021-01-02T23:59:59.999999
somecommand.py --daterange=2021-01-03T00:00:00..2021-01-03T23:59:59.999999
:
because these commands still cannot catch 2021-01-01T23:59:60.000000. The user cannot simply say 2021-01-01T23:59:60 because python's datetime.datetime does not permit 60 for the second. Second, if I make the end of a time span exclusive, then it is confusing to keep using ... Should I use '/' as in https://en.wikipedia.org/wiki/ISO_8601 ? |
| Comment by price [ 13/Jan/21 ] |
|
What about having separate --begin and --end arguments? Both could default to unset, specifying an open range. |
| Comment by sogo.mineo [ 13/Jan/21 ] |
|
I added - |
| Comment by price [ 14/Jan/21 ] |
|
Very nice work, thanks! |
| Comment by sogo.mineo [ 14/Jan/21 ] |
|
Merged to master. Thank you. |