Uploaded image for project: 'DRP 1D pipeline'
  1. DRP 1D pipeline
  2. PIPE1D-14

Run 1D-DRP for the resulting products from 2D in the end-to-end processing test

    Details

      Description

      For the first step of the end-to-end processing test, run 1D-DRP for the resulting products from 2D-DRP, and compare the results to the input spectra of the existing simulated 2D image processed by 2D-DRP.

        Attachments

          Issue Links

            Activity

            kiyoto.yabe Kiyoto Yabe created issue -
            kiyoto.yabe Kiyoto Yabe made changes -
            Field Original Value New Value
            Link This issue is blocked by INFRA-278 [ INFRA-278 ]
            kiyoto.yabe Kiyoto Yabe made changes -
            Link This issue blocks PIPE2D-608 [ PIPE2D-608 ]
            kiyoto.yabe Kiyoto Yabe made changes -
            Assignee Kiyoto Yabe [ kiyoto.yabe ]
            kiyoto.yabe Kiyoto Yabe made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            Confirmed that the test script to run the example datapack included in the pipeline can be processed.

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - Confirmed that the test script to run the example datapack included in the pipeline can be processed.
            kiyoto.yabe Kiyoto Yabe made changes -
            Attachment example_0x37.png [ 12827 ]
            kiyoto.yabe Kiyoto Yabe made changes -
            Attachment example_0x37.png [ 12827 ]
            kiyoto.yabe Kiyoto Yabe made changes -
            Attachment example_0x37.png [ 12828 ]
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            This is an example result for objId=55. 1D spectrum shows [OII], Hb, [OIII] at z=0.853 clearly, but DRP1D returns the best-fit z of 0.08. Needs to further investigation...

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - This is an example result for objId=55. 1D spectrum shows [OII] , Hb, [OIII] at z=0.853 clearly, but DRP1D returns the best-fit z of 0.08. Needs to further investigation...
            kiyoto.yabe Kiyoto Yabe made changes -
            Resolution Done [ 10000 ]
            Status In Progress [ 3 ] In Review [ 10001 ]
            Hide
            pychabau Pierre-Yves CHABAUD added a comment -

            Hi Kiyoto,

            Sorry to introduce inside your ticket. If you want any help on this spectrum, we'll be glad to contribuate.

            Show
            pychabau Pierre-Yves CHABAUD added a comment - Hi Kiyoto, Sorry to introduce inside your ticket. If you want any help on this spectrum, we'll be glad to contribuate.
            kiyoto.yabe Kiyoto Yabe made changes -
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            I confirmed that it returns a good redshift result if we limit the redshift range very narrowly, like  

                "redshiftrange": [
                    "0.8",
                    "0.9"
                ],
            

            Pierre-Yves CHABAUD Could you run drp1d for the spectrum below in your side? I also appreciate if you tell me a good parameter configuration?

            pfsObject-001-00000-0,0-0000000000000037-002-0xe0241444a9af118c.fits

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - I confirmed that it returns a good redshift result if we limit the redshift range very narrowly, like   "redshiftrange" : [ "0.8" , "0.9" ], Pierre-Yves CHABAUD  Could you run drp1d for the spectrum below in your side? I also appreciate if you tell me a good parameter configuration? pfsObject-001-00000-0,0-0000000000000037-002-0xe0241444a9af118c.fits
            Hide
            pychabau Pierre-Yves CHABAUD added a comment -

            I observe a non physical high pick value at pixel 346. By reducing lambda range to

            "lambdarange":["3800","13000"]
            

            the pick is ignored and the right redshift appears in the candidates list. However, the right redshift remains in lower probability position. We are still investigating on this case.

            Show
            pychabau Pierre-Yves CHABAUD added a comment - I observe a non physical high pick value at pixel 346. By reducing lambda range to "lambdarange" :[ "3800" , "13000" ] the pick is ignored and the right redshift appears in the candidates list. However, the right redshift remains in lower probability position. We are still investigating on this case.
            vlebrun vlebrun made changes -
            Attachment unidentified_noise_spike.png [ 12904 ]
            Hide
            vlebrun vlebrun added a comment -

            Dear Kyoto,

            I think I have identified the problem : there is a strong noise peak at 6330 A which is not identified in the variance. This ear is thus used as Ly-alpha, CIII and OII as first solution since it is very strong and has thus more weight than the true emission lines. 

            Show
            vlebrun vlebrun added a comment - Dear Kyoto, I think I have identified the problem : there is a strong noise peak at 6330 A which is not identified in the variance. This ear is thus used as Ly-alpha, CIII and OII as first solution since it is very strong and has thus more weight than the true emission lines. 
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            Thank you for letting me know your results vlebrun & Pierre-Yves CHABAUD (I should have replied earlier). So, in this case, the best redshift is ~4, right? I could not reproduce the same result in my environment. Could you send me your configuration file in json format?

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - Thank you for letting me know your results vlebrun  & Pierre-Yves CHABAUD  (I should have replied earlier). So, in this case, the best redshift is ~4, right? I could not reproduce the same result in my environment. Could you send me your configuration file in json format?
            Hide
            pychabau Pierre-Yves CHABAUD added a comment -

            Hi Kiyoto Yabe , by looking at this issue, we've just found a bug in the pipeline on parameters file overloading ( https://pfspipe.ipmu.jp/jira/browse/PIPE1D-22 ). I propose to wait for fixing it, before going further. Sorry for the delay.

            Show
            pychabau Pierre-Yves CHABAUD added a comment - Hi Kiyoto Yabe , by looking at this issue, we've just found a bug in the pipeline on parameters file overloading ( https://pfspipe.ipmu.jp/jira/browse/PIPE1D-22 ). I propose to wait for fixing it, before going further. Sorry for the delay.
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            Pierre-Yves CHABAUD  OK, thank you for letting me know 

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - Pierre-Yves CHABAUD   OK, thank you for letting me know 
            kiyoto.yabe Kiyoto Yabe made changes -
            Link This issue blocks PIPE2D-606 [ PIPE2D-606 ]
            kiyoto.yabe Kiyoto Yabe made changes -
            Link This issue is blocked by PIPE1D-20 [ PIPE1D-20 ]
            kiyoto.yabe Kiyoto Yabe made changes -
            kiyoto.yabe Kiyoto Yabe made changes -
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            I'm investigating the feature at around 633 nm, which was reproduced with the latest 2D pipeline (`w.2020.39`). As shown in the plots below, this seems to come out after making `pfsMerged` spectrum in only `visit=24`. The plots are for only 2 objects (objId=0x37 and 0x12), but I see the same feature for other fibers. price do you have any idea?

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - I'm investigating the feature at around 633 nm, which was reproduced with the latest 2D pipeline (`w.2020.39`). As shown in the plots below, this seems to come out after making `pfsMerged` spectrum in only `visit=24`. The plots are for only 2 objects (objId=0x37 and 0x12), but I see the same feature for other fibers. price  do you have any idea?
            Hide
            price price added a comment -

            I am investigating (PIPE2D-633; the number of the ticket is completely coincidental!).

            Show
            price price added a comment - I am investigating ( PIPE2D-633 ; the number of the ticket is completely coincidental!).
            hassan hassan made changes -
            Link This issue relates to PIPE2D-633 [ PIPE2D-633 ]
            price price made changes -
            Link This issue relates to PIPE2D-635 [ PIPE2D-635 ]
            kiyoto.yabe Kiyoto Yabe made changes -
            Attachment example_0x37_20201012.png [ 13117 ]
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            This is the same object as before but with a fix of PIPE2D-635

            The 1D output looks reasonable.

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - This is the same object as before but with a fix of  PIPE2D-635 The 1D output looks reasonable.
            Hide
            vlebrun vlebrun added a comment -

            good to see we are able to measure the redshift on such strong emission lines...

            Show
            vlebrun vlebrun added a comment - good to see we are able to measure the redshift on such strong emission lines...
            Hide
            kiyoto.yabe Kiyoto Yabe added a comment -

            I just confirmed that the processing is successfully done with the master branch after PIPE2D-635 is merged. So, I will close this ticket.

            Show
            kiyoto.yabe Kiyoto Yabe added a comment - I just confirmed that the processing is successfully done with the master branch after  PIPE2D-635  is merged. So, I will close this ticket.
            kiyoto.yabe Kiyoto Yabe made changes -
            Status In Review [ 10001 ] Done [ 10002 ]

              People

              • Assignee:
                kiyoto.yabe Kiyoto Yabe
                Reporter:
                kiyoto.yabe Kiyoto Yabe
              • Votes:
                0 Vote for this issue
                Watchers:
                Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: