[PIPE1D-14] Run 1D-DRP for the resulting products from 2D in the end-to-end processing test Created: 19/Jun/20  Updated: 22/Oct/20  Resolved: 22/Oct/20

Status: Done
Project: DRP 1D pipeline
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Normal
Reporter: Kiyoto Yabe Assignee: Kiyoto Yabe
Resolution: Done Votes: 0
Labels: DRP_end-to-end_processing
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File example_0x37_20201012.png     PNG File example_0x37.png     PNG File pfsArm_pfsMerged_v000024_objId0x12.png     PNG File pfsArm_pfsMerged_v000024_objId0x37.png     PNG File pfsArm_pfsMerged_v000025_objId0x12.png     PNG File pfsArm_pfsMerged_v000025_objId0x37.png     File pfsObject-001-00000-0,0-0000000000000037-002-0xe0241444a9af118c.fits     PNG File unidentified_noise_spike.png    
Issue Links:
Blocks
blocks PIPE2D-608 Confirm that 2D-DRP products can be p... Done
blocks PIPE2D-606 Run test scripts for the end-to-end p... Done
is blocked by INFRA-278 Install 2D-DRP & 1D-DRP on a server a... Done
is blocked by PIPE1D-20 Update parameters values with last se... Done
Relates
relates to PIPE2D-633 Debug false lines in extracted spectra Done
relates to PIPE2D-635 Improve FitFocalPlaneTask Done

 Description   

For the first step of the end-to-end processing test, run 1D-DRP for the resulting products from 2D-DRP, and compare the results to the input spectra of the existing simulated 2D image processed by 2D-DRP.



 Comments   
Comment by Kiyoto Yabe [ 20/Jul/20 ]

Confirmed that the test script to run the example datapack included in the pipeline can be processed.

Comment by Kiyoto Yabe [ 17/Aug/20 ]

This is an example result for objId=55. 1D spectrum shows [OII], Hb, [OIII] at z=0.853 clearly, but DRP1D returns the best-fit z of 0.08. Needs to further investigation...

Comment by Pierre-Yves CHABAUD [ 22/Aug/20 ]

Hi Kiyoto,

Sorry to introduce inside your ticket. If you want any help on this spectrum, we'll be glad to contribuate.

Comment by Kiyoto Yabe [ 30/Aug/20 ]

I confirmed that it returns a good redshift result if we limit the redshift range very narrowly, like  

    "redshiftrange": [
        "0.8",
        "0.9"
    ],

Pierre-Yves CHABAUD Could you run drp1d for the spectrum below in your side? I also appreciate if you tell me a good parameter configuration?

pfsObject-001-00000-0,0-0000000000000037-002-0xe0241444a9af118c.fits

Comment by Pierre-Yves CHABAUD [ 31/Aug/20 ]

I observe a non physical high pick value at pixel 346. By reducing lambda range to

"lambdarange":["3800","13000"]

the pick is ignored and the right redshift appears in the candidates list. However, the right redshift remains in lower probability position. We are still investigating on this case.

Comment by vlebrun [ 02/Sep/20 ]

Dear Kyoto,

I think I have identified the problem : there is a strong noise peak at 6330 A which is not identified in the variance. This ear is thus used as Ly-alpha, CIII and OII as first solution since it is very strong and has thus more weight than the true emission lines. 

Comment by Kiyoto Yabe [ 09/Sep/20 ]

Thank you for letting me know your results vlebrun & Pierre-Yves CHABAUD (I should have replied earlier). So, in this case, the best redshift is ~4, right? I could not reproduce the same result in my environment. Could you send me your configuration file in json format?

Comment by Pierre-Yves CHABAUD [ 09/Sep/20 ]

Hi Kiyoto Yabe , by looking at this issue, we've just found a bug in the pipeline on parameters file overloading ( https://pfspipe.ipmu.jp/jira/browse/PIPE1D-22 ). I propose to wait for fixing it, before going further. Sorry for the delay.

Comment by Kiyoto Yabe [ 11/Sep/20 ]

Pierre-Yves CHABAUD  OK, thank you for letting me know 

Comment by Kiyoto Yabe [ 28/Sep/20 ]

I'm investigating the feature at around 633 nm, which was reproduced with the latest 2D pipeline (`w.2020.39`). As shown in the plots below, this seems to come out after making `pfsMerged` spectrum in only `visit=24`. The plots are for only 2 objects (objId=0x37 and 0x12), but I see the same feature for other fibers. price do you have any idea?

Comment by price [ 29/Sep/20 ]

I am investigating (PIPE2D-633; the number of the ticket is completely coincidental!).

Comment by Kiyoto Yabe [ 12/Oct/20 ]

This is the same object as before but with a fix of PIPE2D-635

The 1D output looks reasonable.

Comment by vlebrun [ 12/Oct/20 ]

good to see we are able to measure the redshift on such strong emission lines...

Comment by Kiyoto Yabe [ 22/Oct/20 ]

I just confirmed that the processing is successfully done with the master branch after PIPE2D-635 is merged. So, I will close this ticket.

Generated at Tue Apr 08 01:35:08 JST 2025 using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.