Wouldn't it be easier and more realistic to degrade the spots a little?

Discovered that the simulator and Subaru PSFs aren't extremely different (x and y sigma for 2D Gaussian fit to arc lines was about 1.3 for both), and that we're having the same problems with CR finding on the Subaru quartz observations as we're having with the simulator. Tried tuning CR parameters but wasn't able to reliably find CRs while avoiding false positives from the traces. Decided to try subtracting a vertical median filter to remove the traces before CR finding, and this works very nicely, finding CRs that I had trouble identifying visually on the original. It does find a sprinkling of false-positives, but they are not systematically distributed along the traces as they were before. Perhaps we might benefit from careful parameter tuning, but this change already results in a dramatic improvement in CR finding quality.

Can you attach some images?  This sounds like a good and simple approach, but something more sophisticated (extraction; median filter; project back onto traces; subtract) might work better.

The below image shows the calexp image, the trace-subtracted image, and the calexp mask.

I tried the more sophisticated approach, but currently extraction and image generation from fiber traces introduces step functions into the data that makes CR finding much worse; I'm hoping PIPE2D-907 (and odd-even quartz observations) will fix this.

Kiyoto Yabe ran the weekly with this change and found poor redshift performance, but it was not clear that CR masking was to blame.

Note: I need to convert the Gen3 pipeline to also implement the trace subtraction before searching for CRs. This will involve moving things around some, as I want to do it after adjusting the detectorMap (so we have an opportunity to mask sky lines).

Kiyoto Yabe writes:

The 1D redshift determination is much improved with the new one (although there are some mis-identification but only a few). The CRs (and other features) are masked properly in this version as far as I checked.

rhl requests checking visit=80748 arm=r, where sky lines are being masked as CRs.

Code changes provided in pull requests look fine to me. Minor comments in PR for drp_stella: https://github.com/Subaru-PFS/drp_stella/pull/284

Ran visit=80748 arm=r with the ticket branch. CRs are getting masked just fine, and sky lines are not causing false positives.

Merged.

I confirm that things are going back to normal, 93% success rate in  the redshift measurement in the latest weekly run 

But this merged after the most recent weekly...?

ah... the improvements come from other modification then? but the fact is that the bad pixels I used to see in the September weekly runs are not present in the October 10 one...

Maybe the data you received were processed with this branch, but it could not have been part of the Proper weekly.

According to header, drp_stella version is w.2022.40a-12-g033bd60+1

OK, so that's this branch on top of the last weekly.

Yes, I forgot to switch back to the normal mode after I checked the new fix, so yes, the results are from that branch. Sorry.

well, don't be sorry, now we know that the fix worked fine