https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-689 DRP 2-D Pipeline <p>Using the weekly,</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-python"> dataId = <span class="code-object">dict</span>(visit=47, arm=<span class="code-quote">"r"</span>, spectrograph=1) pfsArm = butler.get(<span class="code-quote">"pfsArm"</span>, dataId) detMap = butler.get(<span class="code-quote">"detectorMap"</span>, dataId) detMap.findPoint(pfsArm.fiberId[0], pfsArm.wavelength[0][0]) <span class="code-comment"># succeeds </span>detMap.findPoint(pfsArm.fiberId[0], pfsArm.wavelength[0][0:2]) <span class="code-comment"># fails</span> </pre> </div></div> <p>generates the error:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> TypeError: findPoint(): incompatible function arguments. The following argument types are supported: 1. (self: pfs.drp.stella.DetectorMap.DetectorMap, fiberId: <span class="code-object">int</span>, wavelength: <span class="code-object">float</span>) -&gt; lsst.geom.Point2D 2. (self: pfs.drp.stella.DetectorMap.DetectorMap, fiberId: <span class="code-object">int</span>, wavelength: ndarray::Array&lt;<span class="code-object">float</span>, 1, 1&gt;) -&gt; ndarray::Array&lt;<span class="code-object">float</span>, 2, 1&gt; 3. (self: pfs.drp.stella.DetectorMap.DetectorMap, fiberId: ndarray::Array&lt;<span class="code-object">int</span>, 1, 1&gt;, wavelength: ndarray::Array&lt;<span class="code-object">float</span>, 1, 1&gt;) -&gt; ndarray::Array&lt;<span class="code-object">float</span>, 2, 1&gt; Invoked with: &lt;pfs.drp.stella.SplinedDetectorMap.SplinedDetectorMap object at 0x2b3b8add2928&gt;, 2, array([620.33496, 620.4214 ], dtype=float32) </pre> </div></div> PIPE2D-689

Vector version of DetectorMap.findPoint fails

Story Normal Done Done price rhl Mon, 21 Dec 2020 17:50:49 +0000 Mon, 7 Mar 2022 23:22:50 +0000 Mon, 1 Feb 2021 19:09:00 +0000 0 1 <p>I think this is working as expected.</p> <p>The solution is:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> &gt;&gt;&gt; detMap.findPoint(<span class="code-object">int</span>(pfsArm.fiberId[0]), pfsArm.wavelength[0][0:2].astype(np.float32)) array([[ nan, nan], [4.0264922e+03, 1.0000000e+00]], dtype=float32) </pre> </div></div> <p>There were two problems:<br/> 1. <tt>pfsArm.fiberId<span class="error">&#91;0&#93;</span></tt> is an <tt>np.uint64</tt>, which isn't recognised as an `int`.<br/> 2. <tt>pfsArm.wavelength<span class="error">&#91;0&#93;</span><span class="error">&#91;0:2&#93;</span>.dtype</tt> is <tt>dtype('&gt;f4')</tt>, which isn't recognised as <tt>np.float32</tt>.</p> <p>I'm reopening this.  If you can't pass the arrays in a <tt>PfsArm</tt> to a <tt>DetectorMap</tt> then something in the API needs to be fixed.</p> <p> </p> <p>And I don't think that the cast to <tt>int</tt> is necessary.</p> <p>OK, then I think we need to do a couple of things:</p> <p>1. Convert arrays on read (byte swap, cast) in datamodel.<br/> 2. Ensure all pybind APIs in drp&#95;stella use consistent types.</p> <p>An alternative would be to add more pybind overloads for <tt>float</tt> and <tt>double</tt> arrays, but that doesn't catch byte-swapping, and I'm concerned that the number of combinations might make maintenance difficult. I guess it might be possible to add automatic translations between <tt>float</tt> and <tt>double</tt> arrays in the pybind layer, but that goes against the ndarray package's philosophy of only doing explicit conversions (because of the potential for silent memory and CPU inefficiency).</p> <p>I propose to use <tt>double</tt> for all arrays (pixel position, flux, wavelength and associated errors) instead of single-precision <tt>float</tt>. <tt>float</tt> would give us ~6 decimal digits of precision, and I'm afraid that that's getting a little tight for wavelengths (dispersion is 0.04 nm/pixel for <tt>m</tt>, I believe our centroids are good to around a 1/100 of a pixel, so we need a precision of 0.04/100 = 4e-4; precision of {float}} for values between between 2^9 and 2^10 is 2^(9-23) ~ 6e-5, which is getting a bit close for comfort). We could mix and match by using <tt>double</tt> for wavelengths and <tt>float</tt> for fluxes, but I think a desire for consistency and the fact that arrays default to <tt>double</tt> in numpy encourages us to always use <tt>double</tt> for arrays.</p> <p>Implemented the above. <tt>detMap.findPoint(pfsArm.fiberId<span class="error">&#91;0&#93;</span>, pfsArm.wavelength<span class="error">&#91;0&#93;</span><span class="error">&#91;0:2&#93;</span>)</tt> now works out-of-the-box.</p> <p>Merged to master.</p> Relates DAMD-130 Development Rank 0|02qps3:1y Reviewers hassan Sprint 2DDRP-2021 A Story Points 3.0