https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/PIPE2D-682 DRP 2-D Pipeline <p>We have been using Travis for CI, but they are now setting limits that we have exceeded.</p> <p>Rubin <a href="https://community.lsst.org/t/converting-travis-to-github-actions/4570" class="external-link" rel="nofollow">moved their Travis operations to GitHub Actions</a>, but they only use it for linting, which is fast. We've been using Travis for integration testing, which takes much more compute power (we only have a handful of packages, so we could get away with it), so we're in a different regime.</p> <p>I propose to move our CI to Jenkins running on Tiger. We already run weekly production runs and builds through this mechanism, and it wouldn't take much to extend it to run smaller-scale integration tests on demand.</p> <ul> <li>Write a back-end script to run the integration test under Jenkins, and report the results via e-mail and/or Slack.</li> <li>Add Jenkins job.</li> <li>Add a front-end script to trigger the Jenkins job.</li> <li>Remove Travis.</li> <li>Circulate instructions.</li> </ul> PIPE2D-682

Move CI to Tiger

Story Normal Done Done price price Thu, 10 Dec 2020 17:35:19 +0000 Fri, 28 Jan 2022 17:16:25 +0000 Fri, 29 Jan 2021 16:20:44 +0000 0 1 <p>Instructions for use, which I'll circulate once this procedure is approved and the changes have merged:</p> <blockquote> <p>To start an integration test automatically via Jenkins, run on the tiger cluster:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> setup pfs_pipe2d $PFS_PIPE2D_DIR/jenkins/trigger_integrationTest.py &lt;branch&gt; </pre> </div></div> <p>This is a fire-and-forget mechanism: you can start the run and then forget all about it until it notifies you of the result via Slack (in <tt>#drp-2d-travis</tt> with an <tt>@</tt> notification) and e-mail. Those notifications will contain the build number, a brief description of the result (success or failure) and the directory where you can poke around the run. The directory will contain a log file (<tt>integration&#95;&lt;build number&gt;.log</tt>), a shallow clone of each of the software products used in the test, and the integration test data repo (<tt>INTEGRATION</tt>), so you can reproduce and debug the failures if you need.</p></blockquote> <p>In addition to the main feature in pfs&#95;pipe2d, there's a single change in each of the DRP2D packages (removing the <tt>.travis.yml</tt> file) that I didn't bother creating PRs for.</p> <p>Merged to master.</p> <p>Sent instructions on using this system to <tt>reduction@pfspipe.ipmu.jp</tt>.</p> Development Rank 0|02qpq3:r1c Reviewers hassan