https://pfspipe.ipmu.jp/jira This file is an XML representation of an issue en-us 8.3.4 803005 13-09-2019 https://pfspipe.ipmu.jp/jira/browse/INSTRM-71 Instrument control development <p>Define organization of dnsmasq configuration files and put documents in ics_doc. ics_dnsmasq repo is designed to be mapped to /etc/dnsmasq.d/ directory, and all of non configuration lines shall start with '#', which breaks rst (or markdown).<br/> This document need to include:</p> <ul> <li>Organization of configuration files (directory and file schema/name, containts)</li> <li>Service (dnsmasq) global configurations</li> <li>Branch organization</li> </ul> <p>Operational procedures (e.g. to register new hardware, to exchange hardware for maintenance) is planned to be developed in separated ticket (ref. <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-70" title="Define procedure of configuration and coordination on dnsmasq (ics_dnsmasq repo) for production and development" class="issue-link" data-issue-key="INSTRM-70"><del>INSTRM-70</del></a>)</p> <p>Based on current working configuration used in labs at JHU and LAM (#19c9f47):</p> <ul> <li>Store hostname to IP address bindings in hosts/ directory</li> <li>Store MAC address to hostname bindings in the top directory, which will be changed on hardware exchange for maintenance</li> <li>Have one set of files for above two bindings (in total two files) per one set of functional modules, e.g. BCU1, ENU1, PFI<img class="emoticon" src="https://pfspipe.ipmu.jp/jira/images/icons/emoticons/help_16.png" height="16" width="16" align="absmiddle" alt="" border="0"/></li> <li>All dnsmasq global configuration (e.g. domain, log-dhcp) shall be in pfs.conf</li> <li>All external configuration (e.g. external DNS server) shall be in machine.conf</li> </ul> <p> from <span class="error">&#91;1&#93;</span>, points 1,2 of 1st section, 1,2,4 of 2nd section. (5,6 of 1st section and 3 of 2nd section need to be defined in global configuration; 3,4 of 1st section is specific to JHU/LAM branch)</p> <p>Also following restrictions/ways have confirmed, which need to be cared in configurations:</p> <ul> <li>addn-hosts=xxx is to set an additional directory for hosts files to be read by dnsmasq DNS service</li> <li>hosts files in addn-hosts directory can contain normal definition lines, such like "ipaddr name1 name2 ..."</li> <li>dhcp-host line can contain IP address, and can rely on hostname-ipaddr conversion defined in hosts files in addn-hosts</li> <li>(so, seems no way to have DNS records for not leased dhcp-host entry, if we don't take a way with hosts file)</li> </ul> <p>*1 <a href="https://github.com/Subaru-PFS/ics_dnsmasq/blob/947663ffc3c63d4a9d9392cdc279627bffab6f95/PFS.README" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ics_dnsmasq/blob/947663ffc3c63d4a9d9392cdc279627bffab6f95/PFS.README</a></p> INSTRM-71

organization of dnsmasq configuration files - both DHCP and DNS

Task Major Done Done shimono shimono Wed, 11 Jan 2017 07:57:56 +0000 Wed, 21 Feb 2018 17:51:56 +0000 Wed, 21 Feb 2018 17:51:56 +0000 ics_doc 0 11 <p>addn-hosts and dhcp-hostsfile could have directory as its value.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>addn-hosts=/etc/dnsmasq.d/hosts/ dhcp-hostsfile=/etc/dnsmasq.d/dhcp/ Feb 7 09:14:31 disk-01 dnsmasq[5074]: read /etc/dnsmasq.d/hosts//hostname - 0 addresses Feb 7 09:14:31 disk-01 dnsmasq-dhcp[5074]: read /etc/dnsmasq.d/dhcp//infra Feb 7 09:14:31 disk-01 systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server. </pre> </div></div> <p>as similar to conf-dir, any files whose names end in ~, start with . or start and end with # are (<b>seems to be</b>) always skipped.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> /etc/dnsmasq.d# ls -al dhcp hosts dhcp: total 0 drwxr-xr-x 2 root root 33 Feb 7 13:46 . drwxr-xr-x 4 root root 78 Feb 7 09:14 .. -rw-r--r-- 1 root root 0 Feb 7 13:46 .dotfile -rw-r--r-- 1 root root 0 Feb 7 09:14 infra hosts: total 0 drwxr-xr-x 2 root root 53 Feb 7 13:45 . drwxr-xr-x 4 root root 78 Feb 7 09:14 .. -rw-r--r-- 1 root root 0 Feb 7 13:45 .dotfile -rw-r--r-- 1 root root 0 Feb 7 08:53 hostname -rw-r--r-- 1 root root 0 Feb 7 13:45 no-dotfile Feb 7 13:46:05 disk-01 dnsmasq[5227]: reading /etc/resolv.conf Feb 7 13:46:05 disk-01 dnsmasq[5227]: using nameserver 10.100.200.1#53 Feb 7 13:46:05 disk-01 dnsmasq[5227]: read /etc/hosts - 5 addresses Feb 7 13:46:05 disk-01 dnsmasq[5227]: read /etc/dnsmasq.d/hosts<span class="code-comment">//no-dotfile - 0 addresses </span>Feb 7 13:46:05 disk-01 dnsmasq[5227]: read /etc/dnsmasq.d/hosts<span class="code-comment">//hostname - 0 addresses </span>Feb 7 13:46:05 disk-01 dnsmasq-dhcp[5227]: read /etc/dnsmasq.d/dhcp<span class="code-comment">//infra </span>Feb 7 13:46:05 disk-01 systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server. </pre> </div></div> <p>so, we may be possible to host some readme or tips file with .xxxx filename.</p> <p>options we may need or may be better to have are:</p> <ul> <li>local-ttl: to set DNS reply TTL (default 0)</li> <li>log-queries</li> <li>expand-hosts</li> <li>domain-needed</li> <li>log-dhcp</li> </ul> <p>options we may be better to have, but better to think seriously on environment:</p> <ul> <li>bogus-priv : no reverse lookups for private IP ranges</li> <li>dhcp-sequential-ip : sequential DHCP relase to clients (default hash)</li> </ul> <p>basic idea</p> <ul> <li>have only global configurations (dnsmasq-wide, pxe, etc.) in the top directory, one file per one (specific) group like pxe</li> <li>have directories for DHCP (dhcp-hostsfile mac-hostname pair, with tag for host - e.g. pxe) and DNS (addn-hosts; hostname-IP pair)</li> <li>use the same name in two directories for each group and use canonical abbreviation and cluster ID, like bcu1, pfi, mcs. (OR several files of DHCP for one DNS could be an additional option, esp. SpS)</li> </ul> <p>For dhcp/dns conf files,</p> <ul> <li>set different name for each port (MAC address) of physical host, with postfix like r410-1a</li> <li>have one hostname for one physical host (like r410-1) and set all normal ethernet ports to one IP address (like "10.0.0.10 r410-1 r410-1a r410-1b")</li> <li>fix hostname to function, which means BEE of BCU1 shall keep the same hostname (and IP address) even if MAC address (host hardware) changed</li> </ul> <p>This will make:</p> <ul> <li><b>mostly</b> only files in dhcp-hostsdir will be modified/replaced on replace of broken hardware</li> <li>files in root (dhcp-range etc.) and in addn-hosts (IP address) are different among branches (sites)</li> <li>files in dhcp-hostsdir will be similar among branches (sites)<br/> and, this leads simple merge/push among branches (even copy of selected commits) to be difficult. Also hardware replacement will happen at <b>one</b> site, independent from other sites even which will get modified on hardware delivery.</li> </ul> <p>So, inter-branch management will be</p> <ul> <li>add/modify global configurations (e.g. dhcp-option) on demand</li> <li>add/modify DNS configurations (addn-hosts) at the initial moment of hardware delivery like LAM or ASIAA to Subaru</li> <li>add/modify DHCP configurations (dhcp-hostsdir) at point of delivery or hardware replacement</li> </ul> <p>also,</p> <ul> <li>set master as one at Subaru, and each site (institute) has one branch</li> <li>have only already delivered hardware in master both for DNS and DHCP</li> </ul> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=fmadec" class="user-hover" rel="fmadec">fmadec</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=arnaud.lefur" class="user-hover" rel="arnaud.lefur">arnaud.lefur</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=jeg" class="user-hover" rel="jeg">jeg</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=sywang" class="user-hover" rel="sywang">sywang</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=chihyi" class="user-hover" rel="chihyi">chihyi</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=naoyuki.tamura" class="user-hover" rel="naoyuki.tamura">naoyuki.tamura</a>, <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=philip" class="user-hover" rel="philip">philip</a><br/> any comments?</p> <p>Additions for system-wide items</p> <ul> <li>physical computer can have only one IP address from PFS dnsmasq even it is with multiple ethernet interface</li> <li>system-wide shared boxes shall be configured as fixed IP address, but also entries need to be registered to dnsmasq (both DNS, DHCP)</li> <li>one physical computer shall be tighten to the one IP address, with e.g. "10.0.0.10 r410-1 r410-1a r410-b" - host will have canonical name "r410-1" but interface (MAC address) to "r410-1a" or "r410-1b"</li> </ul> <p>If host is configured as bonding (either by LACP or rr), DHCP request will be from one of bonded interfaces, so this configuration shall be fine.</p> <p>Draft of 1st version added as PR at github.<br/> If any comment/suggestion/correction, comment to this ticket or PR by 2017/Apr/20 1200 UTC.</p> <p>Hi Atushi,</p> <p>Regarding the hostname, product-tree base name should be fine, but I feel that it would be better to describe guideline "hostname" more specifically, to avoid confusion and/or duplication.<br/> Also, are there any mechanism to check duplication before one defines a hostname?</p> <p>In the current proposal, following line is included. Isn't it enough?<br/> &gt; Subparts of 'hostname' is RECOMMENDED to be well defined name in the PFS product tree, such as bcu1 but not just b1, to make hostname to be self described.</p> <p>One point I may need to update is to define more solid way to perform merge (or copy and add) from branch to master (or even to some branch for AIT). We may need to have some review and filtering on such event to be more secure...</p> <p>Only a few comments:</p> <p>General:</p> <ul class="alternate" type="square"> <li>I would be happy to read this kind of a document and write comments, but should certainly not the only person to judge whether this is in a good enough shape and give it a go, due simply to my limited expertise and experiences of this kind of subject. I recommend this to be reviewed by ~1-2 more experts who could say "what if he/she designs this ..." type of things. Suggestions are: Lupton, Jescke, Tait, etc. - Does this apply to virtual machine configurations? There <b>may</b> be any other cases where it could be hard for target names to be defined from the hardware product tree e.g. since its cross-item nature?</li> <li>There are a few places with "SHALL not", "not is REQUIRED", in which cases "not" is critical not to be separated from the other, so it should be tied up together with all UPPER cases.<br/> o I would be happy to read this kind of a document and write comments, but am certainly not a person to judge whether this is in a good enough shape and give it a go. I recommend this to be reviewed by ~1-2 more experts who could say "what if he/she designs this ..." type of things. Suggestions are: Lupton, Jescke, Tait, etc.<br/> o Branch management:</li> <li>Merging two branches will happen only when hardware delivery happens and this delivery completes the task of one party. Is this correct? For example, JHU keeps delivering cryostats to LAM one after another, but until the last cryostat is delivered, they should need to manage their branch.</li> <li>I suggest to assign a branch manager per institute, just to clarify the location of responsibility.</li> </ul> <ul> <li>review persons</li> <li>due to system restriction, I could not set multiple reviewers, so I just put manager for reviewer. I shall be set as review from the wind, sorry. although I set reviewer as <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=naoyuki.tamura" class="user-hover" rel="naoyuki.tamura">naoyuki.tamura</a>, all relevant persons are putted into watchers, so they should receive notification and may give review comments/suggestions.</li> <li>cases</li> <li>I will correct them</li> <li>merging two branches</li> <li>all branches shall exists over all the time and to be operated, so actual operation is similar to rebase than merge, but merging changes in some branch to another on hardware delivery could be called as "merge", so I used "merge" but left section title as "branch management"</li> <li>assigning branch managers</li> <li>yes. as recent comment/discussions, I plan to add more solid way of branch management.</li> </ul> <p>Getting no comment from others, I'll merge this shortly, with reminding following points (mostly) to me.</p> <ul> <li>Need to file a ticket to add a section on branch management, to include solid way of branch management, such as to point manager(s) per each branch (incl. master) on review and approval of commit(s).</li> <li>Need to work on <a href="https://pfspipe.ipmu.jp/jira/browse/INSTRM-43" title="[dnsmasq] Change branch from master to sites (LAM, JHU)" class="issue-link" data-issue-key="INSTRM-43"><del>INSTRM-43</del></a></li> <li>Need to file a ticket to add IPMU branch to try and track dnsmasq configuration in operation for pfs.ipmu.jp backend.</li> </ul> <p>Atsushi, Arnaud, and I talked a bit more, and are now proposing the following. I will convert the JHU system under this ticket as a proof-of-concept.</p> <p>Basically, </p> <ul class="alternate" type="square"> <li>we stick to master only</li> <li>we split configuration into files in a PFS-wide directory and files in a per-site directory.</li> <li>the per-site configuration goes into /etc/dnsmasq.d/$SITE directories</li> <li>we make a symbolic link from site -&gt; $SITE</li> <li>the system-defined dnsmasq command-line arguments specify the new configuration directories.</li> </ul> <p>A few more details:</p> <p>Configuration which will be valid only at one site goes into <tt>/etc/dnsmasq.d/site/</tt>,<br/> which is a manually maintained symbolic link to {{/etc/dnsmasq.d/(LAM,JHU,ASIAA,SUBARU) }}. As little as possible should be put here.</p> <p>Configuration which will be valid in all locations goes in <tt>/etc/dnsmasq/PFS/</tt><br/> <tt>dnsmasq</tt> configuration location argument becomes something like <tt>-7 /etc/dnsmasq.d/site,&#42;.conf -7 /etc/dnsmasq.d/PFS,&#42;.conf</tt>. As much as possible should be put here.</p> <p>For now, we will leave hostnames and MAC addresses in separate files. This may popup in a separate ticket.</p> <p>Note that the standard Debian <tt>/etc/default/dnsmasq</tt> does not support two configure directories. But you can add arbitrary args to <tt>DNSMASQ_OPT</tt></p> <p>I don't understand this, and I think we haven't talked as so.<br/> &gt; the per-site configuration goes into /etc/dnsmasq.d/$SITE directories<br/> &gt; we make a symbolic link from site -&gt; $SITE</p> <p>&gt; dnsmasq configuration location argument becomes something like -7 /etc/dnsmasq.d/site,<b>.conf -7 /etc/dnsmasq.d/PFS,</b>.conf. As much as possible should be put here.</p> <p>I suppose -7 with following "," sections are for rejecting specific extensions. READ MAN BEFORE PROPOSING SOMETHING!<br/> &gt; Read all the files in the given directory as configuration files. If extension(s) are given, any files which end in those extensions are skipped.</p> <p>Sounds good. Please ensure that a missing symbolic link generates useful and early error messages (and that no-one commits a link to git &#8211; so it needs to go in .gitignore)</p> <p>dnsmasq has countless too-cute features. This is one.</p> <ul class="alternate" type="square"> <li><tt>-7 /dir/path,*.conf</tt> matches all and only <tt>.conf</tt> files.</li> <li><tt>-7 /dir/path,.conf</tt> matches all <em>except</em> <tt>.conf</tt> files.</li> </ul> <p>If you have a better scheme than symbolic links we should use it. I can certainly see moving all the <tt>-7</tt> args into an internal /etc/dnsmasq.d/dirs.conf file, but am stuck at that point.</p> <p>I had tried that (,*.conf) at some point, but it did not work, actually. Could be some issue in configuration loading process, but not sure why. </p> <p>Configuration at the dhcp/dns server will be one time, and for normal operation we will just pull from git and update configurations (w/ reloading them), so both way (symlink or in upper configuration) seems fine for me as operation point of view.</p> <p>FYI, (,*.conf) is what we use at LAM.</p> <p>Hi <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a>, I'm quite sorry but it worked now.<br/> It seems I was totally failed to pass options to command line from daemon execution configuration.</p> <p>Also it worked with</p> <blockquote><p>CONFIG_DIR=/etc/dnsmasq.d,*.conf</p></blockquote> <p>in /etc/default/dnsmasq file, and could be easier to config by script??? (sorry not sure nor tried by Ansible).</p> <p>Updated proposal (from <a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a> and me):</p> <ul> <li>accept different IP address range per site. although we have different IP address, we would recommend to have the same range as at Subaru for final AIT sites - LAM and ASIAA</li> <li>configure dnsmasq to load /etc/dnsmasq.d/*.conf by /etc/default/dnsmasq file</li> <li>have two configuration files - one is globally the same (something like dnsmasq.conf to have project-wide configuration incl. mac addr to host directory, and host.conf as symlink to e.g. host.ipmu to have IP address range with host to IP address directory</li> <li>have one directory to have MAC address to hostname configurations, loaded in dnsmasq.conf</li> <li>have site specific directory to have hostname to IP address configurations, loaded in (e.g.) host.ipmu</li> </ul> <p>We must be getting close.</p> <p>I just pushed the configuration which is running at IDG, and suggest that it can be merged and run at LAM.</p> <p>In short:</p> <ul> <li>All sites have a common <tt>dnsmasq.conf</tt>.</li> <li>Each $site (JHU, LAM, ASIAA, IPMU, Subaru) has a <tt>dnsmasq-site.$site</tt></li> <li>That file must be pointed to by the link <tt>dnsmasq-site.conf</tt>. This is the only real manual hack.</li> <li>All hostname to MAC bindings are in files in <tt>macs/</tt> or <tt>macs-$site</tt>/</li> <li>All hostname to IP bindings are in files in <tt>hosts-subaru</tt>/, <tt>hosts-10.1</tt>/, or <tt>hosts-$site</tt>/.</li> </ul> <p>The <tt>dnsmasq-site.$site</tt> file contains all the configuration which is not common to all. In particular, it gives the directories for site-specific MAC and IP binding files. For LAM, say:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># hostname - MAC dhcp-hostsfile=/etc/dnsmasq.d/macs-lam # hostname - IP addn-hosts=/etc/dnsmasq.d/hosts-10.1 addn-hosts=/etc/dnsmasq.d/hosts-lam </pre> </div></div> <p>declares that there are additional LAM device files in <tt>macs-lam</tt>/ and <tt>hosts-lam</tt>/, and that all the common PFS hosts are in the existing <tt>10.1</tt> network.</p> <p>I suspect that JHU and LAM (and ASIAA?) will switch to using the final observatory address range, as defined in <tt>hosts-subaru</tt>. But for this ticket we are just re-organizing.</p> <p>One LAM-specific note. I merged in the entries in <tt>pfs-ait-server:/etc/hosts</tt> into <tt>hosts-lam</tt>/. You can (and should!) put <tt>nameserver 127.0.0.1</tt> as the <em>first</em> nameserver in <tt>/etc./resolv.conf</tt> to have that host also use <tt>dnsmasq</tt> to resolve names.</p> <p>If this is acceptable, <tt>ics_doc</tt>'s <tt>SSN-00028</tt> will need to be updated.</p> <p>Thanks, that's very clear and I'm willing to test it as soon as possible.</p> <blockquote><p>One LAM-specific note. I merged in the entries in pfs-ait-server:/etc/hosts into hosts-lam/. You can (and should!) put nameserver 127.0.0.1 as the first nameserver in /etc./resolv.conf to have that host also use dnsmasq to resolve names.</p></blockquote> <p>I'm not against it, but I'm not sure to clearly understand the benefits, can you be a bit more specific just for my own curiosity ?</p> <p>It's to enable DNS name resolution on the dnsmasq host itself. If you don't run anything except dnsmasq, it is not required. But it might be useful if you are running some service on the same host.</p> <p>tftp-root configuration need to be an option, or we need to add a remark to have directory specified in tftp-root. Without directory, dnsmasq service does not start with error of missing directory.</p> <p><a href="https://pfspipe.ipmu.jp/jira/secure/ViewProfile.jspa?name=cloomis" class="user-hover" rel="cloomis">cloomis</a> please check updated document at<br/> <a href="https://github.com/Subaru-PFS/ics_doc/pull/29" class="external-link" rel="nofollow">https://github.com/Subaru-PFS/ics_doc/pull/29</a></p> <p>let's merge ics_dnsmasq side to production from migration trial branch.</p> <p>merged</p> Blocks INSTRM-291 Relates INSTRM-43 Development Epic Link INSTRM-70 Rank 0|ii03b1: Sprint 2017-10A