addn-hosts and dhcp-hostsfile could have directory as its value.

addn-hosts=/etc/dnsmasq.d/hosts/
dhcp-hostsfile=/etc/dnsmasq.d/dhcp/

Feb  7 09:14:31 disk-01 dnsmasq[5074]: read /etc/dnsmasq.d/hosts//hostname - 0 addresses
Feb  7 09:14:31 disk-01 dnsmasq-dhcp[5074]: read /etc/dnsmasq.d/dhcp//infra
Feb  7 09:14:31 disk-01 systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.

as similar to conf-dir, any files whose names end in ~, start with . or start and end with # are (seems to be) always skipped.

/etc/dnsmasq.d# ls -al dhcp hosts 
dhcp:
total 0
drwxr-xr-x 2 root root 33 Feb  7 13:46 .
drwxr-xr-x 4 root root 78 Feb  7 09:14 ..
-rw-r--r-- 1 root root  0 Feb  7 13:46 .dotfile
-rw-r--r-- 1 root root  0 Feb  7 09:14 infra

hosts:
total 0
drwxr-xr-x 2 root root 53 Feb  7 13:45 .
drwxr-xr-x 4 root root 78 Feb  7 09:14 ..
-rw-r--r-- 1 root root  0 Feb  7 13:45 .dotfile
-rw-r--r-- 1 root root  0 Feb  7 08:53 hostname
-rw-r--r-- 1 root root  0 Feb  7 13:45 no-dotfile

Feb  7 13:46:05 disk-01 dnsmasq[5227]: reading /etc/resolv.conf
Feb  7 13:46:05 disk-01 dnsmasq[5227]: using nameserver 10.100.200.1#53
Feb  7 13:46:05 disk-01 dnsmasq[5227]: read /etc/hosts - 5 addresses
Feb  7 13:46:05 disk-01 dnsmasq[5227]: read /etc/dnsmasq.d/hosts//no-dotfile - 0 addresses
Feb  7 13:46:05 disk-01 dnsmasq[5227]: read /etc/dnsmasq.d/hosts//hostname - 0 addresses
Feb  7 13:46:05 disk-01 dnsmasq-dhcp[5227]: read /etc/dnsmasq.d/dhcp//infra
Feb  7 13:46:05 disk-01 systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.

so, we may be possible to host some readme or tips file with .xxxx filename.

options we may need or may be better to have are:

• local-ttl: to set DNS reply TTL (default 0)
• log-queries
• expand-hosts
• domain-needed
• log-dhcp

options we may be better to have, but better to think seriously on environment:

• bogus-priv : no reverse lookups for private IP ranges
• dhcp-sequential-ip : sequential DHCP relase to clients (default hash)

basic idea

• have only global configurations (dnsmasq-wide, pxe, etc.) in the top directory, one file per one (specific) group like pxe
• have directories for DHCP (dhcp-hostsfile mac-hostname pair, with tag for host - e.g. pxe) and DNS (addn-hosts; hostname-IP pair)
• use the same name in two directories for each group and use canonical abbreviation and cluster ID, like bcu1, pfi, mcs. (OR several files of DHCP for one DNS could be an additional option, esp. SpS)

For dhcp/dns conf files,

• set different name for each port (MAC address) of physical host, with postfix like r410-1a
• have one hostname for one physical host (like r410-1) and set all normal ethernet ports to one IP address (like "10.0.0.10 r410-1 r410-1a r410-1b")
• fix hostname to function, which means BEE of BCU1 shall keep the same hostname (and IP address) even if MAC address (host hardware) changed

This will make:

• mostly only files in dhcp-hostsdir will be modified/replaced on replace of broken hardware
• files in root (dhcp-range etc.) and in addn-hosts (IP address) are different among branches (sites)
• files in dhcp-hostsdir will be similar among branches (sites)
  and, this leads simple merge/push among branches (even copy of selected commits) to be difficult. Also hardware replacement will happen at one site, independent from other sites even which will get modified on hardware delivery.

So, inter-branch management will be

• add/modify global configurations (e.g. dhcp-option) on demand
• add/modify DNS configurations (addn-hosts) at the initial moment of hardware delivery like LAM or ASIAA to Subaru
• add/modify DHCP configurations (dhcp-hostsdir) at point of delivery or hardware replacement

also,

• set master as one at Subaru, and each site (institute) has one branch
• have only already delivered hardware in master both for DNS and DHCP

fmadec, arnaud.lefur, cloomis, jeg, sywang, chihyi, naoyuki.tamura, philip
any comments?

Additions for system-wide items

• physical computer can have only one IP address from PFS dnsmasq even it is with multiple ethernet interface
• system-wide shared boxes shall be configured as fixed IP address, but also entries need to be registered to dnsmasq (both DNS, DHCP)
• one physical computer shall be tighten to the one IP address, with e.g. "10.0.0.10 r410-1 r410-1a r410-b" - host will have canonical name "r410-1" but interface (MAC address) to "r410-1a" or "r410-1b"

If host is configured as bonding (either by LACP or rr), DHCP request will be from one of bonded interfaces, so this configuration shall be fine.

Draft of 1st version added as PR at github.
If any comment/suggestion/correction, comment to this ticket or PR by 2017/Apr/20 1200 UTC.

Hi Atushi,

Regarding the hostname, product-tree base name should be fine, but I feel that it would be better to describe guideline "hostname" more specifically, to avoid confusion and/or duplication.
Also, are there any mechanism to check duplication before one defines a hostname?

In the current proposal, following line is included. Isn't it enough?
> Subparts of 'hostname' is RECOMMENDED to be well defined name in the PFS product tree, such as bcu1 but not just b1, to make hostname to be self described.

One point I may need to update is to define more solid way to perform merge (or copy and add) from branch to master (or even to some branch for AIT). We may need to have some review and filtering on such event to be more secure...

Only a few comments:

General:

• I would be happy to read this kind of a document and write comments, but should certainly not the only person to judge whether this is in a good enough shape and give it a go, due simply to my limited expertise and experiences of this kind of subject. I recommend this to be reviewed by ~1-2 more experts who could say "what if he/she designs this ..." type of things. Suggestions are: Lupton, Jescke, Tait, etc. - Does this apply to virtual machine configurations? There may be any other cases where it could be hard for target names to be defined from the hardware product tree e.g. since its cross-item nature?
• There are a few places with "SHALL not", "not is REQUIRED", in which cases "not" is critical not to be separated from the other, so it should be tied up together with all UPPER cases.
  o I would be happy to read this kind of a document and write comments, but am certainly not a person to judge whether this is in a good enough shape and give it a go. I recommend this to be reviewed by ~1-2 more experts who could say "what if he/she designs this ..." type of things. Suggestions are: Lupton, Jescke, Tait, etc.
  o Branch management:
• Merging two branches will happen only when hardware delivery happens and this delivery completes the task of one party. Is this correct? For example, JHU keeps delivering cryostats to LAM one after another, but until the last cryostat is delivered, they should need to manage their branch.
• I suggest to assign a branch manager per institute, just to clarify the location of responsibility.

• review persons
• due to system restriction, I could not set multiple reviewers, so I just put manager for reviewer. I shall be set as review from the wind, sorry. although I set reviewer as naoyuki.tamura, all relevant persons are putted into watchers, so they should receive notification and may give review comments/suggestions.
• cases
• I will correct them
• merging two branches
• all branches shall exists over all the time and to be operated, so actual operation is similar to rebase than merge, but merging changes in some branch to another on hardware delivery could be called as "merge", so I used "merge" but left section title as "branch management"
• assigning branch managers
• yes. as recent comment/discussions, I plan to add more solid way of branch management.

Getting no comment from others, I'll merge this shortly, with reminding following points (mostly) to me.

• Need to file a ticket to add a section on branch management, to include solid way of branch management, such as to point manager(s) per each branch (incl. master) on review and approval of commit(s).
• Need to work on INSTRM-43
• Need to file a ticket to add IPMU branch to try and track dnsmasq configuration in operation for pfs.ipmu.jp backend.

Atsushi, Arnaud, and I talked a bit more, and are now proposing the following. I will convert the JHU system under this ticket as a proof-of-concept.

Basically,

• we stick to master only
• we split configuration into files in a PFS-wide directory and files in a per-site directory.
• the per-site configuration goes into /etc/dnsmasq.d/$SITE directories
• we make a symbolic link from site -> $SITE
• the system-defined dnsmasq command-line arguments specify the new configuration directories.

A few more details:

Configuration which will be valid only at one site goes into /etc/dnsmasq.d/site/,
which is a manually maintained symbolic link to {{/etc/dnsmasq.d/(LAM,JHU,ASIAA,SUBARU) }}. As little as possible should be put here.

Configuration which will be valid in all locations goes in /etc/dnsmasq/PFS/
dnsmasq configuration location argument becomes something like -7 /etc/dnsmasq.d/site,*.conf -7 /etc/dnsmasq.d/PFS,*.conf. As much as possible should be put here.

For now, we will leave hostnames and MAC addresses in separate files. This may popup in a separate ticket.

Note that the standard Debian /etc/default/dnsmasq does not support two configure directories. But you can add arbitrary args to DNSMASQ_OPT

I don't understand this, and I think we haven't talked as so.
> the per-site configuration goes into /etc/dnsmasq.d/$SITE directories
> we make a symbolic link from site -> $SITE

> dnsmasq configuration location argument becomes something like -7 /etc/dnsmasq.d/site,.conf -7 /etc/dnsmasq.d/PFS,.conf. As much as possible should be put here.

I suppose -7 with following "," sections are for rejecting specific extensions. READ MAN BEFORE PROPOSING SOMETHING!
> Read all the files in the given directory as configuration files. If extension(s) are given, any files which end in those extensions are skipped.

Sounds good. Please ensure that a missing symbolic link generates useful and early error messages (and that no-one commits a link to git – so it needs to go in .gitignore)

dnsmasq has countless too-cute features. This is one.

• -7 /dir/path,*.conf matches all and only .conf files.
• -7 /dir/path,.conf matches all except .conf files.

If you have a better scheme than symbolic links we should use it. I can certainly see moving all the -7 args into an internal /etc/dnsmasq.d/dirs.conf file, but am stuck at that point.

I had tried that (,*.conf) at some point, but it did not work, actually. Could be some issue in configuration loading process, but not sure why.

Configuration at the dhcp/dns server will be one time, and for normal operation we will just pull from git and update configurations (w/ reloading them), so both way (symlink or in upper configuration) seems fine for me as operation point of view.

FYI, (,*.conf) is what we use at LAM.

Hi cloomis, I'm quite sorry but it worked now.
It seems I was totally failed to pass options to command line from daemon execution configuration.

Also it worked with

CONFIG_DIR=/etc/dnsmasq.d,*.conf

in /etc/default/dnsmasq file, and could be easier to config by script??? (sorry not sure nor tried by Ansible).

Updated proposal (from cloomis and me):

• accept different IP address range per site. although we have different IP address, we would recommend to have the same range as at Subaru for final AIT sites - LAM and ASIAA
• configure dnsmasq to load /etc/dnsmasq.d/*.conf by /etc/default/dnsmasq file
• have two configuration files - one is globally the same (something like dnsmasq.conf to have project-wide configuration incl. mac addr to host directory, and host.conf as symlink to e.g. host.ipmu to have IP address range with host to IP address directory
• have one directory to have MAC address to hostname configurations, loaded in dnsmasq.conf
• have site specific directory to have hostname to IP address configurations, loaded in (e.g.) host.ipmu

We must be getting close.

I just pushed the configuration which is running at IDG, and suggest that it can be merged and run at LAM.

In short:

• All sites have a common dnsmasq.conf.
• Each $site (JHU, LAM, ASIAA, IPMU, Subaru) has a dnsmasq-site.$site
• That file must be pointed to by the link dnsmasq-site.conf. This is the only real manual hack.
• All hostname to MAC bindings are in files in macs/ or macs-$site/
• All hostname to IP bindings are in files in hosts-subaru/, hosts-10.1/, or hosts-$site/.

The dnsmasq-site.$site file contains all the configuration which is not common to all. In particular, it gives the directories for site-specific MAC and IP binding files. For LAM, say:

# hostname - MAC
dhcp-hostsfile=/etc/dnsmasq.d/macs-lam

# hostname - IP
addn-hosts=/etc/dnsmasq.d/hosts-10.1
addn-hosts=/etc/dnsmasq.d/hosts-lam

declares that there are additional LAM device files in macs-lam/ and hosts-lam/, and that all the common PFS hosts are in the existing 10.1 network.

I suspect that JHU and LAM (and ASIAA?) will switch to using the final observatory address range, as defined in hosts-subaru. But for this ticket we are just re-organizing.

One LAM-specific note. I merged in the entries in pfs-ait-server:/etc/hosts into hosts-lam/. You can (and should!) put nameserver 127.0.0.1 as the first nameserver in /etc./resolv.conf to have that host also use dnsmasq to resolve names.

If this is acceptable, ics_doc's SSN-00028 will need to be updated.

Thanks, that's very clear and I'm willing to test it as soon as possible.

One LAM-specific note. I merged in the entries in pfs-ait-server:/etc/hosts into hosts-lam/. You can (and should!) put nameserver 127.0.0.1 as the first nameserver in /etc./resolv.conf to have that host also use dnsmasq to resolve names.

I'm not against it, but I'm not sure to clearly understand the benefits, can you be a bit more specific just for my own curiosity ?

It's to enable DNS name resolution on the dnsmasq host itself. If you don't run anything except dnsmasq, it is not required. But it might be useful if you are running some service on the same host.

tftp-root configuration need to be an option, or we need to add a remark to have directory specified in tftp-root. Without directory, dnsmasq service does not start with error of missing directory.

cloomis please check updated document at
https://github.com/Subaru-PFS/ics_doc/pull/29

let's merge ics_dnsmasq side to production from migration trial branch.

merged