Ubuntu and systemd-resolved
I’ve been slowly setting up my homelab environment, and one problem I ran into recently was my salt minion wasn’t connecting to my salt master
Looks like DNS errors:
ubuntu@kvm02:~$ sudo systemctl status salt-minion
● salt-minion.service - The Salt Minion
Loaded: loaded (/lib/systemd/system/salt-minion.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-03-03 16:20:53 UTC; 8h ago
Docs: man:salt-minion(1)
file:///usr/share/doc/salt/html/contents.html
https://docs.saltstack.com/en/latest/contents.html
Main PID: 2246 (salt-minion)
Tasks: 4 (limit: 4730)
Memory: 70.9M
CGroup: /system.slice/salt-minion.service
├─2246 /usr/bin/python3 /usr/bin/salt-minion
├─2251 /usr/bin/python3 /usr/bin/salt-minion
└─2285 /usr/bin/python3 /usr/bin/salt-minion
Mar 04 01:01:19 kvm02 salt-minion[2251]: [ERROR ] DNS lookup or connection check of 'salt' failed.
Mar 04 01:01:19 kvm02 salt-minion[2251]: [ERROR ] Master hostname: 'salt' not found or not responsive. Retrying in 30 seconds
I hadn’t run into this problem with my debian machines, so I was a bit confused:
The default salt-minion configuration uses the salt
hostname, so I was thinking there was a DNS configuration issue. I’ve configured my edgerouter to resolve
the salt
hostname to the salt master’s ip address.
# Set the location of the salt master server. If the master server cannot be
# resolved, then the minion will fail to start.
#master: salt
Turns out, systemd-resolved (which ubuntu uses), won’t pass simple names: https://askubuntu.com/questions/907246/how-to-disable-systemd-resolved-in-ubuntu
The page links to the following github issue: https://github.com/systemd/systemd/issues/2514
Basically, single-label hostnames are considered “private”, something that should be made sense of locally only, and as such should never end up on the public Internet.
Bleh, so I have two options here:
- Disable systemd-resolved (see this https://askubuntu.com/questions/907246/how-to-disable-systemd-resolved-in-ubuntu)
- Use a fully qualified domain name (fqdn) for my salt master’s hostname
I ended up using a fqdn instead of trying to mess around with disabling systemd-resolved.