Specifying known_hosts file in Semaphore (especially under Docker)

[linsomniac]

Running Semaphore in the Docker environment, I was struggling with how to get around the "Host key verification failed" message. First I tried adding the environment variable "ANSIBLE_HOST_KEY_CHECKING=False", which worked but was not ideal.

I finally realized I could set the "extra variable" in the "Environment" to be: "ansible_ssh_common_args": "-o 'UserKnownHostsFile semaphore/known_hosts'"} to pick up a known_hosts from my repo. In my case, I use signed SSH host keys, so I really only have one line in that file to cover all my hosts.

I post this here to help others if they have similar struggles (I had searched the discussion before coming up with these solutions), and to get feedback if there is a better way to do it. I would think this would be something everyone runs into when running Semaphore from Docker.

[ansibleguy]

That was also one of the first issues I ran into with using a manual installation.
As SSH is one of the most used Ansible connection-types, there should be a good solution provided by Semaphore itself.
Maybe an internal SSH-hostkey-store with a functionality to manually add and fetch-on-click (ssh-keyscan) new ones.
The known_hosts file could be written to disk on runtime.

[linsomniac]

Pro Tip: Signed host keys are awesome!

[ansibleguy]

I agree.
But not everybody will manage their host-keys and some services/devices might not even support supplying custom ones.

[linsomniac]

Agreed, just wanted to point it out, because it was sure a game-changer when I implemented it in my environment (~120 hosts)

[Pshemas]

I've hit that problem as well - it really threw me off in the beginning. When I realized what's going on I chuckled.

On my end I've implemented a simple playbook that appends fingerprints to known_hosts file inside Docker container.

---
- name: Add hosts fingerprints
  hosts:
    - 127.0.0.1

  tasks:
    - name: loop over hosts and append to known_hosts
      shell:
        cmd: 
          "ssh-keyscan -t rsa {{ item }} >> /root/.ssh/known_hosts"
      loop:
        - 192.168.10.181
        - 192.168.10.182
        - 192.168.10.192
...

Of course you could pass the list of hosts to append as a variable in environment etc. to make it more flexible.

[kingp0dd]

wouldn't this hang when sempahoreui does its "Gathering Facts" stage of the playbook? it will connect to ssh when in Gathering Facts right?

[hmoffatt]

Add your host keys to the DNS (SSHFP records) and use ssh option VerifyHostKeyDNS ?

[kingp0dd]

Running Semaphore in the Docker environment, I was struggling with how to get around the "Host key verification failed" message. First I tried adding the environment variable "ANSIBLE_HOST_KEY_CHECKING=False", which worked but was not ideal.

I finally realized I could set the "extra variable" in the "Environment" to be: "ansible_ssh_common_args": "-o 'UserKnownHostsFile semaphore/known_hosts'"} to pick up a known_hosts from my repo. In my case, I use signed SSH host keys, so I really only have one line in that file to cover all my hosts.

I post this here to help others if they have similar struggles (I had searched the discussion before coming up with these solutions), and to get feedback if there is a better way to do it. I would think this would be something everyone runs into when running Semaphore from Docker.

by signed SSH keys, you mean CA signed keys? wouldn't that require adding this to every host's sshd config file?

HostCertificate /etc/ssh/ssh_host_rsa_key-cert.pub