Customised fedora cloud image not completing the startup on Linux (fedora host). #2335

Ranjandas · 2024-05-10T14:07:25Z

Ranjandas
May 10, 2024

I am using Packer to customise the Fedora 40 cloud image. The customised image is launched using a custom lima template.

This works fine on macOS (with vmType: both vz and qemu). However, when I did the same on a fedora host, lima was stuck at the following stage:

...
INFO[0003] SSH Local Port: 42575
INFO[0002] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0012] [hostagent] Waiting for the crucial requirement 1 of 2: "ssh"
...
...

When I checked the serial.log, I could see that the cloud-init failed to create my user account.

[    7.825978] cloud-init[910]: 2024-05-10 13:57:32,482 - schema.py[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.
[    9.564942] BTRFS info (device vda4): resize device /dev/vda4 (devid 1) from 4212109312 to 106217582592
[   10.602674] cloud-init[910]: 2024-05-10 13:57:35,257 - util.py[WARNING]: Failed to create user ranjan
[   10.611336] cloud-init[910]: 2024-05-10 13:57:35,267 - util.py[WARNING]: Running module users_groups (<module 'cloudinit.config.cc_users_groups' from '/usr/lib/python3.12/site-packages/cloudinit/config/cc_users_groups.py'>) failed
[   10.840111] cloud-init[910]: Generating public/private rsa key pair.

Any hint on where to look for the issue? I'm just wondering why it fails only on Linux.

Answered by Ranjandas

May 13, 2024

Thank you for your inputs @afbjorklund. I managed to get into the VM using serial.sock and could inspect the cloud-init.log.

Looks like it is failing because the useradd command is using UID 1000 (from my linux host), which is already taken by my fedora user.

2024-05-13 06:49:13,815 - __init__.py[DEBUG]: Adding user ranjan
2024-05-13 06:49:13,815 - subp.py[DEBUG]: Running hidden command to protect sensitive input/output logstring: ['useradd', 'ranjan', '--home', '/home/ranjan.linux', '--shell', '/bin/bash', '--uid', '1000', '-m']
2024-05-13 06:49:13,943 - util.py[WARNING]: Failed to create user ranjan
2024-05-13 06:49:13,943 - util.py[DEBUG]: Failed to create user ranjan
Traceback (most r…

View full answer

afbjorklund · 2024-05-10T16:33:12Z

afbjorklund
May 10, 2024
Maintainer

Are there any details in /var/log/cloud-init.log?

11 replies

Ranjandas May 12, 2024
Author

I am unable to access that file because I can't get inside the VM, as the user provisioning itself fails. Interestingly, the user account I created as part of the packer build is also not working.

Btw, this is the user-data that I am using (passing as -cdrom cidata.iso) with the packer build to create a user account . Would this in any way conflict with the user-data that lima is using (specifically in lLima on Linux hosts, as this whole stuff works on macOS).

#cloud-config

system_info:
  default_user:
    name: fedora

chpasswd:
  list: |
    fedora:fedora
  expire: False
ssh_pwauth: True
resize_rootfs: True

afbjorklund May 13, 2024
Maintainer

Possibly, I don't think the lima start is able to handle multiple cloud-init images (cidata.iso) even if default users are common

But it is just guesses, until we see the error. You can run with a display to access the console, or use the serial.sock created

Ranjandas May 13, 2024
Author

Thank you for your inputs @afbjorklund. I managed to get into the VM using serial.sock and could inspect the cloud-init.log.

Looks like it is failing because the useradd command is using UID 1000 (from my linux host), which is already taken by my fedora user.

2024-05-13 06:49:13,815 - __init__.py[DEBUG]: Adding user ranjan
2024-05-13 06:49:13,815 - subp.py[DEBUG]: Running hidden command to protect sensitive input/output logstring: ['useradd', 'ranjan', '--home', '/home/ranjan.linux', '--shell', '/bin/bash', '--uid', '1000', '-m']
2024-05-13 06:49:13,943 - util.py[WARNING]: Failed to create user ranjan
2024-05-13 06:49:13,943 - util.py[DEBUG]: Failed to create user ranjan
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/cloudinit/distros/__init__.py", line 742, in add_user 
    subp.subp(useradd_cmd, logstring=log_useradd_cmd)
  File "/usr/lib/python3.12/site-packages/cloudinit/subp.py", line 322, in subp
    raise ProcessExecutionError(
cloudinit.subp.ProcessExecutionError: Unexpected error while running command.
Command: ['useradd', 'ranjan', '--home', '/home/ranjan.linux', '--shell', '/bin/bash', '--uid', '1000', '-m']
Exit code: 4
Reason: -
Stdout: 
Stderr: useradd: UID 1000 is not unique
2024-05-13 06:49:13,953 - handlers.py[DEBUG]: finish: init-network/config-users_groups: FAIL: running config-users_groups with frequency once-per-instance
2024-05-13 06:49:13,954 - util.py[WARNING]: Running module users_groups (<module 'cloudinit.config.cc_users_groups' from '/usr/lib/python3.12/site-packages/cloudinit/config/cc_users_groups.py'>) failed
2024-05-13 06:49:13,954 - util.py[DEBUG]: Running module users_groups (<module 'cloudinit.config.cc_users_groups' from '/usr/lib/python3.12/site-packages/cloudinit/config/cc_users_groups.py'>) failed 
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/cloudinit/config/modules.py", line 255, in _run_modules
    ran, _r = cc.run(
              ^^^^^^^
  File "/usr/lib/python3.12/site-packages/cloudinit/cloud.py", line 60, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/cloudinit/helpers.py", line 172, in run
    results = functor(**args)
              ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/cloudinit/config/cc_users_groups.py", line 240, in handle
    cloud.distro.create_user(user, **config)
  File "/usr/lib/python3.12/site-packages/cloudinit/distros/__init__.py", line 800, in create_user
    self.add_user(name, **kwargs)
  File "/usr/lib/python3.12/site-packages/cloudinit/distros/__init__.py", line 745, in add_user
    raise e
  File "/usr/lib/python3.12/site-packages/cloudinit/distros/__init__.py", line 742, in add_user
    subp.subp(useradd_cmd, logstring=log_useradd_cmd)
  File "/usr/lib/python3.12/site-packages/cloudinit/subp.py", line 322, in subp
    raise ProcessExecutionError(
cloudinit.subp.ProcessExecutionError: Unexpected error while running command.
Command: ['useradd', 'ranjan', '--home', '/home/ranjan.linux', '--shell', '/bin/bash', '--uid', '1000', '-m']
Exit code: 4
Reason: -
Stdout: 
Stderr: useradd: UID 1000 is not unique

Got it working by updating my packer cloud-init to have a different UID.

#cloud-config

system_info:
  default_user:
    name: fedora
    uid: 1001 <------ change the UID to avoid conflict with the UID of the host user account

chpasswd:
  list: |
    fedora:fedora
  expire: False
ssh_pwauth: True
resize_rootfs: True

Figured out that lima gets the user information from the host here:

lima/pkg/osutil/user.go

Line 112 in 7e493aa

cache.u, cache.err = user.Current()

Everything is working fine now. Thank you once again 🙏

Answer selected by Ranjandas

afbjorklund May 13, 2024
Maintainer

Default passwords are considered bad security practice, but Ubuntu Server is also doing this (unlike Cloud without it)

Lima is using keys for access, so disables password by default. You do have to add passwords, to access through GUI

Ranjandas May 13, 2024
Author

Thank you, @afbjorklund; yes, I am aware of the security implications. I have replaced the password with ssh_authorized_keys.

The new user-data is:

#cloud-config

system_info:
  default_user:
    name: fedora
    uid: 1001

resize_rootfs: True
ssh_authorized_keys:
  - ssh-rsa AAAAB3Nza....

afbjorklund May 13, 2024
Maintainer

I don't understand why you need a default user in the image, instead of adding one during instantiation

i.e. why not just use template://fedora

Ranjandas May 14, 2024
Author

@afbjorklund, I was doing that initially, but the package installations during the provisioning stage added to the startup time (especially when spawning a multi-node setup), so I decided to bake custom images and launch them using Lima. Currently, I am using this --set option to launch the VM from the output of packer builds.

## the vm.yaml doesn't have images[] in it by default
$ limactl start vm.yaml --set='. |= .images[0].location="file://...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Customised fedora cloud image not completing the startup on Linux (fedora host). #2335

{{title}}

Replies: 1 comment 11 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Customised fedora cloud image not completing the startup on Linux (fedora host). #2335

Ranjandas May 10, 2024

Replies: 1 comment · 11 replies

afbjorklund May 10, 2024 Maintainer

Ranjandas May 12, 2024 Author

afbjorklund May 13, 2024 Maintainer

Ranjandas May 13, 2024 Author

afbjorklund May 13, 2024 Maintainer

Ranjandas May 13, 2024 Author

afbjorklund May 13, 2024 Maintainer

Ranjandas May 14, 2024 Author

Ranjandas
May 10, 2024

Replies: 1 comment 11 replies

afbjorklund
May 10, 2024
Maintainer

Ranjandas May 12, 2024
Author

afbjorklund May 13, 2024
Maintainer

Ranjandas May 13, 2024
Author

afbjorklund May 13, 2024
Maintainer

Ranjandas May 13, 2024
Author

afbjorklund May 13, 2024
Maintainer

Ranjandas May 14, 2024
Author