WIP : Run as snapshot for apple vm

[PicoCreator]

Note: This is the "redone" version of a previous WIP PR - #3792

---

#2688

The QEMU counterpart is

#3067

The general idea is to do an APFS clone (Meaning it will only require the delta additional space, and make no changes on the original) on VM startup. ie. cp -c command. This allows us to mimic a qemu <disk> --snapshot like behaviour.

The goal is to support the disposable VM workflow (not so much on the snapshot management)

[PicoCreator]

Update - it now works !!! Both of this options is available

• A persistent single option to always run the VM as snapshot
• Snapshot option per-drive

The setup function has also been designed to potentially be used with future "context menu" option.

---

Follow up questions to consider ?

• should we have a "Tmp" folder for such snapshot instead of populating the "Data" folder.

[PicoCreator]

Short demo of the feature options

https://www.youtube.com/watch?v=XdqdCxRbsec&ab_channel=EugeneCheah

@ktprograms

[osy]

@PicoCreator I know this PR is marked as WIP but is it ready to be reviewed?

[osy]

Some high level comments:

1. I don't think the logic for this should go in UTMAppleConfiguration. In division of responsibility, I see UTMAppleConfiguration mostly as an interface between UTM and VZVirtualMachineConfiguration and should only have logic to convert between the two. I think it's more fitting to have the logic in UTMAppleVirtualMachine, preferably as an extension.
2. I don't think this should be a setting option that is saved. To be consistent with Add support for running VM as temporary snapshot #3067 we will have a context menu option to run with snapshot.
3. Should we disallow it for shortcut VMs? I'm just not sure how we can detect if the VM is not on another volume like a network volume or FAT32 drive which will not support shallow copy.

[osy]

For 3, it seems like you can use volumeSupportsFileCloningKey to detect if COW is supported by the volume.

[iby]

@PicoCreator thanks for pushing this so far! Would so be awesome to have this feature! ❤️

[PicoCreator]

@oxy & @iby : Sorry for the delay, Had a flood of work, followed by covid, to catching up with work post-covid 😅

Will try to incorporate the changes required. Once things stabilise.

[galaxy4public]

@PicoCreator , unfortunately, the title is misleading since what you are working on is in no way APFS snapshots (which require the com.apple.developer.vfs.snapshot entitlement to be performed). You stipulated in a comment that FileManager.copyItem() does a "shallow" copy. I could not find any evidence to support that. More likely, you see the quick creation of the file due to the disk cache and other FS optimisations, but the resulting file is most likely a full copy and not a delta. (it turns out to be correct, since it seems to use clonefile(2), which is CoW on APFS, so you do get a delta in the majority of scenarios).

Anyway, I am playing with the idea of true APFS snapshots for UTM and so far it looks as follows:

1. for each VM I provision a disk image containing an APFS Volume group (this is to enable the snapshot functionally regardless of the backend virtualisation);
2. each virtual disk for the VM is allocated its own APFS volume inside the group above
3. I wrote a small tool (10 lines of code) that requests APFS to take a snapshot or revert to the snapshot taken of the specified volume in the group with a sensible label attached to it (the tool is using fs_snapshot_create(2) and is signed with the com.apple.developer.vfs.snapshot and com.apple.private.apfs.revert-to-snapshot entitlements attached to it (See below). So, at any point in time I can take a snapshot of the VM and later restore to that point using my tool. I also can manage snapshots (like listing and deleting them) using diskutil.
4. UTM does not need to know anything about that backend activities and works as it is just fine.

The ideal approach, in my opinion, would be to actually support dynamic allocation of an APFS volume group and the corresponding volumes per disk image inside UTM, this will open the opportunity to have a separate tool that handles snapshots/restores with a nice UI (basically, something like my tool, but on steroids and accessible from UTM's interface).

This was the good news part. The bad news part is that Apple in their wisdom are explicitly restricting developers ability to do APFS snapshots. The aforementioned entitlements are only issued to enrolled Apple Developers after a very close review (usually backup software companies are getting these entitlements). The way my tool works is that I have a macOS VM with both SIP and AMFI (through nvram boot-args=amfi_get_out_of_my_way=0x1) disabled, then I signed my binary with the corresponding entitlements using a fully trusted self-signed certificate. In that VM, I can mount the host folder with UTM VM disk images and do snapshotting and restoring back. It is inconvenient for a common user, but in my case is fully scripted.

If UTM decides to go this route, which is a universal approach for managing snapshots on macOS for any virtualisation backend, I think the project can apply for the corresponding entitlements and get it integrated in a nice to use way.

I hope somebody would find this useful.

P.S. I've spent around 3 hours to come up with a working solution using the "duct tape and sticks" approach, so I would estimate this effort (bar getting the entitlements from Apple) as a "small to medium" in effort.
P.P.S. So far, I am in love with the project and am going to ditch my paid subscriptions to Parallels and VMware (and instead will donate this money to the project).

[grantwest]

The qcow2 format supports snapshots and qemu already supports using this. Why add a dependency on APFS?

[ideologysec]

The qcow2 format supports snapshots and qemu already supports using this. Why add a dependency on APFS?

Because this is for Apple Virtualization machines, that don't use qcow2. If you want snapshots of those machines (or Linux apple-virt), this is the only way.

[esaruoho]

any luck with progressing with this, @PicoCreator ? would be a lovely feature to have indeed.