Skip to content

Driver: Native

Jump to bottom
Sergei Parshev edited this page Mar 4, 2024 · 8 revisions

Native driver

Native is the close-to-baremetal provider, which allows to run your workload nearby the Fish node, but as separated user. The user creates dynamically and could have all the benefits of pre-isntalled software on the system. Images here are mostly simple tar archives which contains layers with dirs and files unpacked on top of each other.

How it works?

How Native driver works?

It creates user and needed volumes, then starts workload as background task (fork) as the created user. When workload is done - destroy will clean up all the user's processes and leftovers.

For that it needs additional sudo/runas permissions, so make sure you setup them properly.

Usage

To use the driver you need:

  • Setup the sudo/runas permissions:
    • MacOS:
      1. On modern MacOS you will need to set Full Disk Access for aquarium-fish binary, otherwise it will be unable to delete the user and will fail with next message during verification on startup (PR#57 comment): 
        2024/02/23 10:45:35 ERROR:	Native: Unable to delete user: Native: Command exited with error: exit status 40: <main> delete status: eDSPermissionError
<dscl_cmd> DS Error: -14120 (eDSPermissionError)
        To fix that you will need to go to MacOS settings into Privacy & Security and enable Full Disk Access for aquarium-fish when it will fail the first time. If it will not fail to delete the user - you would not need to enable Full Disk Access.
      2. Modify sudoers by saving the next content using sudo visudo /etc/sudoers.d/aquarium-fish-driver-native. Please create /etc/sudoers.d directory if it does not exist. 
        # Configuration allows aquarium-fish to run native driver
# Here _aquarium-fish - is user, for other things please check `man sudoers`

# Create new user
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/dscl . create /Users/fish-[a-z][a-z][a-z][a-z][a-z][a-z] *
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/dscl . append /Groups/* GroupMembership fish-[a-z][a-z][a-z][a-z][a-z][a-z]
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/sbin/createhomedir -c -u fish-[a-z][a-z][a-z][a-z][a-z][a-z]
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/sbin/chown -R fish-[a-z][a-z][a-z][a-z][a-z][a-z]\:* /Volumes/fish-[a-z][a-z][a-z][a-z][a-z][a-z]_*/
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/tar -xf - --uname fish-[a-z][a-z][a-z][a-z][a-z][a-z] -C /Users/fish-[a-z][a-z][a-z][a-z][a-z][a-z]/
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/tar -xf - --uname fish-[a-z][a-z][a-z][a-z][a-z][a-z] -C /Volumes/fish-[a-z][a-z][a-z][a-z][a-z][a-z]_*/
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/mdutil -i off /Volumes/fish-[a-z][a-z][a-z][a-z][a-z][a-z]_*/

# Run env init script
_aquarium-fish ALL=(root) NOPASSWD: /usr/bin/su -l fish-[a-z][a-z][a-z][a-z][a-z][a-z] *

# Delete user
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/killall -INT -u fish-[a-z][a-z][a-z][a-z][a-z][a-z]
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/killall -KILL -u fish-[a-z][a-z][a-z][a-z][a-z][a-z]
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /usr/bin/dscl . delete /Users/fish-[a-z][a-z][a-z][a-z][a-z][a-z]
_aquarium-fish ALL=(root) NOPASSWD:NOEXEC: /bin/rm -rf /Users/fish-[a-z][a-z][a-z][a-z][a-z][a-z]
  • Create the image - Aquarium Bait will be the simplest way
  • Run the Aquarium Fish node, create Label and send Application to receive the resource you want

Configuration

Describes the driver options in the drivers section in the aquarium-fish config file:

drivers:
  - name: native
    cfg:
      su_path:        string  # Path to the su (login as user) binary
      sudo_path:      string  # Path to the sudo (privilege escalation) binary
      sh_path:        string  # Path to the sh (simple user shell) binary
      tar_path:       string  # Path to the tar (unpacking images) binary
      mount_path:     string  # Path to the mount (list of mounted volumes) binary
      chown_path:     string  # Path to the chown (change file/dir ownership) binary
      chmod_path:     string  # Path to the chmod (change file/dir access) binary
      killall_path:   string  # Path to the killall (send signals to multiple processes) binary
      rm_path:        string  # Path to the rm (cleanup after execution) binary

      images_path:    string  # Abs/rel path to the directory where images will be downloaded and cached, by default `$CWD/fish_native_images`
      workspace_path: string  # Abs/rel path to the directory where the disks will be placed, by default `$CWD/fish_native_workspace`

      dscl_path:          string  # Path to the dscl (macos user control) binary
      hdiutil_path:       string  # Path to the hdiutil (macos images create/mount/umount) binary
      mdutil_path:        string  # Path to the mdutil (macos disable indexing for disks) binary
      createhomedir_path: string  # Path to the createhomedir (macos create/prefill user directory) binary

      # Alter allows you to control how much resources will be used:
      # * Negative (<0) value will alter the total resource count before provisioning so you will be
      #   able to save some resources for the host system (recommended -2 for CPU and -10 for RAM
      #   for disk caching)
      # * Positive (>0) is also available, but you're going to put more load on the scheduler
      #   Please be careful here - noone wants the workload to fail allocation because of that...
      cpu_alter: int  # 0 do nothing, <0 reduces number available CPUs, >0 increases it (dangerous)
      ram_alter: int  # 0 do nothing, <0 reduces amount of available RAM (GB), >0 increases it (dangerous)

      # Overbook options allows tenants to reuse the resources
      # It will be used only when overbook is allowed by the tenants. It works by just adding those
      # amounts to the existing total before checking availability. For example if you have 16CPU
      # and want to run 2 tenants with requirement of 14 CPUs each - you can put 12 in CpuOverbook -
      # to have virtually 28 CPUs. 3rd will not be running because 2 tenants will eat all 28 virtual
      # CPUs. Same applies to the RamOverbook.
      cpu_overbook: uint  # How much CPUs could be reused by multiple tenants
      ram_overbook: uint  # How much RAM (GB) could be reused by multiple tenants

      download_user:     string  # User used to download the images (from artifact storage), uses HTTP Basic Auth
      download_password: string  # Password for the user

Label definition

Describes the available options of the driver label definition:

definition:
  driver: native

  options:
    images: list  # Contains the locations of images to use
      - url: string      # Url address of the remote image to download it
        sum: string      # Optional (but really recommended) checksum of the image in format "<algo>:<checksum>"
        name: string     # Optional name of the image, if not set will use a part of the Url file name prior to last minus ("-") or ext
        version: string  # Optional version of the image, if not set will use a part of the Url file name after the last minus ("-") to ext
        tag: string      # Optional identifier used by native driver to know which disk to use to unpack the image, if empty - uses user home dir
    entry: string  # Path to binary to be executed as workload. Could use go templates like "{{ .Disks.disk_name }}"
    groups: list   # Which groups to add to the dynamic user during creation, first one is primary one
      - string
      - ...

  resources:
    node_filter: list  # List of the Fish Node identifiers, which allow to select the appropriate node for execution. Could use wildcards from path.Match
      - string
      - ...

    cpu:   uint  # Amount of CPUs (threads) required for the environment
    ram:   uint  # Amount of memory (in GB) required for the environment
    disks: map   # The disks to create/use in the environment
      <name>:          # The name of the disk file and label of the filesystem
        size:  uint    # The size of the disk (in GB), "dir" type will not restrict the amount of space for the resource

    # The modificators to simultaneous execution
    multitenancy: bool  # Tolerate to run along with the others
    cpu_overbook: bool  # Tolerate to CPU overbooking
    mem_overbook: bool  # Tolerate to RAM overbooking

    lifetime: duration  # Lifetime of the Resource in "1h2m3s" format. If "" or "0" - then default will be used, if negative - no timeout.

Available ApplicationTask's

None

Examples:

Clone this wiki locally