vac.conf

vac.conf − Vac configuration file

vacd is a daemon which implements the Vacuum model on a factory (hypervisor) machine. vacd reads its configuration from /etc/vac.conf and .conf files in /etc/vac.d and these files are also read by the vac utility command to find default values.

The configuration files use the Python ConfigParser syntax, which is similar to MS Windows INI files. The files are divided into sections, with each section name in square brackets. For example: [settings]. Each section contains a series of option=value pairs. Sections with the same name are merged and if options are duplicated, later values overwrite values given earlier.

For ease of management, any configuration file ending in .conf in the directory /etc/vac.d will be read, in alphanumeric order by name, and then /etc/vac.conf is read if present.

The [settings] section is required and has options which apply to all virtual machines.

vac_space is required and gives the name of this Vac space. A single space will be sufficient for many sites. The space name should be a fully qualified domain name, like vac01.example.com and may be used as a virtual CE name in some monitoring and accounting systems. It should not be the canonical hostname of any of the factory machines.

factories is a space separated list of the fully qualified domain names of all the factories in this Vac space, including this factory. The factories are queried using UDP when a factory needs to decide which machinetype to start. The Vac responder process on the factories replies to these queries with a summary of the VM and the outcome of recent attempts to run a VM of each machinetype.

udp_timeout_seconds is how long to wait before giving up on more UDP replies. Defaults to 10.0 seconds.

cpu_per_machine sets the number of virtual CPUs to assign to each VM. Along with cpu_total, this is also used when calculating how many virtual machines can be created. Defaults to 1.

mb_per_cpu sets the memory allocated for each CPU in a VM in MiB (1024^2). If enough VMs will underuse memory or KSM is enabled, then it may be more efficient to increase this value and overcommit the factory’s physical memory to prevent the VM kernels from believing they need to swap. Defaults to 2048.

hs06_per_cpu gives the HEPSPEC06 power of each CPU in a virtual machine. This is used to calculate the total value /etc/machinefeatures/hs06 inside the VM and when writing APEL accounting records. Defaults to 1.

shutdown_time can be set to apply a limit on the shutdown time for all VMs. This value is used if it is ever earlier than the shutdown time calculated from the machinetype’s max_wallclock_seconds. In this case $MACHINEFEATURES/shutdowntime is updated for the VM and Vac will kill the VM if that shutdown time is reached. As shutdown_time is approached, VMs will be created with shorter and shorter lifetimes if their min_wallclock_seconds settings permit. shutdown_time is given in Unix seconds since 00:00:00 UTC on 1st of January 1970.

gocdb_sitename gives the GOCDB site name to use when writing APEL accounting record files to /var/lib/vac/apel-outgoing and /var/lib/vac/apel-archive. Please use your official site name to avoid the risk of misnamed records getting into the central APEL database. If gocdb_sitename is not given, then records are only written to apel-archive and the Vac space name is used as a placeholder in the files.

version_logger can be set to 0 to disable the logging of the version in use. Normally Vac sends one UDP factory_status message per day to vac-version-logger.gridpp.ac.uk on port 8884 to report the Vac version number. This is used by the Vac developers to target patches for security and bugs at the versions currently in use. Defaults to 1.

vacmon_hostport If set, this option gives a space-separated list of HOST:PORT to send VacQuery UDP messages to. This can be used to monitor the ongoing status of factories and VMs via site or central VacMon services.

cpu_total is derived from /proc/cpuinfo by default and does not usually need to be set explicitly. If set, then it provides an additional limit on the number of virtual machines that will be created, calculated as cpu_total divided by cpu_per_machine. This allows you to reduce the number of running VMs on a factory machine without having to kill running VMs. Instead, they are allowed to finish one by one and are only recreated if cpu_total allows.

overload_per_cpu sets the level of load per processor on the factory machine which will temporarily prevent the creation of more VMs. The number of processors is counted using /proc/cpuinfo and the one minute load average is taken from /proc/loadavg. VMs typically generate high loads during their initial set up, and this mechanism throttles the VM creation rate in response to the current overall load figure. Values of around 2.0 are ok with well-behaved VMs, but the default is more cautious. Default 1.25.

volume_group can be used to set the volume group to be searched for a logical volume which can be used by a particular VM. The logical volumes must have the VMs’ fully qualified domain names as their names. For example, /dev/vac_volume_group/factory1-00.example.com/ would be used by the VM factory1-00.example.com. Defaults to vac_volume_group.

disk_gb_per_cpu explicitly sets the size of disks to create for VMs in GB (1000^3). For logical volumes, Vac normally creates the disk size using the space available to Vac in volume_group, cpu_total, and cpu_per_machine. For file-backed virtual disks, the default value of 40 GB is used unless overriden by disk_gb_per_cpu.

fix_networking can be set to False to stop Vac trying to reset the lowlevel networking state if the vac_169.254.0.0 virtual network does not exist and cannot be created with libvirt. /var/log/vacd-factory receives details of what Vac detects and tries to do. Default True.

forward_dev can be used to specify which network interface on the factory will be used for outgoing network connections from the VMs to other machines. For example, eth1. This option is only needed if you have multiple network interfaces on the factory and only one should be used for traffic originating from the VMs.

The VM names are formed by adding a hyphen and the VM number to the hostname component of its fully qualified domain name. For example, factory1.example.com would have factory1-00.example.com, factory1-01.example.com, ... as its VMs. MAC addresses are formed with the prefix 56:4D as the first two bytes, and the four bytes of the IP address as the remaining four bytes.

One [machinetype ...] section must exist for each machinetype in the system, with the name of the machinetype given in the section name, such as [machinetype example]. A machinetype name must only consist of lowercase letters, numbers, and hyphens. Each of these sections contain option=value pairs that are specific to that machinetype.

machine_model is required and tells Vac how to configure the virtual hardware seen by the VMs of this machinetype. Currently cernvm3 or vm-raw. Default cernvm3.

target_share gives the desired share of the total VMs available in this space for this machinetype. The shares do not need to add up to 1.0, and if a share is not given for a machinetype, then it is set to 0. Vac factories consult these shares when deciding which machinetype to start as VMs become available.

backoff_seconds is the delay after a VM of this machinetype aborts. If a VM aborts, then no new VMs of this type will be created for this amount of time. This can be used to prevent the unnecessary creation of many VMs when no work is available, and avoid overloading the matcher or task queue of the VO.

fizzle_seconds is used in three places within the backoff procedure and in two other parts of Vac:
(1) First, if a VM finishes without producing a shutdown message code and has lasted less than fizzle_seconds, then it is treated as aborted.
(2) Secondly, after the backoff_seconds time has expired for a VM abort, once at least one VM has been started in this Vac space, then no more new VMs can be started for another fizzle_seconds.
(3) Thirdly, these new VMs are identified because they are still in the starting phase of creating files, or because they have been running for less than fizzle_seconds.
(4) Additionally, when writing the accounting log files, any VMs which run for less than fizzle_seconds are excluded.
(5) Finally, the heartbeat file checking is only carried out once an initial period of fizzle_seconds has passed.

heartbeat_file allows the machinetype to nominate a file which will be created in $JOBOUTPUTS before fizzle_seconds has passed. If this file is not created by then and maintained for the lifetime of the VM, the VM will be destroyed.

heartbeat_seconds gives the frequency at which the heartbeat_file must be updated after fizzle_seconds has passed. If the file is not updated for heartbeat_seconds then the VM will be destroyed. If heartbeat_seconds is 0, then only the existence of the file will be checked. Default 0.

max_wallclock_seconds gives the maximum lifetime of a VM. Vac will set $MACHINEFEATURES/shutdowntime for the VM using this value to communicate it to the VM. Vac will destroy the VM if it is still running after this amount of time. Default 86400.

min_wallclock_seconds gives the minimum remaining time required when creating a VM. This can be used to stop Vac creating VMs with short lifetimes when shutdown_time has been set. Default max_wallclock_seconds.

root_device is the device name exposed to the VM that is associated with the root disk image. Default vda.

scratch_device is the device name exposed to the VM that is associated with a scratch logical volume in the vm-raw model. Ignored for CernVM. Default vdb.

machines_dir_days sets the expiration time in days for per-VM directories created under /var/lib/vac/machines. Default 3.

accounting_fqan is used to specify a FQAN to include when writing APEL accounting records, to associate usage with particular experiments.

For the remaining options, if the file name begins with ’/’, then it will be used as an absolute path; otherwise the path will be interpreted relative to the machinetype’s subdirectory of /var/lib/vac/machinetypes

root_image is the path to the image file from which the VM will boot. With the cernvm3 machine_model, this can also be a remote HTTP or HTTPS URL which Vac will cache in /var/lib/vac/imagecache. The remote server must supply a Last-Modified timestamp and Vac will re-request the image each time a VM starts using an If-Modified-Since request to minimise network load. Alternatively, the images may be files in the local filesystem. With cernvm3 machine_model, the files are ISO CDROM-style boot images; with the cernvm2 machine_model, they are the root hard disk image itself.

cernvm_signing_dn is used to specify a regular expression to match the DN of an X.509 certificate used to verify the authenticity of the root image. Vac attempts to obtain the certificate and signature from a CernVM Signature Block at the end of the image file, verifies the certificate using the CA files in /etc/grid-security/certificates, and compares the certificate DN to cernvm_signing_dn. If this option is given, all these verification steps must be satisified for the image to be used. As of 2015, CernVM images are signed with a DN matching the regular expression /CN=cvm-sign01\.cern\.ch$

root_public_key is the file name of a public key supplied to the contextualization which will be allowed root ssh access. Setting this option to /root/.ssh/id_rsa.pub will give access from the factory machine.

user_data is the path of a contextualization file provided by the VO and perhaps modified by the site. If the path is a remote HTTP or HTTPS URL, Vac will fetch it over the network each time a VM is started. However the file is obtained, Vac will apply a series of default and locally defined ##user_data___## substitutions to it. See USER_DATA SUBSTITUTIONS below for a list of the default substitutions.

user_data_option_XXX and user_data_file_XXX are locally defined substitutions which will be applied to the user_data file before the VM is started. user_data_option_XXX takes the string to be substituted. user_data_file_XXX takes the relative or absolute path to a file whose contents will be substituted for the pattern in the user_data file.

user_data_proxy_cert and user_data_proxy_key are the locations of files containing X.509 certificate(s) and an RSA private key, all in PEM format, which will be used to make a limited X.509 proxy. The same file can be given for both options if desired.

legacy_proxy can be set to True to generate Globus legacy proxies rather than RFC 3820 proxies. Default False.

Before the user_data file is used in starting a VM, several pattern based substitutions are performed by Vac. These patterns are in the form ##user_data___##. String values given to the option user_data_option_XXX replace patterns of the form ##user_data_option_XXX##. The contents of the files given to user_data_file_XXX options also replace patterns of the form ##user_data_option_XXX##. In both cases XXX are arbitrary strings consisting of letters, numbers, and underscores.

The pattern ##user_data_option_x509_proxy## is replaced by the proxy created if the user_data_proxy_cert and user_data_proxy_key options are given.

In addition, the following substitutions are performed automatically by Vac using data it holds internally:

##user_data_uuid## is the UUID assigned to the VM by Vac.
##user_data_space## is the Vac space name.
##user_data_machinetype## and ##user_data_vmtype## (deprecated) are the name of the machinetype of this VM.
##user_data_machine_hostname## and ##user_data_vm_hostname## (deprecated) are the hostname given to the VM by Vac.
##user_data_manager_version## and ##user_data_vmlm_version## (deprecated) have the form "Vac v.v.v" where v.v.v is the Vac version.
##user_data_manager_hostname## and ##user_data_vmlm_hostname## (deprecated) are the hostname of the Vac factory machine.

Andrew McNab <Andrew.McNab@cern.ch>

More about Vac: http://www.gridpp.ac.uk/vac/

vacd(8), vac(1), check-vacd(8)