Autosave for IOCs#
Introduction#
Autosave saves the state of an IOC to disk and restores it on restarts. epics-containers manages autosave configuration in a prescriptive fashion that is discussed here.
The official autosave documentation in here.
Different facilities have used different approaches to configuring autosave. epics-containers has settled on using .req
files. These files simply list the set of PVs to save with autosave. The naming convention for these files is the same as that which the AreaDetector modules have adopted. EPICS Db info tags were also considered but rejected because supplying overrides to them would be problematic.
Because of this, AreaDetector Support modules can use autosave in epics-containers without any additional configuration (unless you disagree with the default choice of saved PVs). Many other support modules have adopted the same approach and will just work.
Other support modules can have their autosave configuration specified in ibek-support or in individual IOC instances.
Configuring autosave for IOCs#
To use autosave, the generic IOC being used must have compiled in the autosave module. The following snippet in the Dockerfile
will do this:
COPY ibek-support/autosave/ autosave
RUN ansible.sh autosave
To enable autosave in an IOC instance that uses the above generic IOC requires adding the following to the ioc.yaml instance configuration (our examples are for the motion IOC BL45P-MO-IOC-01):
- type: autosave.Autosave
P: BL47P-MO-IOC-01 # a prefix for all autosave monitoring PVs
Startup Script#
The startup script entries that handle autosave will look the similar for all IOC instances as follows.
First in pre init stage we have:
# Autosave pre iocInit
set_requestfile_path("/epics", "autosave")
set_savefile_path("/autosave")
save_restoreSet_status_prefix BL47P-MO-IOC-01
save_restoreSet_Debug 0
save_restoreSet_NumSeqFiles 3
save_restoreSet_SeqPeriodInSeconds 600
save_restoreSet_DatedBackupFiles 1
save_restoreSet_IncompleteSetsOk 1
set_pass0_restoreFile autosave_positions.sav
set_pass1_restoreFile autosave_settings.sav
asSetFilename $(PVLOGGING)/src/access.acf
save_restoreSet values can be adjusted with arguments in the yaml but the pass0 and pass1 save files are fixed for epics-containers. These files will be saved into /autosave
inside the IOC instance’s container filesystem. Along side them will be the sequence files and dated backup files.
/autosave
will be mounted into the namespace’s autosave Persistent Volume Claim with a subfolder named after the IOC instance. For our BL47P-MO-IOC-01 example we have:
namespace: p47-beamline
PVC name: p47-autosave-claim
subPath in the PVC: bl47p-mo-ioc-01
Hence a pod that mounts the root of the PVC p47-autosave-claim
will be able to browse the autosave files of all IOC instances in the namespace.
After iocInit we have
# Autosave post iocInit
create_monitor_set autosave_positions.req, 5, ""
create_monitor_set autosave_settings.req, 30, ""
The two req files list the PVs and have fixed names in epics-containers:
autosave_positions.req: lists PVS to be restored at phase 0 before record processing starts (usually motor positions)
autosave_settings.req: lists PVS to be restored at phase 1 after record processing starts
These files are generated by ibek from the req files in the support modules included in the generic IOC but can be also overridden on an instance basis.
Sources of req files#
There are 3 places ibek sources autosave req files from:
The built support module may have req files in its
Db
database template folder alongside the Database templatesThe ibek-support subfolder for the support module, these may override the above
The config folder for the ioc instance, these override both of the above
In all cases the filenames must follow a strict pattern to identify the template from which the referenced PVs are being sourced. (this is the Areadetector approach). Overriding is simply a matter of creating the same file name in ibek-support or instance config.
Pattern: <template_stem_name>_{positions|settings}.req
e.g.
template: basic_asyn_motor.template
phase 0 .req file: basic_asyn_motor_positions.req
phase 1 .req file: basic_asyn_motor_settings.req
Build time behaviour#
The initial gathering of the .req
files is done at container build time. Each time ansible.sh
is executed in the Dockerfile it builds the given support module, but also symlinks all .req
files from ibek-support/<module>
to the \epics\autosave
folder in the container.
These files are provided by the creator of the ibek-support/<support_module>
module recipe. If the upstream support module includes req files that match the AreaDetector naming convention then there is no need to provide any .req
files here.
If the support module has .req
files with a different naming convention then one way to support this is to copy those files into the ibek-support/<support_module>
folder and rename them appropriately. This is in keeping with the epics-containers philosophy of not requiring changes to the upstream support modules in order to support them. However, if the support module maintainer is open to adopting the defacto standard for .req
files then that is the preferred approach.
Adding .req
files to ibek-support
gives us a way to supply autosave information that has historically been done out of band. DLS have a specific use for this as discussed in Diamond Light Source Autosave Approach.
Runtime behaviour#
All of the .req
files supplied by support modules include the same macros as the templates that their PVs come from. Hence these need to be substituted with values that the individual IOC instance is using before passing to autosave.
In addition the multiple .req
files from multiple support modules need to be gathered into a single file for each of the autosave phases. These are to be called autosave_positions.sav.req
and autosave_settings.req
and passed to the create_monitor_set
function as we saw above.
Both file gathering and substitution are handled by ibek in the start.sh
script that all epics-containers IOC instances use. The command that performs this step is:
ibek runtime generate-autosave
This performs the following steps:
symlinks all
.req
files found in the support modules of the generic IOC to\epics\autosave
if the file exists then don’t overwrite - allowing the
ibek-support
to provide overrides
symlinks all
.req
files found in\epics\ioc\config
to\epics\autosave
overwriting any existing files thus providing instance overrides
generates two substitution files
/epics/runtime/autosave_positions.subst
/epics/runtime/autosave_settings.subst
runs MSI over the above substitution files, with the path pointing to
\epics\autosave
The substitution files are copies of /epics/runtime/ioc.subst
except that the EPICS Db template file names are replaced with autosave req file names using the naming convention described in Sources of req files.
The MSI output is two files autosave_positions.sav.req
and autosave_settings.req
which are in turn passed to create_monitor_set
in the startup script.z
Diamond Light Source Autosave Approach#
At DLS we generate our .req files from comments added to the template files. This has had the unfortunate side effect of making DLS forks of support modules deviate from upstream. This does not fit well with the epics-containers philosophy of using upstream support modules without modification.
For this reason we supply a tool to extract the comment information from the internal DLS fork of the support modules and generate the .req
files.
This tool should be used when:
the upstream support module does not provide
.req
files in the AreaDetector stylethe DLS fork of the support module has autosave comments in the template files
the module is DLS internal or is public but originated from DLS
The tool is called builder2ibek
and can be installed from PyPi.
The following example was used to extract the autosave information from the pmac
module:
cd /dls_sw/prod/R3.14.12.7/support/pmac/2-5-22
builder2ibek autosave db/* --out-folder /scratch/hgv27681/work/ioc-pmac/ibek-support/pmac