# Create an IOC Instance ## Introduction The last section covered deploying and managing the example IOC Instance that came with the template services repository. Here we will create a new IOC Instance that implements a simulated detector. For this tutorial some familiarity with the EPICS AreaDetector framework is useful. Take a look at this documentation if you have not yet come across AreaDetector: . (create-new-ioc-instance)= ## Add a New IOC Instance to t01-services ### Introduction To create a new IOC Instance simply add a new folder to the `services` folder in your services repo. The name of the folder will be the name of the IOC. This folder needs to contain these items: ```{eval-rst} =================== ==================================================== **compose.yml** Instructions to compose to say what to deploy **config** A folder that contains the IOC instance configuration files. =================== ==================================================== ``` The configuration files in the config folder can take a number of forms [listed here](https://github.com/epics-containers/ioc-template/blob/main/template/ioc/start.sh). The recommended contents of this folder is a single ibek IOC description yaml file named `ioc.yaml`. But the generic IOCs support raw startup script, substitution files and database files as well. Thus, users that have their own preferred way of generating startup assets can use that method, including hand written files. ### Using .ioc-template The template project added a folder to `services` called `.ioc-template` this is a useful starting point to make a new service. Let us call our new IOC `bl01t-ea-cam-01`. We will copy the `.ioc-template` folder to a new folder called `bl01t-ea-cam-01` and then edit the `compose.yml` file to reflect the new IOC name: ```bash cd t01-services code . # navigate to the services folder and copy the .ioc-template folder # paste it into the services folder and rename it to bl01t-ea-cam-01 ``` ### Compose.yml Our new example IOC will be a simulation detector using the AreaDetector SimDetector. There is already a Generic IOC for the SimDetector, therefore to create an IOC Instance, we just need to refer to that Generic IOC container image and provide some configuration for it. You can find the Generic IOC container source for SimDetector here: . This repository publishes its container image at: `ghcr.io/epics-containers/ioc-adsimdetector-runtime:2024.9.1`. Later tutorials will cover how to build and publish your own Generic IOC container images. Edit the `compose.yml` file in the `bl01t-ea-cam-01` folder to reflect the new IOC name and to refer to the Generic IOC container image for the SimDetector: - find and replace **replace_me** with **bl01t-ea-cam-01** - replace **replace_with_image_uri** with **ghcr.io/epics-containers/ioc-adsimdetector-runtime:2024.9.1** That's it for the `compose.yml` file. This file is essentially boilerplate and would look very similar for every IOC Instance you create. The two unique things that this file does are: - determine the name of the IOC - determine the container image to use Your compose.yml should now look like this: ```yaml services: bl01t-ea-cam-01: extends: service: linux_ioc file: ../../include/ioc.yml image: ghcr.io/epics-containers/ioc-adsimdetector-runtime:2024.9.1 labels: version: 0.1.0 environment: IOCSH_PS1: bl01t-ea-cam-01 > IOC_NAME: bl01t-ea-cam-01 volumes: - ../../opiauto-generated/bl01t-ea-cam-01:/epics/opi configs: - source: bl01t-ea-cam-01_config target: epics/ioc/config configs: bl01t-ea-cam-01_config: file: ./config include: - path: ../../include/networks.yml ``` You can read about the format of a compose file here: . Below is a brief description of the fields in our compose file: ```{eval-rst} ================== ======================================================= field description ================== ======================================================= bl01t-ea-cam-01 declares that we are creating a service called **bl01t-ea-cam-01** extends most of the definition of an ioc comes from the file **include/ioc.yml**, we are extending that, which means the dictionaries described by the two files are merged image the container image to use labels arbitrary labels that can be used to filter services - we use one to indicate the version of the IOC Instance (as opposed to the version of the Generic IOC) environment environment variables to set in the container IOCSH_PS1 ioc shell prompt IOC_NAME the name of the IOC volumes mount the IOC OPI files into the container at **/epics/opi** configs mount the IOC config folder into the container at **/epics/ioc/config** include include a shared definition of the container networks we use ================== ======================================================= ``` ### IOC Instance Config The config folder can contain a variety of different files [as listed here](https://github.com/epics-containers/ioc-template/blob/main/ioc/start.sh). In this case we are going to define the Instance using an `ibek` IOC instance yaml file. *IOC yaml* files are a sequence of `entities`. Each entity is an instance of an `entity_model` declared in the *Support yaml* that one of the support modules provides. `entity_models` can: - define a set of parameters, that are to be used as substitutions below - add templated lines of code to the startup script with substitutions - instantiate 1 more more EPICS Database templates with a set of macro substitutions Each `entity` listed in the *IOC yaml* file will create an instance of the support module `entity_model` that it refers to. It will pass a number of arguments to the `entity_model` that will be used to generate the startup script entries and EPICS Database entries for that entity. The `entity_model` is responsible for declaring the parameters it expects and how they are used in the script and DB entries it generates. It supplies types and descriptions for each of these parameters, plus may supply default values. We will be creating a simulation detector from the `ioc-adsimdetector` Generic IOC. The following *Support yaml* for the simulation detector is baked into the container. Once you have your container up and running you can use `docker compose exec bl01t-ea-cam-01 bash` to get a shell inside and see this file at **/epics/ibek_defs/ADSimDetector.ibek.support.yaml**. ```yaml # yaml-language-server: $schema=https://github.com/epics-containers/ibek/releases/download/3.0.1/ibek.support.schema.json module: ADSimDetector entity_models: - name: simDetector description: |- Creates a simulation detector parameters: P: type: str description: Device Prefix R: type: str description: Device Suffix PORT: type: id description: Port name for the detector TIMEOUT: type: str description: Timeout default: "1" ADDR: type: str default: "0" description: Asyn Port address WIDTH: type: int default: 1280 description: Image Width HEIGHT: type: int default: 1024 description: Image Height DATATYPE: type: int description: Datatype default: 1 BUFFERS: type: int description: Maximum number of NDArray buffers to be created for plugin callbacks default: 50 MEMORY: type: int description: Max memory to allocate, should be maxw*maxh*nbuffer for driver and all attached plugins default: 0 pre_init: - type: text value: | # simDetectorConfig(portName, maxSizeX, maxSizeY, dataType, maxBuffers, maxMemory) simDetectorConfig("{{PORT}}", {{WIDTH}}, {{HEIGHT}}, {{DATATYPE}}, {{BUFFERS}}, {{MEMORY}}) databases: - file: $(ADSIMDETECTOR)/db/simDetector.template args: P: R: PORT: TIMEOUT: ADDR: pvi: yaml_path: simDetector.pvi.device.yaml ui_macros: P: R: pv: true pv_prefix: $(P)$(R) ``` You can see that this lists a number of parameters that it requires and several others that have defaults. It then declares how these will be used to substitute values into the simDetector database template. Finally it declares some lines to go into the startup script (`pre_init` means this goes before `iocInit`). Note that the process for taking a *Support yaml* entity_model with values from *IOC yaml* entity and generating a startup script and EPICS Database uses Jinja2 templating. In its simplest form this just means that you can use `{{ }}` to substitute values from the *IOC yaml* arguments into the *Support yaml* `pre_init` and `databases` sections. When the database section provides no value for the parameters it lists this means that the argument is used verbatim, e.g. **$(ADSIMDETECTOR)/db/simDetector.template** is instantiated with `PORT=$(PORT)`, `P=$(P)` etc. To learn more about Jinja templating see here: . Therefore, we can update our *IOC yaml* file by adding an ADSimDetector entity to the entities list. In vscode, open **services/bl01t-ea-cam-01/config/ioc.yaml** and edit the boilerplate template so that it looks like the following: ```yaml # yaml-language-server: $schema=https://github.com/epics-containers/ioc-adsimdetector/releases/download/2024.6.1/ibek.ioc.schema.json ioc_name: "{{ _global.get_env('IOC_NAME') }}" description: An IOC instance that simulates a detector entities: - type: epics.EpicsEnvSet name: EPICS_TZ value: GMT0BST - type: devIocStats.iocAdminSoft IOC: "{{ ioc_name | upper }}" - type: ADSimDetector.simDetector PORT: DET.DET P: BL01T-EA-CAM-01 R: ":DET:" ``` :::{note} If you are unfamiliar with YAML then you could take a look at the YAML spec here: . It is an extension of JSON (javascript object notation) that is designed to be human readable/writeable. It is also the format for all Kubernetes configuration files. Be aware that white space is significant. i.e. indentation represents nesting. Above we have a list of entities, each list item is denoted by `-`. There are currently 3 entities in the list, each of which is a dictionary of key value pairs. The last entry we just added has first key of `type` with value `ADSimDetector.simDetector`. ::: This will create us a simulation detector driver with PV prefix `BL01T-EA-CAM-01:DET:` that publishes its output on the Asyn port `DET.DET`. Be aware that YAML tries to minimize the amount of punctuation used. So strings can usually be written without quotes. But ':' and '{' are significant characters and if your string starts with one of these you must quote it. Note that the Generic IOC includes all of the support modules that are dependencies of `ADSimDetector` and each of those contributes its own set of `entity_models` in its own *Support yaml* file. Let us also add an `AreaDetector` plugin and wire it to our simulation detector by adding this to our *IOC yaml* file: ```yaml - type: ADCore.NDStdArrays PORT: DET.ARR P: BL01T-EA-CAM-01 R: ":ARR:" NDARRAY_PORT: DET.DET TYPE: Int8 FTVL: CHAR NELEMENTS: 1048576 ``` This adds a Standard Arrays plugin to the IOC that will publish the output of the simulation detector via channel access. The *Support yaml* that declared the plugin came from the ADCore module. This is a dependency of ADSimDetector and so is included in the Generic IOC container. You have now defined your first IOC instance. ## Trying Out The IOC Instance ### Update the Services List In the root of our services repository is the root compose.yml file that includes all of the services. We will edit that to add our new IOC. The result should look like this: ```yaml include: # test and deploy profiles - services/example-test-01/compose.yml - services/bl01t-ea-cam-01/compose.yml - services/gateway/compose.yml # dev and test profiles - services/phoebus/compose.yml # deploy profile only - services/epics-opis/compose.yml ``` ### Change the OPI screen To make this tutorial more interactive, the template includes a hand coded bob screen for the ADSimDetector you just made. It has the few widgets necessary to start the detector, enable the stdarrays plugin and view the stdarrays plugin output. You can make phoebus automatically load this screen upon startup by changing the `command` entrypoint in the `phoebus` service's `compose.yml` file. The bob file you need is called opi/demo-simdet.bob. i.e. ```yaml command: phoebus-product/phoebus.sh -settings /config/settings.ini -resource /opi/demo-simdet.bob -server 7010 ``` ### Launch the IOC Instance We can launch all the services in the beamline as we did in the earlier tutorial as follows: ```bash cd t01-services source ./environment.sh docker compose up -d ``` The new screen will allow you to hit 'Acquire' on the CAMERA pane, and 'Enable' on the Standard Array pane. You should see the moving image from the simulation detector in the right hand image pane. The screen has a link to the autogenerated OPI screens for the IOC Instance. If you click the `bl01t-ea-cam-01` button you will see a list of the 'entities' that were instantiated in the IOC Instance. You can click on each of these to see the engineering screen of the entity which includes all of the PVs that it made. ## Ibek Explanation ### Overview Above we looked at some ibek *Support yaml* and created an *IOC yaml* file. The details of where *Support yaml* files come from and how to create your own are covered in a later tutorial {any}`generic_ioc`. However, without looking into the set of *Support yaml* files that are inside a given Generic IOC we can still make a meaningful *IOC yaml* file. That is because every Generic IOC publishes an *IOC schema* that describes the set of entities that an instance of that IOC may instantiate. The Generic IOC we used was released at this location: . This page includes the assets that are published as part of the release and one of those is `ibek.ioc.schema.json`. This is the *IOC schema* for the `ioc-adsimdetector` Generic IOC. This is what we referred to at the top of our *IOC yaml* file like this: ```yaml # yaml-language-server: $schema=https://github.com/epics-containers/ioc-adsimdetector/releases/download/2024.8.2/ibek.ioc.schema.json ``` When editing with a YAML aware editor like VSCode this will enable auto completion and validation of the *IOC yaml* file. To enable this in VSCode you will need to install the YAML extension from here: Now is a good time to try installing the extension and experimenting with editing the *IOC yaml* file. Using `ibek` yaml files to describe IOC instances has the following advantages: - there is pre-runtime checking that the IOC Instance is valid - instance authors are guided by schema - details of what a support module needs to be instantiated are under the control of the support module author (at Generic IOC specification time). - functions with long argument lists are made easier to use because the instance author supplies named arguments only. However, if you already have a framework for generating startup assets or you prefer hand coding them, this is also supported. ### Experiment With Changes The engineering screens are generated by PVI at startup. We will look into PVI in more detail in a later tutorial. But for now you could experiment with adding more AreaDetector plugins to your IOC Instance and seeing how the additional engineering screens get added. If you try editing your *IOC yaml* file you will see that what you can add is controlled by the schema that the Generic IOC publishes. This makes it easier to create valid IOC Instances. First make sure that you have the 'Redhat YAML' extension installed in VSCode. This will give you auto completion and validation of your *IOC yaml* file based on the schema line at the top of the file. Now try adding the following to the entities list in your *IOC yaml* file: ```yaml - type: ADCore.N ``` You should find that auto-completion will list all the types of plugin that you can add. Go ahead and complete it with 'NDProcess'. Now you will see a red squiggle at the start of 'type'. Hover over this and it will show you the parameters that you are required to supply to NDProcess. :::{figure} ../images/ioc-yaml-schema.png using schema to add an NDProcess plugin ::: Fill out the rest of your NDProcess entity as follows: ```yaml - type: ADCore.NDProcess PORT: DET.PROC P: BL01T-EA-CAM-01 R: ":PROC:" NDARRAY_PORT: DET.DET ``` Now restart your simulation detector IOC: ```bash docker compose restart bl01t-ea-cam-01 ``` Once it is back up you can click on the bl01t-ea-cam-01 button in the 'Autogenerated Engineering Screens' pane and you will see a new 'NDProcess' entity. If you know about wiring up AreaDetector you can now wire this plugin into your pipeline and make modifications to the image data as it passes through. (raw-startup-assets)= ## Raw Startup Script and Database This section demonstrates how to use your own startup assets. This involves placing your own `st.cmd` and `ioc.subst` files in the **config** folder. Or alternatively you could override behaviour completely by placing `start.sh` in the **config** folder, this can contain any script you like. To see what ibek generated you can go and look inside the IOC container: ```bash docker compose exec bl01t-ea-test-02 cd /epics/runtime/ cat ioc.subst cat st.cmd ``` :::{note} The startup script and database are generated at container run time, by `ibek`. They are generated in the /epics/runtime folder of the container. In Kubernetes this will be a persistent volume so that it can be shared for easy debugging of IOC Instances. ::: If you would like to see an IOC Instance that uses a raw startup script and database then you can copy these two files out of the container and into your IOC Instance config folder like this: ```bash docker cp bl01t-ea-cam-01-1:/epics/runtime/st.cmd services/bl01t-ea-cam-01/config docker cp bl01t-ea-cam-01-1:/epics/runtime/ioc.subst services/bl01t-ea-cam-01/config/ioc.subst # no longer need an ibek ioc yaml file rm services/bl01t-ea-test-02/config/ioc.yaml ``` You will need to make a minor change to the `ioc.subst` file. Edit this and remove references to the two template files with `.pvi` in their name. These are PVI generated templates for use with OphydAsync and are not available in manually built IOC Instances. Your IOC Instance will now be using the raw startup script and database. But should behave exactly the same as before. You are free to experiment with changes in the startup script and substitution file and re-deploy the IOC. Restart the IOC to see it operating as before (except that engineering screen generation will no longer happen): ```bash docker compose restart bl01t-ea-cam-01 ``` :::{note} This same IOC instance will be used in later tutorials. Therefore we should move these changes to a branch and restore the original **ioc.yaml** based version: ```bash git checkout -b raw-startup git add . git commit -m "use raw startup script and database" git checkout main ``` :::