.. SPDX-FileCopyrightText: 2024 - 2025 Univention GmbH .. .. SPDX-License-Identifier: AGPL-3.0-only .. _nubus-packaged-integrations-udm-data-loader: UDM data loader =============== .. dropdown:: Deployment — Kubernetes :color: info :icon: rocket This page refers to the Nubus for Kubernetes deployment. As a developer or DevOps engineer, you need directory objects in the directory service for your packaged integration. If this scenario describes your customization case, then the *UDM data loader* is for you. During its initial deployment, Nubus for Kubernetes creates a bunch of directory objects in the directory service. Data in the YAML format describes the wanted state of directory objects. The *UDM data loader* creates or modifies UDM objects accordingly, which applies business logic in UDM, creates and saves the directory objects. You can use the same mechanism to add directory objects for your packaged integration. This page describes how you define the data, bundle, and apply it. .. admonition:: Example For the *UDM data loader*, see the following implementation example: * `UDM data loader in OX Connector `_ .. seealso:: :external+uv-nubus-kubernetes-architecture:ref:`component-directory-manager-startup-nubus` in :cite:t:`uv-nubus-kubernetes-architecture` for information on the architecture for populating data on initial startup of Nubus. :numref:`nubus-packaged-integrations-udm-data-loader-fig` shows the procedure for creating and using the *UDM data loader* plugin. The top row illustrates the involved roles *Software Developer* and *Operator*. The *Software Developer* creates and documents the packaged integration. The tasks for the *Software Developer* are the following: #. Describe the wanted directory objects in YAML, suitable for the *UDM data loader*. #. Document the template variables defined in the YAML data. The *Operator* adjusts the :file:`custom_values.yaml` deployment value file. As part of it, they also assign values to the template variables that relate to the deployed Nubus for Kubernetes environment. The bottom three rows illustrate what happens at the application level. By installing the packaged integration through the :command:`helm upgrade` command, the operator triggers an update of the Nubus for Kubernetes installation, which also triggers a re-run of the *UDM data loader*. The *UDM data loader* reads the UDM data from the YAML files, and acts on it by calling the *UDM HTTP REST API* to create or modify the appropriate directory objects. .. _nubus-packaged-integrations-udm-data-loader-fig: .. figure:: /images/Nubus_packaged_integrations_UDM_data_loader.* :target: ../_images/Nubus_packaged_integrations_UDM_data_loader.svg :alt: Procedure for UDM data loader Procedure for *UDM data loader* and involved roles .. _nubus-packaged-integrations-udm-data-loader-add: Provide data to the loader -------------------------- To provide data for the *UDM data loader* plugin to Nubus for Kubernetes, use the following steps: #. Create the YAML files. See :ref:`nubus-packaged-integrations-udm-data-loader-format` and its following sections. #. Save the YAML files in the directory :file:`udm-data-loader` in your packaged integration project. #. Follow the instructions for building the container image in :ref:`nubus-packaged-integrations-bundle`. #. Copy the plugin file to your packaged integration project, for example :file:`packaged-integration/udm/udm-data-loader/`. For the *UDM data loader*, the ``plugin-type`` is ``udm-data-loader``. See :ref:`nubus-packaged-integrations-bundle-plugins`. .. code-block:: console $ export PLUGIN_TYPE="udm-data-loader" $ export PLUGIN_FILE="udm/udm-data-loader" .. _nubus-packaged-integrations-udm-data-loader-format: Data loader format ------------------ The *UDM data loader* reads the wanted state of directory objects from YAML files and prepares that state by creating or modifying UDM objects. Your packaged integration can contain several YAML files. The data loader reads them in lexical order. For example, you can define groups in one YAML file and then define users in subsequent files to add them to the already defined groups. :numref:`nubus-packaged-integrations-udm-data-loader-format-files-listing` shows an example for the order. .. code-block:: console :caption: Example for YAML files in data loader :name: nubus-packaged-integrations-udm-data-loader-format-files-listing . ├── 01-user-groups.yaml └── 05-users-to-groups.yaml Each YAML file consists of several YAML directives separated by ``---``. :numref:`nubus-packaged-integrations-udm-data-loader-format-yaml-listing` shows an example of a YAML directive for the *UDM data loader*. It creates a mail domain object at a defined position using the ``ldapBaseDn`` in the directory service with the name of the configured ``externalMailDomain``. .. code-block:: yaml :caption: Example of a YAML directive for the *UDM data loader* :name: nubus-packaged-integrations-udm-data-loader-format-yaml-listing --- action: "create" module: "mail/domain" position: "cn=domain,cn=mail,{{ ldapBaseDn }}" properties: name: "{{ externalMailDomain }}" --- ``ldapBaseDn`` and ``externalMailDomain`` in the example in :numref:`nubus-packaged-integrations-udm-data-loader-format-yaml-listing` are template variables, that are placeholders which Helm substitutes with the actual values provided by the operator at deployment time. For information about how to define and use template variables, see :ref:`nubus-packaged-integrations-udm-data-loader-template-variables`. Each YAML directive contains the following mapping: ``action`` Type: ``string``. See :ref:`nubus-packaged-integrations-udm-data-loader-actions`. ``module`` Type: ``string``. See :ref:`nubus-packaged-integrations-udm-data-loader-modules`. ``position`` Type: ``string``. Defines the position in the directory service tree. For the LDAP base DN you can use the template variable ``ldapBaseDn``. ``properties`` Type: mapping of ``strings``. The properties mapping defines the properties of the UDM object and depends on the UDM module used. .. _nubus-packaged-integrations-udm-data-loader-actions: Actions in the data loader -------------------------- The data loader supports the following actions in the YAML directives: ``create`` The action creates the UDM object with the properties provided. For an example, see :numref:`nubus-packaged-integrations-udm-data-loader-format-yaml-listing`. ``ensure_list_contains`` The action ensures that a list in the given UDM object contains the given list entry. For an example, see :numref:`nubus-packaged-integrations-udm-data-loader-actions-ensure-list-contains-listing`. .. code-block:: yaml :caption: Example for ``ensure_list_contains`` action :name: nubus-packaged-integrations-udm-data-loader-actions-ensure-list-contains-listing --- action: ensure_list_contains module: groups/group position: cn=Domain Admins,cn=groups,{{ ldapBaseDn }} policies: policies/umc: - "cn=default-umc-all,cn=UMC,cn=policies,{{ ldapBaseDn }}" --- ``ensure_list_does_not_contain`` The action allows to enforce that a specific object isn't in a list. For example, enforce that a user object isn't in a specific group. .. versionadded: 1.7.0 Added in Nubus for Kubernetes 1.7.0. ``modify`` The action modifies the given properties of the particular UDM object. For an example, see :numref:`nubus-packaged-integrations-udm-data-loader-actions-modify-listing`. .. code-block:: yaml :caption: Example for ``modify`` action :name: nubus-packaged-integrations-udm-data-loader-actions-modify-listing --- action: "modify" module: "portals/entry" position: "cn=login-saml,cn=entry,cn=portals,cn=univention,{{ ldapBaseDn }}" properties: activated: true --- ``modify_if_exists`` The action only modifies a UDM object if it exists. In contrast to the ``modify`` action, it doesn't raise an error if the object doesn't exist. .. versionadded: 1.7.0 Added in Nubus for Kubernetes 1.7.0. ``create_or_modify`` The action creates the object, if it doesn't exist. In case it exists, UDM modifies the object accordingly. ``delete_if_exists`` The action deletes the object if it exists, otherwise it does nothing. .. _nubus-packaged-integrations-udm-data-loader-template-variables: Template variables in the data loader ------------------------------------- Template variables provide the capability to adjust the data for loading into Nubus for Kubernetes depending on their installation environment. The example in :numref:`nubus-packaged-integrations-udm-data-loader-format-yaml-listing` uses the template variables ``ldapBaseDn`` and ``externalMailDomain``. You can define template variables in your YAML directives in the *position* and *properties* mappings using the double curly braces notation ``{{ variable }}``. It's the Jinja2 notation. You can also use other syntax from Jinja2, for example loops. Variable declaration To declare template variables, you as a **software developer** who creates the packaged integration, pick unique variable names and add it to your YAML directives. For example in :numref:`nubus-packaged-integrations-udm-data-loader-format-yaml-listing`. Recommendation: Use a prefix as a namespace to distinguish your variables from template variables in other packaged integrations. Variable value assignment To assign values to the variables, you as an **operator** who updates the Nubus deployment with the packaged integration, assign values to the template variables in your deployment values file, such as :file:`custom_values.yaml`. :numref:`nubus-packaged-integrations-udm-data-loader-template-variables-deployment-listing` shows an example for assigning values to template variables. .. code-block:: yaml :caption: Assign values to template variables in deployment values file :name: nubus-packaged-integrations-udm-data-loader-template-variables-deployment-listing nubusStackDataUms: templateContext: externalMailDomain: "" variable2: "