1. Administration#
This section describes the administration tasks for the ID Connector. It covers the topics installation, configuration of the sending system and the receiving systems and lifecycle tasks for the connector.
The ID Connector replication system consists of the following components as shown in Fig. 1.1:
An LDAP server containing user data.
A process on the data source UCS server, receiving user creation/modification/deletion events from the LDAP server and relaying them to multiple recipients through HTTP, called the ID Connector Service.
A process on the data source UCS server to monitor and configure the UCS@school ID Connector service, called the ID Connector HTTP API.
Multiple recipients of the directory data relayed by the ID Connector Service. They run an HTTP-API service, that the ID Connector Service pushes updates to.
1.1. Definitions#
For the administration of and ID Connector setup or the integration with ID Connector, you need to make sure to know about the following aspects of a UCS environment.
- LDAP and LDAP listener#
UCS uses LDAP because of its optimization for reading in a hierarchical structure. Don’t accessed directly, use UDM instead. OpenLDAP can have plugins, such as the notifier UCS heavily uses. Upon changes in the LDAP directory, the notifier triggers listeners locally and on remote systems.
The listener service connects to all local or remote notifiers in the domain. When notified, the listener calls listener modules, which are scripts in shell and Python.
You need to understand the basic concepts of LDAP.
See also
For more information, see LDAP directory service in UCS Manual [1].
- Univention Directory Manager#
UCS uses Univention Directory Manager (UDM) for handling user data and other data stored in the LDAP server. The LDAP server is one of two core storage locations. The other storage location is UCR. Examples for data are users, roles, or machine info.
UDM adds a layer of functionality and logic on top of LDAP, hence don’t use LDAP directly, but only through UDM.
You need to:
understand the concept of UDM.
know the basic structure of UDM objects and their attributes.
add and manage extended attributes.
See also
For more information in a developer context, see Univention Directory Manager (UDM) in Univention Developer Reference [2].
For an architecture overview, see Univention Directory Manager (UDM) in Univention Corporate Server 5.0 Architecture [3].
- Univention Configuration Registry#
Univention Configuration Registry (UCR) stores configuration variables and settings to run the system, creates and changes actual configuration text files as configured by these variables upon setting said variables.
You need to:
understand basic UCR concepts.
know how to set and read UCR variables.
See also
For more information, see Administration of local system configuration with Univention Configuration Registry in UCS Manual [1].
For an architecture overview, see Univention Configuration Registry (UCR) in Univention Corporate Server 5.0 Architecture [3].
- Univention App Center settings#
Univention App Center is an ecosystem similar to app stores known from mobile platforms such as Apple or Google. It provides an infrastructure to deploy and run enterprise applications on Univention Corporate Server (UCS). The Univention App Center uses well-known technologies like Docker.
Within the App Center, you can configure settings for the individual apps.
See also
For more information, see the following resources:
App settings in Univention App Center for App Providers [4]
Setting of an application in the App Center in UCS Manual [1]
- UCS@school basics#
Schools have special requirements for managing entities about what is going on inside them, such as teachers, students, staff, computer rooms, exams, and more. For managing the relation between multiple schools, their operator organizations (“Schulbetreiber”), and possibly ministerial departments above them.
There are several components used within UCS@school, and Kelvin is one of them.
You need to:
know about UCS@school objects.
know the difference between UCS@school objects and UDM objects.
See also
For more information, see the following resources:
- UCS@school Kelvin REST API#
The UCS@school Kelvin REST API (Kelvin) provides HTTP endpoints to create and manage individual UCS@school domain objects such as school users, school classes and schools (OUs). It uses FastAPI, hence in Python 3.
You need to be able to install and configure Kelvin.
See also
For more information, see the following resources:
UCS@school-Objekte im LDAP-Verzeichnisdienst in UCS@school - Handbuch für Administratoren [6]
If you want to also develop for the ID Connector, please also see the following section Development.
1.2. Installation#
This section describes the installation of the UCS@school ID Connector app. For a working setup, you need a sending system with the ID Connector app, and receiving systems with the UCS@school Kelvin REST API.
1.2.1. Sending system#
The UCS@school ID Connector app is available in the Univention App Center. You can install it with the following command:
$ univention-app install ucsschool-id-connector
The installation process runs the join script 50ucsschool-id-connector.inst
and creates the following:
a key for signing the JWT tokens in the file
/var/lib/univention-appcenter/apps/ucsschool-id-connector/conf/tokens.secret
.the group
ucsschool-id-connector-admins
with the distinguished name (DN)cn=ucsschool-id-connector-admins,cn=groups,$ldap_base
, whose members can access the ID Connector HTTP-API.
The section Authentication explains both files in detail.
If the installation process didn’t create the files listed before, you can re-run the join script with the following command.
$ univention-run-join-scripts --run-scripts --force 50ucsschool-id-connector.inst
Tip
You can validate the existence of the group with:
$ udm groups/group list --filter cn=ucsschool-id-connector-admins
Tip
The installation process registers join scripts in the LDAP and runs them on any UCS system before, during, or after the join process.
For more information, see KB 13034 - A script shall be executed on each or a certain UCS systems before/during/after the join process
The app is using the CA trust store from the UCS host system. The app is restarting if the certificates are updated by update-ca-certificates
, you can disable the automatic restart in the Univention App Center settings.
1.2.2. Receiving system#
The UCS@school ID Connector app needs an HTTP-API on the target system to create, modify, and delete users. The ID Connector supports UCS@school Kelvin REST API.
The UCS@school ID Connector app synchronizes users for different domains between a sender and receiving systems. UCS takes care of the user synchronization within the domain.
To install the Kelvin API on each receiving system, run the following command:
$ univention-app install ucsschool-kelvin-rest-api
To allow the UCS@school ID Connector app on the sender system to access the Kelvin-API on the receiving system,
it needs an authorized user account.
By default, the Administrator
account on the receiving system is the only authorized user.
To add a dedicated Kelvin API user for the UCS@school ID Connector,
consult UCS@school Kelvin REST API documentation [5].
If the receiving system is using a private CA, you need to add that CA to the UCS system you installed the UCS@school ID Connector app on.
1.3. Configure receiving system - HTTP-API (Kelvin)#
You need to install and configure the Kelvin API as described in UCS@school Kelvin REST API documentation [5]. The following sections assume that you have installed the current version of Kelvin.
Note
For the authorization of the UCS@school ID Connector at the target system
it needs credentials with special privileges.
Create a user account with the name and password of your choice
and add them to the group ucsschool-kelvin-rest-api-admins
.
$ udm users/user create --position "cn=users,$(ucr get ldap/base)" \
--set username=USERNAME-OF-YOUR-CHOICE --set lastname=Kelvin --set firstname=UCS \
--set password="PASSWORD-OF-YOUR-CHOICE"
$ udm groups/group modify --dn "cn=ucsschool-kelvin-rest-api-admins,cn=groups,$(ucr get ldap/base)" \
--append users="uid=USERNAME-OF-YOUR-CHOICE,cn=users,$(ucr get ldap/base)"
Write down the credentials. You need them for the school authority configuration on the sending system.
Warning
You used the password before as input to a command. It’s now in the command history. It’s recommended to delete the command with the password from the command history. Run the following command:
$ history -d -2
After installation and basic configuration you can configure mapped UDM properties.
Beyond the standard object properties in UCS@school, you can define additional UDM properties and make them available in the Kelvin API on the target system.
To define additional UDM properties, you first create a mapping in the configuration file
/etc/ucsschool/kelvin/mapped_udm_properties.json
. The following example makes the listed properties additionally available for the resourcesuser
andschool
.{ "user": ["title", "phone", "e-mail"], "school": ["description"] }
Restart Kelvin with the following command, for the configuration changes to take effect:
$ univention-app restart ucsschool-kelvin-rest-api
Caution
When configuring Kelvin in detail, the password hashes for LDAP and Kerberos authentication are collectively transmitted in one JSON object to one target attribute. This means it’s all or nothing: all hashes are synced, even if empty. You can’t select individual hashes.
Caution
Ensure that you configure all the mapped properties that the sending system sends,
for example displayName
.
If the sender sends more than the receiver is configured to process,
you experience unexpected errors, for example 404
in the log file.
1.4. Configure sending system#
The school authorities configuration must be done through the ID Connector HTTP API. Don’t edit configuration files directly.
1.4.1. UCS@school ID Connector HTTP API#
The HTTP-API of the ID Connector app offers the following resources:
https://FQDN/ucsschool-id-connector/api/v1/queues/
/queues/
for the monitoring of queueshttps://FQDN/ucsschool-id-connector/api/v1/school_authorities/
/school_authorities/
for the configuration of school authoritieshttps://FQDN/ucsschool-id-connector/api/v1/school_to_authority_mapping/
/school_to_authority_mapping/
for the configuration of which school you want to synchronize to which authority
You can discover the API interactively using one of two web interfaces. You can visit them with a browser at their respective URLs:
Swagger UI:
https://FQDN/ucsschool-id-connector/api/v1/docs
ReDoc:
https://FQDN/ucsschool-id-connector/api/v1/redoc
The Swagger UI page is especially helpful as it allows sending queries directly from the browser.
The equivalent curl
command lines are then displayed.
You can download an OpenAPI v3 (formerly “Swagger”) schema
from https://FQDN/ucsschool-id-connector/api/v1/openapi.json
.
1.4.2. Authentication#
Only users being member of the group ucsschool-id-connector-admins
are allowed to access the HTTP-API.
The user Administrator
is automatically added to this group for testing purposes.
In production, you should only use the regular administration user accounts.
You can authorize yourself, for example in the Swagger UI using the Authorize button.
To use the ID Connector HTTP API from a script,
the script must retrieve a
JSON Web Token (JWT)
from https://FQDN/ucsschool-id-connector/api/token
.
The token is valid for a configurable amount of time,
the default value for the time to life (TTL) is 60 minutes.
To change the TTL of the token,
open the corresponding app settings in the Univention App Center.
$ curl --include \
--insecure \
--request POST
--data 'username=Administrator&password=s3cr3t' \
https://FQDN/ucsschool-id-connector/api/token
1.4.3. Monitor processing status#
This section describes how to monitor the processing status of the ID Connector queues. The status gives hints about how well the connector performs and if it works as intended.
When users and groups change in UCS, the ID Connector processes these changes as transactions and synchronizes them to the connected UCS@school domains. The connector puts the changes into queues to increase the robustness of the connector and to keep the load at a manageable level.
The ID Connector has an inbound queue that contains data coming from the App Center listener converter. The data uses a JSON representation of the changed objects. The ID Connector transforms the data from the inbound queue into transaction requests to the receiving systems. It buffers each transaction in an outbound queue. The ID Connector has one outbound queue per connected school authority (UCS@school domain) for outbound data.
Each queue is a directory and each transaction is a file in a queue directory. The queues locate at the following directories:
- Inbound queue
/var/lib/univention-appcenter/apps/ucsschool-id-connector/data/listener
- Outbound queues
/var/lib/univention-appcenter/apps/ucsschool-id-connector/data/out_queues/queue_name
The ID Connector provides the resources /queues/
and /queues/name/
that list the size of each queue.
With the resource /queues/name/
you can query a distinct queue and
monitor it.
To retrieve the size of a queue, use the following steps:
Authenticate yourself with the API as described in Authentication.
Request the data from the ID Connector API.
Example#
The following example shows how to query the API for all queue lengths. You can choose between a user interface approach with Swagger UI, or a command-line approach.
Use the following steps with the Swagger UI:
To authorize, click Authorize and enter the credentials of a legitimate user.
In the queues section, open the GET
/ucsschool-id-connector/api/v1/queues
resource and click Try it out.Click the button Execute. In the Server response section, in the Response body area, you see a result similar to Listing 1.2.
Use the following commands for the command line:
Authorize yourself with the API and receive an
access_token
:$ FQDN='<YOUR_FULLY_QUALIFIED_HOST_NAME>' $ curl \ --include \ --insecure \ --request POST \ --data 'username=Administrator&password=s3cr3t' \ https://"$FQDN"/ucsschool-id-connector/api/token $ TOKEN='<YOUR_TOKEN>'
Request a list of queues and use the
access_token
you retrieved before:$ curl \ --insecure \ --request 'GET' \ 'https://"$FQDN"/ucsschool-id-connector/api/v1/queues' \ -H 'accept: application/json' \ -H "Authorization: Bearer ${TOKEN}"
You see a result similar to Listing 1.2.
Hint
If you want to use a secure connection, you need download the UCS root certificate and pass it to curl:
$ F_PATH_CA_CERT="PATH_TO_UCS_ROOT_CERTIFICATE" $ curl \ --cacert "$F_PATH_CA_CERT" \ --request 'GET' \ 'https://"$FQDN"/ucsschool-id-connector/api/v1/queues' \ -H 'accept: application/json' \ -H "Authorization: Bearer ${TOKEN}"
[
{
"name": "InQueue",
"head": "",
"length": 0,
"school_authority": ""
},
{
"name": "auth1",
"head": "2024-01-11-13-43-36-196082_ready.json",
"length": 2,
"school_authority": "auth1"
},
{
"name": "auth2",
"head": "",
"length": 0,
"school_authority": "auth2"
}
]
Alerts for monitoring#
If you want to add the UCS@school ID Connector to your monitoring environment and let the monitoring send you alerts, you may monitor the following problematic states.
- Monotonous growth over a period of time
If an ID Connector queue on the sending system grows continuously over a period of time, such as a day, the ID Connector isn’t able to process transactions at the required speed as transactions arrive. Under normal circumstances, it may happen that the ID Connector can’t process the transactions fast enough. If the queue sizes don’t decrease at all for days, this could be a problem.
- No change in queue size over a period of time
If a queue size greater than
0
remains the same over a period of time, such as an hour, it indicates that the ID Connector isn’t working or is stopping on corrupt transactions. If nothing changes in a queue and the size remains the same, you need to investigate. For more information, see Interrupted processing.- Queues don’t reach a size of
0
for a period of time If the queues don’t run empty over a period of time, such as a week, this can mean that transactions are coming in at the same rate as the connector is processing them. Or, the ID Connector is running too slowly overall. Or, the target system may be unreachable due to network problems or incorrect configuration.
- The amount of files located in the trash directory is rising continuously
If the queues can’t handle transactions because of internal errors, this can mean that the Kelvin API on the target system may be unreachable or the Kelvin API has an incorrect configuration. For more information, see Interrupted processing.
Hint
The right period of time to trigger an alarm depends on your specific environment.
Interrupted processing#
If the queue processing doesn’t go as planned, for example, because the service is unavailable, the receiving system is unreachable, the ID Connector app has crashed, transactions are corrupt, or for any other reason, the queues grow in size or remain at a certain level. If the ID Connector app service doesn’t run, the queues can quickly grow to a considerable size, such as more than 1 million files after some days.
If a transaction has a valid JSON format,
but the receiver can’t process it,
the ID Connector moves the JSON file with the transaction
from the queue to the trash
directory for the outgoing queue of the respective school authority located below
/var/lib/univention-appcenter/apps/ucsschool-id-connector/data/out_queues/SCHOOL_AUTHORITY
.
The value for SCHOOL_AUTHORITY
reflects your respective school authority name.
The files located in the trash folder contain information that you can use to fix the issues.
You can use the DNs of the objects to find and fix the UDM objects:
$ jq -r ".dn" /var/lib/univention-appcenter/apps/ucsschool-id-connector/data/out_queues/SCHOOL_AUTHORITY/trash/*.json | sort | uniq
You can use the names of a
TRANSACTION_FILE
located in thetrash
directory to find out which error the ID Connector raised and logged in the log file:$ grep -C 3 TRANSACTION_FILE /var/log/univention/ucsschool-id-connector/queues.log
You can re-schedule the objects after you fixed the issue.
Reschedule a users:
$ univention-app shell ucsschool-id-connector schedule_user USERNAME
Reschedule a group:
$ univention-app shell ucsschool-id-connector schedule_group GROUPNAME
Reschedule a school and all associated objects:
$ univention-app shell ucsschool-id-connector schedule_school SCHOOL
The ID Connector moves transactions with invalid or not accepted JSON formats
to the trash
directory for the outgoing queue of the respective school authority located below
/var/lib/univention-appcenter/apps/ucsschool-id-connector/data/out_queues/SCHOOL_AUTHORITY/
.
If a transaction in JSON format located in any queue is corrupt, it may stay in the queue forever. To resolve an interrupted queue, use the following steps:
Find out, which transaction is corrupt.
Have a look at the queue log file of the ID Connector on the sending system at
/var/log/univention/ucsschool-id-connector/queues.log
.Resolve the error in the ID Connector setup.
If other errors caused the corrupt transaction, you need to look at other parts. The ID Connector either logs a qualitative error happening on the sending system, or a generic error if a receiving system has an error with its Kelvin API. Review the configurations on the sending system and the receiving systems:
On the sending system, validate the configurations of school authority objects in the UCS@school ID Connector. Have a close look at the URL, credentials, and attribute mappings.
On the receiving systems, validate the attribute mapping and the incoming transactions through the Kelvin API. Have a look at the Kelvin log files.
For information about watching the transaction processing, see Trying it out.
Remove the corrupt transaction from the queue.
Use the transaction you found in the log file, find the JSON file that contains the transaction, and remove the JSON file.
1.4.6. Role specific attribute mapping#
Warning
This section describes an advanced scenario. If you don’t explicitly need it, jump to the next section Trying it out.
Back to the example about telephone numbers. Imagine you don’t want to transfer telephone numbers for students, you actually need them for teachers. This means, that you need to define per role which properties you want to transfer.
New in version 2.1.0: The default Kelvin plugin version 2.1.0
received the role specific
attribute mapping feature.
This allows to define additional user mappings for each
role such as student
, teacher
, staff
and school_admin
by adding a
mapping next to the users
mapping suffixed by _$ROLE
,
for example users_student: {}
.
If the Kelvin plugin handles a user object, the mapping looks like the following:
In the current school authority configuration, determine the schools that the authority handles.
Determine all roles the user has in these schools.
Order the roles by priority:
school_admin
being the higheststaff
teacher
and then
student
with the lowest priority.
Find a
users_$ROLE
mapping from the ones configured in the plugin settings, pick the one with the highest priority.If you couldn’t fine any, fall back to the
users
mapping as the default.
You find an example for such a configuration in Role specific Kelvin plugin mapping.
Note
The priority order for the roles aligns with the order of common specificity in UCS@school.
A student only ever has the role student
.
Teachers, staff, and school administrators can have multiple roles.
Note
The mappings for the different roles aren’t additive, because that approach would complicate the option to remove mappings from a specific role. Therefore, choose only one mapping by the rules just described.
Warning
Users have the field school_classes
, which describes which school classes they belong to.
You can prevent certain user roles from adding to or removing from school classes.
Be aware that leaving out the school_classes
from the mapping isn’t sufficient to achieve this.
Changing the school classes of a user doesn’t only result in a user change event,
but also a school class change event, which you need to handle separately.
You therefore need to use a derivative of the Kelvin plugin, as described in Partial group sync mapping.
1.4.7. Partial group sync mapping#
Warning
This section describes an advanced scenario. If you don’t explicitly need it, jump to the next section Trying it out.
Remember that in the last examples you had a property that you would send for some users, but not others, depending on their role? Turns out that you can have the same problem for groups.
Imagine that a school manages locally which teachers belong to which class.
In the role specific mapping you would not synchronize the classes attribute school_classes
,
to prevent overwriting the local managed settings, see above.
This isn’t enough though. You would also need to make sure
that you don’t synchronize the property users
of groups
which contains those teachers.
New in version 2.1.0: Kelvin plugin version 2.1.0
adds the derivative kelvin-partial-group-sync
.
This plugin alters the handling of school class changes
by allowing you to specify a list of roles that you want to ignore when
synchronizing groups.
The following steps determine which members the connector sends to a school authority, when an administrator adds a school class:
Add all users that are members of the school class locally. This is the default normal Kelvin plugin behavior.
Remove all users that have a configured role to ignore in any school handled by the school authority configuration.
Get all members of the school class on the target system that have one of the configured roles and add them.
Get all members of the school class on the target system that are unknown to the ID Connector and add them.
This results in school classes that have only members with roles not configured to ignore:
members with roles to ignore that were added on the target system,
any users added on the target system which are unknown to the ID Connector.
Warning
To achieve this behavior, several additional LDAP queries on the ID Connector and one additional request to the target system are necessary. This affects performance.
To activate this alternative behavior,
replace the kelvin
plugin in a school authority configuration with the kelvin-partial-group-sync
plugin.
The configuration options are exactly the same as for the kelvin
plugin,
except for the addition of school_classes_ignore_roles
,
which holds the list of user roles to ignore for school class changes.
For an example configuration, see Partial group sync.
Warning
Be aware that this plugin can only alter the handling of dedicated school class change events. Due to the technical situation, changing the members of a school class often results in two events:
a school class change
a user change
To actually prevent the ID Connector to add users of certain roles to school classes at all,
it’s necessary to leave out the mapping of the users school_class
field
in the configuration as well - see the previous section.
1.5. Trying it out#
Time has come to verify the setup. In this section, you go through the following steps:
Create a test user.
Import the test user on the sending side.
Watch the synchronization of the test user to the receiving side.
Caution
The ID Connector only synchronizes users with the properties ucsschoolRecordUID
and ucsschoolSourceUID
.
You need to ensure that the user accounts used for testing have these properties.
The slow way is to create a user individually and to ensure to amend the required properties, or to use the UCS@school import. For more information about user import in UCS@school, see UCS@school - Handbuch zur CLI-Import Schnittstelle [7], German only.
With the fast way, you create and import the user in one step.
The following command creates a user within a class in the school DEMOSCHOOL
.
$ /usr/share/ucs-school-import/scripts/ucs-school-testuser-import \
--students 1 --classes 1 DEMOSCHOOL
To see the synchronization action on the sender system, you can watch the log file with the following command:
$ tail -f /var/log/univention/ucsschool-id-connector/queues.log
To see the synchronization action on the receiving system, you can watch the log file of Kelvin with the following command:
$ tail -f /var/log/univention/ucsschool-kelvin-rest-api/http.log # kelvin log
You may need to wait a short moment before the queue picks up the created user.
If everything went fine,
you see some messages in the Kelvin log file on the receiving system.
You can confirm that the ID Connector created the user
in either the Kelvin web interface at https://FQDN/ucsschool/kelvin/v1/docs
,
or in the UMC.
The following log files are also a good starting point for debugging in case something went wrong:
On the sending system:
/var/log/univention/ucsschool-id-connector/queues.log
On the receiving system:
/var/log/univention/ucsschool-kelvin-rest-api/http.log
Important
During debugging, you must always ensure that the following is correct and matches:
School authority configuration on the sender system, including authentication credentials.
School to authority mapping on the sender system.
mapped_udm_properties.json
on the receiving system has all extra attributes that are defined in the school authority mapping.
1.6. Starting / Stopping services#
Both services, ID Connector Service and ID Connector HTTP API, run in a Docker container. You can start or stop the container by using the regular service facility of the UCS system:
$ univention-app start ucsschool-id-connector
$ univention-app status ucsschool-id-connector
$ univention-app stop ucsschool-id-connector
$ univention-app restart ucsschool-id-connector
1.7. Updates#
To install updates for ID Connector, use one of the two usual UCS ways. Either through UMC or on the command line:
$ univention-app upgrade ucsschool-id-connector