Univention Corporate Server - Performance guide#

By default UCS is suitable for environments with up to 5,000 users. This document describes configuration modifications which can increase performance in larger environments.

OpenLDAP and listener/notifier domain replication#

As a core element in the operation and administration of a UCS domain, the performance of the LDAP server plays a central role in the overall performance.

Indexes#

Comparable with other database systems, OpenLDAP uses indexes about commonly requested attributes. For indexed attributes a search isn’t performed through the full database contents, but over an optimized subsection.

With recent UCS versions, the indexes are occasionally expanded and automatically activated. You can deactivate the automatic activation with the UCR variable ldap/index/autorebuild. In this case, you must set the indexes manually to ensure that there is no loss of performance as a result.

Optimizing an LDAP index requires some thoughts before hand. It isn’t recommended to add all wanted attributes to the LDAP index. The index requires maintenance and updating an index costs performance on write and sometimes also on read operations. Evaluate your frequent searches and add the frequent used search attributes to the LDAP index.

Manage indexes with the index setup script#

Instead of editing the UCR variables manually to configure indexes, UCS provides the /usr/share/univention-ldap/ldap_setup_index script that runs all the steps from the manual procedure. Use this script because it updates the UCR variable, validates the resulting configuration, and rebuilds the index database so that the UCR configuration and the on-disk index remain aligned.

The script does the following:

It reads the current index configuration from the following UCR variables:
It adds or removes the attributes that you pass to the script.
It writes the changed UCR variables and lets UCR rewrite the OpenLDAP server configuration file /etc/ldap/slapd.conf.
It validates the new configuration with slaptest.
It runs slapindex to re-index the existing entries in the LDAP database.

Use ldap_setup_index#

To add an attribute, use one of the options --add-eq, --add-pres, --add-sub, or --add-approx with the attribute name.

To remove an attribute, use the corresponding --rm-eq, --rm-pres, --rm-sub, or --rm-approx option. You can pass an option more than once and combine several index types in a single call.

Example

The example in Listing 1 adds the memberOf attribute to the equality index.

Important

Stop the OpenLDAP server with the systemctl stop slapd.service command before you run the script. slapindex doesn’t run while the OpenLDAP server is running. Otherwise, the script aborts before it rebuilds the index. Start the server again afterwards.

The example in Listing 1 shows this order.

Listing 1 Example for adding memberOf to the LDAP index#

$ systemctl stop slapd.service
$ /usr/share/univention-ldap/ldap_setup_index --add-eq memberOf
$ systemctl start slapd.service

The script rebuilds the index only when ldap/index/autorebuild has the value yes. yes is the default. The variable controls the rebuild behavior as follows:

Value is yes

If ldap/index/autorebuild has the value yes, the script writes the UCR variables and runs slapindex to rebuild the index.

Value is no

If ldap/index/autorebuild has the value no and the UCR variable already contains that value, the script doesn’t change anything.

To update only the UCR variables without rebuilding the index, use the option --only-update-ucr. In this case, run slapindex later, for example during a maintenance window, so that slapindex indexes the existing LDAP entries.

Note

If you manually change any of the UCR variables ldap/index/eq, ldap/index/pres, ldap/index/sub, or ldap/index/approx instead of using the ldap_setup_index script, UCR rewrites the OpenLDAP server configuration, but it doesn’t rebuild the index database. In this case, existing entries stay unindexed until you run slapindex. The script triggers the index rebuild for you.

Option reference for ldap_setup_index#

ldap_setup_index offers the following options:

--add-eq#: Add an LDAP attribute to the equality index.

--add-pres#: Add an LDAP attribute to the presence index.

--add-sub#: Add an LDAP attribute to the sub-string index.

--add-approx#: Add an LDAP attribute to the approximate-match index.

--rm-eq#: Remove an LDAP attribute from the equality index.

--rm-pres#: Remove an LDAP attribute from the presence index.

--rm-sub#: Remove an LDAP attribute from the sub-string index.

--rm-approx#: Remove an LDAP attribute from the approximate-match index.

--add-defaults#: Adds the attributes that Univention recommends for indexing to the existing configuration.

--only-update-ucr#: Only updates the UCR configuration for the indexes, but doesn’t rebuild the index. You need to manually rebuild the index with the slapindex command.

--force-defaults#: Resets the index configuration to the recommended attributes and discards any custom additions.

--remove-unknown#: Removes attributes that slaptest reports as undefined, for example because of a typo or because the attribute no longer exists.

--verbose#: Increases the output verbosity. You can repeat the option for more output.

Configuration of the database backend#

The memory mapped database (MDB) has been used for new installations since UCS 4.0. The following flags have impact on the performance of the MDB database and can be set with the Univention Configuration Registry Variable ldap/database/mdb/envflags (multiple values are separated by spaces):

nosync: Specify that on-disk database contents should not be immediately synchronized with in memory changes. Enabling this option may improve performance at the expense of data security. In particular, if the operating system crashes before changes are flushed, some number of transactions may be lost. By default, a full data flush/sync is performed when each transaction is committed.
nometasync: Flush the data on a commit, but skip the sync of the meta page. This mode is slightly faster than doing a full sync, but can potentially lose the last committed transaction if the operating system crashes. If both nometasync and nosync are set, the nosync flag takes precedence.
writemap: Use a writable memory map instead of just read-only. This speeds up write operations but makes the database vulnerable to corruption in case any bugs in slapd cause stray writes into the memory mapped region.
mapasync: When using a writable memory map and performing flushes on each commit, use an asynchronous flush instead of a synchronous flush (the default). This option has no effect if writemap has not been set. It also has no effect if nosync is set.
nordahead: Turn off file read-ahead. Usually the OS performs read-ahead on every read request. This usually boosts read performance but can be harmful to random access read performance if the system’s memory is full and the DB is larger than RAM.

ldap/database/mdb/envflags#

Specifies additional flags for the mdb database backend. Separate multiple flags with spaces.

Default value:: not set
Possible values:: nosync, nometasync, writemap, mapasync, nordahead
Type:: list

OpenLDAP ACLs#

Access to the information contained in the LDAP directory is controlled by access control lists (ACLs) on the server side. General information on the configuration of ACLs in UCS can be found in LDAP access control configuration in Nubus for UCS 5.2 - Operation Manual [1].

Nested groups are also supported. The Univention Configuration Registry Variable ldap/acl/nestedgroups can be used to deactivate the nested groups function for LDAP ACLs, which will result in a speed increase for directory requests.

Univention Directory Listener#

The Univention Directory Listener can perform safety checks to prevent a user name being added into a group twice. These checks add some overhead to replication and can be deactivated by setting the Univention Configuration Registry variables listener/memberuid/skip and listener/uniquemember/skip to no. Starting with UCS 3.1 the variables are not set and the checks are not activated any longer by default.

listener/memberuid/skip#

Controls whether the Univention Directory Listener performs additional consistency checks to prevent a user from being added to a group multiple times. To deactivate these checks, set this variable and listener/uniquemember/skip to no.

Default value:: not set
Possible values:: yes, no, not set
Type:: boolean

listener/uniquemember/skip#

Default value:: not set
Possible values:: yes, no, not set
Type:: boolean

Performance during initial provisioning of users and groups#

There are several ways in which you can provision users and groups into UCS. Each method has its own performance implications and use cases. Especially for large environments it’s important that you choose an efficient method.

The following recommendations can improve performance when creating large numbers of users and adding them to groups:

Use the UDM Python library in case you can do the provisioning locally on the UCS system.

Use the UDM HTTP REST API to provision users and groups to a remote UCS system.

It isn’t recommended to use the UDM command line interface, because it’s significantly slower than the previously mentioned options.
Create the users first and then the groups. This prevents unnecessary LDAP operations, because after the creation of the users, LDAP only needs to update the groups one time.
For the duration of the provisioning, deactivate the automatic update of the primary group, typically Domain Users, when you create or remove a user. Set the Univention Configuration Registry variable directory/manager/user/primarygroup/update to false.

directory/manager/user/primarygroup/update#

Controls whether Nubus updates a user’s primary group when adding or removing a user. When set to false, Nubus doesn’t update the primary group.

Default value:: not set
Possible values:: true, false, not set
Type:: boolean

You can use the following example as a guide for using the UDM Python library:

#!/usr/bin/python3

from univention.admin import modules, uldap
from univention.config_registry import ucr


lo, position = uldap.getAdminConnection()
base = ucr['ldap/base']

modules.update()

users = modules.get('users/user')
modules.init(lo, position, users)

groups = modules.get('groups/group')
modules.init(lo, position, groups)

def create_user(name):
    position.setDn('cn=users,%s' % (base,))
    res = users.lookup(None, lo, "uid=%s" % name)
    if res:
        user = res[0]
    else:
        user = users.object(None, lo, position)
        user.open()
        user["lastname"] = name
        user["firstname"] = name
        user["password"] = "univention"
        user["username"] = name
        user.create()
    return user.dn

def create_or_modify_group(name, members=None):
    """
    Parameters:
        name (str): name of the group
        members (list[str]): list of user DNs
    """
    position.setDn('cn=groups,%s' % (base,))
    res = groups.lookup(None, lo, "cn=%s" % name)
    if res and members:
        group = res[0]
        group.open()
        group["users"].extend(members)
        group.modify()
    else:
        group = groups.object(None, lo, position)
        group.open()
        group["name"] = name
        if members:
            group["users"] = members
        group.create()

username_list = ["exampleuser1", "exampleuser2"]
userdn_list = []
for name in username_list:
    userdn = create_user(name)
    if userdn:
        userdn_list.append(userdn)

if userdn_list:
    create_or_modify_group("examplegroup1", userdn_list)

Performance issues during the join process#

The size of the UCS domain can have an impact on the duration of the join process. Here is some information how to deal with such problems.

Samba#

One of the join scripts for samba requires that the samba connector has synchronized all domain objects into samba. This script has a timeout of 3h (from UCS 4.4-7 on). This is sufficient for normal sized environments. But in large environments this script may hit the timeout and abort the join process. To increase the timeout the Univention Configuration Registry Variable create/spn/account/timeout can be set prior to the join process.

The join scripts 97univention-s4-connector and 98univention-samba4-dns wait for the replication of the DNS record of the joining system to verify that the local Samba backed DNS server can answer requests for Active Directory related requests. By default the scripts wait for 600 seconds, but in case there are a lot of objects that need to be replicated (e.g. DNS zones) then this default may be too short. In that case the timeout can be adjusted by setting the Univention Configuration Registry Variable join/samba/dns/replication/timeout to a bigger value before joining.

Samba traditionally uses TDB as backend database storage, that has an internal 32 bit address space limitation. UCS supports provisioning Samba using LMDB instead, which doesn’t have this strict limitation. For more information, see Lightning Memory-Mapped Database Manager (LMDB) documentation [2]. KB 18014 describes how to migrate a productive UCS domain.

To use LMDB instead of TDB, set the corresponding Univention Configuration Registry Variable samba/database/backend/store to mdb before you install the app Active Directory-compatible Domain Controller in your UCS domain.

Added in version 5.2-0: LMDB is the default for provisioning the Samba backend database.

The Univention Configuration Registry Variable samba/database/backend/store/size defines the current maximal size of the individual backend database store files and has the default value of 8GB. Since there is one backend storage file per Active Directory naming context, this amounts to a total of 40GiB. Take care that the storage can accommodate this amount of space.

If required, you can increase the value monotonically. After changing the Univention Configuration Registry Variable, you need to restart Samba, so that the value can take effect. You can check the number of used storage pages, 4KiB each, by running the command mdb_stat -nef on the individual files and calculating Number of pages used minus Free pages. The value at Max pages shows the current effective limit. The backend storage files locate in /var/lib/samba/private/sam.ldb.d/ and have the file extension .ldb.

LMDB uses fdatasync to persist transactions. As an optimization, the operating system should only write modified memory pages to the disk. On Amazon EC2, writing modified memory pages can result in a higher number of IOPS as compared to TDB, which can significantly slow down provisioning and operations such as bulk group membership changes. As a temporary speed-up option, Samba offers the ldb:nosync parameter. Enabling ldb:nosync provides performance benefits during a provisioning phase. However, don’t enable ldb:nosync permanently during day-to-day operations, as this compromises durability of changes committed to the SAM database. Instead, consider other means of improving I/O performance. To set this parameter, use the following commands:

$ echo -e "\n[global}\n\tldb:nosync = true" >> /etc/samba/local.conf
$ ucr commit /etc/samba/smb.conf
$ /etc/init.d/samba restart

The following is a short reference for the UCR variables mentioned before:

create/spn/account/timeout#

Sets how long in seconds the system waits for SPN account synchronization between the Nubus for UCS system and Samba during a domain join. In large environments, you may need to increase this value.

Default value:: 10800
Type:: integer

join/samba/dns/replication/timeout#

Sets how long in seconds a domain join waits for the initial replication of the DNS host record.

Default value:: 600
Type:: integer

samba/database/backend/store#

Specifies the database backend for Samba. Changing this value requires re-joining univention-samba4.

Default value:: mdb
Possible values:: mdb, tdb
Type:: string

samba/database/backend/store/size#

Sets the maximum database size for the Samba mdb backend. This variable only applies when samba/database/backend/store is set to mdb.

Default value:: 8Gb
Type:: string

Local group cache#

By default the group cache is regenerated every time changes are made to a group. This avoids cache effects whereby group memberships only become visible for a service after the next scheduled group cache rewrite (by default once a day and after 15 seconds of inactivity in the Univention Directory Listener). In larger environments with a lot of group changes, this function should be deactivated by setting the Univention Configuration Registry Variable nss/group/cachefile/invalidate_on_changes to false. This setting takes effect immediately and does not require a restart of the Univention Directory Listener.

When the group cache file is being generated, the script verifies whether the group members are still present in the LDAP directory. If only the Univention Management Console is used for the management of the LDAP directory, this additional check is not necessary and can be disabled by setting the Univention Configuration Registry Variable nss/group/cachefile/check_member to false.

UCS management system#

Disabling automatic search#

By default all objects are automatically searched for in the domain management modules of the Univention Management Console. This behavior can be disabled by setting the Univention Configuration Registry Variable directory/manager/web/modules/autosearch to 0.

directory/manager/web/modules/autosearch#

Controls whether the domain management modules in the Management UI run an automatic search for all objects when opened. You can override this behavior per module.

Default value:: not set
Possible values:: true, false, not set
Type:: boolean

Imposing a size limit for searches#

The Univention Configuration Registry Variable directory/manager/web/sizelimit is used to impose an upper limit for search results. If, e.g., this variable is set to 2000 (as is the default), searching for more than 2000 users would not be performed and instead the user is asked to refine the search.

directory/manager/web/sizelimit#

Sets the upper limit for search results in the Management UI. If a search returns more results, the system aborts it and asks you to refine the search. When unset, the limit is 2000.

Default value:: not set
Type:: integer

Adjusting the limit on open file descriptors#

The Univention Configuration Registry Variable umc/http/max-open-file-descriptors is used to impose an upper limit on open file descriptors of the univention-management-console-web-server. The default is 65535.

umc/http/max-open-file-descriptors#

Defines the maximum number of open file descriptors for the UMC web server.

Default value:: 65535
Type:: integer

Vertical performance scaling#

A single Univention Management Console instance does not use multiple CPU cores by design, therefore it can be beneficial to start multiple instances. Set the following Univention Configuration Registry Variable umc/http/processes and restart the Univention Management Console:

$ systemctl restart apache2 \
  univention-management-console-server

The number of instances to configure depends on the workload and the server system. As a general rule of thumb these should not be higher than the machines CPU cores. Good throughput values had resulted in tests with the following combinations:

Automatically detect available CPU cores: umc/http/processes=0
6 CPU cores: umc/http/processes=3
16 CPU cores: umc/http/processes=15
32 CPU cores: umc/http/processes=25

Note that the number of Apache processes may also need to be increased for the customization to take effect.

Further services and components#

Apache#

In environments with many simultaneous accesses to the web server or Univention Portal and Univention Management Console, it may be advisable to increase the number of possible Apache processes or reserve processes. This can be achieved via the UCR variables apache2/server-limit, apache2/start-servers, apache2/min-spare-servers and apache2/max-spare-servers. After setting, the Apache process must be restarted via the command systemctl restart apache2.

Detailed information about useful values for the UCR variables can be found at ServerLimit Directive and StartServers Directive in Apache HTTP Server Version 2.4 Documentation [3].

apache2/max-spare-servers#

Defines the Apache MaxSpareServers setting.

Type:: integer

apache2/min-spare-servers#

Defines the Apache MinSpareServers setting.

Type:: integer

apache2/server-limit#

Defines the Apache ServerLimit setting.

Type:: integer

apache2/start-servers#

Defines the Apache StartServers setting.

Type:: integer

SAML#

By default, SAML assertions are valid for 300 seconds, after which clients must renew them to continue using them. In scenarios where refreshing SAML assertions at such short intervals is too expensive for clients or servers, you can increase the lifetime of SAML assertions. For instructions about how to configure the SAML assertion lifetime, refer to SAML assertion lifetime in Univention Keycloak app documentation [4].

Carefully consider the increase of the SAML assertion lifetime, because it has implications on the security.

Squid#

If the Squid proxy service is used with NTLM authentication, up to five running NTLM requests can be processed in parallel. If many proxy requests are received in parallel, the Squid user may occasionally receive an authentication error. The number of parallel NTLM authentication processes can be configured with the Univention Configuration Registry Variable squid/ntlmauth/children.

squid/ntlmauth/children#

Sets the maximum number of authentication processes for NTLM authentication in Squid.

Default value:: 10
Type:: integer

BIND#

BIND can use two different backend for its configuration: OpenLDAP or the internal LDB database of Samba/AD. The backend is configured via the Univention Configuration Registry Variable dns/backend. On UCS Directory Nodes running Samba/AD, the backend must not be changed to OpenLDAP.

When using the Samba backend, a search is performed in the LDAP for every DNS request. With the OpenLDAP backend, a search is only performed in the directory service if the DNS data has changed. For this reason, using the OpenLDAP backend can reduce the load on a Samba/AD domain controller.

dns/backend#

Specifies whether Bind uses the OpenLDAP directory or the Samba LDB database as backend for its configuration. On Directory Nodes running Samba, you must not set this variable to ldap.

Possible values:: ldap, samba4
Type:: string

Kernel#

In medium and larger environments the maximum number of open files allowed by the Linux kernel may be set too low by default. As each instance requires some unswappable memory in the Linux kernel, too many objects may lead to a resource depletion and denial-of-service problems in multi-user environments. Because of that the number of allowed file objects is limited by default.

The maximum number of open files can be configured on a per-user or per-group basis. The default for all users can be set through the following Univention Configuration Registry Variables:

security/limits/user/default/hard/nofile: The hard limit defines the upper limit a user can assign to a process. The default is 32768.
security/limits/user/default/soft/nofile: The soft limit defines the default settings for the processes of the user. The default is 32768.

A similar problem exists with the Inotify sub-system of the kernel, which can be used by all users and applications to monitor changes in file systems.

kernel/fs/inotify/max_queued_events#

Sets the upper limit of queued events per inotify instance.

Default value:: 16384
Type:: integer

kernel/fs/inotify/max_user_instances#

Sets the upper limit of inotify instances per user ID.

Default value:: 512
Type:: integer

kernel/fs/inotify/max_user_watches#

Sets the upper limit of watched files per user ID.

Default value:: 32768
Type:: integer

When the UCS system is part of a network of a very large number of devices, it is possible that the ARP garbage collector thresholds are insufficient. For those scenarios, raise the following thresholds:

kernel/net/ipv4/neigh/default/gc_thresh1#

Sets the threshold of ARP cache entries below which the kernel garbage collector doesn’t run.

Default value:: 1024
Type:: integer

kernel/net/ipv4/neigh/default/gc_thresh2#

Sets the threshold at which the kernel garbage collector starts removing ARP cache entries older than 5 seconds.

Default value:: 2048
Type:: integer

kernel/net/ipv4/neigh/default/gc_thresh3#

Sets the maximum number of non-permanent ARP cache entries.

Default value:: 4096
Type:: integer

Samba#

Samba uses its own mechanism to specify the maximum number of open files. This can be configured through the Univention Configuration Registry Variable samba/max_open_files. The default is 32808.

If the log file /var/log/samba/log.smbd contains errors like Failed to init inotify - Too many open files, the kernel and Samba limits should be increased and the services should be restarted.

samba/max_open_files#

Sets the maximum number of files that the Samba service can open. If you increase this value, you may also need to increase the Unix process limits.

Type:: integer

System statistics#

The log file /var/log/univention/system-stats.log can be checked for further performance analyses. The system status is logged every 30 minutes. If more regular logging is required, it can be controlled via the UCR variable system/stats/cron.

Dovecot high-performance mode#

Univention Corporate Server configures Dovecot to run in High-security mode by default. Each connection is served by a separate login process. This security has a price: for each connection at least two processes must run.

Thus installations with 10.000s of users hit operating system boundaries. For this case Dovecot offers the High-performance mode. To activate it, login processes are allowed to serve more than one connection. To configure this run

$ ucr mail/dovecot/limits/imap-login/service_count=0

If client_limit=1000 and process_limit=100 are set, only 100 login processes are started, but each serves up to 1000 connections — a total of 100.000 connections.

The cost of this is that if a login process is compromised, an attacker might read the login credentials and emails of all users this login process is serving.

To distribute the load of the login processes evenly between CPU cores, mail/dovecot/limits/imap-login/process_min_avail should be set to the number of CPU cores in the system.

mail/dovecot/limits/imap-login/process_min_avail#

Sets the minimum number of imap-login processes that should always be available to accept new client connections.

Default value:: 0
Type:: integer

UDM HTTP REST API performance scaling#

A single UDM HTTP REST API instance does not use multiple CPU cores by design, therefore it can be beneficial to start multiple instances. By setting the Univention Configuration Registry Variable directory/manager/rest/processes the number of processes can be increased. Afterwards the UDM HTTP REST API needs to be restarted:

$ systemctl restart univention-directory-manager-rest

The number of instances to configure depends on the workload and the server system. As a general rule of thumb these should not be higher than the machines CPU cores. With directory/manager/rest/processes=0 all available CPU cores are used.

directory/manager/rest/processes#

Sets how many UDM HTTP REST API processes the system starts in parallel. When set to 0, the service starts one process per CPU core.

Default value:: 1
Type:: integer

Bibliography#

[1]

Nubus for UCS 5.2 - Operation Manual. Univention GmbH, 2025-2026. URL: https://docs.software-univention.de/ucs-operation/5.2/en/.

[2]

Lightning Memory-Mapped Database Manager (LMDB) documentation. Symas Corporation, 2015. URL: http://www.lmdb.tech/doc/.

[3]

Apache HTTP Server Version 2.4 Documentation. The Apache Software Foundation. URL: https://httpd.apache.org/docs/2.4/en/.

[4]

Univention Keycloak app documentation. Univention GmbH, 2023. URL: https://docs.software-univention.de/keycloak-app/latest/.

Univention Corporate Server 5.2 - Performance Guide

Univention Corporate Server - Performance guide

Contents

Univention Corporate Server - Performance guide#

OpenLDAP and listener/notifier domain replication#

Indexes#

Manage indexes with the index setup script#

Use ldap_setup_index#

Option reference for ldap_setup_index#

Configuration of the database backend#

OpenLDAP ACLs#

Univention Directory Listener#

Performance during initial provisioning of users and groups#

Performance issues during the join process#

Samba#

Local group cache#

UCS management system#

Disabling automatic search#

Imposing a size limit for searches#

Adjusting the limit on open file descriptors#

Vertical performance scaling#

Further services and components#

Apache#

SAML#

Squid#

BIND#

Kernel#

Samba#

System statistics#

Dovecot high-performance mode#

UDM HTTP REST API performance scaling#

Bibliography#