6. Troubleshooting#
When you encounter problems with the operation of the OX Connector app, this section provides information where you can look closer into and to get an impression about what’s going wrong.
6.1. Log files#
The OX Connector app produces different logging information in different places.
- Listener Converter:
/var/log/univention/listener_modules/ox-connector.log Contains log information from the Listener Converter about create, update and delete actions of objects.
It also shows warnings and errors when the OX Connector configuration isn’t correct, or the connector can’t establish a connection to the SOAP API.
- Database management script:
/var/lib/univention-appcenter/apps/ox-connector/data/db.log Contains log information from the Database management script that is described below.
- App Center:
/var/log/univention/appcenter.log Contains log information around activities in the App Center.
The App Center writes OX Connector relevant information to this file, when you run app lifecycle tasks like install, update and uninstall or when you change the app settings.
- Domain join:
/var/log/univention/join.log Contains log information from the join processes. When the App Center install OX Connector, the app also joins the domain.
6.2. Checking the Listener#
Before checking the OX Connector, you may want to have a look at the connection between the Listener and the Listener Converter. The Listener should create files and the Listener Converter should translate these files rather quickly.
$ DIR_LISTENER="/var/lib/univention-appcenter/listener/ox-connector"
$ ls -1 "$DIR_LISTENER"/*.json 2> /dev/null | wc -l
0
If files here are not created upon a change or are piling up, this indicates a problem in the Listener or Listener Converter. See Log files.
6.3. CLI to monitor the current state#
The OX Connector ships a command-line interface that you can use to query and manipulate the database it uses to keep track of current tasks, objects already synced and errors it may have found.
$ /usr/sbin/univention-ox-connector-task-management --help
The tool operates on the SQLite database
/var/lib/univention-appcenter/apps/ox-connector/data/listener/ox-connector.db.
The terminology of the tool is as follows:
- Tasks#
A database table managed by the OX Connector. A row represents an active task. The OX Connector iterates over all tasks and synchronizes them to the OX App Suite.
- Old#
A database table managed by the OX Connector. A row represents the state of an item at the moment it was successfully synchronized. It is more or less a copy of a former task. Needed when certain items are synchronized and reference other items (e.g., when synchronizing a group that contains users). Also used for faster look-ups by storing the database ID given by OX.
- Morgue#
A database table managed by the OX Connector. A row represents a failed task. It was moved automatically or manually to this table and is not actively processed by the OX Connector. Administrators can examine the items in the morgue and decide how to proceed with them (see below).
6.3.1. Health check#
First, have a look at the log file for the Listener Converter and look for warnings and errors, see Log files.
Second you can get a brief summary of current tasks. This can indicate if the OX Connector can process the items fast enough or at all.
$ /usr/sbin/univention-ox-connector-task-management summarize-tasks
$ /usr/sbin/univention-ox-connector-task-management search-tasks
Third, you can get a brief summary of past errors. Every item is an object not synchronized. Note that this only makes sense should you have chosen OX Connector continues after faulty items.
$ /usr/sbin/univention-ox-connector-task-management search-morgue
6.3.2. Handling errors#
You can decide what to do with the items that have been moved to the morgue.
All commands assume that you have the UniventionObjectIdentifier of that
object. For each item you have the option to
Delete it from the list: It is as if this item never hit the OX Connector. The underlying object can of course be synchronized again if it is modified in the LDAP directory (creating a completely new item in the OX Connector’s tasks).
$ /usr/sbin/univention-ox-connector-task-management remove-from-morgue --obj-id=...
Retry the very same item: The erroneous item in the list is again copied to the list of tasks, assuming that the problem is now fixed (e.g., a validation on the OX App Suite’s side has been disabled).
$ /usr/sbin/univention-ox-connector-task-management retry-from-morgue --obj-id=...
Fresh synchronization of the object: The object is again put into the list of tasks but not with the attributes it had when the synchronization happened (and failed). Instead, it is freshly fetched from the LDAP database.
$ /usr/sbin/univention-ox-connector-task-management resync-item --obj-id=...
6.4. Provisioning stops working#
When the provisioning stopped working, a previous change in UDM is a probable reason and the OX Connector doesn’t know how to proceed. The connector retries the action over and over again until an administrator repairs the cause manually.
First, see the Log files and look for warnings and errors. If it’s not a temporary problem like for example network connectivity, the fix requires manual action.
As a last resort, the administrator can move the task aside. The log file
reveals the Database ID of that object (e.g. uid=...; $object_identifier;
tasks:$database_id).
$ /usr/sbin/univention-ox-connector-task-management move-task-to-morgue --task-id=$database_id --error-msg="Manual intervention after careful consideration"
6.4.1. Re-provision all data#
Warning
Depending on the number of users and groups in the UCS LDAP directory, this task may take a lot of time.
Reprovisioning all data isn’t recommended.
The following command reads all UDM objects from the UCS LDAP directory and adds them to the provisioning queue:
$ univention-directory-listener-ctrl resync ox-connector
The re-provisioning won’t run any delete operations, because the Listener only adds existing UDM objects to the queue.
Caution
The OX Connector may decide to delete objects based on data in the JSON
files. For example isOxGroup = Not in a group object.
6.5. Duplicated displaynames#
In OX Connector version 2.2.0 the UDM property oxDisplayName does not have a unique constraint anymore.
If duplicate values are used, but OX is not prepared for that, the SOAP API calls will fail with the following exception.
2023-05-30 11:59:31 WARNING Traceback (most recent call last):
2023-05-30 11:59:31 WARNING File "/tmp/univention-ox-connector.listener_trigger", line 324, in run_on_files
2023-05-30 11:59:31 WARNING f(obj)
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/__init__.py", line 86, in run
2023-05-30 11:59:31 WARNING modify_user(obj)
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/users.py", line 420, in modify_user
2023-05-30 11:59:31 WARNING user.modify()
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/backend.py", line 477, in modify
2023-05-30 11:59:31 WARNING super(SoapUser, self).modify()
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/backend.py", line 180, in modify
2023-05-30 11:59:31 WARNING self.service(self.context_id).change(obj)
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/services.py", line 536, in change
2023-05-30 11:59:31 WARNING return self._call_ox('change', usrdata=user)
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/services.py", line 163, in _call_ox
2023-05-30 11:59:31 WARNING return getattr(service, func)(**kwargs)
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/zeep/proxy.py", line 46, in __call__
2023-05-30 11:59:31 WARNING return self._proxy._binding.send(
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/zeep/wsdl/bindings/soap.py", line 135, in send
2023-05-30 11:59:31 WARNING return self.process_reply(client, operation_obj, response)
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/zeep/wsdl/bindings/soap.py", line 229, in process_reply
2023-05-30 11:59:31 WARNING return self.process_error(doc, operation)
2023-05-30 11:59:31 WARNING File "/usr/lib/python3.9/site-packages/zeep/wsdl/bindings/soap.py", line 329, in process_error
2023-05-30 11:59:31 WARNING raise Fault(
2023-05-30 11:59:31 WARNING zeep.exceptions.Fault: The displayname is already used; exceptionId 1170523631-4
To fix this issue, a change in the OX App Suite configuration is required.
Add the following lines to the user.properties file.
com.openexchange.user.enforceUniqueDisplayName=false
com.openexchange.folderstorage.database.preferDisplayName=false
Note
This is configured by default in the OX App Suite installation from the App center.
6.6. Traceback provisioning groups#
When an ox group is synchronized, the OX Connector obtains information about all its users by reading from the listener/old directory where the latest version of the objects that have already been synchronized is stored. If any user is part of such group but is not in listener/old, the OX Connector will fail with a traceback like the following:
2024-11-15 16:06:33 INFO Group oxgroup will be OX Group
2024-11-15 16:06:33 INFO Error while processing /var/lib/univention-appcenter/apps/ox-connector/data/listener/2024-11-15-15-51-31-669745.json
2024-11-15 16:06:33 INFO This is consecutive error #11
2024-11-15 16:06:33 INFO Sleeping for 0 sec
2024-11-15 16:06:33 WARNING Traceback (most recent call last):
2024-11-15 16:06:33 INFO Successfully processed 0 files during this run
2024-11-15 16:06:33 WARNING File "/tmp/univention-ox-connector.listener_trigger", line 419, in run_on_files
2024-11-15 16:06:33 WARNING function(obj)
2024-11-15 16:06:33 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/__init__.py", line 115, in run
2024-11-15 16:06:33 WARNING for new_obj in get_group_objs(obj):
2024-11-15 16:06:33 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/__init__.py", line 173, in get_group_objs
2024-11-15 16:06:33 WARNING user_obj = univention.ox.provisioning.helpers.get_old_obj(user)
2024-11-15 16:06:33 WARNING File "/tmp/univention-ox-connector.listener_trigger", line 76, in _get_old_object
2024-11-15 16:06:33 WARNING raise Exception(f"Old object file {path_to_old_user} for {distinguished_name} does not exist!\nYou need to re-provision \"{distinguished_name}\" (see https://docs.software-univention.de/ox-connector-app/latest/troubleshooting .html#traceback-provisioning-groups).")
2024-11-15 16:06:33 WARNING Exception: Old object file /var/lib/univention-appcenter/apps/ox-connector/data/listener/old/d10338de-3144-103f-8ea2-f39fa7a811dd.json for uid=oxuser1,cn=users,dc=example,dc=com does not exist!
2024-11-15 16:06:33 WARNING You need to re-provision "uid=oxuser1,cn=users,dc=example,dc=com" (see https://docs.software-univention.de/ox-connector-app/latest/troubleshooting.html#traceback-provisioning-groups).
You need to re-provision the user object (uid=oxuser1,cn=users,dc=example,dc=com in this case) manually. Follow the instructions in Handling errors to synchronize the missing users. After this manual intervention the connector automatically continues with the synchronization of the group object.
6.7. Collect information for support ticket#
Before you open a support ticket, make sure to collect and provide relevant details about your case, so that the Univention Support team can help you:
Provide the relevant messages and tracebacks from Log files, specifically the Listener Converter.
Describe the steps that can reproduce the faulty behavior.
Describe the expected behavior.
Provide data from the provisioning that causes the error.
6.8. Invalid values for OX_USER_IDENTIFIER or OX_GROUP_IDENTIFIER#
Only a UDM user property (or UDM group property in case of OX_GROUP_IDENTIFIER) that contains a single value which is not None is a valid option. In case a UDM property that contains an empty value or a list of values is specified, the OX Connector will enter an error state which needs to be resolved manually by simply setting a valid value.
Setting invalid values for the app settings OX_USER_IDENTIFIER or OX_GROUP_IDENTIFIER will lead to the following errors:
2024-01-11 13:57:39 WARNING Traceback (most recent call last):
2024-01-11 13:57:39 WARNING File "/tmp/univention-ox-connector.listener_trigger", line 351, in run_on_files
2024-01-11 13:57:39 WARNING function(obj)
2024-01-11 13:57:39 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/__init__.py", line 86, in run
2024-01-11 13:57:39 WARNING modify_user(obj)
2024-01-11 13:57:39 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/users.py", line 454, in modify_user
2024-01-11 13:57:39 WARNING user.modify()
2024-01-11 13:57:39 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/backend.py", line 475, in modify
2024-01-11 13:57:39 WARNING super(SoapUser, self).modify()
2024-01-11 13:57:39 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/backend.py", line 176, in modify
2024-01-11 13:57:39 WARNING assert self.name is not None
2024-01-11 13:57:39 WARNING No name for this attribute. Missing or misconfigured identifier app settings
2024-01-11 13:57:39 WARNING (OX_USER_IDENTIFIER or OX_GROUP_IDENTIFIER) might be the reason, see
2024-01-11 13:57:39 WARNING https://docs.software-univention.de/ox-connector-app/latest/troubleshooting.html#invalid-values-for-ox-user-identifier-or-ox-group-identifier
2024-01-11 13:57:39 WARNING for more information.
setting "users" udm property for groups
2024-01-11 13:59:36 WARNING Traceback (most recent call last):
2024-01-11 13:59:36 WARNING File "/tmp/univention-ox-connector.listener_trigger", line 351, in run_on_files
2024-01-11 13:59:36 WARNING function(obj)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/__init__.py", line 108, in run
2024-01-11 13:59:36 WARNING modify_group(new_obj)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/provisioning/groups.py", line 146, in modify_group
2024-01-11 13:59:36 WARNING group.modify()
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/backend.py", line 180, in modify
2024-01-11 13:59:36 WARNING self.service(self.context_id).change(obj)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/services.py", line 607, in change
2024-01-11 13:59:36 WARNING return self._call_ox('change', grp=grp)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/univention/ox/soap/services.py", line 194, in _call_ox
2024-01-11 13:59:36 WARNING return getattr(service, func)(**kwargs)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/proxy.py", line 46, in __call__
2024-01-11 13:59:36 WARNING return self._proxy._binding.send(
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/wsdl/bindings/soap.py", line 123, in send
2024-01-11 13:59:36 WARNING envelope, http_headers = self._create(
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/wsdl/bindings/soap.py", line 73, in _create
2024-01-11 13:59:36 WARNING serialized = operation_obj.create(*args, **kwargs)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/wsdl/definitions.py", line 224, in create
2024-01-11 13:59:36 WARNING return self.input.serialize(*args, **kwargs)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/wsdl/messages/soap.py", line 79, in serialize
2024-01-11 13:59:36 WARNING self.body.render(body, body_value)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/element.py", line 232, in render
2024-01-11 13:59:36 WARNING self._render_value_item(parent, value, render_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/element.py", line 256, in _render_value_item
2024-01-11 13:59:36 WARNING return self.type.render(node, value, None, render_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/types/complex.py", line 307, in render
2024-01-11 13:59:36 WARNING element.render(node, element_value, child_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/indicators.py", line 256, in render
2024-01-11 13:59:36 WARNING element.render(parent, element_value, child_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/element.py", line 232, in render
2024-01-11 13:59:36 WARNING self._render_value_item(parent, value, render_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/element.py", line 255, in _render_value_item
2024-01-11 13:59:36 WARNING return value._xsd_type.render(node, value, xsd_type, render_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/types/complex.py", line 307, in render
2024-01-11 13:59:36 WARNING element.render(node, element_value, child_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/indicators.py", line 256, in render
2024-01-11 13:59:36 WARNING element.render(parent, element_value, child_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/element.py", line 232, in render
2024-01-11 13:59:36 WARNING self._render_value_item(parent, value, render_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/elements/element.py", line 256, in _render_value_item
2024-01-11 13:59:36 WARNING return self.type.render(node, value, None, render_path)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/types/simple.py", line 96, in render
2024-01-11 13:59:36 WARNING node.text = value if isinstance(value, etree.CDATA) else self.xmlvalue(value)
2024-01-11 13:59:36 WARNING File "/usr/lib/python3.9/site-packages/zeep/xsd/types/builtins.py", line 27, in _wrapper
2024-01-11 13:59:36 WARNING raise ValueError(
2024-01-11 13:59:36 WARNING ValueError: The String type doesn't accept collections as value