UMC modules

Python API for UMC modules

The Python API for UMC modules primary consists of one base class that must be implemented. As an addition the Python API provides some helper functions and classes:

• exception classes

• translation support

• logging functions

• UCR access

The XML file defining the UMC module specifies functions for the commands provided by the module. These functions must be implemented as methods of a class named Instance that inherits Base.

The following Python code example matches the definition in the previous section:

from univention.management.console import Translation
from univention.management.console.config import ucr
from univention.management.console.modules import Base
from univention.management.console.modules.decorators import sanitize
from univention.management.console.modules.sanitizers import IntegerSanitizer
from univention.management.console.log import MODULE

_ = Translation('univention-management-console-modules-udm').translate

class Instance(Base):

    @sanitize(end=IntegerSanitizer(minimum=0),)
    def query(self, request):
        end = request.options['end']
        result = list(range(end))
        self.finished(request.id, result)

Each command methods has one parameter that contains the HTTP request of type Request. Such an object has the following properties:

id

is the unique identifier of the request

options

contains the arguments for the command. For most commands it is a dictionary.

flavor

is the name of the flavor that was used to invoke the command. This might be None

• username: The username of the owner of this session

• password: The password of the user

• auth_type: The authentication method which was used to authenticate this user

The query method in the example above shows how to retrieve the command parameters and what to do to send the result back to the client. Important is that returning a value in a command function does not send anything back to the client. Therefore the function finished must be invoked. The first parameter is the identifier of the request that will be answered and the second parameter the data structure containing the result. As the result is converted to JSON it must just contain data types that can be converted.

The base class for modules provides some methods that could be useful when writing UMC modules:

Methods * init: Is invoked after the module process has been initialised. At that moment, the settings, like locale and username and password are available.

class univention.management.console.base.Base(*args, **kwargs)

Bases: Translation

The base class for UMC modules

update_language(locales)

set_locale(_locale)

property username

Deprecated since version 5.0-4: use request.username instead!

property user_dn

Deprecated since version 5.0-4: use request.user_dn instead!

property password

Deprecated since version 5.0-4: use request.password instead!

property auth_type

Deprecated since version 5.0-4: use request.auth_type instead!

property tornado_routes

prepare(request)

this function is invoked after the module process started.

init()

this function is invoked after the module process started.

destroy()

this function is invoked before the module process is exiting.

execute(method, request, *args, **kwargs)

security_checks(request, function)

thread_finished_callback(thread, result, request)

error_handling(etype, exc, etraceback)

Translate generic UDM exceptions back to LDAP exceptions.

Parameters:

• etype – The exception class.

• exc – The exception instance.

• etraceback – The exception traceback instance; may be None.

default_response_headers()

get_user_ldap_connection(no_cache=False, **kwargs)

Deprecated since version 5.0-4: use request.get_user_ldap_connection() instead!

bind_user_connection(lo)

Deprecated since version 5.0-4: use request.bind_user_connection() instead!

require_password()

Deprecated since version 5.0-4: use request.require_password() instead!

finished(id, response, message=None, success=True, status=None, mimetype=None, headers=None, error=None, reason=None)

Should be invoked by module to finish the processing of a request. ‘id’ is the request command identifier

result(response)

Convenience decorators for developers of UMC modules

Functions exposed by UMC modules often share some logic. They check the existence and formatting of variables or check permissions. If anything fails, they react in a similar way. If everything is correct, the real logic is often as simple as returning one single value.

This module provides functions that can be used to separate repeating tasks from the actual business logic. This means:

• less time to code

• fewer bugs

• consistent behavior throughout the UMC in standard cases

Note that the functions defined herein do not cover every corner case during UMC module development. You are not bound to use them if you need more flexibility.

univention.management.console.modules.decorators.file_upload(function)

This decorator restricts requests to be UPLOAD-commands. Simple, yet effective

univention.management.console.modules.decorators.log(function=None, sensitives=None, customs=None, single_values=False)

Log decorator to be used with simple_response():

@simple_response
@log
def my_func(self, var1, var2):
    return "%s__%s" % (var1, var2)

The above example will write two lines into the logfile for the module (given that the UCR variable umc/module/debug/level is set to at least 3):

<date>     INFO [         -] my_func got: var1='value1', var2='value2'
<date>     INFO [         -] my_func returned: 'value1__value2'

The variable names are ordered by appearance and hold the values that are actually going to be passed to the function (i.e. after they were sanitize() ‘d or set to their default value). You may specify the names of sensitive arguments that should not show up in log files and custom functions that can alter the representation of a certain variable’s values (useful for non-standard datatypes like regular expressions - you may have used a PatternSanitizer ):

@sanitize(pattern=PatternSanitizer())
@simple_reponse
@log(sensitives=['password'], customs={'pattern':lambda x: x.pattern})
def count_ucr(self, username, password, pattern):
    return self._ucr_count(username, password, pattern)

This results in something like:

<date>     INFO [         -] count_ucr got: password='********', username='Administrator', pattern='.*'
<date>     INFO [         -] count_ucr returned: 650

The decorator also works with multi_response():

@multi_response
@log
def multi_my_func(self, var1, var2):
    return "%s__%s" % (var1, var2)

This results in something like:

<date>     INFO [         -] multi_my_func got: [var1='value1', var2='value2'], [var1='value3', var2='value4']
<date>     INFO [         -] multi_my_func returned: ['value1__value2', 'value3__value4']

univention.management.console.modules.decorators.multi_response(function=None, with_flavor=None, single_values=False, progress=False)

This decorator acts similar to simple_response() but can handle a list of dicts instead of a single dict.

Technically another object is passed to the function that you can name as you like. You can iterate over this object and get the values from each dictionary in request.options.

Default values and flavors are supported.

You do not return a value, you yield them (and you are supposed to yield!):

@multi_response
def my_multi_func(self, iterator, variable1, variable2=''):
    # here, variable1 and variable2 are yet to be initialised
    # i.e. variable1 and variable2 will be None!
    do_some_initial_stuff()
    try:
        for variable1, variable2 in iterator:
            # now they are set
            yield '%s_%s' % (self._saved_dict[variable1], variable2)
    except KeyError:
        raise UMC_Error('Something went wrong')
    else:
        # only when everything went right...
        do_some_cleanup_stuff()

The above code will send a list of answers to the client as soon as the function is finished (i.e. after do_some_cleanup_stuff()) filled with values yielded.

If you have just one variable in your dictionary, do not forget to add a comma, otherwise Python will assign the first value a list of one element:

for var, in iterator:
    # now var is set correctly
    pass

class univention.management.console.modules.decorators.reloading_ucr(ucr, timeout=0.2)

Bases: object

univention.management.console.modules.decorators.require_password(function)

univention.management.console.modules.decorators.sanitize(*args, **kwargs)

Decorator that lets you sanitize the user input.

The sanitize function can be used to validate the input as well as change it.

Note that changing a value here will actually alter the request object. This should be no problem though.

If the validation step fails an error will be passed to the user instead of executing the function. This step should not raise anything other than ValidationError or UnformattedValidationError (one should use the method raise_validation_error()).

You can find some predefined Sanitize classes in the corresponding module or you define one yourself, deriving it from Sanitizer:

class SplitPathSanitizer(Sanitizer):
    def __init__(self):
        super(SplitPathSanitizer, self).__init__(
            validate_none=True,
            may_change_value=True)

    def _sanitize(self, value, name, further_fields):
        if value is None:
            return []
        try:
            return value.split('/')
        except BaseException:
            self.raise_validation_error('Split failed')

Before:

def my_func(self, request):
    var1 = request.options.get('var1')
    var2 = request.options.get('var2', 20)
    try:
        var1 = int(var1)
        var2 = int(var2)
    except (ValueError, TypeError):
        self.finished(request.id, None, 'Cannot convert to int', status=400)
        return
    if var2 < 10:
        self.finished(request.id, None, 'var2 must be >= 10', status=400)
        return
    self.finished(request.id, var1 + var2)

After:

@sanitize(
    var1=IntegerSanitizer(required=True),
    var2=IntegerSanitizer(required=True, minimum=10, default=20)
)
def add(self, request):
    var1 = request.options.get('var1')  # could now use ['var1']
    var2 = request.options.get('var2')
    self.finished(request.id, var1 + var2)

The decorator can be combined with other decorators like simple_response() (be careful with ordering of decorators here):

@sanitize(
    var1=IntegerSanitizer(required=True),
    var2=IntegerSanitizer(required=True, minimum=10)
)
@simple_response
def add(self, var1, var2):
    return var1 + var2

Note that you lose the capability of specifying defaults in @simple_response. You need to do it in @sanitize now.

univention.management.console.modules.decorators.sanitize_dict(sanitized_attrs, **kwargs)

univention.management.console.modules.decorators.sanitize_list(sanitizer, **kwargs)

univention.management.console.modules.decorators.simple_response(function=None, with_flavor=None, with_progress=False, with_request=False)

If your function is as simple as: “Just return some variables” this decorator is for you.

Instead of defining the function

def my_func(self, response): pass

you now define a function with the variables you would expect in request.options. Default values are supported:

@simple_response
def my_func(self, var1, var2='default'): pass

The decorator extracts variables from request.options. If the variable is not found, it either returns a failure or sets it to a default value (if specified by you).

If you need to get the flavor passed to the function you can do it like this:

@simple_response(with_flavor=True)
def my_func(self, flavor, var1, var2='default'): pass

With with_flavor set, the flavor is extracted from the request. You can also set with_flavor=’varname’, in which case the variable name for the flavor is varname. True means ‘flavor’. As with ordinary option arguments, you may specify a default value for flavor in the function definition:

@simple_response(with_flavor='module_flavor')
def my_func(self, flavor='this comes from request.options',
    module_flavor='this is the flavor (and its default value)'): pass

Instead of stating at the end of your function

self.finished(request.id, some_value)

you now just

return some_value

Before:

def my_func(self, request):
    variable1 = request.options.get('variable1')
    variable2 = request.options.get('variable2')
    flavor = request.flavor or 'default flavor'
    if variable1 is None:
        self.finished(request.id, None, message='variable1 is required', success=False)
        return
    if variable2 is None:
    variable2 = ''
    try:
        value = '%s_%s_%s' % (self._saved_dict[variable1], variable2, flavor)
    except KeyError:
        self.finished(request.id, None, message='Something went wrong', success=False, status=500)
        return
    self.finished(request.id, value)

After:

@simple_response(with_flavor=True)
def my_func(self, variable1, variable2='', flavor='default_flavor'):
    try:
        return '%s_%s_%s' % (self._saved_dict[variable1], variable2, flavor)
    except KeyError:
        raise UMC_Error('Something went wrong')

Sanitize classes for the sanitize decorator

This module provides the Sanitize base class as well as some important and often used Sanitizers. They are used in the sanitize function. If the provided classes do not meet your requirements you can easily make one yourself.

The main job of sanitizers is to alter values if needed so that they cannot do something harmful in the exposed UMC-functions. But they are also very helpful when one needs to just validate input.

class univention.management.console.modules.sanitizers.BooleanSanitizer(**kwargs)

Bases: Sanitizer

BooleanSanitizer makes sure that the value is a bool. It converts other data types if possible.

class univention.management.console.modules.sanitizers.ChoicesSanitizer(choices: Iterable[str], **kwargs: Any)

Bases: Sanitizer

ChoicesSanitizer makes sure that the input is in a given set of choices.

Parameters:

choices (Any) – the allowed choices used.

class univention.management.console.modules.sanitizers.DNSanitizer(regex_pattern: Pattern[str] | str | None = None, re_flags: int = 0, minimum: int | None = None, maximum: int | None = None, **kwargs: Any)

Bases: StringSanitizer

DNSanitizer is a sanitizer that checks if the value has correct LDAP Distinguished Name syntax

class univention.management.console.modules.sanitizers.DictSanitizer(sanitizers: dict[str, Sanitizer], allow_other_keys: bool = True, default_sanitizer: Sanitizer = None, **kwargs: Any)

Bases: Sanitizer

DictSanitizer makes sure that the value is a dict and sanitizes its fields.

You can give the same parameters as the base class. Plus:

Parameters:

• sanitizers (dict[str, Sanitizer]) – will be applied to the content of the sanitized dict

• allow_other_keys (bool) – if other keys than those in sanitizers are allowed.

• default_sanitizer (Sanitizer) – will be applied to the content if no sanitizer is defined

class univention.management.console.modules.sanitizers.EmailSanitizer(**kwargs: Any)

Bases: StringSanitizer

EmailSanitizer is a very simple sanitizer that checks the very basics of an email address: At least 3 characters and somewhere in the middle has to be an @-sign

class univention.management.console.modules.sanitizers.IntegerSanitizer(minimum: int | None = None, maximum: int | None = None, minimum_strict: bool | None = None, maximum_strict: bool | None = None, **kwargs: Any)

Bases: Sanitizer

IntegerSanitizer makes sure that the value is an int. It converts other data types if possible and is able to validate boundaries.

You can give the same parameters as the base class. Plus:

Parameters:

• minimum (int) – minimal value allowed

• minimum_strict (bool) – if the value must be > minimum (>= otherwise)

• maximum (int) – maximal value allowed

• maximum_strict (bool) – if the value must be < maximum (<= otherwise)

class univention.management.console.modules.sanitizers.LDAPSearchSanitizer(**kwargs)

Bases: SearchSanitizer

Sanitizer for LDAP-Searches. Everything that could possibly confuse an LDAP-Search is escaped except for *.

ESCAPED_WILDCARD = '\\2a'

class univention.management.console.modules.sanitizers.ListSanitizer(sanitizer: Sanitizer | None = None, min_elements: int | None = None, max_elements: int | None = None, **kwargs: Any)

Bases: Sanitizer

ListSanitizer makes sure that the value is a list and sanitizes its elements.

You can give the same parameters as the base class. Plus:

Parameters:

• sanitizer (Sanitizer) – sanitizes each of the sanitized list’s elements. If None, no sanitizing of elements takes place.

• min_elements (int) – must have at least this number of elements

• max_elements (int) – must have at most this number of elements

class univention.management.console.modules.sanitizers.MappingSanitizer(mapping: Mapping[str, Any], **kwargs: Any)

Bases: ChoicesSanitizer

MappingSanitizer makes sure that the input is in a key in a dictionary and returns the corresponding value.

Parameters:

mapping ({object : object}) – the dictionary that is used for sanitizing

exception univention.management.console.modules.sanitizers.MultiValidationError

Bases: ValidationError

Error used for validation of an arbitrary number of sanitizers. Used by DictSanitizer and ListSanitizer.

add_error(e: ValidationError, name: int | str) → None

Adds a ValidationError

number_of_errors() → int

Cumulative number of errors found

has_errors() → bool

Found any errors

result() → dict[int | str, str]

Returns a errors in a similar way like the arguments were passed to the sanitizers.

class univention.management.console.modules.sanitizers.PatternSanitizer(ignore_case: bool = True, multiline: bool = True, **kwargs: Any)

Bases: SearchSanitizer

PatternSanitizer converts the input into a regular expression. It can handle anything (through the inputs __str__ method), but only strings seem to make sense.

The input should be a string with asterisks (*) if needed. An askterisk stands for anything at any length (regular expression: .*).

The sanitizer escapes the input, replaces * with .* and applies the params.

You can give the same parameters as the base class.

If you specify a string as default, it will be compiled to a regular expression. Hints: default=’.*’ -> matches everything; default=’(?!)’ -> matches nothing

Plus:

Parameters:

• ignore_case (bool) – pattern is compiled with re.IGNORECASE flag to search case insensitive.

• multiline (bool) – pattern is compiled with re.MULTILINE flag to search across multiple lines.

class univention.management.console.modules.sanitizers.Sanitizer(**kwargs)

Bases: object

Base class of all sanitizers.

For reasons of extensibility and for ease of subclassing, the parameters are **kwargs. But only the following are meaningful:

Parameters:

• further_arguments (list) – names of arguments that should be passed along with the actual argument in order to return something reasonable. Default: None

• required (bool) – if the argument is required. Default: False

• default (Any) – if argument is not given and not required, default is returned - even when not may_change_value. Note that this value is not passing the sanitizing procedure, so make sure to be able to handle it. Default: None

• may_change_value (bool) – if the process of sanitizing is allowed to alter request.options. If not, the sanitizer can still be used for validation. Default: True

• allow_none (bool) – if None is allowed and not further validated. Default: False

sanitize(name: str, options: Mapping[str, Any]) → Any

Sanitize function. Internally calls _sanitize with the correct values and returns the new value (together with a flag indicating whether the value was found at all). If you write your own Sanitize class, you probably want to override _sanitize().

_sanitize(value: Any, name: str, further_arguments: Mapping[str, Any]) → Any

The method where the actual sanitizing takes place.

The standard method just returns value so be sure to override this method in your Sanitize class.

Parameters:

• value (Any) – the value as found in request.options.

• name (str) – the name of the argument currently sanitized.

• further_arguments (dict[str, Any]) – dictionary holding the values of those additional arguments in request.options that are needed for sanitizing. the arguments come straight from the not altered options dict (i.e. before potentially changing sanitizing happened).

raise_validation_error(msg: str, **kwargs: Any) → NoReturn

Used to more or less uniformly raise a ValidationError. This will actually raise an UnformattedValidationError for your convenience. If used in _sanitize(), it will be automatically enriched with name, value und formatting in sanitize().

Parameters:

**kwargs (dict) – additional arguments for formatting

raise_formatted_validation_error(msg: str, name: str, value: Any, **kwargs: Any) → NoReturn

Used to more or less uniformly raise a ValidationError. name and value need to passed because the sanitizer should be thread safe.

Parameters:

• msg (str) – error message

• name (str) – name of the argument

• value (Any) – the argument which caused the error

• **kwargs (dict) – additional arguments for formatting

class univention.management.console.modules.sanitizers.SearchSanitizer(**kwargs)

Bases: Sanitizer

Baseclass for other Sanitizers that are used for a simple search. That means that everything is escaped except for asterisks that are considered as wildcards for any number of characters. (If use_asterisks is True, which is default)

Handles adding of asterisks and and some simple sanity checks. Real logic is done in a to-be-overridden method named _escape_and_return().

Currently used for LDAPSearchSanitizer and PatternSanitizer.

Like the Baseclass of all Sanitizers, it accepts only keyword-arguments (derived classes may vary). You may specify the same as in the Baseclass plus:

Parameters:

• add_asterisks (bool) –

  add asterisks at the beginning and the end of the value if needed. Examples:

  • ”string” -> “*string*”

  • ”” -> “*”

  • ”string*” -> “string*”

  Default: True

• max_number_of_asterisks (int) – An error will be raised if the number of * in the string exceeds this limit. Useful because searching with too many of these patterns in a search query can be very expensive. Note that * from add_asterisks do count. None means an arbitrary number is allowed. Default: 5

• use_asterisks (bool) –

  treat asterisks special, i.e. as a substring of arbitrary length. If False, it will be escaped as any other character. If False the defaults change:

  • add_asterisks to False

  • max_number_of_asterisks to None.

  Default: True

class univention.management.console.modules.sanitizers.StringSanitizer(regex_pattern: Pattern[str] | str | None = None, re_flags: int = 0, minimum: int | None = None, maximum: int | None = None, **kwargs: Any)

Bases: Sanitizer

StringSanitizer makes sure that the input is a string. The input can be validated by a regular expression and by string length

Parameters:

• regex_pattern (str or re._pattern_type) – a regex pattern or a string which will be compiled into a regex pattern

• re_flags (int) – additional regex flags for the regex_pattern which will be compiled if regex_pattern is a string

• minimum (int) – the minimum length of the string

• maximum (int) – the maximum length of the string

exception univention.management.console.modules.sanitizers.UnformattedValidationError(msg: str, kwargs: Mapping[str, Any])

Bases: Exception

Unformatted error raised when the sanitizer finds a value he cannot use at all (e.g. letters when an int is expected). Should be “enhanced” to a ValidationError.

exception univention.management.console.modules.sanitizers.ValidationError(msg: str, name: str, value: Any)

Bases: Exception

Error raised when the sanitizer finds a value he cannot use at all (e.g. letters when an int is expected).

number_of_errors() → int

1…

result() → Any

Returns the message

Mixins for UMC module classes

This module provides some mixins that can be incorporated in existing UMC modules. These mixins extend the functionality of the module in an easy-to-use way. Just let your module derive from Base (as before) and the mixin classes.

class univention.management.console.modules.mixins.Progress(progress_id, title, total)

Bases: object

Class to keep track of the progress during execution of a function. Used internally.

progress(detail=None, message=None)

finish()

finish_with_result(result)

initialised()

exception(exc_info)

poll()

class univention.management.console.modules.mixins.ProgressMixin

Bases: object

Mixin to provide two new functions:

• new_progress to create a new Progress.

• progress to let the client fetch the progress made up to this moment.

The progress function needs to be made public by the XML definition of the module. To use this mixin, just do:

class Instance(Base, ProgressMixin):
    pass

new_progress(title=None, total=0)

progress(request, *args, **kwargs)

thread_progress_finished_callback(thread, result, request, progress)

exception univention.management.console.error.UMC_Error(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: HTTPError

include_traceback = False

msg = None

status = 400

exception univention.management.console.error.BadRequest(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

msg = 'Bad request'

status = 400

exception univention.management.console.error.Unauthorized(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

msg = 'Unauthorized'

status = 401

exception univention.management.console.error.Forbidden(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

msg = 'Forbidden'

status = 403

exception univention.management.console.error.NotFound(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

msg = 'Not found'

status = 404

exception univention.management.console.error.MethodNotAllowed(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

msg = 'Method not allowed'

status = 405

exception univention.management.console.error.NotAcceptable(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

status = 406

exception univention.management.console.error.UnprocessableEntity(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

status = 422

exception univention.management.console.error.ServerError(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

msg = 'Internal error'

status = 500

exception univention.management.console.error.BadGateway(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

status = 502

exception univention.management.console.error.ServiceUnavailable(message=None, status=None, result=None, headers=None, traceback=None, reason=None)

Bases: UMC_Error

status = 503

exception univention.management.console.error.PasswordRequired

Bases: Unauthorized

exception univention.management.console.error.LDAP_ServerDown

Bases: UMC_Error

exception univention.management.console.error.LDAP_ConnectionFailed(exc)

Bases: LDAP_ServerDown

exception univention.management.console.error.OpenIDProvideUnavailable(*args, **kwargs)

Bases: ServiceUnavailable