Nubus for Kubernetes - Operation Manual 1.x

Use external S3-compatible object storage

6.5. Use external S3-compatible object storage#

Nubus for Kubernetes needs an S3-compatible object storage for various application components. This section describes how to configure Nubus for Kubernetes to use an external S3-compatible object storage. First, you need to prepare a bucket in your S3-compatible storage. Second, you must configure your Nubus deployment to use the storage.

Important

Nubus for Kubernetes includes a deployment for MinIO. However, Univention doesn’t provide support for the MinIO deployment within Nubus.

Recommendation

Use an external S3-compatible object storage with implemented concepts for backup, restore, and redundancy.

See also

S3-compatible object storage as architectural piece in Nubus

in Univention Nubus for Kubernetes - Architecture Manual [2] for information about application components using S3-compatible object storage.

6.5.1. Scenarios#

The S3-compatible object storage receives objects from the Portal Consumer and provides them to the Portal Server. It involves the following actors:

  • Inside the Kubernetes cluster

    • Portal Consumer

    • Portal Server

  • Outside the Kubernetes cluster

    • Portal Frontend running in the user’s browser

Depending on the accessibility of the S3-compatible object storage for those actors, you need to set different configuration parameters for Nubus for Kubernetes. Consider the following scenarios:

  1. Public availability, your S3-compatible object storage matches one of the following criteria:

    • Is available on the internet

    • Is available on the same network as the user’s browser.

    The sections later on this page cover this scenario.

  2. Restricted public availability, your S3-compatible object storage matches one of the following criteria:

    • Isn’t reachable from the internet.

    • Isn’t reachable from the network that the user’s browser is in.

    • Doesn’t serve public, unauthenticated requests.

    In this scenario, don’t configure a custom value for nubusPortalConsumer.portalConsumer.assetsBaseUrl. Instead, use something like the NGINX S3 Gateway to serve requests to /univention/(portal|selfservice)/icons/(logos|entries)/(.*) from your S3 bucket. However, this kind of configuration is beyond the scope of this documentation.

    See also

    nginx/nginx-s3-gateway - GitHub

    for the project page of the NGINX S3 Gateway.

6.5.2. Prepare external S3-compatible object storage for Nubus#

Before you can connect Nubus for Kubernetes to your S3-compatible storage, you need to prepare it properly. Also consider the Scenarios section. This section describes the necessary steps.

  1. Create a private bucket in your S3-compatible storage for the Nubus assets.

    Nubus creates various folders in it. Most are for private use, one folder must allow public access.

    Important:

    Make sure and verify that you configure the bucket as a private bucket.

  2. Create a policy for the folder portal-assets that allows public read access.

    Important:

    Actors, such as the Portal Server and users with their web browser need to resolve and access objects in this folder.

  3. Create an account and a policy for read access to portal-data for the Portal Server. Remember the access key and the secret key for each account. You need those credentials for the configuration of the Nubus deployment. In MinIO the policy looks similar to Listing 6.7.

    Listing 6.7 Read-only access policy in MinIO as required by the Portal Server#
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::portal-data/*",
        "arn:aws:s3:::portal-data"
      ]
    }
  ]
}

  4. Create an account and a policy for read and write access to portal-data for the Portal Consumer. Remember the access key and the secret key for each account. You need those credentials for the configuration of the Nubus deployment. In MinIO the policy looks similar to Listing 6.8.

    Listing 6.8 Read and write access policy in MinIO as required by the Portal Consumer#
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::portal-data",
        "arn:aws:s3:::portal-data/*"
      ]
    }
  ]
}

  5. You need to collect the following information about your S3-compatible object storage:

    • Bucket name

    • Access key and secret key for each of the users in your storage you created before.

    • Endpoint URL of your S3-compatible storage. The endpoint URL is from the perspective of actors inside the Nubus deployment in the Kubernetes cluster.

      Example:

      https://external-storage.example.com:9000

      If your S3-compatible object storage is in the same cluster, it’s enough to use the Kubernetes service name for the pod of the object storage.

      Example:

      https://s3-storage.kubernetes-namespace:9000

After you have configured the bucket and the folders in your S3-compatible object storage, you can continue with Configure deployment for using external S3-compatible object storage.

6.5.3. Configure deployment for using external S3-compatible object storage#

Before you can configure the external S3-compatible object storage, consider the Scenarios section and Prepare external S3-compatible object storage for Nubus. To use an external S3-compatible object storage for Nubus, go through the following steps:

  1. Deactivate the provisioning jobs for the S3-compatible object storage. Add minio.enabled and set it to false as shown in Listing 6.9 to your custom_values.yaml values file.

    Listing 6.9 Deactivate the provisioning jobs for MinIO#
    minio:
  enabled: false

  2. Deactivate the ingress for the internal MinIO. Add ingress.minio.enabled and set it to false as shown in Listing 6.10 to your custom_values.yaml values file.

    Listing 6.10 Deactivate ingress for the internal MinIO#
    ingress:
  minio:
    enabled: false

  3. Add the connection parameters for your external S3-compatible storage to your custom_values.yaml values file. The listing uses the following Helm Chart values:

    nubusPortalConsumer.objectStorage.bucketName

    Configures the name of the bucket in the object storage. The Portal Consumer needs to upload and write objects to this bucket.

    nubusPortalConsumer.objectStorage.endpoint

    Configures the S3-compatible API endpoint that the Portal Consumer uses to upload objects. The point of view is the service Portal Consumer inside the Kubernetes cluster.

    nubusPortalConsumer.portalConsumer.assetsBaseUrl

    Configures the URL prefix that the Portal Frontend and the Central Navigation API use. The asset base URL is part of the URL to the item from the bucket. The point of view is an actor outside the cluster, such as the user’s web browser.

    nubusPortalServer.objectStorage.bucketName

    Configures the name of the bucket in the object storage. The Portal Server uses it to read objects.

    nubusPortalServer.objectStorage.endpoint

    Configures the S3-compatible API endpoint. The Portal Server uses it to read objects. The point of view is the service Portal Server inside the Kubernetes cluster.

    In general, Listing 6.11 is a good start, where you configure the same bucketName for the Portal Consumer and the Portal Server, as well as, the endpoint. You need to configure the application components separately, as shown in Listing 6.12.

    Listing 6.11 Start configuration for object storage settings#
    nubusPortalConsumer.objectStorage.bucketName
   == nubusPortalServer.objectStorage.bucketName

nubusPortalConsumer.objectStorage.endpoint
   == nubusPortalServer.objectStorage.endpoint
    Listing 6.12 Configure parameters for external S3-compatible storage#
    nubusPortalConsumer:
  objectStorage:
    bucketName: "nubus"
    endpoint: "https://external-storage.example.com:9000"
  portalConsumer:
    assetsBaseUrl: "https://external-storage.example.com/assets-bucket/"

nubusPortalServer:
  objectStorage:
    endpoint: "https://external-storage.example.com:9000"
    bucketName: "nubus"

  4. Add the credentials to the S3-compatible object storage. Make sure to use the right user account for the right section. You can either configure the custom credentials, or you can refer to existing Kubernetes Secrets objects. See the following examples.

    Custom credential configuration

    Use the example in Listing 6.13, if you want to provide custom credentials. The listing uses the following Helm Chart values:

    Listing 6.13 Configure custom credentials for external S3-compatible storage#
    nubusPortalConsumer:
  objectStorage:
    auth:
      accessKey: "<access-key-for-portal-server-user>"
      secretKey: "<secret-key-for-portal-server-user>"

nubusPortalServer:
  objectStorage:
    auth:
      accessKey: "<access-key-for-portal-server-user>"
      secretKey: "<secret-key-for-portal-server-user>"
    Kubernetes secrets for credentials configuration

    Use the example in Listing 6.14, if you want to use existing Kubernetes Secrets. The listing uses the following Helm Chart values:

    Listing 6.14 Configure existing Kubernetes Secrets for external S3-compatible storage#
    nubusPortalConsumer:
  portalConsumer:
    objectStorageCredentialSecret:
      name: "name-of-secret-object-for-portal-consumer"
      accessKeyKey: "<key-in-secret-object-for-access-key>"
      secretKeyKey: "<key-in-secret-object-for-secret-key>"

nubusPortalServer:
  portalServer:
    objectStorageCredentialSecret:
      name: "name-of-secret-object-for-portal-server"
      accessKeyKey: "<key-in-secret-object-for-access-key>"
      secretKeyKey: "<key-in-secret-object-for-secret-key>"

  5. To apply the configuration, follow the steps in Apply configuration.

    After you configured the settings in your values file, you need to make them known to your Nubus deployment.

  6. To verify the configuration, you can retrieve a default icon. Use the value from nubusPortalConsumer.portalConsumer.assetsBaseUrl for the ASSETS_BASE_URL variable. Use the command in Listing 6.15.

    Listing 6.15 Example to verify the configuration through retrieving an icon from the bucket#
    $ export ASSETS_BASE_URL = "value from nubusPortalConsumer.portalConsumer.assetsBaseUrl"
$ curl -i "$ASSETS_BASE_URL"/icons/entries/login-saml.svg
HTTP/2 200
[...]

<svg xmlns="http://www.w3.org/2000/svg" ... />

See also

Secrets in Nubus for Kubernetes

for information about different options using secrets in Nubus for Kubernetes.

On this page