Configuring OpenID Connect (OIDC) / SSO based authentication

How to set up OIDC / SSO for the Anka Build Cloud Controller UI.

Many organizations and developers are already familiar with OpenID Connect (OIDC). OIDC is a layer that sits on top of OAuth 2.0 and performs the authorization necessary to access protected resources, such as the Anka Build Cloud Controller. Let’s walk through what you need to know to set it up and protect your Controller Dashboard/UI.

Important
  • Requires an Anka Enterprise Plus license.
  • It currently only protects the UI/Dashboard and is not available for API or others types of protection.
  • You must have at least one node with a Enterprise or higher license joined to the Controller for these features to work.
  • Your Nodes will lose connection until you join them using the new credential.
  • We currently only support Code/Explicit Flow.

Usage Instructions

When using OIDC, you’ll need an Authorization Provider or Server. Most of our customers use Providers like Okta, Cyberark’s Idaptive, and others. we won’t get into the specifics for these tools as they often differ greatly. However we will go through several general things you need to know, regardless of provider.

Required Changes

Anka Build Cloud Controller & Registry

  1. Set ANKA_OIDC_CLIENT_SECRET to the Client Secret you generate at your provider application.
  2. Set ANKA_OIDC_PROVIDER_URL to the appropriate provider URL for oauth2. For example, in Okta, I would set this to https://dev-123456.okta.com/oauth2/default.
    Don’t know what URL to use for your provider? The provider url + /.well-known/openid-configuration must lead to the issuer’s OIDC config.
  3. Set ANKA_OIDC_CLIENT_ID to the client ID for the application.

At Your Provider

  1. Your provider config must allow a redirect to the Anka Controller at /oidc/v1/callback. The URL doesn’t need to be public, but must match the hostname or IP (and port) you use locally. As an example, in Okta, you’ll set Sign-in redirect URIs to https://anka.controller:8090/oidc/v1/callback.
  2. The controller/your browser will request certain scopes from the provider. These scopes have claims attached. By default, we look for openid, profile, and groups.
  3. Within the scopes, we look for claims. The following claims are required (by default): name (part of profile) & groups. These are changeable with ANKA_OIDC_GROUPS_CLAIM and ANKA_OIDC_USERNAME_CLAIM in your controller’s config. In version 1.29.0, we will also look for scopes using the values of those ENVs.
  4. Once the scopes are requested successfully, the data returned needs to be in a specific format (id_token & token). We make the request with response_type to ensure this.
  5. The groups claim is expected to be an array of strings, each correlating to a Controller permission group.

Here is an example config showing what it looks like for a user with Okta:

  anka-controller:
    container_name: anka-controller
    build:
       context: .
       dockerfile: anka-controller.docker
    ports:
       - "8090:80"
    volumes:
       - /Users/myUserName:/mnt/cert
    depends_on:
       - etcd
       - anka-registry
    restart: always
    environment:
      ANKA_REGISTRY_ADDR: "https://anka.registry:8089"
      ANKA_USE_HTTPS: "true"
      ANKA_SKIP_TLS_VERIFICATION: "false"
      ANKA_SERVER_CERT: "/mnt/cert/anka-controller-crt.pem"
      ANKA_SERVER_KEY: "/mnt/cert/anka-controller-key.pem"
      ANKA_CA_CERT: "/mnt/cert/anka-ca-crt.pem"
      ANKA_ENABLE_AUTH: "true"
      ANKA_ROOT_TOKEN: "1111111111"
      ANKA_OIDC_DISPLAY_NAME: "Okta SSO"
      ANKA_OIDC_PROVIDER_URL: "https://dev-1234567.okta.com/oauth2/default"
      ANKA_OIDC_CLIENT_ID: "0oa7a07mc0kQxyfrus11"
      ANKA_OIDC_CLIENT_SECRET: "aHWQYCbH0mTYwLwwIfBvT-JWotYQAR8HAn7glnSB"

  anka-registry:
    container_name: anka-registry
    build:
        context: .
        dockerfile: anka-registry.docker
    ports:
        - "8089:8089"
    restart: always
    volumes:
      - "/Library/Application Support/Veertu/Anka/registry:/mnt/vol"
      - /Users/myUser/:/mnt/cert
    environment:
      ANKA_USE_HTTPS: "true"
      ANKA_SKIP_TLS_VERIFICATION: "false"
      ANKA_SERVER_CERT: "/mnt/cert/anka-controller-crt.pem"
      ANKA_SERVER_KEY: "/mnt/cert/anka-controller-key.pem"
      ANKA_CA_CERT: "/mnt/cert/anka-ca-crt.pem"
      ANKA_ENABLE_AUTH: "true"
      ANKA_OIDC_DISPLAY_NAME: "Okta SSO"
      ANKA_OIDC_PROVIDER_URL: "https://dev-1234567.okta.com/oauth2/default"
      ANKA_OIDC_CLIENT_ID: "0oa7a07mc0kQxyfrus11"
      ANKA_OIDC_CLIENT_SECRET: "aHWQYCbH0mTYwLwwIfBvT-JWotYQAR8HAn7glnSB"
The OIDC ENVs must be set for both services.

After that, just docker-compose down -t 50 && docker-compose up -d and try accessing the Controller at its HTTPS URL or IP. If you did everything correctly (you enabled certificate authentication and joined your node right?), you should see a Log In box with two options: Login with Okta SSO and Login with superuser.

OpenID Login Buttons

Managing User/Group Permissions (Authorization)

You can then add the exact groups claim name from your SSO provider (or other authorization server software) to the controller’s permission management panel. This gives any users associated to the group in that cloud permission group the specific permissions to the API and even controller UI.

Authorization allows you to control access to specific actions/endpoints of the API and even specific Resources like Nodes and Templates in your Controller and Registry. It has four parts to it that are important to understand:

  • Component (Controller or Registry)
    • Groups
      • Permissions
      • Resources
        • Permissions

  1. Groups are the wrappers for all Permissions and Resources. You attach a Groups to a Authentication credential to enable certain access.

    • Group Permissions are given to allow a credential access to perform a specific action, like listing or creating a VM Instances. Because these control general access to the API to make calls, it overrides any Resource Permissions.

  2. Resources limit which Nodes and Templates the credential can control.

    • Resource Permissions are given to a Resource, further limiting the Group’s access per Resource. As an example, these Resource Permissions allow an admin to prevent a specific Group from deleting a Node from the cloud, yet allow changes to its config, and more. It’s critical to understand that if the Groups Permission “Instances > Start” is not checked, the Resource Permissions for the Group allowing Create Instance won’t matter as the credential has no access to make a Start VM API call.
    • Resource Management, which enables Resource control, is optional. If disabled, all Resources (and Resource Permissions) are available to all credentials.

After a Group is created, you’ll assign it to a specific credential. The credential can have one or more Groups attached and it’s important you consider how much access for each Group you need to provide for your use-case and security requirements.

In this guide we will start with the most common use-case of sharing Node between multiple teams (Groups), then explain how to isolate Nodes per team (similar to how the older Node Groups worked) later on.

Prerequisites

To enable Authorization features, you’ll need to ensure:

  1. Authentication is enabled by setting ANKA_ENABLE_AUTH and ANKA_ROOT_TOKEN ENVs in your Controller & Registry config. You’re also familiar with one of the existing Authentication methods and have one set up.

  2. Groups + Group Permissions are enabled by enabling Authorization. The following ENVs can be added to your config to achieve this:

    • ANKA_ENABLE_CONTROLLER_AUTHORIZATION (boolean) works for both combined (macOS) and standalone (docker) packages.
    • ANKA_ENABLE_REGISTRY_AUTHORIZATION (boolean) is for the macOS combined (controller + registry in one) package only.
    • ANKA_ENABLE_AUTHORIZATION (boolean) is only for the standalone (macOS or docker) registry packages.

    This should expose the https://<controller address>/#/permission-groups page in your Controller.

  3. Resource Permissions are enabled by setting ANKA_ENABLE_RESOURCE_MANAGEMENT to true in your configuration.

License Considerations: These features are only available for Enterprise tier licenses.

  • Enterprise license customers cannot control authorization and each authentication credential will have full access to the system.
  • However, Enterprise Plus customers will be able to use Permission Groups and Resources to have more fine-grained control over access.

Groups + Group Permissions

Do not confuse Node Groups with Groups/Group Permissions.

Under the Permissions section of the Controller UI (https://<controller address>/#/permission-groups), let’s create a group for our nodes to be able to connect.

  1. First choose the Controller Component from the drop down.

  1. Next, create a new group named node by clicking the [+] button.

  2. You can now target and add specific Permissions or Resources for the Group. Click the circular (💻) icon on the right to highlight what permission should be set for Nodes to communicate with the Controller. Then check/enable the highlighted permissions and click Save Permissions at the bottom of the page. Important: Finally, do the same but under the Registry Component.

Note: We’ll not be setting Resources right now.

  1. The group can now be attached to a specific Authentication Credential and the credential used to access and perform the permitted actions. For example, create a UAK and attach the group.
Be sure to download the key if you’re creating a new UAK.

You will now see the UAK in the list.

The node certificate now has permissions to perform the specifically set actions against all Resources (if Resource Management is disabled).

  1. You can now try joining the node to the Controller using the UAK and confirm it’s all joined by checking the Controller Nodes page, or the agent logs.
❯ sudo ankacluster join http://anka.controller:8090 --api-key-file ~/node.cer --api-key-id "node"
Testing connection to the controller...: Ok
Testing connection to the registry...: Ok
Success!
Anka Cloud Cluster join success

Resources + Resource Permissions

Node Groups are disabled while Resource Management is enabled.

The Resource Management/Permissions feature is enabled by setting ANKA_ENABLE_RESOURCE_MANAGEMENT to true in your configuration. Once enabled, it unlocks the Resources tab under /permission-groups.

Within the Resources tab you will be able to, depending on the Component selected, add specific Nodes or Template Resources to the Group. This limits the Group to certain actions that can be performed against those Resources.

An example of this is allowing the iOS team/group to distribute specific Templates to specific Nodes, but not create VM Instances or delete the Node.

In the previous section on Group Permissions, we joined a Node using a credential with the Group node attached. This Group only has Permissions to perform the minimum required actions to run as an Anka Node and communicate with the Controller. We did not add Resources to it though (even though we technically could) so we can instead have team specific Groups and credentials.

In this example, each team gets only their Node or Template Resources assigned to their Group, with no ability to make API calls to Components/Permissions. Instead, other non-team Groups handle the ability to call certain endpoints and perform certain actions to the Controller and Registry. Individual groups (team and non-team related) are then assigned to certain credential, combining the Permissions and Resources.

It’s important to understand that a single credential, like a UAK or a Certificate, should only ever be used by a single user or client. You wouldn’t ever want to share the Node credential with a team for example. Create a second credential for that team, then in order for the Node to be able to access the team’s credential, add the team’s Group to the Node credential.

  1. Create two groups: ios and instance-control.

  2. Under the ios Group > Resources, add the Template Resource you want this team to access. The bare minimum permissions are seen in the image below.

  1. Under the instance-control Group > Permissions, add all of the Instances Permissions like in the image below.

  1. Now create a new credential (not Group) named service-user and attach those two groups. We’ll use UAKs for this example. There is no need for the service-user to have its own group as it will get the VM start and terminate permissions from instance-control group and Template/Node permissions from the ios group.

  2. Attach the group ios to the node credential we created in the previous section and joined our node with so that that it can also collect information about the Template in order to start the VM Instance properly (it checks the download size of the template before pulling). Your UAK setup should look something like this image.

We can now use the service-user credential in your CI/CD tools to communicate with the Controller when an ios team member triggers a job. When the service-user credential makes an API call to start a VM Instance, it will pass in the ios group. That ios group has access to the template being targeted and is required.

Node Groups

The method described above works well for sharing the same nodes amongst all teams in an organization. But what if you want to isolate specific nodes to specific teams? Node Groups are disabled when Resource Management is enabled, but that should make sense to you by now as Node access/permissions are now bound to a specific Group + a specific credential. There are are several other scenarios possible which we’ll detail below.

You cannot have a shared Node credential and also limit by Node Resources for specific teams.
Scenario 1: Team Specific Anka Nodes

This configuration allows isolating certain Nodes to certain teams. The service-user credential is necessary to make API calls. In the Controller > Instances page or the API the team member can now start VM Instances with the group team1 (or 2, 3) and it will only start on the Nodes assigned to the team’s group. This is due to the Resource Permission for the Node being attached to the team group.

High Level Overview
  • Three Anka Nodes joined to the Controller, each with separate credentials.
  • Three Node credentials used to join:
    • UAK: team1-nodes - Groups attached: node,team1
    • UAK: team2-nodes - Groups attached: node,team2
    • UAK: team3-nodes - Groups attached: node,team3
  • Three “service user” credentials for teams to make API calls:
    • UAK: team1-su | Groups attached: service-user,team1
    • UAK: team2-su - Groups attached: service-user,team2
    • UAK: team3-su - Groups attached: service-user,team3
  • Group node has
    • Permissions: Recommended Node permissions (see UI).
    • Resources: None.
  • Group service_user has
    • Permissions: Instances, Nodes, and Distribute.
    • Resources: None.
  • Group team1/team2/team3 has
    • Permissions: None.
    • Resources: The specific Node (and other Resources) for the team.
Scenario 2: Shared Anka Nodes / Teams limited by VM Template Resource Permissions

This configuration allows all teams to share all Anka Node capacity, but only be able to start specific VMs from specific Templates Resources assigned to their team’s Permission Group.

High Level Overview
  • Three Anka Nodes joined to the Controller, each with the same credential.
  • Three Node credentials used to join:
    • UAK: nodes - Groups attached: node,team1,team2,team3
    • UAK: nodes - Groups attached: node,team1,team2,team3
    • UAK: nodes - Groups attached: node,team1,team2,team3
  • Three “service user” credentials for teams to make API calls:
    • UAK: team1-su | Groups attached: service-user,team1
    • UAK: team2-su - Groups attached: service-user,team2
    • UAK: team3-su - Groups attached: service-user,team3
  • Group node has
    • Permissions: Recommended Node permissions (see UI).
    • Resources: None.
  • Group service_user has
    • Permissions: Instances, Nodes, and Distribute.
    • Resources: None.
  • Group team1/team2/team3 has
    • Permissions: None.
    • Resources: The specific Template Resources for that team. No Nodes.
Scenario 3: Team Specific Anka Nodes + Dynamic Nodes

This configuration allows teams to have specific nodes guaranteed, then admins to increase capacity by assigning the specific team’s group to the nodes.

High Level Overview
  • Four Anka Nodes joined to the Controller, each with separate credentials. Three get assign to specific teams, but the fourth can dynamically change to provide extra capacity to specific teams when needed.
  • Four Node credentials used to join:
    • UAK: team1-nodes - Groups attached: node,team1
    • UAK: team2-nodes - Groups attached: node,team2
    • UAK: team3-nodes - Groups attached: node,team3
    • UAK: dynamic-nodes - Groups attached: node
      • Dynamically update this UAK’s groups to add any team or teams that need the extra capacity
  • Three “service user” credentials for teams to make API calls:
    • UAK: team1-su | Groups attached: service-user,team1
    • UAK: team2-su - Groups attached: service-user,team2
    • UAK: team3-su - Groups attached: service-user,team3
  • Group node has
    • Permissions: Recommended Node permissions (see UI).
    • Resources: None.
  • Group service_user has
    • Permissions: Instances, Nodes, and Distribute.
    • Resources: None.
  • Group team1/team2/team3 has
    • Permissions: None.
    • Resources: The specific Node (and other Resources) for the team.
Answers to Frequently Asked Questions
  • The Nodes joined to the Controller must have Permissions to access the Template being used to start a VM. This mean at least one of the groups attached to a node credential must have the template resource attached.
  • Node Groups differ from Permissions Groups and are disabled when this feature is enabled.
  • Save Image requests can only target Instances that belong to a group the user has access to.

Once you’ve added all of the proper permissions, you can now go back to the main Controller page and log out of the superuser. You can now choose Login with Okta SSO, which will redirect you to your Okta to have you log in with your user. You will then be taken to the Controller UI and be logged in as that user.

Final Notes

  • UI sessions will have the same length as the OIDC Access Token’s lifetime. Your provider can set this value lower if needed.