Okctl reference app

Guide from zero to running with a reference application

This guide will give you a complete example of how to set up a reference application in an Okctl environment. It can be useful
to see how various parts are configured together and help gain a better understanding of how you configure your own application.

Following this guide will set up:

  • A running reference application in a new or existing cluster
  • Postgres database for the application (RDS)
  • A Persistent volume claim (PVC) file system for the application

Get your cluster ready

Use an existing cluster or, set up a new cluster.

For simplicity, this guide will assume that your cluster declaration file is simply named cluster.yaml. Edit any commands that reference this file where appropriate

If setting up a new cluster, these are the lines you need to change. If using an existing cluster, make sure it has a database.
Databases set up with an Okctl cluster is a list. So if you already have a database in your cluster declaration, just add okctlreference to the list, as shown below.

  accountID: '123456789123' #set to your own AWS account id
...
  name: my-cluster-name # The name of your cluster, IE okctl-reference-dev
...
clusterRootDomain: my-cluster-name.oslo.systems # your oslo.system sub-domain
...
   repository: my_iac_repo_name #your iac repo name IE okctl-reference-iac
...
users:
- email: user.email@emailprovider.org #your email
  # name, namespace and username do not have to be the same, but it should be the same as you chose for
  # production, since it will make the configuration much more straight-forward later
...
databases:
  postgres:  
  - name: okctlreference
    namespace: okctlreference
    user: okctlreference

NB: Remember to re-runokctl apply cluster -f cluster.yaml if you added a new database or did other changes to your cluster declaration file.

Add application to cluster

We are going to apply the okctl reference app. You can also have a look at the reference IAC repository and the application running in our cluster, but this is not required to continue this guide.

Copy the YAML below into a file refapp.yaml in the root of your IAC-repository. Alternatively, use okctl scaffold application > refapp.yaml and edit values manually.

apiVersion: okctl.io/v1alpha1
kind: Application

metadata:
  # A name that identifies your app
  name: okctl-reference-app
  # The Kubernetes namespace where your app will live
  namespace: reference

# The Docker image containing the application. image.uri and image.name is mutually exclusive. Either specify the URI or
# define a name of an ECR repository for which okctl will create for you.
image:
  # uri defines where the image can be pulled from
  uri: ghcr.io/oslokommune/okctl-reference-app:v0.0.36

# The subdomain of the URL your app should be available on
# Example in a cluster with okctl-reference.oslo.systems as root cluster URL (as defined by primary DNS zone in the
# cluster declaration):
#
# subDomain: okctl
# result: okctl.okctl-reference.oslo.systems
# Comment this out to avoid setting up an ingress, in other words - avoid exposing it on the internet
#
subDomain: app

# The port your app listens on
# Comment this out to avoid setting up a service (required if url is specified)
#
port: 8080

# Enable prometheus scraping of metrics
prometheus:
  path: /metrics

# Enable integration with a Postgres database
postgres: okctlreference

# Volumes to mount
volumes:
  - /okctl/reference/storage: # Requests 1Gi by default

When you have refapp.yaml file enter a virtual environment for your cluster:

okctl venv -c cluster.yaml

Apply the reference application:

okctl apply application -f refapp.yaml

After applying the application configuration, follow the instructions from the output of okctl apply:

  • Add to Git the new files created by okctl in the infrastructure directory
  • Commit and push the changes to your IAC-repository
  • Wait up to 5 minutes for the routing to configure in ArgoCD

You should then be able to access the reference application at https://app.my-cluster-name.oslo.systems/ (replace cluster root domain and app subdomain with the correct values from your cluster and application configuration above).

Mount a volume in the application

By mounting a volume, the application can write to and read from the file system. The volume is by default only available to the root user. We therefore add the fsGroup to make the volume writeable to the application user (specified in the application Dockerfile). The reference application also expects the PVC_PATH environment variable to point to the path of the file to read and write.

Add the following to the configuration file <iac-repo>/infrastructure/applications/okctl-reference-app/base/deployment.yaml under the second nested spec element:

spec:
  ...
  template:
    ...
    spec:
      securityContext:
        fsGroup: 1001
      containers:
      ...
        env:
          - name: PVC_PATH
            value: "/okctl/reference/storage/myfile.txt" # Normally you would link a directory, but we only use one file in this example

Setup application user in the database

Create a temporary secret directory in your IAC repo, add it to gitignore (so you don't accidentally push it). Finally, generate your new password in a file. You can delete this when you are done, but you will need it in the later steps of this guide.

mkdir secret && printf "\nsecret/*\n" >> .gitignore && \
    uuidgen | sed 's/-//g' > secret/password.secret

Go to AWS console -> Systems manager -> Parameter store - > Click Create parameter

  • Name : /okctl/<cluster-name>/okctl-reference-app/db_password
  • Description : Database password for app user
  • Set type to 'Secure string'
  • Copy content from the secret/password.secret file generated in the first step into the Value box
  • Click Create parameter

Create an external secret in your IAC repo

  • Create file <iac-repo>/infrastructure/applications/okctl-reference-app/overlays/<cluster-name>/postgres-external-secret.yaml
  • Add the following code to it, remember to change cluster name, and region if applicable:
apiVersion: 'kubernetes-client.io/v1'
kind: ExternalSecret
metadata:
  name: postgres
spec:
  region: eu-west-1
  backendType: systemManager
  data:
    - key: /okctl/<cluster-name>/okctl-reference-app/db_password
      name: db_password
  • Add a line to the file <iac-repo>/infrastructure/applications/okctl-reference-app/overlays/<cluster-name>/kustomization.yaml, so the last three lines look like the example below:
resources:
- ../../base
- postgres-external-secret.yaml
  • Git commit and push

Forward Postgres to your local machine, and create a new user

  • First, create a password file:
echo $(uuidgen | sed 's/-//g') > secret/tmp.password
  • Now forward Postgres: Use your own cluster definition file (-c), and the name of your database -n, the username (-u) should not exist in the database already:
okctl forward postgres -c cluster.yaml -n okctlreference \
    -u tempuser -p secret/tmp.password
  • In a separate window, run the following script (copy and paste the whole thing) to generate SQL we will use to insert into your database
password=$(cat secret/password.secret) && \
    read -p "Enter appuser name [appuser]: " username && \
    username=${username:-appuser} && \
    cat << EOF > secret/newuser.sql
-- Same as create role with option 'login', and set a password
create user "$username" with password '$password';

-- this depends on app usage
alter role "$username" with inherit; -- note, inherit means inherit privileges, not role stuff (createrole, createdb, etc)
alter role "$username" with createrole;
alter role "$username" with nocreatedb;

-- this depends on app usage
grant all privileges on all tables in schema public to "$username";
EOF
  • Add the user to your database, make any changes to sql if applicable

NOTE: You need to have Postgres client (psql) installed on local machine for this script to work.

read -p "Enter database name [okctlreference]: " database && \
    database=${database:-okctlreference} && \
    export PGPASSWORD=$(cat secret/tmp.password) && \
    psql -h localhost -U tempuser $database < secret/newuser.sql
  • Stop the Postgres forwarding by e.g. Ctrl+C in your original command window.
  • Finally, clean up the secrets you have temporarily stored on your machine
rm -rf secret

Connect application to the database

Add the following to <iac-repo>/infrastructure/applications/okctl-reference-app/base/deployment.yaml:

      containers:
      ...
        env:
          - name: PVC_PATH
            value: "/okctl/reference/storage/myfile.txt" # From above
          - name: DB_NAME
            value: okctlreference
          - name: DB_USERNAME
            value: appuser

View example in context

Replace content in infrastructure/applications/okctl-reference-app/overlays/<cluster-name>/deployment-patch.json with the code-block below. You need to use the appropriate value for DB_ENDPOINT to your own database, which can be found under AWS console -> RDS -> DB instances - > [your database]:

[
    {
        "op": "add",
        "path": "/spec/template/spec/containers/0/image",
        "value": "ghcr.io/oslokommune/okctl-reference-app:v0.0.36"
    },
    {
        "op": "add",
        "path": "/spec/template/spec/containers/0/env/3",
        "value": {
            "name": "DB_PASSWORD",
            "valueFrom": {
                "secretKeyRef": {
                    "name": "postgres",
                    "key": "db_password"
                }
            }
        }
    },
    {
        "op": "add",
        "path": "/spec/template/spec/containers/0/env/4",
        "value": {
            "name": "DB_ENDPOINT",
            "value": "okctl-reference-dev-okctlreference.c7d1uu67i7fm.eu-west-1.rds.amazonaws.com"

        }
    }
]

Also, pay attention to the number at the end of each env variable in the path. Since env is one array, you need to start counting from after common env variables defined in deployment.yaml.
Since deployment.yaml specifies 3 env variables (index 0, 1 , 2) we start counting from 3.

Deploy the changes

Finally, in your IAC repository:

git commit -m "Setup environment for reference app" && git push

Once ArgoCD has redeployed the reference application, you can test out writing and reading entries to and from the database and file system from the application.

This concludes this guide, good luck!