Securing internal applications with IAP on GKE and Terraform

, , , , ,

What’s the fuzz ?

As a devops, I find myself from time to time deploying application for internal use (surprising i know) inside our GKE cluster. Those apps are generally added to provide a better experience for developers, for example, deploying the openapi schema of their services with redoc.
Sadly for us redoc doesn’t provide authentication by default, meaning that the documentation would be open on the internet if we would deploy it as-is, which isn’t a good thing.

Enter a feature of GCP called Identiy Aware Proxy, here their marketing schema on how that works:

iap schema

Put into words, we are just adding a proxy in front of our application that will authenticate (with their Google Account) any users that try to connect to it.

You could have used …

Someone might stop me right there and tell that i could have used:

  • A ip whitelist: Yes but as it’s pandemic time nowadays so everyone is working at home, i don’t want to maintain the ip that each developer use.
  • A VPN; Indeed but within a small company (~6 developers) you don’t want to require everyone to have it running all the time and also put a few hours to setting it up (by that i meant assist every dev to setup it on their laptop).
  • Use any proxy (ex: oauth2 proxy): again you are right i could used that but instead of waiting time to setup a custom proxy, i will just use the tool that GCP gives me and saves that precious time.

Using IAP works fine and integrate with the permission system of my gcp project, that tick all the boxes for me.

Terraforming time

As a fellow adept of Hype Driven Development, i picked Terraform to do Infrastructure As Code so i will show you how to setup IAP using Terraform.

NOTE: As the time of writing (4th April 2020), you must use the version v3.15.0 (and above of course) of the google-beta provider.

Okay so from the documentation, i should setup a OAuth Consent Screen, which in terraform translate to the google_iap_brand resource:

provider "google-beta" {
  credentials = var.gcp_credentials
  project     = var.project
  region      = var.region
}

data "google_client_config" "current" {
  provider = google-beta
}

resource "google_iap_brand" "iap_brand" {
  provider          = google-beta

  support_email     = data.google_client_openid_userinfo.current_identity.email
  application_title = "OAuth Tooling"
}

If i read correctly the next step is to setup access for my users, for which i will use the google_iap_web_iam_member (there are other ways to configure that):

resource "google_iap_web_iam_member" "access_iap_policy" {
  provider  = google-beta

  project   = var.project
  role      = "roles/iap.httpsResourceAccessor"
  member    = "domain:reelevant.com"
}

NOTE: Defining project is currently mandatory due to a bug.
NOTE 2: I’m directly giving access to my whole company domain (using domain: prefix but you can be more precise if you want to)

Next task is actually setting up an OAuth Application which in terraform is a google_iap_client:

resource "google_iap_client" "iap_redoc_client" {
  provider      = google-beta
  display_name  = "Redoc Auth"
  brand         =  google_iap_brand.iap_brand.name
}

Few, quite hard right ? Now i need to inject a secret in my Kubernetes cluster (i will skip on how to setup the kubernetes provider) which will contains the OAuth credentials of my app:

resource "kubernetes_namespace" "redoc_namespace" {
  metadata {
    name      = "redoc"
  }
}

resource "kubernetes_secret" "iap_redoc_client_k8s_secret" {
  metadata {
    name      = "redoc-iap-secrets"
    namespace = "redoc"
  }

  data = {
    "client_secret": google_iap_client.iap_redoc_client.secret
    "client_id": google_iap_client.iap_redoc_client.client_id
  }
  depends_on = [ kubernetes_namespace.redoc_namespace ]
}

And we are done for Terraform ! If you are using Terraform to deploy your application (please contact me because i have a few questions on how you are doing it) you might as well write the last bit with terraform as well.

Yaml Engineering

The last step is to configure a BackendConfig CRD and tell your Service to use it, which in plain yaml translates to:

apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
  name: redoc-backend-config
  namespace: backend
spec:
  iap:
    enabled: true
    oauthclientCredentials:
      secretName: komiser-iap-secrets
---
kind: Service
apiVersion: v1
metadata:
  name: api-redoc-service
  namespace: redoc
  annotations:
    beta.cloud.google.com/backend-config: '{"default": "redoc-backend-config"}'
spec:
  selector:
    app: api-redoc
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: NodePort

For the curious, here are the deployment definition (quite simple actually):

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: redoc
  name: api-redoc
spec:
  selector:
    matchLabels:
      app: api-redoc
  template:
    metadata:
      labels:
        app: api-redoc
    spec:
      containers:
      - name: api-redoc
        image: redocly/redoc:latest
        resources:
          {{- toYaml .Values.resources | nindent 10 }}
        env:
          - name: SPEC_URL
            value: "https://api-service.default.svc.cluster.local/openapi"
          - name: PAGE_TITLE
            value: "API OpenAPI"
        ports:
        - containerPort: 80

The only thing that you need to configure is the SPEC_URL, in our case each service is exposing its schema on the /openapi path so we just point to the application service.

Conclusion

As i’ve shown above, securing your internal application is quite straightforward (even if you don’t use Terraform, doing it by hand is literally 5min of copy/pasting) so i would advise everyone to do that for security sake !

Thats conclude my first article, don’t hesitate to reach me if you have any questions (contact at vmarchaud dot fr) and please give any feedback you may have about my writings, i’m really trying to progress on that !