Securing SSH connections using an Identity Aware Proxy on GCP

In a series of blog posts we will focus on some of the best practices we use within Merapar to evolve our DevOps practices we have built around the GCP platform. Why? Because we think it’s fun to share knowledge and to learn from others in the industry.

Situation

Secure access to environments within the cloud is always a top priority for Merapar when deploying new projects. SSH is often still used, for example when developers need to access remote environments. We never want to maintain SSH keys, or even more costly, a VPN infrastructure for secure access. When using GCP, a combination of Identity Aware Proxy and temporary ssh credentials ensure us to achieve this.

Solution

Traditionally, we would deploy a bastion host / jump box with a public IP address. This would expose the instance publicly on the internet, and as such we will need to control the exposure of this instance on the public internet. When connecting through an Identity Aware Proxy, we do not have to provide the instance with a public IP address. Although good security practices are still preferred, it does limit the exposure of instances on the internet. Note that we do need to allow traffic from the IP ranges which Google uses to host the Identity Aware Proxy infrastructure into our VPC.

Network access for IAP

To provide access from the Identity Aware Proxy we should create a firewall rule which allows the applicable IP ranges access (35.235.240.0/20), see https://cloud.google.com/iap/docs/using-tcp-forwarding#create-firewall-rule Or the following code when using terraform:

resource "google_compute_firewall" "allow-iap-in" {
  description   = "Allow IAP ingress"
  direction     = "INGRESS"
  name          = "allow-iap-ingress"
  network       = <vpc>
  priority      = 100
  source_ranges = ["35.235.240.0/20"]

  allow {
    protocol = "tcp"
    ports    = ["22"]
  }
}

User controls for accessing the Identity Aware Proxy

By providing the role “roles/iap.tunnelResourceAccessor” we can control who has access to the identity aware proxy. This can be achieved by using the following terraform resource:

resource "google_project_iam_member" "iam_member" {
  role    = “roles/iap.tunnelResourceAccessor”
  member  = group:<my-group@my-org.com>
}

Before moving to the required permission at the instance level, let’s construct a compute instance which we can use as a bastion host. As per best practice, we will deploy the compute instance with custom service account.

resource "google_service_account" "bastion-sa" {
  account_id   = "bastion-sa"
}

resource "google_compute_instance" "bastion" {
  name         = "vm-bastion"
  machine_type = "e2-micro"
  zone         = "<location>-a"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }

  network_interface {
    network            = <vpc>
    subnetwork         = <subnetwork>
    subnetwork_project = <subnetwork_project>
  }

  metadata = {
    enable-oslogin = "TRUE"
  }

  service_account {
    # Google recommends custom service accounts that have cloud-platform scope and permissions granted via IAM Roles.
    email  = google_service_account.bastion-sa.email
    scopes = ["cloud-platform"]
  }
}

Next to providing the user control to the Identity Aware Proxy, we need to allow the user “roles/compute.instanceAdmin.v1” to the instance (at least on instance level) we can apply the following resources:

resource "google_compute_instance_iam_member" "bastion_access" {
  project       = google_compute_instance.bastion.project
  zone          = google_compute_instance.bastion.zone
  instance_name = google_compute_instance.bastion.name
  role          = "roles/compute.instanceAdmin.v1"
  member        = "group:<my-group@my-org.com>"
}

Now finally we can connect to our instance, tunnelled through an Identity Aware Proxy using the following command:

gcloud compute ssh vm-bastion --project <project name> --zone <location>-a --tunnel-through-iap

See https://cloud.google.com/sdk/gcloud/reference/compute/ssh for additional SSH options, and for example on how to use this as an SSH tunnel to your environment. For this to be effective, you might want to run a proxy on the instance. By using the combination of Identity Aware Proxy and bastion principle with temporary SSH credentials we ensure secure, auditable access to our bastion host. As our user accounts follow HR processes, we are ensured of no rogue employees having access to our resources.