From zero to live GCP benchmarks with SiliconBoutique

Most cloud setup guides start at the point where you already have a project, a billing account, a service account, and somehow already understand Workload Identity Federation. This one doesn’t. If you’ve never opened the Google Cloud console and you want SiliconBoutique running fully automated benchmark jobs on GCP by the end, this is the guide. Every command is spelled out, every IAM concept is explained before you’re asked to use it, and by the last section you’ll have a workflow that provisions a GKE cluster, collects performance metrics, and stores results in BigQuery without a single manual step.

Before touching any command, it helps to see the full picture. There are five distinct setup stages, and each one builds on the previous. Skipping ahead typically means backtracking.

Stages 1 through 4 are one-time setup steps. Stage 5 provisions the durable BigQuery table and only needs to run once unless you want to change the dataset location or table schema. Stage 6 is what you repeat every time you want a benchmark run.

Go to cloud.google.com and click Get started for free. Google offers a 90-day free trial with $300 in credits, which is more than enough to run several benchmarks on spot VMs.

You need a Google account to sign up. If you already have a Gmail account, use that. During registration, Google asks for a credit card to verify your identity. You won’t be charged during the free trial, and you won’t be charged after it ends unless you explicitly upgrade to a paid account. Keep that in mind before you enter your card details.

Once registration completes, you land in the Google Cloud console. The URL looks like console.cloud.google.com. Keep this tab open throughout the setup.

Google Cloud organizes everything inside projects. A project is the billing and permissions boundary for all the resources you create. SiliconBoutique needs one project to hold the ephemeral GKE benchmark clusters and the durable BigQuery history table.

In the console, click the project selector dropdown at the top of the page. It might say “Select a project” or show an existing project name.

Important: Google Cloud may have automatically created a default project for you during account setup. If you see an existing project listed, inspect it first. If you’re comfortable using that auto-created project (most users are), note its Project ID and skip to the next section. Otherwise, proceed to create a new one.

To create a new project, click New Project.

Fill in the fields:

• Project name — something like silicon-boutique or my-benchmarks. This is just a display name.
• Project ID — GCP auto-generates one from your project name. You can edit it, but once you create the project the ID is permanent. Choose something memorable because you’ll type it in many commands. A format like silicon-boutique-yourname-2026 works well. The ID must be 6 to 30 characters, lowercase letters, numbers, and hyphens only, and must start with a letter.
• Organization — leave as “No organization” if you’re working from a personal account.

Click Create. After a few seconds the console switches into your chosen project.

Write down your project ID. You’ll need it frequently.

APIs and cluster creation won’t work until billing is enabled on the project, even during the free trial.

In the console, open the navigation menu (the three horizontal lines in the top left), go to Billing, then click Link a billing account. If you just created your account, you’ll have one billing account available from registration. Select it and click Set account.

You can confirm billing is active by running this command after installing gcloud in the next stage.

The field billingEnabled should show true.

Google Cloud APIs are disabled by default on a new project. SiliconBoutique uses seven of them. Enable them all now so you don’t hit “API not enabled” errors mid-setup.

When searching for APIs in the Google Cloud console, always look for the full service name listed below. The search interface may return multiple results with similar names—use the API ID (the googleapis.com identifier) to confirm you’re enabling the correct one.

To enable via console: Go to APIs & Services > Library. For each API above, search by the API ID (e.g., compute.googleapis.com). The search results may show several options—select the one whose full name matches the table above, then click Enable.

To enable via command line: Use the single command below once gcloud is installed (easier and less error-prone).

The gcloud CLI is how you manage GCP resources from your terminal or the devcontainer. If you’re working inside the SiliconBoutique devcontainer, the Google Cloud SDK is already installed. Verify it with:

If you’re working on your own machine, follow the gcloud CLI installation guide for your OS. On macOS it’s a package download. On Linux, a curl script handles it.

After installation, initialize the CLI and log in:

The init command walks you through logging in with your Google account and selecting your project. When it asks you to choose a project, pick the one you created (or validated) in Stage 1. When it asks about a default compute region, enter us-central1.

After init completes, run one more login command that provides application default credentials. Terraform uses these when you run it locally.

A browser window opens and asks you to authorize the SDK. Click Allow. This writes a credentials file to ~/.config/gcloud/application_default_credentials.json.

Now enable all the required APIs in a single command. Replace YOUR_PROJECT_ID with your actual project ID.

This takes about 30 seconds. You’ll see each API listed as it enables. If you get an error about billing not being enabled, go back and complete the billing setup from Stage 1 first.

A service account is an identity that a program uses to authenticate to Google Cloud APIs. Unlike a human account, it doesn’t have a password or a browser login. Instead, programs prove their identity by holding a short-lived token issued by Google. The GitHub Actions workflow acts as this program during benchmark runs.

The service account you create here will be granted exactly the permissions SiliconBoutique needs, nothing more.

This creates the account. Its full email address will be silicon-boutique-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com. You’ll use this email in several commands, so store it in a variable to avoid typing it repeatedly.

The service account needs four roles to run benchmarks and load results into BigQuery.

Compute Admin lets the account create and delete GKE clusters, node pools, VPCs, and subnets during each run.

Kubernetes Engine Admin grants control over GKE operations.

Service Account User allows the account to act as itself when creating clusters.

BigQuery Job User allows it to submit queries and load jobs.

Verify the bindings with:

You should see four role entries.

In a traditional setup, you’d create a long-lived key for the service account and store it as a GitHub secret. Every time the workflow runs, it uses that key to authenticate. Keys are static credentials, and static credentials are a security liability—if someone finds the key, they can impersonate the service account forever.

Workload Identity Federation (WIF) is a better approach. Instead of storing a key, GitHub Actions gets a short-lived token from Google that’s valid for exactly one workflow run. The token is tied to a specific GitHub repository, branch, and workflow, and expires in hours. This reduces the blast radius if a token is leaked.

A Workload Identity Provider is the bridge. It tells Google: “I trust tokens from GitHub Actions in this repository, issued for this GitHub repository context. Exchange them for short-lived GCP service account tokens.”

Create the workload identity pool first:

Now create the provider inside that pool. This is where you specify which GitHub repository can authenticate.

Replace:

• GITHUB_OWNER with your GitHub username (e.g., octocat).
• GITHUB_REPO with your repository name (e.g., silicon-boutique).

The attribute condition is case-sensitive. It must match your GitHub URL exactly, in lowercase, in the format owner/repo.

Retrieve the full provider resource name (you’ll need this for GitHub secrets):

Save this output. It looks like projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/github-pool/providers/github-provider. You’ll need it in Stage 6.

The provider can now validate GitHub tokens, but it can’t use them to impersonate the service account yet. Grant that permission:

Replace GITHUB_OWNER/GITHUB_REPO with the same owner and repo name from above.

Verify the binding:

You should see one binding for principalSet://goog/workload-identity-pool/....

This stage creates the BigQuery dataset and benchmark summary table that stores your results. It uses Terraform to define the resources as code so they can be versioned and recreated if needed.

Before running Terraform, create a secure environment file to manage your credentials. This is a best practice that keeps sensitive values out of shell history and makes scripts more portable.

In the root of the SiliconBoutique repository, create a file named credential.env:

Replace the values with your actual project ID and service account email (from Stage 3).

Important: Add credential.env to your .gitignore to prevent accidental commits:

Before running Terraform, load your environment file. This approach keeps credentials out of your shell history:

Now initialize Terraform:

Terraform downloads the Google Cloud provider and prepares the workspace. This completes quickly.

Validate the configuration:

If validation passes, apply the configuration to create the BigQuery dataset and table:

This command does three things:

1. Creates the BigQuery dataset named silicon_boutique in the US region.
2. Creates the benchmark summary table named benchmark_summaries with the schema defined in the Terraform module.
3. Grants your service account two permissions on the dataset:
   • roles/bigquery.jobUser at the project level (permission to run queries and load jobs).
   • roles/bigquery.dataEditor on the dataset (permission to create, read, and delete tables).

The second grant is critical. It allows the workflow to create and delete the scratch table used during the BigQuery preflight check (a step that verifies write access before attempting to load the full results). Without this grant, the workflow will fail with a “Permission denied” error when trying to create the preflight table.

Terraform outputs the created resource names:

Verify the grants were applied correctly:

You should see your service account listed with EDITOR role in the output.

If you see an error like Permission denied on dataset silicon_boutique, it means the Terraform summary_writer_service_accounts variable wasn’t applied correctly with your service account email. This often happens if:

• The service account email is missing or malformed.
• The Terraform was applied without the service account variable in a previous run.

Fix: Re-run the apply command with the correct service account email:

After apply completes, verify the bindings again with bq show.

The workflow needs three secrets to authenticate and know which GCP project to use. Go to your GitHub repository settings, then Secrets and variables > Actions.

Create three secrets:

1. GCP_PROJECT_ID— the value from Stage 1 (your project ID, e.g., silicon-boutique-yourname-2026).

2. GCP_SERVICE_ACCOUNT— the email from Stage 3 (e.g., silicon-boutique-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com). Do not include the serviceAccount: prefix.

3. GCP_WORKLOAD_IDENTITY_PROVIDER— the full resource name from Stage 4. It looks like:

   projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/github-pool/providers/github-provider

Paste the exact output from gcloud iam workload-identity-pools providers describe. Do not edit or abbreviate it.

In your GitHub repository, go to Actions. You should see the SiliconBoutique workflow listed. Click on it, then click Run workflow. GitHub shows a dropdown menu—leave it on the default branch and click the green Run workflow button.

The workflow begins immediately. Watch the job logs in real time by clicking the job name.

The workflow has six phases, each with multiple steps. Here’s what to expect:

If any step fails, the teardown phase still runs because all cleanup steps use if: always(). The workflow fails the job at the end if Terraform destroy reported a non-zero exit code. This prevents orphaned GKE clusters from accumulating costs.

While the job is running, you can watch the GKE cluster appear in the Google Cloud console. Go to Kubernetes Engine > Clusters. You’ll see a cluster named sb-gha-{run-id} appear during the provision phase and disappear after teardown completes.

The VPC and subnet are visible under VPC network > VPC networks, named sb-{run-id}-vpc and sb-{run-id}-subnet.

All of these resources carry the run_id label set by Terraform. If a run fails mid-way and teardown doesn’t complete, you can find the orphaned resources with:

Delete them manually with the values from terraform output teardown_check_commands if needed.

After the workflow completes successfully, the benchmark row is in BigQuery. Query it from the console or the bq CLI.

A complete run produces a summary_status of complete. A partial status means something was missing from the metric extraction, which is usually caused by a very short benchmark window or a Prometheus warmup issue.

You can also download the full benchmark-summary.json from the workflow artifacts. In the GitHub Actions UI, open the completed run and look for the artifact named benchmark-gha-{run_id}-1. It contains the summary JSON, the raw Prometheus metrics, load-generator stats, comparability report, BigQuery load report, teardown logs, and the acceptance demo report if you enabled it.

Once you have more than one benchmark row in BigQuery, you can compare them directly.

Or use the generate_comparison_report.py script to get a ranked Markdown table:

A few things go wrong regularly on first setup. Here’s what to check before filing a bug.

The workflow fails at the “Authenticate to Google Cloud” step

This almost always means the Workload Identity Provider resource name in the GCP_WORKLOAD_IDENTITY_PROVIDER secret is wrong. The value must include the full projects/PROJECT_NUMBER/... path, not just the provider ID. Re-run the gcloud iam workload-identity-pools providers describe command from Stage 4 and copy the exact output.

Also confirm the attribute-condition in the provider matches your actual repository name. The assertion.repository claim uses the format owner/repo, lowercase, exactly as it appears in the GitHub URL.

Terraform apply fails with “required APIs are not enabled”

Run the gcloud services enable command from Stage 2 again. Sometimes one API takes longer to propagate than expected. Wait 60 seconds and retry the workflow.

The node pool creation fails with a quota error

GCP enforces per-region CPU quotas on new accounts. For a single e2-standard-4 node, the default quota is usually sufficient, but spot VM quotas are separate from on-demand quotas. If you hit a quota error, either switch pricing_model to on_demand for the first run, or request a quota increase from the GCP console under IAM & Admin > Quotas.

The BigQuery load step fails with “Access Denied” or “Permission denied”

This means the table-level permissions from Stage 5 weren’t applied correctly. The error message typically looks like:

This happens when the BigQuery Terraform root hasn’t been applied with your service account email, or the summary_writer_service_accounts variable is missing or malformed.

Fix: Re-apply the BigQuery Terraform with the correct service account:

This Terraform root grants two permissions to each service account in the list:

• roles/bigquery.jobUser at the project level (permission to run load jobs).
• roles/bigquery.dataEditor on the dataset (permission to create, read, and delete tables).

The second grant is critical for the preflight write probe step, which creates and deletes a scratch table to verify write access before loading the full results.

After apply completes, verify the grants were applied:

You should see your service account email listed with EDITOR role. Then re-trigger the workflow.

Terraform destroy fails and the job shows red

The workflow is designed to fail the job when destroy doesn’t succeed. Check the teardown-destroy.log file in the uploaded artifacts to see the actual error. Common causes are IAM permission gaps on specific resource types and GKE node pool deletion timeouts. The Terraform state file is preserved in the job workspace, so you can inspect and retry destroy from the workflow logs rather than hunting for orphaned resources manually.

Once your first benchmark completes and the results land in BigQuery, you have several options:

• Run benchmarks on different machine types by editing the workflow dispatch inputs.
• Schedule automated runs on a cron schedule by adding a schedule trigger to the workflow.
• Compare multiple runs using the generate_comparison_report.py script.
• Export results to Google Sheets or a dashboard using BigQuery’s export tools.

For questions or issues beyond this guide, check the SiliconBoutique repository issues or documentation.