Enhanced cluster management
Introduction
Managing multiple Kubernetes clusters across different customer environments presents significant operational challenges. The Enhanced Cluster Management feature was introduced to streamline this process, enabling Gitpod to provide superior support while maintaining strict security boundaries and full transparency.
Key Benefits
- Reduced resolution time for technical incidents
- Full visibility into cluster operations and security boundaries
- Decreased operational overhead for your organization
Opting Out
While we recommend keeping Enhanced Cluster Management enabled, you can choose to disable this feature. Before doing so, please note:
- Incident resolution may take longer
- Your organization may be required to do manual intervention
- Support processes may require additional steps
To disable the feature:
- Access CloudFormation in your installation’s AWS account and region
- Select your “Instance” stack and “Update” it
- Set the “Opt-out of enhanced cluster management” parameter to “Yes”
Questions about disabling this feature? Your Account Manager is ready to help evaluate the best approach for your organization.
How It Works
The Enhanced Cluster Management feature operates through a secure, asynchronous mechanism:
- A management agent runs within your Kubernetes cluster
- The agent monitors a cell-specific S3 bucket for command requests
- When a command is detected, the agent:
- Executes the approved command in the cluster
- Sanitizes the output to remove sensitive information
- Writes the sanitized response back to the S3 bucket
- All activities are logged for full auditability
Security and Data Protection
- Your cluster remains private with no direct external access
- All communication occurs through a S3 bucket owned by Gitpod and its Gitpod Enterprise control plane. Read access to this bucket will be limited to the AWS Account of the customer’s Gitpod installation.
- Response data is sanitized at the source before being written to S3
- Commands are limited to a pre-approved set defined by RBAC policy defined and reviewable in the Infrastructure Template
- You maintain full audit capability through S3 logs
Allowed Operations
The following operations are permitted based on the RBAC policy:
Read Operations
- List and view: pods, services, nodes, jobs, namespaces, persistent volumes, events
- Access pod logs
- View deployments, daemonsets, replicasets, statefulsets
- Monitor metrics and custom metrics
Management Operations
- Delete pods when necessary
- Update node status
- Scale deployments and replicasets
- Create pod evictions for maintenance
- Create jobs for maintenance
Workspace Management
- View and manage Gitpod workspaces
Please verify the full policy in the Cloudformation Template
Auditing Activity
To review commands executed in your cluster, you can access audit logs via AWS CloudWatch. These logs are stored in the log group /aws/containerinsights/meta/audit
and contain detailed information about requests and responses.
Accessing Audit Logs with AWS Logs Insights
Follow these steps to view audit logs in AWS CloudWatch using Logs Insights:
-
Open the AWS Management Console and navigate to CloudWatch.
-
In the left navigation pane, select Logs Insights.
-
Choose the log group
/aws/containerinsights/meta/audit
. -
Use the following query to filter relevant logs:
-
Click Run Query to retrieve and analyze logs.
Example Query Output
When executing the query, the results may appear as follows:
Field | Value |
---|---|
… | … |
@timestamp | 1739363063721 |
… | … |
kubernetes.container_name | genie |
log_processed.request.cmd | kubectl |
log_processed.request.args.0 | get |
log_processed.request.args.1 | pods |
log_processed.request.args.2 | -n |
log_processed.request.args.3 | default |
log_processed.response.output | NAME READY STATUS RESTARTS AGE |
This allows you to track cluster activities and ensure accountability for executed commands.
Was this page helpful?