Infrastructure Update Changelog

This update might incur a downtime of up to 15 minutes due to EKS cluster maintenance and redis updates. We recommend doing this outside of working hours.

How to update

Follow the process laid out on Updating the Enterprise Infrastructure

Changelog

• Use encryption in transit with S3
• Upgrade Redis to version 7.1 and configure that minor version upgrades are automatically performed by AWS
• Upgrade EKS clusters to 1.28
• Increase retention period for CloudWatch Log Groups to two weeks, thereby making it easier to debug issues that occured further in the past

This is an optional infrastructure upgrade releasing a bug fix. You can choose to skip this release if the improvements listed are not urgently needed. This upgrade does not incur a downtime.

Changelog

• Cleanup the default security group of the Gitpod cell to not allow any traffic
• Fixes some policy requirements for a AWS Lambda role
• Your workspace can now access resources via a proxy. Please contact your Gitpod account manager to know how to enable these.
• Clean up of an unused log group

This update does not incurr downtime. Running workspaces are not affected. For extra caution, it is advised to perform the update outside of working hours.

Changelog

• HTTP2Preferred is now set on the HTTPS listeners for the workspace and meta load balancers. Results in better browser performance and reduces the amount of stale HTTP connections
• Support nameservers for allPrivate networking mode when using BYO Hosted Zone
• Enable providing only one hosted zone
• Support for proxy settings in workspace pods
• Routine version update for EKS addons

This update might incur a downtime of upto 5 minutes due to EKS cluster maintenance. We recommend doing this outside of working hours.

Changelog

• Introduce Container Image Signing for all images used by the dedicated cell
• Add support for Bringing Your Own(BYO) VPC and Hosted Zone to install Enterprise in
• Upgrade kube-proxy and vpc-cni cluster addons
• Add Role ARN to the output of the role template
• Fix AMI update workflow to cleanup all older nodes
• CloudFormation stack deletion now also cleans up the loadbalancer logs
• Fix policy issues in the pods in workspace cluster

This update does not incur any downtime.

Changelog

• Fix autoscaler configuration to use instance’s local timezone
• RDS access has been limited to the meta subnet, away from internal subnets
• CloudFormation stack deletion now fully deletes the instance without additional user input
• Remove duplicate entry for services DNS entry for public API gateway
• Remove inbound traffic traffic rules to the lambda security group
• Add Transit Gateway attachment ID and VPC ID to the CloudFormation stack output

This update transitions MySQL from v5.7 to v8 and incurs full downtime of up to 30 minutes.

Changelog

• Major MySQL upgrade to v8
• Switches CA to rds-ca-rsa2048-g1 to avoid expiry in Aug 2024
• Policy to enable lambda updates
• Policies to support cleaner stack deletion

This update impacts running workspaces and should not be done during working hours. You can expect a downtime of 4 minutes after the CloudFormation Change Set is applied as new nodes are spun up.

Changelog

• Support for in-place AMI updates for all clusters
• Improved cell lambda image update workflow, triggered by new Enterprise releases. This equips a faster rollout of cell Lambda code changes. This does not change the permission scope of the lambdas.
• Automated changeset creation process, thereby simplifying future infrastructure upgrades. After this upgrade you will be able to preview all the changes automatically in the stack’s Changesets section
• Dynamic autoscaler configurations enabling fine tuning of cluster scaling capacity to optimally accommodate varying workloads. You can talk to your Gitpod Account manager to get this configured
• Export of historical logs, thereby enhancing the ease of debugging issues that customers encounter
• Various bug fixes

This update impacts running workspaces and should not be done during working hours. You can expect a downtime of 5 minutes after the CloudFormation Change Set is applied as new nodes are spun up.

Creating the change set can take longer than usual. Further, once the change stack is applied, the clean up step will take longer than usual - up to 40 minutes (see below for reasoning). The Gitpod instance can be used as normal during this time. Future updates will take less time again.

Changelog

• Support for custom CA certificates (important: An application release is necessary to fully roll out this feature. You can ask your Gitpod Account Manager whether your instance has received the required release)
• Disabled scaling the instance to 0 nodes during working hours (6:00 to 22:00 local time to the instance) to speed up the workspace starts in the morning. Scale to 0 is still enabled on weekends.
• Improvements of log groups associated with Lambda functions to reduce cost and align function names with AWS conventions. This requires all lambdas to be recreated, leading to the longer than usual clean up time mentioned above.
• Enforce use of IMDSv2 AWS metadata endpoint for EC2 instances
• Various bug fixes

Changelog

• Update to the application controller (Lambda) to improve the ordering of its operations
• Turn off debug mode for the telemetry controller as it was logging too much
• Turn off AZ rebalancing which was impacting the stability of some nodes and thus workspaces
• Set workspace DNS resolvers to be local VPC resolver IP instead of public DNS lookup. This resolves networking issues in environments where public DNS lookups are blocked. This is the first of a two part roll out process, the second part is an application change.