Solved: Not enough RAM to run docker-compose?

Lou Bichard Product Manager at Gitpod

May 9, 2024

Have you been adding new services to your docker-compose configuration and are reaching a breaking point? Is your application now running with more services than the available resources on your laptop or device? Does your laptop fan spin so loud that it might take off?

Whilst there are some application-level optimizations that provide some pain relief, such as setting memory limits, or increasing RAM allocations, eventually you’ll reach the physical limits of your hardware, and a more comprehensive strategy is needed to tackle these challenges.

How to solve the not enough RAM for docker-compose issue? Mock out services to reclaim memory, use the cloud with “Remote - SSH” for VS Code, “JetBrains Gateway”, or a managed cloud development environment (CDE) solution such as Gitpod.

Mock your microservices and build contracts

A core principle of microservices is to have strict contracts between services. It should be possible to deploy each microservice locally without needing to have full integrations with other services. If you do need these other services—you’re building a distributed monolith, and you’re not getting the true benefits of microservices and container-based development.

With mocked services, you build  fast feedback locally. Once you’re happy with your local change to your microservice, you will typically push, using an automated build pipeline to a remote environment, to the cloud. That remote environment then runs all your full application integrations for testing. To help with building out these mocks we have tools like LocalStack for mocking cloud services, or PACT for contract testing.

However, mocking services like this does have its limits, and it’s not easy. It might mean that you need to re-architect your application, and within large organizations, using complicated tech stacks, re-architecting is never an easy task. So, what else can you do?

Self-host a virtual machine in the cloud

A simple way to get pain relief if you don’t have enough memory on your machine is to leverage the cloud. This might be easier than you imagine. The most heavily used editors, VS Code and JetBrains, both support remote development using plain SSH. Which means that all you need is a remote running VM with the right specifications, a hook up your editor, and you have an experience just like you’re developing locally, but with data center performance.

Simply download either the VS Code remote - SSH extension, or the JetBrains Gateway application, spin up a remote VM, for instance using EC2 on Amazon, and you’ll be able to connect up directly using SSH.

This can be a great option for short-term pain relief if you’re maxed out on your docker-compose memory. However, if you’re successful, and you want to rollout this pattern to your entire organization, that’s when things can get challenging. For instance:

• How do you manage the costs of your running cloud based VMs?

• How do you ensure your VMs are secure behind a VPN or private network?

• How do you enable developers to spin up and tear down VMs?

This is where cloud development environments come in.

Use a cloud development environment to increase RAM when running docker-compose

A cloud development environment like Gitpod is a professional, enterprise-grade version of the self-hosted cloud development solution described above. Gitpod covers all of the use cases typically required from large and regulated enterprises such as private network setup, access via VPN and SSO, cost management, analytics, and much more. A CDE is a very versatile solution and solves many organizational challenges, but one challenge it solves extremely well is getting access to more compute and RAM without compromising your developer experience.

Running a docker-compose file inside of Gitpod is simply a case of copying your existing docker-compose command to a .gitpod.yml file committed at the root of your repository.

For example:

tasks:
  - command: |
      docker-compose up

Now, take your repository URL, and prefix it with gitpod.io/# to open in Gitpod.

You can click the above link to see an example docker-compose configuration in action using one a Gitpod sample repository.

Now, you have a workspace running in the cloud with an editor of your choice. Use up to 8 cores and 16GB RAM on pay-as-you-go, or up to 30 cores and 54GB of RAM on the Gitpod Enterprise tier. Gitpod free plan includes 50 hours free a month.