Ruby Memory Allocations: How Upgrading Reduces Cloud Infrastructure Costs

Before the advent of cloud computing, scaling an application meant physically racking new servers — a capital expense that required planning, purchase orders, and lead time. When a Ruby application needs more resources now, auto-scaling groups spin up new containers or dynos.

While this operational ease is a profound benefit, it obscures a critical financial reality: in the cloud, memory is money.

When your cloud hosting bill grows, it is often not strictly because you have more users, but rather because your application’s memory footprint requires larger, more expensive instance types to serve those users. For many Ruby applications running on older versions of the language, memory bloat and inefficient garbage collection are the primary drivers of these escalating costs.

Before we discuss how to fix this, though, let’s take a step back and examine how Ruby allocates memory, and why upgrading your Ruby version is one of the most effective levers you have for reducing your infrastructure spend.

The Mechanics of Ruby Memory Allocation

To understand how Ruby consumes your cloud budget, we first need to understand how it consumes RAM.

Ruby, strictly speaking, does not allocate memory for objects exactly the way a lower-level language like C does. Instead, the Ruby virtual machine maintains a structure called the ObjectSpace, which is divided into heaps. These heaps contain slots, and each slot holds an RVALUE — a C struct that represents a single Ruby object.

Historically, every RVALUE slot was exactly 40 bytes (on a 64-bit system). This is perfectly fine for a simple integer or a boolean. However, if you create a string that is larger than what can fit inside those 40 bytes, Ruby cannot make the slot larger. Instead, it must reach out to the operating system using the C malloc function to allocate additional memory elsewhere, and then store a pointer to that external memory inside the RVALUE.

This architecture leads directly to two distinct challenges that inflate cloud costs: memory fragmentation and garbage collection overhead.

Memory Fragmentation

By way of a memory aide, imagine a shipping warehouse where you only have boxes of exactly two sizes: tiny and massive. If you need to ship a medium-sized item, you are forced to use the massive box, wasting most of the space inside.

Memory fragmentation works similarly. As your application runs, it allocates and deallocates millions of objects. The external malloc allocations leave gaps in the system’s memory. Over time, the operating system struggles to reclaim these fragmented gaps efficiently. As a result, your Puma worker process might report using 500MB of RAM, even if the actual Ruby objects only account for 200MB. The rest is essentially wasted space trapped by fragmentation.

How Newer Ruby Versions Solve the Problem

Of course, the Ruby core team is acutely aware of these challenges. Over the last several major releases — specifically from Ruby 3.1 through 3.3 — significant architectural changes have been introduced to address memory bloat.

Variable Width Allocation (VWA)

Perhaps the most impactful change for memory utilization is Variable Width Allocation, introduced experimentally in Ruby 3.1 and expanded in subsequent versions.

Rather than forcing every object into a rigid 40-byte slot and using malloc for the overflow, VWA allows the Ruby heap to maintain slots of different sizes. This means that a moderately sized string or array can now be stored entirely within the Ruby heap.

One may wonder: if we are only changing the size of the slot, why does this dramatically reduce memory fragmentation? The answer is straightforward. By storing data inside the heap itself, VWA avoids triggering an external malloc call. The operating system no longer has to manage millions of tiny, scattered allocations in external memory.

The practical result of this change is profound:

1. It drastically reduces memory fragmentation by keeping memory contiguous inside the heap.
2. It improves CPU cache locality, making garbage collection faster.
3. It directly lowers the baseline memory footprint of your application.

Object Shapes

Introduced in Ruby 3.2, Object Shapes completely reimagined how Ruby stores instance variables. In older versions, every object maintained an internal hash table or an array to keep track of its instance variables. This required significant memory overhead for every single instance of a class you instantiated.

With Object Shapes, Ruby tracks the shape (the specific set and order of instance variables) at the class level. Individual objects store their values in a flat array, and reference the shared shape to know which index corresponds to which variable. For an ActiveRecord model with dozens of attributes instantiated thousands of times per request, this optimization yields a massive reduction in allocated bytes.

First, let’s discuss a few possibilities besides upgrading your Ruby version. You could, of course, manually tune your garbage collection environment variables like RUBY_GC_HEAP_INIT_SLOTS or RUBY_GC_MALLOC_LIMIT. Some developers prefer this philosophy of fine-grained tuning. However, these settings are complex to optimize and require constant benchmarking as your application changes. Upgrading your Ruby version, on the other hand, provides architectural improvements that benefit your memory usage automatically without manual tuning. Therefore, a version upgrade is generally a more robust long-term solution, which is why we will focus on the upgrade path here.

Translating Memory Savings to Infrastructure Costs

Let’s ground this in practical economics. How do fewer allocations and reduced fragmentation actually lower your AWS or Heroku bill?

The math is straightforward. Most Ruby web applications are deployed using Puma, a threaded web server that utilizes multiple worker processes. The limiting factor for how many Puma workers you can run on a single cloud instance is almost always memory, not CPU.

Suppose you are running an application on Heroku Performance-M dynos (which provide 2.5GB of RAM), and each of your Puma workers consumes 500MB. You can safely run 4 workers per dyno.

If upgrading from Ruby 2.7 to Ruby 3.3 reduces your memory footprint by 20% (a conservative estimate for many applications), your worker memory drops to 400MB.

# Before Upgrade (Ruby 2.x)
2500MB Total RAM / 500MB per worker = 4 workers per instance

# After Upgrade (Ruby 3.x)
2500MB Total RAM / 400MB per worker = 6 workers per instance

By fitting 6 workers onto the same instance instead of 4, you have increased your throughput capacity by 50% without changing your hardware. Consequently, if you were previously running 10 dynos to handle your peak traffic, you can now handle that exact same traffic with only 7 dynos.

That is a direct, recurring 30% reduction in your cloud hosting costs, achieved entirely through a software upgrade.

Verifying the Impact in Your Application

In order to dig a bit deeper into the mechanics of memory allocations in your own codebase, you don’t need to guess. You can measure it directly using tools like the memory_profiler gem.

Let’s illustrate this with a small script. First, you must install the gem. You can do this by adding it to your Gemfile:

gem 'memory_profiler'

Next, you can wrap a typical operation — such as rendering a complex view or processing a background job — in a profiling block. Let’s create a file named profile-memory.rb in our Rails application root. Although memory_profiler is an excellent tool for finding allocations, running it will significantly slow down the block being measured; therefore, it is strongly recommended to restrict this kind of profiling to development or staging environments:

require 'memory_profiler'

report = MemoryProfiler.report do
  User.includes(:orders, :preferences).limit(1000).to_a
end

report.pretty_print(scale_bytes: true)

We can run this script using the rails runner command:

$ bundle exec rails runner profile-memory.rb

Note the use of bundle exec; we could have simply used rails runner, but that won’t necessarily give us the correct version of the gems we are looking for. As we can see, running the above script does not modify any records in the database; because we are only reading records into memory with .to_a, it is a read-only process.

The output will be quite long, but the most important metrics are at the very top:

Total allocated: 14.2 MB (125000 objects)
Total retained:  145.0 kB (1300 objects)

Allocated memory by gem
-----------------------------------
      12.0 MB  activerecord-7.0.8
       2.2 MB  my_app/app/models

...snip...

I’ve abbreviated the rest of the output for the sake of brevity, but the ...snip... section contains detailed breakdowns of memory allocation by file, location, and class. Additionally, note that the exact memory numbers you see will likely vary; your specific object counts and bytes allocated will depend heavily on your data structure, application complexity, and the Ruby version you are currently running.

You also may notice that the profiler explicitly separates “Total allocated” from “Total retained”. Allocated memory represents the objects created during the block that were eventually garbage collected. Retained memory represents objects that survived and permanently increased your footprint.

By running this script in your older Ruby environment, and then again after upgrading, you can objectively quantify the reduction in allocated objects and bytes.

Conclusion

Upgrading a legacy Ruby on Rails application is often framed purely as a matter of security, compliance, or developer happiness. While those are valid justifications, they can sometimes be difficult to quantify to business stakeholders.

Cloud infrastructure costs, however, are highly quantifiable. By taking advantage of modern Ruby features like Variable Width Allocation and Object Shapes, you can significantly reduce memory fragmentation and bloat. This allows you to pack more worker processes into fewer cloud instances, directly lowering your monthly spend.

A Ruby upgrade is not a technical chore; it is a powerful financial lever. Maintaining a regular upgrade cadence ensures that your application remains not only secure and maintainable, but also cost-effective to operate at scale.