Comments on: Idle Memory Tax

By: jason

jason — Wed, 03 Nov 2010 05:19:13 +0000

@Forbes Guthrie
Almost everything I know or have written about Idle Memory Tax comes from reading VMware ESX Server Advanced Technical Design Guide by Oglesby and Herold as well as VMware Infrastructure 3 Advanced Technical Design Guide by Oglesby, Herold, and Laverick. The former (older) book actually contains more detailed information about IMT.

Excerpt from page 60 of the VI3 ATDG:
“Idle Memory Tax
When memory share allocation takes effect, VMware provides a mechanism to
prevent virtual machines from hoarding memory they may not be utilizing. Just
because a particular server has four times the memory share priority than another
does not mean it requires it at the time allocation takes place. VMware has
a process that applies an idle memory tax. This associates a higher “cost value”
to unused allocated shares than it does to memory that is actively used within a
virtual machine. This allows the virtual machine to release it for use on other
guests that may require it. If the virtual machine in question has a need for the
memory, it still has the proper authority to reclaim it as it still has priority over
the memory space. A default value of 75% of idle memory may be reclaimed by
the tax.”

By: Forbes Guthrie

Forbes Guthrie — Tue, 02 Nov 2010 22:32:33 +0000

OK, I’ve spent quite a bit of time digging into the depths of this one. To cut a long story short, IMT is always being calculated, but the result of the calculation is only used during contention. So when TPS isn’t cutting the mustard and a host decides it wants to reclaim memory (calculated via free memory states), then it uses IMT to figure out which VMs to recover from. It can use ballooning and swap for this. Presumably compression also fits in now.
So IMT is really used just to figure out which VMs to reclaim from and in what proportions.
Anyway, I’ll be writing something about this in the next couple of days. Jason, I’ll let you see a preview
If I get time, I’ll create a blog post as well.
Cheers, Forbes.

By: Forbes Guthrie

Forbes Guthrie — Tue, 02 Nov 2010 04:28:42 +0000

Hi Jason,
Following on from your last comment that ballooning is only used during oversubscribing, whereas IMT is used all the time. I actually think that perhaps this is on its head. Although the term is “idle”; it is as you said related and applied through shares. As we know shares only apply during contention. Therefore surely IMT is only used during contention.
To me this makes more sense. TPS is used all the time (as soon as a VM is turned on, and with the large pages cavet), but ballooning and then swapping only happens during contention (via memory state percentages). And now we can add in compression just as swapping kicks in.
What do you think? I have been going round in circles with this as many of the VMware papers use very loose terminology which tends to confuse things. They use “enabling overcommitment on the host” with “being overcommitted” as interchangeable even though they obviously aren’t.
Forbes.

By: Ballooning And Hypervisor Swapping – Common Misunderstandings « DeinosCloud

Thu, 30 Sep 2010 11:37:34 +0000

[...] Sources: vSphere Resource Management Guide, Understanding Memory Resource Management in VMware ESX 4.1, Yellow-Bricks.com, Boche.net [...]

By: jason

jason — Thu, 04 Feb 2010 14:00:58 +0000

Memory ballooning is a mechanism and the VMkernel uses it when needed to reclaim memory during periods of memory oversubscribing.

It is my understanding that mem tax idle is ALWAYS running and just so happens to use the balloon driver as a mechanism by default to reclaim the memory because it is a proven technology/method to use.

Jas

By: Jb

Jb — Thu, 04 Feb 2010 08:35:59 +0000

Hi,

I read your article and I found it very interressting. But there is something I can’t understand. You said Idle Memory is reclaims by balloon driver (by default). But I’m wondering when this memory taxing is working (always ? only during contention ?). I was told Ballooning was only working contention but by reading you I’ve the feeling that idle memory taxing is always “running” in the background… Can you explain me this ?

Thanks a lot

By: Resource allocation for Virtual Machines « A glimpse into the life of IT

Resource allocation for Virtual Machines « A glimpse into the life of IT — Sun, 03 Jan 2010 01:06:38 +0000

[...] Memory Tax in ESX http://www.boche.net/blog/index.php/2009/01/29/idle-memory-tax/ Possibly related posts: (automatically [...]

By: jason

jason — Wed, 25 Nov 2009 01:13:59 +0000

The Balloon Driver is a mechanism that forces memory pages in the guest to be paged out to disk. Essentially it’s a little application in the guest (installed with VMware Tools) which the ESX host will “inflate” (think intentional memory leak). Bytes in RAM get paged to disk and ESX “borrows” the memory that was made availalble, giving it to another VM.

Idle Memory Tax is a configurable setting which leverages either the Balloon Driver (preferred) or VMkernel swap (more detrimental to the performance of the VM) to reclaim idle memory in the VM, giving it to another VM.

What’s the difference you ask? Not a whole lot. The Balloon Driver is a memory reclamation mechanism in and of itself. Idle Memory Tax is a configurable setting which leverages the Balloon Driver memory reclamation mechanism. Also, Idle Memory Tax only borrows idle (unused) memory (think of it as a luxury tax with the luxury being excess memory in the guest not being used) whereas the Balloon Driver borrows used or unused memory in the guest at the discretion of the ESX VMkernel.

By: rafee

rafee — Wed, 25 Nov 2009 00:32:13 +0000

Hi
I am new to this VMware,I have question
what is the difference between Balloon drive and Memory tax?
If both services same purpose? why do we need two different concept.
Its like reinventing a wheel?

By: VMware ESX Memory Over Commit Technology Explained | VM /ETC

VMware ESX Memory Over Commit Technology Explained | VM /ETC — Fri, 30 Jan 2009 15:59:53 +0000

[...] Boche’s post titled Idle Memory Tax is a great read if you are trying to understand ESX memory allocation between virtual machines [...]

By: Rbrambley

Rbrambley — Fri, 30 Jan 2009 14:39:24 +0000

Balloons, Robin Hood, and taxes! Oh my! Hmm, didn’t Prince have a CD with that title?

Great post. Thanks for taking a tough concept and making it easier to understand.