[virt-tools-list] How does the memory usage calculated?

[Cole Robinson]

On 11/18/2015 09:54 AM, Cole Robinson wrote:
> On 11/18/2015 09:44 AM, Richard W.M. Jones wrote:
>> On Wed, Nov 18, 2015 at 01:04:32PM +0000, 张强 wrote:
>>> Hi all,
>>> I’m studying the source code, and I saw this in domain.py around line 164:
>>> …
>>>         curmem = stats['rss']
>>>         totalmem = stats['actual']
>>> except libvirt.libvirtError, err:
>>>     logging.error("Error reading mem stats: %s", err)
>>>
>>> pcentCurrMem = curmem * 100.0 / totalmem
>>> pcentCurrMem = max(0.0, min(pcentCurrMem, 100.0))
>>> …
>>>
>>> But using this algorithm, I’m always getting results that are greater than 100:
>>>>>> dom.memoryStats()
>>> {'actual': 16777216L, 'rss': 17062900L}
>>>
>>> 17062900 * 100 / 16777216 = 101.7028…
>>>
>>> Is this normal?
>>
>> The VIR_DOMAIN_MEMORY_STAT_RSS statistic returned for qemu/KVM guests
>> is the resident set size (RSS) of the qemu process.  Of course that
>> includes all the overhead of qemu, such as host-side structures used
>> when emulating devices.  Plus the guest RAM, which is just a regular
>> malloc-style allocation inside qemu and so also contributes to the RSS.
>>
>> The 'actual' field seems to come from the libvirt
>> VIR_DOMAIN_MEMORY_STAT_ACTUAL_BALLOON statistic which is found by
>> sending the query-balloon monitor command to the guest.  The returned
>> value is the guest's get_current_ram_size() (or less if the balloon is
>> "inflated", but for the majority of guests the balloon is never used).
>>
>> Given all that, it seems reasonable that rss > actual, and so you'd
>> get a number > 100%.  Sometimes.  It also seems to me that if the
>> guest RAM had been allocated but not accessed, you might see rss < actual.
>>
>> rss / actual seems like a fairly meaningless number from a
>> virt-manager point of view.
>>
> 
> Yeah I think this calculation is leftover from when virt-manager's memory
> reporting just tracked memory ballooning, so pcentCurrMem was <currentMemory>
> / <memory>. I'll look into cleaning it up
> 

We still need to do the 'rss / actual' allocation to get meaningful memory
stats from the lxc driver, but I pushed a commit to virt-manager.git now that
uses qemu's proper guest coordinated memory stats. In order to get these, we
need to call DomainSetMemoryStatsInterval so qemu actually does the polling for us

commit e68efe810375826a0e72864f4e60b3ee3e0d9bb8
Author: Cole Robinson <crobinso at redhat.com>
Date:   Wed Nov 18 21:17:22 2015 -0500

    domain: Use setMemoryStatsPeriod for QEMU

    Since that's what is required to get cooperative memory stats from
    the guest balloon driver.


- Cole