[virt-tools-list] How does the memory usage calculated?

[张强]

Hi Cole, Richard,

Thanks for your answers.

Cole, I looked at your commit, I think we should increase required libvirt-python version to use 
SetMemoryStatsPeriod. 

This function seems fairly new, my libvirt is 0.10.x, and still it doesn't support this function, let 
alone the 0.4.0 we listed in pre-requisite in the INSTALL file... I think adding install_requires to 
setup.py is a good idea.


-----邮件原件-----
发件人: Cole Robinson [mailto:crobinso at redhat.com] 
发送时间: 2015年11月19日 10:26
收件人: Richard W.M. Jones; 张强
抄送: virt-tools-list at redhat.com
主题: Re: [virt-tools-list] How does the memory usage calculated?

On 11/18/2015 09:54 AM, Cole Robinson wrote:
> On 11/18/2015 09:44 AM, Richard W.M. Jones wrote:
>> On Wed, Nov 18, 2015 at 01:04:32PM +0000, 张强 wrote:
>>> Hi all,
>>> I’m studying the source code, and I saw this in domain.py around line 164:
>>> …
>>>         curmem = stats['rss']
>>>         totalmem = stats['actual']
>>> except libvirt.libvirtError, err:
>>>     logging.error("Error reading mem stats: %s", err)
>>>
>>> pcentCurrMem = curmem * 100.0 / totalmem pcentCurrMem = max(0.0, 
>>> min(pcentCurrMem, 100.0)) …
>>>
>>> But using this algorithm, I’m always getting results that are greater than 100:
>>>>>> dom.memoryStats()
>>> {'actual': 16777216L, 'rss': 17062900L}
>>>
>>> 17062900 * 100 / 16777216 = 101.7028…
>>>
>>> Is this normal?
>>
>> The VIR_DOMAIN_MEMORY_STAT_RSS statistic returned for qemu/KVM guests 
>> is the resident set size (RSS) of the qemu process.  Of course that 
>> includes all the overhead of qemu, such as host-side structures used 
>> when emulating devices.  Plus the guest RAM, which is just a regular 
>> malloc-style allocation inside qemu and so also contributes to the RSS.
>>
>> The 'actual' field seems to come from the libvirt 
>> VIR_DOMAIN_MEMORY_STAT_ACTUAL_BALLOON statistic which is found by 
>> sending the query-balloon monitor command to the guest.  The returned 
>> value is the guest's get_current_ram_size() (or less if the balloon 
>> is "inflated", but for the majority of guests the balloon is never used).
>>
>> Given all that, it seems reasonable that rss > actual, and so you'd 
>> get a number > 100%.  Sometimes.  It also seems to me that if the 
>> guest RAM had been allocated but not accessed, you might see rss < actual.
>>
>> rss / actual seems like a fairly meaningless number from a 
>> virt-manager point of view.
>>
> 
> Yeah I think this calculation is leftover from when virt-manager's 
> memory reporting just tracked memory ballooning, so pcentCurrMem was 
> <currentMemory> / <memory>. I'll look into cleaning it up
> 

We still need to do the 'rss / actual' allocation to get meaningful memory stats from the lxc driver, but I pushed a commit to virt-manager.git now that uses qemu's proper guest coordinated memory stats. In order to get these, we need to call DomainSetMemoryStatsInterval so qemu actually does the polling for us

commit e68efe810375826a0e72864f4e60b3ee3e0d9bb8
Author: Cole Robinson <crobinso at redhat.com>
Date:   Wed Nov 18 21:17:22 2015 -0500

    domain: Use setMemoryStatsPeriod for QEMU

    Since that's what is required to get cooperative memory stats from
    the guest balloon driver.


- Cole