[virt-tools-list] [vhostmd virtio PATCH v3 5/6] Activate virtio support in vhostmd
Jim Fehlig
jfehlig at suse.com
Wed Nov 14 23:46:13 UTC 2018
On 11/12/18 8:13 AM, Michael Trapp wrote:
> Activate the virtio code and extend DTD/XML to support a virtio section
> for virtio related configuration and a new transport type virtio.
> ---
> README | 84 +++++++++++++++++++++++++++++++++++++++++----
> vhostmd.changes | 5 +++
> vhostmd.dtd | 6 +++-
> vhostmd.xml | 6 ++++
> vhostmd/Makefile.am | 4 +--
> vhostmd/vhostmd.c | 61 ++++++++++++++++++++++++++++++--
> 6 files changed, 154 insertions(+), 12 deletions(-)
>
> diff --git a/README b/README
> index cb23226..f473079 100644
> --- a/README
> +++ b/README
> @@ -50,8 +50,15 @@ includes a few examples of user-defined metrics, which provide a
> <path>/dev/shm/vhostmd0</path>
> <size unit="k">256</size>
> </disk>
> + <virtio>
> + <max_channels>1024</max_channels>
> + <expiration_time>15</expiration_time>
> + <path>/var/lib/libvirt/qemu/channels</path>
We should update this to /var/lib/libvirt/qemu/channel/target, depending on the
answer to my question in 4/6.
> + </virtio>
> <update_period>5</update_period>
> <path>/usr/bin:/usr/sbin:/usr/share/vhostmd/scripts</path>
> + <transport>vbd</transport>
> + <transport>virtio</transport>
> </globals>
> <metrics>
> <metric type="string" context="host">
> @@ -115,11 +122,17 @@ includes a few examples of user-defined metrics, which provide a
> </vhostmd>
>
> A valid configuration file must contain the root element <vhostmd>.
> -The <globals> element contains configuration global to vhostmd, such as
> -the metrics disk path and the metrics refresh interval. The <metrics>
> -element is a container for all of the <metric> elements. A metric element
> -is used to define a metric, giving it a name and an action that produces
> -the metric value.
> +
> +The <globals> element contains configuration global to vhostmd, such as the
> +metrics refresh interval and the metrics transport mechanism. The <transport>
> +element defines how the metrics are transported between the host and VMs. The
> +vbd transport uses a virtual disk, described in the <disk> element, to share
> +metrics data between host and VM. The virtio transport, described by the
> +<virtio> element, uses a virtio-serial connection to share the metrics data.
> +
> +The <metrics> element is a container for all of the <metric> elements.
> +A metric element is used to define a metric, giving it a name and an action
> +that produces the metric value.
>
> The supplied vhostmd configuration file provides a useful set of default
> metrics to be collected. This can be extended or modified by editing
> @@ -277,14 +290,71 @@ section:
> (Note: Change target dev and bus as appropriate for the domain)
>
>
> +
> +Notes on Virtio Transport
> +-------------------------
> +
> +The virtio transport uses a virtio serial device to transport metrics data
> +between the host and VMs. Basically for a virtio serial device, QEMU creates
> +- a unix domain socket on the Host
> +- a serial port on the VM
> +- 'connects' both to a 'communication channel'
> +
> +It can be configured in the virtio section of the vhostmd configuration file.
> +<max_channels> defines the maximum number of virtio channels/VMs supported
> +by the vhostmd instance with a default value of 1024.
> +<expiration_time> is the time after which the virtio serial channel of a VM
> +is detached when vhostmd did not receive updates. It's default value is
> +'3 * update_period' of the <globals> section and the configured value is
> +checked to be at least '3 * update_period'.
> +<path> is the directory where the virtio serial channels are accessible and
> +it must be the directory of the path attribute of QEMUs VM/channel
> +configuration. The default path is '/var/lib/libvirt/qemu/channels'.
> +
> + <vhostmd>
> + <globals>
> + <virtio>
> + <max_channels>1024</max_channels>
> + <expiration_time>15</expiration_time>
> + <path>/var/lib/libvirt/qemu/channels</path>
Same here.
> + </virtio>
> +
> +Sample VM config with virtio serial:
> +
> + <channel type='unix'>
> + <source mode='bind' path='/var/lib/libvirt/qemu/channels/\
> +org.github.vhostmd.1.a70605c8-7d69-8c44-7e1a-5ecd092cb1e1'/>
Ideally we'd like to leave out the path and let libvirt generate it.
> + <target type='virtio' name='vhostmd'/>
Here is where we use the reverse domain name and protocol version. E.g.
<target type='virtio' name='org.github.vhostmd.1'/>
> + <address type='virtio-serial' controller='0' bus='0' port='1'/>
> + </channel>
> +
> +The name of a channel in a QEMU VM configuration must be
> + <path from vhostmd-globals-virtio-path>/org.github.vhostmd.1.<VM-UUID>
> +
> +
> +Vhostmd accepts metric requests 'GET /metrics/XML\n\n' and responds with
> +
> + <metrics>
> + <metric type='real64' context='host'>
> + <name>TotalCPUTime</name>
> + <value>179645.910000</value>
> + </metric>
> + ...
> + <metric type='uint64' context='vm' id='9' uuid='a70605c8-7d69-8c44-7e1a-5ecd092cb1e1'>
> + <name>UsedMemory</name>
> + <value>524288</value>
> + </metric>
> + </metrics>
> +
> +
> Guest Tool/Library for Accessing Metrics Data
> ---------------------------------------------
>
> Tool: vm_dump_metrics
> Stand alone static utility will read all the metrics and write them
> to stdout or optionally an argumented file.
> - Usaga:
> - vhostmd [-f dest_file]
> + Usage:
> + vm_dump_metrics -b|-i|-x [-d dest_file]
>
> Library: libmetrics.so.0
> Dynamic library that supports individual metrics gathering
> diff --git a/vhostmd.changes b/vhostmd.changes
> index c7b453d..c1aa06f 100644
> --- a/vhostmd.changes
> +++ b/vhostmd.changes
> @@ -1,3 +1,8 @@
> +-------------------------------------------------------------------
> +Wed Jul 25 10:00:20 CEST 2018 - michael.trapp at sap.com
> +
> +- add virtio as transport mechanism
> +
> -------------------------------------------------------------------
> Mon Jun 29 16:42:39 MDT 2009 - jfehlig at novell.com
>
> diff --git a/vhostmd.dtd b/vhostmd.dtd
> index 471c72f..6480532 100644
> --- a/vhostmd.dtd
> +++ b/vhostmd.dtd
> @@ -9,7 +9,7 @@ Virtual Host Metrics Daemon (vhostmd). Configuration file DTD
> -->
>
> <!ELEMENT vhostmd (globals,metrics)>
> -<!ELEMENT globals (disk,update_period,path,transport+)>
> +<!ELEMENT globals (disk,virtio,update_period,path,transport+)>
>
> <!ELEMENT disk (name,path,size)>
> <!ELEMENT name (#PCDATA)>
> @@ -20,6 +20,10 @@ Virtual Host Metrics Daemon (vhostmd). Configuration file DTD
> <!ELEMENT update_period (#PCDATA)>
> <!ELEMENT transport (#PCDATA)>
>
> +<!ELEMENT virtio (max_channels,expiration_time,path)>
> +<!ELEMENT max_channels (#PCDATA)>
> +<!ELEMENT expiration_time (#PCDATA)>
> +
> <!ELEMENT metrics (metric*)>
> <!ELEMENT metric (name,action,variable*)>
> <!ELEMENT action (#PCDATA)>
> diff --git a/vhostmd.xml b/vhostmd.xml
> index 9b048df..5400981 100644
> --- a/vhostmd.xml
> +++ b/vhostmd.xml
> @@ -33,9 +33,15 @@ the logical && operator must be replaced with "&&".
> <path>/dev/shm/vhostmd0</path>
> <size unit="k">256</size>
> </disk>
> + <virtio>
> + <max_channels>1024</max_channels>
> + <expiration_time>15</expiration_time>
> + <path>/var/lib/libvirt/qemu/channels</path>
Another spot potentially needing an adjustment.
Regards,
Jim
> + </virtio>
> <update_period>5</update_period>
> <path>/usr/sbin:/sbin:/usr/bin:/bin:/usr/share/vhostmd/scripts</path>
> <transport>vbd</transport>
> + <transport>virtio</transport>
> <!-- <transport>xenstore</transport> -->
> </globals>
> <metrics>
> diff --git a/vhostmd/Makefile.am b/vhostmd/Makefile.am
> index 3585970..34a15e0 100644
> --- a/vhostmd/Makefile.am
> +++ b/vhostmd/Makefile.am
> @@ -3,9 +3,9 @@ INCLUDES = \
> -I../include
>
> sbin_PROGRAMS = vhostmd
> -vhostmd_SOURCES = vhostmd.c util.c metric.c virt-util.c
> +vhostmd_SOURCES = vhostmd.c util.c metric.c virt-util.c virtio.c
> vhostmd_CFLAGS = $(LIBXML_CFLAGS) $(LIBVIRT_CFLAGS)
> -vhostmd_LDADD = -lm $(LIBXML_LIBS) $(LIBVIRT_LIBS)
> +vhostmd_LDADD = -lm $(LIBXML_LIBS) $(LIBVIRT_LIBS) -lpthread
>
> if WITH_XENSTORE
> vhostmd_SOURCES += xenstore-update.c
> diff --git a/vhostmd/vhostmd.c b/vhostmd/vhostmd.c
> index e7ec2fc..e13fa2d 100644
> --- a/vhostmd/vhostmd.c
> +++ b/vhostmd/vhostmd.c
> @@ -42,10 +42,11 @@
> #include <time.h>
> #include <libxml/parser.h>
> #include <libxml/xpath.h>
> +#include <pthread.h>
>
> #include "util.h"
> #include "metric.h"
> -
> +#include "virtio.h"
>
> /*
> * vhostmd will periodically write metrics to a disk. The metrics
> @@ -85,6 +86,7 @@ typedef struct _mdisk_header
> */
> #define VBD (1 << 0)
> #define XENSTORE (1 << 1)
> +#define VIRTIO (1 << 2)
>
> /* Global variables */
> static int down = 0;
> @@ -103,6 +105,9 @@ static mdisk_header md_header =
> };
> static char *search_path = NULL;
> static int transports = 0;
> +static char *virtio_path = "/var/lib/libvirt/qemu/channels";
> +static int virtio_max_channels = 1024;
> +static int virtio_expiration_time = 15;
>
>
> /**********************************************************************
> @@ -470,6 +475,8 @@ static int parse_transports(xmlDocPtr xml,
> return -1;
> #endif
> }
> + if (strncasecmp((char *)str, "virtio", strlen("virtio")) == 0)
> + transports |= VIRTIO;
> free(str);
> }
> }
> @@ -605,6 +612,19 @@ static int parse_config_file(const char *filename)
> goto out;
> }
>
> + if (transports & VIRTIO) {
> + char *p = NULL;
> +
> + if (vu_xpath_long("string(./globals/virtio/max_channels[1])", ctxt, &l) == 0)
> + virtio_max_channels = (int)l;
> +
> + if (vu_xpath_long("string(./globals/virtio/expiration_time[1])", ctxt, &l) == 0)
> + virtio_expiration_time = (int)l;
> +
> + if ((p = vu_xpath_string("string(./globals/virtio/path[1])", ctxt)) != NULL)
> + virtio_path = p;
> + }
> +
> /* Parse requested metrics definitions */
> if (parse_metrics(xml, ctxt)) {
> vu_log(VHOSTMD_ERR, "Unable to parse metrics definition "
> @@ -838,7 +858,8 @@ static int metrics_disk_create(void)
> static int metrics_host_get(vu_buffer *buf)
> {
> metric *m = metrics;
> -
> + unsigned start = buf->use;
> +
> while (m) {
> if (m->ctx != METRIC_CONTEXT_HOST) {
> m = m->next;
> @@ -850,12 +871,18 @@ static int metrics_host_get(vu_buffer *buf)
>
> m = m->next;
> }
> +
> + if (transports & VIRTIO)
> + virtio_metrics_update(&buf->content[start], (int) (buf->use - start),
> + NULL, METRIC_CONTEXT_HOST);
> +
> return 0;
> }
>
> static int metrics_vm_get(vu_vm *vm, vu_buffer *buf)
> {
> metric *m = metrics;
> + unsigned start = buf->use;
>
> while (m) {
> if (m->ctx != METRIC_CONTEXT_VM) {
> @@ -868,6 +895,11 @@ static int metrics_vm_get(vu_vm *vm, vu_buffer *buf)
>
> m = m->next;
> }
> +
> + if (transports & VIRTIO)
> + virtio_metrics_update(&buf->content[start], (int) (buf->use - start),
> + vm->uuid, METRIC_CONTEXT_VM);
> +
> return 0;
> }
>
> @@ -912,11 +944,30 @@ static int vhostmd_run(int diskfd)
> int *ids = NULL;
> int num_vms = 0;
> vu_buffer *buf = NULL;
> + pthread_t virtio_tid;
>
> if (vu_buffer_create(&buf, MDISK_SIZE_MIN - MDISK_HEADER_SIZE)) {
> vu_log(VHOSTMD_ERR, "Unable to allocate memory");
> return -1;
> }
> +
> + if (transports & VIRTIO) {
> + int rc;
> +
> + if (virtio_expiration_time < (update_period * 3))
> + virtio_expiration_time = update_period * 3;
> +
> + if (virtio_init(virtio_path, virtio_max_channels, virtio_expiration_time))
> + return -1;
> +
> + rc = pthread_create(&virtio_tid, NULL, virtio_run, NULL);
> +
> + if (rc != 0) {
> + vu_log(VHOSTMD_ERR, "Failed to start virtio thread '%s'\n",
> + strerror(rc));
> + return -1;
> + }
> + }
>
> while (!down) {
> time_t run_time,
> @@ -948,6 +999,12 @@ static int vhostmd_run(int diskfd)
> vu_buffer_erase(buf);
> }
> vu_buffer_delete(buf);
> +
> + if (transports & VIRTIO) {
> + virtio_stop();
> + pthread_join(virtio_tid, NULL);
> + }
> +
> return 0;
> }
>
>
More information about the virt-tools-list
mailing list