[virt-tools-list] libosinfo - RFC v2

[Arjun Roy]

Hi all,

This is a second draft of the proposed libosinfo API.

Warning: pretty long.

The basic idea remains:

1. OS Data would be stored in several XML files.
2. The data would have a flat, no hierarchy layout.

I've made some changes, mostly based on suggestions in response to my first
email:

1. Primary identifier will now use RDF. In addition, there will be a short name
parameter that is supposed to be unique as well, for use in console apps and
other uses where the user will want to refer to a distro.

  <distro rdf:about="http://fedoraproject.org/fedora-10">
    <name>Fedora 10</name>
    <kernel>linux</kernel>
    <kernel-version>2.6.30</kernel-version>
  </distro>

  <distro rdf:about="http://fedoraproject.org/fedora-11">
   <upgrades rdf:about="http://fedoraproject.org/fedora-10">
   <short-id>fedora11</short-id>
   <name>Fedora 11</name>
  </distro>

2. A new type will be added to the library, the osi_hypervisor_t. It will
represent information dependent on the choice of hypervisor, such as supported
devices. The goal is to query libvirt for hypervisor information, and storing
the data in the backing store as a fallback.

In XML, a hypervisor could be represented like this:

  <hypervisor rdf:about="http://qemu.org/qemu-kvm-0.11.0">
    <name>KVM</name>
    <version>0.11.0</version>
    <audio-driver>ac97</audio-driver>
    <audio-driver>es1371</audio-driver>
    <video-driver>vesa</video-driver>
  </hypervisor>

3. All non-structure properties, including version numbers, are now plain
strings. Version numbers will be available as courtesy information only; it will
not be used to determine capabilities. There will be no notion of searching for
distros with a kernel greater than 2.6.29, for example.

4. Querying properties for an individual distro/hypervisor will be slightly
different. In the first revision, I assumed that at most, a distro only had one
value to a given property (kernel = linux, for example). But in the above
example for hypervisor, we see that there are several properties of type
'audio-driver'. The idea is that a property can now have several values,
one of which will be the preferred value. So we will now have methods to return
just one value like before (which would be the preferred value), and methods to
return all values. For example, the preferred sound driver could be ac97 while
the total set could be ac97, es1371 and intel_hda.

For simplicity, we will limit ourselves to properties that are strings having
(possibly more than one) values that are also strings. Specifically, properties
won't have subproperties.

>From an XML point of view, that means this is invalid:

<some-tag some-property='foo'>bar</some-tag>

Just to keep things simple.

________________________________________

API
________________________________________


The library would be implemented in C, and define a few major types:

osi_lib_t : an opaque handle to a library instance.
osi_distro_t : represents a distro object.
osi_distro_id_t : represents the unique ID for a distro.
osi_distro_list_t : a list of osi_distro_t objects.
osi_filter_t: filter for searching through distros.
osi_hypervisor_t : represents a hypervisor

And a bunch of methods:

osi_lib_t osi_get_lib_handle();

/* Setting parameters like libvirt version, etc. */
int osi_set_lib_param(osi_lib_t lib, char* key, char* val);
int osi_get_lib_param(osi_lib_t lib, char* key);
int osi_set_hypervisor(osi_lib_t lib, char* hvname, char* hvversion);
osi_hypervisor_t osi_get_hypervisor(osi_lib_t lib);

/* Initializing and closing down the library */
int osi_init_lib(osi_lib_t lib);
int osi_close_lib(osi_lib_t lib);

/* Getting parameters for hypervisor */
char* osi_get_hv_property_pref_value(osi_hypervisor_t hv, char* propname);
char** osi_get_hv_property_all_values(osi_hypervisor_t hv, char* propname, int* num);
char** osi_get_all_property_keys(osi_hypervisor_t hv, int* num);

/* Querying for distros and iterating over them */
osi_distro_list_t osi_get_distros_list(osi_lib_t lib, osi_distro_filter_t filter);
int osi_put_distros_list(osi_distro_list_t distros);
int osi_distro_list_length(osi_distro_list_t distros);

/* Methods for filtering results when querying distros */
osi_filter_t osi_get_filter(osi_lib_t lib);
int osi_put_filter(osi_filter_t filter);
int osi_add_constraint(osi_filter_t filter, char* propname, char* propval);
int osi_add_relation_constraint(osi_filter_t filter, enum_t relationship, os_distro_t distro);
int osi_clear_constraint(osi_filter_t filter, char* propname);
int osi_clear_relation_constraint(osi_filter_t filter, enum_t relationship);
int osi_clear_all_constraints(osi_filter_t filter);

/* Get a single distro, either from a list or by ID */
osi_distro_t osi_get_distro_by_index(osi_distro_list_t distros, int index);
osi_distro_t osi_get_distro_by_id(osi_lib_t lib, osi_distro_id_t id);

/* Query Properties for a given distro */
osi_distro_id_t osi_get_distro_id(osi_distro_t distro);
osi_distro_t osi_get_related_distro(osi_distro_t distro, enum_t relationship);
char* osi_get_property_pref_value(osi_distro_t distro, char* propname);
char** osi_get_property_all_values(osi_distro_t distro, char* propname, int* num);
char** osi_get_all_property_keys(osi_distro_t distro, int* num);

/* Query unique values for a given property */
char** osi_unique_property_values(osi_lib_t lib, char* propname, int* num);
os_distro_list_t osi_unique_relationship_values(osi_lib_t lib, enum_t relationship);


Here is a sample program that sets libvirt version, hypervisor type and version,
opens the library, gets a handle to RHEL5.4's representation, finds all the 
distros deriving from it, and lists the audio drivers available for each.

#include <stdio.h>
#include "libosinfo.h"

int main(void)
{
  int i, ret, count;
  osi_lib_t lib = osi_get_lib_handle();

  ret = osi_set_lib_param(lib, "libvirt-version", "3.4");
  if (ret != 0) {
    printf("Error: Could not set libvirt version!\n");
    exit(1);
  }

  ret = osi_set_hypervisor(lib, "kvm", "1.2");
  if (ret != 0) {
    printf("Error: Could not set hypervisor!\n");
    exit(1);
  }

  ret = osi_init_lib(lib);
  if (ret != 0) {
    printf("Error: Could not set initialize libosinfo!\n");
    exit(1);
  }

  osi_filter_t filter = osi_get_filter(lib);
  ret = osi_add_constraint(filter, "short-id", "rhel5.4");
  if (ret != 0) {
    printf("Error: Could not set constraint!\n");
    exit(1);
  }

  osi_distro_list_t results = osi_get_distros_list(lib, filter);
  if (osi_bad_object(results))
    printf("Bad result list!\n");
    exit(1);
  }

  if (osi_distro_list_length(results) == 0) {
    printf("No results. Quitting...\n");
    exit(0);
  }
  else if (osi_distro_list_length(results) > 1) {
    printf("Failed sanity check. 'short-id' should be unique...\n");
    exit(1);
  }

  osi_distro_t rhel = osi_get_distro_by_index(results, 0);

  // Now that we have a handle to rhel5.4, we can free the results list
  // that we used to get to it. The handle to the single distro is still
  // valid though, and we use it for the next step
  ret = osi_put_distros_list(results); // Done with that list so get rid of it
  if (ret != 0) {
    printf("Error freeing distro list!\n");
    exit(1);
  }

  // We shall reuse the filter
  ret = osi_clear_all_constraints(filter);
  if (ret != 0) {
    printf("Error clearing constraints!\n");
    exit(1);
  }

  if (osi_add_constraint(filter, "kernel", "linux") != 0 ||
      osi_add_constraint(filter, "kernel-version", "2.6.30") != 0 ||
      osi_add_relation_constraint(filter, DERIVES_FROM, rhel) != 0)
  {
    printf("Error adding constraints!\n");
    exit(1);
  }

  osi_distro_list_t more_results = osi_get_distros_list(lib, filter);
  if (osi_bad_object(more_results))
    printf("Bad result list!\n");
    exit(1);
  }

  // For each distro:
  count = osi_distro_list_length(more_results);
  for (i = 0; i < count; i++) {
    int j, num;
    osi_distro_t distro = osi_distro_by_index(more_results, i);
    char* distroname = osi_get_property_pref_value(distro, "name");
    char** audiodrivers = osi_get_property_values(distro, "audio-driver", &num);

    // For each audio driver:
    for (j = 0; j < num; j++) {
      printf("Audio driver for %s is: %s\n", distroname, audiodrivers[j]);
      free(audiodrivers[j]);
    }
    free(audiodrivers);
    free(distroname);
  }

  ret = osi_put_distros_list(more_results); // Done with that list
  if (ret != 0) {
    printf("Error freeing distro list!\n");
    exit(1);
  }

  ret = osi_put_filter(filter); // Done with the filter
  if (ret != 0) {
    printf("Error freeing filter!\n"):
    exit(1);
  }

  ret = osi_close_lib(lib);
  if (ret != 0) {
    printf("Error cleaning up library handle!\n");
    exit(1);
  }

  printf("Done.\n");
  return 0;
}

In the event that any single operation fails and we do a hard exit, we don't
need to do any library cleanup since the process is exiting anyways. The library
is designed to only ever use memory and open file handles to the backing store,
both of which the OS will cleanup on a bad exit.

So. I think this is expressive enough in terms of what is needed. Notice I
haven't mentioned implementation yet, and haven't locked in the backing data
schema. However, in my opinion everything that I have proposed for the API is
doable.

Awaiting comments/concerns/criticisms and anything really. Thanks for the read.

-Arjun