Showing posts tagged with libvirt

in libvirt, node, node.js, node-libvirt, promises

After adding a couple things to Node-Libvirt one of the maintainers pointed out my original article on Node-Libvirt referenced the old callback style of using the library so I wanted to take a moment to write up a new guide for using Node-Libvirt with promises rather than callbacks. Since my first post offers a fair amount of detail regarding Node-Libvirt in general, I'll limit this one to showing syntax only.

Hypervisor

First lets take a look at connecting to the hypervisor using promises. Of the changes, this is the most prominent (in my opinion anyway).

Connection

var libvirt = require('libvirt');  
var hypervisorObject = libvirt.hypervisor;

var hypervisor = new hypervisorObject('qemu:///system');

hypervisor.connect(function() {  
    // The hypervisor object is now connected. From here you can do just about any call you want...
    // Like hypervisor.lookupDomainById(1);
});

You'll probably want to wrap the code you need to access the hypervisor with in the hypervisor.connect() function to ensure the connection has been made before you attempt to communicate with the hypervisor.

Everything Else

It isn't really worth noting each and every function I mentioned originally, but I will give you a couple examples for how things look using promises.

The basic idea is you add Async to the name of the function you want to call, and then use then instead of a callback. For example:

hypervisor.helloWorld(function(callback) {  
    // Old style
});

hypervisor.helloWorldAsync(arguments).then(function(return_value) {  
    // New style
});

For real example take a look at a call to getNodeInfo:

hypervisor.getNodeInfoAsync().then(function(nodeStats) {  
    // nodeStats now has the return value from getNodeInfo
});

Domains

Domains use pretty much the same syntax shown above but I'll run through a few examples:

// Lets make a new domain
hypervisor.defineAsync(domainXML).then(function(domain) {  
    // Use domain to access this domain now

    // Now lets restart that domain
    domain.startAsync().then(function() {
        // Domain has been restarted
    });
});

// Lets lookup a domain with the id 1
hypervisor.lookupDomainByIdAsync(1).then(function(domain) {  
    // Use the domain object to do other stuff
});

Would you like to know more?

From here, everything looks almost identical to the syntax you see above. Just append Async to the name of the function and use then instead of a callback and you will be good to go.

If there are any specific calls you are interested in seeing please don't hesitate to leave a comment. Most of these calls do have examples in the Node-Libvirt tests. If you are planning on using Node-Libvirt consider following me on Twitter as I intend to document the full Node-Libvirt API in the future.

Thanks for reading!

in Getting Started, libvirt, node, node.js, domain, hypervisor, node-libvirt

Recently I've been doing a fair amount of work with the libvirt API, specifically using the Node.js bindings found here. The bindings are great and work as expected but I was rather disappointed in the lack of documentation.

The authors reference the tests for the project as documentation, but this can be a bit of a pain to read through especially if you are new to the language. Given this, I'm going to start documenting some of the functions I use most often.

This post assumes you are using the latest version of node-libvirt (>=0.1.3). Please note this is important because earlier versions of this library were not asynchronous - code snippets in this post will not work for earlier versions.

Hypervisor

Before you can do anything else, you must communicate with the hypervisor its self.

Connection

Communication with the hypervisor can be achieved by instantiating a new hypervisor object:

var libvirt = require('libvirt');  
var hypervisor_object = new libvirt.Hypervisor(CONNECT_URI);  

Connect URI will vary depending on your specific hypervisor (KVM, Xen, etc.) and how you are connecting exactly. For KVM this would be qemu:///system.

Next, you need to actually connect to the hypervisor:

hypervisor_object.connect(function(err) {  
    if (!err) {
        console.log("I'm connected!"); 
    }
});

Statistics

Most projects will want to know a bit about the hypervisor, you can gather all sorts of information but I have found the virNodeInfo call the most useful. This provides you with the model of the CPU, memory allocated to the host, number of CPUs, speed of CPUs, nodes, sockets, cores and threads per core. This call is made using the hypervisor object:

hypervisor_object.getNodeInfo(function(err, info) {  
    console.log(info); 
});

All remaining hypervisor functions are called in a similar fashion with a callback. Other stats functions of interest include:

  • listDefinedDomains - Returns an array of defined, but not active domain IDs
  • listActiveDomains - Returns an array of active domain names.
  • getNumberOfDefinedDomains - Returns an integer representing the total number of defined but not active domains.
  • getNumberOfActiveDomains - Returns an integer representing the total number of active domains.
  • getNodeFreeMemory - Returns an integer representing the amount of free memory on the host.

From here you can use the hypervisor object to do other things.

Domains

Next, lets get to the meat of the library and discuss domain handling.

Create a Domain

Creating a domain is done using the hypervisor object we created before. You'll need to magic up some XML in the appropriate format to make this work. Once you have that being generated and accessible from Node you can use the following call to make a domain:

hypervisor_object.createDomain(domain_xml, function(err, domain) {  
    // Domain is created if no error
});

This will create a non-persistent domain which basically means if the domain is destroyed (completely stopped) it ceases to exist. This is okay if you have all the information you need to reconstruct the domain XML elsewhere (like in a database), if you don't you might want to look into using persistent domains. These can be created using the define function instead of the createDomain wrapper:

hypervisor_object.define(domain_xml, function(err, domain) {  
    // Persistent domain created
});

Actions

There are a number of actions available for a domain including:

  • reset
  • reboot
  • suspend
  • resume
  • shutdown
  • start

All of these are called the same way using the domain object. You can get the domain object by accessing domain in the create domain example or you can look a domain up using lookupDomainById or lookupDomainByName, both are covered later.

domain.ACTION(function(err) {  
    // If no error action successful
});

Substitute an action from the list above for ACTION.

Get

Getting a domain can be done using either lookupDomainById or lookupDomainByName. Realistically, unless you have a unique application lookupDomainByName will be better used because the domain ID is not static and there is no reliable way of predicting it, the name however is static and should be unique.

hypervisor_object.lookupDomainByName(DOMAIN_NAME_HERE, function(err, domain) {  
    // You've got your domain! 
});

Should you need to look up the domain by ID you call that function the same way as shown above.

Destroy

Some people get confused by the nomenclature used by libvirt regarding destroying a domain. In the context of libvirt, destroy does not mean delete in the traditional sense, it essentially means stop the domain now (force it to stop if it cannot be stopped otherwise). This is like pulling the plug on the machine so data loss is a possibility. There is an exception however, if the domain is not persistent destroy will actually remove the domain from libvirt. This is done like the other actions mentioned above:

domain.destroy(function(err) {  
    // I'm stopped!
});

If you have yourself a persistent domain and you want to actually make libvirt forget about it completely you would need to use the undefine function:

domain.undefine(function(err) {  
    // Poof, I'm gone!
});

Would you like to know more?

I originally started writing this guide with the intention of documenting all the things, but in truth it would be a waste of time. All calls are made using the same basic format:

the_object.WHAT_I_WANT(function(err, RESPONSE_IF_THERE_IS_ONE) {  
});

If you find anything particularly challenging regarding hypervisor or domain functions please let me know - I'm happy to help!

While a bit annoying, the tests for node-libvirt are useful if you want to see the context of these calls. You can find the tests for domains here and hypervisors here.

If there is any interest I'd be happy to continue covering the basic calls for other libvirt areas - please let me know if you are interested in that at all.

Thanks for reading!

in python, space, virtualization, libvirt, development

When I first started working on Space I knew I would need to use something like libvirt to make all of the calls to the hypervisor its self. I had heard of libvirt several times during my tenure in the IT industry so I figured it would be realtively easy to get things running fairly quickly but when I started reading the libvirt documentation I was very surprised at how poor it was.

I'm not hoping to rewrite all of their documentation, but I would like to give a few examples for doing basic things using the libvirt API in Python.

Connecting to the Hypervisor

You'll need to establish a connection with your hypervisor before you can interact with it. A simple function to build that connection looks like this:

import libvirt

def connect():  
    connection = libvirt.open("SYSTEM_TYPE:///system")
    return conn

You'll end up returning a connection object that you can then use to do other things. SYSTEM_TYPE will be something like QEMU or XEN.

Getting a List of Domains (Virtual Machines)

There are actually two calls required to do this, listDefinedDOmains() which returns a list of all defined domains (running and not running), and listDomainsID() which returns a list of only running domains.

import libvirt

connection = libvirt.open("SYSTEM_TYPE:///system")

# This lists all domains on the host
connection.listDefinedDomains()  
# Returns
['vm1','vm2','vm3']

# This lists running domains on the host
connection.listDomainsID()  
# Returns
[2] 
# These are the ID's as used by the hypervisor - you'll need to store this somewhere to tie this to a virtual machine as it will change each time the virtual machine is started.

Find a Domain

You can find a domain two different ways, you can use the ID that was mentioned above (this changes, so it's tough to use this unless you maintain this information somewhere) or you can use the name of the virtual machine.

I generally use the name because I set it and it stays the same regardless of what happens to the virtual machine. The name is what you'll see if you run virsh list --all:

 virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     vm54d42cd2a5ee223e6be95bc2     running

So you can do that using the following functions:

# Assume the connection function is included here

virtual_machine = connection.lookupByName("vm54d42cd2a5ee223e6be95bc2")  
# Returns the object representing this virtual machine

virtual_machine = connection.lookupByID(2)  
# Returns the same object, but uses the ID as shown in the virsh output. Probably not the one you want to use, but it is there if you need it.

Start Domain

Once you have the domain object you can start/stop/redfine it fairly easily. Let's look at starting it:

virtual_machine.create()  
# Will create (start) this virtual machine if it is defined
Stop Domain
virtual_machine.destroy()  
# Will destroy (stop) this virtual machine if it is defined

Create Domain

Now lets make a brand-spankin'-new domain. This wil take a bit of work, so lets dig in.

First you'll need to make a domain configuration file - this is an XML file that defines a bunch of things about the domain. I've provided an example below:

<domain type="kvm">  
    <name>vm11</name>
    <cpu>
        <topology cores="4" sockets="1" threads="4" />
    </cpu>
    <uuid>90e7bca4-97b2-11e4-86bf-001e682ee78a</uuid>
    <memory unit="MB">512</memory>
    <currentMemory unit="MB">512</currentMemory>
    <vcpu placement="static">2</vcpu>
    <os>
        <type>hvm</type>
        <boot dev="hd" />
    </os>
    <features>
        <acpi />
        <apic />
        <pae />
    </features>
    <clock offset="utc" />
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>restart</on_crash>
    <devices>
        <emulator>/usr/libexec/qemu-kvm</emulator>
        <disk device="disk" type="file">
            <driver cache="none" name="qemu" type="raw" />
            <source file="/var/disks/vm11.img" />
            <target dev="hda" />
            <address bus="0" controller="0" target="0" type="drive" unit="0" />
        </disk>
        <disk device="cdrom" type="file">
            <source file="/var/images/ubuntu-14.04.1-server-amd64.iso" />
            <driver name="qemu" type="raw" />
            <target bus="ide" dev="hdc" />
            <readyonly />
            <address bus="1" controller="0" target="0" type="drive" unit="0" />
        </disk>
        <interface type="network">
            <source network="default" />
        </interface>
        <graphics port="-1" type="vnc" />
    </devices>
</domain>  

I won't dig into how this is made (perhaps I'll do that in another post), but long story short you'll need to make this file and save it somewhere on your filesystem. You can take a look here to see how I did this in Space.

Next, we'll need to use this configuration to define a new virtual machine. Defining a virtual machine essentially means you are creating it, once it is defined you can undefine it (delete it), start it, and shut it down. To define the domain, you can use defineXML(xml):

# First you'll want to get your XML file into memory
xml = ""  
with open(path_to_xml_config, "r") as file:  
    xml = file.read()
connection.defineXML(xml)  
# If you don't get an error, the domain has been created, woot!

Delete Domain

Deleting a domain is significantly easier than making one. You'll just need to use the undefine() function:

virtual_machine = connection.lookupByName("your_domain_name")  
virtual_machine.undefine()  
# Bye bye virtual machine, its dead! 

Updating Domain

I'm sure some of you have put two and two together here and have pieced together how you would update a domain from that - if not, I'll detail that now.

You'll first need to overwrite the old configuration file (or make a new one). The new configuration file should include the new options you want to update the domain to. Next, you'll need to undefine the current domain using the method described above.

Once the domain has been undefined, simply define it again as described in the create a domain section. Start it up and you'll have a domain with updated settings.

Conclusion

I hope this saves somebody time in the future because it took me a while to figure out how to get all this stuff working my first time around. If you have any questions about anything I didn't cover, please feel free to comment or hit me up on Twitter @joseph_pettit.

in space, control panel, virtualization, celery, mongodb, kvm, libvirt, spacepanel

One of my goals for 2015 was to learn more about virtualization - for me, learning requires a little reading and a ton of doing. I'd like to introduce my latest project that is a result of the aforementioned "doing" - Space.

Space is a virtualization control panel similar to cPanel, minus ripping out the guts, heart and soul of your operating system. Space is built in Flask and utilizes some new technologies for me, including libvirt, KVM, Celery and MongoDB.

Screen shot of Space Panel

Space is a web based GUI for libvirt essentially, meaning it has all of the flexibility offered by libvirt (in theory). Currently it runs on Centos 6.6 and KVM only, but I plan on expanding that to Debian/Ubuntu and Xen in the coming weeks/months/*.

You can do nearly everything you would need to do to manage a virtual machine directly in Space, including:

  • Create new virtual machines
  • Delete virutal machines
  • Start/stop/restart VMs
  • Access virtual machine via web console
  • Resize disks
  • Manage disk images
  • Manage networking
  • Access logs, see events

I'm making Space freely available to anybody who wants to use it, the source is available on Github. Feel free to git clone and give it a whirl. Be warned though, there are still some bugs that need to be worked out, so I wouldn't recommend using this in any environment even closely resembling production.