Showing posts tagged with python

in python, space, virtualization, libvirt, development

When I first started working on Space I knew I would need to use something like libvirt to make all of the calls to the hypervisor its self. I had heard of libvirt several times during my tenure in the IT industry so I figured it would be realtively easy to get things running fairly quickly but when I started reading the libvirt documentation I was very surprised at how poor it was.

I'm not hoping to rewrite all of their documentation, but I would like to give a few examples for doing basic things using the libvirt API in Python.

Connecting to the Hypervisor

You'll need to establish a connection with your hypervisor before you can interact with it. A simple function to build that connection looks like this:

import libvirt

def connect():
	connection = libvirt.open("SYSTEM_TYPE:///system")
    return conn

You'll end up returning a connection object that you can then use to do other things. SYSTEM_TYPE will be something like QEMU or XEN.

Getting a List of Domains (Virtual Machines)

There are actually two calls required to do this, listDefinedDOmains() which returns a list of all defined domains (running and not running), and listDomainsID() which returns a list of only running domains.

import libvirt

connection = libvirt.open("SYSTEM_TYPE:///system")

# This lists all domains on the host
connection.listDefinedDomains()
# Returns
['vm1','vm2','vm3']

# This lists running domains on the host
connection.listDomainsID()
# Returns
[2] 
# These are the ID's as used by the hypervisor - you'll need to store this somewhere to tie this to a virtual machine as it will change each time the virtual machine is started.

Find a Domain

You can find a domain two different ways, you can use the ID that was mentioned above (this changes, so it's tough to use this unless you maintain this information somewhere) or you can use the name of the virtual machine.

I generally use the name because I set it and it stays the same regardless of what happens to the virtual machine. The name is what you'll see if you run virsh list --all:

 virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     vm54d42cd2a5ee223e6be95bc2     running

So you can do that using the following functions:

# Assume the connection function is included here

virtual_machine = connection.lookupByName("vm54d42cd2a5ee223e6be95bc2")
# Returns the object representing this virtual machine

virtual_machine = connection.lookupByID(2)
# Returns the same object, but uses the ID as shown in the virsh output. Probably not the one you want to use, but it is there if you need it.

Start Domain

Once you have the domain object you can start/stop/redfine it fairly easily. Let's look at starting it:

virtual_machine.create()
# Will create (start) this virtual machine if it is defined
Stop Domain
virtual_machine.destroy()
# Will destroy (stop) this virtual machine if it is defined

Create Domain

Now lets make a brand-spankin'-new domain. This wil take a bit of work, so lets dig in.

First you'll need to make a domain configuration file - this is an XML file that defines a bunch of things about the domain. I've provided an example below:

<domain type="kvm">
    <name>vm11</name>
    <cpu>
        <topology cores="4" sockets="1" threads="4" />
    </cpu>
    <uuid>90e7bca4-97b2-11e4-86bf-001e682ee78a</uuid>
    <memory unit="MB">512</memory>
    <currentMemory unit="MB">512</currentMemory>
    <vcpu placement="static">2</vcpu>
    <os>
        <type>hvm</type>
        <boot dev="hd" />
    </os>
    <features>
        <acpi />
        <apic />
        <pae />
    </features>
    <clock offset="utc" />
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>restart</on_crash>
    <devices>
        <emulator>/usr/libexec/qemu-kvm</emulator>
        <disk device="disk" type="file">
            <driver cache="none" name="qemu" type="raw" />
            <source file="/var/disks/vm11.img" />
            <target dev="hda" />
            <address bus="0" controller="0" target="0" type="drive" unit="0" />
        </disk>
        <disk device="cdrom" type="file">
            <source file="/var/images/ubuntu-14.04.1-server-amd64.iso" />
            <driver name="qemu" type="raw" />
            <target bus="ide" dev="hdc" />
            <readyonly />
            <address bus="1" controller="0" target="0" type="drive" unit="0" />
        </disk>
        <interface type="network">
            <source network="default" />
        </interface>
        <graphics port="-1" type="vnc" />
    </devices>
</domain>

I won't dig into how this is made (perhaps I'll do that in another post), but long story short you'll need to make this file and save it somewhere on your filesystem. You can take a look here to see how I did this in Space.

Next, we'll need to use this configuration to define a new virtual machine. Defining a virtual machine essentially means you are creating it, once it is defined you can undefine it (delete it), start it, and shut it down. To define the domain, you can use defineXML(xml):

# First you'll want to get your XML file into memory
xml = ""
with open(path_to_xml_config, "r") as file:
	xml = file.read()
connection.defineXML(xml)
# If you don't get an error, the domain has been created, woot!

Delete Domain

Deleting a domain is significantly easier than making one. You'll just need to use the undefine() function:

virtual_machine = connection.lookupByName("your_domain_name")
virtual_machine.undefine()
# Bye bye virtual machine, its dead! 

Updating Domain

I'm sure some of you have put two and two together here and have pieced together how you would update a domain from that - if not, I'll detail that now.

You'll first need to overwrite the old configuration file (or make a new one). The new configuration file should include the new options you want to update the domain to. Next, you'll need to undefine the current domain using the method described above.

Once the domain has been undefined, simply define it again as described in the create a domain section. Start it up and you'll have a domain with updated settings.

Conclusion

I hope this saves somebody time in the future because it took me a while to figure out how to get all this stuff working my first time around. If you have any questions about anything I didn't cover, please feel free to comment or hit me up on Twitter @joseph_pettit.

in open source, project, ptpb, pastebin, python, flask

I've tried a few different times to make a decent command line pastebin - each ended up being super bloated and slow. I think I've finally succeeded in that goal though - with the help of Zack Buhman I made ptpb.

ptpb is ultralight, ultrafast and feature rich. It is written entirely in Python and is completely open source. I'm a big advocate for keeping your own data, so I wanted to make sure ptpb could be deployed by anybody who wants to roll their own version of it so I/we have written detailed deployment instructions in the repo.

A quick overview of things ptpb can do:

  • Make pastes (obviously)
  • Edit pastes (using a UUID that is given to you when you make a paste, secure and easy)
  • Handle all types of data (images, documents, pdf's, bits in bits out)
  • Display all types of data (append the extension and your browser does the rest)
  • Render different types of text with syntax highlighting
  • Forced SSL everywhere

Thanks to Zack, this is all done extremely quickly and in a (relatively) light fashion. My latest tests show ptpb outperforming all of the major competitors in this realm (ix.io, sprunge.us).

I'm super pumped about this project - give it a whirl!