Idempotency is one of the fundamental ideas when managing systems: it’s both convenient and natural to demand that any management action has the same result whether it’s performed once or multiple times. For example, if the management action is ‘make sure that httpd is running’, we only want to say...
Handling SSL certificates is not a lot of fun, and while Puppet’s use of client certificates protects the server and all its deep, dark secrets very well from rogue clients, it also leads to a lot of frustration. In many cases, users would configure their autosign.conf to allow any (or...
DHH has a post on some of the hoopla around hypermedia API’s over at SvN, complete with a cool picture of the WS-*. While I agree with most of his points, he’s missing the larger point of API discoverability. The reason discoverability is front and center in RESTful API’s isn’t...
This morning, the DMTF officially announced the availability of CIMI v1.0. After two years of hard work, heated discussions, and many a vote on proposed changes, CIMI is the best shot the fragmented, confusing, and in places legally encumbered, landscape of IaaS API’s has at a universally supported API. Not...
Like everything, REST API’s change over time. An important question is how these changes should be incorporated into your API, and how your clients should behave to survive that evolution. The first reflex of anybody who’s thought about API’s and their evolution is to stick a version number on the...
The upcoming release of Deltacloud 1.0 is a huge milestone for the project: even though no sausages were hurt in its making, it is still chockful of the broadest blend of the finest IaaS API ingredients. The changes and improvements are too numerous to list in detail, but it is...
TL;DR: have a look at sinatra-rabbit. When we converted Deltacloud from Rails to Sinatra, we needed a way to conveniently write the controller logic for RESTful routes with Sinatra. On a lark, I cooked up a DSL called ‘Rabbit’ that lets you write things like collection :images do description "The...
Installing Deltacloud is work. Not a lot of work, in fact it is very easy, but it still involves installing a package/gem and starting a server. For simple development and test uses, even that is not necessary any more. There’s two of them: one, https://api.deltacloud.org/ runs the latest stable release,...
We just released Apache Deltacloud 0.4.0, part of the Apache Incubator. The release contains a huge number of enhancements and additions. The full list can be found in the release announcement, but some of them bear highlighting separately. The biggest new feature is probably a driver for VMWare’s vSphere. This...
As we all know by now, cloud computing is a veritable goat rodeo, an unseemly sight for anybody’s stomach. Disconcerted by these proceedings, Mark Shuttleworth lets his stomach have the better of him, and declares it over by picking the winner. That, of course, is not how you end a...
Somehow, I find myself writing the same email to introduce people to git over and over again. But no more ! Now, I will only send out links to this blog entry. Git can be intimidating at first, even though it is probably the most forgiving source control system out...
The initial Deltacloud project consisted of two components: the RESTful API and an inter-cloud broker, a web application. When we submitted the API part of the project to Apache, also under the name Deltacloud, it was clear that we’d have to find a new name and home for the cloud...
ApacheCon takes place next week in Atlanta. And Deltacloud will be there, too: we are having a Deltacloud meetup on Thursday, 11/4, at 8pm in Room 4. You can find more details about the meetup on the ApacheCon Wiki The goal of the meeting is to bring everybody from the...
I learned two things yesterday browsing the web idly: Webzero as a name for the minimalist style of the new web applications we are seeing, that so far I’ve just been referring to as “a new Rails app with great UX”: light, easy on the eyes, easy to use, without...
We’ve been working on it for a while, but today, I finally managed to import the existing code for Deltacloud Core into Apache’s subversion repository. From now on, development of the Deltacloud API will happen within the Apache Incubator. The main reason for this move is that we’ve heard from...
We have several projects at Red Hat that are currently implementing REST API’s, from . At first, REST is nice and easy, and has a lot of appeal because it’s not SOAP. When you get into the nitty-gritty of designing a REST API though, there are numerous niggling issues; to...
When you use Augeas to add users to groups, you might do that by first adding a new user node to the corresponding group (we’ll use kvm for the sake of this example), and then set its value to the name of the user: augtool> ins user after /files/etc/group/kvm/user[last()] augtool>...
A few weeks ago, I came across Sinatra, a minimalist Ruby web framework, much leaner, meaner and simpler than Rails — while it’s probably not a good fit for traditional database-backed web applications, it seemed like an ideal framework for Deltacloud Core. Deltacloud Core is a cross-cloud API providing abstraction...
For a while now, I’ve been unhappy with my blog setup; when I started blogging, the only blog my hosting provider had was b2evolution, a monster of a traditional blogging engine, more suitable for hosting many blogs than just the one I needed. Add to that the fantastico, allegedly a...
The Eucalyptus folks yesterday gave the Deltacloud API a spin against a Eucalyptus instance. No surprises there, seeing how Eucalyptus strives for compatibility with the EC2 API — Deltacloud API just works. All you need to do is use the Deltacloud EC2 driver and make sure it has the EC2_URL...
Virtualization management software needs to control, in addition to virtual machine lifecycles, pretty much any aspect of virtualized hosts. The basic open-source building block for this is libvirt — one of the features it was missing up until release 0.7.0 was managing the network setup of a host. When we...
Often, when editing a config file from a program, one also wants to leave a comment alongside the change. When the change adds something to the config file, that’s fairly simple. For example, to add a new entry to /etc/sysctl.conf that sets vm.mmap_min_addr to 64k and puts a comment just...
Michael Stahnke wrote some excellent slides on developing infrastructure for the Red Hat Summit, partly based on his experience within Fedora Infrastructure. The gist if the talk is something most sysadmins somehow know, but management rarely likes to admit: infrastructure needs to grow organically with an organization, and there isn’t...
One of the nice things about having your configuration data in a uniform data structure like Augeas’ tree is that you can query it in a uniform way. A couple of releases ago, I added support for querying the tree with XPath — it was surprising, even to me, how...
The ‘g’ key on my laptop is dyin. Now all my typin is goin to be lookin like I can see Russia from my front porch.
Like many people, I have the problem that I have way too many home directories - one on my laptop, one on my main desktop etc. For the longest time, I tried to keep them in sync with crazy schemes involving rsync or NFS mounting and the like. And it...
LXer has a very nice summary of Raphael’s talk on Augeas — though, contrary to what the author says, writing lenses (i.e., the plumbing that explains to Augeas how to map a config file into the tree) is not hard. You can mostly focus on how to take your file...
Looking at the schedule for FOSDEM next week makes me regret that I am halfway around the world from Brussels — there’s a ton of talks I’d love to go to. Raphaël is going to talk about Augeas. He’s done a lot of work on Augeas in the past, and...
Raphael has been doing a lot of work to make processing of INI-style files in Augeas easy and painless. It’s now at the point that you can describe /etc/php.ini in a few lines: module PHP = autoload xfm let entry = IniFile.entry /[a-z][a-z0-9\._-]+/ let record = IniFile.record "section" entry let...
Thanks to the tireless efforts of Dominique Dumont, Augeas now has Perl bindings in addition to the existing Python, Ruby, and OCaml bindings. And the corresponding perl-Config-Augeas package should be coming to a Fedora mirror near you very soon.
linux.com has a very nice article on Augeas. It’s a very nice overview and introduction. As embarrassed as I am by the bug the author ran into (blank lines in /etc/hosts threw Augeas’ parser off), I am glad to say that it’s fixed in the most recent version 0.2.0. The...
I just released Augeas 0.1.1; without really planning it, it turned out that the last two weeks were mostly spend on fixing bugs (besides the regular expression enhancement I blogged about previously — even though the real reason for doing that was that the typechecker had a serious bug, and...
For Augeas, I wanted to support subtraction of regular expresions, so that you can say let key_re = /[A-Za-z]+/ - /(Allow|Deny)(Groups|Users)/ which would make key_re match all words made up of lower and upper case letters except for AllowGroups, AllowUsers, DenyGroups and DenyUsers — the reason being, that those four...
A while ago I had what would be a hallway conversation with Mark if we worked in the same office (or country, for that matter.) Something he said set me thinking that getting a better handle on the mess of file formats in /etc would be possible, and in a...
Recently, I needed a finite automata library written in C (for those of you who don’t remember their formal language classes too well, finite automata are the theoretical underpinning of regular expressions) In a nutshell, a finite automaton represents the set of all strings matching a regular expression. Such a...
At long last, Ruby on Rails 2 is part of Fedora. Packages are already in rawhide and will show up in the testing repos for F-7 and F-8 really soon now. The package is called rubygem-rails, as it’s based on the rubygems for Rails. For those few who installed the...
Yesterday, while playing with a tool that shall remain unnamed, I needed some more details on its command line options. Out of habit, I ran tool --help, to which it politely replied see 'man tool' for instructions. Turduckalope ! What a tool ! To prove a sense of humor, the...
JumpBox packages open source applications as appliances. Since I wanted to see what they are all about, I downloaded one of them. After unpacking, the first roadblock is that the appliance doesn’t come with any config file for libvirt - JumpBox includes a Xen config file, but I wanted to...
In theory, appliances are easy: download an appliance, point your favorite virt management tool at the downloaded image, and you have yourself a running, working virtual machine doing great things for you in no time (bandwidth limits may apply). In practice, that’s not quite the case: after picking the appliance...
When you’re using kvm to run virtual machines, you more than likely also want them to have some sort of network access. There’s two very basic setups that seem useful: Create a private network on the host that gets NAT'd to the 'outside', similar to how your home network is...
Appliances of the ‘software’ or ‘virtual’ variety are popular, and everybody has to have a few. And that’s your problem right there: once you have not just a few, but many appliances, how do you keep track of them, how do you keep them running, and how do you keep...
I just made the very first release of cft, the sabayon-like config file tracker. This version supports almost all the types that puppet understands natively, except for cron, sshkey and package. Package support, i.e. understanding what packages the sysadmin has installed/updated/removed during a session, will be the focus for the...
I’ve started working on a tool that helps tracking changes to a system’s configuration, especially when it is controlled by Puppet. The tool is called cft (pronounced ‘sift’) and is inspired by Gnome’s Sabayon — it watches how an admin changes a system and produces a puppet manifest based on...
Konstantin, your assertion that puppet is a “cfengine lookalike written in Ruby,” could not be farther from the truth. There is a long list of reasons why that is not the case; the comparison on the puppet site does a good job at explaining the most important of them. For...
The topic of kickstarting and enabling puppet came up on et-mgmt-tools. In addition to what I’ve said previously on the subject, a few more tips: 1. Installing puppet in the main %packages section If you are using Fedora Core 6, you can specify additional yum repos right in the kickstart...
It turns out that some kind soul added buildsys-build and buildsys-macros packages to the Fedora buildgroups repository. That considerably simplifies setting up a mock buildroot for RHEL, much simpler than my previous entry. All you need to do now to create a mock buildroot for, say, RHEL4 is: Create a...
Mock, the Fedora tool for maintaining standard buildroots and building rpm’s in them, can also be used to build packages for Red Hat Enterprise Linux (RHEL). As an example, to create a mock buildroot for RHEL4, you need to: Create a yum repository with the RPM's for RHEL4 in them....
In a previous post, I mentioned that puppetca can be used to pregenerate certificates for puppet clients. Pregenerated certificates can be used very effectively to avoid transferring the client certificate in the clear over a network, for example, by storing them on a USB key, and carrying them by hand...
Puppet stores configuration files for all its clients on a central fileserver, which by default is the same as the puppetmaster. If a manifest needs to refer to a file on the fileserver without knowing the hostname of the puppetmaster explicitly, it can use the variable $servername in the manifest,...
Sometimes it is necessary in a puppet manifest to run a command with exec every time puppet changes a file. The subscribe metaparameter is ideal for this purpose, since it causes the dependent element to be executed whenever the referenced element changes — with the small caveat that the dependent element will...
A configuration management tool like [Puppet](http://reductivelabs.com/projects/puppet) ensures that the local configuration of a machine agrees with its configuration stored on a central server, and makes it possible to install packages, change configuration files, start and stop services etc. by editing the puppet manifest on the central server. But before you...
Spammers have found this blog (before anybody else, it seems) Since I’ve been inundated with bogus trackbacks, I have turned them off for now.
You can get at a stacktrace in Ruby through Kernel#caller To dump out stacktraces in your code simply add caller.each { |l| $stderr.puts l } (For some reason, I keep forgetting how to do that, and have to Google for it every time)
I’ve written up a detailed example of using Puppet to deploy and configure a database backed web application. Apart from discussing the ins and outs of using Puppet, the writeup also explains how the modularity of Puppet manifests enables sharing of them.
A very simple problem in managing Fedora machines with Puppet is that of controlling which yum repositories the machines get their updates from. Puppet does not (yet) have a way to specify the client configuration of yum repositories; to work around this, we need to deploy the complete configuration for...
Package management with tools such as rpm or apt is such a staple of life on Linux that we have mostly forgotten what an enormous step forward these tools are. In a nutshell, these tools package software together with metadata that makes certain types of sanity checks possible; most important...
While installing the [tomcat5](http://jakarta.apache.org/tomcat/) RPM from [jpackage](http://jpackage.org/rpm.php?id=1901), it once again hit me what a shoddy job Sun is doing in the open source world. If you listen to Sun's executives long enough, you might think they invented open source software, and have been driving it ever since. But when you...
Watzmann.Blog by David Lutterkort is licensed under
a Creative Commons Attribution-Share Alike 3.0
United States License.
Generated with Jekyll