If this makes sense to you, HELP, otherwise, nothing to see here, move along.
I’m trying to simplify the task of configuring and maintaining Linux servers at work and I want to build on some existing configuration management system to do so. We are using Ubuntu Linux distribution, and I was thinking of just building on the APT package management tools they’ve borrowed (among other things) from Debian, but I decided to look for something distro agnostic.
I’ve spent a lot of time and frustration the last day trying to get the server working for Chef, a new system written in Ruby. I spent time scanning their bugtracker and asking for help in their IRC group to no avail. It still doesn’t work, and I have no more idea why than when I started.
I’m really doubting my decision:
- Chef has only been packaged for the bleeding edge version of Ubuntu. Um yeah, great, I really want to use beta software on my SERVERS.
- The installation documemtation advises that I dowload and install Ruby Gems from a tarball becuase the version in the Ubuntu repositories isn’t to their liking. Great, I have to install extra shit by hand before I can use the software I want to use so I don’t have to install shit by hand. That’s efficient, right?
- Chef relies on OpenID for authentication. Sweet! I can use my MySpace account to manage my servers! Well, I could, if only I could figure out the appropriate URL for the myspace authentication endpoint (and I was batshit insane). As for how I integrate OpenID authentication with anything else I’m using, I’m sure it will be easy and obvious what to do, in a year or two.
- Oh yeah, I forgot the most important thing: It doesn’t work. At least it doesn’t work for me. I’ve installed all the prerequisites, I’ve run their “installer,” and I can even get to the login page of “chef-server” but when I actually try to log in, it falls down and goes BOOM. I get a generic error page warning me about a socket error. I tried to diagnose it myself to no avail, there wasn’t anything in the log files because…
- Chef server truncates its log files willy nilly. It actually writes a fair amount of info to its log file, but you’d never know by looking at it after the fact, because after every request, it ends up as a zero-length file. Useful, huh? The trick is to ‘tail -F’ the file before restarting chef-server. This prints the output as it is written to the file, and reopens the file each time it gets truncated, which happens multiple times during the request.
- For what it is worth, I figured out what was wrong here, for some bizzarre reason, the hosts file on the machine was only readable by root, which casued lookups for localhost to fail when chef-server was trying to connect to the couchDB server.
Now, to be fair, the Chef site makes it clear in a nice green sidebar that Chef is young and a work in progress. I knew that when I started with it. I didn’t expect it to be production ready, but I thought it was far enough along to start working with. Clearly, I’m reconsidering that.
I’m also reconsidering the assumption that sent me to Chef in the first place, that it was desirable, at this point, not to take a dependancy on a specific Linux distribution by trying to build off of APT, the package distribution and management system at the heart of Debian and Ubuntu. The truth is APT is awesome. One of the reasons given for creating Chef was that Puppet, an earlier Ruby-based configuration management system choked on dependancy management. I haven’t seen that complaint about APT, not lately, in fact, that’s one of the things they love most about Debian and Ubuntu, people love it so much that say things like “I want apt to bear my children,” or words to that effect.
So, my thought is that I create my own apt repository. I’ll create derivates of the ubuntu packages I need custom versions of, and I’ll create configuration packages derived from their configuration packages whenever possible. Machine and role specific packages can be used to manage rollouts and/or I can different repository tiers for different classes of servers, in much the same way that Debian and Ubuntu have different tiers for testing, stable, unstable, etc. I’m sure I’ll run in to headaches on the way, but it least they will be headaches that other people have suffered and I can learn from their experience.
Pingback: A Tempermental Chef, or Something More APT for Configuration … | Linux Affinity
Well, I’m sad to hear of your issues with Chef, but a little learning and fact finding on your behalf wouldn’t have gone amiss.
Debian and Ubuntu packages have been released
OpenID is getting removed in the next version in favour of keypairs (although you can setup your own OpenID internal provider)
The issues you faced with logging in must be entirely down to your infrastructure or lack of knowledge. Chef is very choosy about using FQDNs (as it should be, that might have been your issue).
After testing out every configuration management tool under the sun for the past 6-9 Months, I can honestly say that Chef is by far the best.
Horses for courses, but I think your analysis of the software is hampered by your lack of understanding.
Good luck with the other tools if you try and use them!
Josh also took the time to email me. I’m republishing it below because it goes into a lot more depth than his comment above:
Hello Erik!
I’m sorry to hear you’ve had some trouble getting up and running with Chef. I work for Opscode, and I’m primarily responsible for helping people implement Chef in their environment. Let me address some of the concerns you brought up in your blog post.
Documentation, installation: First off, I’m responsible for maintaining the documentation and making sure the installation procedures work as broadly as possible. Obviously I can’t test every single environment possibility, but I try to reach the most common.
IRC help: We might have missed your queries! Please don’t be discouraged, we try to be as helpful as possible, but if you’re not seeing any response on IRC, please post to the Chef mailing list[0]. Be sure to include detail about your environment and relevant configuration and logfiles.
Ubuntu packages: The Debian/Ubuntu packaging process is detailed (which is why APT is so awesome!), it is a lengthy process, and it is difficult to get new packages into current releases. Now that Chef is in Ubuntu Karmic, it will be easier to get Chef packages into the other releases of Ubuntu and Debian through backporting and syncing. In the meantime, you may be interested in the APT repository where we host Chef and its dependencies[1].
RubyGems: When using the Debian/Ubuntu rubygems package, binary programs aren’t copied to /usr/bin like with stock RubyGems, and instead live in /var/lib/gems/1.8/bin, which is not in the user’s default path. This may be confusing to some people. The packaged RubyGems also tends to lag quite a bit behind the current releases. We initially released Chef and associated software (Ohai, the Mixlibs) as Gems because for Ruby software, it is the most common distribution method, and works on any platform. Yes, Chef even installs on Windows :-).
Server log: This is resolved for the next release (0.7.10) per ticket CHEF-500[2].
Server doesn’t work: Please try IRC again, or post on the mailing list. Keep in mind that Chef relies on the fully qualified domain name (FQDN) being set correctly on the server and also on clients. An incorrectly set FQDN causes a number of common problems, including SSL errors in the Apache+Passenger server stack.
Production readiness: We certainly think that Chef is production ready, it manages our own infrastructure :-). Several users in the community are also using Chef for their companies, including 37Signals[3], Engine Yard[4], and others.
I hope these explanations help. If you need further assistance, please stop by #chef on FreeNode, my nick is jtimberman.
[0] http://lists.opscode.com/sympa/info/chef
[1] http://bit.ly/aptopscode
[2] http://tickets.opscode.com/browse/CHEF-500
[3] http://37signals.com/svn/posts/1872-nuts-bolts-configuration-management-with-chef
[4] http://www.engineyard.com/cloud-services/solo (uses chef-solo)
Hello!
Joshua here. Just to clarify, Joel had initially responded via comment, not me. I hope my email helped, though. If you run into further issues, please email the mailing list, or stop by #chef on IRC and we’ll help you get up and running.
Thank you!
Good points. Chef really isn’t there.. I’ve been using puppet since it started, (migrated off cfengine) and have loved it ever since. Done some extremely complex things, too. Dependency problems? Never heard that before.. are Chef people spreading FUD?
Hello Charlie!
On the dependency topic, we’re not trying to spread any FUD about Puppet. It is simply a different approach. As you are probably aware, with Puppet you have to specify dependencies between resources (or with 0.25.0, classes). The difference with Chef is it processes recipes in the order specified. This is covered in the FAQ and on our blog. I believe the config management panel at OSBridge earlier this year covered the topic as well.
http://blog.opscode.com/2009/01/9-things-to-like-about-chef.html (Dependencies, Ordinality and Expectations)
http://wiki.opscode.com/display/chef/FAQ (how is it different than puppet)
Chef actually uses the native package system (apt, rpm, etc). So Apt isn’t a replacement for Chef.
Some jobs “could go either way”. For example, let’s say you want to change the default Apache config. You could use Chef to fix the config at runtime, or you could package up a new Apache with a fixed config. Either way is sane.
But let’s say you manage virtual hosts, and you have to get the hosts from the database and update various Apache config files. Of course any admin would start writing shell scripts.
But down the road, you’ll find a mess of shell scripts with implicit dependencies. For example, today you happen to need to run 3 scripts, and each one needs to restart Apache. How do you make sure Apache gets restarted, but make sure it doesn’t get restarted 3 times? Or how do you document the ordering and dependencies of your scripts, especially those that aren’t run often? Chef deals with issues like that.
(On the other hand, I think people should start with Chef Solo [because that’s the hard part — learning the ‘language’ and writing recipes]. Later, install Chef client/server so you can centralize both your cookbooks and your configuration. Then you can do really neat tricks like “find all the databases on all my mysql servers” or “what version of perl is installed on these servers?”)