I recently gave a talk at the Web 2.0 Expo giving an overview of how (and why) we build automated infrastructure.
I’ll be giving a more technical version of his talk (titled “Building an Automated Infrastructure”) at the Velocity Conference, June 23-24 down in Burlingame, CA.
Velocity is going to be the best place to send your Operations people this year: they’ll be able to get all the best advice from some of the best operations folks on the planet, all in one place. (There is a discount code at the end of the slide deck!)
Hope to see you there!
Maybe I should ask this one more time… and why are people paying in excess of 3 to 5 million a year for this kind of software?
I think the answer revolves around two things.
People don’t realize that manual systems administration is a problem until it’s too late.
By “people” I mean everyone involved… many excellent systems administrators just don’t see the need for it. (“Why would I spend two months building out a configuration management infrastructure when I could just spend a week building my current systems by hand?”)
The result of this is that automation comes in too late in the game, after you’ve already got a couple of hundred (or couple of thousand) systems being managed by hand, huge teams of systems administrators, and entire support structures built around doing it by hand. If you’re a growing startup, this winds up impacting your agility and stability as a platform, and you maybe start looking for tools like Puppet to help out. You talk to companies like mine to help, or at least talking to Luke’s own Reductive Labs to help get you running quickly.
If you are the Fortune 1000, it’s another thing altogether. You start to look for a way out, and someone tells you that a software package and 3 to 5 million in licensing and consulting will get you to the end game. In this case, it’s hard to be nimble enough to realize that Open Source provides a better path here.
You have to see it to believe it.
If you ask anyone working in technology whether they want the ability to rebuild the entire infrastructure, from bare metal, in an afternoon the answer is yes.
What follows, though, is “but our infrastructure can’t work that way because of X”. Or “your systems must not be very complex.” As of today, if we took a poll, the majority of enterprise grade IT environments don’t have this kind of functionality. They’ve never even seen it. So they believe you when they say it, but they don’t have any confidence that it will really work out that way for them.
When you realize you must have it, and you already believe that it’s impressively difficult to do, having someone say it will only work with a 3-5 million investment and a fleet of consultants actually affirms what you already believe.
Your own experience tells you it is that hard, and it ought to require all of that investment. You want to believe it.
Lets hope that the growing movement of practical, real, automated infrastructures being built on Open Source software starts to change that perception.
In addition to lots of insightful commentary on how Puppet is constructed, and a nice compare/contrast with how Tivoli is built (and you would be hard pressed to talk to someone who knows more about how Tivoli is built than John Willis, I expect,) there is also a section about iLike and HJK:
Uncomfortable with his recent celebrity at conferences, Luke told me that he has difficulty measuring his successes because he has his head so deep in the development and services of Puppet. One of his better success stories is with iLike.com, a website that allows users to download and share music. When iLike created one of the first Facebook applications, it grew from about ½ million users to over 6 million in a week. Luke, being the entrepreneur that he is, asked how iLike planned to manage that growth. He discovered that a services company in Seattle was managing iLike.com’s infrastructure build out using Puppet. In fact, one of the owners of that company told Luke that he makes a healthy living installing Puppet. Luke admitted that he felt feel pretty good to know that other people can make a living from his product.
Well, Luke, we feel pretty good about the fact that we make a living with your product too. :)
Puppet enables us to get a huge jump-start on building automated, scaleable, easy to manage infrastructures for our clients. Using puppet, we:
- Automate as much of the routine systems administration tasks as possible.
- Get 10 minute unattended build times from bare metal, most of which is data transfer. Puppet takes it the rest of the way, getting the machines ready to have applications deployed on them. It’s down to two and a half minutes for Xen.
- Bootstrap our clients production environments while building their development environment. I can’t stress how cool this really is. Because we are expressing the infrastructure at a higher level, when it comes time to deploy your production systems, it’s really a non-event. We just roll out the Puppet Master and an Operating System auto-install environment, and it’s finished.
- Cross-pollinate between clients with similar architectures. We work with several different shops using Ruby on Rails, all of whom have very similar infrastructure needs. By using Puppet in all of them, when we solve a problem for one client, we’ve effectively solved it for the others. I love being able to tell a client that we solved a problem for them, and all it’s going to cost is the time it takes for us to add the recipe.
Puppet, today, is a tool that is good enough to handle the vast majority of issues encountered in building scalable infrastructures. Even the places where it falls short are almost always just a matter of it being less elegant than it could be, and the entire community is working on making those parts better.