Viraj's Weblog

I always thought this song was a good job resignation song.

I setup a load balancing Apache setup to log its access log to MySQL with mod_log_sql. This module works nicely. Check it out.

Well after doing alot of work with cfengine, I've come to the realization that the convergence methodology of system administration is the best one. That means many systems slowly converging to a desired setup, and staying that way.

One of the major hassles with sysadmin is making sure servers that should be identical, are actually identical. In terms of software and libraries installed, processes running or not, etc, any sort of divergence is usually a problem.

It takes alot longer to 'administrate' this way, but the extra time you put in at the beginning pays off. I've been setting up a group of machines for a client with cfengine, setting up packages and making configuration changes as needed, all from a central location. I'm not quite to the point where I don't have to even login to other systems, but for many tasks I don't. What's also great is I was able to test how well my 'converging' setup worked. A new server was brought online, and needed to be just like the others. After installing cfengine and letting it go, it worked pretty good, with only minor issues.

I believe we need to move away from fixing systems and instead work on making systems fix themselves. It's more than just automation, it's predicting failures and coming up with solutions. cfengine is a step in that direction, towards an immunological system applied to computers. When you start talking about hundreds of servers, you don't want to ever have to login to any of them. The more 'intelligent' I can make the servers I setup, the better.

In fact I am almost against even setting up monitoring for them. I don't want to be paged or even notified about some server Apache process that died. I want it to restart the process. The less I know the better . We sysadmins need to stop being mechanics and start designing systems that react to change and fix themselves.

Ahh, but we don't live in an ideal world. Anyhow, any steps that can be made towards this goal I think are good ones. cfengine is really cool in its ability.

I recently posted a question to the cfengine mailing list, and seemed to have started a storm about whether a system's hostname should be a FQDN. I did however get my question answered in all that hoopla.