Monthly Archives: October 2011

Request Tracker, migrating from RT 3.8.0 to RT 4.0.2

We’ve been using Request Tracker for tracking requests, incidents and resolutions for years now. A few days ago, we decided to migrate our RT server to a virtual machine so we can retire the old machine. We decided to upgrade to RT 4.0.2 as well.

The upgrade process became complicated because we decided to keep our old data. We managed to upgrade successfully but not without roadblocks.

These are the 2 major issues that we encountered:

1. RT 4.0.2 won’t start because of an issue with the Attachments table, wrong type:

fix: alter table `Attachments` change column `Content` `Content` longblob;

2. Once RT 4.0.2 is running, Perl code in Templates are not parsed properly. Our server ended up sending mails like this:

Greetings,
This message has been automatically generated in response to your request regarding
{$Ticket->Subject}

There is no need to reply to this message right now. Your ticket has been assigned an ID of
[rt.wyrls.net #{$Ticket->id()}].

Please include [rt.wyrls.net #{$Ticket->id()}] in the Subject line of all future correspondence about this issue.

Thank you,
{$Ticket->QueueObj->CorrespondAddress()}
————————————————————————-
{$Transaction->Content()}

The resolution email was much worse 🙂

Request Tracker was nice enough to give us some hint on how to fix it though. These errors keep on appearing on our apache logs:

[Tue Oct 18 14:48:46 2011] [warning]: Use of uninitialized value in string eq at /home/rt/sbin/../lib/RT/Template.pm line 410. (/home/rt/sbin/../lib/RT/Template.pm:410)

After reviewing the Template.pm module, we came up with a fix.

fix: update Templates set Type = 'Perl';

Advertisements

Munin plugin – MegaRAID HDD temperature using MegaCLI

Munin Exchange approved my plugin recently. I submitted it for approval a few months ago that I already forgot about it. The plugin is written in Bash and it graphs temperatures of HDDs attached to a LSI MegaRaid controller.

It uses the serial numbers of the HDDs as labels:

Most of our servers, circa 2008+, uses LSI cards especially our Supermicro blades. So if you’re using LSI cards as well, check it out.

UPDATE: Munin Exchange is down. They’re moving to github so the links above are not working anymore.

UPDATE: I moved the code to GitHub. Just follow this link.

ganeti and KVM Virtualization

We’ve been using KVM Virtualization for almost 2 years now and we’re happy with it. But as the number of hypervisors & VM instances increases, so is the complexity of server management which can be frustrating at times.

I realized that we have to find a way to manage it somehow.  I’ve been scouring the net for possible solutions. I’ve read about OpenStack & Eucalyptus but the disparity of deploying VM instances against our current deployment is big that migrating one will be difficult.

I have 6 requirements for the target platform:

  1. cost
  2. centralized management
  3. learning curve / ease of deployment
  4. migration constraints (lesser, the better)
  5. performance / high availability
  6. community support

My boss forwarded me this blog about ganeti a few months ago. I was skeptical to try it at first because deployment was debian-centric. We’re using CentOS so that could be a problem. But after reading the documentation + mailing-lists, I realized that migrating to ganeti will be less painful than other solutions (in theory), so I decided to install a test cluster and ran it for a few weeks.

Testing phase is over and ganeti is promising (drbd + live migration rocks!). Our current cluster has 5 nodes but that will surely change as we go into full production 🙂