[personal profile] tara_hanoi
Over the past 2 months, I have occasionally found myself spending a lot of time working in the server labs in work. The job was re-racking machines; that is, taking machines from one rack and placing them on another. The reasons for this were daft enough as they were, so I won't go into it. However, I have learned a few lessons along the way. Some of these were hard learned, given that my job doesn't normally require me to spend much time in there.

I will dispense this advice now (Not in the form of Baz Luhrmann's Sunscreen Song):

Firstly: This is Murphy's Domain - Murphy's law applies more than usual when you're in the server room. But like all instances of Murphy's law, there are a certain amount of pre-emptive countermeasures. I guess this is the core rule of it all. Once you understand that all the tips after this are merely real-life examples of such countermeasures, life gets much easier.

  • The Server-room Gods ALWAYS DEMAND BLOOD - I'm not joking. If you physically work with machines, you will get your battle wounds. If it's heavy machinery, you might get a finger bruised, or if you're working inside a machine case, you'll get cut. This is inevitable if you're working with machines for any length of time. Don't take it personally, it's your tribute to the Server Gods. Instead, prepare by getting some anti-septic wipes and plasters. If you happen to know of any good any bruising remedies, send them on. I think my entire team would appreciate it by now.

  • TAKE BREAKS - I cannot stress this one enough. If you haven't read Zen and the Art of Motorcycle maintenance, do so now. Depending on your views, you might consider it to be a pile of pretentious wank; I know I did. However, if you read it, you will come away with what Pirsig calls, "Gumption". Gumption basically refers to your mental state. If you're in a bad mental state you'll take the bad stuff more personally, and it will affect you more, leading to more screwups; you see the cycle here.

    Remember, a server room is not a hospitable environment. It's like you're flying in a really loud plane, so much so that you have to wear ear protectors (not doing so is a dismissable offense in my company). For me, that makes me tense from the get-go, never mind having to shout extra-hard so people can hear me through the cans (my voice has gotten huskier these days). Anyway, tensions fray, so have set points you can walk away, and just get yourself back into a good headspace. Personally, I think the best way is to set various wall-clock waypoints, and walk away before you start feeling tense; it's easier to prevent getting into that state than it is to get out of it.

  • The optimum number of hands on a job is probably 3 - Originally, my estimate was that you really need 2.5 people in the lab. I've since reviewed that, as early in the project we got under each other's feet, and there wasn't enough solid work for all 3 of us to do. Even with 2 people, there are too many hands, but the job is always too much for one person. If three people do end up in the lab, get 2 people who work well together and set them on a task, while the third member scouts ahead and prepares the scene for the next task (even if it's something as pithy as putting washers on the rack screws because it's going to be finicky if you try and put them all on in a hurry).

  • DON'T GO IN THERE ALONE! - Obey the cardinal Horror Movie rule: Never go in there alone. It's ok if you're just checking the status of a machine, but if you're going to be moving stuff, it's best to have a second person (see the rule about needing 3 hands).

    One time, after mounting a nice little 2 unit server on its rail kit, I decided to go into the lab alone and check that it slid out and in properly. The problem was, it was 5pm, and the server wouldn't slide back in. In the end I managed to prop it up, and then call a colleague to help me detach it; had he not been there, I would've had a major problem.

  • Know the style of your colleagues - When I was an intern (in the same group), I worked with a guy who was our main lab-head, and I helped him out whenever he had to do work. Neither of us were physically strong, meaning that Brute Force and Ignorance was rarely an option so we tended to try thinking and planning what we wanted to do. That guy's since left (and it's a pity, he was a good guy), and I spent most of my time working with 2 guys who are physically strong, and have always had the privilege of that strength; brawn-over-brains was a strategy that generally worked for them. As a result, there was a fundamental mismatch. While I was still thinking about how to readjust something or if we even needed it to happen, they were already stuck in, doing it. It was frustrating for all sides, especially as I'd be a little ungracious saying, "Told you to wait".Fortunately, there was a 4th team member who had similar hours to me, was more of a brains-over-brawn type; we worked well together (especially when I bossed him around).

    Basic point is, get to know how your colleagues work, and act accordingly. In my case, it was let the two brawny guys do their work, and then myself and the 4th would go in and think our way through stuff.

  • NO SHORTCUTS! (aka "Patience is a Virtue" aka "Premature Optimisation can be an embarassing little problem") - Larry Wall might believe that the 3 great qualities of a programmer are laziness, impatience and hubris, but that's not useful in the lab. I learned a good while before that trying to save yourself effort will cost you more time.

    For example, 3 of us were working on putting some storage devices on shelves (they didn't have rail kits that would fit in the new racks so we had to put them on shelves). The storage devices themselves were about 4 rack units high (let's assume they were 4u just for simplicity). So myself and brawny #2 screwed in the first rack while Brawny #1 unscrewed and brought over the storage device. In the meantime brawny #2 was bored and decided to screw in the rack for the second storage device 4 rack units above where the first shelf was screwed. The minor problem was that the shelf itself took up space, so we couldn't fit the device into the first gap. So we left brawny #2 holding the storage device while brawny #1 and I move the shelf up a unit. The device was still difficult to get in, as we didn't have as much lee-way, thanks to the second shelf, so it would've been better if it wasn't put in at all.

    Basically, don't solve problems that haven't happened yet, you could easily find yourself meticulously undoing all your clever, time-saving work. Just do it right when you need to.

  • Communication is key - This is probably most people's Golden Rule. I'm putting it at the bottom because, even though it's as important as the others, the others generally get less air time in general (even though morale and patience are almost always important). But do get talking about what you plan to do, because in an environment like this, you want to have a clear idea of what you want to do before you do it. You know "Measure twice, cut once" and all that jazz... and making sure all both know what you're measuring, and how.

I'm sure I've worked other little rules, but I think these are the main ones (namely: Morale, Patience, Team awareness and Communication - same ingredients for any situation, but I really just wanted to kvetch and tell stories about how it applies in this particular domain). I'll probably add more ideas and stories if I think of them.



September 2015


Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Sep. 21st, 2017 03:27 am
Powered by Dreamwidth Studios