Amazon's S3 provides unlimited, relatively cheap storage, accessible via HTTP (using REST/SOAP). This is ideal for personal offsite backups. The barrier to entry is low as are the costs.

Sun's ZFS turns everything you know about managing filesystems on its head. When I recently upgraded my storage array, I decided to load my machine with extra RAM for ZFS rather than buying a dedicated RAID card. One of the best features of ZFS is the low-cost snapshots. You can snapshot a filesystem, and said snapshot will not take any space on disk until the original is modified. To put it another way, the snapshots only store the (block-level) diffs.

The icing on the snapshot cake is the ability to send a snapshot as a stream. This can be piped over the network (i.e. ssh) or simply output to a file (then bzipped and uploaded to S3!). ZFS also allows you to send a differential of two snapshots.

To put this into action, I have written a script which will create a snapshot, bzip it, encrypt it and upload it to S3. I use a threshold to determine whether I should upload the full or incremental to save space/bandwidth (and time, as my cable upstream isn't that great...). To interface with S3, I'm using s3tools.

I realized afterward that s3tools have GPG encryption built in, but I think it is simpler to use openssl and a passphrase for this use-case.

Here's the script:

#!/bin/sh
# 
# Copyright 2010 Richard Kolkovich. All rights reserved.
# 
# Redistribution and use in source and binary forms, with or without modification, are
# permitted provided that the following conditions are met:
# 
#    1. Redistributions of source code must retain the above copyright notice, this list of
#       conditions and the following disclaimer.
# 
#    2. Redistributions in binary form must reproduce the above copyright notice, this list
#       of conditions and the following disclaimer in the documentation and/or other materials
#       provided with the distribution.
# 
# THIS SOFTWARE IS PROVIDED BY Richard Kolkovich ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL Richard Kolkovich OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# 
# The views and conclusions contained in the software and documentation are those of the
# authors and should not be interpreted as representing official policies, either expressed
# or implied, of Richard Kolkovich.
# 

ZFS=/sbin/zfs
BZIP=/usr/bin/bzip2
OPENSSL=/usr/bin/openssl
MD5=/sbin/md5
S3CMD=/opt/s3cmd/s3cmd

BACKUP_DIR=/tank/backup
TEMP=$BACKUP_DIR/tmp
PASSFILE=$BACKUP_DIR/.password

if ! test -d $TEMP; then
    mkdir $TEMP
fi

# Backup a given zfs store
# arguments: name, zfs fs, threshold (0 to always use incremental), S3 bucket
backup() {

    NAME=$1
    FS=$2
    THRESHOLD=$3
    BUCKET=$4

    # incremental 
    SNAP=$FS@incremental
    SUFFIX=incremental
    $ZFS destroy $SNAP
    $ZFS snapshot $SNAP
    $ZFS send -i $FS@full $SNAP > $TEMP/$NAME

    export BLOCKSIZE=1024
    SIZE=$(( `du $TEMP/$NAME | awk '{print $1}'` * 1024 ))
    if test $THRESHOLD -eq 0 || test $SIZE -gt $THRESHOLD; then

        rm $TEMP/$NAME
        $ZFS destroy $SNAP
        SNAP=$FS@full
        SUFFIX=full
        $ZFS destroy $SNAP
        $ZFS snapshot $SNAP
        $ZFS send $SNAP > $TEMP/$NAME

        # a new full invalidates old incrementals
        $S3CMD del s3://$BUCKET/$NAME-incremental.bak
        $S3CMD del s3://$BUCKET/$NAME-incremental.bak.md5

    fi

    # compress
    $BZIP $TEMP/$NAME
    FILE=$TEMP/$NAME.bz2

    # encrypt snapshot
    $OPENSSL enc -aes-256-cbc -salt -pass file:$PASSFILE -in $FILE -out $FILE.bak
    rm $FILE
    FILE=$FILE.bak
    $MD5 $FILE > $FILE.md5

    # send snapshot to S3
    $S3CMD put $FILE s3://$BUCKET/$NAME-$SUFFIX.bak
    $S3CMD put $FILE.md5 s3://$BUCKET/$NAME-$SUFFIX.bak.md5

    # clean up
    rm $FILE
    mv $FILE.md5 $BACKUP_DIR/$NAME.md5

}

And I call backup() thusly:

backup "private" "tank/private" 52428800 "my.backup.bucket"

Done with Seagate

| 1 Comment | No TrackBacks

I've been a proponent of Seagate drives since somewhere around my second or third PC build. I think my first "high capacity" (13.2GB) drive was a Seagate; I then went the Deathstar route with a 40GB Hitachi/IBM followed by an 80 (which, apropos, arrived on the day the 40 died). After that, I actually thought about brand before buying hard drives. I think I stepped up with Seagate in the following order: 160GB, 320GB and dual 400GB drives. When my 400GB mirror became full, I decided to upgrade once again.

If it's worth building, it's worth over-engineering the piss out of. I decided to roll with 4x1.5TB drives in a ZFS raidz pool (RAID5-equivalent capacity). I knew that initial shipments in Seagate's 1.5TB 7200.11 series had firmware issues aplenty; these had been resolved by the time I decided to build this array (early August 2009). I was doubly re-assured by the drive's placement on the Maximum PC Best of the Best list (where it reigned, I believe, until the recent arrival of the WD 2TB Caviar Black). I pulled the trigger on August 4.

After basking in the glow of 4.5 (4.0, effective)TB of empty space and the awesomeness that is zfs, I moved the contents of my 400GB array over. Everything went as smoothly as could be expected. About two days after this, however, I awoke to a S.M.A.R.T. error on one drive accompanied by its subsequent detachment from my system. I quickly started the RMA process at Newegg only to have a second drive begin exhibiting the Click of Death (before I had shipped the other drive back). ZFS's raidz can tolerate one drive failure, so I immediately shut my machine down to stymie the impending doom of the second failing drive. I amended my RMA to be a refund for two drives and ordered two new drives with expedited shipping (makeshift advanced replacement). With my array once again happy, I continued on my merry way.

Before October's passing, it decided to leave me a gift: the dreaded Click of Death had begun on another drive. Being out of Newegg's 30-day zone, I RMA'd the drive through Seagate. The drive was still accessible, but since access was painful, I shut my machine down. I paid the $19.99 for an advanced replacement and return shipping label, reslivered the new drive into the array and was off to the races (again). On a side note here, I decided that I would sell my (original) Raptor when I built this array, assuming it would be fast enough for OS/applications.

Fast-forward to yesterday. December has given me an early Christmas present: ANOTHER FAILING DRIVE! As I wondered why my machine was running sporadically slow yesterday, I discovered while refilling my coffee once that the Click of Death had once again visited me (my new headphones are designed to block out noise, so I didn't hear it until taking them off...). I did my due diligence and ran the long test provided by Seatools; this is just a bootable S.M.A.R.T. agent. The long test is the same as the S.M.A.R.T. long offline test (which would not complete using smartmontools, as the drive would disconnect after extremely long seek delays). This morning, I called Seagate Warranty Support, determined to at the very least not pay $19.99 for an advanced replacement AGAIN.

I initially suggested to the customer service representative that Seagate refund my money for all four drives. I would be much happier buying four new drives from a different manufacturer, such as the WD Caviar Black. Obviously, this suggestion was dismissed. I battled this rep for a good 20 minutes to get the advanced replacement with no charge. Upon requesting a supervisor, she told me her supervisor was on another call but suggested I talk to tech support before they could approve my request. I wasted time talking to a helpful tech support rep. who agreed that any clicking warrants drive replacement. He made his notes on my account and transferred me back to customer service.

I was now speaking to a different rep. than I had initially. She read the notes on my account, again presented my two options (I ship, they ship or advanced replacement) without waiving the $19.99 fee. She told me she could not approve that, so I waited in the queue for her supervisor.

The supervisor was pleasant but worthless. He danced to the same tune - they could not send me an advanced replacement for free. I don't typically let my emotions get the better of me, so I was cool-headed but forceful with him. At this point, I didn't care about Seagate's obligations or policies. I was a loyal customer who had experienced a 57% failure rate in four months of ownership of a drive with an advertised annual failure rate of 0.34%; I wanted him to go out of his way to make me happy. The best he said he could do was to send me a shipping label to ship my drive in, free of charge. That would result in at least a week turnaround which is, to me, unacceptable.

Once I had become exhausted with attempting to garner a concession based on sympathy for my (hopefully) unusual experience with faulty drives, I turned to logic. I asked this supervisor what the difference was between giving me the advanced replacement for free and covering my shipping costs. He stated the $19.99 was their cost for the packaging materials and shipping via UPS, and that's "just how it is". I asked him to explain, logically, how it could be any different. He's offering to cover shipping both ways. They are going to provide their own packaging materials even if I send my drive first. He did not refute these points, but he still could not explain why the options were really any different cost-wise.

They aren't. The only difference between these options is shipping order. In both cases, Seagate was going to pay for shipping from them to me, shipping from me to them and packaging materials for the replacement drive. As this gentleman did not have a supervisor, I am now waiting 24-48h for the department he reports to (Seagate Global Customer Service or something...) to call me back.

I would also like to note that Newegg's packaging and UPS's shipping are suspect here. I was fairly pleased with the packaging - each drive was individually wrapped in a copious amount of bubble wrap and the box was filled with packaging peanuts, but various sources on the Interweb swear against Newegg when it comes to magnetic hard drives. I've seen this same dogma applied to UPS shipping, too. In the future, I will attempt to avoid both these potential pitfalls. My friend and colleague Stephen Shelton ordered four of the same drive from Dell a week before I did. They came in foam packaging designed for four hard drives (just like the first replacement I received from Seagate); he has had no problems with his drives. At the same time, "normal" shipping abuse should be considered by manufacturers - how else are these drives going to get to end users?

I'm beyond the realm of logic now. I've spent more time than the $19.99 is worth between being on the phone and not having my workstation running. Now, I fight a battle of principle. I was, even with this failure, willing to place blame at Newegg and/or UPS. Seagate has thus far shown me they don't care enough about retaining me as a customer and proponent of their products to concede a bullshit $19.99 fee. For this, they will lose my business in the future.

A Strange Language

| 2 Comments

While scanning and shredding all manner of paper record I have accumulated over the years, I stumbled across a poem given to me by an English teacher in middle or high school. There is no author on the print I have, and I could not find the poem online anywhere. To preserve its humor for eternity, here it is:

A lawless language is English;
The plural of box is boxes,
But more than one ox is not oxes.
One fowl's a goose, more are two geese,
But more than one mouse is certainly not meece.
On the contrary for mouse, the plural is mice,
While the plural for house is never hice!
If more than one man is always men,
The plural of pan should then be pen?
One may be that, and two may be those,
But the plural of hat is surely not hose!
Masculine pronouns are he, his and him.
Imagine the feminines she, shis and shim!
The English language, I think you'll agree,
Is as strange a language as ever could be.

If anyone happens to know the proper author of this, please let me know. Otherwise, enjoy what I feel is an accurate and humorous critique of the English language.

I run my own mail server using maildrop to deliver to multiple virtual LDAP users. My personal e-mail is sorted into many folders on the server as it comes in. To reduce my e-mail maintenance overhead, I decided to mark messages as read automatically in certain folders. This is useful for low-priority e-mails such as advertisements, coupons, etc.; I want to view these when I need them - not when I receive them.

It turns out that maildrop doesn't have any facilities built in to do this automatically. It does, however, provide you with the tools to roll your own:

cc "${MAILDIR}.offers/"
`for x in ls ${MAILDIR}/.offers/new/*; do mv $x ${MAILDIR}/.offers/cur/${x##*/}:2,S; done`
to "/dev/null"

Since the 'to' directive exits immediately after execution, you must cc then move. Delivering to /dev/null then stops execution of the filter.

Fun with Asterisk

| No Comments | No TrackBacks

I spend approximately 10-15% of my time wearing the IT hat for both my company and the company whom we are currently consulting. We are running FreePBX for both companies' phone systems. Our setups are relatively simplistic, so I have had little problems managing everything. The first real hurdle I ran into was the inability of Asterisk to allow the same extension to log on from multiple devices. I'm pretty sure this is not a SIP restriction, but I may be wrong. Either way, my goal is not to bash FreePBX or Asterisk.

Initially, I over-engineered a solution to this problem while simultaneously making it more difficult to administer and use. The setup required a ring group for each user's extension. This ring group contained an extension for each phone the user needed. To have a common voicemail box, there was yet another extension for voicemail. Convoluted? Yes, it was. It did work, however, for our use-case. We have three employees, and only one of us needed a multi-homed extension at the time. I set it up and forgot about it.

More recently, I set up another FreePBX system for our customer. At this point, I discovered the Follow-Me module. The original point of using this module was to allow calls to ring through a user's mobile phone. I also adapted my previous abomination to use Follow-Me, making for a much simpler setup. It was simpler, yes, but it was still a bit convoluted.

This system worked well until I was tasked with on-demand call recording. I determined how to enable this in a jiffy with some quick Googling. The problem? Access to these recordings was tied to the extension that initiated the recording.

To solve this, I re-visited "deviceanduser" mode in FreePBX. This is the route I should have taken in my earlier ventures - it decouples devices and users. A device can register via SIP to the server, and a user represents an extension and voicemail box. This can be enabled by setting AMPEXTENSIONS=deviceanduser in /etc/amportal.conf. Be sure to comment out the existing setting.

To enable on-demand recording, add 'w' to the dial command string and 'W' to the outbound dial command string. These settings are found in General Settings in the FreePBX UI.

Add the following to /etc/asterisk/features_general_custom.conf:

featuredigittimeout=3000
courtesytone=beep

This makes sure the PBX will register your keypress (waiting for up to 3 seconds) and beeps to confirm. Feel free to disable the beep.

To enable access to the recordings from the ARI:

ln -s /var/spool/asterisk/monitor/ /var/www/html/recordings/monitor
yum install sox

Finally, tie everything to the user. For every device a user has, add the user's extension in the 'accountcode' field on the device.

Edit /var/www/html/recordings/modules/callmonitor.module. Add:

OR accountcode = '" . $_SESSION['ari_user']['extension'] . "'

After the line:

OR dst = '" . $_SESSION['ari_user']['extension'] . "'

On my FreePBX, this was at line 614.

Now, all calls recorded from any device for a given user will show up in that user's ARI for download.

Goals

| No Comments

My last post promised talk of goals, and I had fully expected to write this post much sooner. Appropriately enough, one of my goals just so happens to involve blogging. While the idea may have been inspired by the beginning of a new year, I don't quite believe in New Year's Resolutions. I have made my fair share of these self-promises, and I don't think I've ever followed through on them. A large part of that can be attributed to lack of a plan of attack. Simply stating that I want to do something only gets me so far. It is important to have your goals clearly laid out with the steps you plan to take to achieve them. It is also important to have a realistic timeline for achieving your goals. This applies not only to duration but also to the starting point.

The number of books I've read since my last English class is appallingly low. I enjoy reading, and there are countless classic works of literature in the world I have yet to enjoy. I have been doing a good job keeping up with my subscription to Scientific American as well as keeping my RSS subscriptions in check, but I have barely touched the world of fiction or a technical book in quite some time. I don't enjoy reading multiple books at once, so I'll keep this goal to making some progress each week on one book for enjoyment and one technical book.

My second consumption-related goal is to pick up where I left off with my study of mathematics. I've always enjoyed math, and I would have liked to double-major (or at least minor) in it. My situation and opportunities dictated that I couldn't do this, but that does not make me love math any less. I picked up The Princeton Companion to Mathematics for a song when joining the SciAm book club, so this will be my starting point.

There is a lot of cool research being done by people with different skillsets than me. The best of this research shows up at TED. I have some TED talks downloaded already, but I'd like to make an effort to actually watch them. Watching at least one TED talk each month will suffice for now (some of them are quite lengthy...).

I love to play chess, but I've never really had a large pool of readily-available opponents. There is an upper bound on your skill acquisition when playing the same opponent(s) repeatedly. Luckily, there is GameKnot. I plan to keep at least one chess match going on GameKnot at all times. Want to play? Find sarumont and challenge me!

I'd also love to learn Go. I know the basic rules, but that barely gets your foot in the door. The rules are simple, and the strategy is endlessly complex. I don't feel I can allocate any time to Go presently, but it will be on the back burner. I feel like Go and chess can both help to hone my critical thinking.

I think bloggers can be separated into two camps: those who keep a schedule and those who do not. I sit firmly in the "those who do not" category, and I agree with this mentality. Keeping to a schedule will inevitably lead to more quantity and less quality. I want to blog more and regularly, but I do not want to sacrifice quality, decreasing the already abysmal SNR of the Internet. I think the solution here is to strive for a reasonable schedule. What's reasonable? For me, at least once per month should do. I would like to think that, in ideal circumstances, my non-blog contributions to the world are of greater value than the other way around.

That being said, I would like to have a blog which is a neutral force in the world, at the very least. My self-improvement motivations behind blogging are to improve my writing and vocabulary skills while purging certain thoughts and ideas from my overly full mind. Back to striving for a schedule, I think the big challenge is becoming your own editor. When you can say to yourself, "I don't have any new, interesting or useful ideas to share this month," you can preserve a high SNR. To these ends, I plan to try to post at least monthly by taking on a project of some sort which could result in a useful post at the end of said month.

My primary function in the world is that of a software developer. I happen to enjoy what I do, but I have not had the time or energy in the past years to sharpen my saw in my free time. Too many things interest me in the computing world, so I have no lack of interest. Working at a start-up does, however, drain your energy (both physical and creative). I want to start siphoning off a bit of that energy into writing code that is in no way related to work. I would prefer this code be using different tools and a different language. This will help me discover new tools and methodologies which I should be able to bring back to work, enriching our codebase with renewed vigor and creativity.

I know many geeks who share the problem I have: too many interests and/or hobbies. I enjoy cooking, baking bread, roasting/drinking coffee, brewing/drinking beer, cycling, and almost everything about computing. Even before my lovely wife, there were not enough hours in the day to devote to all these hobbies. I'm now roasting my own coffee beans, brewing my own beer, baking my own bread and cooking most meals. These hobbies detract precious time from computing, and I feel that is where my energies are best spent when it comes to paying the bills.

With the exception of my most recent (brewing beer), these tasks have become well-learned and routine to a point. The next step is one that, I think, many people do not take: optimization. With my hobbies, there are two forms of optimization: optimizing the task itself and optimizing the task's integration into life. I think that optimizing the integration promises the better ROI, so I will be focusing on that shave time away from these hobbies to apply to my goals.

Now this evil meta-post is out of the way, it's time to move on to productivity. Until next time.

I was recently in Dallas to assist in an IT weekend. My company is in the process of making ourselves as agile as possible. This process involves liquidating everything possible - including the office. We're getting rid of our primary data center (and moving our customers to their own hosting) and making our secondary data center our only - and utilizing it for corporate IT.

This past weekend, we trekked to the data center and proceeded to give our rack a cleaning it so desperately needed. Now, everything is organized quite nicely. Part of this restructure was to be the movement of personal servers to virtual machines. I had started building my VM, but we were not planning on migrating just yet. When I hooked my server back in, it did not boot. I also did not have a CT/2 cable with me, so I took the server out. I got the VM (and my server) up and running the next day. Lesson learned: NEVER buy a server with a proprietary KVM cable.

Tuesday night, I decided to reboot my desktop to remove the region lock on my DVD drive. Some friends of mine had a British DVD they could not view, so I wanted to get at it. My desktop decided it did not want to power back on. I had seen this issue a few times before, but it has passed quickly in the past. This time I was not so fortunate.

Wednesday morning, my machine is still not booting. I do a bit more research on the issue and discover a Supermicro FAQ describing this issue. Supermicro's response? They only support Supermicro power supplies powering their motherboards. Wonderful. I had been talking about an upgrade anyway, so I have fresh parts from Newegg arriving tomorrow...

Today I read a post on Joel on Software regarding goals. I've felt the need for setting some goals recently. Moving to a home office has been great, but I have felt like I am wasting a lot of time. I've really been missing goals. I, therefore, plan on setting some goals for myself for the year and beyond. I've scribbled down a few things which have come to mind, but I want to put a decent amount of effort into it.

I look forward to building my new box tomorrow - unboxing photos should be up this weekend - and orienting myself toward a better me this year. Stick around; it should be fun.

I was forced into Williams-Sonoma last year, as my wife got a gift card for me for Christmas. I had always avoided the store like a dieter avoids candy; I knew I could not be trusted in such a place (I have, however, restrained myself every time I've been since). With that gift card, I purchased a pasta machine. We've been making fresh pasta about 60-70% of the time since.

The instructions for the pasta machine noted that Semolina flour was traditionally used for pasta making. Semolina is made from durum wheat. The flour itself is a bit coarse (compared to white flour) and hard. This makes for a hard time kneading. The last time I made pasta, I was EXTREMELY frustrated, as the 2 servings took me about an hour and a half to make. Last night, I took matters into my own hands. The pasta pictured was made with a 1:1 ratio of semolina to white flour. That made the dough MUCH more pliable and easy to work with. The recipe I used was:

  • 1c semolina flour
  • 1c white flour
  • 2 eggs, beaten
  • 1t fine sea salt
  • 2T extra virgin olive oil
  • water, as needed

Mix dry ingredients and form a well. Pour beaten eggs into the well and work into the dry ingredients. Work olive oil in. Continue to knead, adding water to get to a good consistency (it will be a bit sticky). Follow your pasta machine's instructions for making the pasta. No pasta machine? Roll out to the desired thickness and use a knife to cut the desired width.

Next time, I'll revise that a bit and convert the inaccurate volumetric measurements to weights. The yield was just under a pound. I cooked 9oz., though 8 would probably have sufficed for the two of us.

I've been working on my alfredo for awhile, and I just started writing everything down within the past couple months. I have a Moleskine dedicated to the kitchen, and I take note of everything I create, keeping track of successes, failures and constructive criticism. I'm pretty happy with my basic alfredo:

  • 1c heavy cream
  • 2.5T butter
  • 3oz. grated Parmesan (reggiano, if you can)
  • 1.25oz. grated Asiago
  • a couple drops of extra virgin olive oil
  • a dash of salt and white pepper (be careful with the salt - Parmesan is a salty cheese)

Melt the butter and olive oil over medium-low to medium heat. Add the cream and bring to a simmer. Whisking constantly, start adding the cheeses a little at a time. Add more or less of each to taste. Whisk constantly until a silky consistency is reached.

The chicken? I just sliced a boneless, skinless breast and added salt and pepper. I then pan-seared it in olive oil in an iron skillet over medium to medium-high heat.

It's garnished with some dried parsley and served with some homemade focaccia au levain.

Fettucine

When IT goes awry

| 1 Comment

Information technology (IT) is a field born of increasing complexity and choice in computer technology. Before computer science was considered a discipline, physicists and mathematicians programmed computers. Before there was IT, computer scientists managed hardware and software; they even built their own hardware at some point. Now, IT is a loosely defined discipline which ends up, in the business world, governing and managing choices having to do with technology.

According to Merriam-Webster, technology is the practical application of knowledge especially in a particular area. Throughout history, technology has been incorporated into society as its usefulness becomes apparent. The plow made farming much easier, so farmers began using it. The plow evolved, becoming better with each iteration. The plow is relatively simple next to a computer, but its path to ubiquity is one the computer and its ilk are following.

Following the plow example, who is focused on the plow? The farmer most certainly is not. As long as the plow does its job, the farmer could care less about it. His goal is to produce as much as he can with the highest possible efficiency. The plow is a tool. It enables him to farm and increases his efficiency, thus providing value. The farmer's customers don't care about the plow. They want to buy or barter food from the farmer. How he produced the food is of no concern to them. This leaves the plow manufacturer to care about the plow. The plow manufacturer thrives by selling plows; she cares very much about the plow. But does she use plows to make other plows?

When a company reaches a certain size, the task of managing technology requires a dedicated resource; the IT department is born. This size varies from company to company, depending on that company's specific needs, but the underlying trend is there. We'll assume here that the company in question is Spacely Sprockets, so they do not, in fact, sell any form of IT-related services.

What is the role of technology at Spacely Sprockets? It is a tool. Technology enables them to more efficiently and effectively produce sprockets. In this case, the best IT department is an IT department who is never seen or heard from. Ideally, Spacely's IT department has configured their systems to be completely invisible. Computers never crash, phones never disconnect and printers never make archaic requests such as "PC Load Letter". Sound too good to be true? It is. Though this level of perfection can probably never be attained, it can be approached.

On this sliding scale of IT perfection, there is a point where IT becomes inhibitory of the company's business. I fear many companies in the world are at or below this point. Why? I believe incompetence and arrogance account for most cases.

Incompetence (or its brother, ignorance) can cause solutions to be implemented poorly or not at all. Part of the job of an IT department is to recognize where technology could improve business in any way. If no one in the IT department is able to recognize the need for a given technology, it's unlikely anyone will. These flaws can also yield half-baked policies which end up inhibiting production.

Arrogance is the lower volume offense here, but it can be just as damaging. One case of this type of behavior is the technique of building a fortress. Ever known someone whose job security was that no one knew how to manage what they had built? That's a fortress. Did this person build the fortress with the sole intention of keeping people out? Not necessarily. There are circumstances where some constrained resource (people or time, usually) can limit transparency. I'd say these situations should at least yield a wiki page in the corporate intranet about the system (don't have a corporate intranet/wiki? see the previous paragraph...). Arrogance, in some form, can once again yield inhibitory policies. These are typically knowingly inhibitory, meant to draw attention to the power held by the IT department.

What can be done about these problems? Well, to some extent, these are character flaws. Those in charge of hiring can screen the extreme versions of these people very quickly. Once they are in power, though, those in charge of them must have the ability and willingness to deal with potential problems as they are identified. The worst case scenario has an arrogant IT department gone rampant with a CEO who will not lay down the law. IT is just another department. If your sales team isn't producing, you take action. If your IT department isn't producing, you need to take action. IT is an integral part of your business but should not control it.

Hello, world

| No Comments

The signal-to-noise ratio of the Internet at large is far too low already, so I do not plan on wasting my time and yours with useless regurgitation.  The topics I plan to discuss (in order of frequency/importance to me/knowledge) are:

  • programming (general and project-based)
  • software/technology in general
  • baking (breads, mainly)
  • cooking
  • aquaponics (indoor, small-scale)


This list is not exhaustive by any means.  My overarching goal here is to give something useful to the world while growing personally and professionally (and having more to give back as a result...).  One very important thing I've learned while writing software is the requirements always change as you get to know your problem space.  I know very little about blogging, so I am certain my habits will adapt as I get into this.  Thanks for visiting.