Best Recruiting Email Ever

I just received the greatest recruiting email of all time. This is for a job with a large software company in Redmond, WA. I kid you not. The end of the mail enumerates the requirements as follows.

Needs to have familiarity with:
  • Python and/or Ruby Experience
  • Chef and/or Puppet Experience: Desire persons whom have been exposed to programmatic automation
  • CCNA level network knowledge: Desire network understanding of subnetting, network management protocols such as SSH / SNMP / serial console interactions
  • Linux experience
  • Knows how to use either VI or Emacs editors
If interested, please reply with current Word resume. Thanks!


Wow... Just wow. Bonus points for the incorrect usage of "whom". But at least they're trying. My Pyrrhic victory over Microsoft will be complete when I do not have to ever even contemplate giving them money, directly or indirectly, in order to use software unrelated to them. Progress has definitely been made! The world is definitely a better place. Who knows, once that situation exists, maybe I will work for them.


Fortunetelling Like An Oracle

It starts out with an email from a friend saying that they are looking into acquiring a storage server for their science lab.

Date: Mon, 1 Mar 2010
From: A Friend In That Lab
Subject: Fwd: Fwd: server config pricing spreadsheet
To: Chris X Edwards

We're looking for recommendations for something you know a lot more about than I do, so if you have time, I thought I'd ask what you think...

---------- Forwarded message ----------
Subject: Re: Fwd: server config pricing spreadsheet
To: PI
From: Someone The PI Knows Who Knows-About-Computers
Cc: A Friend In That Lab, Person Responsible For System Administration

Hi folks

I like the x4440 the best, it seems to be the best value. Nothing wrong with the x4275, though the extra power would be good.  If you did purchase the x4275, I would recommend and even number of drives (4 or more) and configure with a RAID 10. [Blah blah blah]

[A bunch of talk about various server options which is uninteresting.]

I offered to go to their lab meeting and tell them my perspective on storage servers. This offer was accepted and I met with this entire group (PI, PRFSA, et al) on March 16, 2010. I explained to them the
McNett Storage Server Architecture, build it yourself from components that are easily replaceable and if in doubt, buy two and put one on the shelf. And, of course, use Linux.

In the previous couple of months I had just built such a storage server of similar capacity. I showed them my notes on that specific build. They were looking to buy a used Sun machine. With storage capacity and RAID configuration similar to my box, their prospect was, despite being used, about three times more expensive.

I clearly told them that the storage server they were looking at would be a proprietary machine which would limit their responsiveness to problems. I enumerated all the ways that storage servers fail and
addressed how the McNett design accommodates those failures. By having easily obtainable cold spares of anything that could fail on hand and ready to go you greatly reduce your dependence on uncontrollable entities in keeping the machine running.

They were very skeptical, instead appearing to favor the apparent greater security of Sun's very reassuring warranty (which this machine apparently would be "covered" by). I explained my experience with warranties and how big companies are more motivated to make the
service seem good before the sale than after.

In March of 2010, I basically described the exact course of events which ultimately took place 3 years later after they bought the Sun...

Date: Jan 2013
From: Person Responsible For System Administration
Subject: [admin-mailinglist]  Spare hard drive for a Sun Fire x4540?
To: admin-mailinglist

Hi everyone.

I realize this is a long shot, but does anyone have any spare hard drives for a Sun Fire X4540 ("thor") ?  We currently have 2 failed drives and are waiting on replacements from Oracle, but they are backordered with no ETA.  If one more drive fails, we could lose all of our data, so naturally I'm very concerned.

Date: Jan 2013
From: Helpful Sys-Admin
Subject: [admin-mailinglist] Re: Spare hard drive for a Sun Fire x4540?
To: Person Responsible For System Administration

What size?

If you have a hardware support contract, Oracle is bound to specific response times (2 hours onsite for "Premier").


Are they not honoring that?
Date: Jan 2013
From: Person Responsible For System Administration
Subject: [admin-mailinglist] Re: Spare hard drive for a Sun Fire x4540?
To: Helpful Sys-Admin

It's a 1 TB drive.  Oracle has responded, but only to tell me that the drives are backordered.

Date: Jan 2013
From: Helpful Sys-Admin
Subject: [admin-mailinglist] Re: Spare hard drive for a Sun Fire x4540?
To: Person Responsible For System Administration

Sorry, our thors have 500GB drives.

Oracle's response sounds unacceptable to me.  They should have stock on hand sufficient to cover their contracts. Obviously that's not entirely feasible.  What if a earthquake took out some major sites with lots of drives?  But barring any such disaster, they should have a replacement drive on-site in 2 hours, as their contract obliges them to.

Have you tried to escalate it?

My storage servers have Linux software RAID1 OS drives, minimal lean secure Gentoo, custom kernels, and daily notifications that failure has not occurred. But I owe a big debt of gratitude to Dr. McNett for showing me the wisdom of doing it right by doing it yourself.


Telephones and Telegraphs - Not Internet

Wow. AT&T kicked me in the balls pretty hard at the beginning of this month. I didn't pay my bill because I didn't know it was due. Apparently AT&T prefers some kind of antique Pony Express kind of notification system using dead trees. Since that's buried under so much AT&T spam packing the mailbox in my yard, it's easy to let a couple of them slip by. But I got that sorted out.

I remember during one of my many hours on the phone with AT&T customer service telling someone of the impossibility of using AT&T's web site to do simple things like pay a freaking bill. But I was assured it was not just possible, it was easy. So I tried again. And shaking my head in disbelief, I requested that AT&T send me an online registration number by Pony Express. That arrived and I have the number! Great! So I fill in the web form. Then I fill it in again because of shockingly stupid restrictions on username composition (which made the stupid restrictions on the password look less criminally insane). Great. Account created! Super.

Except not. Except instead of logging into my account, I see this:

Then I click the "contact us" link in the error message and I click the topic related to logging in. It then suggests I log in. Which produces the same error. Grr. Getting out of that loop I start to compose my description of the situation and there is some unspecified limit to how much I'm allowed to say? WTF? At the very least that space should be as long as AT&T's freaking terms of service which they wanted me to read!
I'm totally not amused here. I don't even use this idiotic land line. I just need it for my internet connection and it follows that I want to do my business over the internet. This would be less ridiculous if AT&T was not claiming to be the post 19th century company that they wish they were. No, AT&T keeps sending me dead tree spam trying to get me to use them as an internet provider too. Hell, why not just go with carrier pigeons?
Internet provider that can't bill its customers using the Internet: Epic Fail


DSLexia Extreme

My internet per se comes from DSLExtreme (which I call DSLexia Extreme) and that requires an old school telephone land line. The economics barely are worth it, but I like to pay AT&T as little as possible. My recent nightmare has fully developed that sentiment. AT&T (ironically for being a major ISP, for example all iPhones before last year) has a shitty web page with shitty features to manage your phone account online. I could not figure out how to pay bills on line. I'm pretty sure it's not simply my computer-using clumsiness. I timed one AT&T promotional web page loading in *eight* minutes - at SDSC.edu, best network in town! So something is rotten with AT&T's web page and that's just a start. They force me to call with a telephone and press buttons to pay my bill. No email reminders, but rather paper bills hidden among as much of their paper spam. I missed a couple of these among the spam bin that is my snail mail and lost track of when I'd last called and suddenly one day, no internet.

I call DSLexia who kept insisting that to figure out why I had no internet, I'd need to give Steve Ballmer a foot massage. Finally *I* figured out that, hey wait, no phone line (because I don't use it for anything else because of AT&Ts dirty surreptitious habit of charging "long" distance rates for calls to points I can reach in 30 minutes - by bicycle!). Once I realized that it was the phone that was really dead, I called AT&T and told them to turn it back on. To their credit they lied to me the good way, they said it'd take 40 hours and it took about 25.

Great! I thought I was back in action. Only the DSL didn't work. Then back to DSLexia. These guys really were exasperating. I called at least a dozen times and spent at least as many hours on hold. Some "tech" "support" personnel still wanted to do the old "Press the Start button" script (moot for Linux) but since I knew the root cause of the problem, I could usually get them to move on to their next phase - being completely stultified. They would consistently elevate my call to tier 2 support and after a while the tier 2 guys would start to realize what was at issue and they believed it wasn't a technical problem, but rather one of re-signing up for service. They'd transfer me to billing. Then billing would deem it a technical problem and transfer me to tier 1. Then the process would repeat. About half of the time the wait times were about 10 minutes. The other half, they were indefinite.

But this was wasting valuable time to experiment with just how long they could keep me on hold after promising to come back in a minute. Eventually I learned that billing and tier 2 will *not* eventually pick up your call if it's not conventional weekday business hours (didn't figure out which US time zone exactly). We could have saved a lot of trouble had the tier 1 staff been honest and/or knowledgeable about that.

Finally I talked to Danielle in California (not the Philippines) who was not clueless. She knew my problem in a familiar been-there-done-that way that made me believe what she said. She also didn't seem eager to lie to me about the prospects of when I might get my connection back. I had been assuming all this time that all that needed to happen was someone just resets my connection and I'm done. But she made the reality sink in that basically a new order was going to have to be placed. This meant a projected outage of almost two weeks.

Even though I had just paid the AT&T weasels $40 to turn my phone back on, I checked into switching to cable from Warner: expensive to setup, maintain, and no quicker to get back online. I asked what it would take to get AT&T to just take over DSL and they really balked at that as if it were hardly even possible. They are emphatically moving to their Uverse technology and it sounds superior, but the up front costs are high (for a non TV watcher) and its lead time was not helpful either, weeks hence. I would add that this makes sense since it is madness to think that true *high* speed internet can come delivered over an analog phone line. Most previous experience with DSL Extreme has been them telling me to do voodoo nonsense with my physical phone wires to improve the connection. If that is really necessary, then I'm afraid those wires are too obsolete for using in that way.

Basically there was going to be no high speed internet service at my house for 2 weeks. It definitely sounds like a white person problem to be so dependent on a stupid internet connection, but it's like with a lot of people how important their car is. If it breaks, they don't get to work. Same with me and the internet. But with internet, there are no "rentals". This is a very big problem that telecommuting pioneers like me are discovering.

I did explore the possibility of borrowing a cup of internet from my friendly neighbors. But my neighbors aren't really any friendlier than I am and when it comes to the internet they are much more hostile. For the past 9 month I've let my neighbors use my open wifi if they aren't doing conspicuously naughty things, but no reciprocity in my time of need. As much as I wanted to "borrow" a connection from access point "GO_F_YOURSELF", alas, they used a strong password wpa psk scheme. I checked.

I looked into tethering and in the future this will be the correct solution, but right now, it required creepy client software (not available for Linux) or rooting the phone. Since my new smart "phone" was my last dwindling ray of light to the outside world, I did not feel like experimenting to see if I could avoid bricking it.

Not quite ready to accept defeat, it was time to open a mighty can of ancient technical whoop ass on my lack of internet connectivity. Since I had already signed up for this moribund 19th century "telephone" "service", I realized that I had, barely, all the ingredients to go back in time, almost 30 years, to use an ancient technology to temporarily solve the problem: dial up.

Getting Linux (or any OS) to know what to do with a telephone modem in 2011 is not exactly easy and without a network connection, it's very challenging. But by ferrying some things (pppd, wvdial, minicom, setserial) from work, a visit to an internet cafe, and reading man pages on my Android phone, I did it. I then found that almost all modems that could be plugged into the inside of a modern PC are "Winmodems" which means that you probably won't find necessary drivers if you use Windows and they probably won't exist if you use Linux. Amazingly some clueless people were clearing out an office building downtown and posted a NIB proper external 56k modem. They asked too much (the clueless part), but it was their lucky day since I was on a serious mission; I guess when they tell the story, I'll be the clueless character.

Once back home, the computer I was preparing for this project stopped booting and started smelling like burnt electronics. Much voodoo later, I was able to drop its power draw down and get it to boot and run. Finally, I got the modem recognized and responding, then, making calls. I'd already laboriously mapped out which of the access numbers from the dial up ISP I'd signed up for were in Zone 1 (not long distance). After some configuration issues that required me to read the C source coded of pppd, I finally got an internet connection. Then I was in more familiar territory (as a practicing professional) when I set this computer up to be a port forwarding IP masquerading gateway firewall router. I rewired a few things on my network and now, my house once again has internet. Whew.

It's not fast, that's for sure, but one of the very nice advantages of doing things in the spartan text-console-based world I like to work in is that I don't *need* very fast. I can not watch lolcat videos and stream music while using on-line office suites, but I can ssh to my main server and do all the stuff I need to do (like compose this entirely over the line, i.e. I'm at home and the editor is running at work). I even did a Gentoo `emerge --sync` which needed to happen to then `emerge iptables`. That was slow and took many hours, but it did work.

I don't know how this story will end. Maybe DSLexia was jacking me around and they'll keep billing me and not providing service indefinitely until I get legal representation to menace them with a law suit. I did, for example, find out in the course of this that their "free" DSL modem actually means "free lease". Apparently "free lease" is a euphemism for "loaned". They *loan* you the modem because when you cancel your service, they want it back or they hit you with a $100 charge. Shady bullshit like that, the necessity for paying two bills, and the questionable technical service is probably enough to make me not recommend them despite how much I hate giving AT&T any money ever.

I may be without high speed internet for between 2 and 4 weeks, but thanks to my serious effort I won't be pressured without internet while I make a more careful decision about which of these predatory companies are the lesser of evils.


Visualizing Types Of Databases

I made this graphic to illustrate how I thought of various database strategies. Enjoy.


The Comforts of Java

Java is such a pain in the ass. I suffer terrible struggles to get it installed for people who need it. I went to the Sun Oracle website to hunt for it and I saw this graphic which instantly reminded me of the pain and discomfort of the entire experience of dealing with Java.


There are many ways to say "Evil" in Spanish: Net10, TracFone

Let's talk about evil first. Net10 has a slogan that says "No Evil". Do you know why? They have this slogan precisely because they are "Evil". Here's how it happened. They started out with a reasonable idea which was squeezing money out of people so wretched that they couldn't qualify for the oppressive plans normally offered by the cell phone "service" providers. They called this evil thing TracFone. But then smart people figured out ways to take advantage of TracFone. I, for example, bought their "lifetime double minutes" card. After using that for two years, a funny thing happened, it mysteriously got converted to a "1 year double minutes" card. Ahem. Evil. So TracFone is clearly evil. I know it, you know it, everyone, even TracFone knows it. So what does TracFone do to fix that? Become less evil? Noooo. They just create a phony new company. This is Net10. From a page on the Net10 website:

"All NET10 prepaid wireless service is provided by TracFone Wireless, Inc, America's largest prepaid wireless service provider. TracFone Wireless is a subsidiary of America Movil (NYSE: AMX, Nasdaq: AMOV), Latin America's largest mobile phone provider, with more than 165 million cellular subscribers across the continent."

America Movil... Hmmm take out the "o" and turn the "M" 90 degrees counter clockwise. I think you're seeing things clearly now. This means that if you've been burned by TracFone's flagrant evil, don't let Net10's "No Evil" (comically displayed on the same web page) fool you. When Net10 says "No Evil" what they really mean is "We needed an alias to avoid being recognized by our hallmark which is being evil".

Now it gets funny. Since TracFone screwed me, I've been hard at work screwing them and I encourage you to screw them too. The first thing to note is that they really want your nuts in their mouse trap and this is why they have "special" "deals" on phones. The interesting thing is that one can buy 300 minutes of phone time for the eponymous 10 cents per minute *and* get a phone thrown in. Note that Net10 does not offer any real incentive to buy a 600 minute card over a 300 minute one. Why not buy 2 phones that are charged with 300 min each? (Hint: google.com/voice is your friend.) If you do this you can basically get a spare battery and charger for free. Or a whole second phone etc. And it gets better. You should always search the web for "promotion codes net10" before ordering. I got my 300 minutes for only 23.91 complete which is less than $.08/min. Oh, and the W375 "phone" thrown in purely as a bonus.

The last time I did this, I got a phone with 200 minutes on it and a 300 minute card. They've caught on to this and now they don't send a separate card (ahem, quite easily fungible (sellable) on eBay) anymore. However, one can get minutes transferred to the new phone and aggregated to the new minutes.

How did all that go? Not too great. The process starts out with "Press 1 for English" but there's no option for intelligible English. After being on a (landline) phone for about 30min with customer "service" everything seemed set up, number transfered, minutes transfered, activated, signal.... yet all calls resulted in "Call Failed". Then about an hour on the phone with another tech support guy. I had given them all the info they could possibly need and since they were able to remotely set the number my "phone" was clearly in contact with a base station. The ball was in their court. The only thing they had to say by way of excuses is that it could take up to 72 hours. That's THREE days for those keeping score at home.

Ok, so at this point this "phone" still can't make or receive phone calls. Maybe its other fantastic functionality will make it worth the electricity it has used.

First off it has a Bluetooth functionality because my computer told me it detected it. Great! Only not so much. Turns out that the Bluetooth is only for headsets (verified by a customer "service" person).

Ok, how about the extremely helpful-looking USB connection? I tried connecting the "phone" to a USB memory device: nope. How about to my computer's USB port: nope. Surely you can charge from the computer's USB port's power supply: nope. So much for the USB port.

On to the "Camera"! This is so ridiculous that it's actually funny. After reading up on the "camera" and asking the tech support person about it, I was left with the question, why? Why did they include fancy CCD electronics on this "phone" when they could have just skipped that part? You see, if you take nearly any object, say a banana, you can say it's a "camera" banana because the photons constantly hitting the banana are indelibly changing the subatomic structure of the fruit. In fact, it's really a "video camera". The problem with our "video camera banana", of course, isn't that this is untrue, but rather how does one extract this video information that does, in fact, exist in the banana? It's such a stupendously hard problem that we tend to think of bananas as not being video cameras. And so it is with this "camera" "phone". Oh sure it's mysterious internal state is recording variation in incident lighting, however, that recorded information is trapped forever inside the device. In other words, you can "take" photos, but they're forever stuck in the "phone". The only thing you can "do" with them is to joyfully delete them. Fun!

So let's see, not only is the "camera" "feature" no better than that of a banana, but the calling service is also no more satisfying than a banana. I think I'll stick with a banana in the future because of its long battery life and pleasant ring tone. Also, with the banana, you can at least feel good about eating it.

Wishing you creative luck at screwing whichever cell phone "service" incurs your wrath!