If you ever read a technical discussion board, you’ll quickly come to the realization that the breakdown of people is maybe 90% people who have kind of figured out how to use the technology, 7% people who are power users, and 3% people who are experts. It’s an arbitrary breakdown, but it seems about right intuitively.

Consider something like Excel. Most people can use it to keep tabular data, and most of those even know how to calculate sums. But very few might have a clue how to use Pivot Tables or formulas spanning multiple sheets, and even fewer will know how to extend it.

MySQL is definitely the same way. A lot of MySQL users can install it on their server and make phpBB use it. They might not understand what MyISAM and InnoDB are or how they’re different, much less the pros and cons of each. And even fewer could make a halfway decent DBA. But the good news with MySQL is that some of that elite 3% of experts are very, very vocal, and doing really, really neat things. Jeremy Zawodny is the first name that comes to mind, and check out his The New MySQL Landscape post. And don’t miss Percona’s announcement of their GPL’ed XtraDB, a replacement for InnoDB that’s supposed to be optimized for performance on more powerful machines. Seems like it’s very new and meant for MySQL 5.1, which some pretty smart people have said isn’t ready for prime-time. One of the MySQL guys at Google has a post about his patches to make MySQL better scale to ‘big iron’ type systems, too. And then there’s Our Delta (found on the Jeremy Zawodny blog) which distributes various patched versions of MySQL. Some are especially intersting to me, like Fast Master Promotion which is designed to allow a slave MySQL box to be promoted to master pretty much instantly, or the KILL IF IDLE command, allowing you to issue KILL statements to a connection and have them not affect non-idle connections. UserStats would be really helpful to run on a development machine to see what your code is impacting.

People who know me well will know that I’m not generally fond of movies. I can think of maybe a half-dozen movies I’ve seen lately. Borat is the only one I can think of that I’d recommend. Most movies just turn out to be a waste of my time. So I’m maybe not the normal movie-watcher.

That said, I’d like to review the latest Indiana Jones movie, which I watched last night. I’ll keep it nice and brief: F Minus. Made no sense.

I’m not sure how it started, but I had recently read something about greywater recyling, re-using water from things like your washing machine (as opposed to something gross, like your toilet) for other uses. It’s sometimes done in small homes, diverting the drain from your shower into your garden or the like, but it’s also done in big commercial places in the desert, where it’s filtered more heavily and reused.

At times, Indiana Jones got boring enough that I pulled out my iPhone and started reading some pages on greywater recycling. Did you know that you shouldn’t store greywater for more than 24 hours? As the action picked up a little more, I’d put it away, only to become seriously bored again and turn back to reading about recycling greywater on Wikipedia.

I don’t recall the last Indiana Jones movie I saw, but I seem to recall him as being a sort of macho, wild west hero who rides horses, kills badguys with his six-shooter and a whip, and defends ancient historical sites. Good ol’ Americana that makes sense, albeit being totally unrealistic. (He must have had about 10,000 bullets fired at him, and not a single one hit him: no special skill was involved, he was just running away and somehow machine gun fire from many guns never, ever made contact with him.)

But this one ended with a magnetized quartz (huh?) skull turning into an alien, which formed a giant UFO-vortex that sucked up the evil Russian lady, and turned what looked a lot like Machu Pichu into an ocean. And then, the end.

If anyone’s thinking of seeing it, I’d recommend you instead stay home. Here is the Wikipedia page on greywater, which includes some good links. Sure, I can think of much more interesting things to do than read up on greywater recycling. But watching this Indiana Jones movie isn’t one of them.

Sure, it had a few scenes that may have beat greywater recycling, but on average, it was slightly less interesting than reading about greywater recycling and how the various plumbing codes in the US regulate it. (Spoiler alert: some plumbing codes permit it, some do not, and most allow it with heavy regulation that usually makes it neither cost-effective nor environmentally friendly.) But besides being slightly more interesting, greywater has the advantage of making sense. Halfway through reading about greywater, it’s not suddenly going to become a magnetic skull made out of quartz, and it definitely won’t spontaneously turn into an alien, form a giant vortex, and flood Machu Pichu, sucking the evil Russians “into another dimension, the space in between spaces.”

Unlike SAV (also known as challenge-response systems), SPF is generally a decent idea. Basically, you publish a DNS record for your domain that lists what IPs are allowed to send mail from your domain. This means that you can say that mail sent from the host ‘mail.yourdomain.com’ is valid, but if a spammer sends mail from a random hijacked box in Tijuana, it will be rejected via SPF. It doesn’t target spam directly, but rather, it targets spam that spoofs the domain. (Which is probably a very good percentage of spam.)

But I’ve recently noticed a problem I hadn’t considered before: forwarders. I can easily set up e-mail addresses on my n1zyy.com domain that will simply point elsewhere. So mail sent to helen@n1zyy.com (which is actually a spamtrap; don’t e-mail it) might just be automatically redirected to another e-mail address, say john.doe@example.com. The headers are rewritten so that the whole thing is transparent.

The problem is that, with SPF, the mailserver that redirects the mail is effectively “forging” the headers, which means that SPF will block it. If example@hotmail.com sends an e-mail to helen@n1zyy.com, and it gets redirected to john.doe@example.com, it will fail if Hotmail has an SPF record. This is because example.com gets mail saying it’s from hotmail.com, but the headers indicate that it was actually sent from n1zyy.com.

There’s a few workarounds, but most are sustainable:

• The person running the original domain could add an SPF record for the mailserver doing the forwarding. This is all well and good if you’re sending mail from n1zyy.com and wanted me to whitelist the Comcast mailserver or something, but consider the example I used, in which case you’d have to call up Hotmail and ask them to add mail.n1zyy.com’s IP to their SPF record. They’d laugh at you.
• The recipient mailserver could override it. You could tell your example.com mailserver that, if the header says n1zyy.com, you shouldn’t check the SPF record. Again, good luck with this, unless you run the mailserver. Also, this is getting into “I could probably hack the Postfix source code to do that…” material.
• The recipient mailserver could be configured so that SPF will check to see if any of the mailservers along the way are listed in the SPF record, and, if so, accept the mail. This sounds like a good idea to me, to be honest, but it’s deviating a bit from what SPF was meant to do.
• The originating mailserver could stop using SPF, and this problem would go away. But then someone would send out a hundred million spam e-mails claiming to be from that domain, and they’d all go through.

Clearly, this is the type of thing that everyone is thinking about on a Friday night.

I consider myself a moderate Democrat. I support the right of citizens to own guns, think government should be as small as possible, and hate paying taxes.

I think Obama is to the right of me, too. I think we’ve screwed up Gitmo so bad that we need to just free everyone today, and arrest those who committed torture. Obama thinks we need to focus on the future, not dwell on the past, left the military and CIA a bit of wiggle room to get confessions, and still has people jailed.

A small, lean government is important. We need most of what we have, though. Same deal with taxes: I hate paying them, and want them as low as possible. But there’s really not a ton of cruft. And something like 50% goes to the military.

I’m appalled by Obama’s opposition to gay marriage. I think anyone who opposes it must have failed civics: terms like “separation of church and state,” “equal protection of the law,” and “tyranny of the majority” all seem pretty applicable. I consider it a moral issue like letting women vote.

So what I really want to know is… If I’m a moderate Democrat, and Obama’s to my right… What is a conservative?

I thought I’d piggyback on Mr. T’s work to ensure we gave fair coverage to more than one beverage. I actually have a couple different things to say.

• A year or two ago, when gas was $4 a gallon, someone mentioned that people are furious paying $4/gallon for gasoline, but merrily pay $2 for a 16 ounce bottle of water, which is $16/gallon. Consider that one comes out of the faucet and is basically free, and the other involves negotiating with third-world cartels and massive amounts of refining.
• Am I the only one that can’t drink water in restaurants? I think most restaurants use a little chlorine (?) in their water to keep things clean and sanitary. Except that I take a sip, feel like I’m drinking pool water, and spit it out all over the table*. It’s revolting. Not only that, but most places use the same water in their soda; if it’s light chlorine, it’s alright, but in some places, I can taste it in my soda, which is even more distressing to me. The thing that I don’t get is that it’s really easy to filter water. We have some fancy machine at work that’s sort of like a water bubbler hooked up to a sink, but it claims to use charcoal filters**, UV rays, and reverse osmosis. And that’s what water should taste like: nothing. I’ve started to drink it often. This is surely overkill, too. Throw a cheap filter in line with the water you’re serving to your customers, and voila! Strangely, there seems to be no correlation between restaurant quality: even in a place where you pay $50/person for dinner, their water might as well have been taken out of the toilet.

• Tremendous hyperbole. I furrow my brow and stop drinking the water.
  
• I am not a chemist, but doesn’t it seem counter-intuitive that running pure water through charcoal makes it cleaner?

A few thoughts on MySQL replication:

• Since a “READ DATA FROM MASTER” will issue locks until the slave is up to sync, it’s a horrible way to bring up a new slave to an existing setup. What’s not written about a lot, but isn’t really a big secret, is that you should use the latest backup/dump of the database to get the slave almost up to speed, and make sure you log its log position (from a SHOW SLAVE STATUS and SHOW MASTER STATUS). From there, you can just tell MySQL the binary log and the log position to resume at. In my case this means catching up on about 45,000 seconds, but it’s better than locking the database until it’s done.
• MySQL replicates temporary tables. I didn’t believe this at first, but it’s true. This means that replicating to a machine used for lots of data mining that creates temporary tables from SELECTs that take 30 minutes to complete is a bad idea. As best as I can see, there’s no, “Don’t copy over temporary tables, silly” option, either.
• Running STOP SLAVE will seemingly wait for the current query to finish. (You could kill it, but that’d be bad…)
• If, in the process of bringing up a replicant for the first time, it’s trying to create temporary tables based on temporary tables…. You can tell it to ignore “table not found” errors by putting slave-skip-errors = 1146 in your my.cnf: 1146 is the error for missing tables. This will keep replication going. Risky, but if you restored from a recent, complete backup, you’ll just skip creating temporary tables…

I got a bunch of kinda-neat apps for my iPhone. The free version of TouchTerm works well, but isn’t anything special. Twitterific is an okay Twitter client. Pandora is kind of cool, but I wish you could put it in the background, and it’s really only good over WiFi, and when there’s WiFi, I’m almost always at a computer. Dictionaire is a nice little dictionary app, but the load time is horrendous. PW is a password wallet; I don’t use it often. “Units” is a nice unit conversion thing, but nothing special…

Shazam, though, is really cool. I listen to the radio a lot while driving, and I’ve come to the realization that, for one reason or another, announcing the names of the songs you’re playing has gone out of style. So if I hear a catchy tune I want to buy, chances are, they’re not going to announce what it is.

Shazam lets you record a brief clip of the song, sends it off to their servers, and then identifies the song. There are so many things that could go wrong with this. I figured it would be kind of like OCR software or voice recognition: a cool concept that’s usually horribly off. Except that Shazam has never, ever, been unable to identify a song. I hear a song, pick up the iPhone, start Shazam, and hit “Tag Now,” and about 30 seconds later, I’ve got the song name, some album art, and links to buy it on iTunes, as well as preview it on Youtube.

And it’s free. I’d highly recommend it for any iPhone user that listens to music.

I just received the following bounced mail:

This is a delivery status notification from %s,
running the Courier mail server, version 0.52.1.

The original message was received on %s
from %s

I love how the one bit of information that came through is the exact version of Courier that they’re running. Because that’s exactly what I need to pinpoint why %s isn’t getting mail that %s sent them at %s.

I think some conservatives get really scared at the ridiculously huge crowds supporting Obama, and maybe even annoyed at the sight of him at the Lincoln Memorial.

I’m psyched for a few reasons. Obama himself is only a small component of those. He’s just a guy, an imperfect human. But what I think is so inspirational about Tuesday is a number of things:

• It’s President Bush’s last day. “01.20.09” bumper stickers have been around for a couple years. Looking back, I almost feel sorry for GWB. He took office amid tremendous controversy, the election being decided by the Supreme Court. The economy wasn’t quite as amazing as it was during the center of Clinton’s term. And 9/11 certainly put a damper on things. I don’t buy into the MIHOP conspiracy theories. But I do believe that the Iraq War (including Guantanamo) has cost about 4,000 American lives, distracted us from finding Osama bin Laden, and made us an international pariah. Hurricane Katrina was an unmitigated disaster, and overhearing the name Michael Brown on the news brought back the full absurdity of having an Arabian horse judge head up FEMA. The full blame for the present economic collapse can’t be heaped on Bush, but I’m certainly with the two-thirds of America that thinks he hasn’t helped. His repeated use of terms like, “The economy cratered” don’t help to inspire confidence. Warrantless wiretapping. Waterboarding. And so forth. I’m not saying that Bush was pure evil, just that I won’t be sorry to see him return to the private sector.
• I’m politically cynical. I didn’t like McCain, hated Hillary Clinton, thought Kucinich was crazy, hoped Romney got run over by a bus, and think Gore is better at his environmental stuff than he would have been in elected office. There are millions of people like me. My demographics—people in their teens and 20s—is one of the most politically-apathetic demographics on the face of the planet. I spent my month-long winter break volunteering for Obama. Many of my fellow volunteers had also never gotten involved in politics, or even really cared.
• But we weren’t just campaigning for a man named Barack Obama. We were campaigning to take the government back from the wealthy and from big corporations. We were fighting to take back the role of America as a defender of international human rights. We were pounding pavement to talk about equal rights. In short, Obama was just the spokesman for a novel new concept: a government of the people, by the people, and, most of all, for the people. The election of Obama, then, stands as an important reminder: if we, the American people, don’t like the government, we, the American people, can rise up and change it, to make it serve our needs.
• As much as some on the right love to argue that he’s only half-black, I think the election of a black man to the Oval Office is a major milestone in America. And anything that gets the KKK riled up is probably a good thing for America. Without meaning to imply that people who preferred McCain were racist, there were undeniably some overtly racist people out there protesting Obama. His election shows that, although racism is alive and well in America, it can’t hold a candle (or a burning cross?) to the tolerance of America.

For people like me who will be stuck in the office Tuesday 11:56 a.m., here’s a discussion of some places that will be streaming it. I rarely listen to WBCN these days, but they mentioned that they would be streaming it on their site, too.

In the meantime, check out the song. Though I didn’t like it musically at first, it’s grown on me, to the point that it’s really powerful and moving. A bit of trivia: the Yes We Can song, by will.i.am of the Black Eyed Peas, is sampled from … at Nashua High. Our spirits were high on election day, with one poll showing Obama leading Clinton by as much as 10%. But in an upset victory, Clinton won the NH Democratic Primary. Obama’s concession speech at Nashua High, though cordially recognizing Clinton’s success, was called “anything but a concession” (though I can’t come up with the citation). About nine minutes in is the start of the “famous” part of the speech.

Of course, his loss in New Hampshire wasn’t a major setback. Here’s on election night, addressing the nation for the first time with what would soon become a tiresome phrase: President-elect Barack Obama.

But in 38 hours, we’ll drop the “-elect,” and begin the process of reshaping America. And what a day it will be.

One thing that I always thought was neat, in a really strange way, is how users and coders see things totally differently.

At work, we have some admin tools to manage users, but periodically, we’ll come across a spammer who’s just signed up. They have tools to stop them and delete their posts, but I’ll often go the extra mile and log into a MySQL replicant and do a “SELECT * FROM users WHERE ip LIKE ‘$spammers_ip’G”, and see if it turns up any other users. Sometimes it’ll unearth a bunch.

The other thing I’ll do is something like, “SELECT username, signup_time, email, ip FROM users ORDER BY signup_time DESC LIMIT 20;” to view the most recent users, and re-run it every 30 seconds or so to keep an eye on who’s signing up. I have much better things to do than sit around watching this all day, but it’s useful when spammers do their periodic, “Register thousands of accounts at once” thing.

So today I was doing this for one of the customer service people, and they asked, “Where is that tool? I can’t find it.”

“Oh, I’m logged into the database… There’s no way for–actually, give me a few minutes.”

I threw together a couple PHP scripts that basically do the above, and display it in a table. It’s the type of thing that might bore a Programming 101 student, and the only thought involved, really, was in trying to figure out why it was aborting with an error. (Hint: searching for “” doesn’t do what you expect it will. I must have looked at the bit of code 30 times trying to figure out what could possibly be wrong with the line, and then it suddenly dawned on me. I’m a giant idiot. The is an escape character, effectively telling PHP to treat the next quote as a literal quote, not a closing quote on the string. This is something I know well: I’ll often do something like echo "link", using the exact functionality that tripped me up today. And yet I kept missing the error.)

Anyway, that’s not my point. My point is that I’m consistently amazed at how the user’s view is so radically different than the programmers. From my perspective, I just wrapped a little PHP around one of the simplest SQL queries imaginable. Heck, much of the logic was copied-and-pasted from another script. But from the perspective of the people who will use this script, it’s an immensely helpful tool that fills a void that’s existed for quite some time.

Of course, sometimes it’s the opposite way. Sometimes all the user wants is a report on newly-created groups, the username of the person that created them, and the number of members. That’s just a simple SQL query! Actually, I suppose it’ll take a join. No, wait, two joins… But do we actually store the number of members in a group? I wonder if I can do that with a subquery… Maybe this needs to be a script. You know what? This is way too complicated.