Politics

I think the best analogy for modern-day politics is sports teams. I root for the Red Sox, because they’re my local team. Yankees suck! This is foolish, of course. The Yankees are a great team. They’re not an enemy, but friendly competition.

Politics today are the same way. Democrats hate Republicans, and Republicans think Democrats are destroying America. And we are the fans, cheering for our side and booing the other.

But the problem is that the teams don’t matter, and a drama-filled game isn’t good in politics. We sit back and watch heated political debates, and start cheering for our side and vehemently opposing the other. But at the end of the day, politics isn’t a game we play for fun. It’s the operation of our country, and it impacts each of us every day.

It’s not just that we root for our teams that’s the problem, it’s that we do so without caring about the game. It would be like if I cheered fanatically for the Red Sox and talked the next day about how bad the Yankees were, yet I never bothered to watch a moment of the game. In politics, the teams don’t matter, and yet the “game” we all ignore is profoundly important.

This is interesting, because it doesn’t talk about political parties. I evaluated a whole bunch of proposals to fix problems, and did so without having much of an indication if I was backing a Republican idea or a Democratic idea. Doing so forced me to think about the pros and cons of individual bills, without hearing any twisted rhetoric from either side. This is what politics should be.

Thinking Like an Engineer

Lately a lot of my work as a web developer has been way at the back-end, and, for whatever reason, it tends to focus heavily on third parties. I spent a while fixing a bizarre intermittent error with our credit card processor, moved on to connecting with Facebook, and am now working on a major rewrite of the API client we use to talk to our e-mail provider. Sometimes it starts to bleed over into my personal life.

This kind of turned into crazy-person babble, but I haven’t posted in a while, so here goes a perhaps-horrifying look into how my mind works:

  • Driving home one night, I went through the FastLane / EZPass lane, as I often do. Only this time, instead of thinking, “I hate that I have to slow down for this,” I started thinking about latency. Latency is one of the biggest enemies of people working with third parties. It was at the crux of our problems with the credit card processor — we’d store a card and immediately try to charge it, when sometimes we had to wait “a little bit” before the card was available throughout their system to be charged. So I had to introduce a retry loop with exponential backoff. The email API work has major hurdles around latency and timeouts. We’ve moved almost all of it into a background queue so that it doesn’t delay page load, but even then we have intermittent issues with timeouts. So driving through the FastLane lane today, I slowed to about 45, and thought how remarkable it was that, even at that speed, it was able to read the ID off my transponder, look it up in a remote database somewhere, and come back with a value on what to do. I’d have assumed that they’d just queue the requests to charge my account, but if my prepaid balance is low, I get a warning light shown. It seems that there’s actually a remote call. It’s got to happen in a split-second, though, and that’s pretty impressive. I wonder how they do it. I thought a lot about this, actually.
  • I work on the fourth floor of a building with one, slow elevator. A subsection of Murphy’s Law indicates that the elevator will always be on the exact opposite floor: when I’m on the first floor, it’s on the fourth, for example. So one day while waiting for the elevator, I started thinking that it needed an API. I could, from my desk, summon it to our floor to lessen my wait time. Likewise, I could build an iPhone app allowing me to call the elevator as I was walking towards it. The issue of people obnoxiously calling the elevator way too early seems like a problem, but I think it’s okay — if you call it too soon, it will arrive, and then someone else will call it and you’ll miss out entirely. It’s in everyone’s interest to call it “just right” or err on the side of a very slight wait.
  • While thinking more about the elevator API, I started thinking about how elevators aren’t really object-oriented. (I’m pretty sure that’s never been written before.) It seems an elevator is really pretty procedural, running something like goToFloor(4). The obvious object would be Floors, but that’s not really right. You’re not adding Floors to the building, or even changing properties of Floors. The object is really CallRequest, and it would take two attributes: an origin and a direction. “Come to floor two, I’m going up.” It made me think that there are some places that being object-oriented just doesn’t make a ton of sense.
  • You really want to add authentication. To get to our floor, you need to swipe your badge. The elevator API needs to account for the fact that some requests require validating a user’s credentials to see if they’re authorized to make the request they are.
  • “Code an elevator” would actually be an interesting programming assignment. But I fear it’s too far removed from most normal coding. I started thinking that you’d want to sort CallRequests in some manner, use some algorithms, and then iterate over CallRequests. I think you actually want to throw out that concept. You have a tri-state to control direction: Up, Down, and Idle. Then you have two arrays: UpwardCalls and DownwardCalls. They don’t even need to be sorted. As you near a floor, you see if UpwardCalls contains that floor. If so, you stop. If not, you continue. If you’ve reached the highest floor in UpwardCalls, you check to see if DownwardCalls has an elements. If so, you set your direction to Down and repeat the same procedure for DownwardCalls. If there are no DownwardCalls, you set your state to Idle. The problem is that this is really not how I’m used to thinking. I want to iterate over CallRequests as they come in, but this means that the elevator is going all over the place. The person on the 4th floor wants go to the 2nd, so we make that happen. But right as they put that request in, the person on the 3rd wants to go to the 1st. So you’d go 4 to 2 to 3 to 1. “Fair” queuing, but ridiculously inefficient. On your way from the 4th to the 2nd, stop on the 3rd to pick the person up.
  • I wonder how things work when you have multiple elevators. In big buildings you’ll often have something like 8 elevators. I’m far too tired to try to figure out the ideal way to handle that. They need to be smart enough to have a common queue so that I don’t have to hit “Up” on all eight elevators and just take whatever comes first, but deciding what elevator can service my request first is interesting. I kind of think it’s another case of elevators not being the same as the programming I’m used to, and it’s just whatever elevator happens to pass my floor in its service going in the right direction. But what if there’s an idle elevator? Can it get to me first, or will an already-running elevator get there first? Do you start the idle elevator first and make it event-driven? What if the already-running elevator has someone request another floor between its present location and my floor? You’d need to recompute. You’re probably better off dispatching an idle elevator and just giving me whatever gets there first.
  • You then need to figure out what’s important. If you have an idle elevator that can get to me more expediently than an already-running elevator, but the wait time wouldn’t be that much longer, do you start up the idle elevator, or do you save power and have me wait? How do you define that wait? Is this something elevator-engineers actually tune?
  • I think you want to track the source of a request — whether it came from within the elevator or from the external button on a floor. If it’s within the elevator, you obviously need to stop, or the elevator has effectively “kidnapped” the person. But if it’s an external button, you might want to just skip it and let another elevator get to it, if you have a bunch of CallRequests you’re working through. Ideally, you’d also approximate the occupancy of the elevator based on the weight (from reading the load on the motors?), and when the elevator was at perhaps 75% capacity, stop processing new external requests.
  • Should the elevator controller try to be proactive? It might keep a running log of the most “popular” floors out of, say, the last 50 CallRequests, and, when it was done processing all CallRequests, go to whatever the most popular was and sit idle there? Or perhaps it should work its way towards the middle floor? If you had multiple elevators you could split them apart that way. Is it worth the power conservation?
  • The office thermostat should have an API, too. (I bet the good ones do. But we don’t have those.) Thermostats are a pain to use. readTemperature and setTemperature are the obvious API calls, though advanced thermostats would have a TemperaturePlan concept.

sync_binlog on ext3

I’ve mentioned before about how the sync_binlog setting in MySQL can be especially slow on ext3. Of course, I wasn’t the first to discover this; the MySQL Performance Blog mentioned it months ago.

I was reading through some of the slides I mentioned in my last post, and remembered that I’d left sync_binlog off on an in-house replicated slave. You’re able to set it on the fly, so a quick set global sync_binlog=1 was all it took to ensure we flushed everything to disk.

A while later I blogged about dstat and thought to run it on the in-house MySQL slave. I was confused to notice that the CPU was about 50% idle, 50% “wai” (I/O wait). For a box that’s just replaying INSERTs from production, that’s really bad. Below is a graph of load on the system. Care to guess when I enabled sync_binlog?

Performace with and without sync_binlog enabled

Performace with and without sync_binlog enabled

Disk I/O does roughly the same thing, but it’s less pronounced, “only” doubling in volume. But the difference is still pretty insane!

Absolutely Nothing Happens

We were talking quite some time ago at college about how gay marriage had been legal in Massachusetts for a few years. Despite all the hubbub about it being the end of the world–or the greatest thing since sliced bread–we came to the conclusion that it was a complete non-issue. None of us knew anyone who knew anyone who had ever been to a gay wedding, and even my conservative peers came to not care in the slightest that same-sex couples could get married. It’s gone on for years, and the impact to conservatives who felt threatened turned out to be nil.

And then I remembered that Massachusetts had decriminalized the possession of small amounts of marijuana. Like gay marriage, it drew a lot of criticism, and had many panicked that the sky was falling. About four months (I think?) later, absolutely nothing has happened as a result. It’s had zero impact on my life or those I know. Massachusetts doesn’t suddenly have a drug catastrophe.

I wonder what else would turn out that way. Drivers licenses for illegal immigrants? (It’s no secret they drive; I’d much rather that they had to follow the same rules I did to get my license.) Loosening up gun laws on non-criminals? (Gotta get both sides in there! I do think Boston and New Hampshire are very different, but I live in a state with incredibly lax gun laws and would be hard-pressed to think of a single gun crime.)

Though I think there’s an interesting lesson in this. For all the political apathy we’re accused of, it seems that there’s an awful lot of Chicken Little FUD (on both sides) about things that turn out to be non-issues to most of us.

Pirates

After reading about a series of pirate attacks last year—back then an almost laughably bizarre occurrence—I became interested in the concept of modern piracy, something I, like many average citizens, was unaware still went on. I picked up a copy of John Burnett’s Dangerous Waters: Modern Piracy and Terror on the High Seas after hearing him talk on NPR, but didn’t get far into it.

Recent events revived my interest, and I made some headway in the book this weekend. It turns out that piracy has been a major problem for ships in third-world areas, which is problematic since many major international shipping lanes progress right through these areas. No ship is immune, from small sailboats to “VLCCs”: Very Large Crude Carriers, commercial oil tankers rivaling our military’s biggest ships in size. As we learned with the recent hostage situation, pirates tend to be destitute teenagers from the poverty-stricken nations who have little to lose and everything to gain.

This afternoon, I read an interesting observation: some private ships, including cruise ships, are known to employ “heavies,” gun-toting mercenaries, to protect the ship and those onboard. Guns are otherwise uncommon: there are many thorny legal issues, including the need to declare them to customs when docking in a foreign port, at which point they’re seized until you leave again; the fact that pulling a gun on pirates, unless you’re a well-trained marksman, is likely to get you shot; and the fact that, on many of the oil tankers, a single stray round could blow the whole ship up.

So imagine my surprise when I checked out Google News, and saw that an Italian cruise liner off the coast of Somalia actually used its heavies to deter pirates. Besides idle fascination n the escalating pirate wars, I think this is a good thing: if pirates are becoming brazen enough to fire on cruise ships, there’s a much more pressing need for the international community to aggressively put an end to piracy. Piracy is no longer an obscure issue affecting an incredibly small number of commercial ships, but something threatening anyone on a boat in international waters, and the latest escalation is likely to cause an even greater escalation in piracy defenses.

Unapologetic

Apparently, a company wrote an application for the iPhone called Baby Shaker. It depicts a crying baby, and you vigorously shake the iPhone to make it stop, at which point its eyes are replaced by X’s.

Apple pulled the application from its store and apologized, saying, “This app is deeply offensive and should not have been approved for distribution on the App Store.”

The Sarah Jane Brain Foundation, however, has had enough, with a spokesperson calling it “the most cynical apology I have ever seen.” They plan to picket Apple stores, calling on them to “mitigate the harm they’ve now caused.”

What I find so interesting is how the Sarah Jane Brain Foundation has had “The PETA Effect” here, at least for me: so vehemently overstating your cause that you steer people to the other side. If I’d seen the application distributed, I’d surely have joined the Sarah Jane Brain Foundation in finding it horrifically offensive. It’s in bad taste and makes light of an abusive practice that kills many babies and leaves even more with permanent injuries and brain damage.

And yet, with them coming across as so overzealous, my “That’s really kind of funny” sense is triggered, just a tiny bit. I guess I find their position so outrageous since:

  • I don’t like Apple having sole control of what I can run on my iPhone. Apple pulled the app, which means that, unless I jailbreak my iPhone (voiding my warranty), I can’t have the application. I’m not sure I want this application, but it’s a sore spot with me. The fact that Apple pulled the app just drives home Apple’s exclusive control.
  • Apple promptly pulled the app. The “most cynical apology” actually seemed to be a pretty emphatic, “That application was horribly offensive. We screwed up big time in approving it!” from Apple. I’ve definitely heard much more cynical. (I’m sorry you feel that way…)
  • The application shows that shaking babies kills them. Sure, it demonstrates it in an awful way, but it’s like showing tapes of babies’ skulls being crushed to lobby against abortion. Isn’t this exactly what the Sarah Jane Brain Foundation should enjoy?
  • I think that, and then get the sneaking suspicion that they are loving this, because it’s giving them tremendous publicity. And calling for protests outside Apple stores, weeks after they pulled the application and apologized for it, only furthers that point.

What do you think? Was Apple’s apology (and prompt retraction) of the app good enough? Should Apple have left it up even though it was controversial?

Building a Better Camera

Thinking about a $22,000 lens got me thinking about “real” cameras a bit more. And it occurred to me that Canon is in kind of a weird spot right now.

Their flagship camera has always been the EOS-1. With digital it was the 1D, which was followed by a 1Ds. The s designates that it’s meant for studio work, with a higher resolution but lower framerate. After a while Canon replaced them with the “Mark II” edition of the 1D and 1Ds, and a few years (?) ago, the Mark III edition.

The Mark IIIs were well received. The 1D Mark III supported up to ISO6400 if unlocked, allowing great low-light performance. The 1Ds Mark III is what really got people drooling, though, with a 21-megapixel resolution. I think it was around 10 megapixels that people started saying that resolution wars should really be considered over. 21 megapixels, in the eyes of many, bests medium-format cameras. People shoot for two-page magazine spreads and billboards with lower resolutions.

The awkward sitution comes from the Canon 5D Mark II. The 5D is still a very high-end line, but it’s meant to be second fiddle to the 1D. But the 5D Mark II boasts 21 megapixels, the same as their flagship 1Ds Mark III. It records 1080p video. And what really wins me over is that it gives Nikon’s D3 a run for its money: ISO6400 out of the box, and you can enable “High ISO” support for ISO 12,800 and 25,600, allowing photos to be taken in absurdly low light. It sells for $2,700, less than half of the $7,000 1Ds Mark III.

So it’s high time for a Mark IV series. I haven’t even seen rumors about it yet, which tend to start long before the camera’s released. But here are some of the things I’d really like to see Canon release in a Mark IV edition:

  • Higher ISO support with lower noise. I’m not sure many people even imagined ISO6400 in the days of film (though it looks like there may have been such a thing, though it certainly wasn’t sold in Walmart), but the trend has been started. ISO12,800 and ISO25,600 are kind of experimental modes that remain very noisy (grainy). When I’m in the market for a new digital SLR in a few years, I hope it’s got a noise-free ISO25,600, or higher. Consider that increasing sensitivity just twice more would bring “ISO 100K.” Canon and Nikon, it’s a race. You heard it here first. I want the 1D Mark IV to put Canon in the lead, and Nikon to come out with a D4 to try to one-up them, with the end result being a camera that can take photos in dimly-lit rooms without five-figure lenses.
  • Get rid of the mechanical shutter. Sample the sensor for the necessary duration. It seems there are design challenges in eliminating the shutter, but it’s really a vestigial organ on a digital camera. This removes a common spot of mechanical failure, and paves the road to higher shutter speeds. I don’t think any camera (possibly excluding ultra-expensive scientific gear) can exceed 1/8000th of a second shutter speeds right now. Accidentally shoot outside at f/1.8 and ISO1600 on a sunny day and tell me it’s not a limit. (Yes, yes, why would you want to do that? Because I needed the shallow depth of field and forgot my camera was cranked to ISO1600. The real question is: why couldn’t the camera handle it?)
  • RAM is cheap. Like $10 for a 1GB DIMM. I doubt cameras have DDR2 DIMMs, but why can I only take a couple shots in rapid succession before I have to wait for the camera to write things out to the card? On the flagship model, give us a crazy-huge buffer.
  • For the love of God, give us an LCD that we can see when we’re working outside. And while you’re at it, spend the money on a great LCD. Look at an iPhone screen for a while, in fact, and see what 150 dpi looks like.

If I Made Computers

I think you could say with relative accuracy that there are three main bottlenecks in a computer: CPU, memory, and disk. There are some outliers that people might try to pile in: video card performance, or network throughput if you’re tweaking interrupts on your 10GigE card. But the basic three are pretty universal.

To cut to the chase: I hit disk bottlenecks sometimes, CPU bottlenecks almost never, and RAM bottlenecks all the time. And sometimes high load that looks to be on the CPU is really just I/O wait cycles. But RAM is special: if you have enough RAM, disk throughput becomes less important. At least, redundant disk I/O, which seems to account for a lot of it.

What interests me, though, is that almost everything is RAM starved in my opinion. My laptop has 2GB and I get near the limit fairly often. I’m thinking of trying to take it to 4GB. The jury’s out on whether or not it’ll see more than 3GB, and others complain that 3GB causes you to lose out a bit on speed.

But here’s the thing. I maintain things like a MySQL server with 32GB RAM. It’s not RAM-bound per se: we could switch to a machine with 1GB RAM and MySQL would still run fine. The memory is overwhelmingly configured for various forms of cache. But it’s not enough: there’s still a steady stream of disk activity, and a non-negligible number of queries that have to write temporary tables to disk.

RAM is cheap. It’d cost me about $50 to buy 4GB of RAM for my laptop. The reason RAM stops being cheap is that most motherboards don’t give you enough room. Both of my laptops can only take two DIMMS, which means I need dual 2GB sticks. They’re both based on older 32-bit chipsets, so I can’t exceed 4GB, but if I wanted to, I’d need dual 4GB sticks, and those are expensive. Even on decent servers, it’s hard to find many that give you more than 8 slots, making 32GB hard to exceed.

So what I’d really like to see someone bring to market is a 1U box with as many memory slots as it’s physically possible to fit in. 1U is still tall enough to have standard DIMMs standing up, and most of them are extremely deep. I bet you could fit 256 slots in. Then throw in a compact power supply, a standard LGA775 slot (allowing a quad-core chip), a good Gigabit NIC or four, and an optional FibreChannel card. No hard drives. Maybe a 4GB CompactFlash card if you really want it to have its own storage. Oh, and make sure the motherboard is pretty versatile in terms of RAM requirements and FSB. Oh, and don’t force me to go with ECC. If this were a single database server, it might be worth buying top-notch ECC RAM. But if this were just for caching things, I don’t care. Cache isn’t meant to be permanent, so an error is no big deal.

256 slots, and you could fill it with ultra-cheap 1GB DDR2 DIMMs. (Heck, at work, we have a bag of “useless” 1GB sticks that we pulled out.) You can get ’em for $10 a pop, meaning 256GB RAM would cost about $2,560. I suspect the system would command a high premium, but really, it’s just $2,560 worth of RAM and a $200 processor. A 2GB DIMM is about twice as much ($20/stick), but $5,000 for half a terabyte of RAM isn’t bad. Though 4GB DIMMs are still considerably more: they’re hard to find for under $100.

I think this would be a slam-dunk product. memcache is pretty popular, and it’s increasingly being used in previously unheard of roles, like a level 2 cache for MySQL. There are also a lot of machines that just need gobs of RAM, whether they’re database servers, virtual machine hosts, or application servers. And tell me a file server (sitting in front of a FibreChannel array) with 256GB RAM for caches and buffers wouldn’t be amazing.

So, someone, hurry up and make the thing. The key is to keep it fairly cheap. Cheaper than buying 4GB DIMMs, at least.

bash: Looping over a range

This falls into the category of things that are simple yet I’m always trying to remember. Sometimes in bash I find myself wanting a for loop, not of the usual foreach type (for i in `ls /etc/`; do ...), but a more classical one (a la C-style for var=0; var<100; var++ ... syntax). Here’s the bash way:


for i in {0..99}; do
echo $i
done

The output is a list of numbers from 0 to 99. For a fun example of when you might need this, consider the case of noticing that Postfix isn’t running on a machine and starting it. Your mail client promptly crashes, and the flurry of output in the log indicates that it’s delivering mail as fast as it can, but complaining that some mail has been in the queue for 180 days. “Holy smokes,” you exclaim, stopping Postfix. “Just how much mail is there?” You’re not entirely sure because you aborted your ls after it sat for a good minute with no output. That’s never good. du -sh shows about a gig and a half.

“I’ve got this under control,” you think. rm -f /var/spool/postfix/*. Ta-da! Wait, what’s this “/bin/rm: Argument list too long” business? That error can’t possibly be good. rm very rarely complains. So we tried a smaller delete size, thinking we could do postfix/1*, postfix/2*, etc. Just step through it. That, too, was too much for rm.

So it ended up taking an absurd for i in {0..99}; do rm -f /var/spool/postfix/$i; done to purge the mail. (And that didn’t catch all of it, either; I’m not sure off-hand how to do this in hex.) Each iteration deleted somewhere around 2,000 mail messages, making me think there were something like a half-million messages. (00-FF, times a little under 2,000.)

Politics

Lately I’ve felt that things were going pretty well. I was reading a bit of international news and looking at the international reaction to our presence at the G20 summit, for example. Of course not everyone in the world loves us, but I couldn’t help but feel that our presence was a little different than last time. Our President helped get disagreeing parties to agree, and in general seems to have the world eager to work with us. (I don’t really mean this as a condemnation of Bush, nor is it my intention to heap praise on Obama.)

And then I read what conservatives are saying, and it almost seems like we’re looking at two vastly divergent realities. I see a statesman, they see a closet Muslim who was all too eager to bow to an Arab leader and who went out of his way to apologize for being American. I see a fiscal plan inspired by John Maynard Keynes, they see someone deliberately wasting money for his own gain. I see the first black President, they see the first illegal immigrant President. I see a President who came in after Bush’s first round of financial bailouts and pretty much continued the policy, they see a President who nationalized the banks because he’s a Socialist. Oh, and he wants to take everyone’s guns away, and destroy Christianity.

I’d gone a while without reading the “wingnut propaganda,” and in that time period, I’d come to think that things were pretty good. Obama’s approval rating is something like 70%, and the two parties have been known to work with each other a bit lately, even if it’s been far less than I’d like. (And even if it’s been largely Democrat-led, which doesn’t really make for impressive bipartisanship…) And then I realized that there’s a lunatic fringe that seriously believes he’s a Muslim or a Socialist, and became truly worried. Fiscal conservatives and social conservatives may dislike Obama, and I respect their different views. Divergent views, discussed and brought to compromise, truly leave us better off. But there are thousands, if not millions, of Americans who have literally lost touch with reality. They’re like the MIHOPs of the Democrats.

I also want to caution that when I use terms like “neocon” and “wingnut,” I mean them more literally, not as terms to refer to all Republicans. Similarly, I respect Republicans and hate the artifically-created divide between the parties. What I’m complaining about is the wingnut Republicans who use utter lies to advance their own causes. There are Democrats who do the same, surely, but with Democrats leading Congress and the White House, those people aren’t noteworthy right now.

Anyway, two things have interested me lately. Besides the thousands of dead Americans, one thing that always bothered me about the Iraq War was the exorbitant cost. If the money were spent domestically, it could have gone an amazing distance. The military takes up something like 50% of our spending. So I’m waiting to see what the wing-nut faction of Republicans says. They’ve spent weeks protesting Obama’s Socialist spending. But the Democrats have long complained that the Iraq War is too expensive, and Republicans have argued that not giving the military a blank check amounts of waving the white flag of surrender. So I’m curious where this will go, because it could leave Republicans in an awkward state either way. Hopefully it will just be passed and nasty politics will be left out of it.

But then I was reading this article about how Obama may be looking to get the ball rolling on immigration reform. And the general description of his plan seems to amount to increasing border patrol and cracking down on illegal immigration. During the campaign trail one of the things he discussed was a path to citizenship, but with a pretty steep burden: you’d have to learn English, pay back taxes for as long as you’ve been in the country, pay a fine, and only then would you “get in line, behind everyone who came here legally” to become a citizen. Of course, that was something discussed during the campaign trail. The “official” Administration hasn’t even released a plan yet, but has merely made mention of strengthening border control, and the article is little more than speculation.

Yet some Republican activists have already denounced Obama’s (currently non-existent) plan as “dangerous” and “amnesty.” Seriously.