Category Archives: Ideas

This is How We Do

Posted on by
Reply

I finally got around to processing some of the pictures I took this past weekend. I wanted to share a few.

title=”IMG_1184 by n1zyy, on Flickr”>IMG_1184

How does that one look? I like to think it passes as a ‘normal’ shot. What’s not evident is that it was underexposed, poorly metered, had a nasty green color cast, and had everything in perfect focus. Tweaking photos in Photoshop requires striking a delicate balance. Too little touch-up and the image doesn’t look that great, and too much and the image looks pathetically artificial.

I like to think this one is a good compromise. I used Levels as well as the Shadows & Highlights tool to bring out a lot of the detail that was lost: some areas were too dark (people), and others were too bright (the court). I bumped up saturation and contrast every so slightly (that’s an easy one to take too far), and then used my new favorite Photoshop tool, a ‘smart selector,’ which let me easily select the crowd, and nothing else. It worked remarkably well at letting me not get any of the players or the court. With that, I applied a slight Gaussian blur to the crowd, to throw them a little bit out of focus. It’s what it should have looked like anyway, had it been shot with a faster lens.

Here’s the same shot straight out of the camera. You can see that it’s not really bad, and, side-by-side, it actually look a little more “natural.” But the players are a little too pale, the crowd is a little too distracting, and so forth.

title=”No Post-Processing by n1zyy, on Flickr”>No Post-Processing

Now here’s another one I took:

title=”The Shot by n1zyy, on Flickr”>The Shot

At a glance, this looks like a decent photo. Your eyes should be drawn right to the player as he makes his shot (I love shooting a camera with negligible shutter lag!) (The motion blur on the ball which looks kind of cheesy to me is actually legitimate.) The players, the rim, and the backboard all look nice and sharp. But if your eyes wander the crowd, you can quickly see that my attempt at a few tiers of Gaussian blur were amateurish. Having stuff like the open doors and scaffolding in the back of the gym is distracting. Again, had I been shooting a nice 200mm f/2.0 (not yet released, much less within my budget), it would have been thrown out of focus. But I was shooting with a lens at f/5.6, leaving stuff like that remarkably in focus. I started by selecting all the background junk and applying a Gaussian blur, but to mimic real life blur gets complicated in this setup. I wanted to throw the fans near the court slightly out of focus, but let the doors and walls be further blurred. This required multiple tiers of blurring, and left some strange effects. You’ll notice a few people with bodies that are mostly in focus, but heads that are exceptionally blurry. With a little more time, I could probably improve on this, but this was one of my first attempts at working seriously to manage background blur, so I instead offer it, with its myriad flaws, as an example.

Stomatron

Posted on by
Reply

I’ve been working on my resume as I seek to apply for a job that’s a neat blend of multiple interests–managing web projects (even in my preferred LAMP environment), politics, and even a management potential. And as I do it, I’m remembering all the stuff I did at FIRST, and reflecting on how much better it could be.

I was “fluent” in SQL at the time, but didn’t know some of the neater functions of MySQL. For example, when I wrote the web management interface to the Stomatron, I didn’t know that I could make MySQL calculate times. So I’d retrieve a sign-in and sign-out time and use some PHP code to calculate elapsed time. This wasn’t terrible, really, but it just meant that I did more work than was necessary.

More significantly, I didn’t know about the MySQL query cache. (Actually, I don’t know when it was introduced… This was five years ago.) Some of the queries were quite intense, and yet didn’t change all that often. This is exactly where the query cache is indicated.

Worse yet, I really didn’t do much with the idea of caching at all. Being the stats-freak that I am, I had a little info box showing some really neat stats, like the total number of “man hours” worked. As you can imagine, this is a computation that gets pretty intense pretty quickly, especially with 30+ people logging in and out every day, sometimes repeatedly. Query caching would have helped significantly, but some of this stuff could have been sped up in other ways, too, like keeping a persistent cache of this data. (Memcache is now my cache of choice, but APC, or even just an HTML file, would have worked well, too.)

And, 20/20 hindsight, I don’t recall ever backing up the Stomatron box. (I may well be wrong.) Especially since it and our backup server both ran Linux, it’d have been trivial to write a script to run at something like 3 a.m. (when none of us would be around to feel the potential slowdown) to have it do a database dump to our backup server. (MySQL replication would have been cool, but probably needless.) If I were doing it today, I’d also amend that script to employ our beloved dot-matrix logger, to print out some stats, such as cumulative hours per person, and maybe who worked that day. (Which would make recovery much easier in the event of a catastrophic data loss: we’d just take the previous night’s totals, and then replay (or, in this case, re-enter) the day’s login information.)

I’m not sure it was even mainstream back then, but our website could have used a lot of optimization, too. We were admittedly running up against a really slow architecture: I think it was a 300 MHz machine with 128MB RAM. With PostNuke, phpBB, and Gallery powering the site, every single pageload was being generated on the fly, and used a lot of database queries. APC or the like probably would have helped pretty well, but I have to wonder how things would have changed if we used MySQL query caching. Some queries (like WordPress’s insistence on using exact timestamps in every one) don’t benefit. I wonder if phpBB is like that. I have a feeling that at least the main page and such would have seen a speedup. We didn’t have a lot of memory to play with, but even 1MB of cache probably would have made a difference. As aged as the machine was, I think we could have squeezed more performance out of it.

I’m still proud of our scoring interface for our Lego League competition, though. I think Mr. I mentioned in passing a day or two before the competition that he wanted to throw something together in VB to show the score, but hadn’t had the time, or something of that sort. So Andy and I whipped up a PHP+MySQL solution after school that day, storing the score in MySQL and using PHP to retrieve results and calculate score, and then set up a laptop with IE to display the score on the projector. And since we hosted it on the main webserver, we could view it internally, but also permitted remote users to watch results. It was coded on such a short timeline that we ended up having to train the judges to use phpMyAdmin to put the scores in. And the “design requirements” we were given didn’t correctly state how the score was calculated, so we recoded the score section mid-competition.

I hope they ask me if I have experience working under deadlines.

Torrent Hosting

Posted on by
Reply

So I’m contemplating posting my BlueQuartz VMware image on VMware’s “Appliances” page, where it’d probably get a decent amount of downloads. I strongly doubt I’ll run into my bandwidth limit (it’d have to be downloaded about 3,000 times in a month), but I still don’t want to use bandwidth I don’t have to. When you’re distributing a big file to lots of people all of a sudden, BitTorrent is the perfect solution.

Unlike distributing, say, a bootleg movie, there’s an ‘official source’ for a lot of legitimate torrent hosting. This doesn’t mean anything in BitTorrent, but I think it should. The official source wants to ‘host’ it, but get people to help with bandwidth over BitTorrent.

There should be an easy way for them to host the file. Run a single command, pass it the file you want to distribute, and it’ll automatically create a .torrent file, register with some trackers (or host your own?), and begin seeding the file. In practice, this would probably take 10-15 minutes of work by hand. That’s pathetic.

There’s also a catch 22 at first: you want seeders (people who have the whole file and upload it to their peers), since, without them, no one can get the file. But you need a seeder before anyone can be a seeder. The obvious solution is to seed your own file, and this is how it’s done. But, as the ‘official’ distributor of a file, you don’t want to burn through bandwidth, so it makes sense that you’d want to throttle your available bandwidth: if there were lots of other seeders, you’d only use a small amount of bandwidth. By keeping the ‘server’ up as a permanent seeder, you alleviate the really annoying problem of no one having the full file, which, obviously, prevents anyone from ever getting it.  This is sort of a “long tail” problem: after the rush is over, you often end up with BitTorrent not being so awesome.  (And, if you set your throttled upload bandwidth to be inversely proportional to the number of seeders, when no one else is seeding it, there’s really no difference between someone downloading your file over BitTorrent and downloading it directly from your server.)

Of course, you’ll still have to distribute over FTP/HTTP, since not everyone can use BitTorrent. But, if you distribute it ‘normally’ over HTTP, you create an incentive for people to just download it from you, bypassing BitTorrent, which ruins the whole plan. So you also need to be able to throttle your bandwidth on those services, to make sure that it’s never faster than BitTorrent.

I really think there should be an all-in-one package to do this, so the host just runs a quick command on the server, and the file’s immediately being seeded on BitTorrent and available on HTTP/FTP. And for all of “us,” just think of situations that, say, Linux distributions must have with distributing large files.

This could even be a hosted service: a decent amount of people providing things like games have been smart enough to embrace BitTorrent. The market’s there. There’s just nowhere offering this.

Cool Stuff

Posted on by
Reply
  • FDC (FDCServers.net) has come a long way since I last dealt with them. (I remember back when they had a couple Cogent lines). They’ve now got 81 Gbps of connectivity.
  • Internap has long been the Internet provider when latency/speed matters. They basically buy lines from all the big providers, and peer with lots of the smaller ones, so that, unless your hosting company has their own private peering agreements, it’s basically impossible to find a shorter route. People hosting gameservers, or really just anything “high quality,” love Internap. I’ve seen prices in the $100-200 range for 1 Mbps. (This is purely for the transit: it’s all well and good to envision $100 for a 1 Mbps line to your house as good, but that’s not what it is. This is when you’re in a data center where they have a presence and run a line to them. The cost is just for them carrying your packets.)
  • FDC now has a 10 Gbps line to Internap. “Word on the street” is that Internap had some sort of odd promotion at $15/Mbps if you bought in bulk, and FDC wisely jumped, getting a 2 Gbps commit on a 10 Gbps line.
  • I’m working on getting Xen running on my laptop. It’s interested me for a long time–it’s a GPL’ed virtualization platform. You can use it on your desktop to experiment with various OSs inside VMs, but it’s also awesome on servers to run multiple virtual machines as virtual private servers.
  • Do you remember Cobalt RaQs? I distinctly remember ogling them and thinking they were the best things ever. (Of course, now we see them as 300 MHz machines…) It turns out that, when Cobalt went belly-up, they released a lot of the code under the GPL or similar. The BlueQuartz project is an active community-developed extension of that, and, combined with CentOS, it apparently runs well on “normal” computers now. (True, you don’t get the spiffy blue rackmount server or the spiffy LCD, but you do get to run it on something ten times as powerful.)
  • I’m still itching to host a TF2 server. I’ve found that they’re all either full or empty, with few in-betweens, and that a lot of them aren’t ‘adminned’ as tightly as I’d like: games like this seem to attract irritating people, and not enough servers kick/ban them.
  • cPanel seems to have come a distance since I last used that, too, and you can now license it for use just inside a VPS at $15/month.
  • Mailservers are hard to perfect. There are lots and lots of mediocre ones, but it’s rare to come across an excellent one, something that can deflect spam seamlessly, make it easy to add lots of addresses, and provide a nice web GUI. All of the technology’s out there, but for some reason, mailservers are among the hardest things in the world to configure. (Even my thermostat is easier to use!) Especially given my affinity for spamd, it’s no wonder that I’m so impressed with the Mailserver ‘appliance’ that Allard Consulting produces. It’s essentially all of the best things about mailservers (greylisting, whitelisting, SpamAssassin, Postfix with MySQL-based virtual domains, a spiffy web interface with graphs, Roundcube…), hosted on OpenBSD, coming as a pre-assembled ISO.
  • Computer hardware’s come a long way lately. I’d imagine it’d be fairly easy to assemble a machine with a good dual-core (or quad-core!) processor, 4 GB RAM, and a few 500 GB disks for around $1,000.
  • Colocation + 1,000 GB transfer on Internap at FDC is $169. (Or $199 for 5 Mbps unmetered, but that’s probably overkill.) Are you thinking what I’m thinking? (Hint: everything on this list indirectly leads to these last two point!)

Geolocation

Posted on by
5

The concept of matching an IP to a country is known as IP geolocation, often just “IPGeo” or “GeoIP.” There are lots of reasons for using IP geolocation, ranging from the mundane (identifying countries in your webserver logfiles) to the questionable (banning countries from your server to cut down on spam) to the neat (doing it at firewall/router level and redirecting a user to the closest data center).

Most of the work is just done on a country level. You take an IP (72.36.178.234, my server) and look it up in a database, and get “UNITED STATES” as an answer. There do exist databases on finer levels, down to the city, but they’re expensive and often wrong. (I keep getting ads to find hot singles in Mashpee, more than 100 miles away and in a different state… Or maybe it’s Mattapan. Whatever the case, they’re not even close.)

It turns out that you can download a free database of IP-country mappings. It’s not infallible, but they say it’s 98% accurate. The database itself won’t do you any good. It’s a compressed CSV (comma-separated variable).

In the comments section here, there’s a snippet of PHP code to take the CSV and convert it to a huge series of SQL inserts, which you input into a database… (Hint: for whatever reason, his preg_match is imperfect and leaves a few instances of the word “error” in the middle of the file. It’s probably a bad idea, but I just commented out the “echo error” line. I end up with a 5.7MB SQL query. You can also just download the thing directly here (warning: 5.7 MB SQL file). Note that, per the license terms, I disclose in the comments that it’s a derivative work of their CSV file.

The other important catch is that IPs are stored as long integers, not ‘normal’ IPs. You’ll presumably want to use PHP + MySQL to get the country associate with PHP, so I’ll provide pseudocode in a minute. PHP provides an ip2long() function, but it only takes you halfway, but leaves you with sign problems. (Argh!) It’s an easy fix, though, and you want something like the following:

$long = sprintf("%u", ip2long($ip));
$query = "SELECT a2,a3,country FROM ip2c WHERE start <= $long AND end >= $long";

You then, of course, run $query and parse through it… You get 2- and 3-letter country codes, as well as the full country name. I use it, with good results, in seeing what country comment spam is coming from. (Most of it comes from the US.)

A MySQL query isn’t the proper way to do this: there exist binary files with the same data that result in faster lookups. But this is the simplest way to start doing IP geolocation in ten minutes time, and, with the query cache enabled, there’s not a ton of overhead.

I’m tempted to write some scripts to allow people to ‘browse’ the database, either looking up an IP, or to view it by country.

Update: Weird Silence has a binary implementation of this same database that’s supposedly much faster. The main page is here, the PHP one is here, and the C one is (t)here. (I’m wondering if it makes sense to write a PHP script to call the C version, and what the performance implications would be?)

Update 2: Get your country flags here.

Whoa’8

Posted on by
Reply

One thing that I find oddly fun is thinking about possible Pres-VP combinations.

Some that come to mind are obvious: Clinton-Edwards, Obama-Edwards… Each has its own nuances that are neat to explore. But there’s another reason I think it’s interesting. In the business world, if you have a fragmented market–many sellers in a market all competing–it makes sense to try to merge some of the small guys to become a powerhouse. (Obviously, you can take this too far and become an anti-competitive monopoly.) Where this tactic is especially important is when the markets are bad. (We’ve discussed at length whether Ford and GM should merge.)

I think the Democratic race is fragmented. (Republicans, too, but in a different way right now.) We have three candidates all attracting substantial support. I have to wonder what would happen if, say, Obama somehow convinced, say, Edwards to be his running mate. Would they form a powerhouse?

There are a lot of combinations that are laughably improbable. I don’t think we’ll ever see {Clinton, Obama}-{Romney, Giuliani}. They’re at opposite ends of the spectrum, and I think {Clinton, Obama} fans would be turned off that they’d picked {Romney, Giuliani} as a running mate, and vice versa. But I do like the idea of bipartisan couplings. I also don’t think that an Obama-Clinton (or Clinton-Obama) ticket is likely. They’ve spent so much time at each others’ throats that I can’t see it working.

But here are two that I find, to quote Kucinich, viable:

Obama-Richardson: They complement each other well, and, in my opinion, are both awesome candidates. Richardson is far behind in the polls, and thus doesn’t really stand a chance of getting the nomination; I’m far from the first to talk about him being in it for VP. Obama has Senate experience; Richardson has gubernatorial experience. Obama doesn’t have much foreign policy experience; Richardson has heaps of it. Obama brings an exciting, fresh perspective; Richardson brings decades of solid experience. (I’m not implying that Obama has no experience, nor that Richardson is ‘stale’–neither is true.) And neither of them are white, which is neat in a way.

Obama-Huckabee: Hear me out! Of the Republicans, I think Huckabee is my favorite. I certainly don’t agree with every position of his, but there are two things I really like about him. One is that he’s a good, honest guy. I think anything he does will be because he thinks it’s truly the right thing to do, not because it’ll make him rich. I think Obama-Huckabee would be the “cleanest” Administration in history. (Not in borderline-racist “clean and articulate” terms, but in “actually fighting for the American people and not doing anything crooked” terms.) And the second thing is that I love the way he views his faith–a call for him to do good on Earth. A religious, conservative Republican against the death penalty and in favor of helping the poor? Wow-a-wee-wow! There are some big differences between them, and I don’t know how reconcilable they are. But there comes a third benefit, too: done right, I think a bipartisan running ‘couple’ attracts the most votes. A Republican who would never go for Obama-Clinton might be convinced to vote for Obama-Huckabee. Not to mention centrist independents.

Desk Cleaning for the OCD

Posted on by
Reply

I tend to border on obsessive-compulsive. I wax the wheels on my car. I’m still not happy with the enormous performance gains I’ve gotten by tuning my WordPress setup to move from 4 to 400 (dynamic) pages per second.

So it stands to reason that how I clean my desk is… unique. The problem is that the desk is synthetic wood, so there exist a total of zero cleaning products that work on it. Using something like Windex cleans it but leaves it looking even more dull. And don’t even think about using a wood cleaner. I used Pledge once. It looked great, but it remained slippery for about a week. The Pledge is apparently supposed to sink into the wood. This is somewhat difficult when the desk isn’t made of wood, so you instead get an incredibly slick, oily desk.

I think I finally found the key, though: car wax. You clean with one of the many cleaners just to get junk off of it–I used Simple Green, but Windex would work just as well. And then you pour liquid car wax on and apply it with a cloth as if you were applying it to your car. Wait a while for it to dry, and wipe the white hazy stuff off. The result is a high sheen. It didn’t do as well as I’d like with filling in the scratches, but it looks much better overall.

I admit it: I’m a total dork. I just waxed my desk. But… It works.

Datacenter Fiend

Posted on by
1

No matter what I do, I keep finding myself thinking about webhosting.

Netcraft does a monthly survey of hosts with the top uptime, and mentioned that DataPipe is usually on top. I’ve found that, at least for what I do, any “real” data center has just about 100% uptime. I have never not been able to reach my server. You’re either with a notoriously bad host (for example, when Web Host “Plus” bought out Dinix, they took the servers offline for a few days with no notice… that’s noticeable downtime), or you’re with a reputable host where downtime just doesn’t really happen.

So 0.00% downtime, as opposed to 0.01%, isn’t a huge deal for me. (That doesn’t mean it’s not impressive.) But what impressed me about DataPipe is that I clicked their link and their webpage just appeared. No loading in the slightest. I browsed their site, and there was never any waiting. I might as well have had the page cached on my computer, except I know it’s not cached anywhere.

Their data center is in New Jersey, but they clearly have excellent peering. I’m getting 20ms pings. They don’t (directly, at least) offer dedicated hosting, VPS hosting, or shared hosting.

One of my big concerns is that I wonder about long-term viability. The market’s full of hosts. A lot of them are “kiddie hosts,” inexperienced people just reselling space often with poor quality. That’s room for competition. But the problem is that there are hosts selling the moon: 200 GB of disk space and 3 terabytes of bandwidth for $5 a month? That’s ludicrous: that’s more than I get with my dedicated server! They can get away with it because no one uses that much, but it concerns more “honest” hosts–you’d have to charge ten times as much if everyone actually used it! But for hosts that offer, say, 1GB of space and 10 GB of transfer–a ‘realistic’ amount–they’re left vulnerable to people thinking they’re getting a better deal.

I realized the other day that, while a lot of people offer VPS (virtual private server: several people share a server, but software ‘partitions’ give each of them their own server software-wise, with root access and separation from other users), I’m really not aware of any good ones. It’s also hard to find any that offer significant amounts of disk space, or any that are particularly cheap.

Big Hosting

Posted on by
1

I tend to think of web hosting in terms of many sites to a server. And that’s how the majority of sites are hosted–there are multiple sites on this one server, and, if it were run by a hosting company and not owned by me, there’d probably be a couple hundred.

But the other end of the spectrum is a single site that takes up many servers. Most any big site is done this way. Google reportedly has tens of thousands. Any busy site has several, if nothing else to do load-balancing.

Lately I’ve become somewhat interested in the topic, and found some neat stuff about this realm of servers. A lot of things are done that I didn’t think were possible. While configuring my router, for example, I stumbled across stuff on CARP. I always thought of routers as a single point of failure: if your router goes down, everything behind it goes down. So you have two (or more) routers in mission-critical setups.

One thing I wondered about was serving up something that had voluminous data. For example, suppose you have a terabyte of data on your website. One technique might be to put a terabyte of drives in every server and do load balancing from there. But putting a terabyte of drives in each machine is expensive, and, frankly, if you’re putting massive storage in one machine, it’s probably huge but slow drives. Another option would be some sort of ‘horizontal partitioning,’ where five (arbitrary) servers each house one-fifth of the data. This reduces the absurdity of trying to stuff a terabyte of storage into each of your servers, but it brings problems of its own. For one, you don’t have any redundancy: if the machine serving sites starting with A-G goes down, all of those sites go down. Plus, you have no idea of how ‘balanced’ it will be. Even if you tried some intricate means of honing which material went where, the optimal layout would be constantly changing.

Your best bet, really, is to have a bunch of web machines, give them minimal storage (e.g., a 36GB SCSI drive–a 15,000 rpm one!), and have a backend fileserver that has the whole terabyte of data. Viewers would be assigned to any of the webservers (either in a round-robin fashion, or dynamically based on which server was the least busy), which would retrieve the requisite file from the fileserver and present it to the viewer. Of course, this places a huge load on the one fileserver. There’s an implicit assumption that you’re doing caching.

But how do you manage the caching? You’d need some complex code to first check your local cache, and then turn to the fileserver if needed. It’s not that hard to write, but it’s also a pain: rather than a straightforward, “Get the file, execute if it has CGI code, and then serve” process, you need the webserver to do some fancy footwork.

Enter Coda. No, not the awesome web-design GUI, but the distributed filesystem. In a nutshell, you have a server (or multiple servers!) and they each mount a partition called /coda, which refers to the network. But, it’ll cache files as needed. This is massively oversimplifying things: the actual use is to allow you to, say, bring your laptop into the office, work on files on the fileserver, and then, at the end of the day, seamlessly take it home with you to work from home, without having to worry about where the files physically reside. So running it just for the caching is practically a walk in the park: you don’t have complicated revision conflicts or anything of the sort. Another awesome feature about Coda is that, by design, it’s pretty resilient: part of the goal with caching and all was to pretty gracefully handle the fileserver going offline. So really, the more popular files would be cached by each node, with only cache misses hitting the fileserver. I also read an awesome anecdote about people running multiple Coda servers. When a disk fails, they just throw in a blank. You don’t need RAID, because the data’s redundant across other servers. With the new disk, you simply have it rebuild the missing files from other servers.

There’s also Lustre, which was apparently inspired by Coda. They focus on insane scalability, and it’s apparently used in some of the world’s biggest supercomputer clusters. I don’t yet know enough about it, really, but one thing that strikes me as awesome is the concept of “striping” across multiple nodes with the files you want.

The Linux HA project is interesting, too. There’s a lot of stuff that you don’t think about. One is load balancer redundancy… Of course you’d want to do it, but if you switched over to your backup router, all existing connections would be dropped. So they keep a UDP data stream going, where the master keeps the spare(s) in the loop on connection states. Suddenly having a new router or load balancer can also be confusing on the network. So if the master goes down, the spare will come up and just start spoofing its MAC and IP to match the node that went down. There’s a tool called heartbeat, whereby standby servers ping the master to see if it’s up. It’s apparently actually got some complex workings, and they recommend a serial link between the nodes so you’re not dependent on the network. (Granted, if the network to the routers goes down, it really doesn’t matter, but having them quarreling over who’s master will only complicate attempts to bring things back up!)

And there are lots of intricacies I hadn’t considered. It’s sometimes complicated to tell whether a node is down or not. But it turns out that a node in ambiguous state is often a horrible state of affairs: if it’s down and not pulled out of the pool, lots of people will get errors. And if other nodes are detecting oddities but it’s not down, something is awry with the server. There’s a concept called fencing I’d never heard, whereby the ‘quirky’ server is essentially shut out by its peers to prevent it from screwing things up (not only may it run away with shared resources, but the last thing you want is a service acting strangely to try to modify your files). The ultimate example of this is STONITH, which sounds like a fancy technical term (and, by definition, now is a technical term, I suppose), but really stands for “Shoot the Other Node in the Head.” From what I gather from the (odd) description, the basic premise is that if members of a cluster suspect that one of their peers is down, they “make it so” by calling external triggers to pull the node out of the network (often, seemingly, to just reboot the server).

I don’t think anyone is going to set up high-performance server clusters based on what someone borderline-delirious blogged at 1:40 in the morning because he couldn’t sleep, but I thought someone else might find this venture into what was, for me, new territory, to be interesting.

Geekostat

Posted on by
2

Disclaimer: I can tell right now that this is one of those late-night posts where I should be sleeping, not posting about a technical topic. But these not-entirely-lucid ones are sometimes the most fun to read.

I consider myself extremely tech-savvy. I can build a computer from parts, make my own Ethernet cables, run some performance tuning on interactive websites, write applications in numerous programming languages (as well as SQL and HTML), and much more.

But I still don’t get our digital thermostat. They’re programmed to go down to 58 at night, come up to 67 on weekends and from something like 6 to 9 a.m., and 3 to 9 p.m. on weekdays. In other words, when people are home.

Of course, me being home on vacation isn’t quite compatible with this. There’s a simple override, where you can hit the up or down arrows to set it to a temperature. While I use (and appreciate!) this, it’s also a pain. It’s really no fun waking up and having it be 58. I’d really like to reprogram it to automatically come up to 63 or so around 10:30.

I still don’t get why the whole thing isn’t on the LAN. This would have two obvious benefits right out of the gate–it’d be much easier to configure (even if you let someone with no clue about usability design the GUI, it’ll be better than the myriad knobs, switches, and buttons on our thermostat!), and it’d be more convenient in many cases to pull up a new tab in your web browser than to walk down the hall to the thermostat. (Plus, the thermostat is in my parents’ bedroom. I’d have loved to have turned the heat up a few degrees around 11 tonight, since it’s 9 outside and almost as cold inside. But something tells me they really wouldn’t have appreciated it.)

I’m also not sure that the ‘simple’ thermostat algorithm is that efficient. You figure it works something like:

while(1) {
$temp = getTemperature();
$desired = readDial();
if($temp<$desired) furnace.enable;
if($temp>$desired) furnace.disable;
}

When we view it at ‘computer speed,’ I think we can see one of the basic problems: in theory, the furnace could start flapping, where on one loop iteration it turns the furnace on, and just a fraction of a second later, it turns it off. I don’t profess to know a lot about the overhead in starting a furnace, but I’d imagine that it’s most efficient to let it run for a few minutes.

I think a much better system would be to have a programmed minimum run time: if the furnace is turned on, we should run it for at least 5 minutes. After 5 minutes, we again evaluate the temperature: if it’s at the target, we turn it off. If not, we drop into a quicker polling, maybe once every minute. Incidentally, this is much better for the thermostat’s processor, but if its sole purpose is determining whether to turn something on or off, no one really cares about minimizing overhead.

So you give it a secondary purpose: handling a TCP/IP stack and a basic webserver! All of a sudden, instead of an infinite loop, you run a tiny bit of code every 30 seconds.

You can also generate some interesting statistics. For example, how long does the furnace need to run to raise the temperature one degree? How does this scale–if you want to raise it three degrees, does it take three times as long? How does the temperature of my house look when graphed across a day? How about telling me how long the furnace ran yesterday? And, given information about my furnace’s oil consumption and our fuel costs, it’d be cool to see how much it’s costing. And it could give us suggestions: “If you drop the temperature from 68 to 67, you’ll save $13.50 a month,” or such. This would require some storage, but a gig of solid-state media (e.g., a camera’s SD or CF card) is around $10-20 now. Plus, with the advent of AJAX, you can push some of the processing off to the client–let the client use a Flash applet or some good Javascript to draw the graphs if the thermostat is underpowered!

In conclusion, I’m freezing.