There’s a bit of buzz around slowloris, which aims to take down webservers via resource starvation via a low-bandwidth DoS attack. It’s actually somewhat like a SYN flood, but it targets HTTP servers specifically, not TCP. Basically, it opens many HTTP connections and “stutters” requests, forcing the server to handle a number of concurrent requests. What’s interesting is that the apparent “defense,” limiting the maximum number of threads the webserver can have, actually makes the attack even easier: if you configure Apache to not serve more than 150 concurrent connections, an attacker just needs to get 150 concurrent connections and send data just slowly enough not to time out.

FreeBSD has accf_http (possibly sometimes known as HTTPReady?) that will “proxy” connections by buffering the request until the full one comes in. It doesn’t seem like it was meant for security per se, so much as to help cut down the load on webservers. As is pointed out elsewhere, accf_http doesn’t handle POST requests. Even though 95% (made-up stat) of new UNIX-ish server tools are aimed at Linux users, I sometimes feel like BSD gets all the really cool ones like accf_http and spamd and OpenBGPD and so on. We Linux people get good virtualization support, though. 🙂

IIS and lighttpd aren’t affected; squid, Apache, and a few I haven’t heard of are. Of note, squid is a proxy server, not a webserver, but it’s sometimes used as a “reverse proxy” in front of webservers. From its description, varnish seems as if it may be vulnerable, too. I’m curious about why IIS and lighttpd aren’t affected. lighttpd is meant to be super-fast and pretty small, so I wonder if it just doesn’t launch new threads to handle connections. I don’t know about IIS.

What seems to kill Apache is that it’s configured to start refusing incoming connections after a set limit (150 is a common default), which is meant to keep it from dying under heavy load. I have a hunch that removing this limit would make things worse, though, since you’d then be able to exhaust “real” resources (e.g., drive the machine into swap).

As a cheap, temporary hack, you can probably get creative with firewall rules and limit the number of concurrent connections any one IP can have open with you. This makes good sense to do anyway, but it won’t stop someone with access to many machines from doing this. Actually, at a glance, haproxy looks to buffer incoming connections. Might be interesting to let it sit in front of a single Apache box (which seemingly makes no sense: “load balancing” across one server?) and see how it performs.

vim typically supports the :syntax on command, but I just found out about the :colorscheme option. But what I can’t find documented anywhere is what color schemes are options.

On CentOS 5.3, they’re in /usr/share/vim/vim70/colors/ and include blue, darkblue, default, delek, desert, elflord, evening, koehler, morning, murphy, pablo, peachpuff, ron, shine, slate, torte, and zellner. The page is extremely resource intensive since it seems to have hundreds of iframes, but this page [may crash your browser?] shows many, many other color schemes.

iptables, the Linux firewall engine, is capable of a lot more than I’d previously given it credit for. It seems like it has native support for things like OS fingerprinting and port-scan detection, plus analysis of low-level TCP headers that mere mortals like me probably shouldn’t touch. Tarpitting and ECN support, too.

It also turns out that in addition to commands like /etc/init.d iptables {start, stop, restart...}, there’s a panic that will restart iptables with a policy of dropping all traffic. Kind of a neat thing for the back of your head — just don’t do it over ssh! 🙂

Of course, as with a lot of advanced firewall tools, things get complicated very quickly. I just tried allowing NTP traffic to my machine and it’s still being refused. If I disable iptables momentarily, traffic goes right through. I need to look more into it, but suspect it involves the “custom” chains that CentOS (probably derived from RHEL) includes.

Edit: Ha. They want you to use the “RH-Firewall-1-INPUT” chain instead of just “INPUT,” and the rule I was adding was getting put at the bottom — right after the default deny rule. iptables -I RH-Firewall-1-INPUT 9 -p udp --dport 123 -j ACCEPT (don’t run without understanding: it refers to a specific position in your rules) put it right before the deny; -I lets you specify the position (9th in the RH-Firewall-1-INPUT chain) to put the rule into, as opposed to -A (add to the bottom).

I wrote off Tor a while ago. It seemed to me that 95% of people who wanted to be anonymous were up to something bad. Spammers, hackers, and child predators all seem to love Tor. But all of the talk about Iran and their attempts to silent dissent by blocking popular Internet sites got me thinking: what they need is Tor.

I kind of want to grab a cheap 1U machine and colocate it as a Tor exit node. I blogged before about some good bandwidth deals; 10 Mbps unmetered can be had for $39/month. It seems that Tor nodes can attract a lot of complaints due to bad users, though. Though it’s worth noting that Tor expected this and has some prepared documentation on how to deal with it. The question is whether the host would tolerate it, and whether I want the headache.

Actually, besides the fact that I don’t really want to pay $39/month for something that wouldn’t benefit me any, this post is making me reconsider. Of course it’s from the UK which seems to be rapidly turning into Big Brother, but it’s not as if the same thing couldn’t happen here in the US.

The highway was horribly backed up on my way home today, so I stopped at a Burger King halfway home today. The place was mobbed, and as I stood watching everything while I waited for my food, I reflected on the days when I worked–and ran shifts–at the bowling center, and on my time in college studying Management.

The first thing that I noticed is something that I find surprising that any manager would do. Don’t yell at your employees in front of customers. It’s humiliating for the employee, no doubt, but it makes the manager look inexperienced and the business look unprofessional. Take them aside and talk to them, or just politely coach them on what they should be doing. The person who took my order apologized because he only had four nickels and a couple pennies as change. He asked the manager to bring him change. She snapped back that she’d been trying to get change ever since he came into work at 5:30pm and that if he had been on time it wouldn’t have been an issue. Besides the fact that it seemed a non-sequitir, it was really unprofessional. The teenage kid being yelled at seemed like the more professional one, really.

But what I found happening in general is something that happened to us all the time when I worked at the bowling center. You started getting slammed in one department, so you’d shift more resources there, but it wasn’t enough. Your service slowed down, but it slowed down everywhere. There was a long wait for lanes, and it was made even worse because one was broken and we’d asked the mechanic to come help with customers. Food was backlogged. The trashcans were overflowing so people were just piling garbage up on counters nearby, and there was spilled popcorn all over the floor.

I started thinking today that, even though it makes intuitive sense to try to take resources from one department and throw them into the most-overwhelmed department, you’re setting yourself up for even more problems. If one department is having problems, don’t devote resources you can’t spare. It’d be like having one of the cooks in a restaurant come out and help seat people. It works in the short term, but makes things even worse in the long-run.

At Burger King, the trashcans were all overflowing and all of the napkin dispensers were empty. An employee threw a carton of napkins on the counter, since no one had time to go put them away or empty the trash. But I contend that it would have been better if one of the employees had taken the time during the rush to start in on stuff. Don’t go polishing the kitchen sinks, but take an employee out to start putting out fires as soon as they start. Fill up the napkins and empty the main trash cans that are overflowing the most. Yes, the kitchen needs all the help it can get, but it means that people eating won’t be getting in the way of customers trying to order to get their napkins, and it means that new customers won’t be greeted by overflowing trash bins and garbage on the floor.

Oh, and another thing: don’t rush your way through to keep up. I waited maybe 2 minutes for my burger and fries. It wasn’t a problem at all. But if I had known that my fries would be undercooked and literally dripping with oil, I’d gladly have waited another five minutes for them to be done right.

I guess another reason for what I’m arguing is one of the overall experience. Which of the following partially-hypothetical situations seems worse?

• I waited 10 minutes for my meal. It seemed to take a really long time. After a long wait, I got my food, grabbed some napkins, sat down in a clean booth, and ate it. It was good. By the time I was done, I had forgotten about the slow service. On the way out, I threw my trash out in the garbage can and walked away satisfied.
• I waited 5 minutes for my meal. It seemed to take a really long time. When it finally came, I realized that there were no napkins in the dispenser, so I had to wander around the restaurant trying to find them. I finally found them, after walking by some disgusting overflowing trash cans. Then I had to try to find a clean booth to sit at, before settling for one that only had some crumbs but no mustard smeared across the table or soda spilled on the booth. After cleaning my own table and thinking that the place was a dump, I finally sit down to enjoy my meal, only to realize that the French fries were undercooked and that the whole meal is really pretty crappy. While eating, I pass the time chewing the crappy food by looking around and realizing what a dump the place is. On my way out, I find that all of the trash cans are overflowing, so I leave my tray of half-eaten food and napkins on the counter by the drink fountain. I drive home and am so irked that I write a rambling blog post about how awful the experience was.

North Carolina’s Rusty DePass, described as a GOP Activist, came under fire after remarking that an escaped gorilla was probably an ancestor of Michelle Obama. When the remark (posted on Facebook) hit the news, he explained that Michelle Obama, not him, had made the comparison. When news outlets failed to find any supporting evidence of her making such a remark, he issued a waffling attempt at an apology.

At least he didn’t do my favorite thing and try to pass it off as freedom of speech.

David Letterman, meanwhile, delivered a crude joke meaning to have referenced Sarah Palin’s 18-year-old daughter, but it really fell apart when people took it to refer to the 14-year-old daughter, and it just got worse and worse as people talked about it. So Letterman issued a (lengthy!) on-air apology: “But there was a joke that I told, and I thought I was telling it about the older daughter being at Yankee Stadium. And it was kind of a coarse joke. There’s no getting around it, but I never thought it was anybody other than the older daughter… But the joke really, in and of itself, can’t be defended…  I told a joke that was beyond flawed, and my intent is completely meaningless compared to the perception.”

Sorry for all the HuffPo links. I got there by watching this Daily Show clip about Iran’s elections, which is a pretty apt summary of the elections there.

As a grammar “nut,” haphazardly-placed quotation marks tend to “annoy” me greatly. They just “jump” out at me and really mess up the “flow” of whatever I’m reading. Sometimes people use them for “emphasis,” which never made any sense, but other times it seems that people use them truly at “random” to make their text look more interesting.

When they’re not “quoting someone else’s words,” I tend to read them as “air quotes,” as something you’d say with over-dramatic winks to imply that the exact opposite of what you were saying were the case. The underage kids drank “apple juice” at the party, and people “upgraded” from Windows 98 to Windows ME when it came out.

So it’s little wonder that my friends and I cracked up laughing when an on-campus eatery put up a sign: “Fresh” Sushi! Of course, the local seafood place we used to drive by that one day changed its sign from “Fresh Seafood” to just “Seafood” was equally as amusing. (Needless to say, we came to avoid eating any seafood in the area since it was apparently all of questionable freshness.)

But today, I saw a little sign for someone’s home business that may trump the “fresh” sushi sign. Someone’s car bore a sign: Computer “Guy,” with a phone number to call for service.

Let me go on record as stating that I don’t discriminate. Male or female, gay or straight, it wouldn’t matter if I were looking for someone to fix my computer. Heck, I even blogged about supporting transgender rights a while ago. (What ever happened to that bill anyway?) But all I know is that, if I ever invite someone into my home to work on my computers, I’m going to hire someone whose advertisement doesn’t pose troubling questions.

The New MacBook Pro looks amazing. It’s smaller, faster, better, and cheaper. I practically had my mind made up to buy one, although I was dogged by an awful lot of hesitation.

But one thing irks me. Apple’s been criticized for a long time about built-in batteries. Batteries fail on the iPod and you have to send the whole thing in for service, at which point you might as well buy a new one. The battery on my work MacBook Pro failed (it held 400 mAh), but it was no big deal because we had a new spare so I just swapped them out. It’s right on the bottom: just slide a latch and it pops out.

But the new ones? In the interest of being sleek, they built the batteries inside the laptop. I think I’m going to use this as my excuse to not drop $1800 on a laptop: I’m not going to spend money on a machine that needs to be shipped to a depot if the battery fails. Maybe I’ll scoop up a used MacBook Pro on eBay as prices drop, and just be careful to get one that will take 4GB RAM as opposed to the 2GB on mine.

For a long time, I tended to think that a lot of brand-name servers were overpriced, given that I could build one with similar specs for less.

After working with some Dells at work, I came to see that there’s good reason they charge more. There are a lot of nifty onboard diagnostic tools that don’t exist on “whitebox” machines. And now that I’ve got a Proliant under my bed* until it’s ready to ship out, I’m finding even more reasons to run “real” servers. I can monitor most server health, and even connect to an onboard “lights out” management card. (Though I’m yet to figure that one out. It looks like an option you have to pay for. Plus I need a second Ethernet drop under my bed.)

• The bed muffles the roar of the fans so that it’s merely annoying instead of unbearable. Yes, I’m monitoring the temperature.

I’ve been itching to toy with spamd for ages, so I’d really hoped to get it running on my new machine. spamd relies on OpenBSD’s pf firewall, so it really won’t run on anything that’s not BSD, which means that I either need a dedicated BSD machine (not practical), or I need to get a BSD flavor up and running as a Xen virtual guest.

The new server came today — in an enormous box that I could barely lift. UPS left the box in the drizzling outdoors, but thought to cover 1/3 of the cardboard with a plastic bag. (Frankly, after several futile attempts to lift the thing, I’m just glad the UPS deliveryman didn’t smash it when trying to handle it. The cardboard was just a little damp, but the machine was well-packaged and dry as a bone.)

It’s an older dual-Xeon setup, before processors supported hardware virtualization, which limits me to operating systems that support paravirtualization. That’s a pretty limited list (most Xen development is on Linux), but fortunately, FreeBSD is one of them. Until recently, it wasn’t easy.

I’m using CentOS 5.3 for the Dom0 (host), but worried that FreeBSD in DomU was going to be really hard, especially since I’m a Linux guy, not a BSD guy. I found the AdrianChadd Xen Images page, and am now sitting inside the console on a very minimalistic FreeBSD setup. Essentially you pull down his image and tweak the Xen config; I pulled out the swap file since I didn’t feel like creating one (this is for testing, not production!), and pointed his reference to a real disk to just use the file:/ reference. It defaults to bridged networking. Then I fired off an xm create pointing to the config file he uses (with my slight modifications), and pygrub had me select the FreeBSD OS. In astonishingly little time (1 second tops?), I was inside FreeBSD. It probably helps that it’s a minimal install and that the whole disk image fits inside RAM right now.

I’m not claiming victory quite yet. I was able to bring up the “xn0” network interface and see the LAN, but then ran into trouble upon finding that there’s no such thing as /etc/resolv.conf. This is where my incredible lack of knowledge about BSD becomes apparent. There’s also the issue that it’s an older FreeBSD release and that it’s in a 512MB disk image, but the latter should be easy to remedy. I’m not sure if the upgrade will be easy; I’ve seen people mention that they had trouble with FreeBSD 7. But for a minimalistic mailserver, maybe it’s good enough.

The freebsd-xen mailing list suggests that it’s hardly a finished product, but that active development is taking place. So we’ll see how I fare getting a mailserver setup.