DB Stats

I’ve been playing with phpMyAdmin and doing a bit of optimization of it. A few stats:

  • Since I upgraded the kernel, MySQL has been up for a little under 3 days and 11 hours.
  • The DB server has moved 841 MiB of traffic. This is 10 MiB an hour.
  • It’s processed 131,048 queries. This is about 1,580 an hour.
  • 132,000 inserted rows.
  • 96K queries served out of MySQL’s query cache.
  • 1,393 temporary tables created on disk to handle queries. This seems like a bottleneck, although it is only a tiny percentage.

I’ve just restarted MySQL to apply some configuration changes. (Actually, I could have changed them on the fly now that I think about it…) I tweaked the settings a bit: MySQL allows you to set limits on how much RAM it can use for various operations, and I tend to be very frugal. But I think I was shooting myself in the foot there: it was relying on disk a bit too much. It’s not like I’m running a load average of 25 and am moving gigs of traffic a day, where tuning is really vital, but it still bothers me that it’s not as efficient as it could be.

No, No You Don’t

I periodically peruse access_log and error_log, Apache’s logfiles. There’s always weird crap. Here’s todays (odd linebreaks to make it fit):

[Thu Nov 22 07:59:15 2007] [error] [client 80.32.3.251] File does not exist: /var/www/ardentdawn.org/htdocs/drupal, referer: http://72.36.178.236/drupal/?_menu[callbacks][1][callback]=drupal_eval &_menu[items][][type]=-1&-312030023=1 &q=1/IROCKTHEWORLD%22);

There’s also some inept script kiddie later on who tried requesting the same non-existent page 112 times in a row… o_O

Undoing bad tar files

Proper ‘etiquette’ for packaging a tar file is to package it so that it extracts to a directory. But sometimes, the files are packaged by an idiot, and, when extracted, just extract the files right to whatever directory it’s in. (Which is fine, if you expect it.)

tar takes a “t” option (“t” for “list”–get it? I don’t…) to list (list?) the files in a directory. You can use it two ways:

  • Pre-emptively: tar ft file.tar will show you how it’d extract.
  • Retroactively: rm `tar ft file.tar` will list the files, and pass them as an argument to rm, deleting the mess it just made.

Geekery

One of my weird OCD concerns is that some of the scripts I host place a heavy load on the server. I want to make sure that, in busy times, they don’t weigh down things further. Here’s a neat little bit of PHP I wrote to simply have PHP abort the page load if the 1-minute load average is over 2.00:

// Check the uptime first
$fh = fopen('/proc/loadavg', 'r');
$uptime = fread($fh, '4');
fclose($fh);

if ($uptime>2) {
die("Sorry, we're too busy.");
}

Rather than die(), you might throw a redirect to a cache or something else. And I should point out that, of course, running this code does take some CPU time… And that this script doesn’t always make sense: you’re basically forcing a failure before the server itself forces the failure. The time it makes sense is in the way I’m using it — when some unimportant, tangential project requires inordinate resources and you want to make sure it doesn’t slow the server down too excessively, at the expense of the more important projects (e.g., the blogs).

Today’s Deals

This week, Office Depot wins the award for best deals:

  • eMachines desktop and monitor combo. Athlon 3800, 1 GB RAM, 160GB disk, dual-layer CD/DVD-R, Vista Home Premium, and a 21.6″ LCD. Their website seems devoid of information on it, so I can’t figure out what I really want to know–the resolution on that monitor. But at $399.99 after rebates ($649.99 in store), it seems like a good deal any way you spin it.
  • If you’re more of a laptop person, they have a Toshiba laptop with a dual-core Athlon TK-53 (which means nothing to me), 120 GB disk, 17″ “TruBrite” widescreen monitor, and 2 GB (!) RAM. Typical dual-layer CD/DVD burner and Vista Home Premium. Integration 802.11b/g (but not a or N)…. $599.99 after rebates, $749.99 in store.
  • “Becker Traffic Assist GPS” with voice directions and a 3.5″ LCD (touchscreen). I’ve never even heard of Becker, but at $129.99 (after rebates: $199.99 in store), it seems like an incredible value.
  • 2GB CompactFlash card, $19.99 in store. I’ve been watching CF prices for a while now that I have a CF camera, and this is definitely a good deal. The problem is that I’m not sure I need a 2GB card? Same price for a 2GB SD card.
  • Epson all-in-one machine (printer/copier/scanner/fax), $44.99. Includes an LCD so you can print right from digital media. As usual, price is after rebate
  • $649.99 ($799.99 in store) for a quad-core Intel (Q6600), 2GB RAM, 400 GB hard drive.
  • $6.99 for a 1GB USB thumbdrive. Those came down in price awfully quickly!
  • $27.99 for a 4GB (!) thumbdrive.

I really wish I could justify the purchase of a new computer, because that $400 desktop system is very tempting.

The Bible

I found a script that does that sort of Markov chains mentioned. I use it in PHP.

I needed a large body of text, though. Just using someone’s blog posts, for example, just results in a lot of repetitiveness. It’s no good. For bonus points, I wanted a large body of text that sounded kind of strange no matter how it was read.

So I found the Bible. It’s doubly good because the wording is pretty archaic, so you’re use to having to carefully analyze it to divine some meaning. While a guy on a forum saying he recently spent an evening with a grain of salt comes across as nonsense, in the context of the Bible you might try to read into it. This is perfect for this script!

Here’s the page. A lot of it’s sheer nonsense, but some of it’s incredibly good. In lieu of actual verse numbers, the script picks up on the numbers and very consistently plugs in two numbers in front of text.

Some recent highlights:

22 7 And David said unto Saul, I saw gods ascending out of the land to bury with the passengers those that remain upon the face of the earth, and upon every high mountain…

The zombies are coming? To kill the living?

5 11 Woe unto them! for their day is come, the time that David was escaped from Keilah; and he forbare to go forth. 23 14 And he went through the corn fields on the sabbath days. 4 32 And they were offended in him. But Jesus stooped down, and with his mouth, and began at the same scripture, and

This is one of those ones that almost tells a ‘coherent’ story about David escaping from Keilah, running through cornfields even on the Sabbath, which offended people. But Jesus stooped down to begin scripture. I’m fairly certain that no such verse appears in the Bible, though.

22 3 And David prepared iron in abundance for God had made them rejoice with great joy

That’s not much of a party….

Of course, sometimes it seems to get in a sort of loop… Anyone who’s read the Bible will recall that it, at various times, launches into really lengthy lists of people’s names and the relations between them. So I cringe whenever it begins doing that, because sometimes it just doesn’t stop. Here’s a good illustration of that:

are honest, whatsoever things are lovely, whatsoever things are honest, whatsoever things are just, whatsoever things are pure, whatsoever things are true, whatsoever things are honest, whatsoever things are honest, whatsoever things are honest, whatsoever things are true, whatsoever things are lovely, whatsoever things are honest, whatsoever things are just, whatsoever things are pure, whatsoever things are true, whatsoever things are lovely, whatsoever things are lovely, whatsoever things are just, whatsoever things are pure, whatsoever
29 2 And he placed forces in all the coasts thereof, from two years old was Jehoash when he began to reign, and he reigned eleven years in Jerusalem.

My biblical history isn’t so hot, but I’m fairly certain that rulers had to be at least three to begin their reign.

15 6 In the morning sow thy seed, and in the water

o_O

40 4 And the glory of their strength in the tabernacles of Ham
16 59 For thus saith the Lord GOD; Behold, I will stand upon my watch, and set me in dark places, as they that must give account, that they may lay hold on bow and spear; they are cruel, and have no child, and her husband were dead, she bowed herself and travailed; for her pains came upon her.

Say what?

33 25 Wherefore say unto them, My little finger shall be thicker than my father’s loins.

Is that an actual verse? It sounds like it may have been the equivalent of a your-mom insult from the biblical era?

Anyway, go see for yourself. Just don’t expect every verse to be good.

vim Trick of the Day

:g/nbsp/d

That command sets up a “range” of all lines (hence the g for global) that match “nbsp”, and runs the command “d” (delete) on them.

I’m working with a file that was converted by a script from HTML, but had some carryovers that were whole unnecessary lines… No lines with ‘desirable’ content had the nbsp in them, just the junk ones. So we delete them all.

Kudos to this site for the inspiration.

There Goes My Hero

Watch him as he goes! It was the usual “wasting time on Wikipedia” path — I started reading about nuclear fission, and then read about Los Alamos, and then read about the supercomputers, one of which ran Plan 9, so I read about Plan 9, and then its GUI, and then the guy who wrote the GUI. And there was an allusion to someone else, Mark V. Shaney. So I read about him.

In a nutshell, it was a script a few of the Plan 9 guys wrote that would process a lengthy body of text and do some statistical analysis, and use that to spit out writing. It was AI, in a sort, but “schizophrenic” is the best way I’ve seen it described. You read it and it’s one of those things where, for a minute, it makes sense, but then it radically shifts topics or draws some sort of completely irrelevant conclusion. Kind of like a lot of people on the Internet, actually.

They had some fun with textbooks. Here‘s an example, in which the code was fed a basic arithmetic textbook:

Why do we count things in groups of five. When people learned how to count many things, they matched them against their fingers. First they counted out enough things to match the fingers of both hands. Then they put these things aside in one quart. A giant-size bottle that will hold four quarts is a three-digit number….

It starts of making good sense, but suddenly they go from counting on your fingers to putting “these things” in a quart, and is pretty incomprehensible from there.

Here’s another really funny one. You read it, and can kind of comprehend it. But the first reply summarizes it well: it suddenly shifts from constipation to understanding the 19th century, with no logical shift. I think that commenter may have been aware of what was going on. The second guy accurately nails what’s going on.

Finnegan’s Wake? This one cracks me up a lot. But you read this, and doesn’t it exactly sum up what’s wrong with Internet forums? The people just seem totally bonkers, and like they’re ranting but not really sure what they’re ranting about. He manages to talk about being good in bed and using the latest version of BSD in the same sentence. The reply is hilarious, because it’s exactly what you’d think if you didn’t know what was going on: that the “guy” posting was on some serious drugs.

This one, though, is my all-time favorite. It starts off as some religious rant, but clearly not a coherent one. But the fifth paragraph is the best paragraph ever written:

When I meet someone on a professional basis, I want them to shave their arms. While at a conference a few weeks back, I spent an interesting evening with a grain of salt. I wouldn’t take them seriously!

I’m fairly certain there are AI ‘bots’ out there that do this same thing, maybe in more coherent forms. I want to acquire one. Badly. I’ve always been interested in the ‘bounds’ of nonsense—when something kind of makes sense, you work with it. We “understand” people shaving their arms in professional settings, and we can visualize someone spending an evening with a grain of salt, and I surely wouldn’t take them seriously afterwards. But we’re making ‘sense’ out of sheer nonsense generated by a computer. How far will it go before we think, “This is complete nonsense.”

Defragging for the OCD

My Windows hard drive is a 60GB drive, and is always full-ish. With 8% free space (really good for this drive!), a defrag doesn’t get a lot accomplished. The small files are reassembled, but none of the big ones.  There’s not enough room to piece together the paging file.

So here’s how I, a definite OCD-sufferer, am cleaning up my Windows machine:

  • Create a desktop folder, “Crap,” and drag everything on the desktop into it, except for things that I know should definitely stay.
  • Plug in external 500GB hard drive.
  • Move the Crap folder to the external drive.
  • Move everything in My Documents to the external drive.
  • Empty the trash bin.
  • Run CCleaner.
  • Fire up your paid version of Diskeeper (it’s worth it, I promise: and I hate paying for software). Set up a boot-time defrag, and have it get the paging file and MFT as well.
  • Move everything back. Or, realize that you don’t need 3/4 of it and don’t move it back.

Warning: I’m somewhat concerned that some things might not take well to being moved around, like my iTunes Library. I’m posting this as I’m finishing up copying everything over, so it’s possible that this isn’t going to work out as I planned. We’ll see…

Crappy Apps

Am I the only one that has to put up with terrible interfaces all day, every day?

The work order system for submitting requests to facilities management (Datastream) has a number of irritating flaws:

  • It only works in IE… I just happened to be in Windows right now, so it didn’t require anything other than switching browsers.
  • It requires pop-ups. SP2, by default, doesn’t allow them. It’s simple enough to allow them, but it’s a nuisance.
  • The link isn’t at all easy to find from the main Bentley site.
  • You need to log into this specific URL that specifies what building you’re in. The problem is that this information is tied to your username anyway, so you really don’t need to specify it in the URL. Except that, apparently, you do.
  • You log in with your student ID. Our student IDs begin with an @, and are then an eight-digit number. I never got the @ sign, but you can’t log in with it. It’d take a one-line script to strip the @ out if provided.
  • It’s some obnoxious Javascript/Flash interface that requires double-clicking on page elements. There is no reason this can’t be done with HTML forms? Which would also allow the interface to be used outside of IE.
  • You can view tickets, past and present, for anyone in your building. I suppose it’s not exactly confidential information, but why do you let me see that the guy on the first floor has to have someone come spray for ants?

In all seriousness, I could write the code to do this in a day, by myself?

And then our library has this interface to schedule meeting rooms. We have about 20 rooms. It’s terrible. It seems to connect to about 20 IPs when loading, which gives me strong reason to believe that every page load goes out and connects to every room. (Haven’t they ever heard of caching?!) There are always 2 or 3 rooms that don’t load, and often the tables load all funky. And it takes about 30 seconds to load. The problem is that it auto-refreshes every minute or so. So you’ll finally get the room to finish loading, and as soon as you lay eyes on an open room and go to click on it, the page refreshes and starts the whole process all over again.

And even when it does work, if you try to click on a certain date, instead of you showing you the room schedule for that day, it takes you to a little page with a picture of one of the rooms. How this isn’t a bug I don’t quite understand.

Again, this is a Programming 101 assignment.  Any of us on here could write something that would work better in a spare afternoon.

But then I started thinking… That’s maybe 5 web interfaces I use, 2 of which are unbearable. That’s 40% garbage. That’s a pretty bad statistic?