Category Archives: Programming

AIM

Posted on by
Reply

I frankly don’t use AIM that much these days, but will often sign on and think, “Wow, lots of people are on tonight!” or, “Wow, almost no one is on tonight!” So I just wanted to list my thought process after noticing this:

  1. I’d be interested in seeing a graph of my “buddies” online over time.
  2. It wouldn’t be too hard to write a little script to sit on AIM 24/7 and watch this.
  3. If I was doing that, I might as well log each time someone signed on and off, which would let me answer those, “I wonder if x has been online in at all lately?” questions.
  4. As long as I have a stalker bot going, it’d be even more interesting to grab their away message text and buddy profile.
  5. And as long as I’m doing that, I might as well add support for using diff to show changes in the above between any two points in time.

Is there anything that can’t be graphed? Or made into a shell script?

CraigsList

Posted on by
1

I’ve always been a little creeped out by some of the stuff on Craigslist. There’s pretty obvious prostitution and drugs going on, in addition to people seeking affairs. And if you read through the “personals” section (which is pretty entertaining), watch out for ones with pictures… Something they’re, uhh, graphic.

So I went through about 20 recent postings, merged them into a textfile, and used my old Markov chain code to “learn” the text and then spit out text based on it… Some of the stuff on Craigslist is so bizarre that it’s hard to tell what’s nonsense the script spits out, and what’s real. (I’ve omitted anything wildly obscene.)

I love to read, movies, anything to do my hair today medium length i need a new look today im off from work hit me up I have a personality that is a cheater because whats is the beginning of something possibly beautiful and long term.

I love to have a great day! A little about me…I am very mature. I am a very comfortable passenger seat. I may or may not have a degree

Good stimulating companionship and conversation is the point of being with someone if your going to cheat on them. I’m new to this online service and hoping to make new friends.Hope it works…

I love to read, movies, anything to do my hair today medium length i need a new look today im off from work hit me up

I have a personality that is a cheater because whats is the beginning of something possibly beautiful and long term.

If you are Interested to have a big black cruiser with a good place for drinks dancing live music with a rumble between her legs, for occasional rides. Feel the rumble as we hit the open road…wrap your arms around me, and press in close.

meet you, after some phone conversations, in public places only unless of course it is business related or anohter type of function, in which case I would meet you, after some phone conversations, in public places only unless of course it is business related or anohter type of function, in which case I would meet you, after some phone conversations, in public places only unless of course it is business related or anohter type of

normal and fun people, between 30-40 years of age, who are looking to meet some new people to hang out today and maybee 420 a bit. I love being out in the rewards you crave. Where you do not. Once a week, I will visit you. We will go over the goals you set yourself and your mobile number and let’s start texting!

I’m looking for someone to be a hypocrite.

Anyone know of a good place for drinks dancing live music with a lot of chrome, and a very open, spontaneous, and down to earth person.

I’m not looking for a coffee and a very comfortable passenger seat. I may or may not have a great companion. My friends think Im mischievious and I hate writing, so that’s it for me.

I have all camping gears. I am burned out of shape so don’t be shy, just be sensual.

Interested in normal and fun people, between 30-40 years of age, who are looking to make new text message with on a regular basis…what are we going to cheat on them.

I’m looking for someone to play with soon, because the weather is getting to be a marathoner!!!

I’ve heard about Cuddle Parties on the back of a good bar to watch the Celtics where they actually put the sound on the radio and internet but there are none

I’m certainly not someone who puts a twinkle in my stomache

hey im a leo male looking for possible another mom who is self motivated

I am open to a totally awesome 2-year-old boy. My problem is that I am looking to spend some time with an older 30-50.

so please be mature and not very interested in going out at night.

I consider diversity to be really nice !

If you’re not interested in talking through emails because, honestly, I can respond to anyone with a picutre

Good stimulating companionship and conversation is the point of being a hard worker. It’s simply that you have one? Reply now for flirty fun on the TV.

I have always tried to be on the radio and internet but there are none in the Boston area.

I am thankful for every day that I feel like I have no friends!!

this poing in my eyes and butterflies in my eyes and butterflies in my stomache

Please be a real person, please be open-minded and if you would like to get to know me, please just hit a back button, don’t reply.

Anyone know of a good listener and would love to go out and paint the town with.

I look forward to talking to a loser

I work 2 jobs and do not allow myself to be really nice !

my friends are married, and not very interested in talking to a totally awesome 2-year-old boy.

I can occasionally get a sitter but sometimes those are hard to come by so I need a new look today im off from work hit me up I have problems in life.

I am open to speaking to people of all races.

Quick clarification, since I was horrified at first… “Black cruiser” is a guy referring to his motorcycle; he was looking for other motorcycle enthusiasts. As was the “rumble” bit. It just happens to come up in the most inappropriate places. Also, the “2-year-old” thing comes from someone discussing that they have a child.

Faster Compression

Posted on by
Reply

It’s no secret that gzip is handy on UNIX systems for compressing files. But what I hadn’t really considered before is that you don’t have to create a huge file and then gzip it. You can simply pipe output through it and have it compressed on the fly.

For example:
[root@oxygen]# mysqldump --all-databases -p | gzip > 2008May10-alldbs.sql.gz

That backed up all the databases on this machine and compressed them. (It’s a 31MB file, but that’s nothing when you realize that one of my databases is about 90MB in size, and I have plenty others at 10-20MB each.)

Tip o’ the Day

Posted on by
Reply

The Web Developer toolbar, which is (1) the #1 hit on Google for “Web Developer,” and (2) now compatible with Firefox 3 beta, is totally awesome. You may recall that, in the past, if you had text after a bulleted list or similar on this page, the text would suddenly be mashed together. I never took the time to fully look into it, but it always irked me.

A quick “Outline… Outline Block Level Elements” drew colored boxes around each element of the page, which was exceptionally helpful. This shows the problem: posts start off inside a <p> tag, and adding a list or similar closes the <p> tag. This would have been an easy catch, except that the list looked fine. Upon a closer review, it’s because the lists specified the same line-spacing, thus looking right. While I most likely could have solved this by staring at the code for a long time, Web Developer made it much easier to spot: the first text is inside one box, followed by the list, but the other text is floating outside, leading to a quick, “Oh, I should look at how the <div> is set up” thought, which ended up being exactly the problem. (There’s a bit of excessive space now, but that’s caused by me using PHP to inject linebreaks.)

Web Developer also includes a lot of other useful tools, including the ability to edit the HTML of the page you’re viewing, view server headers, resize non-resizeable elements frames, show page comments, change GETs to POSTs and vice-versa, and much more. Whether you do design full-time, or if you just occasionally fix things, it’s worth having. And you can’t beat the fact that it’s free.

Web Compression

Posted on by
2

I’ve alluded before to using gzip compression on webserver. HTML is very compressible, so servers moving tremendous amounts of text/HTML would see a major reduction in bandwidth. (Images and such would not see much of a benefit, as they’re already compressed.)

As an example, I downloaded the main page of Wikipedia, retrieving only the HTML and none of the supporting elements (graphics, stylesheets, external JavaScript). It’s 53,190 bytes. (This, frankly, isn’t a lot.) After running it through “gzip -9” (strongest compression), it’s 13,512 bytes, just shy of a 75% reduction in size.

There are a few problems with gzip, though:

  • Not all clients support it. Although frankly, I think most do. This isn’t a huge deal, though, as the client and server “negotiate” the content encoding, so it’ll only be used if it’s supported.
  • Not all servers support it. I don’t believe IIS supports it at all, although I could be wrong. Apache/PHP will merrily do it, but it has to be enabled, which means that lazy server admins won’t turn it on.
  • Although it really shouldn’t work that way, it looks to me as if it will ‘buffer’ the whole page then compress it, then send it. (gzip does support ‘streaming’ compression, just working in blocks.) Thus if you have a page that’s slow to load (e.g., it runs complex database queries that can’t be cached), it will appear even worse: users will get a blank page and then it will suddenly appear in front of them.
  • There’s overhead involved, so it looks like some admins keep it off due to server load. (Aside: it looks like Wikipedia compresses everything, even dynamically-generated content.)

But I’ve come across something interesting… A Hardware gzip Compression Card, apparently capable of handling 3 Gbits/second. I can’t find it for sale anywhere, nor a price mentioned, but I think it would be interesting to set up a sort of squid proxy that would sit between clients and the back-end servers, seamlessly compressing outgoing content to save bandwidth.

Google Charts

Posted on by
2

Have you guys seen Google Charts? It’s a quirky little API I didn’t know existed until I saw a passing allusion to it. Essentially, you pass it a specially-crafted URL (essentially the definition of an API) and it will generate a PNG image.

Here’s a fairly random line graph. My labeling of the axes makes no sense, but it was nonsensical data anyway.

One of the cooler things they support is a “map” type of chart, like this US map. The URL is a bit tricky, though this one of South America is easier to understand: chco (presumably “CHart COlor”) sets the colors, with the first being the ‘default’ color. chld lists the countries, as they should map up to the colors: UYVECO is “Uruguay,” “Venezuela,” and “Colombia.”

What has me particularly interested is that I’ve recently installed code to watch connections to my NTP servers. Here’s my Texas box, a stratum 2 server in the NTP pool (pool.ntp.org). I bumped it up to list a 10 Mbps connection speed to signal that I could handle a lot more queries than the average, although it’s still nowhere near its limit. In addition to the stats you see there, it keeps a “dump file” of every single connection. (As an aside, this strikes me as inefficient and I want to write an SQL interface to keep aggregate stats… But that’s very low-priority right now.)

Further, I have some IPGeo code I played with. More than meets the eye, actually: a separate database can give a city/state in addition to the country. (It comes from the free MaxMind GeoLite City database.) Thus I could, in theory, parse the log file created, match each IP to a state, and plot that on a US map.

This reminds me that I never posted… I set up NTP on the second server Andrew and I got, where we’re intending to consolidate everything, but haven’t had time yet. It sat unused for a while, keeping exceptionally good time. So, with Andrew’s approval, I submitted it to the NTP pool. I set the bandwidth to 3 Mbps, lower than the 10 Mbps my Texas box is at.

I was somewhat surprised to see it handling significantly more NTP queries. (They’re not online, since the box isn’t running a webserver, but for those in-the-know, ~/ntp/ntp_clients_stats | less produces the same type of output seen here.) It turns out that a flaw in the IPGeo code assigning the server to the right ‘zones’ for some reason thought our server was in Brazil. Strangely, while the United States has 550 (at last count) servers in the pool, South America has 16. Thus I got a much greater share of the traffic. It’s still low: at its peak it looks like me might use 2GB of bandwidth.

So there are a few graphs I think would be interesting:

  • A line graph of the number of clients served over time. Using Google Charts would save me from having to deal with RRDTool / MRTG.
  • A map of South American countries, colored to show which of the countries are querying the server most frequently. (The same could be done for my US server, on a state-by-state basis.)

Geekery

Posted on by
1

Trying a different style for this post…

# # #

We held our “Rock Band Night” event tonight. The turnout wasn’t that great, but it’s a long weekend and gorgeous, so we were happy with the people we got. I brought my Xbox VGA cable, so we ran the Xbox into a projector at 1280×1024. We also pulled out an awesome sound system and hooked into that. What made things even more awesome, though, was that we realized that the projector not only has a Computer In, but a Computer Out, which just mirrors the input. So we hooked up a big monitor, and ended up with the band in front of the screen, facing the crowd, as if they were a normal band. That is how you play Rock Band. It was essentially like having a live band performing, minus the actual musical talent. The crowd was also just right, happily listening, periodically singing along, cheering good people and (good-naturedly) heckling those who missed strings of notes.

# # #

While listening, I spoke with a student who works in the admissions office, and she mentioned that she gets asked surprisingly often about video games at Bentley. We talked a bit about what we do, and then she asked if we had a website. We do, but the URL was long. So on a whim, I picked up bsgo.org.

# # #

The process of registering a new domain was interesting. It’s been a while since I went ahead with it. Initially, I inadvertently went to register.com, and merrily proceeded through the registration until it presented me with the total and asked for my credit card information. $79?! It was for a few years, but I forgot that they inexplicably charged a lot. I went to GoDaddy, which charges a more sane rate, but was constantly having to uncheck offers I didn’t want. I wanted to register the domain for one year, not several. I wanted to register bsgo.org, not bsgo.biz and bsgo.info and bsgo.tv. Every time I progressed to the next step, there were more offers for me to turn down.

On the flip side, a couple minutes later, it was live.

# # #

In the process of adding DNS records, I discovered that some of my existing ones seem corrupt / absent. www.ttwagner.com doesn’t resolve to an IP. If you notice any other assorted weirdness, let me know.

# # #

We’d talked before about making the webpage more than a barebones site with a couple of pages. Of course, then we get into all sorts of problems with preserving look and feel, and all that happens if we want to update navigation, etc. So I figured this was a great chance to try out SilverStripe, a spiffy-looking CMS. It looks very promising, although it uses some newer features in PHP that require me to update it, which has given me roundabout cause to do a lot of side-projects. (Like working on moving over to the VPS…)

# # #

I run a mailing list for the club on my machine, using Mailman. It works great, but as I graduate, I want to make sure that they’re not reliant on my server. I intend to keep hosting the list, but I’d hate for critical data to be in the hands of the aging server of someone that doesn’t even go to their school anymore. I wanted to back up the list, but Mailman lacks an “Export list…” feature. (Which annoys me almost enough to want to pick up Python just to add one in?) It turns out that it’s easy, but it took me some poking around.

Mailman, at least on Gentoo, keeps its stuff in /usr/local/mailman. There’s a lists/ folder, with a config.pck that seems to list all the members, as well as all the configuration. This might be good for backing up the list itself, but it’s pretty useless if you just want a list of members to pass on. I figured I could write a script to parse the file and extract the addresses, but I started to Google to see if it was done.

And then I found this page talking about it. And it turns out that there’s a tool to do it included with Mailman, in the bin/ folder. For me, then, /usr/local/mailman/bin/list_lists will list all of the mailing lists on the server. In addition to list_lists, there’s list_members [listname], which will do exactly what I wanted: provide a plain-text list of each member. I then redirected the output to my e-mail address….

./list_members BSGO | mail matt@example.com

# # #

I’m back in Ubuntu for the first time in a while, and I’ve got it upgrading in the background to Hardy Heron, the latest build of Ubuntu. I’m hoping that Xen will work in Hardy for me, allowing me to stay in Ubuntu permanently: I have too much Windows stuff I need to access. It’s hardly credible without data, but my dad has told me that he did some benchmarking and found that Windows running as a virtual guest on Linux actually outperforms native Windows in cases where you have a VT-capable chip. So I’m not concerned with performance, as much as whether it’ll work.

I’ve found that, for whatever reason, I’m just more comfortable in Linux than in Windows. As I was upgrading to the latest distribution, the process seemed to be slowing down. So I pulled up a command line and ran iftop, which showed me a list of my active network connections with a visualization of bandwidth on each connection. And little things like my ability to pipe the output of a command to an e-mail. This isn’t to say that one platform is “better” than the other, just that I feel more ‘at home’ when I’m on a Linux machine these days.

UCWords

Posted on by
2

Every now and then I find that I need to do something that seems remarkably obscure, and am surprised to learn that PHP has a function to do it. I found myself cleaning up a really sub-par Wikipedia article, where someone had entered a massive list of things in ALL CAPS.

It’s easy enough to upper-case everything, or to lower-case everything. But what I really wanted to was to capitalize the first letter of each word, because they’re all proper nouns. It turns out that PHP has a ucwords function to do precisely this.

And thus this page, which I suspect will never be useful to anyone, was born. Enter text, and it’ll covert it to lowercase, but upper-case the first letter of each word. And in this case, the time required to write the script was less than the time it’d have taken me to change it all by hand.