Quantcast


MySQL Replication – Permission Denied Errors

I spent a long time beating my head against the wall trying to solve this, so I thought I’d share it here in the hopes that it helps someone else.

I picked up a very cheap VPS to serve as a backup for various services, and decided to use it as a MySQL slave for backup purposes, and to provide secondary DNS (with a MySQL backend). Despite having no problem maintaining a kind-of-complicated cluster of MySQL boxes at work, I could not for the life of me get basic replication running. It consistently failed:

100829 15:56:56 [ERROR] Slave I/O thread: error connecting to master 'replication@127.0.0.1:7777': Error: 'Access denied for user 'replication'@'localhost' (using password: YES)'  errno: 1045  retry-time: 60  retries: 86400

(Yes, 127.0.0.1 is the master. Port 7777 is an ssh tunnel.) I tore my hair out, and repeatedly dropped and re-created the replication user. I could connect by hand. I tripled-checked credentials. It just didn’t work. Some people advised that master.info was sometimes saved with incorrect information; I blew that file away and re-issued the CHANGE MASTER command. Still nothing.

Finally, someone mentioned an oddity they discovered: with long passwords, replication just refused to connect. I’d used my favorite method of assigning passwords: “head /dev/urandom | shasum” So I changed my password to ‘password’ (just for testing!), flushed privileges on the master, changed the password with CHANGE MASTER, and started the slave again. Viola.

This wouldn’t be nearly as annoying if I could find it documented anywhere. I would argue that it’s a bug. But there it is — shorten your password. (12 characters is the limit I’ve seen put forth.)

Live Scanner Feeds

Radio Reference is a popular website for (police) scanner enthusiasts. It offers a pretty revolutionary concept: enthusiasts hook their scanners up to computers which relay the traffic to RadioReference’s servers, which host them as Shoutcast feeds. (I would also argue that the site is the best resource out there for scanner frequencies, and there are also forums that comprise what I can only assume is the world’s largest scanner-related community.)

For some inexplicable reason, I ignored the Live Feeds for a long time, until I got an iPhone and discovered an abundance of “police scanner” apps that were built on top of those feeds. The feeds are quite popular — the site’s owner posted a blog entry showing his latest AWS bill, which shows about 14TB of incoming traffic and more than 35TB outgoing traffic.

Besides being really valuable for those who simply don’t own a scanner, some of the feeds provide audio that is out of reach for most people. I could easily listen to the LAPD, for example, despite living thousands of miles away. There’s now a feed for Mass State Police in the area; I own a scanner capable of receiving them, but their signal is very weak where I live, so it’s advantageous to listen over the Internet, despite a bit of lag. Waltham has a feed of its own (well, plus Newton Fire). Someone else provides a Merrimack Police feed, which is advantageous because it’s a digital voice (P25) signal and a capable scanner will run you $500.

Boston Police are conspicuously absent, even though a lot of smaller cities have feeds. It seems that there’s considerable interest in setting one up, but to date, no one has. (I’d love to myself, but I have very poor reception here — part of the reason I want a BPD feed in the first place!)

In any event, I’d recommend checking the feeds out. You can listen in with a browser, or use one of many smartphone apps.

Special Interests

If anyone has ever doubted that special interests were out of control, here’s proof: the National Association of Broadcasters and the RIAA have proposed legislation that would require all cell phones to contain FM radios.

I’m not really sure why I would want an FM radio on my iPhone, when I have over a thousand of my favorite songs on it, or when I can just pull up Pandora and stream music tailored specifically to my tastes. This isn’t to say that having an FM radio on a cell phone wouldn’t be cool, just that making it the law seems rather absurd.

Gary Shapiro, the CEO of the Consumer Electronics Association, blasted the idea as “the height of absurdity” and said something everyone has thought about the RIAA for years: “Rather than adapt to the digital marketplace, NAB and RIAA act like buggy-whip industries that refuse to innovate and seek to impose penalties on those that do.” FM radio and the RIAA are yesterday’s news, and they don’t even seem to be aware of it.

My favorite part of the article, though, is the paraphrased argument from a NAB executive: “Most Web-based music services don’t include emergency alerts that radio stations broadcast, he said. Requiring FM receivers in mobile phones would help better inform the public about emergencies or bad weather nearby, he said.”

Without realizing it, he proves once and for all just how out of touch they are. I have the Internet in my pocket, and can be reached instantly 24/7. In the middle of a sandwich shop, I can pull a device out of my shirt pocket and check the weather radar. While waiting in line for the train, I check the news. They look around and see people connected like never before, and what they conclude is apparently, “Those poor people have no way of knowing about major emergencies. What they need is FM radio.”

Snooping on memcache

Not that it’s a user-friendly form, but ever wanted to snoop on memcache and see how many pointless queries are sent in?

tcpdump -vvxXs 1500 -i bond0 'port 11211'

Substitute bond0 with whatever you call your NIC.

New Main Page Theme

It seems I do this often, but the main page was getting ugly to look at it. I replaced it with another theme. It’s obviously still got some styling issues and some places where I haven’t changed the default text that came with the template I bought… But I’ll get to that in time. :) Image display isn’t working yet. I’m thinking of switching back to showing the full text of posts instead of truncating to a few hundred bytes. Any opinions?

WordPress 3.0

I’ve just upgraded to WordPress 3.0. Everything seems peachy, but let me know if there are problems. (You can reach me on gmail.com email, where my username is ‘mwaggy’ if comments don’t work?)

Anonymous.

The Oregon Tea Party put out a bumper sticker that begins, “We are Anonymous.”

Anonymous was apparently displeased, as the Tea Party Oregon Facebook page may indicate:

I’m not linking to the page, because I had to crop my screenshot to leave out the Goatse guy. If you have retinal bleach, you can search Facebook yourself.

Eating Your Own Dogfood

A big concept in the software development world is eating your own dogfood — that is, using your own code as a customer would. (Places like 37signals will — justly — argue that this is of paramount importance.) A while ago we made an effort to use our own site for internal communication, and ended up fixing a couple long-standing bugs that never seemed important until we had to deal with them.

I wonder how often this happens in the real world. Do any executives at Kraft/Ritz ever eat Handi-Snacks bought from a store? If so, they would have noticed that, about 95% of the time, at least one cracker will break in half as you lift it out. Worse, the cracks are often incomplete, so you take it out and it’s in one piece, but you start applying cheese and then it snaps, invariably landing cheese-side-down on your lap. I bet a simple change in packaging or distribution could make smashed crackers a rarity. But I also bet that executives are oblivious to the problem, even though I’ve had this problem for about a decade.

And how about manufacturers of small electronics? Has the CEO of Sandisk ever gone into BestBuy, bought an SD card, and tried to open the freaking package to get his SD card out when he got home? You need industrial scissors or a machete to get through the plastic. And then when you’ve cut the top off, it isn’t enough, so you try to rip the package, and the SD card goes flying out and lands behind the couch or something. I’m envisioning this happening to the company president one day, and him going into work the next morning in a fury of rage. “Do you have any idea how horrible our packaging is?!” Yes, everyone else has known for years.

Has Bill Gates ever had his mother call his cell phone and ask how to change the border style of a table in Word? I have, and my answer was something like, “I have no idea without seeing it… Right-click on the table — not the whole table, but just the very outline of the table — and select something near the bottom, maybe Properties? And then some menu with like five tabs will come up, and it looks like the controls to a 747. And one of them will let you change it. I seem to recall that there’s something counter-intuitive about it, though, so you might have to play with it for a while.” The response was surely something along the lines of, “That doesn’t help me at all.”

Does the CEO of Verizon have FiOS at his house? Has he ever tried calling customer service and realized that some of the people do not speak English? I couldn’t understand them, and they couldn’t understand me. It went on like that for 30 minutes. If it were my company, I would have shot myself in the head upon realizing that this is what I had built. Do the executives on the AT&T/iPhone deal all carry iPhones with AT&T service? Do they get important text messages hours after the fact, with no apparent reason for the delay? Even though they’re sitting next to someone else who got the same message the second it was sent? How many times are they out driving when they realize that they have no service?

Has the president of AAA ever broken down on the side of the highway? Did he call the 800 number, give a barrage of information, and then wait 45 minutes for a tow truck? Does the editor of the Boston Globe ever visit boston.com without a popup blocker? Does the head of the MBTA take the commuter rail to work? Has he ever almost been knocked over when the train randomly lurched forward while it was stopped? Has he ever had to deal with a rude conductor? Did he ever get splashed with water from the air conditioner that was leaking onto hte seat through a light fixture? Did he sit in one of those two-person seats and then have a second person actually try to use the two-person seat, and realize that it only seats two if both people are anorexic, but that people do it anyway?

I’m convinced that the answer to every question in this post is “Certainly not.” Because otherwise, there is no way that these things would continue to be problems.

Load Balancers and Crappiness

I was reading something about Array Networks and the load balancers with SSL offload that they offer. I followed a link to their main page, and the images slowly drew on the page. I’m on a cable modem, and thumbnail images generally load near-instantly. This is one of the images embedded on their main page. It’s 5440 x 2448 pixels, and they use the height and width parameters of the img tag to display it in browser. This is a major pet peeve of mine, and it screams of amateurism at best, or cluelessness at worst. It’s only moderately annoying on a cable modem, perhaps adding about a second to the page load. Can you imagine if you were on dialup?

What really gets me, though, is that these people should know better! Although proper HTML design isn’t their industry per se, making sites responsive is arguably their company’s main purpose. It would be like a marketing firm having misspellings on their homepage — grammar isn’t their industry, but they’re all about presenting a pristine image, so it’s inexcusable to make mistakes like that.

But while you’re at it, here’s an interesting article about using a dedicated nginx proxy as an “SSL accelerator.” Most people doing SSL offload move it to a machine with hardware crypto cards (a la the Cavium Nitrox line), or, more commonly, dedicated appliances. This article argues that a decent level of commodity hardware can match the performance, and, in the case of some top-shelf equipment, do it for a tenth of the cost. (Granted, it replaces an appliance with a whizz-bang GUI and 24/7 support with a homebrew system…)

It’s also interesting to note that you can easily turn this into something that does all (or at least, most) of the magic that something like a BigIP box does. Rather than using it for simple SSL acceleration, you could also use it as a load balancer by throwing something like haproxy on it. (I admit: I have a little bit of a crush on its request buffering capability. Look at the Apache thread scoreboard sometime and note how much time is ultimately just waiting for clients.) And throw in varnish (with SSDs!) and your SSL-offloading, load-balancing appliance is suddenly a caching reverse proxy, too. Granted, there’s a big downside here: you’ve just moved an awful lot of components onto one single box, introducing a horrible single point of failure. At some level you’ll get this with an enterprise load balancer too, but it’s the same issue described above: it’s some sort of custom homebrew thing running on commodity hardware, so the single point of failure is pretty scary. You could mitigate it somewhat with redundancy, though.

I also found a reference to Strangeloop. Their site is light on details, but reading between the lines, it seems like, among other tasks, it might consolidate and minify CSS and JS files, etc., which can reduce multiple calls. This task should really happen when building a site, not on the fly when serving the page, but at the same time I can understand how you get there — perhaps you’re responsible for the upkeep of a site where all of the developers have left the company, or perhaps the systems and network people are sufficiently divorced from the web people…

Now if only they could make their product automatically resize images on the fly to fit the sizes described by the image tags, they could sell to Array Networks!

Sapeurs

A recent Cracked.com article — yes, I’m citing it as a serious source — introduced me to the concept of Congo’s SAPE, from Société des ambianceurs et des personnes élégantes, which translates to the “Society for the Advancement of People of Elegance.” Adherents, known as sapeurs, dress in dapper, sometimes garish attire, despite living of modest means in an impoverished nation.

The origins of the practice trace back to French colonization of the area, but have an intensely regional twist. President Mobutu attempted to ban three-piece suits as part of his effort to eradicate colonial influence, but not everyone could be convinced. Musician Papa Wemba is also credited with popularizing the look in the seventies, performing in fine European suits.

A decent introduction can be found on Oddity Central, and an article presenting some critical views can be found on AfricaFeed. And there are plenty of galleries such as here, here, and here.

Oh, and there’s a book: Gentlemen of Bacongo.

Page 1 of 12212345...Last »