Mirror Idea

My server is allowed 1 terabyte of transfer a month. I would be shocked if I exceeded 10 GB any month.

Lots of services need mirrors. I’m becoming re-interested in streaming radio stations. Some of the good ones have limited bandwidth and fill up. Most open-source packages have a series of mirrors, too. Most distributions have elaborate mirror networks, in fact.

Here’s what someone should do. Set up a mirror ‘controller.’ I hit a generic name like us.something.com wanting to download something from a US mirror. This goes on all the time, and DNS does round-robin ‘load balancing’ across mirrors.

But you take it a little further. As a site admin with 900+ GB of bandwidth going unused each year, I can sign up and say, “I’ll take up to 25 GB a day,” and “I can spare 6 GB of disk space for mirrors,” and select a list of matching projects. I might end up hosting an Ubuntu mirror. I install a daemon on my server that communicates with the mirror network, but when someone someone hits the us.whatever.com pool, I’m in the list. But, it’ll detect that it’s forwarded me enough traffic for the day and pull me out. Furthermore, the daemon on my machine can also send a “temporarily remove me” notice, either for a duration of time or until further notice. That alleviates my final fear: that I won’t exceed disk quotas or bandwidth, but that serving all those files will really tax my system. When to send the signals is entirely up to me.

I’d like to volunteer to help, but I don’t want to blindly commit to something. And because I don’t see any good way to let me commit to help within my means, I have at least 20 GB of disk space and 900 GB of bandwidth that the community can’t use.

It seems like it wouldn’t be that hard, either. It might require a little more CPU power on the part of the mirror network management, but it’s not exceedingly complex. I think a simple PHP script might be easiest… You load, say, us.project.com/project/latest.tar.gz, but latest.tar.gz is actually a script that grabs a list of available mirrors and throws a redirect to the file on one of the mirrors.

The irony is that the argument against this idea — that it’d require more servers — is exactly what the problem is trying to solve.

Leave a Reply

Your email address will not be published. Required fields are marked *