Archive for the ‘Computers’ Category
Kingston (who I never knew made CF cards) has a 4GB CompactFlash card for $24 at NewEgg. At 8MB writes and 10MB reads, it’s hardly blazing, but if you don’t do any high-volume activity, it’s not a bad deal.
Actually, if you’re not looking for speed or name brands…
They seem better-suited for a ultra-low-power PC than a camera, of course: no one needs a 32GB CF card, and the type of people that would ‘need’ one are the type of people that couldn’t put up with anything but the highest throughput available.
A really random thought just popped into my head…
UNIX systems have their “swap” partition, a disk partition where unused stuff in RAM is “swapped out” to disk to make room for newer stuff in RAM. Of course, no hard drive is as fast as RAM, so you obviously want lots of RAM so you never touch swap. Windows has the same concept but calls it a paging file.
But what if your disk was as fast as RAM? I remain fascinated by OCZ’s new 64GB SSD, which has an almost-zero seek time, and throughput rivaling the best of hard drives. (Though I’m yet to read any actual reviews, as it seems to have still not quite shipped.) I suspect that, given factors like the direct bus between your CPU and RAM, and all the work on boosting RAM “speeds,” a solid-state disk wouldn’t literally be as fast as RAM. But I also think that the difference between having more RAM and “swapping out” to SSD would be somewhat negligible.
I think it’d be interesting to test the extent of this… Plunk an SSD (one with high throughput!) into a system, and run it on as little memory as you can. (Though I think you’d be hard-pressed to find anything less than 256MB DIMMs these days, and even those might be hard to find? I wonder if Linux has a facility for deliberately overlooking a quantity of its RAM?) And with that set up, go crazy with stuff to ensure that you’re using several gigs of “memory.”
We can sit around all day and measure bus speeds and Mbps throughput and latency and seek time, but I think the only meaningful measure would be to try this and see how it feels. I have a hunch that it wouldn’t be that big of a slowdown (compared to RAM), but that the biggest problem would be ensuring your SSD was on a separate controller/ bus/ channel, so you didn’t obliterate your hard drive performance. While it’s easy to get an affordable system with a couple gigs of RAM now, RAM remains expensive if you need a decent amount of it. Buying a system with 64GB of RAM would still be extraordinarily expensive, but with a 64GB SSD for under $300, you could imitate it fairly well.
I’d posted before about my interest in picking up a low-capacity SSD card for my laptop, to drastically speed up disk access. (This actually has nothing to do with my recent posts about slow hard drives…)
Newegg seems to have a 64 GB SSD, 2.5″ SATA disk for $240 after rebate. Interestingly, from the specs, it seems as if not only are the seek times nill (on account of being solid-state), but the throughput exceeds that of your average hard disk. It won’t be released for four days, however. (Found via FatWallet, which also links to a review here.)
For those who aren’t major geeks, SSD is short for “solid-state disk.” Your ordinary hard drive is a bunch of spinning platters, whereas solid-state is the technology you see in a USB thumb drive or the like: no moving parts. The major benefit of SSDs thus far has been seek time: with a normal hard disk, the disk has to find the right spot on the disk and read it. Seek times average 8-10ms on most normal drives, but that adds up quickly with fragmentation or concurrent I/O. With an SSD, there are no moving parts, so “seek time” is pretty much non-existent: files are ready instantly. Early SSDs seemed to not be capable of moving as much data (in terms of MB/sec), though, meaing that SSDs were great for lots of small “random” access, but not so hot for handling big, contiguous files. Now, it’s looking as if OCZ has made SSDs kick butt over normal hard drives, and somehow offered the product at a fraction of what it normally costs. (This 64GB SSD is more normally-priced, to give you an idea of why they haven’t caught on so quickly.)
Incidentally, today I came across deals on two different notebooks for about $700, both of which have 4GB RAM, but 1280×800-pixel screens. The RAM is incredible, as are most of the other specs (though it’s 5400RPM drives), but I think you can do much better on the resolution.
I think I’ve alluded earlier to the fact that I’ve been trying to speed up some systems at home, and how some of them are really slow. (I’m starting to suspect Norton, actually, but more on that when I find out more.)
I just came across this spiffy application, which will write and then read a test file to measure disk performance. My laptop gets 27.1 MB/sec. (sequential) write, 41 MB/sec. sequential read, and 29.9 MB/sec. random reads. This was on a 1GB file; it wanted to do a ~4 GB file, but I really didn’t feel like spending the time. I suspect the goal is to make sure that it’s not being “fooled” by caching, but I figured 1GB was sufficient for that. Some of the results show read speeds of 600+ MB/sec., which is most definitely coming from cache. (That said, this is a more “real-life” test… Just don’t think you have a hard drive that does 800MB/sec. reads!)
This post christens my newest category, Thinking Aloud. It’s meant to house random thoughts that pop into my head, versus fully fleshed-out ideas. Thus it’s meant more as an invitation for comments than something factual or informative, and is likely full of errors…
Aside from “time geeks,” those who deal with it professionally, and those intricately familiar with the technical details, most people probably are unaware that each of the GPS satellites carries an atomic clock on board. This is necessary because the way the system works, in a nutshell, by triangulating your position from various satellites, where an integral detail is knowing precisely where the satellite is at a given time. More precise time means a more precise location, and there’s not much margin of error here. The GPS satellites are also syncronized daily to the “main” atomic clock (actually a bunch of atomic clocks based on a few different standards), so the net result is that the time from a GPS satellite is accurate down to the nano-second level: they’re within a few billionths of a second of the true time. Of course, GPS units, since they don’t cost millions of dollars, rarely output time this accurately, so even the best units seem to have “only” microsecond accuracy, or time down to a millionth of a second. Still, that’s pretty darn precise.
Thus many–in fact, most–of the stratum 1 NTP servers in the world derive their time from GPS, since it’s now pretty affordable and incredibly accurate.
The problem is that GPS isn’t perfect. Anyone with a GPS probably knows this. It’s liable to be anywhere from a foot off to something like a hundred feet off. This server (I feel bad linking, having just seen what colocation prices out there are like) keeps a scatter plot of its coordinates as reported by GPS. This basically shows the random noise (some would call it jitter) of the signal: the small inaccuracies in GPS are what result in the fixed server seemingly moving around.
We know that an error in location will also cause (or, really, is caused by) an error in time, even if it’s miniscule.
So here’s the wondering aloud part: we know that the server is not moving. (Or at least, we can reasonably assume it’s not.) So suppose we define one position as “right,” and any deviation in that as inaccurate. We could do what they did with Differential GPS and “precision-survey” the location, which would be very expensive. But we could also go for the cheap way, and just take an average. It looks like the center of that scatter graph is around -26.01255, 28.11445. (Unless I’m being dense, that graph seems ’sideways’ from how we typically view a map, but I digress. The latitude was also stripped of its sign, which put it in Egypt… But again, I digress.)
So suppose we just defined that as the “correct” location, as it’s a good median value. Could we not write code to take the difference in reported location and translate it into a shift in time? Say that six meters East is the same as running 2 microseconds fast? (Totally arbitrary example.) I think the complicating factors wouldn’t whether it was possible, but knowing what to use as ‘true time,’ since if you picked an inaccurate assumed-accurate location, you’d essentially be introducing error, albeit a constant one. The big question, though, is whether it’s worth it: GPS is quite accurate as it is. I’m a perfectionist, so there’s no such thing as “good enough” time, but I have to wonder whether the benefit would even show up.
From my “Random ideas I wish I had the resources to try out…” file…
The way the “pretty big” sites work is that they have a cluster of servers… A few are database servers, many are webservers, and a few are front-end caches. The theory is that the webservers do the ‘heavy lifting’ to generate a page… But many pages, such as the main page of the news, Wikipedia, or even these blogs, don’t need to be generated every time. The main page only updates every now and then. So you have a caching server, which basically handles all of the connections. If the page is in cache (and still valid), it’s served right then and there. If the page isn’t in cache, it will get the page from the backend servers and serve it up, and then add it to the cache.
The way the “really big” sites work is that they have many data centers across the country and your browser hits the closest one. This enhances load times and adds in redundancy (data centers do periodically go offline: The Planet did it just last week when a transformer inside blew up and the fire marshalls made them shut down all the generators.). Depending on whether they’re filthy rich or not, they’ll either use GeoIP-based DNS, or have elaborate routing going on. Many companies offer these services, by the way. It’s called CDN, or a Contribution Distribution Network. Akamai is the most obvious one, though you’ve probably used LimeLight before, too, along with some other less-prominent ones.
I’ve been toying with SilverStripe a bit, which is very spiffy, but it has one fatal flaw in my mind: its out-of-box performance is atrocious. I was testing it in a VPS I haven’t used before, so I don’t have a good frame of reference, but I got between 4 and 6 pages/second under benchmarking. That was after I turned on MySQL query caching and installed APC. Of course, I was using SilverStripe to build pages that would probably stay unchanged for weeks at a time. The 4-6 pages/second is similar to how WordPress behaved before I worked on optimizing it. For what it’s worth, static content (that is, stuff that doesn’t require talking to databases and running code) can handle 300-1000 pages/second on my server as some benchmarks I did demonstrated.
There were two main ways to enhance SilverStripe’s performance that I thought of. (Well, a third option, too: realize that no one will visit my SilverStripe site and leave it as-is. But that’s no fun.) The first is to ‘fix’ Silverstripe itself. With WordPress, I tweaked MySQL and set up APC (which gave a bigger boost than with SilverStripe, but still not a huge gain). But then I ended up coding the main page from scratch, and it uses memcache to store the generated page in RAM for a period of time. Instantly, benchmarking showed that I could handle hundreds of pages a second on the meager hardware I’m hosted on. (Soon to change…)
The other option, and one that may actually be preferable, is to just run the software normally, but stick it behind a cache. This might not be an instant fix, as I’m guessing the generated pages are tagged to not allow caching, but that can be fixed. (Aside: people seem to love setting huge expiry times for cached data, like having it cached for an hour. The main page here caches data for 30 seconds, which means that, worst case, the backend would be handling two pages a minute. Although if there were a network involved, I might bump it up or add a way to selectively purge pages from the cache.) squid is the most commonly-used one, but I’ve also heard interesting things about varnish, which was tailor-made for this purpose and is supposed to be a lot more efficient. There’s also pound, which seems interesting, but doesn’t cache on its own. varnish doesn’t yet support gzip compression of pages, which I think would be a major boost in throughput. (Although at the cost of server resources, of course… Unless you could get it working with a hardware gzip card!)
But then I started thinking… That caching frontend doesn’t have to be local! Pick up a machine in another data center as a ‘reverse proxy’ for your site. Viewers hit that, and it will keep an updated page in its cache. Pick a server up when someone’s having a sale and set it up.
But then, you can take it one step further, and pick up boxes to act as your caches in multiple data centers. One on the East Coast, one in the South, one on the West Coast, and one in Europe. (Or whatever your needs call for.) Use PowerDNS with GeoIP to direct viewers to the closest cache. (Indeed, this is what Wikipedia does: they have servers in Florida, the Netherlands, and Korea… DNS hands out the closest server based on where your IP is registered.) You can also keep DNS records with a fairly short TTL, so if one of the cache servers goes offline, you can just pull it from the pool and it’ll stop receiving traffic. You can also use the cache nodes themselves as DNS servers, to help make sure DNS is highly redundant.
It seems to me that it’d be a fairly promising idea, although I think there are some potential kinks you’d have to work out. (Given that you’ll probably have 20-100ms latency in retreiving cache misses, do you set a longer cache duration? But then, do you have to wait an hour for your urgent change to get pushed out? Can you flush only one item from the cache? What about uncacheable content, such as when users have to log in? How do you monitor many nodes to make sure they’re serving the right data? Will ISPs obey your DNS’s TTL records? Most of these things have obvious solutions, really, but the point is that it’s not an off-the-shelf solution, but something you’d have to mold to fit your exact setup.)
Aside: I’d like to put nginx, lighttpd, and Apache in a face-off. I’m reading good things about nginx.
In case anyone else is struggling like I am… After you install MySQL on CentOS, it doesn’t seem to want to start… You run mysqld and it barfs up an error.
It turns out that it’s because you need to run service mysqld start instead.
(And if you’re looking for how to install GD2 for PHP, and frustrated because a search of “gd2″ in yum turns up nothing… That’s because it’s called php-gd, not php-gd2.)
I’m going to use the default Apache and PHP that come from the CentOS repositories (until I get irritated enough with how old they are…).
So I just did an install of php-pecl-memcache and let yum fetch the “dependencies,” which, in this case, included Apache and PHP. O:-)
In other news, I’m slightly confused about why memcache isn’t a package I can install, but the libraries for it are?
I made a bunch of changes all at once, and suddenly my system froze when I tried to log in, just saying “Loading your personal settings…”
For a long time, I’ve wanted Windows to show me exactly what it was doing, since “Loading your personal settings…” means nothing. Is it choking on a config file? Trying to reconnect to the network share that doesn’t exist anymore? Is my new anti-virus software conflicting with the old?
I’m still not entirely satisfied, but it turns out that Windows does support extended messages in the login dialog: in HKEY_LOCAL_MACHINE / Software / Microsoft / Windows / CurrentVersion / Polices / system, create a DWORD called “verbosestatus” and set it to 1. (And, per some of the online guides, make sure you don’t have a “DisableStatusMessages” key, or at least make sure it’s set to 0.)
Now, instead of getting “Loading your personal settings…” I can see exactly what file it’s loading. Although to be honest, that wasn’t at all helpful in this case, but this is a setting I’m going to leave on.
As an aside, don’t ever run two anti-virus programs at once. I’m pretty sure that’s the program. Safe Mode doesn’t let you remove software (stupid! stupid! That’s why I needed to get into Safe Mode), but I remembered the old msconfig (Start -> Run -> “msconfig”), where I was able to be judicious in disabling both anti-virus applications, along with some other services that I really don’t need running in the background anyway. And now it works like a charm.
I joined my family today in central/Northern New Hampshire. It was opening day at Clark’s Trading Post, a favorite of my brother. My brother encouraged me to go with them, and the forecast called for a decent day, so I figured I’d tag along. The area’s always been quite photogenic, and I figured it’d be a good chance to continue my exploration into HDR.
That’s the train at Clark’s. This is what I like to think of as a halfway-decent HDR shot: a close inspection will reveal some technical flaws, but it’s normally a difficult subject to shoot well. It’s largely a black train, glossy in parts, matte in others, but it also has shiny metal highlights, plus the sky. This isn’t to say that it can’t be photographed, just that the results are ordinarily less than stunning. What I like about this shot is that your first thought isn’t, “What type of surrealist artwork is this?!” As seems to be typical of my shots, the sky looks kind of wonky, but overall, I’m happy with the shot.
There’s an example of the type of stuff I’m still on the fence about. It’s just kind of jarring in a way, as the colors, while “correct” in a sense, are unnatural. Rather than correcting for the fact that the camera can’t capture the whole scene the same way the human eye might, it goes further and does something even our eyes can’t. It’s a little surreal, but the style is growing on me. (Trivia: look closely and see how many things you can spot wrong. When stitching together multiple photos in which people are moving, things are bound to not quite match up right. They’re fairly subtle in this photo.)
There’s another building, again showing the more “legitimate” aspects of HDR photography in my mind. (Besides the ghostly half-man.) This would ordinarily be a lighting disaster. The building was receiving what was almost direct sunlight, while a dark shadow existed. And the sky was somewhere in between. In this case, I think the blended exposures work perfectly. The same goes true for this shot:
I didn’t know how it’d turn out at first… It looks like a simple shot, but it wasn’t! My typical method is to set my camera up for auto-bracketing, taking three shots in a row, one properly exposed, one too dark, and one too bright, thus increasing the odds that there’s a good shot in there somewhere. Often the main one looks good, but I know that some of the details from the others will boost it when converting to HDR. In this case, though, none of the three worked. If the bottles looked good, the wooden headstones were washed out. If the wall looked good, the grass was far too bright. I had a nicely-bracketed set of three bad photos. Fortunately, Photomatix worked it magic and produced a good shot.
We eventually tired of Clark’s and went exploring the area. Lost River was nearby… I pondered what lens to take, since it wasn’t practical to carry all of them, and made the right choice to bring my wide-angle 18-50mm lens. And here I realized that shooting for HDR, much like switching to using two computer monitors, starting to carry a cell phone, or buying high-thread-count sheets, is a habit that rapidly becomes very hard to break. I’d take a “normal” shot, but realize that parts were under- or over-exposed, so I’d retake the shot as a bracketed set of three, and spent some time in the car ride back home on my laptop merging them.
This is at the Lost River section; hardly the best shot, but a quick-and-dirty example of an ‘acceptable’ use of HDR. The trees are well-lit, and so is the sky. Difficult to pull off with one exposure, but a piece of cake with three and HDR!
As we went along, I noticed various parts of running water. (I thought this was a lost river… Pretty easy to find.) I’d never actually taken the stereotypical long-exposure moving-water shot, so this served as a good opportunity. I set the camera to ISO 100 (pretty insensitive to light, very low noise, but meaning slower shutter speeds), and stopped the lens down to f/22, which gave me about a one-second shutter speed. I set the camera down on a railing and pressed the shutter. Viola!
But I was curious… How would my newfound obsession with HDR play into this? Could you “bracket” that type of shot, and merge them with any success?
I actually didn’t expect this to work, but it ended up being one of my favorite shots from the day. The shots were something like 1/3 second, 1 second, and 2 seconds, so I expected that the water / person would have moved too much. As luck would have it, they didn’t, and the result was that shot. (A nice side-effect is that I rarely remember to stop the lens down for landscapes, but I necessarily did here… At f/22, everything, in theory, is in focus. Although if you look closely, you’ll notice that some of the photo is kind of soft where things got matched up slightly off-kilter.
We then went to the Indian Head Resort, where, in a welcome break from $15 admission tickets, we paid 50 cents to climb the tower. (In hindsight, they should have paid me to climb that thing!)
Doesn’t it look big and scary? Nevermind that I used a wide-angle lens feet away from the base to distort the perspective, nor that I made it an HDR exposure to boost the ominous dark clouds that really weren’t that ominous or dark.
That’s the Indian Head. For those easily confused like me, the Indian Head, and the Old Man of the Mountain are two separate things. I initially remarked, “It kind of still looks like a face,” before realizing that it was the Old Man that came tumbling down, not the Indian Head.
This was yet another one of those shots that was pretty tricky. I pulled out a polarizing filter for this one to try to boost contrast and get the sky to not look so gloomy; you wouldn’t know from the picture, but it helped. You also wouldn’t know from the picture, but this, too, is an HDR shot.
Facing the other way, I decided to take a series of shots holding the camera vertical, intended to be stitched together into a panorama. Since I have under 500MB free on my hard drive (?!) and since I couldn’t find PanoTools or any of its ilk, I ended up trying Windows Live Photo Gallery, which I installed at Mr. T’s suggestion but never got around to using much. (I also brought the image into Photoshop, where I cropped it and tweaked it.)
I’d like to give it good reviews, as it was very easy and quite intuitive. The problem is that I’m fairly certain this isn’t actually how things looked. The pond looks right, but I’m fairly certain that there were more ‘humps’ off to the left. I’m really not sure what happened, but the end result looks good, so I’m happy.