Check out your Pipes

While we’re on the topic of the Linux command line, I figured I’d highlight a useful tool that I accidentally discovered a few days ago. I don’t remember what I was searching portage for, but I happened across Pipe Viewer, or pv. From the website:

Pipe Viewer – is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

And using it? You just insert pv anywhere you’re piping data between processes — it simply passes the data from stdin to stdout — and you get a nice little wget style progress bar. For instance, I could insert it in Matt’s previous example, and I’d get something like:

# pv /var/log/messages | grep bumttwagnerfor 
>  | awk '{ print $10}' | sed "s/[/ /g" 
>  | sed "s/]/ /g" | awk '{print $2}'
94.5MB 0:00:03 [34.3MB/s] [=======>            ] 41% ETA 0:00:04

The only thing I’ve noticed is that you really need to be working with a lot of data to make the information it provides useful; most of the time the piping is completed by the time you even get any output. But for situations where you’re dealing with a lot of data (like log files), it’s golden!

2 Comments so far

  1. n1zyy on October 15th, 2007

    I’ve have to give that a try some time.

    It’s probably like the download timer in wget. Sometimes I’ll download things at like 400 Mbps from my server with a 10 Mbps link.

  2. andrew on October 15th, 2007

    It can also very handily demonstrate the Linux kernel’s filesystem caching; on the first run of a file (small enough to fit entirely within memory), you’ll see moderate transfer speeds (appropriate for a hard drive), on the second, it’ll be lightning fast. Really large files tend to push their beginnings out of the cache, so you won’t see the speed increase.

Leave a Reply