One thing I’ve found with computers is that most people only learn the basics of many of the tools they use daily. Articles like My top 8 time-saving Firefox shortcuts are great ways to quickly pick up on the little things I never knew about (Ctrl+K to jump to the search bar, and Ctrl+L to jump to the URL bar!).

Today I found myself with a badly-formatted CSV file. Many of the last columns had newlines in them, which isn’t something a CSV should have. I fired up vim, and after a couple minutes, found out the regular expression to find any line that doesn’t end with a quotation mark: /[^”]$/ will do that. (Aside: regular expressions are giant pains, but I don’t know of any easier way to do what they do. I could write code to iterate over every line, but then I’ve got 15 lines of code instead of five obscure characters.)

Pressing “n” jumps to the next pattern matching that, and “J” pulls up the previous line to the current line. “n” again to the next match, and then “.” (easier than Shift+j for an uppercase) repeats the J. I soon realized that there were more instances than I thought, though. The ideal way would be have a regular expression to do the replacement, too, but I couldn’t find an easy way, so I did the next best thing and defined a macro. “qr” defined a macro named “r”, and pressing “n” and then “J”, the two steps from above, recorded them as a macro. “q” again stopped recording, and then “@r” ran the macro. Of course, “@r” (@@ does the same thing) wasn’t much easier to type than “n.n.n.n.n.n.n.” over and over.

So I ran “100@r” to run the macro 100 times. I realized that there were way more cases than I thought, and “10000@r” finished it off.

I was left noticing a few things. One is that many of these things were really obscure. To a UNIX power-user, it’s perhaps cake, but to a more average user, it’s black magic. I decided a while ago to make an effort to really learn vi. I’m only a small fraction of the way to mastery, but I’ve found that the time I invested in learning the lesser-known features has been well worth it.

Another thing I wonder about is how I could have done this more effectively. It seems like there ought to be a way to run a vim command for every spot that a regular expression matches. And putting a huge number in front of a macro to run it as many times as needed seems like a hack. At the same time, I know that using a heavyweight text editor was a no-go. This was an enormous file and text editors were dying under the load of trying to deal with the whole document. vim doesn’t mind a gigantic file.

I’ve learned some handy shell tricks and more about MySQL in the past year, too, and both have gotten me far. It sometimes doesn’t seem worth the time, but every time I’ve made an effort to really master all the nuances of a tool, I’ve recouped my time investment many times over.