terriko: (Pi)
[personal profile] terriko
Back in one of my early, unpaid co-op jobs, I discovered that my otherwise reasonably experienced boss hadn't ever used tab completion, and it got me thinking a lot about how I learned a lot of command line habits through a combination of word of mouth and a personal conviction that the computer should be able to do anything I found repetitive (alas, I have not taught it to load the dishwasher). But the real take-home message is that there's a lot of little linux tricks that aren't really obvious to everyone. So in that spirit, here's an incredibly tiny script I wrote today that might be useful to someone else:

Moving files found with grep

I had a bunch of output files from my experiments, and I wanted to know at a glance which ones had failed, and then move those files to a subdirectory, leaving me with a smaller list of successes to evaluate in more detail.

Here's the script as a one-liner, the way I'd enter it:
for a in `grep -l -z "No repair found" repair.debug.*` ; do echo $a; mv $a notfound/; done

And here's some explanation:

grep -l "No repair found" repair.debug.*

My particular experiment prints a line "No repair found" when the run fails, so that's what I'm searching for in the output files it generates (repair.debug.*). The -l makes grep print just the filenames so I don't have to do any special work to parse them from the output. (You can also use the longer but easier-to-read --files-with-matches. I'm guessing -l was intended as "l for list" but I don't know.)

When I was googling for the -l flag, I did find some people with fancy xarg stuff you could do here, but seriously, if all you need is the filename save yourself some hassle. If your filenames have spaces in them, you may find it useful to do that and some fanciness with -z to change the delimiters to be \0s, but I didn't need to do that.

for a in ` ... `; do ... ; done

This is my favourite little bash for loop with the functional bits cut out. It iterates over whatever you gave it in ` ... ` putting each item in $a as it goes through. In this case, each $a is one of the found filenames. You can do away with the backticks all together if you just want a list of filenames that you could get from ls, though. If I'd wanted to move all my repair.debug.* output files, I could have done for a in repair.debug.*; do mv $a output/; done -- no backticks! I do this all the time for moving files out of my way before I start a new experiment, using directories with the date to keep track of what ran when.

Another useful command to put in there other than a grep is `seq 10` which will give you a standard counted for loop that goes up to 10. Very useful when I want my computer run an experiment 10 times while I go to lunch!

echo $a
I almost always run a version of the loop with *just* "echo $a" in the middle before I make one that does anything, just as a sanity check to make sure I got the expression right and I am actually doing stuff to the right files. I usually leave it in the final version so I can scan the output easily and see what was done. Sometimes I actually output the whole command as an echo for debug purposes

mv $a notfound/

The easy part: moving each file that matched into my notfound/ directory.


And... there you have it! A quick way to move a set of files out of your way and a little bit about how to automate other repetitive tasks on the command line. Probably obvious to many, but who knows, maybe this is exactly the script that someone else needs.

filenames

Date: October 17th, 2012 08:08 am (UTC)
From: (Anonymous)
Bash rocks for such things, especially when you've got control over the filenames yourself so you can be certain that they don't contain nonsense like spaces, tabs, newlines or quote-characters.

On the other hand, it's *annoyingly* hard to do the above in a way that is safe and works even with arbitrary, possibly maliciously crafted filenames.

Moving

Date: October 29th, 2012 04:04 pm (UTC)
unregisteredpseudonymspls: (Default)
From: [personal profile] unregisteredpseudonymspls
The first thing I thought of when reading the title was to translate it mentally to "Emotionally affecting files you found with grep".

As in, "Those files I found with grep are so moving". Or "I need to affect those files I found with grep emotionally."

Admit it: it works well with "the joy of for loops".

Profile

terriko: (Default)
terriko

September 2014

S M T W T F S
 123456
78 910111213
141516 17181920
2122 2324252627
282930    

Most Popular Tags

Page Summary

Style Credit

Expand Cut Tags

No cut tags
Page generated Oct. 1st, 2014 03:57 am
Powered by Dreamwidth Studios