![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
terrikoBack in one of my early, unpaid co-op jobs, I discovered that my otherwise reasonably experienced boss hadn't ever used tab completion, and it got me thinking a lot about how I learned a lot of command line habits through a combination of word of mouth and a personal conviction that the computer should be able to do anything I found repetitive (alas, I have not taught it to load the dishwasher). But the real take-home message is that there's a lot of little linux tricks that aren't really obvious to everyone. So in that spirit, here's an incredibly tiny script I wrote today that might be useful to someone else:
Moving files found with grep
I had a bunch of output files from my experiments, and I wanted to know at a glance which ones had failed, and then move those files to a subdirectory, leaving me with a smaller list of successes to evaluate in more detail.
Here's the script as a one-liner, the way I'd enter it:
And here's some explanation:
My particular experiment prints a line "No repair found" when the run fails, so that's what I'm searching for in the output files it generates (repair.debug.*). The -l makes grep print just the filenames so I don't have to do any special work to parse them from the output. (You can also use the longer but easier-to-read --files-with-matches. I'm guessing -l was intended as "l for list" but I don't know.)
When I was googling for the -l flag, I did find some people with fancy xarg stuff you could do here, but seriously, if all you need is the filename save yourself some hassle. If your filenames have spaces in them, you may find it useful to do that and some fanciness with -z to change the delimiters to be \0s, but I didn't need to do that.
This is my favourite little bash for loop with the functional bits cut out. It iterates over whatever you gave it in ` ... ` putting each item in $a as it goes through. In this case, each $a is one of the found filenames. You can do away with the backticks all together if you just want a list of filenames that you could get from ls, though. If I'd wanted to move all my repair.debug.* output files, I could have done
Another useful command to put in there other than a grep is `seq 10` which will give you a standard counted for loop that goes up to 10. Very useful when I want my computer run an experiment 10 times while I go to lunch!
I almost always run a version of the loop with *just* "echo $a" in the middle before I make one that does anything, just as a sanity check to make sure I got the expression right and I am actually doing stuff to the right files. I usually leave it in the final version so I can scan the output easily and see what was done. Sometimes I actually output the whole command as an echo for debug purposes
The easy part: moving each file that matched into my notfound/ directory.
And... there you have it! A quick way to move a set of files out of your way and a little bit about how to automate other repetitive tasks on the command line. Probably obvious to many, but who knows, maybe this is exactly the script that someone else needs.
Moving files found with grep
I had a bunch of output files from my experiments, and I wanted to know at a glance which ones had failed, and then move those files to a subdirectory, leaving me with a smaller list of successes to evaluate in more detail.
Here's the script as a one-liner, the way I'd enter it:
for a in `grep -l -z "No repair found" repair.debug.*` ; do echo $a; mv $a notfound/; doneAnd here's some explanation:
grep -l "No repair found" repair.debug.*My particular experiment prints a line "No repair found" when the run fails, so that's what I'm searching for in the output files it generates (repair.debug.*). The -l makes grep print just the filenames so I don't have to do any special work to parse them from the output. (You can also use the longer but easier-to-read --files-with-matches. I'm guessing -l was intended as "l for list" but I don't know.)
When I was googling for the -l flag, I did find some people with fancy xarg stuff you could do here, but seriously, if all you need is the filename save yourself some hassle. If your filenames have spaces in them, you may find it useful to do that and some fanciness with -z to change the delimiters to be \0s, but I didn't need to do that.
for a in ` ... `; do ... ; doneThis is my favourite little bash for loop with the functional bits cut out. It iterates over whatever you gave it in ` ... ` putting each item in $a as it goes through. In this case, each $a is one of the found filenames. You can do away with the backticks all together if you just want a list of filenames that you could get from ls, though. If I'd wanted to move all my repair.debug.* output files, I could have done
for a in repair.debug.*; do mv $a output/; done -- no backticks! I do this all the time for moving files out of my way before I start a new experiment, using directories with the date to keep track of what ran when.Another useful command to put in there other than a grep is `seq 10` which will give you a standard counted for loop that goes up to 10. Very useful when I want my computer run an experiment 10 times while I go to lunch!
echo $aI almost always run a version of the loop with *just* "echo $a" in the middle before I make one that does anything, just as a sanity check to make sure I got the expression right and I am actually doing stuff to the right files. I usually leave it in the final version so I can scan the output easily and see what was done. Sometimes I actually output the whole command as an echo for debug purposes
mv $a notfound/The easy part: moving each file that matched into my notfound/ directory.
And... there you have it! A quick way to move a set of files out of your way and a little bit about how to automate other repetitive tasks on the command line. Probably obvious to many, but who knows, maybe this is exactly the script that someone else needs.
filenames
Date: October 17th, 2012 08:08 am (UTC)On the other hand, it's *annoyingly* hard to do the above in a way that is safe and works even with arbitrary, possibly maliciously crafted filenames.
Re: filenames
Date: October 17th, 2012 05:13 pm (UTC)Moving
Date: October 29th, 2012 04:04 pm (UTC)As in, "Those files I found with grep are so moving". Or "I need to affect those files I found with grep emotionally."
Admit it: it works well with "the joy of for loops".