Loop to delete the first line of multiple files using Bash Script

Question

Just a new script and Bash programming in general. I would like to automate the removal of the first line from multiple .data files in a directory. My script looks like this:

#!/bin/bash for f in *.data ; do tail -n +2 $f | echo "processing $f"; done 

I get an echo message, but when I am a cat, the file has not changed anything. Any ideas?

Thank you in advance

Answer 1

You do not change the file itself. Using tail , you simply read the file and print its parts to stdout (terminal), you must redirect this output to a temporary file and then overwrite the original file with a temporary one.

 #!/usr/bin/env bash for f in *.data; do tail -n +2 "$f" > "${f}".tmp && mv "${f}".tmp "$f" echo "Processing $f" done 

Also, it is not clear what you would like to achieve with the echo command. Why are you using a pipe ( | )?

sed will give you an easier way to achieve this. See devnull answer.

Answer 2

I get an echo message, but when I am a cat, the file has not changed anything.

Because just tail ing won't change the file.

You can use sed to modify files in place with the first line excluded. Statement

 sed -i '1d' *.data 

will delete the first line of all .data files.

---

EDIT: BSD sed (on OSX) expects the -i argument, so you can specify the extension to back up old files or edit the files in place, say:

 sed -i '' '1d' *.data 

Answer 3

I would do it like this:

 #!/usr/bin/env bash set -eu for f in *.data; do echo "processing $f" tail -n +2 "$f" | sponge "$f" done 

If you don't have sponge , you can get it in the moreutils package.

The quotes around the file name are important - they will work with file names containing spaces. And the top of env is for people to be able to determine which Bash interpreter they want to use through their PATH if someone doesn't have a default value. set -eu makes a Bash output if an error occurs that is usually more secure.

Answer 4

ed - standard editor:

 shopt -s nullglob for f in *.data; do echo "Processing file \`$f'" ed -s -- "$f" < <( printf '%s\n' "1d" "wq" ) done 

shopt -s nullglob here only because you should always use this when using globes, especially in a script: this will make the globes unchanged if there are no matches; You do not want to run commands with uncontrolled arguments.

Next, we focus on all of your files and use ed with the commands:

• 1 : go to the first line
• d : delete this line
• wq : write and exit

ed options:

• -s : tell ed shut up! we don’t want ed print its garbage on our screen.
• -- : end of options: this will make your script more reliable if the file name starts with hypen: in this case the hyphen confuses ed , trying to treat it as an option, C -- , ed knows that after that there are no more parameters, and it will successfully process any files, even those starting with a hyphen.