Starting a new line after connecting to the command: is there any standard?

Question

Answer How to remove the last CR char with cut I found out that some programs add a new trailing line to the end of the line, while others do not:

Say we have a foobar string and print it with printf so that we don't get an extra new line:

 $ printf "foobar" | od -c 0000000 foobar 0000006 

Or using echo -n :

 $ echo -n "foobar" | od -c 0000000 foobar 0000006 

(The echo default behavior is to return output, followed by a newline, so echo "foobar" returns foobar \n ).

Neither sed nor cat add an extra character:

 $ printf "foobar" | sed 's/./&/g' | od -c 0000000 foobar 0000006 $ printf "foobar" | cat - | od -c 0000000 foobar 0000006 

While both awk and cut do. Also xargs and paste add this new new line:

 $ printf "foobar" | cut -b1- | od -c 0000000 foobar \n 0000007 $ printf "foobar" | awk '1' | od -c 0000000 foobar \n 0000007 $ printf "foobar" | xargs | od -c 0000000 foobar \n 0000007 $ printf "foobar" | paste | od -c 0000000 foobar \n 0000007 

So I was wondering: why is this a different behavior? Does POSIX have anything to offer about this?

Note. I do all this in my Bash 4.3.11, and the rest:

• GNU Awk 4.0.1
• sed (GNU sed) 4.2.2
• cat (GNU coreutils) 8.21
• cut (GNU coreutils) 8.21
• xargs (GNU findutils) 4.4.2
• paste (GNU coreutils) 8.21

Answer 1

So I was wondering: why is this a different behavior? Does POSIX have anything to offer about this?

Some commands (for example, printf ) are a simple interface for calls to the libc library (for example, printf() ) that do not automatically add \n . Most NIX text processing commands have added \n to the end of the last line.

From the POSIXv7 Definition , the text string should have a newline at the end:

3.206 Line

A sequence of zero or more characters is not a <newline> plus a trailing character.

If newline missing, it becomes the following:

3.195 Incomplete line

The sequence of one or more characters is not <newline> at the end of the file.

The general idea is that a text file can be thought of as a list of entries, where each entry ends with \n . In other words, \n not something between the lines - it is part of the line. See For example, fgets() : \n always on and serves to determine whether the text line was read completely or not. If the last line does not contain \n , then you need to do more checks to read the file correctly.

In general, as long as your text files are created in * NIX using * NIX programs / scripts, it is quite normal to expect the last line to finish correctly. But many Java applications, as well as Windows applications, cannot do this correctly or sequentially. Not only do they often forget to add the last \n , they often also incorrectly treat the trailing \n as an extra blank line.