[NCLUG] bash question

[Bob Proulx]

Warren Turkal wrote:
> Can anyone explain why the following doesn't result in a newline (0x0a)
> being piped to od?

It is in the definition of the shell command language as defined by
POSIX.  Because bash tries to be POSIX compatible it complies.

> wt at pyrus:~/test/ctest$ echo -n "$(echo -ne "\n")" | od -t x1 -t a
> 0000000

First, the use of 'echo' is one of the more troubled commands for
portability.  Some systems interpret backslash-escaped characters in
its argument strings by default and others do not.  Bash's echo is
modeled on the 9th edition Research Unix version of echo and does not
and the -e option is required to interpret.  But System V machines
such as HP-UX for example always interpret backslash-escaped
characters by default and the -e is them printed.  It is better to
avoid echo when using escape sequences.  Better to use printf in that
case.

That is effectively the same as the following.

  $ echo | wc -c
  1
  $ printf "\n" | wc -c
  1

Using printf we can see that escape characters are interpreted.
Therefore we can generate newlines.  And your interesting case with
the embedded newline in the string is printed with printf.  (My PS2 is
'> ' and so this is prompted for command continuation.  Don't
cut-n-paste it if testing this behavior.)

  $ printf "
  > " | wc -c
  1

But as you pointed out when these are embedded in $(...) command
substitution the newline disappears and is not printed.

  $ printf "$(printf "\n")" | wc -c
  0

No characters are output.  Why?  Where did it go?  The man page
documents the behavior.

   man bash

   Command Substitution
       ...
       Bash performs the expansion by executing command and replacing
       the command substitution with the standard output of the
       command, with any trailing newlines deleted.  Embedded newlines
       are not deleted, but they may be removed during word splitting.

And this follows POSIX.

  http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_03

  The shell shall expand the command substitution by executing command
  in a subshell environment (see Shell Execution Environment) and
  replacing the command substitution (the text of command plus the
  enclosing "$()" or backquotes) with the standard output of the
  command, removing sequences of one or more <newline>s at the end of
  the substitution.

So we now see that it is documented behavior.  But why does POSIX
require it?  It turns out that POSIX was implemented upon the original
ksh behavior and therefore does so because ksh introduced this syntax.
POSIX is documenting the behavior of the existing implementation.  In
the original implementation of the syntax ksh removed trailing
newlines from the string.  All else followed that original
implementation.

Bob