Shell one liner to prepend to a file

This is probably a complex solution.

I am looking for a simple operator like ">>", but for prepending.

I am afraid it does not exist. I'll have to do something like

mv myfile tmp
cat myheader tmp > myfile

Anything smarter?

This still uses a temp file, but at least it is on one line:

echo "text" | cat - yourfile > /tmp/out && mv /tmp/out yourfile

Credit: BASH: Prepend A Text / Lines To a File

echo '0a
your text here
.
w' | ed some_file

ed is the Standard Editor! http://www.gnu.org/fun/jokes/ed.msg.html

The hack below was a quick off-the-cuff answer which worked and received lots of upvotes. Then, as the question became more popular and more time passed, people started reporting that it sorta worked but weird things could happen, or it just didn't work at all. Such fun.

I recommend the 'sponge' solution posted by user222 as Sponge is part of 'moreutils' and probably on your system by default. (echo 'foo' && cat yourfile) | sponge yourfile

The solution below exploits the exact implementation of file descriptors on your system and, because implementation varies significantly between nixes, it's success is entirely system dependent, definitively non-portable, and should not be relied upon for anything even vaguely important. Sponge uses the /tmp filesystem but condenses the task to a single command.

Now, with all that out of the way the original answer was:

---

Creating another file descriptor for the file (exec 3<> yourfile) thence writing to that (>&3) seems to overcome the read/write on same file dilemma. Works for me on 600K files with awk. However trying the same trick using 'cat' fails.

Passing the prependage as a variable to awk (-v TEXT="$text") overcomes the literal quotes problem which prevents doing this trick with 'sed'.

#!/bin/bash
text="Hello world
What's up?"

exec 3<> yourfile && awk -v TEXT="$text" 'BEGIN {print TEXT}{print}' yourfile >&3

John Mee: your method is not guaranteed to work, and will probably fail if you prepend more than 4096 byte of stuff (at least that's what happens with gnu awk, but I suppose other implementations will have similar constraints). Not only will it fail in that case, but it will enter an endless loop where it will read its own output, thereby making the file grow until all the available space is filled.

Try it for yourself:

exec 3<>myfile && awk 'BEGIN{for(i=1;i<=1100;i++)print i}{print}' myfile >&3

(warning: kill it after a while or it will fill the filesystem)

Moreover, it's very dangerous to edit files that way, and it's very bad advice, as if something happens while the file is being edited (crash, disk full) you're almost guaranteed to be left with the file in an inconsistent state.

Not possible without a temp file, but here goes a oneliner

{ echo foo; cat oldfile; } > newfile && mv newfile oldfile

You can use other tools such as ed or perl to do it without temp files.

If you need this on computers you control, install the package "moreutils" and use "sponge". Then you can do:

cat header myfile | sponge myfile

It may be worth noting that it often is a [good idea][1] to safely generate the temporary file using a utility like [mktemp][2], at least if the script will ever be executed with root privileges. You could for example do the following (again in bash):

(tmpfile=`mktemp` && { echo "prepended text" | cat - yourfile > $tmpfile && mv $tmpfile yourfile; } )

[1]: http://www.linuxsecurity.com/content/view/115462/81/ "Safely Creating Temporary Files in Shell Scripts, by Stefan Nordhausen at LinuxSecurity" [2]: http://www.mktemp.org/ "mktemp homepage"

Using a bash heredoc you can avoid the need for a tmp file:

cat <<-EOF > myfile
  $(echo this is prepended)
  $(cat myfile)
EOF

This works because $(cat myfile) is evaluated when the bash script is evaluated, before the cat with redirect is executed.

assuming that the file you want to edit is my.txt

$cat my.txt    
this is the regular file

And the file you want to prepend is header

$ cat header
this is the header
 

Be sure to have a final blank line in the header file.
Now you can prepend it with

$cat header <(cat my.txt) > my.txt

You end up with

$ cat my.txt
this is the header
this is the regular file

As far as I know this only works in 'bash'.

When you start trying to do things that become difficult in shell-script, I would strongly suggest looking into rewriting the script in a "proper" scripting language (Python/Perl/Ruby/etc)

As for prepending a line to a file, it's not possible to do this via piping, as when you do anything like cat blah.txt | grep something > blah.txt, it inadvertently blanks the file. There is a small utility command called sponge you can install (you do cat blah.txt | grep something | sponge blah.txt and it buffers the contents of the file, then writes it to the file). It is similar to a temp file but you dont have to do that explicitly. but I would say that's a "worse" requirement than, say, Perl.

There may be a way to do it via awk, or similar, but if you have to use shell-script, I think a temp file is by far the easiest(/only?) way..

Like Daniel Velkov suggests, use tee.
To me, that's simple smart solution:

{ echo foo; cat bar; } | tee bar > /dev/null

EDIT: This is broken. See https://stackoverflow.com/questions/25334938/weird-behavior-when-prepending-to-a-file-with-cat-and-tee

The workaround to the overwrite problem is using tee:

cat header main | tee main > /dev/null

sed -i -e "1s/^/new first line\n/" old_file.txt

The one which I use. This one allows you to specify order, extra chars, etc in the way you like it:

echo -e "TEXTFIRSt\n$(< header)\n$(< my.txt)" > my.txt

P.S: only it's not working if files contains text with backslash, cause it gets interpreted as escape characters

Mostly for fun/shell golf, but

ex -c '0r myheader|x' myfile

will do the trick, and there are no pipelines or redirections. Of course, vi/ex isn't really for noninteractive use, so vi will flash up briefly.

With $( command ) you can write the output of a command into a variable. So I did it in three commands in one line and no temp file.

originalContent=$(cat targetfile) && echo "text to prepend" > targetfile && echo "$originalContent" >> targetfile

A variant on cb0's solution for "no temp file" to prepend fixed text:

echo "text to prepend" | cat - file_to_be_modified | ( cat > file_to_be_modified ) 

Again this relies on sub-shell execution - the (..) - to avoid the cat refusing to have the same file for input and output.

Note: Liked this solution. However, in my Mac the original file is lost (thought it shouldn't but it does). That could be fixed by writing your solution as: echo "text to prepend" | cat - file_to_be_modified | cat > tmp_file; mv tmp_file file_to_be_modified

WARNING: this needs a bit more work to meet the OP's needs.

There should be a way to make the sed approach by @shixilun work despite his misgivings. There must be a bash command to escape whitespace when reading a file into a sed substitute string (e.g. replace newline characters with '\n'. Shell commands vis and cat can deal with nonprintable characters, but not whitespace, so this won't solve the OP's problem:

sed -i -e "1s/^/$(cat file_with_header.txt)/" file_to_be_prepended.txt

fails due to the raw newlines in the substitute script, which need to be prepended with a line continuation character () and perhaps followed by an &, to keep the shell and sed happy, like this SO answer

sed has a size limit of 40K for non-global search-replace commands (no trailing /g after the pattern) so would likely avoid the scary buffer overrun problems of awk that anonymous warned of.

Why not simply use the ed command (as already suggested by fluffle here)?

ed reads the whole file into memory and automatically performs an in-place file edit!

So, if your file is not that huge ...

# cf. "Editing files with the ed text editor from scripts.",
# http://wiki.bash-hackers.org/doku.php?id=howto:edit-ed

prepend() {
   printf '%s\n' H 1i "${1}" . wq | ed -s "${2}"
}

echo 'Hello, world!' > myfile
prepend 'line to prepend' myfile

Yet another workaround would be using open file handles as suggested by Jürgen Hötzel in https://stackoverflow.com/questions/2585438/redirect-output-from-sed-s-c-d-myfile-to-myfile

echo cat > manipulate.txt
exec 3<manipulate.txt
# Prevent open file from being truncated:
rm manipulate.txt
sed 's/cat/dog/' <&3 > manipulate.txt

All this could be put on a single line, of course.

Here's what I discovered:

echo -e "header \n$(cat file)" >file

sed -i -e '1rmyheader' -e '1{h;d}' -e '2{x;G}' myfile

You can use perl command line:

perl -i -0777 -pe 's/^/my_header/' tmp

Where -i will create an inline replacement of the file and -0777 will slurp the whole file and make ^ match only the beginning. -pe will print all the lines

Or if my_header is a file:

perl -i -0777 -pe 's/^/`cat my_header`/e' tmp

Where the /e will allow an eval of code in the substitution.

If you're scripting in BASH, actually, you can just issue:

cat - yourfile <<<"text" > /tmp/out && mv /tmp/out yourfile

That's actually in the http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/x13320.html" target="new">Complex Example you yourself posted in your own question.

If you have a large file (few hundred kilobytes in my case) and access to python, this is much quicker than cat pipe solutions:

python -c 'f = "filename"; t = open(f).read(); open(f, "w").write("text to prepend " + t)'

A solution with printf:

new_line='the line you want to add'
target_file='/file you/want to/write to'

printf "%s\n$(cat ${target_file})" "${new_line}" > "${target_file}"

You could also do:

printf "${new_line}\n$(cat ${target_file})" > "${target_file}"

But in that case you have to be sure there aren’t any % anywhere, including the contents of target file, as that can be interpreted and screw up your results.

current=`cat my_file` && echo 'my_string' > my_file && echo $current >> my_file

where "my_file" is the file to prepend "my_string" to.

I'm liking @fluffle's ed approach the best. After all, any tool's command line switches vs scripted editor commands are essentially the same thing here; not seeing a scripted editor solution "cleanliness" being any lesser or whatnot.

Here's my one-liner appended to .git/hooks/prepare-commit-msg to prepend an in-repo .gitmessage file to commit messages:

echo -e "1r $PWD/.gitmessage\n.\nw" | ed -s "$1"

Example .gitmessage:

# Commit message formatting samples:
#       runlevels: boot +consolekit -zfs-fuse
#

I'm doing 1r instead of 0r, because that will leave the empty ready-to-write line on top of the file from the original template. Don't put an empty line on top of your .gitmessage then, you will end up with two empty lines. -s suppresses diagnostic info output of ed.

In connection with going through this, I discovered that for vim-buffs it is also good to have:

[core]
        editor = vim -c ':normal gg'

variables, ftw?

NEWFILE=$(echo deb http://mirror.csesoc.unsw.edu.au/ubuntu/ $(lsb_release -cs) main universe restricted multiverse && cat /etc/apt/sources.list)
echo "$NEWFILE" | sudo tee /etc/apt/sources.list

I think this is the cleanest variation of ed:

cat myheader | { echo '0a'; cat ; echo -e ".\nw";} | ed myfile

as a function:

function prepend() { { echo '0a'; cat ; echo -e ".\nw";} | ed $1; }

cat myheader | prepend myfile

IMHO there is no shell solution (and will never be one) that would work consistently and reliably whatever the sizes of the two files myheader and myfile. The reason is that if you want to do that without recurring to a temporary file (and without letting the shell recur silently to a temporary file, e.g. through constructs like exec 3<>myfile, piping to tee, etc.

The "real" solution you are looking for needs to fiddle with the filesystem, and so it's not available in userspace and would be platform-dependent: you're asking to modify the filesystem pointer in use by myfile to the current value of the filesystem pointer for myheader and replace in the filesystem the EOF of myheader with a chained link to the current filesystem address pointed by myfile. This is not trivial and obviously can not be done by a non-superuser, and probably not by the superuser either... Play with inodes, etc.

You can more or less fake this using loop devices, though. See for instance this SO thread.

Quick and dirty, buffer everything in memory with python:

$ echo two > file
$ echo one | python -c "import sys; f=open(sys.argv[1]).read(); open(sys.argv[1],'w').write(sys.stdin.read()+f)" file
$ cat file
one
two
$ # or creating a shortcut...
$ alias prepend='python -c "import sys; f=open(sys.argv[1]).read(); open(sys.argv[1],\"w\").write(sys.stdin.read()+f)"'
$ echo zero | prepend file
$ cat file
zero
one
two

for dash / ash:

echo "hello\n$(cat myfile)" > myfile

example:

$ echo "line" > myfile
$ cat myfile
line
$ echo "line1\n$(cat myfile)" > myfile
$ cat myfile
line1
line

The simplest solution I found is:

cat myHeader myFile | tee myFile

or

echo "<line to add>" | cat - myFile | tee myFile

Notes:

• Use echo -n if you want to append just a piece of text (not a full line).
• Add &> /dev/null to the end if you don't want to see the output (the generated file).
• This can be used to append a shebang to the file. Example:

  # make it executable (use u+x to allow only current user)
  chmod +x cropImage.ts
  # append the shebang
  echo '#''!'/usr/bin/env ts-node | cat - cropImage.ts | tee cropImage.ts &> /dev/null
  # execute it
  ./cropImage.ts myImage.png
  

Bah! No one cared to mention about tac.

endor@grid ~ $ tac --help
Usage: tac [OPTION]... [FILE]...
Write each FILE to standard output, last line first.
With no FILE, or when FILE is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
  -b, --before             attach the separator before instead of after
  -r, --regex              interpret the separator as a regular expression
  -s, --separator=STRING   use STRING as the separator instead of newline
      --help     display this help and exit
      --version  output version information and exit

Report tac bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report tac translation bugs to <http://translationproject.org/team/>