sed substitute variable contains newline (preserve it)

sed substitute variable contains newline (preserve it)

Thanks in advance for your help. Solution has to be via SED (but can be pre-processed anyway) as I am using the -i option (edit in place).

Sample text received via LYNX (came from lynx'ing some website - see below):
toast the lemonade
blend with the lemonade
add one tablespoon of the lemonade
grill the spring onions
add the lemonade
add the raisins to the saucepan
rinse the horseradish sauce

The above has been declared as variable $INPUT i.e.
INPUT=lynx --dump 'http://somesite.net/recipes' | python -m json.tool | awk '/steps/,/]/' | egrep -v "steps|]" | sed 's/"//g' |sed 's/,//g' | sed 's/^ *//g' | sed '$d'

At this point, $INPUT is ready for substitution with SED as follows:
sed -i "0,/OLDINPUT/s//$INPUT/" /home/test_file

Of course, sed complains about an unterminated s command - herein lies the problem

The current patch I am using is to echo $INPUT prior to sending it to sed but then the newlines are not preserved. Echo strips newlines...which is the problem.

The correct output should maintain it's new lines. How can SED be instructed to preserve the new lines?

Answer by Wintermute for sed substitute variable contains newline (preserve it)

The hacky direct answer is to replace all newlines with \n, which you can do by adding

| sed ':a $!{N; ba}; s/\n/\\n/g'  

to the long command above. A better answer, because substituting shell variables into code is always a bad idea and with sed you wouldn't have a choice, is to use awk instead:

awk -i inplace -v input="$INPUT" 'NR == 1, /OLDINPUT/ { sub(/OLDINPUT/, input) } 1' /home/test_file  

This requires GNU awk 4.1.0 or later for the -i inplace.

Answer by Jotne for sed substitute variable contains newline (preserve it)

To clean up your code some.

This:

lynx --dump 'http://somesite.net/recipes' | python -m json.tool | awk '/steps/,/]/' | egrep -v "steps|]" | sed 's/"//g' |sed 's/,//g' | sed 's/^ *//g' | sed '$d'  

Can be replaced with this:

lynx --dump 'http://somesite.net/recipes' | python -m json.tool | awk '/]/ {f=0} f {if (c--) print line} /steps/{f=1} {gsub(/[",]|^ */,"");line=$0}'  

It may be shorten more, but I do not now what this does: python -m json.tool

This:

awk '/]/ {f=0} f {if (c--) print line} /steps/{f=1} {gsub(/[",]|^ */,"");line=$0}'  

Does:

1. Print line after pattern steps to line before ] - awk '/steps/,/]/' | egrep -v "steps|]"
2. Removes ", , and all space in front of all lines. - sed 's/"//g' |sed 's/,//g' | sed 's/^ *//g'
3. Then remove last line of this group. - sed '$d'

---

Example:

cat file  my data  steps data   more   do not delet this  hei "you" , more data  extra line  here is end ]  this is good  

awk '/]/ {f=0} f {if (c--) print line} /steps/{f=1} {gsub(/[",]|^ */,"");line=$0}' file  more  do not delet this  hei you  more data  

Answer by glenn jackman for sed substitute variable contains newline (preserve it)

You'll want to use an editor instead of sed's substitution:

$ input="toast the lemonade  blend with the lemonade  add one tablespoon of the lemonade  grill the spring onions  add the lemonade  add the raisins to the saucepan  rinse the horseradish sauce"    $ seq 10 > file    $ ed file <

Answer by tripleee for sed substitute variable contains newline (preserve it)

Assuming your input JSON fragment looks something like this:

{ "other": "random stuff",    "steps": [      "toast the lemonade",      "blend with the lemonade",      "add one tablespoon of the lemonade",      "grill the spring onions",      "add the lemonade",      "add the raisins to the saucepan",      "rinse the horseradish sauce"    ],    "still": "yet more stuff" }  

you can extract just the steps member with

jq -r .steps  

To interpolate that into a sed statement, you'd need to escape any regex metacharacters in the result. A less intimidating and hopefully slightly less hacky solution would be to read static text from standard input:

lynx ... | jq ... |  sed -i -e '/OLDINPUT/{s///; r /dev/stdin' -e '}' /home/test_file  

The struggle to educate practitioners to use structure-aware tools for structured data has reached epic heights and continues unabated. Before you decide to use the quick and dirty approach, at least make sure you understand the dangers (technical and mental).

Answer by Toby Speight for sed substitute variable contains newline (preserve it)

Your shell should be able to substitute \n for the newline:

INPUT="${INPUT//  /\\n}"  

If you don't like the literal linefeed in your parameter expansion, try

nl=$'\n'  INPUT="${INPUT//$nl/\\n}"  

Side note - you probably mean to change the matched lines to your input, not substitute each of them. In which case, you don't want to quote the newlines, after all...

Fatal error: Call to a member function getElementsByTagName() on a non-object in D:\XAMPP INSTALLASTION\xampp\htdocs\endunpratama9i\www-stackoverflow-info-proses.php on line 72