Escape a newline in bash on macOS

While setting up a new step for a client's CI pipeline on a macOS agent, I needed to export an API key containing line breaks to a JSON file. Meaning I had to escape line breaks, in bash, on macOS. Turns out, it's not as easy as it sounds. Who would've guessed?

Escape a newline in bash on macOS
Photo by KOBU Agency / Unsplash

It seems obvious, right? Simply do echo $OUR_VAR | sed 's/\n/\\n/g',  right?

For those of you already convinced it is not that easy, just skip below. For the potential future colleague I need to convince I'm not a hack, let's go ahead and try that with a simple string from our favorite text generator: Samuel L Ipsum.

OUR_VAR="My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?

Normally, both your asses would be dead as fucking fried chicken, but you happen to pull this shit while I'm in a transitional period so I don't wanna kill you, I wanna help you. But I can't give you this case, it don't belong to me. Besides, I've already been through too much shit this morning over this case to hand it over to your dumb ass."

echo $OUR_VAR

Everything good? Now, let's escape our newlines.

echo "$OUR_VAR" | sed 's/\n/\\n/g'
My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?

Normally, both your asses would be dead as fucking fried chicken, but you happen to pull this shit while I'm in a transitional period so I don't wanna kill you, I wanna help you. But I can't give you this case, it don't belong to me. Besides, I've already been through too much shit this morning over this case to hand it over to your dumb ass.

Are you getting the same thing? We should be seeing two \n in there between our paragraphs, right?

I searched around, and mostly found people trying to accomplish the opposite: replacing something with a newline, and finally discovered that, to quote another StackOverflow answer

sed is intended to be used on line-based input. Although it can do what you need.

Despite trying to adapt the suggested solutions, like using tr, all I got weere extra backslashes in my string.

echo "$OUR_VAR" | tr '\n' '\\n'
My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?\Normally, both your asses would be dead as fucking fried chicken, but you happen to pull this shit while I'm in a transitional period so I don't wanna kill you, I wanna help you. But I can't give you this case, it don't belong to me. Besides, I've already been through too much shit this morning over this case to hand it over to your dumb ass.\

But I'll spare you everything else I tried, and skip to the solution. Because we both have better things to do. Like grab a "Proper Hot Chocolate" from one of the best places in Edinburgh: Uplands Roast (their mocha are great too).

How it's actually done

After rephrasing my search queries, and trying something both more and less specific, the answer was finally to be found in Google's second result's second subresult.

The answer? Change the 'output record separator' (ORS in the script) to \\n instead of \n using awk:

echo "$OUR_VAR" | awk -v ORS='\\n' '1'
My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?\n\nNormally, both your asses would be dead as fucking fried chicken, but you happen to pull this shit while I'm in a transitional period so I don't wanna kill you, I wanna help you. But I can't give you this case, it don't belong to me. Besides, I've already been through too much shit this morning over this case to hand it over to your dumb ass.\n

And there they are! Our gloriously escaped newline characters!

A hearty breakfast
Ah, those newlines! Almost as beautiful as some random Unsplash picture! - Photo by Shyam / Unsplash

Epilogue

That's it! Enjoy the rest of your day, and the few hours (if you had the same issue and this somehow popped up in the first results) you've just saved! 😉