Context

While working on a client's app's CI pipeline, I needed to use an API key. It was already used by some of the pipeline's other stages, and thus available to me as an environment variable. However, it was encoded in base64, when the tool I needed to use required it to be plaintext in a JSON file.

Set up

Let's start with a simple string from our favorite text generator: Samuel L Ipsum. Why? Because like Apple's Appstore Connect API keys, it contains line breaks.

My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?

Normally, both your asses would be dead as fucking fried chicken, but you happen to pull this shit while I'm in a transitional period so I don't wanna kill you, I wanna help you. But I can't give you this case, it don't belong to me. Besides, I've already been through too much shit this morning over this case to hand it over to your dumb ass.

Your bones don't break, mine do. That's clear. Your cells react to bacteria and viruses differently than mine. You don't get sick, I do. That's also clear. But for some reason, you and I react the exact same way to water. We swallow it too fast, we choke. We get some in our lungs, we drown. However unreal it may seem, we are connected, you and I. We're on the same curve, just on opposite ends.

Look, just because I don't be givin' no man a foot massage don't make it right for Marsellus to throw Antwone into a glass motherfuckin' house, fuckin' up the way the nigger talks. Motherfucker do that shit to me, he better paralyze my ass, 'cause I'll kill the motherfucker, know what I'm sayin'?

My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?

OUR_VAR="My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?

Normally, both your asses would be dead as fucking fried chicken, but you happen to pull this shit while I'm in a transitional period so I don't wanna kill you, I wanna help you. But I can't give you this case, it don't belong to me. Besides, I've already been through too much shit this morning over this case to hand it over to your dumb ass.

Your bones don't break, mine do. That's clear. Your cells react to bacteria and viruses differently than mine. You don't get sick, I do. That's also clear. But for some reason, you and I react the exact same way to water. We swallow it too fast, we choke. We get some in our lungs, we drown. However unreal it may seem, we are connected, you and I. We're on the same curve, just on opposite ends.

Look, just because I don't be givin' no man a foot massage don't make it right for Marsellus to throw Antwone into a glass motherfuckin' house, fuckin' up the way the nigger talks. Motherfucker do that shit to me, he better paralyze my ass, 'cause I'll kill the motherfucker, know what I'm sayin'?

My money's in that office, right? If she start giving me some bullshit about it ain't there, and we got to go someplace else and get it, I'm gonna shoot you in the head then and there. Then I'm gonna shoot that bitch in the kneecaps, find out where my goddamn money is. She gonna tell me too. Hey, look at me when I'm talking to you, motherfucker. You listen: we go in there, and that nigga Winston or anybody else is in there, you the first motherfucker to get shot. You understand?"

echo "$OUR_VAR"

Encoding it in base64 and storing it in a variable is easy enough:

BASE64_API_KEY=$(echo $OUR_VAR | base64)

Decoding a base64 string

Now, let's decode it and store the decoded string, shall we?

DECODED_API_KEY=$(echo $BASE64_API_KEY | base64 --decode)
echo $DECODED_API_KEY

Still quite simple.

Escaping newlines & line breaks

Now, since we want to store that key in a .json file, we need to escape our newlines. Easy, right? Not on macOS. Thankfully, there's a known solution (shameless plug alert): Escape a newline in bash on macOS.

NEWLINES_ESCAPED_API_KEY=$(echo "$DECODED_API_KEY" | awk -v ORS='\\n' '1')
printf "\n[NEWLINE_ESCAPE_API_KEY]\n%s\n--------\n" "$NEWLINES_ESCAPED_API_KEY"

Creating a new JSON file with the decoded content

And now, instead of playing around with echo and manually escaping newlines and double quotes, lets simply use cat with End Of File (EOF):

cat <<EOF > "file.json"
{
  "key": "$NEWLINES_ESCAPED_API_KEY"
}
EOF

Full example

Here's what a full script would look like:

JSON_FILE_PATH="file.json"

ACCOUNT_ID="3dfae1f9-a8a7-4847-a8c0-306b1d2a1016"

BASE64_API_KEY="TXkgbW9uZXkncyBpbiB0aGF0IG9mZmljZSwgcmlnaHQ/IElmIHNoZSBzdGFydCBnaXZpbmcgbWUgc29tZSBidWxsc2hpdCBhYm91dCBpdCBhaW4ndCB0aGVyZSwgYW5kIHdlIGdvdCB0byBnbyBzb21lcGxhY2UgZWxzZSBhbmQgZ2V0IGl0LCBJJ20gZ29ubmEgc2hvb3QgeW91IGluIHRoZSBoZWFkIHRoZW4gYW5kIHRoZXJlLiBUaGVuIEknbSBnb25uYSBzaG9vdCB0aGF0IGJpdGNoIGluIHRoZSBrbmVlY2FwcywgZmluZCBvdXQgd2hlcmUgbXkgZ29kZGFtbiBtb25leSBpcy4gU2hlIGdvbm5hIHRlbGwgbWUgdG9vLiBIZXksIGxvb2sgYXQgbWUgd2hlbiBJJ20gdGFsa2luZyB0byB5b3UsIG1vdGhlcmZ1Y2tlci4gWW91IGxpc3Rlbjogd2UgZ28gaW4gdGhlcmUsIGFuZCB0aGF0IG5pZ2dhIFdpbnN0b24gb3IgYW55Ym9keSBlbHNlIGlzIGluIHRoZXJlLCB5b3UgdGhlIGZpcnN0IG1vdGhlcmZ1Y2tlciB0byBnZXQgc2hvdC4gWW91IHVuZGVyc3RhbmQ/CgpOb3JtYWxseSwgYm90aCB5b3VyIGFzc2VzIHdvdWxkIGJlIGRlYWQgYXMgZnVja2luZyBmcmllZCBjaGlja2VuLCBidXQgeW91IGhhcHBlbiB0byBwdWxsIHRoaXMgc2hpdCB3aGlsZSBJJ20gaW4gYSB0cmFuc2l0aW9uYWwgcGVyaW9kIHNvIEkgZG9uJ3Qgd2FubmEga2lsbCB5b3UsIEkgd2FubmEgaGVscCB5b3UuIEJ1dCBJIGNhbid0IGdpdmUgeW91IHRoaXMgY2FzZSwgaXQgZG9uJ3QgYmVsb25nIHRvIG1lLiBCZXNpZGVzLCBJJ3ZlIGFscmVhZHkgYmVlbiB0aHJvdWdoIHRvbyBtdWNoIHNoaXQgdGhpcyBtb3JuaW5nIG92ZXIgdGhpcyBjYXNlIHRvIGhhbmQgaXQgb3ZlciB0byB5b3VyIGR1bWIgYXNzLgoKWW91ciBib25lcyBkb24ndCBicmVhaywgbWluZSBkby4gVGhhdCdzIGNsZWFyLiBZb3VyIGNlbGxzIHJlYWN0IHRvIGJhY3RlcmlhIGFuZCB2aXJ1c2VzIGRpZmZlcmVudGx5IHRoYW4gbWluZS4gWW91IGRvbid0IGdldCBzaWNrLCBJIGRvLiBUaGF0J3MgYWxzbyBjbGVhci4gQnV0IGZvciBzb21lIHJlYXNvbiwgeW91IGFuZCBJIHJlYWN0IHRoZSBleGFjdCBzYW1lIHdheSB0byB3YXRlci4gV2Ugc3dhbGxvdyBpdCB0b28gZmFzdCwgd2UgY2hva2UuIFdlIGdldCBzb21lIGluIG91ciBsdW5ncywgd2UgZHJvd24uIEhvd2V2ZXIgdW5yZWFsIGl0IG1heSBzZWVtLCB3ZSBhcmUgY29ubmVjdGVkLCB5b3UgYW5kIEkuIFdlJ3JlIG9uIHRoZSBzYW1lIGN1cnZlLCBqdXN0IG9uIG9wcG9zaXRlIGVuZHMuCgpMb29rLCBqdXN0IGJlY2F1c2UgSSBkb24ndCBiZSBnaXZpbicgbm8gbWFuIGEgZm9vdCBtYXNzYWdlIGRvbid0IG1ha2UgaXQgcmlnaHQgZm9yIE1hcnNlbGx1cyB0byB0aHJvdyBBbnR3b25lIGludG8gYSBnbGFzcyBtb3RoZXJmdWNraW4nIGhvdXNlLCBmdWNraW4nIHVwIHRoZSB3YXkgdGhlIG5pZ2dlciB0YWxrcy4gTW90aGVyZnVja2VyIGRvIHRoYXQgc2hpdCB0byBtZSwgaGUgYmV0dGVyIHBhcmFseXplIG15IGFzcywgJ2NhdXNlIEknbGwga2lsbCB0aGUgbW90aGVyZnVja2VyLCBrbm93IHdoYXQgSSdtIHNheWluJz8KCk15IG1vbmV5J3MgaW4gdGhhdCBvZmZpY2UsIHJpZ2h0PyBJZiBzaGUgc3RhcnQgZ2l2aW5nIG1lIHNvbWUgYnVsbHNoaXQgYWJvdXQgaXQgYWluJ3QgdGhlcmUsIGFuZCB3ZSBnb3QgdG8gZ28gc29tZXBsYWNlIGVsc2UgYW5kIGdldCBpdCwgSSdtIGdvbm5hIHNob290IHlvdSBpbiB0aGUgaGVhZCB0aGVuIGFuZCB0aGVyZS4gVGhlbiBJJ20gZ29ubmEgc2hvb3QgdGhhdCBiaXRjaCBpbiB0aGUga25lZWNhcHMsIGZpbmQgb3V0IHdoZXJlIG15IGdvZGRhbW4gbW9uZXkgaXMuIFNoZSBnb25uYSB0ZWxsIG1lIHRvby4gSGV5LCBsb29rIGF0IG1lIHdoZW4gSSdtIHRhbGtpbmcgdG8geW91LCBtb3RoZXJmdWNrZXIuIFlvdSBsaXN0ZW46IHdlIGdvIGluIHRoZXJlLCBhbmQgdGhhdCBuaWdnYSBXaW5zdG9uIG9yIGFueWJvZHkgZWxzZSBpcyBpbiB0aGVyZSwgeW91IHRoZSBmaXJzdCBtb3RoZXJmdWNrZXIgdG8gZ2V0IHNob3QuIFlvdSB1bmRlcnN0YW5kPw=="
"
printf "\n[ACCOUNT ID]\n%s\n--------\n" "$ACCOUNT_ID"
printf "\n[BASE64 API KEY]\n%s\n--------\n" "$BASE64_API_KEY

DECODED_API_KEY=$(echo $BASE64_API_KEY | base64 --decode)
NEWLINES_ESCAPED_API_KEY=$(echo "$DECODED_API_KEY" | awk -v ORS='\\n' '1')

printf "\n[DECODED_API_KEY]\n%s\n--------\n" "$DECODED_API_KEY"
printf "\n[NEWLINE_ESCAPE_API_KEY]\n%s\n--------\n" "$NEWLINES_ESCAPED_API_KEY"

cat <<EOF > $JSON_FILE_PATH
{
  "accountId": "$ACCOUNT_ID",
  "key": "$NEWLINES_ESCAPED_API_KEY"
}
EOF

printf "\n[JSON FILE]\n%s\n--------\n" "$(cat $JSON_FILE_PATH)"

Side notes

At some points, while working on my script, when using echo to print variables assigned using "command substitution" (ie DECODED_API_KEY=$(echo $BASE64_API_KEY | base64  --decode), the line breaks where simply gone and replaced by spaces. , the line breaks would simply be gone

That's because to quote cuonglm's answer on Stack Exchange:

The newlines were lost, because the shell had performed field splitting after command substitution.

We can solve that by adding quotes our variable's name when printing it, like so:

echo "$DECODED_API_KEY"

Epilogue

As always, should you encounter any issues or have any suggestions for improvements, reach out to me on Twitter! And have a great day! Here's some nice picture of Edinburgh to brighten your day, fresh off Unsplash: