At Spotr, we use Airtable (yes, this is a referral link) as our back-office. It allows the content team to manage all... well, you guessed it, our content, along with user generated content suggestions, and handle (hasn't happened yet; fingers crossed) user content reporting.

But our apps use Firebase as their backend. And I really don't want our users to get the pictures directly from Airtable. Not that I don't trust the Airtable team, but I really don't want to pollute our content team's nice back-office and minds with the x different sizes we have for each image, the custom tailored projections for the apps' views..., and I really don't want to expose Airtable to the outside world. So we need to sync all of the content, including the pictures. But how on Earth does one upload remotefiles to Firebase Storage?

Most answers I've found talk about uploading a local file from a client. No use for us. Some dismiss the matter altogether, and one seemed overly complicated. Rest assured, there is an easy and simple way.

To accomplish this, you'll only need three things: the firebase admin sdk, the request package (along with it's type definitions if you are, like me, using Typescript. And you should), and Node.js streams (hence the picture at the top of this post). Yep, nothing more.

Setup your firebase admin sdk

import admin from "firebase-admin";

admin.initializeApp();

Get the file as a stream from it's url using request

import admin from "firebase-admin";
import request from "request";
import Stream from "stream";

admin.initializeApp();

const getStreamFromUrl = (url: string) => {
	console.log(`Url: ${url}`)
	return request(url);
}

Pipe the stream into Firebase Storage

And return the file's new URL

import admin from "firebase-admin";
import request from "request";
import Stream from "stream";

admin.initializeApp();

const uploadNewFileFromStream = (bucketName: string) => (id: string) => (readStream: Stream) => {
	const fileName = `folder/${id}.jpg`
	const bucket = admin.storage().bucket(bucketName)
	const file = bucket.file(`/${fileName}`);

	const writeStream = file.createWriteStream({
		metadata: {
			contentType: 'image/jpeg'
		}
	})
    
    const pipedStreams = readStream.pipe(writeStream);
	
	const result: Promise<string> = new Promise((resolve, reject) => {
		pipedStreams.on('error', function (err) {
			reject(err);
		});
		pipedStreams.on('finish', function () {	
			const publicUrl = `https://firebasestorage.googleapis.com/v0/b/${bucket.name}/o/${(encodeURI(fileName)).replace(/\//g, "%2F")}`;
			resolve(publicUrl);
    	});
	})
	
	return result
}

const getStreamFromUrl = (url: string) => {
	return request(url);
}

Expose it all using a clean interface and a nice default export

import admin from "firebase-admin";
import request from "request";
import Stream from "stream";


export interface FirebaseStorageGatewayInterface {
	uploadNewFile: (id: string, stream: Stream) => Promise<string>;
	uploadNewFileFromUrl: (id: string, url: string) => Promise<string>;
}

const uploadNewFileFromStream = (bucketName: string) => (id: string) => (readStream: Stream) => {
	const fileName = `folder/${id}.jpg`
	const bucket = admin.storage().bucket(bucketName)
	const file = bucket.file(`/${fileName}`);

	const writeStream = file.createWriteStream({
		metadata: {
			contentType: 'image/jpeg'
		}
	});

    const pipedStreams = readStream.pipe(writeStream);1
	
	const result = Promise((resolve, reject) => {
		pipedStreams.on('error', function (err) {
			reject(err);
		});
		pipedStreams.on('finish', function () {	
			const publicUrl = `https://firebasestorage.googleapis.com/v0/b/${bucket.name}/o/${(encodeURI(fileName)).replace(/\//g, "%2F")}`;
			resolve(publicUrl);
		});
	});

	return result;
}

const getStreamFromUrl = (url: string) => {
	return request(url);
}

export default (bucketName: string) => {
	return {
		uploadNewFile: (id: string, stream: Stream) => uploadNewFileFromStream(bucketName)(id)(stream),
		uploadNewSpotPictureFromUrl: (id: string, url: string) => uploadNewFileFromStream(bucketName)(id)(getStreamFromUrl(url))
	}

TL;DR

Get the file's stream using request. Use that stream to write into Firebase Storage. Return the file's url once you're done writing.

Acknowledgments

Thanks to @aseidma for noticing that in some cases, the previous implementation could lead to the creation of empty files, and offering a solution!

Over and out!

Have some questions? Noticed something wrong? Know of a better and / or simpler way to get it done? Some general feedback? Hit me up on Twitter!