Scheduling Firebase Functions to auto-download images.

The Idea:

Here’s the idea: Yosemite (and other parks) have webcams that they make public. These cameras update every few minutes and there’s a webpage that shows the latest image. For example check out the Half Dome camera. The problem is these are just static images. If you’re looking at the camera at night, you can’t tell what it looks like during the day since there’s no way to scroll back in time. What if you had an app that allowed you to view all the imagery from all the cameras in one place, but also showed animations of previous time intervals (12hr, 24hr, 1w, 1m ,6 m, etc…). That’d be pretty cool right?

The Plan/Tech:

I’m a huge fan of serverless technology and wanted to build something out using firebase cloud functions to download the imagery on a schedule. Then the functions can store the files in firebase storage which is just a GCP storage bucket. Once all the Imagery is stored, another function could assemble the individual images into animations and store those in storage as well. All these functions could then update a Firestore database to clients to the latest imagery.

To set up a new firebase function, you just need the firebase CLI installed and to start a project, use the terminal to go to a directory and use “firebase init” which will walk you through firebase setup. For our cases here we only need functions enabled and I suggest typescript over javascript because it helps to avoid type issues.

Once the project is all initialized, go to the functions>lib>src folder to find the index.ts which is where we will be putting our functions!

The firebase team has provided many ways to trigger functions including via https, request/document updates on firestore or realtime database, pubsub, and a couple other ways. To access these triggers you just use the functions object to find the one you’re looking for. In the following case, I’m using pubsub to schedule the function to trigger every minute. You can change out pubsub for https to create your own api.

A really important thing to know about cloud functions is how to tell the service that your function is done. This can be accomplished by returning a promise or null on a pubsub function or sending a response on https. There’s a fantastic explanation on the firebase youtube video here:

With those basics, its super easy to get going and start building out our functions!

Time to get downloading!

The first thing we need to do is download the image. I did this with the request package which needs a destination to write to. Since the function instances are ephemeral, we can just store the file in the temporary directory. To get the temporary location, just import “os” and call os.tmpdir(). I didn’t want any file name conflicts so I’m just naming the file using a timestamp generated from Date.now() appended by the camera name. To make sure I don’t try to upload a file that doesn’t exist yet, I return the promise and resolve it when the file finishes downloading.

Now that we have the file saved, we just need to upload it to the firebase storage bucket. To do this, we first need to import the firebase admin package and initialize it using admin.initializeApp(). Since we used the firebase cli to set up the project, it’s already authenticated 🎉! With everything set up, uploading couldn’t be more straightforward. Just use admin.storage().bucket(“yourbucketname”).upload(“local file location”,{destination: “remote location your putting the file”});

The upload function returns a promise thats resolved when the file is uploaded so just use an await to wait for the file to finish uploading. Isn’t that easy?

Here’s the function that I used to download the image and upload it to my firebase storage bucket every ten minutes.

Testing your functions

So you’ve put your code together, but you want to test it now? Thankfully the firebase SDK has some cool ways to test your functions! There are two major ways to test your code locally. Once of them is with the firebase emulator which actually can emulate many different aspects of firebase including firestore database, auth, storage, hosting, and pubsub. This is useful if you don’t want to use your production firebase instance to test new features etc. This is a bit overkill for just functions though. If you want to try it out, you can run “npm run-script serve”.

If your function uses an https trigger (my preference for testing/building), the “firebase serve” function will emulate just the function and give you a localhost endpoint to ping the function with. You get the console log in the terminal and it provides a great way to debug and test your functions without deploying!

Time to deploy!

“firebase deploy”

That’s it. Literally just run that and it will package and publish your function on firebase! It does take a few minutes for it to finish, but it does everything automatically and once it’s all finished, your functions will show up in the functions page of firebase for you to keep an eye on them.

You can use the health tab to see if the function has run into any errors and then use the logs tab to dig into the errors and figure out what’s going on.

If you’re interested in learning more about functions, I can’t suggest this youtube playlist enough as they walk through the ins and outs of functions so you can get a better idea of how everything works.

Open Source

Check out the full source code HERE:

https://github.com/emerysilb/firebase-download-image

Thanks for reading everything! Hope you learned a thing or two!

--

--

--

Working to solve to worlds problems starting with technology in agriculture.

Love podcasts or audiobooks? Learn on the go with our new app.

DevOps without Test Automation = Coffee without Cream, Peanut Butter without Jelly

How to use DynamoDB with NodeJS? (The easy way😉)

Connected App in Anypoint Platform

Support FBS - Contact FBS

Perform a certain task whenever Django is started/restarted (runserver)

A roadmap from mechanic to software engineer

Mike Morris Completes AWS Cloud Practitioner Certification

Five Reasons Why Speed Matters …

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Emery Silberman

Emery Silberman

Working to solve to worlds problems starting with technology in agriculture.

More from Medium

Setting profiler and flame-graph in Visual Studio Code

The easy straightforward way to create demo serverless functions on Netlify

Firebase Auth: Sign In Link with Custom Expiration

Breaking the last Monolith