18 Jun 2016
Can I tell you about this doozy of a bug we just resolved at Studio71? When I say doozy, I mean a total mess and the path led us down to some questionable npm package decisions and eventually to Mozilla…and thankfully nothing overly stupid that we did!
Lets back up though…
Studio71’s stack is pretty simple by design. We have an API and offline data system that is written in Clojure which powers the various web applications, the largest of which is built with React.JS on top of a Nodejs middleware.
Make sense? Ok, lets get in to the bug.
Late the other night we got reports of something not working on one of our apps. It seemed like the app just couldn’t connect to a particular outside service. We’ll check on it in the morning. By morning we had even more reports of various, seemingly unrelated, services breaking. We dove in and found a lot of issues in the logs about not being able to verify the local CA cert for SSL connections. For those who aren’t aware, this isn’t an SSL to our servers, but when our app is making SSL connections to other services. The log message was saying we could verify the third party connection because something was wrong with our local Certificate Authority file. Ok…lets update the system’s CA file. No change. I can curl and wget from these web services fine, but the node application is unable to. Why is that?!
It doesn’t help that I’m less than familiar with our Node application. We’re in the middle of some “corporate restructuring” which means I’m down 2/3 of my JS developers at the moment. Nothing else is moving while I frantically read up on CA certs, Node JS in general, and better familiarize myself with the inner workings of how we request outside services in Node. I discover we use the
ssl-root-cas module…but everything looks fine and it’s set to fetch and use the latest CA during the build process. I take a break. I get back on it looking for other ways things can break. Maybe I need to update the calls and point the CA location to the system’s CA file…but the stage server works fine, so why should I have to do that here? I certainly don’t want a bunch of production file paths all over the code every time we make an SSL request. Lets take another break. Back at it, looking through the build logs when I see…
* Mozilla's root CA store
* generated from https://mxr.mozilla.org/nss/source/lib/ckfw/builtins/certdata.txt?raw=1
…and then, thankfully, I visited that URL. Probably out of something to do rather than anything else as it was late and I had run out of viable ideas hours ago…but I’m oh so happy I did because I saw…
Well that doesn’t look like a CA cert…ah, dammit.
Mozilla put that page on maintenance and the node module downloaded an HTML file rather than a CA file. Grrrrrrrrrrrrrr! Ok, ok, the need for a maintenance page can certainly come up. Why did that module download the damn HTML file?! …well, that’s Mozilla’s fault too. If you click that link and watch your Network tab, you’ll see a 302 temporary redirect to the maintenance page (totally fine) and then a 200 OK response from the maintenance page! A 200?! Come on! They return a 200 code and the node module downloads the file, stores it, and then tries to use it as a CA for all request. That’s not gonna work.
I’ve created and submitted a pull request on GitHub that updates the module to check the response content type in addition to the status code: github.com/coolaj86/node-ssl-root-cas/pull/23 I’ll never get that day of work back, but in the end these in the weeds moments are kinda fun…right?
25 May 2016
I recently had the opportunity to perform a “lightening talk” at San Francisco’s Gopherfest 2016, a conference about the Go programming language, about my go-alexa library to create Amazon Alexa skills.
Still no word on when the video of my talk will be released, but I’ve embedded my slides below:
I love speaking to groups on fun topics like this and it’s something I hope to do more of in the near future. Let me know if you’d like me to talk to a group of people you know of…like a conference, or people waiting for the bus…whatever.
14 Mar 2016
Update: I forgot to mention it in the post, but if there is any interest I could release a generic version to the store and/or the source code to the app. Let me know: @thatmikeflynn
I love screens. You love screens. Developers love screens! We have at least two or three on our desk and when that’s not enough we like to hang them all over the place so no matter where we look there’s a screen in there somewhere. Is the graph pointing up? Is that counter a high number? How many tickets has that new guy done? All important questions resolved by easy access to a well designed screen of information.
We love screens. We want screens. How do we get the screens? There are plenty of apps you can buy to create beautiful dashboards for you, and after researching the options, I want to tell you about the solution I came up with at Studio71.
We are a Multi-Channel Video Network, so we have YouTube, Vine, Facebook, Instagram, and [INSERT NEW SOCIAL VIDEO NETWORK] creators all over the globe. Our tech team aggregates those stats, moves the videos around, and creates applications for our creators, employees, and brands to log in and see the information we’ve collected on our network. In short, we already have lots of things to display on screens (Kanban boards, Google Analytics, internal metrics, etc…) we just needed the way to do it.
- Should rotate through $n URLs with an adjustable wait time.
- Easy to run on different machine types, specifically the Chromeboxes we have in the office for our conference rooms.
- Quick and easy setup.
- Remote updating of the application.
- Remote updating of the list of URLs.
- Allow for different displays to run different sets of URLs.
I created a Chrome App that met all of the above needs after a couple of hours of tinkering and off and on over the span of two days applying fixes. A Chrome app was perfect to get up and running quickly as it was uploaded to an unlisted Chrome Web Store page, and will run on nearly anything that runs Chrome.
The app has two layouts: The primary URL rotation screen and a very minimal options panel. The options panel allows the local user to give the app name. The name matches the remote configuration (more on that later). The only other option is to set the wait time between URL rotation, which defaults to 30 seconds.
The application logic is very simple: On load, the application checks the options to get the computer name and rotation delay. It pings a server to get a JSON file that dictates the URLs to rotate for each machine name, downloads the file and looks for it’s name (if the name isn’t in the list, there is a default configuration it will load). The app then creates web views for each of the URLs and sets and interval to rotate through the list, hiding all the views and displaying the current URL’s web view. There’s some additional logic for when to optionally reload each URL, and to handle a handful of events like pausing the rotation, advancing to the next URL, or reloading the page.
Security isn’t much of a concern as any URL authentication is handled by the local user. I could certainly add some option authorization logic to the URLs but that would have be created for each URL and would mean a massive security requirement on the JSON file location, so it’s not worth the trouble.
I launched the solution on Chromeboxes plugged in to two wall-mounted monitors and another Chromebox plugged in to our TV in the waiting area. The wall-mounted boxes are set to a “dev” profile that rotate through Google Analytics dashboards, Jira, our internal status page and a playlist of Studio71’s currently most popular YouTube videos. The big TV has a “lobby” profile that just includes the YouTube playlist.
Everything has worked perfectly! I’ve found that fullscreen YouTube playlists tend to just die after about 24 hours of playtime so having those URLs set to auto-reload at 24 hours works perfectly. I’ve also been able to add new dashboards to rotate through for the “dev” screens by easily updating the JSON file…and I’ve even added a few random gifs to rotate occasionally.
This solution could also easily be rolled out to out other offices and still all be controlled by a central configuration. I’d long since blocked it from my memory, but I actually did some contract work with a company that created a really really overly-complicated system to do this exact thing in banks but with Flash animations…and I think this solution works better after a few hours banging around the Google Chrome App documentation!
07 Feb 2016
We went to Super Bowl City in San Francisco the other day and outside of the excellent policing budget, you would never want to live in that city. Communist Russia-style food lines and prices, no schooling outside of how to “Punt, Pass and Kick,” and it was filled with people wearing jerseys for teams that weren’t in the Super Bowl! Why?! “Well this is a football thing so I have to wear my Patriots jersey…and hat…and pants!” No, you don’t. You look like a douche. Just wear a light jacket like everyone else! Arrg! Ok, here are some pictures.
“Fan Energy Zone”s was a place where you stand and they tell you stuff they want you to do, like watch kids to a half-hearted dance show or to remind you about the Fisherman’s Wharf-type food they were selling everywhere.
There was a big stage.
Watching people cram themselves behind these Manning and Newton was the single best part of Super Bowl City. It’s really an interesting commentary about the NFL: They looked Facebook post worthy cool from the front, but looking at them from a different angle and you realize they are just a fraud. A fraud with their ass sticking out.
It doesn’t take long to figure out what the actual point of Super Bowl City is: A place to put TV sets for live shots with the pier in the background. The NFL Network, CBS, and CNN (???) all had big sets that towered above the confused residents of Super Bowl City. I eagerly await a CNN news break about the New Hampshire primaries butted up against Kathy Griffin making gay jokes with Anderson Cooper in front of San Francisco Pier.
Ok, now lets talk about this thing. What is this supposed to be? It’s a bear…a hipster bear with the hipster hat and hipster glasses and the hipster v-neck tshirt…and icons of hipster stuff for tats. For fucks sake, one of the icons is a picture of a human hipster! From what I can see this mutant hipster bear mascot isn’t being used anywhere nationally, so is this how the NFL thinks San Francisco wants to be portrayed? Yes. The answer is yes. One of the NFL designers has a nephew that lived in San Francisco briefly before they flunked out of Berkley and took his word as the single source of input for how the Bay Area wants to be marketed to.
In a related note, here’s the mascot we designed for NFL City:
25 Jan 2016
We can all talk about my Comedy Hack Day 10 / Sketchfest later, but for now lets check out this git log from this weekend. This is how you Comedy Hack Day!