The Story of BTC Price Alerts

As I refreshed BitcoinCharts.com for the thousandth time in a day, a thought struck me, "I wouldn't need to waste inordinate amounts of my time refreshing this page if there were a service that would email me when bitcoin's price made a significant movement." After a couple minutes of research, I found two services claiming to have this capability, but neither seem to be very fully featured, and frankly, both were a bit sketchy. I was looking for a side project to hack on, and Andy was Looking for a side project to hack on, we decided at that point to create BTCPriceAlerts.com.

The minimum feature set was as follows:
- Allow users to sign up and authenticate with the service
- Allow users to create and configure alerts at a given price threshold
- Automatically send an email or SMS alert when the price of Bitcoin crosses the configured threshold

More than creating a functional service, the purpose of BTCPriceAlerts.com is to explore new technologies, architectures, and workflows. What else are side projects for? After a couple design sessions we settled on a far-flung architecture consisting of at least six different components. Six components is probably a little overkill for such a limited feature set, but that's the point. Additionally, breaking the app into bite size components allows each component to own it's function and allows for the addition of new features without any of the components getting overly complex. After settling on the following components, we divvied up the work and started implementing. Andy wrote the user facing app while I took care of the alert architecture which included Twilio and Sendgrid integrations, the message queue, the message workers, the alerts API, and the market data daemon.

Message Queue
We are using a cloud messaging queue that is provided as a service from Iron.io. I couldn't really be happier with the service so far. Starting with the signup process and documentation and continuing through the automated daily usage reports, the product has been phenomenal. Using IronMQ from Iron.io provides me with a lot of infrastructure that I don't need to build or maintain. Some of my favorite features are the management console and the usage reports. The management console allows me to easily see the state of all of my queues and configure them. Because the app is a notification app, an important metric is the number of notifications sent. Iron.io provides this information both in the management console and in the daily usage report email without me having to track this metric explicitly.

Maybe the coolest feature of the IronMQ message queue is the ability to turn any queue into a "push queue". This means that any message put on the queue is immediately pushed out to every "subscriber". In my case the subscribers are the messaging workers (more on the workers next). The message queue ensures reliable delivery by requeueing the message if the "push" isn't acknowledged with a success indication. This greatly reduces the likelihood that a message is missed.

Message Workers
Iron.io provides another great service called IronWorker. IronWorker is a distributed worker architecture meant for asynchronous and long running tasks. IronWorker exposes an HTTP webhook endpoint that receives the push from the message queue with a JSON payload. That payload is then made available to the worker code. In the case of BTC Price Alerts, one worker handles the SMS notifications, and another worker handles the email notifications. The SMS worker receives messages from the message queue and delivers them through Twilio while the email worker delivers emails through Sendgrid.

The process of creating the workers went smoothly thanks to the quality of the Iron.io docs. They provide a command line tool so that creating a worker is just a matter of writing the code, creating a few configuration files, and running a few command line commands. Once the worker is uploaded, all I had to do was point my push queue at the endpoint exposed by the worker.

Market Daemon
The market daemon's job is to extract pricing data from Mtgox's API and send it to the alerts API so that alerts can be triggered. Mtgox caches their API results for 30 seconds, so polling more frequently than that is futile. I originally wrote this extremely simple polling loop in Go because I was curious to see what that language was all about. I've been following along with the community a little bit, and I wanted to try to implement something. This part of the app was so simple that I figured it couldn't hurt. Mostly due to my inexperience with everything from the syntax to the build process, building this daemon was much more painful than it needed to be. I eventually decided to scrap the Go implementation and fall back to ruby and EventMachine for the polling loop. This implementation has worked much better.

User App
The user facing web app is written in Python on Django. The user app owns all user data including the email and phone number associated with each user account. Importantly, the user app does not own any of the alerts data. The usr app includes a one page app to administer all of a user's alerts. That console faciliates creating, viewing, and deleting email and SMS allerts. All alerts data is queried form the alerts API in real time when it is displayed in the console. All alert creation and deletion requests are basically proxied through the user app to the alerts API. As I mentioned, I didn't write the user app, so you'll have to wait for Andy's highly anticipated blog post to learn the juicy details.

Alerts API
The heart of the alerts architecture I created is the alerts API. Written on Sinatra with a Padrino admin interface, the app owns all of the alerts data. It exposes a standard JSON REST API for CRUD operations on alerts. The app also receives price updates from the market daemon, calculates which alerts have been triggered, and enqueues those alerts on the message queue. The alerts data itself is stored in a Heroku Postgres database. The alerts API app does not have any user facing UI--only an admin interface. The app also doesn't know anything about the actual users other than a unique identifier. This design decision was made so that the alerts API app could focus on storing alerts and calculating which alerts are triggered.

Monitoring
BTC is a "highly critical" service with many "demanding users" monitoring "millions of dollars" worth of bitcoin. Entrusted with this duty, we've set up standard monitoring services to notify us when the app goes down. To monitor the alerts API and the user app, we using Pingdom. Monitoring the market daemon is a little bit trickier. First, it doesn't expose an HTTP endpoint to ping. Second, it's a little harder to monitor when something doesn't happen (i.e. the daemon stops sending prices to the alerts API). To monitor this part of the app, we use a service called Dead Man's Snitch. Basically, we receive a notification if a certain HTTP endpoint isn't pinged for an hour straight. The market daemon just pings that endpoint every time it successfully polls the Mtgox API and sends that price to the alerts API.

BTC Price Alerts is currenly in "private beta". I've been using it for about two weeks now and have gained some insights into changes that need to be made to the product. For intance, notification rate limiting is more or less a requirement to avoid "alert blindness" in which the volume of alerts causes them to be ignored entirely. We also plan to integrate exchanges other than Mtgox for those users who trade on other exchanges. Coinbase (though not truly an exchange) is a likely candidate for the next integration.

The project has definitely fulfilled its pupose of being an entertaining side project and an experiment in architecture. Of course, feel free to try it out and let me know what you think!

P.S. All of the source code is on Github if you want to check it out. I'm 44maagnum. Andy is ajk14.