Going serverless with Bitcoin, Azure Functions and Cosmos DB

With serverless database computing, aspects we would have traditionally concerned ourselves with such as provisioning and managing infrastructure, installing and upgrading software, scaling to meet peak loads, have all been abstracted. This new way of working dramatically lowers the barriers to entry, allowing developers to get started far more quickly and focus on what's important, fast forwarding our ability to unlock value.

In this post, we will walk through an example by integrating Azure Functions with Cosmos DB as our endpoint. Our function will update Cosmos DB with Bitcoin (BTC) price data (e.g. Open, High, Low, Volume, etc.) on a regular interval using a time-based trigger (e.g. every 1 minute).

High-Level Flow

The API

The application programming interface (API) in this example comes from Bitstamp. Bitstamp is a Bitcoin exchange who fortunately offer a number of public data functions. The web service we will be using is called Ticker (https://www.bitstamp.net/api/ticker/) which defaults to USD.

At the time of this post, the API simply requires a GET request with no parameters (i.e. you should be able to see the data by simply punching in the URL into your browser).

Example JSON Response

{
	"high": "16600.00",
    "last": "15350.00",
    "timestamp": "1515415230",
    "bid": "15352.98",
    "vwap": "15894.1",
    "volume": "12238.16855868",
    "low": "15156.00",
    "ask": "15364.61",
    "open": 16173.98
}

Contents

1. Create an Azure Function Application
2. Create a Function
3. Integrate with Cosmos DB
4. Update Function Code
5. Update Function Schedule

1. Create an Azure Function Application

Instructions

1. Click New.
2. Type "Function" in the search bar and hit Enter. 
3. Select Function App and click Create.
4. Populate the required fields (App name (e.g. bitcoinFunctionApp), Resource Group, Location etc.).
5. Click Create.
6. Wait for your resource to be deployed (this may take a minute), click Go to resource.

Note: In the background this step has spun up three resources: an App Service, an App Service plan and a Storage account. For more information as to how resources work with Azure Functions, check out https://docs.microsoft.com/en-us/azure/azure-functions/functions-scale.

Video

2. Create a Function

Instructions

1. Hover your mouse over "Functions" and click the "+" symbol to create new.
2. Under "Choose a scenario" select Timer.
3. Under "Choose a language" select CSharp.
4. Click Create this function.

Video

3. Integrate with Cosmos DB

Instructions

1. Click Integrate.
2. Click New Output.
3. Select Azure Cosmos DB and click Select.
4. Populate the form as below:
   a) Document parameter name: bitcoinDocument
   b) Database name: bitcoinDatabase
   c) Collection name: bitcoinCollection
5. Check If true, creates the Azure Cosmos DB database and collection.
6. Under Azure Cosmos DB account connection click new.
7. Click Create new.
8. Populate the form as below:
   a) ID: A Unique Name (e.g. bitcoindb)
   b) API: Select SQL
   c) Subscription, Resource and Location as per your preferred settings.
9. Click OK.
10. Once the Cosmos DB instance has been successfully created (this will take a minute or two), click Save.

The important step to note here is 4a, the document parameter name. This is the name Azure will be binding with to integrate the Function with Cosmos DB and can be found in the code sample below within the functions parameters.

Video

4. Update Function Code (run.csx)

Instructions

1. Navigate back to your function by clicking on its name.
2. Copy and paste the code sample below into the inline editor.
3. Click Save.
4. If the console logs show "Compilation succeeeded.", click Run.
5. If the execution was successful, you should see "Document added to Cosmos DB!" in the logs.
6. Navigate to your Cosmos DB instance and click Data Explorer.
7. Expand Database > Collection > Documents, click on a Document to see the loaded data.

Important: At this point, the function is set to run every 5 minutes (as per the default settings). Unless the function is stopped or disabled, it will continue to execute every 5 minutes and populate the database. In the next step, I will demonstrate how to update the frequency but if you would like to stop here, you may want to delete the function so it stops consuming Azure resources.

Example Code

#r "Newtonsoft.Json"

using System.Net.Http;
using Newtonsoft.Json;

// HTTP Client
private static HttpClient client = new HttpClient();

public static async Task Run(TimerInfo myTimer, IAsyncCollector bitcoinDocument, TraceWriter log)
{

    try
    {
        // API
        var url = "https://www.bitstamp.net/api/ticker/";

        // 1. HTTP Request
        var request = await client.GetAsync(url);

        // 2. HTTP Response
        var response = await request.Content.ReadAsStringAsync();

        // 3. Deserialize - FROM: [JSON] TO: [.NET object]
        RootObject document = JsonConvert.DeserializeObject(response);

        // 4. Add Document (.NET object) to Cosmos DB
        await bitcoinDocument.AddAsync(document);

        log.Info("Document added to Cosmos DB!");
    }
    catch (Exception ex)
    {
        log.Error("An error occured: ", ex);
    }
}

public class RootObject
{
    public string high { get; set; }
    public string last { get; set; }
    public string timestamp { get; set; }
    public string bid { get; set; }
    public string vwap { get; set; }
    public string volume { get; set; }
    public string low { get; set; }
    public string ask { get; set; }
    public double open { get; set; }
}

Video

6. Update Function Schedule

By default, the Timer is set to run every 5 minutes. To change the frequency, follow the steps below.

1. Navigate back to your function.
2. Click Integrate.
3. Under Triggers click Timer.
4. Update the schedules CRON expression (e.g. 0 */1 * * * * is equal to "every minute").
5. Click Save.

Reminder: Unless you have stopped, delete or disabled your function it would be still executing at the specified interval (i.e. consuming resources). You must either stop the application, disable the function or delete to free up the resources.

Bonus - Data Visualised via Power BI

Using Power BI's Azure Cosmos DB connector (currently in Beta), we can consume, enrich and visualise the data. The trend line below shows data queried directly from Cosmos DB based on the Azure Function running on it's default 5 minute schedule for just over 6 hours, approximately 80 data points (running from 3:45 PM till 10:15 PM on Monday, 8th January 2018).