Jumping Into Javascript – Drawing JSONP Data on the Canvas

The last article in this series was a bit of a digression into the module pattern and namespaces in Javascript. This article will focus on a very simple charting algorithm.

Now that we have the data to draw and know how to move things on the canvas, we need to focus on making the chart look more like a normal chart. This post will focus on the drawing algorithms that will turn our data into a nice display and redrawing that same chart at different time periods/scales. Here’s the screenshot representing our progress from last time.

No Modifications

The first step is to find out the lowest value that will have to be drawn on the chart. We can align all the rest of the values with the bottom of the chart if we subtract the lowest value from each value on the chart. To do this we first have to traverse the sequence of data that will be shown and look for the lowest value.

var width = 600, height = 500
var chart = document.getElementById("chart");
var c = chart.getContext("2d");
chart.width = width; chart.height = height

//___________________________________________________________________________//
// Query and Init Procedures
//___________________________________________________________________________//

function getUrl(symbol) {
  return "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20csv%20where%20url%3D'http%3A%2F%2Fichart.finance.yahoo.com%2Ftable.csv%3Fs%3D" + symbol + "%26d%3D2%26e%3D27%26f%3D2011%26g%3Dd%26a%3D0%26b%3D1%26c%3D2000%26ignore%3D.csv'&format=json&callback=?";
}

$.getJSON(getUrl("ibm"), init);

function init(result) {
  var data = result.query.results.row;
  drawRects(data, 1);
}


//___________________________________________________________________________//
// Chart Drawing Procedures
//___________________________________________________________________________//

function drawRects(d, s) {
  var e = s + 30;
  var low = d[s].col3;
  
  for (var i = s; i < e; i++) {
    if (d[i].col3 < low) { low = d[i].col3 }
  }
  
  for (var i = s; i < e; i++) {
    c.fillRect(i*30, height-(d[i].col1-low), 20, d[i].col1 - d[i].col4);
  }
}

Align at Bottom

The next thing to do is scale up the bar heights to match the height of the chart. For this we also have to determine the highest high. Then we take the difference between the highest high and the lowest low and divide the chart height by that number, this gives us our multiple. Then we take the multiple and use it to compute the lower starting position as well as the height of each bar.

Before we were using the open data, however this won’t always give the lowest price. On any day the stock price fell, the open will represent the highest price. We don’t have to compute if the open is lower or higher than the close because our data gives us the low and high prices for the day. This is also beneficial as the low or high from the day could be significantly lower/higher than the open or close.

Also we’ve been drawing our chart backward, to flip it we need to subtract the offset from the width of the chart. In addition we should put in some color to tell what days are up or down. Since code is probably the best description of this formula I’ll stop trying to explain it and just show you the algorithm. I’m sure there is a cleaner/simpler solution (or maybe this solution is even too simple) however this can be updated a bit later down the road.

var width = 600, height = 500
var chart = document.getElementById("chart");
var c = chart.getContext("2d");
chart.width = width; chart.height = height

//chart.onclick = moveRects;


//___________________________________________________________________________//
// Query and Init Procedures
//___________________________________________________________________________//

function getUrl(symbol) {
  return "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20csv%20where%20url%3D'http%3A%2F%2Fichart.finance.yahoo.com%2Ftable.csv%3Fs%3D" + symbol + "%26d%3D2%26e%4D04%26f%3D2011%26g%3Dd%26a%3D0%26b%3D1%26c%3D2000%26ignore%3D.csv'&format=json&callback=?";
}

$.getJSON(getUrl("ibm"), init);

function init(result) {
  var data = result.query.results.row;
  console.log(data[1].col0);
  drawRects(data, 1);
}


//___________________________________________________________________________//
// Chart Drawing Procedures
//___________________________________________________________________________//

function drawRects(d, s) {
  var e = s + 15;
  var low = d[s].col3;
  var high = d[s].col2;
  
  for (var i = s; i < e; i++) {
    if (d[i].col3 < low) { low = d[i].col3 }
    if (d[i].col2 > high) { high = d[i].col2 }
  }

  var mul = height / (high - low);

  for (var i = e; i > s; i--) {
    if (d[i].col1 < d[i].col4) { c.fillStyle = "#00f"; }
    else { c.fillStyle = "#f00"; }
    c.fillRect(width - (i*30), 
               height - (mul * (d[i].col1-low)),
               20,
               (mul * (d[i].col1 - d[i].col4)));
  }
}

Simple Chart

And here’s the Yahoo Finance chart as comparison…

Yahoo Finance Chart

Finally here’s a snapshot of pyTrade, the father of JamochaTrade. Something similar to this is the ultimate goal of JamochaTrade.

pyTrade

As you can see our chart now looks a lot more like a stock chart! However, we still have a long way to go. Wicks, labeled axes, hover tags and much more need to be added, not to mention different styles of charts (line, mountain, dot, etc.). There are also a few bugs with our current implementation that need to be worked out.

The next article will concentrate on refactoring and debugging. Notably we’ll move our code into a namespace, as was discussed in the previous article. We’ll also rip out the drawing logic and put it into a function so we can make a more general drawing procedure that can handle different types of bar charts (Candlestick, OHLC, HLC).

Get JamochaTrade on GitHub

Advertisements

Jumping Into Javascript – Namespaces, we should have more of these

The last article dealt with cross-domain data retrieval. This article will be a short discussion about namespaces and the module pattern in Javascript.

Here’s a snippet from the Zen of Python that I think should be included in some theoretical incantation of the Zen of Javascript:

Namespaces are one honking great idea — let’s do more of those!

Namespaces are an important concept in any programming language. Programming without them is inevitably an exercise in tracking down hard-to-find bugs that occur when a definition is clobbered by an unrelated assignment.

Here’s the problem: There is a single global namespace for all the scripts running in a page!

This means that defining: a = 42 in the file display.js will clobber the: a = 7 definition in database.js! Let me restate that. When you are creating global variables in a Javascript file, you must know about ALL the global variables created in ALL the other script files.

This may be possible for small apps that you write by yourself. However for large apps and/or when multiple programmers are working on the same project, this is a nightmare. Luckily there is a solution and it’s what I call the namespace pattern.

The namespace pattern is very similar to, but not quite the same as, the module pattern. The module pattern is designed to allow private/public attributes of an object, which has benefits and drawbacks that are outside the scope of this article.

Links that describe these patterns can be found at the bottom of this page. It is not within the scope of this series to describe all the aspects of the module pattern. I will, however, give a short overview of the two approaches that I think are worthy of consideration. The first is the module pattern and the second doesn’t have a name that I have found, so I’ll call it the Prototype Pattern.

Module Pattern

var MODULE = (function () {
  // all code here
  return {
    // return everything you want to be public
  }
}());

Prototype Pattern

var MODULE = {};
MODULE.variable = 42;
MODULE.fn = function (x) {return x*x};

With the module pattern you must declare everything you want to be public. With the prototype approach you must prefix every attribute creation/access with the namespace. I have read some articles that suggest that prototypes are more efficient than constructor functions. I simply don’t know enough about Javascript at the moment to make that claim. Please research this yourself if it is important to you, don’t take my word for it.

You may come across some “Module Pattern Considered Harmful” type articles when researching. To save you some time here’s the argument: the module pattern is not a solution for every conceivable problem, therefore it should be avoided. Some of these articles do raise valid use cases where the module pattern is not applicable, if you’re interested in that they are a good read. Just don’t drink the kool-aid and believe that they are always useless/harmful.

Most of the issues with the module pattern revolve around private attributes. I have spent most of my time programming in Python and Scheme and, in the general case, I haven’t had a need for private attributes. However the real benefit of the module pattern (IMHO) is not private attributes, but avoiding the global namespace problem. A problem for which the module pattern is an adequate solution.

GOTO Part 6 – Drawing JSONP Data on the Canvas

Get JamochaTrade on GitHub

 

Links

JavaScript: The Good Parts
A Google Tech Talk given by Doug Crockford. This talk is a great overview of the good and the bad of Javascript for experienced programmers.

Javascript Programming Patterns
A good article that covers a number of methods of tackling the namespace problem.

JavaScript Module Pattern: In-Depth
More information about the module pattern.

Jumping Into Javascript – Same Origin Policy, JSONP, XMLHttpRequest, and Cheating with jQuery

In the last installment we successfully created an array of rectangle objects and moved them across the screen. This post will deal with retrieving JSONP data from a different domain.

Figuring out how to accomplish this took me longer than I care to admit. I began researching the same origin policy around 14:00 today and after about 30 minutes I knew I needed to find a service that offered stock quotes in JSONP format, which allows cross-domain access. I spent another 2 hours looking for a source of historical stock data in JSONP format before stumbling upon this post by IBM developerWorks. That led me to the Yahoo Query Language (YQL) which is a SQL-like language for retrieving tons of different datasets in XML or JSONP format. I proceeded to fool around with that for another 2 hours or so before figuring out how to access Yahoo historical stock data through YQL.

After 4 1/2 hours of looking for a new data source that was consumable by Javascript, I was not really in the mood to figure out the intricacies of portable XMLHttpRequest (which is bound to be a post, if not series, in itself). So, at least for the moment, I cheated and included the jQuery library from Google for the sole purpose of retrieving the JSONP data. Overkill? Yes, most definitely. For the moment though, it will have to do.

Total time to get this very basic feature implemented? 6 1/2 hours, including 2 meals and a number of games of darts with the new blowgun I bought yesterday. Moral of the story: the security practices in Javascript can make your life a real pain in the…

index.html

<html>
<head>
  <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.5.1/jquery.js"></script>
  <title>JamochaTrade - A Javascript stock charting program.</title>
  <style>
    body {text-align: center;}
    canvas {border: 1px solid #000;}
  </style>
</head>
<body>
  <canvas id="chart"></canvas>
  <script src="chart.js"></script>
</body>
</html>

chart.js

function getUrl(symbol) {
  // This should use the current date instead of a hardcoded date.
  return "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20csv%20where%20url%3D'http%3A%2F%2Fichart.finance.yahoo.com%2Ftable.csv%3Fs%3D" + symbol + "%26d%3D2%26e%3D27%26f%3D2011%26g%3Dd%26a%3D0%26b%3D1%26c%3D2000%26ignore%3D.csv'&format=json&callback=?";
}
  
function parseData(result) {
  // This returns an array of date objects the first row is labels.
  // For now use console.log so we can inspect the result in FireBug
  console.log(result.query.results.row);
}

// Loading and using jquery just for this is definately overkill.
$.getJSON(getUrl("ibm"), parseData)


var chart = document.getElementById("chart");
var c = chart.getContext("2d");

chart.onclick = moveRects;

function drawRects() {
  for (var i in rects) {
    var r = rects[i];
    c.fillRect(r.x, r.y, r.width, r.height);
  }
}

function moveRects() {
  for (var i in rects) {
    rects[i].x += 20;
  }
  c.clearRect(0, 0, chart.width, chart.height);
  drawRects();
}

function Rect(x, y, width, height) {
  this.x = x;
  this.y = y;
  this.width = width;
  this.height = height;
}

var rects = [new Rect(10, 25, 20, 50), new Rect(40, 25, 20, 50)];

drawRects();

For the moment I considered it more instructive to simply simply spit the result out to the console so we can inspect it. If you’re interested in seeing the full result object just use this code at line 9: console.log(result).

 

An Asynchronous Request

jQuery’s getJSON procedure makes an asynchronous request for JSONP data. What does this mean exactly? For one, it means that some things you might have thought would work, simply won’t. Take this as an example:

var data = false;
$.getJSON(getUrl("ibm"), function(result) 
                           { data = result.query.results.row; } );

console.log(data);

In a synchronous environment one would expect data to contain the result of the getJSON request. However, getJSON is an asynchronous method to fetch data. This means that the variable data may or may not hold the result of getJSON depending on how long it takes getJSON to fetch the data. If console.log is called before getJSON returns, the value of data will be false.

There are a number of solutions to this problem and I’m not really sure which is the best practice. First, using jQuery’s ajax procedure and setting async to false will not work with JSONP, just read the async entry in the link. There’s also a note in there about synchronous requests possibly freezing the browser, which is the major downside of this next approach.

data = false;
$.getJSON(getUrl("ibm"), function(result) 
                           { data = result.query.results.row; } );

while (data === false) { }

console.log(data);

Almost every time I try this I get the, “A script has become unresponsive” alert. Though it does work, this is not the kind of user experience I want in my apps.

My next thought was to use a sleep function and check the value of data only so often to prevent the above error. Well it turns out that the timer in Javascript is also implemented asynchronously! In addition it would seem that there is no standard sleep function and no easy way of implementing one, which I find a bit odd, but I digress.

There are a few other approaches that I have found. However there is only one that I actually like; just put the init code for the chart in the callback function! Humorously this is what I intended to do from the beginning and what my first example does. I wanted to make a simple example to show off here using a global to store the data. Well it turns out that the simplest way to do that is to do it in the callback.

$.getJSON(getUrl("ibm"), init);

function init(result) {
  var data = result.query.results.row;
  // do something with your data here
  console.log(data);
}

This approach may require rethinking the design of your app a bit, but it’s usually pretty simple. One approach is to simply put all your top-level commands inside a main or init function and use that as your callback function. A discussion of the different approaches to asynchronous code is outside the scope of this article.

That just about concludes this part of the Jumping Into Javascript series. Before ending this, I’ll leave you with the updated code. I have temporarily removed the ability to move items on the canvas, however the chart is now drawn from real data. It still doesn’t look all that great because I used a very na├»ve drawing algorithm to keep things simple and focused on fetching and handling the data, but you should get the idea.

var width = 600, height = 500
var chart = document.getElementById("chart");
var c = chart.getContext("2d");
chart.width = width; chart.height = height

//chart.onclick = moveRects;

//___________________________________________________________________________//
// Query and Init Procedures
//___________________________________________________________________________//

function getUrl(symbol) {
  return "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20csv%20where%20url%3D'http%3A%2F%2Fichart.finance.yahoo.com%2Ftable.csv%3Fs%3D" + symbol + "%26d%3D2%26e%3D27%26f%3D2011%26g%3Dd%26a%3D0%26b%3D1%26c%3D2000%26ignore%3D.csv'&format=json&callback=?";
}

$.getJSON(getUrl("ibm"), init);

function init(result) {
  var data = result.query.results.row;
  drawRects(data, 1);
}


//___________________________________________________________________________//
// Chart Drawing Procedures
//___________________________________________________________________________//

function drawRects(d, s) {
  var e = s + 30;
  for (var i = s; i < e; i++) {
    c.fillRect(i*30, d[i].col1, 20, d[i].col1 - d[i].col4);
  }
}

A quick screenshot too…

Screenshot

With retrieving and handling data out of the way, the next article will likely be a short digression into the namespace pattern. After that I will most likely cover a few possible implementations of the chart drawing algorithm and play around with Javascript’s first-class functions.

GOTO Part 5 – Namespaces we should have more of these

Get JamochaTrade on GitHub