In a previous post, we installed and tested the Spark job server, turning Spark into a REST API. We now want to actually do something with this component.
We will use the same BED file data that we have used previously, which contains information on transcription factors bindings on different chromosomes. The more binding sites have been registered, the more dense the colour is in the result on the right.
The thing to know with D3 is, first that is uses existing web standards (DOM, CSS, …) and second that its way of dealing with data is not obvious but very powerful.
I will not go into the details of the implementation, but just highlight a few points. First, doing the
POST request to the Spark job server and reading the data can be done using a method that d3 provides:
From this point onwards,
dataset contains the data that is returned from the Spark service. The content is in
JSON format and remember from the earlier post, it contains two fields. The
result field is the effective data.
Making the colors depend on the value in the cell can easily be done using D3 scales:
and later, this scale can be used as a function:
A very nice property of the way D3 deals with data can be seen in the latter code snippet. We provide a function to the
attr method. This function will be called for every entry in the dataset, passing the current value as a parameter
d. Also the index of the entry in the dataset can be passed as a parameter to the function.
Please note that this approach fits nicely with the rest of the architecture because we like to stick to a functional way of programming whenever possible.
Extensions and todo’s are clear at this point: include the locustree in the architecture, interactive zooming and panning through the data, different datasets, etc.