More Polymaps GeoJSON layer hacking

A while ago I started using Polymaps as my mapping library. I like it much better than other alternatives. I put together a rather stable map viewer that layers my own GeoJSON layer on top of OpenStreetMap tiles. My GeoJSON layer is rendered on demand and then cached to the file system in a nifty solution using Connect, its Connect’s static file middleware, Connect’s router middleware, and a database hitting service of my own. It all runs on node and is super fine and dandy. Some other day I’ll need to write that up.

All this map stuff serves a purpose. My idea is to use the map to interact with data. Click on a county, get the county’s data. Click on a freeway, get the freeway’s data. I got that working fine, but then my coworker offered the rather sage advice that most people expect some visual feedback when they click on something. Click on a county area, see the color of the county change to indicate that data has been downloaded.

“You’re using Polymaps, right?” Craig asked. “It’s easy. All you have to do is add a class to the clicked element and you can use CSS to style the border darker or something.”

Selected areas outlined

Persuaded by this flawless logic, I hacked up a quick solution. I already had .on('click', function(...)) handlers for my GeoJSON elements, so all I had to do was style the clicked element with an additional “active” class, and then write the CSS to say that “active” meant “stroke: black; stroke-opacity:.95“. That was easy.

Unsightly tile boundary lines

However, as the next figure shows, it is also wrong.  The problem is that the GeoJSON tiles are squares containing polygons. The tile is the thing I am downloading from my tile server, not the polygon. So outlining the selected path will outline the border of the tile if the selected area’s polygon crosses over the tile boundary. That’s why Los Angeles county and Kern county have a solid vertical line running through them both in the figure to the right. Then I realized that if I were to zoom in on an area the problem would get even worse, as the outlined areas would become just a grid of black squares.

Except that it didn’t! When I rolled my mouse wheel to check the outlining at higher resolutions, the outlining went away completely. That’s when I realized that my click handler was only activated on click, and it was only styling the active tiles for the selected area when the click happened. Once the map zoom changed and different tiles were loaded, they were styled without the “active” class. I realized I would have to hook into the map movement callbacks to update the styling of all selected areas. Oops. This was turning out to be a classic case of “easy” becoming “difficult and tedious”.

The easier problem was to fix the area styling. Because I was already shamelessly lifting the styling via CSS demonstrated in the Polymaps population example (except I’m using class not element id to do the styling, as I don’t like the idea of having multiple elements in the DOM with the same id, as would happen if I zoomed in on a county), I just copied the block of text, added “active” to the CSS selector, and search and replaced 255) with 128), thereby changing the hues from blue to green when an element is selected. Et voila no more unsightly tile lines.

Greens hues indicate selected areas.

Then I spent the rest of the afternoon wiring up the various ways to select and unselect geographic areas such that they all properly turn on and off the colors and the little dialog box that showed what areas were selected. Tedious and bug prone, but it was working pretty well. I also implemented a bunch of other user feedback elements (I think the UI/UX people call them affordances?) that Craig had suggested, and closed out the various tasks I had created in Redmine. As usual, I was winding everything up at about 5:30 and looking out the window as the sunset, thinking about my bike ride home. As I pushed the changes from my desktop to my more public demo server, I ran through the public site to make sure all the new changes were working. All was well, except I had never fixed the zooming-breaks-active-styling bug.

My first thought was to plug into the on('load',function(){}) handler for the GeoJSON layer. After all that was where I was setting up the class styles in the first place. For example, my original code for handling counties was:

 function county_load(e) {
     for (var i = 0; i < e.features.length; i++) {
         feature.element.setAttribute("class", 'fips'+feature.data.properties.fips );
         area_load('counties', feature.data.properties.fips, feature );
     }
 }

This is essentially the same as the Polymaps population example referenced above. The problem is that there is no obvious way to hook into this function to look up whether an area has been selected. At high zoom levels, the entire set of new layers could be within the same county. What I would really like to do is to add the styling after all the new layers are loaded and drawn, so that I only have to do one DOM select per zoom or move event.

But sometimes done is better than perfect, so I just wired up a simple check after the features loop. My first attempt at adding active styling was:

 function county_load(e) {
     for (var i = 0; i < e.features.length; i++) {
         feature.element.setAttribute("class", 'fips'+feature.data.properties.fips );
         area_load('counties', feature.data.properties.fips, feature );
     }
    jQuery('input:checkbox[name=datadown]:checked').each(function(idx,elem){
        handlers['counties'][elem.__data__.fips].setMapActive();
    });
 }

The selector line should be pretty obvious. jQuery('input:checkbox[name=datadown]:checked') filters through the checkbox block named “datadown” and pulls off all of the checked elements. But here is where it gets a little bit interesting. I am creating that checkbox input element dynamically based on what kind of area the user wants to view. If you want counties, you get the counties selection dialog, etc. I am using d3 to do this. While d3 deserves an entire book rather than just a quick paragraph, the neat feature with d3 is that one can attach data to a DOM element. So when I look up the selected counties via jQuery, the DOM element that is passed to the each loop contains valuable information, including the unique id of the selected area. In this example, counties are identified by their “fips” code. Further, what I haven’t shown is that for each area and unique area id, I have a single click handler (generated in the “area_load” function). This way my javascript program isn’t generating thousands of unique click handlers, I’ve just got at most 50 or so that are handling all the clicks. This is easier to debug, and in this case, pretty convenient.

I hacked a bit, and made the unique click handler container a global object. Globals are bad. I am a bad programmer. There is a more elegant way to do it. But again, done is much better than perfect, so I made that hashmap a global object. As a consequence, my tile load function can look up the handler for each of the selected elements by the area type (‘counties’ in this case) and the clicked element’s unique identifier (the county’s fips code in this case). Then I factored out a “setMapActive()” method that did the work of styling the multiple paths of the selected county as being active. That code looks like this:

        var select_area = function (ev){
            var cc = d3.select(select_area.element.element).attr('class');
            // because this could fire on already active elements,
            // let's not get too redundant with active classes!
            cc = cc.replace(' active','');
            var nc = cc + ' active';
            d3.selectAll('path[class="'+cc+'"]').attr('class',nc);

            // var cc = d3.select(select_area.element.element).attr('class');
            // jQuery('path[class$="'+id+'"]').each(function(idx,elem){
            //     d3.select(elem).attr('class',cc + ' active');
            // });
        };

Here I am using d3 to do the selecting, not jQuery. There is no reason I used jQuery in the earlier code block, other than I haven’t recoded it. But here I was using jQuery as you can see in the commented-out code to get access to its “attribute ends with” selector, as I will discuss below.

What the active, not commented out code is doing is still extremely hacky, and probably has a bug in it, but it works. In some code that I do not show here, when the unique handler for the area (county) and element (fips code) is first created, it is passed an SVG element (generated from the downloaded GeoJSON data). This element does not have any __data__ associated with it, as it was produced with Polymaps, not with d3, so I can’t use the __data__ trick I did before. Instead, I look up the element with a d3 select statement and get its class.

Here I must use d3 to select, and not jQuery. Apparently, when getting classes from SVG elements in the DOM, jQuery produces a complicated object with member elements, not a simple string value. I didn’t want to hack around with that. d3 seems to be masking all this trickery and producing a clean DOM-style class value.

The possible bug here is that I’m not entirely certain that the original element passed to the javascript handler creator will still be in memory every time this function is called. I think holding onto a reference like this will force the element to live forever (which is another kind of bug), but if not this routine will fail.

Anyway, the element’s current class name is loaded up into the variable cc (current_class, shortened). In testing I found that the element quickly gets the “active” part written to its class, but I don’t want that for a selector of other elements for this area (county) and id (fips code). So the next line after I generate the cc value is to strip any possible “active” addenda from the class. Then I produce a new class name that is the stripped original class name with exactly one “active” added back in.

    cc = cc.replace(' active','');
    var nc = cc + ' active';

Prior to this fix, I was using jQuery to only get elements that had the current county id as a class, but not the active additional class, using the “Attribute Ends With Selector”. But then within the each function, I had to select again using d3 in order to properly manipulate the class of the SVG element.

To try to streamline things and do fewer DOM selects, I now use d3’s selectAll to accomplish the class switching all at once. What I do is to look for all elements with the “inactive” version of the class (the area’s id plus whatever distinct prefix the area generator added—‘fips_’ in the case of counties), and then I reset the class on all of them to be the active version in the one-liner:

    d3.selectAll('path[class="'+cc+'"]').attr('class',nc);

So that works: as tiles are loaded, they are properly styled according to whether the area they represent is currently selected or not. At the other end of the spectrum, I needed a click handler to handle the case when an area was unselected:

        // unselect the area
        select_area.unselect = function(){
            // some other code handles the data side of things, 
            // this just uncolors the map
            var cc = d3.select(select_area.element.element).attr('class');
            var nc = cc.replace(' active','');
            d3.selectAll('path[class="'+cc+'"]').attr('class',nc);
        };

Largely the same as before. Again, if element.element goes away, I’m sunk, but so far I haven’t had a problem. This unselect click handler properly found all of the DOM elements with classes that contained the given area, id, and the word “active”, and stripped the active bit, turning them back to blue.

But there was a problem I didn’t expect—this unselect code doesn’t touch the tiles in the tile cache, and neither does the select handler shown earlier. So after zooming in and then unselecting the county, as you zoom back out, the county would perversely turn green again as soon as the map tiles were pulled in out of the cache. Similarly, if I zoom in and zoom out and select an element, all those cached tiles stay blue, not green. As my non-technical wife said when looking over my shoulder this morning as areas were brokenly toggling between green and blue “That is so irritating when websites don’t work right.” (She also said I need people to do this sort of thing for me, but that’s a whole ‘nother kettle of fish.)

The final fix was to hook into the map’s move event that Polymaps makes available. I couldn’t find any examples of how to use this feature on the Polymaps documentation and examples, and so that is the real motivation behind this writeup (that and the fact that it is Friday and my New Year’s resolution is to write more on Fridays).

I couldn’t decide whether to add a “show” listener to the GeoJSON layer that would be called every time a show event was triggered on a tile, or whether to listen to the map’s move events. I tried the show approach first, but it seemed like I was doing way to much work every time a tile came out of the cache. Each new showing tile would trigger a look-up all of the selected check boxes and then compare the tile’s area id versus the selected areas’ ids.

So I hooked into the map’s move event instead. It still seems like I’m doing way too much searching and class rewriting, but I can’t see a more elegant way that actually works every time. My current solution is the following:

var map_move = function(){
    var areaname = 'county' // default to something
    var mapmove = function(e){
        if(!areaname) return;
        jQuery('path[class$="active"]').each(function(idx,elem){
            var cc = d3.select(elem).attr('class');
            cc = cc.replace(' active','');
            d3.select(elem).attr('class',cc);
        });
        var checkers = jQuery('input:checkbox[name=datadown]');
        var checked = checkers.filter(':checked');
        checked.each(function(idx,elem){
            var idname = area_types[areaname].idfield;
            var elemid = elem.__data__[idname];
            if(handlers[areaname][elemid]){
                handlers[areaname][elemid]
                    .setMapActive();
            }
        });
    };
    mapmove.area = function(a){
        areaname = a;
    }
    return mapmove;
}();

What this code does is to first switch off all active areas. This catches all green areas that need to be switched back to blue, as well as some that don’t. Then it looks up the active areas in the checkbox block and sets those to active, which turns them back to some shade of blue. I added the area method to the handler to allow me to change behaviors whenever the area selection dialog was used to switch the kind of area being shown. I also allow for that area to be null in case I decide in the future to reactivate the highway-showing layer and need to do something different for that.

What I tried and failed to get working was to selectively switch off only those elements that were not selected. My thought was to use jQuery’s .not() function to not select all elements that were not checked. In practice this just spewed errors that I couldn’t track down. I believe it is because the majority of areas are not active at any given time, and not even on the map. But debugging required stopping the code during a move event, which broke firefox 4 and firebug. Rather than wrestle with bugs in incidental tools, it seemed easier to simply set everything to default, and then turn elements on as needed. In practice, I didn’t notice even a flicker when this was running, so I guess it is working pretty well.

Another approach that I haven’t tried yet is to store a reference to every tile as it is created in the click handler. Right now my code just spits back the existing handler if an area id already has one. But I might try something like:

function areas_click(arearoot,feature,selected){
    var id = selected.area();  //ancient cruft calling area() to get id()!
    if(handlers[arearoot] && handlers[arearoot][id]){
        handlers[arearoot][id].addAnotherElement(feature);
        return handlers[arearoot][id];

In this case, each area’s handler would already have a list of all of its associated SVG elements, and could style them right away. But I’ve found in the past that storing references to DOM elements leads to bloated websites that slow way down and crash sooner or later. The main function of this site is to view data, not to store references to map nodes. If I can’t generate plots for a year’s worth of hourly data for a particular county, then I may as well not be showing the map at all!

Update:

To follow up a bit with questions about actually how the colors are changed, I really did just mimic the approach taken in the the Polymaps population example of using a pre-compiled stylesheet.  Below I have a snippet of the styles, both for active and inactive tiles.

First here is the rule that covers the “inactive” case, with the names representing districts, airbasins, and county fips codes. For this set of counties, they are colored using the rule rgb(0,63,255).

.disAMA ,.disFR  ,.disMOD ,.disVEN ,
.abNEP,.abSCC,
.fips06021, .fips06039, .fips06049, 
.fips06083, .fips06109, .fips06113 {
  opacity: 0.75;
  stroke-opacity: 0.1;
  fill: rgb(   0,  63, 255);}

Next comes the corresponding rule for active areas. The CSS selector will match elements that have both ‘active’ and the related district, airbasin, or fips code class. These
will get colored according to the rule rgb(0,63,128), which has half the blue value as the unselected version. Not a drastic shift, but as you can see from the screenshots above, the effect is to produce a green tint rather than a blue tint. Most importantly, it is easy to see whether an area has been selected or not.

.active.disAMA ,.active.disFR  ,.active.disMOD ,
.active.disVEN ,.active.abNEP,.active.abSCC,
.active.fips06021, .active.fips06039, .active.fips06049, 
.active.fips06083, .active.fips06109, .active.fips06113 {
  opacity: 0.75;
  stroke-opacity: 0.1;
  fill: rgb(   0,  63, 128);}

And to be super redundant, I really am using the above listing of the function var select_area = function (ev)... to set the active class. Line by line:

      var cc = d3.select(select_area.element.element).attr('class');

This is a bit of an implementation detail, but the handler already has a reference to the element that I want to color. This reference is stored within select_area.element.element. The select_area variable is a hidden variable stored in the handler. I use d3 to select the object, then pull out the object’s current class, and save it to a variable cc.

       cc = cc.replace(' active','');
       var nc = cc + ' active';

Next I replace the class ‘active’ in the current string representation of the element’s classes with nothing (”), and then I create a new string that has the current class plus ‘ active’. I do the replace first because in testing there were situations in which the code would append redundant ‘active’ classes to an element. So if this code fires on an element that is already active, we still end up with just a single ‘active’ in the element’s CSS class.

        d3.selectAll('path[class="'+cc+'"]').attr('class',nc);

And finally, I want to color every element that has the same parent class, so I use the d3 selector 'path[class="'+cc+'"]'. This is implementation specific. I know my elements will either have the same fips code, district or air basin, and that aside from ‘active’ there isn’t any other class they will have. So this selector works for me. In a more complicated situation, you might not want to rely on ‘cc’ to be a good selector rule. Once all of these elements are selected, the d3 attr() function will assign the new class, which is the same as the old class but with the addition of ‘active’.