A developer’s status update: Test driven deadlock

No worries, I do value testing. But test driven? It depends.

The last few weeks I’ve been working off and on building a crawler. The thing triggers a series of scheduled tasks that could run in parallel, generating (possibly) tasks (that are consequently scheduled again in their own task-specific queue) on its own and so on. The end goal is structured copies of external resources (read: webpages). But I’ve been stuck close to the start for quite a while, setting up the base architecture using the test driven development approach. And I’m failing, it’s going too slow :(

Small victories

Like many developers, this is not my first parser/crawler. But this time I wanted to make a GoodParser™. Make it more extensible, and flexible and foremost robust, building in fault-tolerance from start. But I’m not getting near the end result and it is frustrating.

The TDD-school says make small victories, and oh yes, I had my sheer number of victories already. But the end is nowher…

Making block elements link

There is no standard way of linking block elements to pages in HTML. But sometimes you do need it. The <a>-element doesn't support any block elements inside (such as <div> etc.). I solve the problem with a jQuery snippet that allows me to simply add 'js_blocklink' to a block elements class and the jQuery code deals with the rest. It does assume that the first link is the 'main link' (also needed for graceful degradation when Javascript for some reason is not available). A bonus feature is that other links may still exist, and stay working. function enableJsBlockLinks() {     $(".js_blocklink").each(function() {         $(this).click(function() {             $(this).find("a").each(function(index) {                 a = $(this).attr("href");                 if (a…