PostgreSQL CSV import

An article, posted 10 days ago filed in , , & .

Since I always forget (database management isn't my day-job): a short guide on how to quickly import large datasets in TXT or CSV into PostgreSQL. For smaller sets I still use ruby and FasterCSV to import the set, but nothing beats native DB imports in terms of speed. And speed doesn't matter when importing a few megabytes of data, but it certainly matters when it gets more than that.

In this example I'll use my current use-case, importing a large Drive-Time Matrix table, with drive times and distances between two postal codes. The head of the TXT file is formatted as such:


Now let's assume we want to import this in a table 'DTM' with the following columns: from_pc (integer), to_pc (integer), time (in…

Continue reading...

Technische schuld

An article, posted 23 days ago filed in , , , , , & .

De software wereld zit vol met Engelse begrippen. “Technical Debt” is ook zo’n begrip. Maar wat is het?

Een dag geleden zag ik dit bericht waarin Technical Debt werd vergeleken met een continue slecht gestapeld Tetris spel. Een rake vergelijking: als je niet oppast wordt de technische structuur van een applicatie een stapeling op zichzelf oplosbare problemen, maar vormt het in z’n geheel een nagenoeg niet te redden constructie.

Maar waar hebben we het dan over?

Technische schuld wijkt af van financiële schuld. Om financiële schuld te maken moet je geld lenen van een ander. Technische schuld ontstaat doordat er geen tijd wordt gestoken in het goed op orde maken van de techniek. Zo is het een goede gewoonte om kritische zaken en/of zaken die erg foutgevoelig zijn in de applicatie te voorzien van automatische tests (“kan die rol wel/niet bij die gegevens?”, “gaat het ook goed als hier een negatief getal …

Continue reading...

Phoenix’ Channels

An article, posted about 2 months ago filed in , , , , , , , , , , & .

I started exploring Phoenix for one thing only: Channels (or actually fast real time communication over websockets). In this post I explore how to use them (yes this is a follow up of My first Phoenix-app-post).

Preparing for the authentication problem

Websockets don’t pass session cookies. Because we don't have access to these we need to transfer the user's identity in a different way. One of the recommendations I found was passing a user_token using a ``-tag (adjusting templates/layout/app.html.eex):

We can access this with a simple query selector in javascript:


But that’s for later. Let’s move to the server side, since we need something to connect to, a Socket.


In our default project there is alrea…

Continue reading...

My first Phoenix app

An article, posted about 2 months ago filed in , , , , , , , , & .

Some time ago I actually initiated my very first Phoenix app, but was a bit disappointed by the lack of a rich box of gems (like that of ruby's) and/or I didn't have the time to invest heavily in researching all the possibilities. One of my new year resolution was to actively pursue more knowledge, hence I'm giving it a second shot.

Why Phoenix?

I'm a full-stack Rails developer, but I needed real time messaging. That is not something Rails is typically good at (although it works), but Elixir (with its Erlang base) is well known for (even in the ruby community :) ). Phoenix wraps Elixir in a nice (Rails-like) package ready for a web and API development


  • Authentication
  • Broadcasting filtered messages based on tags
  • Writing messages


  1. Install Elixir (macOS & homebrew: brew install elixir) and make sure you have [node.js](https://n…

Continue reading...

Op meer kennis!

An article, posted about 2 months ago filed in , , , , , , & .

Dit is er weer één in de serie van murb’s nieuwjaarsredes ;)

Als ik aan de toekomst denk, overkomt mij een vreemd soort gevoel. Enerzijds gaat het mij goed af. Ik heb de allerliefste vriendin en beschik over tal van mogelijkheden en vrijheden. Anderzijds voelt het alsof het nog even flink fout zal gaan. Al lijken de prognoses allemaal weer de gunstige kant op te gaan, er hangt een zweem van onzekerheid over.

Want is deze economische opleving niet allemaal gebaseerd op die kwantitatieve versoepeling waardoor er veel goedkoop geld beschikbaar is gekomen voor banken om uit te lenen? Die extreem lage rente die de huizenprijzen weer tot recordhoogtes heeft doen laten stijgen (en zo de huizenbezitters weer (voor nu?) uit het slop trekt (en het ‘logisch’ maakt dat de huren navenant stijgen))? En hoe gaat dat straks als ook Europa wordt geregeerd door rechts-extremisten, zoals Geert Wilders en Le Pen? Wordt er dan nog wel werk gemaakt van e…

Continue reading...

Facebook is the sender

An article, posted 3 months ago filed in , , , , , , , , & .

TLDR; Without the control and true freedom of speech we should regard Facebook as nothing but a attention draining casino made for people with fear of missing out. Equally so, we should consider it as an active sender as it actively modifies what is being said and consumed. And hold it responsible as such.

Probably most of us have felt fear of missing out. So we try too keep up with too many sources of information and entertainment. Facebook et al. promise to help us by filtering information. It is well described in for example the Facebook guidelines of how their newsfeed algorithm is supposed to function.

The filtering one gets is based on the behaviours of others. It is based on the often correct premise that the more people agree that something is 'worth sharing' or ‘worth interacting with’ makes it more 'worth showing’. Especially amongst ‘like-minded’ (actually: l…

Continue reading...

ActionCable and authentication with Devise (2/2)

An article, posted 5 months ago filed in , , , , , , & .

This is a short follow up on the previous article in which the ActionCable basics were explained. We can now add some level of authentication. Authentication is a bit harder than simply registering some before_action’s, but it is perfectly doable, especially if you've survived the previous tutorial.

From the official Action Cable guide we can simply reuse the full connection.rb template:

module ApplicationCable
  class Connection < ActionCable::Connection::Base
    identified_by :current_user
    def connect
      self.current_user = find_verified_user
      def find_verified_user
        if current_user = User.find_by(id: cookies.signed[:user_id])

Continue reading...

Getting started with Rails ActionCable (1/2)

An article, posted 5 months ago filed in , , , , , , & .

As the lead developer at I wanted to try the new Rails ActionCable technology for a new feature: shopping lists. The idea is that you can walk in a store or on a market, mark an ingredient as checked when you add it to your (physical) basket and continue shopping. ActionCable can make the experience nicer because it, based on WebSockets, allows for real time notifying other viewers and editors of the same shopping list. No more shouting around in the supermarket: I’ve got the milk! Sure, nothing essential, but I needed an excuse ;)

(Note that we recently migrated from Rails 4, so not everything was in place in our app, just ignore the bits Rails already made for you; everything has been tested with Rails

Getting the basics right

To start: You need a web server that can open multiple threads, so if you’re still using Webrick in development (which can’t rece…

Continue reading...

Try not using Javascript first

An article, posted 5 months ago filed in , , , , , , & .

My guiding principle in web-development is (still): Always make things work without (client side) JavaScript first.

Aside from offering a graceful degradation of the experience by progressively enhancing it leads to better code. Three reasons why:

  1. it forces you as a developer to think about logical endpoints for your form submits, your data requests etc. Typically this leads to fewer cases of overloading a resource with all kinds of unrelated functionality (yep, I'm a big REST-first advocate).
  2. your application will probably be more web-native, and hence more future proof, more easily cache-able, etc.
  3. the front-end JavaScript to enrich the experience will typically also be less complex and can be generalized more easily.

Yes, I do shiver when I hear things like CSS in JS, KISS!

Photo by [Dmitry Baranovskiy](…

Continue reading...

We don't need more silos

An article, posted 5 months ago filed in , , , , , , , & .

Apps are a great commercial succes. Every new operating system is claimed to be doomed. Still, most people use only 3 apps, 80% of the time (and 10 apps total 96% of the time).

While we cherish simple, little and focussed, these constitute a too limited view on what software could be. Not because simple, little and focussed is bad, but because many apps lack an important bit: interconnectivity. Most apps are silos.

We need integration

Integration nowadays is limited to simple sharing (typically ironically mostly urls) and a look and feel of the applications which is carefully described in Human Interface Guidelines that make things look integrated, like these: iOS UI guidelines and Android Material Design.

Still, wh…

Continue reading...

Be careful with what you ask for

An article, posted 6 months ago filed in , , , , , , , , & .

Since we all seem to know that every other field in the registration form is another percentage of users failing to register*, we think of alternative ways to gather information. We gamify the user profile completeness by adding a progress bar to our user account, we present a full form after the confirmation link or we ask questions while using the application.

But there is another reason why we might not even ask all the questions. Ask the wrong questions and you may alienate your user.

It can be relatively minor things like picking your favourite colour, where the form just lets you pick one colour, while many have multiple. But it may also be more personal (or one could argue, more political): not everyone defines oneself as male or female, so why only present just these options (and do know that it isn’t particularly nice being referred to as ‘the other’ all of the time).

These issues are well discussed in this [talk by Ca…

Continue reading...

De principes van murb

Toelichting: Niet iedereen maakt applicaties op dezelfde manier. In de loop der jaren heb ik wat principes ontwikkeld waaraan ik mij probeer te houden wanneer ik oplossingen realiseer. Het zijn handvatten waaraan ik mijzelf probeer te houden. Ik hoop ook dat opdrachtgevers mij hieraan helpen herinneren mocht het, vergeef me, toch nodig zijn. Vandaar deze openbare declaratie.


Het idee dat zaken veelal onnodig complex gemaakt worden? Het komt maar al te vaak voor dat IT-bedrijven oplossingen willen bouwen die geschikt zijn voor alles. Maar dat heeft een prijs: complexiteit. De minder complexe software van murb is eenvoudiger te gebruiken en ook nog eens goedkoper aan te passen aan jouw wensen.

De eindgebruiker staat centraal

Heb je het gevoel dat de gebruiker wordt vergeten bij lange lijsten met features? Er wordt door murb in de eerste plaats gewerkt aan gebruiksvriendelijke oplossingen. Met een duidelijke mens-machine interactie en interactieontw…

Continue reading...


An article, posted 7 months ago filed in , , , & .

It seems to happen, people who are at their thirties completely lost in their career.

For life’s problem of feeling lost there is no GPS, nor a map. Don’t expect enjoyment in random short lived ‘interesting’ idea consumption, try week+ long exploration of unknown territory instead.

When young, society allows you to try new stuff, but at the age of 30 you might start to think that society expects you to know it all. That you’re the senior. Stop believing this.

There is no route other than the one you’re taking. Continue, explore and experience, but ignore expectations. Finding optimal happiness is an NP-hard problem, try to find your happiness in traversing.

Continue reading...

Let’s encrypt! It’s easy!

An article, posted 7 months ago filed in , , , , , , , & .

There important reasons to use HTTPS. It makes your systems more secure, helps to protect your users privacy, and will prevent others to hijack your account to deface your site.

If you’ve ever tried to secure your site you may have found how hard it is. You have to generate a private key, a certificate signing request, upload that request somewhere, pay, process the e-mail, upload the certificate, configure your server and set a reminder that in 1, 2, 3 or 5 years you’ve got to go through most of that same process again (which I described before in more detail in an earlier "how I do it"-article. Well, no longer! Enter: Let’s encrypt.

> Actually, Let’s encrypt is so easy that I had doubts whether I should even write this post. But maybe it wins an extra soul or two over.

The recommended way to get sta…

Continue reading...

Notes: How to Hack PR for Your Startup

An article, posted 7 months ago filed in & .

Just attended a free webinar from Founder Institute: “How to Hack PR for Your Startup”. Here are some of their (Adeo Ressi (FI) and Conrad Egusa ( main takeaways:

  • Make sure you have a great design, no-one needs to tell you that
    1. Formulate the announcement: Make sure you’ve got news when you approach TechCrunch etc;
    2. Amplify your dreams; Let’s go to Mars!
    3. Messaging: the press release; make sure to make it as easy as possible: $20 /article; E-mail pitch (sell), press release is the make it easy to write. Make sure you use social proof, interesting stuff that adds to an interesting tool. Link your LinkedIn, Twitter etc. “Hi I’m Maarten (LinedIn, @murb”.
      1. Make sure you follow up, but don’t spam
      2. Don’t sell!
      3. Own: Make it your own
    4. Outreach. Create exclu…

Continue reading...

murb blog