Block troublemakers using fail2ban

An article, posted about 2 years ago filed in how i do it, debian, unix, configuration, server & devops.

I don't mind running my own virtual servers. Fail2ban is a tool I've had running on my servers for years. It helps fencing of requests from ip-addresses that repeatedly misbehave when connecting to SSH and postfix. I never got to creating my own rules. I thought I had to write it in some arcane scripting language, but recently I learned it is pretty easy.

In this case I wanted to block 500 (internal server error) and 422 (Unprocessable Entity) errors. A server error once in a while is expected, but repeated server errors are suspicious. Common source of these errors are scripts that scan for things like SQL injections.

Examples given are for Debian.

/etc/fail2ban/filter.d/nginx-errors.conf

[Definition]

failregex = ^ -.*"(GET|POST|HEAD).*HTTP.*" (500|422)
port = http,https
ignoreregex =
backend = auto
logpath = /var/log/nginx/access.log
bantime = 600
maxretry = 10

And appending to /etc/fail2ban/jail.local

Continue reading...

A local .test domain for development with https using Puma-dev (on macOS or Linux)

An article, posted more than 2 years ago filed in development, server, rails & local.

When you maintain a few projects locally developing against localhost works good enough. npm start or rails s or python manage.py runserver or php -S 127.0.0.1:8000 will boot up a server that binds to a local port and allows you to see your work locally. The advantage of using localhost is that you don't have to bother with https-traffic as browsers don't require https for their latest features, but sometimes you need different domains to test and running multiple services distinguished by nothing more than their port numbers can become hard to manage.

To address this problem not only for websites served by the puma server, puma-dev exists. It is a spiritual successor to Sam Stephenson's Pow, which solved this problem for rack-apps. puma-dev, however, can proxy other servers as well, whether these are written in Javascript, PHP, ruby or other languages; as long as these exposes a port to 127.0.0.1, your local loopback/host you can use…

Continue reading...

Must do speed optimalizations on nginx

An article, posted almost 8 years ago filed in performance, nginx, server, configuration & cache.

Nginx's default configuration needs a bit of extra configuration (at least on Debian systems) to enable gzip and client side caching. Two very quick wins for better performing web-apps.

Enabling gzip for more content types

Compression makes files smaller. By default only HTML is gzipped, but it it makes sense for quite a list of other file types too. This, however, excludes(!) images, which have their own methods of compression: compression over compression delivers you nothing, and costs you and your end-user a few more CPU cycles.

So find the gzip on; line in /etc/nginx/nginx.conf file (make sure it is not turned off or commented), and either uncomment the gzip_types-line or use this, more complete, line (including svg):

gzip_types image/svg+xml text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

Client side caching

Every browser has a cache of its own, …

Continue reading...

Rails and elasticsearch for beginners - follow up

An article, posted more than 8 years ago filed in elasticsearch, rails, search, linux, server, development, ruby, gem, tech & programming.

In a previous post I described how simple integrating elasticsearch is with Rails for beginners. You could've been happy with the fact that you now have implemented full text search, but that too basic set up probably doesn't work that much better than adding a column to your model, throwing in all text in it and running a LIKE query (although elasticsearch does try to rearrange the results a bit).

In this post I will learn you two things that makes elasticsearch worth it.

Analyzers

Analyzers add some fuzziness to your searches. First, make sure your analyzer is in the right language, this will improve your results. You add the following bit to your model (I typically place it just below where the scopes and validation are defined).

settings index: { number_of_shards: 1 } do
  ...

Continue reading...

Rails and elasticsearch for beginners

An article, posted more than 8 years ago filed in elasticsearch, rails, search, linux, server, development, ruby, gem, tech & programming.

I don't like complexity. Adding new items to your stack increases complexity. But sometimes it is worth it. When you need proper search and filtering, elasticsearch is worth it. Mostly because it isn't hard to set up at all, as you'll learn in this post.

Installing it on a Debian server is easy, simply follow their instructions (you'll add their package-server, and run apt-get install. On OS-X you can install it easily with HomeBrew (brew install elasticsearch), but do make sure you have installed a JDK (e.g. openjdk-8-jre-headless)

If you're not using something like Docker, you probably have to repeat the steps on your dev machine, your staging server and your production environment.

Note: When running on a low memory server, which isn't recommended for production, you should make sure that the configured heap size isn't too high, edit `/et…

Continue reading...

Setting up https/spdy communication for your website with nginx

An article, posted more than 9 years ago filed in ssl, https, nginx, server, configuration, security, privacy, certificate & how i do it.

In case you do something with user accounts on your website, you definitely want to make sure you're using https. In general it protects the user's privacy, also when just reading content on your website. The only thing that can be seen by a middleman is that the person is viewing something at your server, the rest is all encrypted. And since Google has started to rank https-websites higher it has even become a SEO technique :) ). This article explains you how to serve your pages over https.

Update: a better option exists nowadays for non-domain validated certificates: Let's encrypt!

While the path to your server from someones desktop could be considered relatively ok in the past (harder to tap, putting a lot of trust in everything from the ISP to the internet exchanges and everything else in between), things have changed now. Wit…

Continue reading...

A somewhat secure Debian server with nginx, Passenger, rbenv for hosting Ruby on Rails with mail support and deployment with Capistrano

Basically this is a technical note to myself, in case I need to setup another server for running yet another personal Ruby on Rails project. And don't worry, I'm not going to replicate all nice guides out there, just filling in the gaps.

So let's start with the list of bookmarks I follow as a start. Note that in these tutorials mostly a user is used named 'deploy'. Typically I create a user per project and name databases etc. accordingly.

  1. Get security right first: My first 5 minutes on a server or essential security for Linux servers
  2. Then I get Rails up and running with this how to install Ruby on Rails with rbenv on Debian
  3. (in case you want to use the server as your remote git repo too) [Git setting up a …

Continue reading...

How to do it: Using screen

An article, posted more than 10 years ago filed in tutorial, linux, server, introduction, ssh, unix, guide, debian, command line & how i do it.

A technical note to myself: One way of doing multiple things simultanenously on a server can be by setting up multiple connections via SSH, that's how I used to do things before. An alternative is to use a single connection and use the command screen on the remote server. Another good reason to use screen is if you have a long running process that you don't want to break just because your SSH connection flips on and off with your computer going in and out of stand-by.

This is for absolute beginners. If you don't know about screen, this is for you. If you are already familiar with screen, I probably won't be able to educate you :o

So what is Screen?

GNU Screen is a kind of window managment system for the terminal (you're ought to say terminal multiplexer) and has several advantages over using multiple SSH connections. Most importantly: the processes keep running when SSH d…

Continue reading...

murb blog