Deploy style: Copy files (from SCP to Git)

An article, posted 2 months ago filed in deployment, linux, hosting, discussion, unix, yaml, virtualisation, files & Git.

This is an article in a series on Deployment.

Copying files is essentially the basis of all deployment styles

Deploy with SCP (or rsync over SSH)

Deploying may not be that hard if you’re deploying something static. A simple copy or rsync command may well suffice.

This will work fine for static sites, but more dynamic sites often require more setup. Hence, you will often find some additional scripts that will take care of this. These scripts are imperative in nature.

Variant: deploy with .git

Small applications can be deployed with a true .git deployment: git push and remote pull, directly serving the git repository via a server.

.git based deploy

First time encounter I had with a .git based deployment approach was with Heroku (US based), a service that made deployment easy. You push (git terminology) your app, would configure your database, and it would figure out the ruby version…

Continue reading...

Data recovery adventures

An article, posted about 2 years ago filed in harddrive, recovery, hdd, files, data & backup.

I’m not a data recovery specialist, but I tend to help people out every now and then with computer problems, and sometimes you have to deal with a disk that doesn’t seem to work anymore. This is a collection of notes I made after I got a new challenge. I had to resort to using testdisk and ddrescue. I’ve been testing a few tools, and read about more, but these were the tools I landed on.

IMPORTANT: I’m not an expert on this topic. Every additional moving of the needle can cause a harddrive to fail even harder. If your data is absolutely critical, use a certified data recovery service.

1. Is it replaceable hardware?

When I get a harddrive in a laptop or external enclosure, my first attempt is to see if there is something between computer and the disk. I try to remove the disk from the computer or enclosure and connect it to a USB3 to 4-types of P/SATA adapter I have bought years ago and has saved me multiple times. If it mounts, copy everything to another drive, to b…

Continue reading...

Copying / syncing files over a local network with rsync

An article, posted about 5 years ago filed in how i do it, rsync, copy, files, unix & macos.

Just a short article to document for myself how to copy a large directory (e.g. a user-folder) over a local network. While (s)cp might work for smaller operations, rsync is my preferred tool as you can restart it when it breaks + in case you found an optimization, you can just abort and restart. Some things to take into account before I share the command:

  • Do not mount a drive, just use ssh
  • if you’re sharing from macOS, make sure file sharing has access to the entire harddrive, otherwise some important folders will sync empty (e.g. Documents(!))
  • Make sure you exclude files you don’t need (a home folder typically contains many cache-files that you don’t want to sync to a new machine
  • Do not enable some form of compression (it waists cpu cycles when your network is fast enough)

So here is the command:

rsync -aWP --inplace --exclude-from=exclude-file.txt murb@someaddress:/Users/username/ .

Breakdown:

  • -a is the archival option, and it is typically what y…

Continue reading...