Copying / syncing files over a local network with rsync

An article, posted 29 days ago filed in how i do it, rsync, copy, files, unix & macos.

Just a short article to document for myself how to copy a large directory (e.g. a user-folder) over a local network. While (s)cp might work for smaller operations, rsync is my preferred tool as you can restart it when it breaks + in case you found an optimization, you can just abort and restart. Some things to take into account before I share the command:

  • Do not mount a drive, just use ssh
  • if you're sharing from macOS, make sure file sharing has access to the entire harddrive, otherwise some important folders will sync empty (e.g. Documents(!))
  • Make sure you exclude files you don't need (a home folder typically contains many cache-files that you don't want to sync to a new machine
  • Do not enable some form of compression (it waists cpu cycles when your network is fast enough)

So here is the command:

rsync -aWP --inplace --exclude-from=exclude-file.txt murb@someaddress:/Users/username/ .

Breakdown:

  • -a is the archival option, and it is typically what y…

Continue reading...

*nix: find the largest files/directories within a directory

An article, posted almost 3 years ago filed in how i do it, unix, command line, terminal, sort, linux, macos & osx.

Every now and then I’m searching for this little snippet in my notes using NotationalVelocity (or currently actually a fork):

du -hsx * | sort -rh | head -100

It’s a variation of a snippet I found somewhere, but hardly invested any time in understanding what it actually does. Let’s decompose, from head to taildu.

head

head -100

head simply limits the results to a maximum of 100 lines. Not much more to explain here

sort

sort sorts. by default it sorts the files by filename, but adding ‘-h’ to it allows it to sort by “human readable numbers” (e.g. 5M > 6K); if ‘-n’ would be added as option 6K would be > 5M. The ‘-r’ options reverses the sort wich is by default ascending.

du

du by defaults crawls a directory recursively for all files. passing '-s' tells it to sum the values of files within directories. the '-x' option is used to n…

Continue reading...

How to do it: Using screen

An article, posted almost 8 years ago filed in tutorial, linux, server, introduction, ssh, unix, guide, debian, command line & how i do it.

A technical note to myself: One way of doing multiple things simultanenously on a server can be by setting up multiple connections via SSH, that's how I used to do things before. An alternative is to use a single connection and use the command screen on the remote server. Another good reason to use screen is if you have a long running process that you don't want to break just because your SSH connection flips on and off with your computer going in and out of stand-by.

This is for absolute beginners. If you don't know about screen, this is for you. If you are already familiar with screen, I probably won't be able to educate you :o

So what is Screen?

GNU Screen is a kind of window managment system for the terminal (you're ought to say terminal multiplexer) and has several advantages over using multiple SSH connections. Most importantly: the processes keep running when SSH d…

Continue reading...

murb blog