Synchronizing Folders

Someone asked me recently if there was an easy way to synchronize folders between two computers. Unfortunately, the easy ways tend not to work very reliably. But here is a method that's a little more complicated, but should be extremely reliable. You'll need to do some unix things and you'll need a place on the server where you can keep all of the files.

Note: You could avoid using the server as an intermediate space, but then you'd be limited to a system that only works inside the building. Or, rather, making it so it would also work outside the building would make it a lot more complicated.

We'll assume you're going to keep a folder in your home directory on a laptop and a desktop synchronized with a folder located on a lab hard-drive on marlin. Don't try to set this up with a folder in your home directory unless you've made prior arrangements with the technical staff, unless you only want to use it for tiny things.

Read more to learn how.

Shared-key Authentication

First, you need to set up shared-key authentication with the server. Open Terminal and type ssh-keygen It will ask you some questions, but you can just hit return for all of them.

Now copy and paste this command:

scp .ssh/id_rsa.pub marlin.bio.umass.edu:mykey

Then log into marlin
ssh marlin.bio.umass.edu

Note: If your username on marlin is different than the username on your Mac, you may need to user username@marlin.bio.umass.edu for both scp and ssh.

Once you've logged in, append the key to the end of the authorized_keys file
cat mykey >>.ssh/authorized_keys

Type exit to logout.

Once you've done that, test the setup by logging in again -- You should now be able to log into marlin, automatically, without having to type a password. If you are prompted to enter a password, there is a problem.

Create a script

Open Textwrangler and paste this into the window:

#!/bin/bash
#sync_me
/usr/bin/rsync -artuz --rsync-path=/usr/local/bin/rsync marlin.bio.umass.edu:/export/scratch/folder/ ~/folder
#sync_other
/usr/bin/rsync -artuz --rsync-path=/usr/local/bin/rsync ~/folder/ marlin.bio.umass.edu:/export/scratch/folder

You'll need to lightly edit this: The word "folder" refers to two directories that needs to be created: one in your home directory on your Macintosh and one in /export/scratch on marlin. NOTE: You can't actually create a folder in that directory on marlin. You'll need to create it somewhere else that you control.

Actually create both folders and then adjust the path(s) accordingly.

Save the file in your home directory as "syncfolders" or similar.

Automator

We can use Automator on the Mac to synchronize the folders every time you put something in.
1. Start Automator.app, choose type "Folder Action".
2. At the top, select the pop-up menu, choose "Other" and select the folder.
3. Type "shell" in the field to the left, and drag "Run shell script" into the area to the right.
4. Replace the word "cat" with ~/syncfolders (or whatever you called your script.)
5. Under the File menu, select "Save".

Testing

Try dragging a file into the folder. If you did everything correctly, a copy should promptly appear in the folder on the server.

If that doesn't work, click the "run" button at the top of the Automator window to see if the script executes successfully -- you may see helpful errors logged below.

Other computers

You can easily repeat the setup on other computers, provided it's all the same user. Don't set up someone else's computer to be able to log into the server as you with no password -- that's a really bad idea.

About removing files

You will find that once you've added a file to the folder on the Macintosh, you can't simply remove it. You can, but the next time you add something, it will get recopied into the folder from the server. You need to remove files from the server first, then remove them from the folder on the Mac.