Code

Update: How To Sync Local Development Environments

Update:

I’ve simplified a way to implement this and threw it into a repo.

In this post I’ll show you how to keep your Varying Vagrant Vagrants (VVV) synced across multiple machines (including databases).

The Problem

  • Inconsistency across development machines.
  • Management of separate environments for the same projects.
  • Remote access to development files.

Solution

  • Resilio Sync (formally BitTorrent Sync) for syncing files
  • Custom vagrant halt to backup database to then import on vagrant up

Why use Resilio Sync instead of Dropbox?

While trying to figure out the best solution, I tried a few different services (Dropbox, Google Drive, iCloud, ownCloud, MEGA). All of these services seem to have a missing feature that’s needed, the ability to ignore files and folders.

  • Dropbox – I really wanted Dropbox to work because of the quality of the product but after syncing all my files (including node_modules) it ends up draining your machine’s resources and perpetually indexes these directories.
  • Google Drive – Same issue as Dropbox.
  • MEGA – Can ignore files but limits your bandwidth on free version so it takes forever to sync your files.
  • iCloud – Really isn’t a sync service. It’s more of a remote drive and you can’t ignore files or see status on synced files.
  • ownCloud – It’s possible to use ownCloud for this since you’re able to edit files to ignore but to run ownCloud you’ll need a Raspberry Pi or to host it on your own server. Regardless, I found ownCloud to be the slowest with syncing.

Resilio is the only option that lets you globally exclude files and folders on a root directory level. When developing with Grunt.js,  a node_modules directory is created with a large number of files that aren’t needed for syncing.

Installing Resilio Sync

Personally I followed this tutorial that shows you how to install Sync to a Raspberry Pi. This isn’t needed but is nice to have. If you don’t have a Raspberry Pi you can install the the syncing software on each of your machines. Once installed, add your /vagrant-local directory to Sync.

Screen Shot 2016-08-19 at 10.46.21 AM
From here you can copy a link to import into your 2nd machine.

Indexing and syncing will take some time depending on the size of your /vagrant-local directory. Mine was around 9GB and took half a day. Compare this to the 2 days it took to sync to Dropbox.

Excluding Files and Folders

This is the feature that all the major syncing services are missing. The ability to ignore files globally.

Before you start syncing both machines, lets exclude files and folders that aren’t needed. Within /vagrant-local/.sync open IgnoreList.

At the bottom of IgnoreList add:

You can also add any files and folders to this list you may want to ignore. (help)

Note: You’ll need to edit this file on both machines.

Syncing Databases

For syncing databases, please follow a post by Zach Adams under the section “Syncing Databases”.


This is where it gets fun. Before we begin make sure you have the vagrant plugin vagrant-triggers installed.

So VVV automatically runs a script called db_backup on halt, which backs up all the databases into the folder database/backups. We’ll create a script to copy these backups to a sync folder, and since Dropbox/Google Drive/etc. syncs your folders it will automatically detect and sync these .sql files for us. First we’ll need to edit the Vagrantfile.

Vagrantfile

We want to tell VVV that on vagrant up we want to import all backed up databases back into VVV. Since we have the vagrant-triggers plugin installed we can tell VVV to run a script on up. Open your Vagrantfile and near the bottom there’s a section for Vagrant Triggers that looks like this:

Add the following snippet directly below the if statement:

This tells VVV to run the sync-sql script on up, which we’ll create next. This section should now look like this.

sync-sql.sh

Now we’ll create a script based on the import-sql.sh file located in the database folder. Go ahead and create a file called sync-sql.sh in the database folder and copy/paste the the script located here. The script is commented so you can see what it’s doing.

vagrant_halt_custom

Now we want to tell vagrant that during the halt command we want to copy the backups it’s made into the sync folder. Copy the script here and create a file called vagrant_halt_custom in the config/homebin folder.

Next you have to run vagrant provision to make sure all the files get copied to the correct locations.

The last thing we want to do is ssh into the vagrant box with vagrant ssh and run the following commands so that vagrant can run the script:


Conclusion

You should now be able to sync your dev environment across multiple machines and have remote access to all your files. Keep in mind you’ll need to halt one machines to then up the next.

Resources

8 thoughts on “Update: How To Sync Local Development Environments

  1. Is the point of the raspberry pi just to have a device synced up that you can leave on all the time? So rather than needing both computers on at the same time each computer can sync with the pi without needing the other one on?

    1. I’m sure there are plenty of ways to achieve what this does but for me this seems to be faster and with less hassle. Everything automatically syncs and I don’t have to pull/push to GitHub.

      One thing that is actually great that I didn’t mention is that your .git folders are synced as well. So you can make changes on one computers (it syncs) then push from a different computer.

  2. We have a tool that’s in beta right now, but soon to be released publicly, that simplifies synchronizing development environments similar to what you discussed in this article. It’s called Nanobox. It’s a micro-platform that runs on top of a Docker container.

Leave a Reply