SVN mirror for Live back-up - High Availability


Other then taking backup at regular interval, we may want to provide keep live data just to make sure data is highly available.

The best option is to backup Live data as soon as it gets changed.

Here is the trick for the SVN Mirroring.

How to create Repository
  • Only user with correct access as admin can create new repository
  • Identify the repository name and cluster. You must follow the approval process before creating any new repository.
  • Create repository svnadmin create /svnroot/<Repo_name>
  • change the owner of repository to apache(if needed) chown -R apache:apache /svnroot/<Repo_name>
  • Now you need to create Mirror repository on another SVN server, following are the steps to create "Mirror repository".

How to create Mirror Repository
  • Take the hotcopy of the repository from 1st SVN server.
svnadmin hotcopy /svnroot/<Repo_name> /tmp/<Repo_name>
  • Remove the hook script if any
  • create a tar file of the repo and scp it to 2nd SVN server.
  • Log in to 2nd server and untar the file.
  • now again do hotcopy to svnroot location on 2nd server
svnadmin hotcopy <Repo_name> /svnroot/<Repo_name>


  • Create 2 hook scripts start-commit and pre-revprop-change on 2st servver, The reason for this is that you can make use of these script to make sure that commit to 2nd server is happening via automated way instead of someone manually committing which you do not want to happen.
  • Change the owner of repository to apache(if needed) chown -R apache:apache /svnroot/<Repo_name>
  • Now you need to initialize the sync mechanism, login to 1st server.
  • Go to repository hook location and add following content in post-commit script.
#Calling sync job
TO=https://svn-mirror.domain.com/<repo_name>/
/svnroot/hook-scripts/sync.sh "$REPOS" "$TO" "2>&1 > /dev/null &


The Above will run after every commit and sync the latest revision to mirror.

Lets assume your main SVN repo is https://svn.domain.com
And mirror SVN repo is https://svn-mirror.domain.com

content of sync.sh

#!/bin/bash

REPOS=$1
MIRROR=$2

#Replace your username and password
SVNSYNC=/usr/local/bin/svnsync
SYNC_USER=bot 
SYNC_PASS=admin
SOURCE_USER=bot
SOURCE_PASS=admin

# Sync to Mirror
$SVNSYNC --non-interactive sync $MIRROR \
--sync-username $SYNC_USER --sync-password $SYNC_PASS \
--source-username $SOURCE_USER --source-password $SOURCE_PASS
if [ $? != 0 ]; then
mail -s "svn sync failed for $REPOS for $MIRROR" your-emain-id@my.com < /dev/null
fi

  • create a post-revprop-change script and add above content with following change
TO=https://svn-mirror.domain.com/<repo_name>/
/svnroot/hook-scripts/sync_prop.sh "$REPOS" "$TO" 2>&1 > /dev/null &

Content of sync_prop.sh

#!/bin/bash

REPOS=$1
MIRROR=$2

SVNSYNC=/usr/local/bin/svnsync
SYNC_USER=bot 
SYNC_PASS=admin
SOURCE_USER=bot
SOURCE_PASS=admin

# Sync to Mirror
$SVNSYNC --non-interactive copy-revprops $MIRROR \
--sync-username $SYNC_USER --sync-password $SYNC_PASS \
--source-username $SOURCE_USER --source-password $SOURCE_PASS
if [ $? != 0 ]; then
mail -s "svn sync_Prop failed for $REPOS for $MIRROR" yourname@my.com < /dev/null
fi

  • Now, you need to initialize the sync, run the following command on Main SVN server(1st one).
svnsync initialize http://svn-mirror.domain.com/<Repo_name>/ http://svn.corp.domain.com/<Repo_name>/ --sync-username bot --sync-password admin --source-username bot --source-password admin


How to debug the svn sync problem

Usualy sync failed and we get the alert saying the sync is getting failed.
if you try to run the sync manualy it may throw following error.

Failed to get lock on destination repos, currently held by 'something:db33884a-d10f-11e1-924a-47b0d8d1042f'
This means last sync couldn't perform the whole action and lock is still hanging for the repo.
In this case you log in to mirror server for which it is failing and run the following command

svn propdel svn:sync-lock --revprop -r 0 https://svn-mirror.domain.com/<reponame>/

This will clear the lock and output would be something like "property 'svn:sync-lock' deleted from repository revision 0".

Then you can go and run the manual sync command and it should work now. :)


Comments

Post a Comment

Popular posts from this blog

Colour formatting - Jenkins Console

Image
Jenkins is the main CI tool used in Software industry. This is the backbone tool for all CI and CD implementation, Build - release and deployment methods. With the help of 100s of plugin available around the jenkins, We can achieve so much of automation around build process including build/deployment pipeline, notification, analysis, pre and post build action etc. Jenkins has tremendous power to perform great amount of work around the core build process. Many time we write script in pre and post build action to perform many tasks which is becoming very command and necessary for automation. But the main limitation in running shell script is that it's very difficult to make it more human readable for errors and make it fancy. Today i am going to cover some steps to make the output of shell script colour formatted. Jenkins Configuration Steps First install following jenkins plugin https://wiki.jenkins-ci.org/display/JENKINS/AnsiColor+Plugin Go to jenkins configurati
Read more

Manage Docker images on local disk

Docker is very powerful containerization technique, and it is becoming famous in short time. People are adapting and containerizing their applications for deployment. Soon container become very heavy and consume too much of disk space, If you do not delete old images and layers you may soon run out of disk space. Here i am trying to cover the ways to regain the disk space consumed by docker images. Moving docker filesystem to bigger mount point. Usually docker keeps all temporary files related to image building and layers at /var/lib/docker This path is local to the system usually at root partition "/" . You can mount a bigger disk space and move the content of /var/lib/docker to the new mount location and make sym link.  This way even docker images occupy space, will not affect  your system as it will be using some other mount location. Remove Old docker images Here are few ways to remove old and unused docker images Removing stopped contai
Read more

How to migrate Parent pom from maven to gradle

Convert Maven parent/super pom to Gradle init file When you think of moving to gradle for some reason from maven, You need to take care of lot of stuff around it, You can find lot of tools and help to convert maven pom.xml to gradle build.gradle But one of the most important task as an engineer or architect would be to convert your super pom (or parent pom) in to gradle init scripts. Gradle init script provides same functionality as Maven super/parent pom. The basic difference is that you can call init script Run time As many as of them This gives us flexibility to change the init script on run time but doubt of not tracking the changes.
Read more