dup, rdiff: implemented and documented simpler and clearer symlinks/globbing

behaviour, as proposed on
http://lists.riseup.net/www/arc/backupninja/2006-07/msg00008.html

8 files changed, 100 insertions, 59 deletions

diff --git a/ChangeLog b/ChangeLog
index 5eab333..913a90f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -40,10 +40,10 @@ version 0.9.4 -- unreleased
 	svn:
 	 . Fixed inversed vsname emptiness check
 	rdiff:
-	 . Symlink and globbing support enhancement and clarification: for
-	   a given include/exclude/vsinclude statement, rdiff handler now
-	   support EITHER globbing with '*' OR symlinks in the path.
-	 . Fixed improper include/exclude symlink dereference
+	 . Symlink and globbing support in include/exclude/vsinclude
+	   clarification: globbing is fully supported again, whereas no attempt
+	   is done to dereference symlinks anymore, due to incompatibilities
+	   between various readlink versions in this field.
 	 . Removed overzealous vsnames check
 	 . Now works if testconnect=no and if $test is not defined.
 	 . add $sshoptions config parameter in [dest] section of config so
@@ -52,9 +52,10 @@ version 0.9.4 -- unreleased
 	   options = --remote-schema 'ssh -p REMOTE-PORT -C %s rdiff-backup
 	   --server'
 	dup:
-	 . Symlink and globbing support enhancement and clarification: for
-	   a given include/exclude/vsinclude statement, dup handler now
-	   support EITHER globbing with '*' OR symlinks in the path.
+	 . Symlink and globbing support in include/exclude/vsinclude
+	   clarification: globbing is fully supported again, whereas no attempt
+	   is done to dereference symlinks anymore, due to incompatibilities
+	   between various readlink versions in this field.
 	 . Removed over zealous vsnames check
 	 . Does not pretend anymore that duplicity can work without
 	   any passphrase
@@ -82,6 +83,8 @@ version 0.9.4 -- unreleased
 	 . Fix configdirectory error that forced you to use /etc/backup.d, thanks anarcat
 	 . When determining which backup actions to list, find now follows
            symlinks for $configdirectory
+	dup.helper:
+	 . Fix: signing was enabled with symmetric encryption.
     changed cron permissions to 644
     minor documentation fixes
     improved RPM build process allowing 'make rpm-package' and 'make
diff --git a/NEWS b/NEWS
index 7b262c4..f866125 100644
--- a/NEWS
+++ b/NEWS
@@ -1,8 +1,25 @@
+backupninja (0.9.4-1) UNRELEASED
+
+	* duplicity: Old (pre-0.9.4) example.dup file used to give false
+	  information about the way the GnuPG-related options are used.
+	  Please read the new /usr/share/doc/backupninja/examples/example.dup
+	  file, and update your own configuration files if needed.
+
+	* duplicity, rdiff: symlinks and globbing support in
+	  include/exclude/vsinclude was unclear and did not work in all
+	  situations, with weird behavious, due to incompatibilities
+	  between various readlink versions in this field. This has been made
+	  clear eventually: globbing is fully supported again, whereas no
+	  attempt is done to dereference symlinks anymore. 
+	  Please read the new /usr/share/doc/backupninja/examples/example.dup
+	  or /usr/share/doc/backupninja/examples/example.rdiff file, and update
+	  your own configuration files if needed.
+	
+backupninja (0.9.2-1) unstable; urgency=low
+
 WARNING FOR DUPLICITY USERS
 
-1. Old (pre-0.9.4) example.dup file used to give false information about the way
-   the GnuPG-related options are used. Please read the new example.dup file, and
-   update your own configuration files if needed.
+Old (pre-0.9.2) example.dup file used to give false information about the way
+the GnuPG-related options are used. Please read the new example.dup file, and
+update your own configuration files if needed.
 
-2. The bandwidthlimit option does not work anymore with duplicity >=0.4.2.
-   Trying to solve this...
diff --git a/examples/example.dup b/examples/example.dup
index f3d3981..457883c 100644
--- a/examples/example.dup
+++ b/examples/example.dup
@@ -55,9 +55,21 @@ password = a_very_complicated_passphrase
 
 [source]
 
-# WARNING: include, exclude and vsinclude statements support EITHER globbing
-# with '*' OR symlinks in the path; usage of both in the same statement is *not*
-# supported and will lead to weird behaviour.
+# A few notes about includes and excludes:
+# 1. include, exclude and vsinclude statements support globbing with '*'
+# 2. Symlinks are not dereferenced. Moreover, an include line whose path
+#    contains, at any level, a symlink to a directory, will only have the
+#    symlink backed-up, not the target directory's content. Yes, you have to
+#    dereference yourself the symlinks, or to use 'mount --bind' instead.
+#    Example: let's say /home is a symlink to /mnt/crypt/home ; the following
+#    line will only backup a "/home" symlink ; neither /home/user nor
+#    /home/user/Mail will be backed-up :
+#      include = /home/user/Mail
+#    A workaround is to 'mount --bind /mnt/crypt/home /home' ; another one is to
+#    write :
+#      include = /mnt/crypt/home/user/Mail
+# 3. All the excludes come after all the includes. The order is not otherwise
+#    taken into account.
 
 # files to include in the backup
 include = /var/spool/cron/crontabs
diff --git a/examples/example.rdiff b/examples/example.rdiff
index dd2ac5e..25cd26e 100644
--- a/examples/example.rdiff
+++ b/examples/example.rdiff
@@ -34,11 +34,20 @@ type = local
 keep = 60
 
 # A few notes about includes and excludes:
-#   - include, exclude and vsinclude statements support EITHER globbing with '*'
-#     OR symlinks in the path; usage of both in the same statement is *not*
-#     supported and will lead to weird behaviour.
-#   - All the excludes come after all the includes. The order is not otherwise
-#     taken into account.
+# 1. include, exclude and vsinclude statements support globbing with '*'
+# 2. Symlinks are not dereferenced. Moreover, an include line whose path
+#    contains, at any level, a symlink to a directory, will only have the
+#    symlink backed-up, not the target directory's content. Yes, you have to
+#    dereference yourself the symlinks, or to use 'mount --bind' instead.
+#    Example: let's say /home is a symlink to /mnt/crypt/home ; the following
+#    line will only backup a "/home" symlink ; neither /home/user nor
+#    /home/user/Mail will be backed-up :
+#      include = /home/user/Mail
+#    A workaround is to 'mount --bind /mnt/crypt/home /home' ; another one is to
+#    write :
+#      include = /mnt/crypt/home/user/Mail
+# 3. All the excludes come after all the includes. The order is not otherwise
+#    taken into account.
 
 # files to include in the backup
 include = /var/spool/cron/crontabs
diff --git a/handlers/dup b/handlers/dup
index b4a7c9d..15d1a02 100644
--- a/handlers/dup
+++ b/handlers/dup
@@ -134,24 +134,14 @@ symlinks_warning="Maybe you have mixed symlinks and '*' in this statement, which
 # excludes
 for i in $exclude; do
    str="${i//__star__/*}"
-   str=`readlink -f $str`
-   if [ -n "$str" ]; then
-      execstr="${execstr}--exclude '$str' "
-   else
-      warning "exclude statement '${i//__star__/*}' will be ignored. $symlinks_warning"
-   fi
+   execstr="${execstr}--exclude '$str' "
 done
 	
 # includes 
 for i in $include; do
    [ "$i" != "/" ] || fatal "Sorry, you cannot use 'include = /'"
    str="${i//__star__/*}"
-   str=`readlink -f $str`
-   if [ -n "$str" ]; then
-      execstr="${execstr}--include '$str' "
-   else
-      warning "include statement '${i//__star__/*}' will be ignored. $symlinks_warning"
-   fi
+   execstr="${execstr}--include '$str' "
 done
 
 # vsincludes
@@ -159,12 +149,8 @@ if [ $usevserver = yes ]; then
    for vserver in $vsnames; do
       for vi in $vsinclude; do
 	 str="${vi//__star__/*}"
-	 str=`readlink -f $VROOTDIR/$vserver$str`
-         if [ -n "$str" ]; then
-	    execstr="${execstr}--include '$str' "
-         else
-            warning "vsinclude statement '${vi//__star__/*}' will be ignored for VServer $vserver. $symlinks_warning"
-         fi
+	 str="$VROOTDIR/$vserver$str"
+	 execstr="${execstr}--include '$str' "
       done
    done
 fi
diff --git a/handlers/dup.helper b/handlers/dup.helper
index dbf9643..da299c6 100644
--- a/handlers/dup.helper
+++ b/handlers/dup.helper
@@ -199,6 +199,8 @@ do_dup_gpg() {
       if [ "$dup_gpg_sign" == yes ]; then
 	 do_dup_gpg_signkey ; [ $? = 0 ] || return 1
       fi
+   else
+      dup_gpg_sign=no
    fi
 
    # a passphrase is alway needed
@@ -310,9 +312,21 @@ password = $dup_gpg_password
 
 [source]
 
-# WARNING: include, exclude and vsinclude statements support EITHER globbing
-# with '*' OR symlinks in the path; usage of both in the same statement is *not*
-# supported and will lead to weird behaviour.
+# A few notes about includes and excludes:
+# 1. include, exclude and vsinclude statements support globbing with '*'
+# 2. Symlinks are not dereferenced. Moreover, an include line whose path
+#    contains, at any level, a symlink to a directory, will only have the
+#    symlink backed-up, not the target directory's content. Yes, you have to
+#    dereference yourself the symlinks, or to use 'mount --bind' instead.
+#    Example: let's say /home is a symlink to /mnt/crypt/home ; the following
+#    line will only backup a "/home" symlink ; neither /home/user nor
+#    /home/user/Mail will be backed-up :
+#      include = /home/user/Mail
+#    A workaround is to 'mount --bind /mnt/crypt/home /home' ; another one is to
+#    write :
+#      include = /mnt/crypt/home/user/Mail
+# 3. All the excludes come after all the includes. The order is not otherwise
+#    taken into account.
 
 # files to include in the backup
 EOF
diff --git a/handlers/rdiff b/handlers/rdiff
index 79577c6..bdc1cbd 100644
--- a/handlers/rdiff
+++ b/handlers/rdiff
@@ -179,23 +179,13 @@ symlinks_warning="Maybe you have mixed symlinks and '*' in this statement, which
 # excludes
 for i in $exclude; do
    str="${i//__star__/*}"
-   str=`readlink -f $str`
-   if [ -n "$str" ]; then
-      execstr="${execstr}--exclude '$str' "
-   else
-      warning "exclude statement '${i//__star__/*}' will be ignored. $symlinks_warning"
-   fi
+   execstr="${execstr}--exclude '$str' "
 done
 # includes 
 for i in $include; do
    [ "$i" != "/" ] || fatal "Sorry, you cannot use 'include = /'"
    str="${i//__star__/*}"
-   str=`readlink -f $str`
-   if [ -n "$str" ]; then
-      execstr="${execstr}--include '$str' "
-   else
-      warning "include statement '${i//__star__/*}' will be ignored. $symlinks_warning"
-   fi
+   execstr="${execstr}--include '$str' "
 done
 
 # vsinclude
@@ -203,7 +193,7 @@ if [ $usevserver = yes ]; then
    for vserver in $vsnames; do
       for vi in $vsinclude; do
 	 str="${vi//__star__/*}"
-	 str=`readlink -f $VROOTDIR/$vserver$str`
+	 str="$VROOTDIR/$vserver$str"
          if [ -n "$str" ]; then
 	    execstr="${execstr}--include '$str' "
          else
diff --git a/handlers/rdiff.helper b/handlers/rdiff.helper
index 904ac98..42bcb1b 100644
--- a/handlers/rdiff.helper
+++ b/handlers/rdiff.helper
@@ -271,12 +271,22 @@ type = local
 keep = $rdiff_keep
 
 # A few notes about includes and excludes:
-#   - include, exclude and vsinclude statements support EITHER globbing with '*'
-#     OR symlinks in the path; usage of both in the same statement is *not*
-#     supported and will lead to weird behaviour.
-#   - All the excludes come after all the includes. The order is not otherwise
-#     taken into account.
+# 1. include, exclude and vsinclude statements support globbing with '*'
+# 2. Symlinks are not dereferenced. Moreover, an include line whose path
+#    contains, at any level, a symlink to a directory, will only have the
+#    symlink backed-up, not the target directory's content. Yes, you have to
+#    dereference yourself the symlinks, or to use 'mount --bind' instead.
+#    Example: let's say /home is a symlink to /mnt/crypt/home ; the following
+#    line will only backup a "/home" symlink ; neither /home/user nor
+#    /home/user/Mail will be backed-up :
+#      include = /home/user/Mail
+#    A workaround is to 'mount --bind /mnt/crypt/home /home' ; another one is to
+#    write :
+#      include = /mnt/crypt/home/user/Mail
+# 3. All the excludes come after all the includes. The order is not otherwise
+#    taken into account.
 
+# files to include in the backup
 EOF
    ## includes ##
    if [ "$host_or_vservers" == host -o "$host_or_vservers" == both ]; then