Chapter 5. Backup Sources

You can configure multiple backups in one configuration file. Each backup is represented by a <backup> Tag. To configure the data that is stored use the <source> Tag.

Table 5.1. Supported types of backup sources:

Type	Description
arangodump	Backup ArangoDB with the arangodump command line tool.
elasticdump	Backup Elasticsearch with the elasticdump command line tool.
influxdump	Backup Influxdb with the influxd command line tool.
ldapdump	Backup LDAP data with the ldapdump command line tool.
mongodump	Backup MongoDB with the mongodump command line tool.
mysqldump	Backup MySQL with the mysqldump command line tool.
pgdump	Backup PostgreSQL with the pg_dump command line tool.
redis	Backup redis with the redis-cli command line tool.
rsync	Collect files via the rsync command line tool.
tar	Backup a directory with the tar command line tool.
xtrabackup	Backup MySQL with the Percona xtrabackup command line tool.

ArangoDB

Backup a ArangoDB database with arangodump.

Table 5.2. arangodump-Options

Name	Value	Required	Default	Description
endpoint	string	no	tcp://localhost:8529	URI to your ArangoDB endpoint.
username	string	no	null	Username for the ArangoDB authentication.
password	string	no	null	Password for the ArangoDB authentication.
disableAuthentication	boolean	no	false	Disable the ArangoDB authentication.
database	string	no	_system	Database to backup.
dumpData	boolean	no	true	Dump database data.
includeSystemCollections	boolean	no	false	Dump system collections as well.
collections	string	no	all	List of collections to dump.
pathToArangoDump	string	no		Custom arangodump executable location.

Example 5.1: arangodump XML example

<!-- source arangodump -->
<source type="arangodump">
  <option name="database" value="myDatabase" />
  <option name="dumpData" value="true" />
  <option name="collections" value="myCollection1,myCollection2" />
</source>

Example 5.2: arangodump JSON example

{
  "type": "arangodump",
  "options": {
    "database": "myDatabase",
    "dumpData": "true",
    "collections": "myCollection1,myCollection2"
  }
}

Elastic

Backup a Elastic index with elasticdump.

Table 5.3. arangodump-Options

Name	Value	Required	Default	Description
host	string	no	localhost:9200	URI to your Elastic host.
user	string	no	null	Username for the Elastic authentication.
password	string	no	null	Password for the Elastic authentication.
index	string	no	all	Index name to backup.
type	string	data	true	Elastic type to dump.
pathToElasticDump	string	no		Custom elasticdump executable location.

Example 5.3: elasticdump XML example

<!-- source elasticdump -->
<source type="elasticdump">
  <option name="index" value="myIndex" />
</source>

Example 5.4: elasticdump JSON example

{
  "type": "elasticdump",
  "options": {
    "index": "myIndex"
  }
}

InfluxDB - influxdump

Backup an InfluxDB database with influxdump.

Table 5.4. influxdump-Options

Name	Value	Required	Default	Description
host	string	no	localhost:8088	Hostname and port of your InfluxDB-Server.
database	string	no	all	Name of database you want to backup, all by default.
pathToInfluxdump	string	no		Custom influxdump executable location.

Example 5.5: influxdump XML example

<!-- source influxdump -->
<source type="influxdump">
  <option name="host" value="localhost:8088"/>
  <option name="database" value="myDatabase"/>
</source>

Example 5.6: influxdump JSON example

{
  "type": "influxdump",
  "options": {
    "host": "localhost:8088",
    "database": "myDatabase"
  }
}

LDAP - ldapdump

Backup you LDAP data with ldapdump.

Table 5.5. ldapdump-Options

Name	Value	Required	Default	Description
host	string	no	localhost	Hostname of you LDAP server.
port	int	no	all	Port to connect to your LDAP server.
searchBase	string	no		LDAP search base.
filter	string	no		LDAP search filter.
attrs	string	no		LDAP search attributes.
pathToLdapdump	string	no		Custom path to ldapdump binary.

Example 5.7: influxdump XML example

<!-- source ldapdump -->
<source type="ldapdump">
  <option name="host" value="localhost"/>
  <option name="searchBase" value="ou=Users,dc=fr"/>
</source>

Example 5.8: influxdump JSON example

{
  "type": "ldapdump",
  "options": {
    "host": "localhost",
    "searchBase": "ou=Users,dc=fr"
  }
}

MongoDB

Backup a MongoDB database with mongodump.

Table 5.6. mongodump-Options

Name	Value	Required	Default	Description
host	string	no	localhost	Hostname of your MongoDB-Server.
user	string	no	OS-User	Username for the MongoDB authentication.
password	string	no	null	Password for the MongoDB authentication.
authenticationDatabase	string	no	null	MongoDB database that holds the user credentials.
databases	string	no	all	List of databases you want to backup, all by default.
excludeCollections	string	no	null	List of collections you don't want to backup.
excludeCollectionsWithPrefix	string	no	null	List of collection prefixes to exclude matching collections from backup.
pathToMongodump	string	no		Custom mongodump executable location.

Example 5.9: mongodump XML example

<!-- source mongodump -->
<source type="mongodump">
  <option name="databases" value="myDatabase"/>
  <option name="user" value="user.name"/>
  <option name="password" value="topsecret"/>
  <option name="excludeCollections" value="collectionFoo,collectionBar"/>
</source>

Example 5.10: mongodump JSON example

{
  "type": "mongodump",
  "options": {
    "databases": "myDatabase",
    "user": "user.name",
    "password": "topsecret",
    "excludeCollections": "collectionFoo,collectionBar"
  }
}

MySQL - mysqldump

Backup a MySQL database with mysqldump.

Table 5.7. mysqldump-Options

Name	Value	Required	Default	Description
host	string	no	localhost	Hostname of your MySQL-Server.
port	integer	no	null	Port of your MySQL-Server.
user	string	no	OS-User	Username for the MySQL authentication.
password	string	no	null	Password for the MySQL authentication.
databases	string	no	all	List of databases you want to backup, all by default.
ignoreTables	string	no	null	List of tables you don't want to backup.
filePerTable	true \| false	no	false	Create directory with separate files for each table.
structureOnly	string	no	null	List of tables where only the table structure will be stored. Can not be used with filePerTable
hexBlob	true \| false	no	false	Use the --hex-blob option to dump blob fields in hex.
lockTables	true \| false	no	null	Use the --lock-tables option to lock tables during the dump.
singleTransaction	true \| false	no	null	Use the --single-transaction option to add BEGIN SQL statement before dumping data from server.
sslCa	string	no		Path to your ssl-ca file.
skipExtendedInsert	true \| false	no	false	Do not use multiple-row INSERT syntax.
skipTriggers	true \| false	no	false	Do not execute triggers during backup.
pathToMysqldump	string	no		Custom mysqldump executable location.

Example 5.11: mysqldump XML example

<!-- source mysqldump -->
<source type="mysqldump">
  <option name="databases" value="myDatabase"/>
  <option name="user" value="user.name"/>
  <option name="password" value="topsecret"/>
  <option name="ignoreTable" value="tableFoo,tableBar"/>
  <option name="structureOnly" value="logTable1,logTable2"/>
</source>

Example 5.12: mysqldump JSON example

{
  "type": "mysqldump",
  "options": {
    "databases": "myDatabase",
    "user": "user.name",
    "password": "topsecret",
    "ignoreTable": "tableFoo,tableBar",
    "structureOnly": "logTable1,logTable2"
  }
}

MySQL - xtrabackup

Backup a MySQL database with Percona xtrabackup.

Table 5.8. xtrabackup-options

Name	Value	Required	Default	Description
dataDir	string	no		MySQL data directory.
host	string	no	localhost	Host to connect to.
user	string	no	OS-User	User to connect with.
password	string	no	false	Password to authenticate the user.
databases	string	no	-	List of databases to backup, all by default.
include	string	no	-	Pattern to include databases and tables to the backup e.g. `^mydatabase[.]mytable`.
pathToXtraBackup	string	no		Custom xtrabackup executable location.

Example 5.13: xtrabackup XML example

<!-- source xtrabackup -->
<source type="xtrabackup">
  <option name="databases" value="myDB1,myDB2" />
</source>

Example 5.14: xtrabackup JSON example

{
  "type": "xtrabackup",
  "options": {
    "databases": "myDB1,myDB2"
  }
}

pgdump

Backup PostgreSQL databases using the pg_dump command line tool.

Table 5.9. pgdump-Options

Name	Value	Required	Default	Description
host	string	no	-	Host to connect to.
port	integer	no	-	Port to use to connect to the PostgreSQL server.
user	string	no	-	User to use to connect.
password	string	no	-	Password to use to authenticate.
database	string	yes	-	Database to backup.
schemaOnly	bool	no	false	Dump only schema information.
dataOnly	string	no	false	Dump no schema information.
schemas	string	no	-	List of schemas to dump, separated by comma.
excludeSchemas	string	no	-	List of schemas to exclude, separated by comma.
tables	string	no	-	List of tables to backup, separated by comma.
excludeTables	string	no	-	List of tables to exclude, separated by comma.
excludeTableData	string	no	-	List of tables where data is not dumped, separated by comma.
noPermissions	bool	no	false	Don't dump any permission schemas.
noOwner	bool	no	false	Don't dump any owner changing statements.
format	string	no	plain	Dump format, plain, custom, directory.
encoding	string	no	-	Set the dump encoding.
pathToPgdump	string	no		Custom pgdump executable location.

Example 5.15: pgdump XML example

<!-- source pgdump -->
<source type="pgdump">
  <option name="database" value="myDstabase"/>
  <option name="user" value="myName"/>
  <option name="password" value="topSecret"/>
</source>

Example 5.16: pgdump JSON example

{
  "type": "pgdump",
  "options": {
    "database": "myDatabase",
    "user": "myName",
    "password": "topSecret",
  }
}

redis

Backup Redis data structure store using redis-cli.

Table 5.10. redis-Options

Name	Value	Required	Default	Description
pathToRedisData	string	yes	-	Path to the Redis data .rdb file.
port	integer	no	6379	Port to use to connect to the redis server.
password	string	no	-	Password to use to authenticate.
pathToRedisCli	string	no		Custom redis-cli executable location.

Example 5.17: redis XML example

<!-- source redis -->
<source type="redis">
  <option name="pathToRedisData" value="/var/lib/redis/6379/dump.rdb"/>
</source>

Example 5.18: redis JSON example

{
  "type": "redis",
  "options": {
    "pathToRedisData": "/var/lib/redis/6379/dump.rdb"
  }
}

rsync

Backup files via the rsync command line tool.

Table 5.11. rsync-Options

Name	Value	Required	Default	Description
path	string	yes	-	Path to file/directory to sync.
host	string	no	-	Host to connecto to.
user	string	no	-	User to use to authenticate.
delete	string	no	false	Remove deleted files locally as well.
isDirSync	string	no	false	Keep the directory instead of archiving it at every PHPBU execution.
pathToRsync	string	no		Custom rsync executable location.

Example 5.19: rsync XML example

<!-- source rsync -->
<source type="rsync">
  <option name="path" value="/tmp/foo"/>
</source>

Example 5.20: rsync JSON example

{
  "type": "rsync",
  "options": {
    "path": "/tmp/foo"
  }
}

Directories

Backup directories using tar.

Table 5.12. tar-Options

Name	Value	Required	Default	Description
path	string	yes	-	Path to the source directory.
exclude	string	no	-	Comma separated list of paths to exclude.
ignoreFailedRead	boolean	no	false	Ignore changing source directory.
incrementalFile	string	no	-	Incremental cache file.
forceLevelZeroOn	string	no	-	Force a level zero backup at specific times.
compressProgram	string	no	-	Set custom compression program f.e. lbzip2.
removeSourceDir	boolean	no	false	Remove the directory after compression.
throttle	string	no	-	Throttle IO throughput via 'pv' (Linux/Unix only).
pathToTar	string	no		Custom tar executable location.

Example 5.21: tar XML example

<!-- source tar -->
<source type="tar">
  <option name="path" value="htdocs/uploads"/>
</source>

Example 5.22: tar JSON example

{
  "type": "tar",
  "options": {
    "path": "htdocs/uploads"
  }
}

Incremental backups

Doing incremental backups with tar is also supported. To activate incremental backups you have to specify a cache file where tar can store state information. Tar will then only do incremental backups unless told otherwise. You can force full backups (level zero backups) by removing the cache file manually or by using the forceLevelZeroOn option. In the following example incremental backups are activated by storing the backup state in a metadata.snar file and phpbu will force a level zero backup every Sunday and Wednesday. The backups on Monday, Tuesday, Thursday, Friday and Saturday will only contain the changes to the day before. To for example restore a Saturday backup you will need the backups from Wednesday, Thursday, Friday and Saturday.

Please configure your cleanup strategies accordingly to not use any data.

Example 5.23: tar XML example

<!-- source tar -->
<source type="tar">
  <option name="path" value="htdocs/uploads"/>
  <option name="incrementalFile" value="path/to/metadata.snar" />
  <option name="forceLevelZeroOn" value="%D@Sun|Wed" />
</source>

forceLevelZeroOn

date mask @ value [|value]

%D @ Sun|Web - Level zero backups on Sunday, and Wednesday

%H @ 3|15 - Level zero backups at 3am, and 15pm

Exclude files or directories

If you want to exclude some files or directories from the created tar file.

Example 5.24: tar XML example

<!-- source tar -->
<source type="tar">
  <option name="path" value="htdocs/uploads"/>
  <option name="exclude" value="folderA,fileA.txt,folderB"/>
</source>

Example 5.25: tar JSON example

{
  "type": "tar",
  "options": {
    "path": "htdocs/uploads",
    "exclude": "folderA,fileA.txt,folderB"
  }
}

IO Throttling

If you are experiencing high CPU load during your backups with tar you can throttle the IO to reduce CPU load.

Example 5.26: Throttling XML example

<!-- source tar -->
<source type="tar">
  <option name="path" value="htdocs/uploads"/>
  <option name="throttle" value="5m"/>
</source>

With a configuration like in the example above you make sure that you only write 5 Megabytes each second. This way your backup may take longer, but your CPU is able to handle other stuff as well.

This is done via the CLI tool pv. To use this feature you must have pv installed on your server. If you are using PHPBU on Windows, this feature is not available.

For more information on pv have a look here