A tool for cloning/syncing a local directory tree with an SFTP server.
Project description
A tool for cloning/syncing a local directory tree with an SFTP server.
Features
Keep in sync a local directory tree with a specified folder of an SFTP server.
Update symbolic links as needed and keep files consistent.
Automatic tilde expansion/handling on the SFTP server.
Public key authentication.
ssh_config entries compatibility.
Syncing exclusion patterns.
Compatible with both Python 2 and Python 3.
Install
You can install sftpclone by using pip:
$ pip install sftpclone --user
Note: Sometimes building required dependencies in user mode doesn’t work. In that case, you’d need to use sudo and to remove the --user flag. Alternatively, you could make use of a virtualenv.
Alternatively, you can clone this repository and then launch:
$ git clone https://github.com/unbit/sftpclone
$ cd sftpclone
$ python setup.py install
In both cases, you’ll find the sftpclone script in your path.
Usage
usage: sftpclone [-h] [-k private-key-path] [-l {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}] [-p PORT] [-f] [-a] [-c ssh config path] [-n known_hosts path] [-d] [-e exclude-from-file-path] [-t] [-o] local-path user[:password]@hostname:remote-path
Where, for each command line argument:
local-path: The path of the local folder. This path must exists and can contain ~ (we use tilde expansion).
sftp-url: It specifies the remote SFTP url having the form: [user[:password]@]hostname:remote-path. Both the password and the user field can be omitted. If you omit the former then you should specify a private key identity file. If you omit the latter then the current user is automatically used. The hostname can refer to a element of your ssh_config file. If the remote path contains ~, then it will be expanded to the default folder in which the user begins her SFTP session.
[h]elp: show the help message and exit.
private-[k]ey-path: the path to your private identity file. Set it if you are not using password authentication. It automatically defaults to ~/.ssh/id_rsa and can be used more than once.
[l]ogging: set the log level (ERROR by default).
[p]ort: SSH remote port (defaults to 22).
[f]ix-symlinks: if you have absolute symlinks pointing to your synced directory, they will remain consistent on the remote server: i.e., they will have an absolute path that reflect the path of the cloned directory on the server. Useful for cluster configurations.
ssh-[a]gent: enable ssh-agent support. Any private-[k]ey-path argument will be ignored.
ssh-[c]onfig-path: in the sftp-url’s hostname you can specify an entry of your ``ssh_config` file <#ssh_config-compatibility>`__. If you are using a non-standard path, you can set it here.
k[n]own_hosts path: path to your `known_hosts <#known_hosts-checking>`__ file. Default to ~/.ssh/known_hosts.
[d]isable-known-hosts: disable remote fingerprint check against local known_host file.
[e]xclude-from-file-path: the path to a file containing a list of patterns. Each file matched by these pattern will be ignored (not synced).
do-not-dele[t]e: do not delete remote files that are missing from the local directory.
all[o]w-unknown: do not ask for confirmation before connecting to unknown hosts.
Warning: be sure to select a proper remote folder. The synchronization process will indeed delete any file that doesn’t exist in the local folder (unless you turn the -t option on).
ssh_config compatibility
The hostname in the sftp-url parameter can be a valid entry in a ssh_config file. Specifically, your entry should have relevant parameters such as:
HostName
User
Port
IdentityFile
Any value not found will fallback to the CLI arguments. Anyway, you have to set the IdentityFile field, otherwise authentication will try to fallback to ~/.ssh/id_rsa and could not work. The first hostname matching the pattern is chosen (in the ssh_config way).
known_hosts checking
By default sftpclone will match the remote host fingerprint against the one contained in your ~/.ssh/known_hosts file. If this file doesn’t exists on your machine, you can specify a different path by using the -n option. Furthermore, you can disable the check with the -d flag. Unknown hosts will require the user to authorize the connection. Please note that, even after authorization, the known_host file won’t be modified.
Exclude list
It takes inspiration from the rsync/tar --exclude-from flag.
You can specify among your command line arguments a file containing a list of patterns, one per each line. All those files that match any pattern will not be synced with the SFTP server.
Lines beginning with ; or # are ignored.
Each pattern is considered relative to the syncing directory. As a consequence, leading / are ignored.
Example
; This will exclude any file or directory beginning with foo
foo*
; This will exclude any file foo in a subdir of the directory bar.
bar/*/foo
Programmatic usage
You can find some examples of programmatic usage inside the examples directory.
Testing
This project uses nose for testing.
You can test it by launching from the project root directory:
$ nosetests
$ python setup.py test # alternatively
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for sftpclone-1.1.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d9289a4285bf9dded51175320856fd81cf8c3eb7803d0bd985360f2e8029ef2a |
|
MD5 | 179134ad2976ff082791e656d2640205 |
|
BLAKE2b-256 | 925c923206aac610fc5bf277c4299f52d65c1145243b7843b83312c268d1a326 |