Skip to main content

CopyXnat is a python utility for copying XNAT projects from one XNAT server to another

Project description

CopyXnat

test docs package Python license PyPI DockerHub

A utility for downloading or copying projects between XNAT servers.

This is a prerelease. It may not work as intended, and may lead to data loss. Please use caution when running on production servers

Quick start

Please make sure your XNAT archive and PostgreSQL database are fully backed up before running CopyXnat on a real XNAT server. CopyXnat may modify and overwrite your data.

Install and run using pip

Python 2.7 or 3.6+ is required, and pip should be installed.

It is recommended that you do not modify your system Python installation. Use a tool such as Homebrew (macOS) to install the latest python version without affecting your system python. Then, create a dedicated virtual python environment using virtualenv and virtualenvwrapper or similar tools.

Once you have a suitable python environment set up, install with pip:

    pip install copyxnat --upgrade
    copyxnat --help

You can use pip's --user flag to install dependencies in the user space instead of globally.

Install and run using Docker

If you have Docker installed you can run CopyXnat in a container. Note that interactive mode (-it) is generally required in order to enter server passwords where prompted.

    docker run --rm -it tomdoel/copyxnat --help

Example use cases

Show project contents

  • Print a list of projects, subjects etc on an XNAT server

    copyxnat show -s "https://YOUR-SERVER-URL" -u "YOUR-USER-NAME"
    
  • Show differences between two projects

    copyxnat compare --src-host "https://SOURCE-SERVER-URL" --src-user "XNAT-USER-SOURCE" --project "PROJECT-NAME-SRC:PROJECT-NAME-DST" --dst-host "https://DEST-SERVER-URL" --dst-user "XNAT-USER-DEST"
    

Download projects

  • Download XNAT projects onto your computer

    copyxnat export --src-host "https://YOUR-SERVER-URL" --src-user "YOUR-USER-NAME" --project "PROJECT1,PROJECT2"
    
  • Download all projects you have access to onto your computer

    copyxnat export --src-host "https://YOUR-SERVER-URL" --src-user "YOUR-XNAT-USER-NAME"
    

Copy projects from one XNAT server to another

  • Duplicate a project on a destination server

    copyxnat copy --src-host "https://SOURCE-SERVER-URL" --src-user "XNAT-USER-SOURCE" --project "PROJECT-NAME" --dst-host "https://DEST-SERVER-URL" --dst-user "XNAT-USER-DEST"
    
  • Duplicate a project and rename

    copyxnat copy --src-host "https://SOURCE-SERVER-URL" --src-user "XNAT-USER-SOURCE"  --project "PROJECT-NAME:NEW-PROJECT-NAME" --dst-host "https://DEST-SERVER-URL" --dst-user "XNAT-USER-DEST"
    
  • Duplicate a project on the same server (with a new name)

    copyxnat copy --src-host "https://SERVER-URL" --src-user "XNAT-USER-SOURCE"  --project "PROJECT-NAME:NEW-PROJECT-NAME" --dst-host "https://SERVER-URL" --dst-user "XNAT-USER-DEST"
    

Maintenance

  • Rebuild catalog for all experiments

      copyxnat rebuild-catalog -s "https://YOUR-SERVER-URL" -u "YOUR-USER-NAME"
    
  • Reset OHIF viewer for all experiments

      copyxnat ohif -s "https://YOUR-SERVER-URL" -u "YOUR-USER-NAME" 
    

Command-line parameters

The parameters vary depending which command you are running. To see help for a particular command, run with the command name and --help, for example:

copyxnat copy --help
Parameter Short form Default Description Notes
--help -h - Shows general help
command --help command -h - Shows detailed help for this command
--version - Shows version number
command - Command to run. One of copy, export, show, compare, check-datatypes, ohif, rebuild-catalog See above for examples of each command.
--src-host hostname -s - hostname of source XNAT server
--src-user username -u - XNAT username for source server
--project project_string -p all projects list of project IDs to process Use a colon if you want to rename project IDs between source and destination servers, eg srcId:dstId. To process multiple projects, separate the IDs with commas, eg proj1,proj2. Do not include any spaces. You can do both, e.g. prj1,prj2:dst2,prj3
--dst-host hostname -d - hostname of destination XNAT server Only for some commands
--dst-user username -w - XNAT username for destination server Only for some commands
--fix-scan-types -f False Fix undefined scan types on the copy If the source server does not correctly specify the scan type, CopyXnat will attempt to determine the scan type and set it on the destination server
--transfer-mode mode -t file Specifies how files will be copied or downloaded. One of file, zip, meta, rsync file to copy one file at a time. zip to zip up each resource collection as a zip file (generally faster). meta to transfer metadata only, not resource files. rsync to transfer files using rsync (experimental)
--limit-subjects limit -l no limit Process a maximum number of subjects specified by the integer limit. During a copy, the limit will exclude subjects which already exist on the destination and have not changed.
--skip-existing -n False Only copy new experiments; do not synchronise existing experiments Speeds up copying process when some experiments have already been copied to the destination server. However will not copy over new scans and resources that have been added to existing experiments.
--verbose -v False Output additional information For debugging and for providing additional information about what the command is doing.
--insecure -k False Do not verify server certificates Do not use in production as is vulnerable to MITM. For testing or working with self-signed certificates within a secure private network.
--dry-run -y False Do not make changes on the destination server Useful for seeing what changes a command may make before running it for real. May cause errors in some commands because the expected changes on the destination server didn't happen.
--overwrite-existing -o False Overwrite existing data on the destination server When copying, will modify and overwrite existing data
--ohif-rebuild -g False When copying, trigger an OHIF viewer rebuild after each experiment copy. The OHIF Viewer does not always detect new data being added to existing experiments. This will cause a session rebuild when data are copied, but this will take additional time.
--rsync-src-user -i - ssh source server username for copy via rsync Experimental. Only used for copy using --transfer-mode rsync
--rsync-dst-user -w - ssh destination server username for copy via rsync Experimental. Only used for copy using --transfer-mode rsync

Supported XNAT data levels

CopyXnat can process the following XNAT data hierarchies.

  • Projects
    • Project Resources
    • Subjects
      • Subject Resources
      • Experiments
        • Experiment Resources
        • Scans
          • Scan Resources
        • Assessors
          • Assessor In-Resources (copy not supported)
          • Assessor Out-Resources
        • Reconstructions (copy not supported)
          • Reconstruction In-Resources
          • Reconstruction Out-Resources

Current limitations

  • Reconstructions can be downloaded but cannot be copied to another server
  • Assessor In-Resources can be downloaded but cannot be copied to another server
  • Support for eCat files is limited and untested

Using CopyXnat in your own code

Importing via pip

You can import and run the module with pip and then import from your code

    from copyxnat.xnat.run_command import run_command
    from copyxnat.commands.show_command import ShowCommand

    result = run_command(command=ShowCommand,
                         src_host=src_host,
                         src_user=src_user,
                         src_pw=src_pw,
                         project_filter=None)

Local development: running the command-line utility directly

You can run directly from the source code. This is primarily useful if you are modifying the code.

   pip install -r requirements-dev.txt
   python copyxnat --help

Links

Authors

  • Tom Doel

Copyright

Copyright 2021 Tom Doel

License

This project is released under the MIT License. Please see the license file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

copyxnat-0.9.6-py2.py3-none-any.whl (62.9 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page