py4sci

Installation

Spring has been successfully run on Linux (CentOS6, RHEL6, Ubuntu, Fedora, OpenSuse, Debian) and MacOSX systems (Mountain Lion, Mavericks, Yosemite). We provide binary distributions of Spring for standard Linux and Mac systems. Check Prerequisites and read quick instructions below to Install binaries. For more time-consuming compilation from scratch, refer to section Install by compilation below.

Prerequisites

Linux

Please make sure the following packages are installed on your system:

Linux (Debian 7.9, 8.2) Linux (Ubuntu 12-17) Linux (OpenSuse13, Leap42.1) Linux (CentOS5-7) Linux (Fedora20) Linux (Fedora23)
root: apt-get install sudo apt-get install sudo zypper install sudo yum install sudo yum install sudo dnf install
  1. g++
  2. libfreetype6-dev
  3. libpng12-dev
  4. pkg-config
  1. g++
  2. libfreetype6-dev
  3. libpng-dev
  4. libgnome2-bin
  1. gcc
  2. gcc-c++ make
  3. freetype2 freetype2-devel
  4. libpng12-devel
  5. patch
  1. gcc
  2. gcc-c++
  3. freetype-devel
  4. libpng-devel
  1. gcc
  2. gcc-c++
  3. libfreetype2-devel.x86_64
  4. libpng-devel.x86_64
  5. patch.x86_64
  1. gcc
  2. gcc-c++
  3. libfreetype2-devel.x86_64
  4. libpng-devel.x86_64
  5. patch.x86_64

Mac

  • Requirements: Xcode including Python with X11
MacOSX (10.8/10.9/10.10/10.11)
  1. Xcode
  2. +command line tools
  3. X-Quartz

Launch the App Store and download Xcode. After it is installed, you need to install the Command Line Tools. This will install the default OS X Python binary.

  1. MacOSX 10.8: in Xcode’s preferences, go to the Downloads tab and install Command Line Tools.
  2. MacOSX 10.9/10.10/10.11:
% xcode-select --install

Wait until install completed. In order to activate Xcode developer tools, make sure to agree with the license:

% sudo xcodebuild -license

Acknowledge reading by typing agree after scrolling down.

Install the X11 window manager X-Qartz on your system.

Install binaries

Download Spring’s linux binary here, copy to desired location, unpack, launch quick install and you are done.

Linux

% tar xvf spring-0.86.1661-linux.tar.gz
% cd spring_v0-86-1661
% sh patches/binary_install_linux.sh

Mac

Download Spring’s mac binary here, copy to desired location using the Finder, launch quick install and you are done.

% cd spring_v0-86-1661
% sh patches/binary_install_mac.sh

Install by compilation

The scripts below are sufficient to build all components from scratch automatically. The execution of the recipe will take ~2 hours to build. The script will download the build environment hosted at bitbucket. In addition, the python code of Spring is regularly deposited at the pypi repository.

Linux

Launch the compile script below.

% sh build_linux.sh
#!/bin/sh
#author: Carsten Sachse
#date: 12-Jul-2017
#version: 0.3    
#usage: sh build_linux.sh
clear
echo '=================================================================================='
echo ' --------------------------- Running build_linux.sh ------------------------------'
echo 'This script will download, build and test Spring including required components on '
echo 'linux. Spring will be created in a new directory named spring_v(version number).'
echo '=================================================================================='

echo '----------------------------------------------------------------------------------'
echo 'Unset all existing library paths to avoid confusion during compilation.'
echo '----------------------------------------------------------------------------------'
export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

echo '----------------------------------------------------------------------------------'
echo 'Get Spring source and prepare directory.'
echo '----------------------------------------------------------------------------------'
STARTDIR=${PWD}
if [ -f ${STARTDIR}/bin/hg ] && [ -d ${STARTDIR}/.hg ]; then
	VERS=`cat ./doc/_static/version.txt`
	VERSION=`echo $VERS | sed "s/\./-/g"`
	SPRINGDIR=spring_v${VERSION}
	./bin/hg clone . ../${SPRINGDIR}
	cp buildout_linux.cfg buildout_mac.cfg ../${SPRINGDIR}
	cd ../${SPRINGDIR}
else
	wget https://bitbucket.org/csachse/emspring/get/tip.tar.gz
	BITDIR=`tar tzf tip.tar.gz  | sed -e 's@/.*@@' | uniq`
	tar -xvf tip.tar.gz && rm -f tip.tar.gz
	
	VERS=`cat ${BITDIR}/doc/_static/version.txt`
	VERSION=`echo $VERS | sed "s/\./-/g"`
	SPRINGDIR=spring_v${VERSION}
	mv ${BITDIR} ${SPRINGDIR}
	cd ${SPRINGDIR}
fi

echo '----------------------------------------------------------------------------------'
echo 'Get version number and define build spring directory.'
echo '----------------------------------------------------------------------------------'
echo '----------------------------------------------------------------------------------'
echo "Spring will be installed in ${SPRINGDIR}."
echo 'Get EMAN2 source. Only the provided EMAN2 version will work with Spring.'
echo '----------------------------------------------------------------------------------'
echo '----------------------------------------------------------------------------------'
echo 'Install EMAN2.'
echo '----------------------------------------------------------------------------------'
cd ../${SPRINGDIR}
mkdir fetch
mkdir parts
if [ -f ${STARTDIR}/bin/hg ] && [ -d ${STARTDIR}/.hg ]; then
	cp -pr ${STARTDIR}/parts/EMAN2 parts/
else
	cd fetch
	wget http://www.sachse.embl.de/emspring/_download/eman2.daily.linux64.gcc4.tar.bz2
	cd ..
	
	cd parts
	tar xjf ../fetch/eman2.daily.linux64.gcc4.tar.bz2
	cd ..
fi

echo '----------------------------------------------------------------------------------'
echo '1. Generate virtualenv within Spring directory.'
echo '----------------------------------------------------------------------------------'
PYTHONPATH=$PWD/lib/python2.7/site-packages LD_LIBRARY_PATH=$PWD/parts/EMAN2/lib \
 ./parts/EMAN2/Python/bin/python patches/virtualenv.py --no-site-packages .

cd patches; wget https://bootstrap.pypa.io/get-pip.py; cd ..

PYTHONPATH=${PWD}/lib/python2.7/site-packages LD_LIBRARY_PATH=${PWD}/parts/EMAN2/lib \
./bin/python patches/get-pip.py -U 

PYTHONPATH=${PWD}/lib/python2.7/site-packages LD_LIBRARY_PATH=${PWD}/parts/EMAN2/lib \
./bin/pip install -U pip setuptools 

PYTHONPATH=${PWD}/lib/python2.7/site-packages LD_LIBRARY_PATH=${PWD}/parts/EMAN2/lib \
./parts/EMAN2/Python/bin/python patches/virtualenv.py --no-site-packages .

echo '----------------------------------------------------------------------------------'
echo '2. Generate buildout environment for Spring.'
echo '----------------------------------------------------------------------------------'
cp buildout_linux.cfg buildout.cfg

PYTHONPATH=${PWD}/lib/python2.7/site-packages LD_LIBRARY_PATH=${PWD}/parts/EMAN2/lib \
./bin/python bootstrap.py

echo '----------------------------------------------------------------------------------'
echo '3. Build all components. This step may take 1 - 2 hours depending on your hardware.'
echo '----------------------------------------------------------------------------------'
PYTHONPATH=${PWD}/lib/python2.7/site-packages LD_LIBRARY_PATH=${PWD}/parts/EMAN2/lib \
./bin/buildout -v 2>&1 | tee build.log

echo '--------------------------------------------------------------------------------'
echo '4. Install latest Python dependencies for Spring'
echo '--------------------------------------------------------------------------------'
PYTHONPATH=${PWD}/lib/python2.7/site-packages LD_LIBRARY_PATH=${PWD}/parts/EMAN2/lib \
./bin/buildout install springpython

if [ ! -d ~/.python-eggs ]; then
	mkdir ~/.python-eggs
	chmod og-w ~/.python-eggs
fi

echo '--------------------------------------------------------------------------------'
echo '5. Copy libpng12 to local folders as latest linux releases do not support it.'
echo '--------------------------------------------------------------------------------'
cd ${PWD}/parts/libpng/lib
cp -p /usr/lib64/libpng12.so.0.49.0 .
ln -s libpng12.so.0.49.0 libpng12.so.0
cd ../../..

echo '----------------------------------------------------------------------------------'
echo 'Install done. Check whether Spring was successfully installed by testing.' 
echo 'The tests will run for approx. 15 min depending on your hardware.'
echo '----------------------------------------------------------------------------------'
EMSPRING_DIR=`find lib/python2.7/site-packages -name "*emspring*"`
for each_test in ${EMSPRING_DIR}/spring/tests/segment2d/test*.py;
	do ./bin/nosetests --nocapture --nologcapture --with-outputsave \
		--save-directory=nose_results ${each_test};
done

echo '=================================================================================='
echo 'Testing done. Check in nose_results directory whether ~80 tests ran successfully. '
echo 'If this is the case Spring was correctly built and installed.' 
echo '=================================================================================='

exit 0

Mac

*Update: In order to compile CFFIND4, wxWidget and fftw3 are expected to be installed in /opt/local. The easiest way to install them is using macports:

% sudo port install wxwidgets-3.2 fftw-3

Launch the compile script below.

% sh build_mac.sh
#!/bin/sh
#author: Carsten Sachse
#date: 12-Jul-2017
#version: 0.3
#usage: sh build_mac.sh
clear
echo '=================================================================================='
echo ' ----------------------------- Running build_mac.sh ------------------------------'
echo 'This script will download, build and test Spring including required components on '
echo 'a Mac. Spring will be created in a new directory named spring_v(version number).'
echo '=================================================================================='

echo '----------------------------------------------------------------------------------'
echo 'Unset all existing library paths to avoid confusion during compilation.'
echo '----------------------------------------------------------------------------------'
unset MPIHOME MPIBIN LIBRARY_PATH LD_LIBRARY_PATH CPATH LD_RUN_PATH DYLD_LIBRARY_PATH PYTHONPATH
export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin

echo '----------------------------------------------------------------------------------'
echo 'Get Spring source and prepare directory.'
echo '----------------------------------------------------------------------------------'
STARTDIR=${PWD}
if [ -f ${STARTDIR}/bin/hg ] && [ -d ${STARTDIR}/.hg ]; then
	VERS=`cat ./doc/_static/version.txt`
	VERSION=`echo $VERS | sed "s/\./-/g"`
	SPRINGDIR=spring_v${VERSION}
	./bin/hg clone . ../${SPRINGDIR}
	cp buildout_linux.cfg buildout_mac.cfg ../${SPRINGDIR}
	cd ../${SPRINGDIR}
else
	curl --remote-name https://bitbucket.org/csachse/emspring/get/tip.tar.gz
	BITDIR=`tar tzf tip.tar.gz  | sed -e 's@/.*@@' | uniq`
	tar -xvf tip.tar.gz && rm -f tip.tar.gz
	
	VERS=`cat ${BITDIR}/doc/_static/version.txt`
	VERSION=`echo $VERS | sed "s/\./-/g"`
	SPRINGDIR=spring_v${VERSION}
	mv ${BITDIR} ${SPRINGDIR}
	cd ${SPRINGDIR}
fi

echo '----------------------------------------------------------------------------------'
echo 'Get version number and define build spring directory.'
echo '----------------------------------------------------------------------------------'
echo '----------------------------------------------------------------------------------'
echo "Spring will be installed in ${SPRINGDIR}."
echo 'Get EMAN2 source. Only the provided EMAN2 version will work with Spring.'
echo '----------------------------------------------------------------------------------'
echo '----------------------------------------------------------------------------------'
echo 'Install EMAN2.'
echo '----------------------------------------------------------------------------------'
cd ../${SPRINGDIR}
mkdir fetch
mkdir parts
if [ -f ${STARTDIR}/bin/hg ] && [ -d ${STARTDIR}/.hg ]; then
	cp -pr ${STARTDIR}/parts/EMAN2 parts/
else
	cd fetch
	curl --remote-name http://www.sachse.embl.de/emspring/_download/eman2.daily.lion.dmg
	cd ..
	
	hdiutil mount fetch/eman2.daily.lion.dmg  
	cp -pr /Volumes/EMAN2\ daily\ for\ Mac\ OS\ X\ lion,\ built\ on\ 2013-08-05/EMAN2 parts
	hdiutil unmount /Volumes/EMAN2\ daily\ for\ Mac\ OS\ X\ lion,\ built\ on\ 2013-08-05/EMAN2 
fi

echo '----------------------------------------------------------------------------------'
echo '1. Generate virtualenv within Spring directory.'
echo '----------------------------------------------------------------------------------'
PYTHONPATH=${PWD}/lib/python2.7/site-packages /usr/bin/python patches/virtualenv.py \
--no-site-packages .

cd patches; curl --remote-name https://bootstrap.pypa.io/get-pip.py; cd ..

PYTHONPATH=${PWD}/lib/python2.7/site-packages LD_LIBRARY_PATH=${PWD}/parts/EMAN2/lib \
./bin/python patches/get-pip.py --target=${PWD}/lib/python2.7/site-packages

PYTHONPATH=${PWD}/lib/python2.7/site-packages /usr/bin/python patches/virtualenv.py \
--no-site-packages .

echo '----------------------------------------------------------------------------------'
echo '2. Generate buildout environment for Spring.'
echo '----------------------------------------------------------------------------------'
cp buildout_mac.cfg buildout.cfg
./bin/python bootstrap.py

echo '----------------------------------------------------------------------------------'
echo '3. Build all components. This step may take 1 - 2 hours depending on your hardware.'
echo '----------------------------------------------------------------------------------'
# 16-Oct 2017 addition required as python's urllib2 does not meet the certification of the openmpi server
# avoiding SSLError: sslv3 alert handshake failure
cd fetch; curl --remote-name https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.1.tar.gz; cd ..
./bin/buildout -v install \
    fetch \
    gcc \
    blas \
    lapack \
    numpy \
    scipy \
    springpython \
    pkg-config \
    libpng \
    matplotlib \
    openmpi \
    mympi_patch \
    mympi \
    mpi4py 2>&1 | tee build.log

PATH=/opt/local/bin:${PATH} ./bin/buildout -v install \
    ctf \
    ctffind_ctftilt \
    cleanup_build 
echo '--------------------------------------------------------------------------------'
echo '4. Install latest Python dependencies for Spring'
echo '--------------------------------------------------------------------------------'
./bin/buildout install springpython

echo '----------------------------------------------------------------------------------'
echo 'Install done. Check whether Spring was successfully installed by testing.' 
echo 'The tests will run for approx. 15 min depending on your hardware.'
echo '----------------------------------------------------------------------------------'
EMSPRING_DIR=`find lib/python2.7/site-packages -name "*emspring*"`
for each_test in ${EMSPRING_DIR}/spring/tests/segment2d/test*.py;
	do ./bin/nosetests --nocapture --nologcapture --with-outputsave \
		--save-directory=nose_results ${each_test};
done

echo '=================================================================================='
echo 'Testing done. Check in nose_results directory whether ~80 tests ran successfully. '
echo 'If this is the case Spring was correctly built and installed.' 
echo '=================================================================================='

exit 0

Activation

You can now execute any of the programs from the spring package by typing the absolute path, e.g.:

% /Applications/spring_v0-86-1661/bin/spring

You can add the absolute spring_v0-86-1661/bin directory to your path by adding the following line into your .bashrc file, e.g.:

% export PATH:/Applications/spring_v0-86-1661/bin:$PATH

or .cshrc file, e.g.:

% setenv PATH:/Applications/spring_v0-86-1661/bin:$PATH

Update

1. As of Spring v83.1432.dev-0 we updated a number of supporting libraries. Therefore, we recommend a clean install using the abovementioned methods (Install binaries or Install by compilation).

2. For future versions from v83.1432.dev-0 you can quickly update Spring automatically within seconds without rebuilding the entire dependencies from scratch by running your buildout script. The buildout script will fetch the latest Spring python code from the pypi repository and install it.

Re-run install springpython.

% ./bin/buildout install springpython

Known Issues

On Ubuntu 14.04, Debian 8.2 users have reported GUI issues due to a conflict with the Gtk+ GUI style. The solution is to change the default GUI style:

% sudo apt-get install qt4-qtconfig
% qtconfig

A Window opens. ’Select GUI style’ from ‘Gtk+’ to ‘Cleanlooks’. Reboot and the Spring GUI will open without complaints.

Changelog

0.86.1661 (2017-12-12)

  • MICHELIXTRACE: release of algorithm as published in Huber et al. 2017 including statistical thresholding, search for optimal parameter combination and persistence length pruning
  • Web-based Helical Diffraction Simulator on http://spring.embl.de based on SEGLAYER2LATTICE.

0.85.1617 (2017-07-12)

  • MICCTFDETERMINE: updated to work with ctffind4.
  • SEGCLASSRECONSTRUCT: updated to work with binning option.
  • SEGMENTCLASS: updates to work with larger datasets for classification to reduce memory issues.
  • SEGCLASSLAYER: addition of ‘B-factor’ option to adjust high-resolution layer-line contrast.
  • SEGREFINE3DINSPECT: Bfactor/Resolution cutoff option separated from long helix option.
  • MICHELIXTRACE: Algorithm update to increase robustness of filament tracing.
  • SEGREFINE3DGRID/SEGGRIDEXPLORE: Now with out-of-plane deviation parameter.
  • SEGMENTPLOT/SEGREFINE3DPLOT: Now plots quantities on micrograph if subset micrograph chosen.
  • Updated to work with SLURM cluster

0.84.1470 (2016-01-04)

General bugfixes and improvement in stability. Addition of explanatory tool tips.

  1. SEGCLASSEXAM
  • Multiple bugfixes including wrong computation of pixelsize.
  1. SEGMENTREFINE3D
  • Addition of new parameter “Choose out-of-plane tilt for amplitude correlation.”
  1. Installation procedures
  • Updated dependencies GCC, matplotlib, libpng including build procedures.

0.83.1449 (2015-1-26)

  • Fixes in SEGMENTREFINE3D procedure, now also works for non-zero out-of-plane tilts.

0.83.1432 (2014-12-30)

  • Fixes and improvements
  1. Installation procedures
  • Updated dependencies GCC, Numpy, Scipy, Openmpi including build procedures.
  1. SEGMENTREFINE3D
  • Computation of persistence length at the end of each refinement run. Can be used now as a selection criterion for discarding less straight helices in the next refinement run.
  • Selection criteria: Straight helices, Layer-line correlation and Projection matching cross-correlation are now selected in percent range of the distribution instead of cutoff values, e.g. upper 20 % –> 80 - 100 %.
  • ‘Continue refinement option’ - in case no reference structure is given a 3D reconstruction based on the provided parameters is now created for further refinement.
  • Improved segment-based motion correction, now accepts mrcs-stacks.
  • Computation of structure-masked FSC with mask deconvolution according to Ultramicroscopy Chen et al. 2013 (Scheres and Henderson) for resolutions better than 12 Angstrom.
  • Added error estimates for forward x-shift, forward out-of-plane tilt and in-plane rotation angles according to Sachse et al, 2007 J Mol Biol.
  • Overhaul in symmetrization of volumes to make them perfectly even.
  • Diagnostic power spectra are now always written out as EMAN2 image files in addition to diagnostic summary.
  1. High-performance computing cluster
  • Added support for SLURM.
  1. SEGMENTCLASS, SEGLAYERLATTICE
  • Many minor fixes.

Note: old refinement.dbs and grid.dbs need to be upgraded to be readable by the latest Spring version.

% spring -udb='grid.db' -inp grid.db -out grid_upd.db
% spring -udb=refinement.db -inp=refinement024.db -out=refinement024_up.db

0.82.1339 (2014-09-15)

  • Improved installation procedures including build and binary install scripts.

0.82.1339 (2014-04-25)

  • General fixes, optimizations and enhancements
  1. SEGMENTREFINE3D
  • Fixes for data sets with large number of asymmetric units
  • Fix/workaround for occasional database lock problems on cluster mounted nodes
  • Fix in ‘Absolute limit of x and y-shifts’ handling. Now properly respects this.
  • Declared some options as experimental: support remains limited.
  • Occasional over-estimation of FSC fixed due to densities at upper and lower ends of filament
  • Implementation of ‘independent half-set’ refinement (a.k.a. gold-standard refinement)

Note: the update will break reading your previous refinement.db files Database files can be updated as follows:

% spring -udb=refinement.db -inp=refinement024.db -out=refinement024_up.db
  1. SEGREFINE3DINSPECT
  • Additional option of signal-to-noise weighting using FSC file.
  1. SEGLAYER2LATTICE
  • Simulation of layer-line pattern takes rotational symmetry into account

0.81.1282 (2014-01-26)

  • Fixes, enhancements and optimization
  1. SEGMENTREFINE3D
  • Disk requirements for temporary directories revised.
  • Improved handling of selected segments. Spring processes helices as one entity and discards them later for 3D reconstruction.
  • FSC only computed with cylinder mask.
  • Improved experimental power spectra for high/maximum resolution analysis.
  1. SEGMENT
  • Fix in frame processing.
  • Rotated stack only written if requested.
  1. SEGMENTEXAM
  • Addition of selection options from spring.db.
  1. SEGCLASSEXAM
  • Addition of mpi option.
  1. SEGLAYER2LATTICE
  • Addition of tooltips of predicted Bessel orders.
  1. SEGCLASSLAYER
  • Accepts also power spectra as input.
  • Added tooltips on Bessel look up table.
  1. General
  • Parameter input from prompt now works using Tab auto completion including file search.