Page tree

cisTEM is an image processing framework.

Homepage: https://cistem.org/

Loading the environment

cisTEM can be used through environment modules:

module load cistem/1.0-beta

CSSB Notes

Getting started

By default cisTEM suggests your home directory for project setup. Since it will contain all analysis files use your BeeGFS space as project directory.

Start the user interface by calling cisTEM on the command line.

Importing TIFF files

With the K3 camera the movies are most likely in LZW compressed TIFF format to save storage space and the file suffix is *.tiff.

cisTEM expects the file pattern *.tif for the movies import. So we make an import folder, link the original files and rename the links to this convention.

mkdir Import; cd Import
cp -s /whereever/*.tiff .
for i in $(/usr/bin/ls -1 *.tiff); do mv $i ${i%.tiff}.tif; done

Now you can import the files in the cisTEM GUI.

Submitting to the DESY maxwell cluster

Now import the DESY/CSSB cluster profile. Go to SettingsImport and choose the following file:

/beegfs/cssb/software/etc/em/cistem/1.0-beta/maxwell_slurm.txt

You should now have a run profile called Maxwell Cluster which you can choose in your Actions.

The cluster usage works in 3 steps:

  1. The cisTEM user interface allocates cluster resources using salloc and runs the cisTEM manager on the first node
  2. The manager spawns its worker tasks by using srun inside the allocated resources
  3. When the manager is finished the maxwell allocation will be canceled automatically.

Changing the preset submission parameters

First look for free resources with the sinfo_detailed alias:

$ sinfo_detailed
NODELIST           PARTITION CPUS NODES NODES(A/I) AVAIL_FEATURES
max-cssb[002-010]  cssb      48   9     3/5        INTEL,Gold-6126,1536G
max-cssbg[001-003] cssbgpu   40   3     3/0        INTEL,V4,E5-2640,GPU,P100,GPUx2,512G
:
max-xxxx[101-188]  all       72   88    12/70      INTEL,Gold-6140,768G
max-xxxx[020-099]  all       80   80    28/52      INTEL,V4,E5-2698,512G
:

Now you see that 5 of the CSSB nodes with each 48 cores are idle (NODES(A/I)) and can be used.

In cisTEM navigate to SettingsRun ProfilesMaxwell Cluster and change the Manager Command like (leave the rest of the line as it is):

salloc -n 240 -p cssb
salloc -n 288 -p all -C INTEL&Gold-6140&768G
salloc -n 320 -p all -C INTEL&V4&E5-2698&512G

-n: the amount of cores e.g. 5*48 = 240

-p: the maxwell queue/partition name

-C: available features like a specific processor for correct amount of cores or a specific memory. In case of doubt just take the whole feature string.

Then edit the No. Copies of the srun command to 239 (240 - 1 manager process) and do not forget to press the Save button.

Now the updated profile will be used to start new actions.

You can use the Run ProfilesDuplicate button to create a list of your favorite submission configurations.

Running cisTEM on a dedicated cluster node

The actions Align Movie and Find CTF execute special external programs which have connections problems on when submitting the jobs to the cluster. So its best to run these steps locally on one dedicated node. Compared to the other actions they are quite fast.

The following commands allocates a 48 core machine which will be referred inside cisTEM as Run Profile: Default Local (48).

$ salloc_cssb_cpu_node 
salloc: Granted job allocation 2310829
salloc: Waiting for resource configuration
salloc: Nodes max-cssb006 are ready for job

$ ssh max-cssb006

$ module load cistem

$ cisTEM
:

Once you are done quit cisTEM and free the resources by exiting the shell twice (one for exiting the node, the second for exiting the allocation):

$ exit
Connection to max-cssb006 closed.

$ exit
salloc: Relinquishing job allocation 2310829
$

Or you can cancel your allocation by:

$ scancel 2310829

Note:

  • salloc_cssb_cpu_node is an alias defined in the global environment

Exporting and manipulating the cisTEM database

The main cisTEM database can exported, backuped etc. via SQLite.

$ sqlite3 CisTEM_TestSetProject.db
sqlite> .output dump.sql
sqlite> .dump
sqlite> .quit
gedit dump.sql                                                    # now do search and replace text finally save it
mv CisTEM_TestSetProject.db CisTEM_TestSetProject_original.db     # backup original database
sqlite3 CisTEM_TestSetProject.db <dump.sql                        # create a new database with the content from the modified file
  • No labels