• Prerequisites
    • Obtaining a User Interface Account
    • Obtaining a certificate
    • Loading your certificate into the browser
    • Registering with the Belle Virtual Organisation
    • Registering with the Belle2.org Virtual Organisation
    • If you have time left
  • Authentication and Authorisation
    • Inspecting personal certificate
    • Creation of a proxy with voms extensions
    • Check your voms proxy
    • Further information and references
  • Information System
    • lcg-infosites
      • USAGE
      • lcg-infosites options
    • Practicals
    • lcg-info
      • USAGE
      • lcg-info options
    • Practicals
      • If you have time left
  • Job Management
    • Login to User Interface
    • Create VOMS proxy
    • Simple JDL
    • Credentials Delegation
    • Job List Match
    • Job Submission
    • Job Status
    • Job Logging
    • Job Output
    • Job cancel
  • Data Management
    • LFC and LCG commands
      • Listing file and directory
      • Create directory
      • Upload file into SE
      • Get the file SURL
      • Replicate file between SE
      • Create symbolic link
      • Download a file from SE to UI
      • Clean all !
    • If you still have time
  • Metadata
    • Setup the .mdclient.config configuration file
    • Connect to the AMGA server
    • General Operations on AMGA
    • Let us start with an example
    • Exercises
    • If you still have time
  • Advanced Job Management
    • Objective
    • MyProxy Server for Long-running job submission
    • Advanced Sandbox Management
    • Real Time Output Perusal
    • JDL Description
    • Credentials Delegation
    • Job List Match
    • Job Submission
    • Job Status
    • Job Real Time Output Perusal
    • Job Logging
    • Job Output
    • Job cancel
    • If you still have time

Prerequisites

Obtaining a User Interface Account

Information on User Interface Accounts will be provided on the first day.

Otherwise, if you have a grid site at your institute, please contact its administrators if you wish.

 

Obtaining a certificate

You need to obtain a grid certificate from your local CA

Loading your certificate into the browser


1) In your home directory, make a directory .globus
2) Copy the public part of your certificate into ~/.globus/usercert.pem
3) Find the private part of the certificate (ususally userkey.pem) and cp into ~/.globus/userkey.pem
4) Use OpenSSL to make a .p12 file so we can load it into your browser:
openssl pkcs12 -export -in ~/.globus/usercert.pem -inkey ~/.globus/userkey.pem -out ~/usercert.p12
5) Load Firefox, Preferences --> Encryption --> View Certificates --> Your Certificate --> Import and select your P12
[You may also want to tick ("Select One Automatically") under Preferences -> Encryption]

Registering with the Belle Virtual Organisation

  1. Visit https://vomrs.cc.kek.jp:8443/vo/belle/vomrs
  2. Click Registration (Phase I) and fill out the form
  3. You willl receive an email - click the confirmation link
  4. Fill out and submit the Phase II form
  5. Your request will be approved by an administrator shortly & you will recieve an email

Registering with the Belle2.org Virtual Organisation

  1. Visit https://vomrs.cc.kek.jp:8443/vo/belle2/vomrs
  2. Click Registration (Phase I) and fill out the form
  3. You willl receive an email - click the confirmation link
  4. Fill out and submit the Phase II form
  5. Your request will be approved by an administrator shortly & you will recieve an email

If you have time left

  • You might like to have a look around the other pages on the VOMS server - or view examples at Gilda

Authentication and Authorisation

 

Inspecting personal certificate

Your personal certificate is split in two separate files in a directory called .globus. These files are effectively your public and private keys, which will be used for the authenticated connections with all the other grid elements. It is essential that they have the correct file permissions otherwise you won't be able to create a proxy. Check the permissions with the command

ls -l .globus

which should give you the following output:

 

# ls -l .globus
total 8
-rw-r--r--    1 user  users        1613 Jun  7 15:16 usercert.pem
-r--------    1 user  users        1914 Jun  7 15:16 userkey.pem

You can have now a look inside your certificate with

grid-cert-info

 

# grid-cert-info
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 3901 (0xf3d)
        Signature Algorithm: md5WithRSAEncryption
        Issuer: C=IT, O=GILDA, CN=GILDA Certification Authority
        Validity
            Not Before: Apr 12 08:32:31 2006 GMT
            Not After : Apr 12 08:32:31 2007 GMT
        Subject: C=IT, O=GILDA, OU=Personal Certificate, L=INFN, CN=Emidio Giorgio/Email=emidio.giorgio@ct.infn.it
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (2048 bit)
                Modulus (2048 bit):
                    00:c7:c9:9a:ae:81:e5:0f:03:c9:f6:0f:5b:3c:c5:
                    [cut...] 
                    9c:54:93:ae:b4:f2:5c:fc:62:f7:ff:f8:50:fc:c7:
                    00:4b
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            Netscape Comment:
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier:
                41:8D:0A:A0:6D:3E:24:00:9E:98:FA:3B:EF:6F:40:DC:CB:86:81:0B
            X509v3 Authority Key Identifier:
                keyid:F2:4E:BA:9C:42:04:41:7F:A0:0A:AA:35:D0:98:93:12:37:EF:8C:76
                DirName:/C=IT/O=GILDA/CN=GILDA Certification Authority
                serial:00

    Signature Algorithm: md5WithRSAEncryption
        91:b1:2b:45:40:74:b1:0a:ba:d5:d2:fb:ad:32:07:9f:a9:3b:
        [cut...]        
        dd:4f

Worth noting things are certificate creation and expiration date, the name and subject of the Certification Authority which issued the certificate, the Common Name (CN) of the certificate owner, and the certificate subject, which uniquely identifies the certificate owner.

 

Creation of a proxy with voms extensions

This step is comparable to a login on the Grid, because without it you can do very little. The command to do it is voms-proxy-init --voms VO-NAME Then, for instance, if your VO is belle2, you have to type

voms-proxy-init --voms belle

and if everything is ok you should have

 

# voms-proxy-init --voms belle
Your identity: /C=AU/O=APACGrid/OU=The University of Melbourne/CN=Tom Fifield
Creating temporary proxy ............................................................................................ Done
Contacting voms.cc.kek.jp:15020 [/C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp] "belle" Done
Creating proxy ....................... Done
Your proxy is valid until Mon Mar 15 13:24:10 2010

 

Check your voms proxy

Once that your proxy has been created, you can gather info on it through the voms-proxy-info command. It is much more useful if ran with the -all option, because it will show also the VO related infos added by the VOMS server, such as belonging groups or roles. You may note also two different lifetimes : first is related to the proxy itself, the second one is referred to the AC infos added by the VOMS server. They have to be valid both in order to be fully enabled to perform operations.

# voms-proxy-info --all
subject  : /C=AU/O=APACGrid/OU=The University of Melbourne/CN=Tom Fifield/CN=proxy
issuer   : /C=AU/O=APACGrid/OU=The University of Melbourne/CN=Tom Fifield
identity : /C=AU/O=APACGrid/OU=The University of Melbourne/CN=Tom Fifield
type     : proxy
strength : 1024 bits
path     : /tmp/x509up_u1056
timeleft : 11:59:15
=== VO belle extension information ===
VO       : belle
subject  : /C=AU/O=APACGrid/OU=The University of Melbourne/CN=Tom Fifield
issuer   : /C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp
attribute : /belle/Role=NULL/Capability=NULL
timeleft : 11:59:15
uri      : voms.cc.kek.jp:15020

 

Further information and references

A Brief Guide to Certificate Management by Marisa Luvisetto

Information System

lcg-infosites

The lcg-infosites command can be used as an easy way to retrieve information on Grid resources for most use cases.

This command is actually just a perl script wrapping a series of LDAP commands and was developed to allow the user to retrieve information on Grid resources for the most common cases.

Before beginning it is worth observing that "lcg-infosites" does not use your VOMS proxy certificate and hence all commands need to include the option "--vo belle".

USAGE

lcg-infosites --vo <vo name> options -v <verbose level> --is <BDII to query>

lcg-infosites options

Practicals

Start by getting a list of the compute elements available in the belle VO and the state of the associated CPU's:

# lcg-infosites --vo belle ce
#CPU   Free   Total Jobs   Running   Waiting   ComputingElement
----------------------------------------------------------
 192     2     0            0     0   agh5.atlas.unimelb.edu.au:8443/cream-pbs-belle
 192     2     0            0     0   agh2.atlas.unimelb.edu.au:2119/jobmanager-lcgpbs-belle
 288   279     0            0     0   ce02.sdfarm.kr:2119/jobmanager-lcgpbs-belle
 376   374     0            0     0   kek2-ce04.cc.kek.jp:2119/jobmanager-lcgpbs-belle
...

A different set of data may be obtained by appending the option "-v 2" (nothing to do with verbose) to the previous command as is shown below

# lcg-infosites --vo belle ce -v 2
AMMemory   Operating System   System Version             Processor   Subcluster name
-------------------------------------------------------------------------------------------------------------------------
  8117            ScientificSL   Boron                                  Xeon5450            agh5.atlas.unimelb.edu.au
  8117            ScientificSL   Boron                                  Xeon5450            agh2.atlas.unimelb.edu.au
 16384            ScientificSL   Boron                                      Xeon                       ce02.sdfarm.kr
  4096            ScientificSL   Boron                                      Xeon                  kek2-ce04.cc.kek.jp


Similarly information about the storage elements may be obtained with the command

# lcg-infosites --vo belle se
Avail Space(Kb) Used Space(Kb) Type   SEs
----------------------------------------------------------
1910857575     5082997543     n.a   dg24.cc.kek.jp
1912791        200236         n.a   se0.sdfarm.kr
3830037086     4673998159     n.a   kek2-se.cc.kek.jp

It is also possible to obtain which SE is closest to which CE. The term "close" in grids can have several interpretations including physical distance apart and speed of data access (available bandwidth). The closest SE to a CE is defined by the manager of the CE. To see the closet SE to each CE use the command:

[ui01] /home/wlueng > lcg-infosites --vo belle closeSE
Name of the CE: agh5.atlas.unimelb.edu.au:8443/cream-pbs-belle
   agh3.atlas.unimelb.edu.au

Name of the CE: agh2.atlas.unimelb.edu.au:2119/jobmanager-lcgpbs-belle
   agh3.atlas.unimelb.edu.au

Name of the CE: ce02.sdfarm.kr:2119/jobmanager-lcgpbs-belle
   se0.sdfarm.kr

Name of the CE: kek2-ce04.cc.kek.jp:2119/jobmanager-lcgpbs-belle
   kek2-se.cc.kek.jp

 

lcg-info

The lcg-info command can list ce or se according to more complicated conditions.

This command is similar to the "lcg-infosites" except that it is used to list either CE's or SE;s satisfying a given set of conditions on their attributes and to print, for each of them, the values of a given set of attributes. This is very similar to the usage of the "Requirements" tag in a JDL file along with the command "glite-job-list-match". The "lcg-info" command can therefore be useful when constructing the "Requirements" tag in a JDL file. Note that "lcg-info" does not in most (but not all) cases need a VO to be specified.

USAGE

lcg-info [--list-ce | --list-se] [--query <query>] [--attrs <attrs>]

 

lcg-info options

Practicals

The first step is to view the list of attributes available for both the matching and printing aspects of "lcg-info". Run the command:

# lcg-info --list-attrs
Attribute name      Glue object class     Glue attribute name           
WorstRespTime       GlueCE                GlueCEStateWorstResponseTime  
CEAppDir            GlueCE                GlueCEInfoApplicationDir      
TotalCPUs           GlueCE                GlueCEInfoTotalCPUs           
MaxRunningJobs      GlueCE                GlueCEPolicyMaxRunningJobs    
CE                  GlueCE                GlueCEUniqueID                
WaitingJobs         GlueCE                GlueCEStateWaitingJobs        
MaxCPUTime          GlueCE                GlueCEPolicyMaxCPUTime        
LRMSVersion         GlueCE                GlueCEInfoLRMSVersion         
MaxTotalJobs        GlueCE                GlueCEPolicyMaxTotalJobs      
CEStatus            GlueCE                GlueCEStateStatus             
LRMS                GlueCE                GlueCEInfoLRMSType            
CEVOs               GlueCE                GlueCEAccessControlBaseRule   
AssignedJobSlots    GlueCE                GlueCEPolicyAssignedJobSlots  
FreeCPUs            GlueCE                GlueCEStateFreeCPUs           
RunningJobs         GlueCE                GlueCEStateRunningJobs        
EstRespTime         GlueCE                GlueCEStateEstimatedResponseTime
FreeJobSlots        GlueCE                GlueCEStateFreeJobSlots       
Cluster             GlueCE                GlueCEInfoHostName            
TotalJobs           GlueCE                GlueCEStateTotalJobs          
Priority            GlueCE                GlueCEPolicyPriority          
CEDefaultSE         GlueCE                GlueCEInfoDefaultSE           
JobManager          GlueCE                GlueCEInfoJobManager          
MaxWCTime           GlueCE                GlueCEPolicyMaxWallClockTime  
Accesspoint         GlueCESEBind          GlueCESEBindCEAccesspoint     
CloseCE             GlueCESEBindGroup     GlueCESEBindGroupCEUniqueID   
CloseSE             GlueCESEBindGroup     GlueCESEBindGroupSEUniqueID   
Root                GlueSA                GlueSARoot                    
AvailableSpace      GlueSA                GlueSAStateAvailableSpace     
Path                GlueSA                GlueSAPath                    
SEVOs               GlueSA                GlueSAAccessControlBaseRule   
UsedSpace           GlueSA                GlueSAStateUsedSpace          
SESite              GlueSE                GlueForeignKey                
SEType              GlueSE                GlueSEType                    
SEArch              GlueSE                GlueSEArchitecture            
SEName              GlueSE                GlueSEName                    
SE                  GlueSE                GlueSEUniqueID                
SEPort              GlueSE                GlueSEPort                    
Protocol            GlueSEAccessProtocol  GlueSEAccessProtocolType      
ArchType            GlueSL                GlueSLArchitectureType        
ServiceID           GlueService           GlueServiceUniqueID           
ServiceOwner        GlueService           GlueServiceOwner              
ServiceWSDL         GlueService           GlueServiceWSDL               
ServiceName         GlueService           GlueServiceName               
ServiceVersion      GlueService           GlueServiceVersion            
ServiceEndpoint     GlueService           GlueServiceEndpoint           
ServiceSite         GlueService           GlueForeignKey                
ServiceStatusInfo   GlueService           GlueServiceStatusInfo         
ServiceVOs          GlueService           GlueServiceAccessControlRule  
ServiceAccesspoint  GlueService           GlueServiceAccessPointURL     
ServiceURI          GlueService           GlueServiceURI                
ServiceType         GlueService           GlueServiceType               
ServiceStatus       GlueService           GlueServiceStatus             
SiteSupport         GlueSite              GlueSiteUserSupportContact    
SiteName            GlueSite              GlueSiteName                  
SiteAdmin           GlueSite              GlueSiteSysAdminContact       
SiteDesc            GlueSite              GlueSiteDescription           
SiteLocation        GlueSite              GlueSiteLocation              
SiteSecurity        GlueSite              GlueSiteSecurityContact       
SiteInfo            GlueSite              GlueSiteOtherInfo             
SiteID              GlueSite              GlueSiteUniqueID              
SMPSize             GlueSubCluster        GlueHostArchitectureSMPSize   
Processor           GlueSubCluster        GlueHostProcessorModel        
PlatformArch        GlueSubCluster        GlueHostArchitecturePlatformType
OSVersion           GlueSubCluster        GlueHostOperatingSystemVersion
VMemory             GlueSubCluster        GlueHostMainMemoryVirtualSize 
CFP2000             GlueSubCluster        GlueHostBenchmarkSF00         
TmpDir              GlueSubCluster        GlueSubClusterTmpDir          
OSRelease           GlueSubCluster        GlueHostOperatingSystemRelease
ClockSpeed          GlueSubCluster        GlueHostProcessorClockSpeed   
LogicalCPU          GlueSubCluster        GlueSubClusterLogicalCPUs     
OutboundIP          GlueSubCluster        GlueHostNetworkAdapterOutboundIP
OS                  GlueSubCluster        GlueHostOperatingSystemName   
Tag                 GlueSubCluster        GlueHostApplicationSoftwareRunTimeEnvironment
CINT2000            GlueSubCluster        GlueHostBenchmarkSI00         
WNTmpDir            GlueSubCluster        GlueSubClusterWNTmpDir        
CPUVendor           GlueSubCluster        GlueHostProcessorVendor       
PhysicalCPU         GlueSubCluster        GlueSubClusterPhysicalCPUs    
InboundIP           GlueSubCluster        GlueHostNetworkAdapterInboundIP
Memory              GlueSubCluster        GlueHostMainMemoryRAMSize     
VORunningJobs       GlueVOView            GlueCEStateRunningJobs        
VOTotalJobs         GlueVOView            GlueCEStateTotalJobs          
VOWorstRespTime     GlueVOView            GlueCEStateWorstResponseTime  
VOEstRespTime       GlueVOView            GlueCEStateEstimatedResponseTime
VOCEVOs             GlueVOView            GlueCEAccessControlBaseRule   
VOFreeJobSlots      GlueVOView            GlueCEStateFreeJobSlots       
VOWaitingJobs       GlueVOView            GlueCEStateWaitingJobs        

One usage of "lcg-info" is to see which software packages are available on which CE's. The software packages are stored in the "Tag" attribute and can be seen by typing:

# lcg-info --vo belle --list-ce --attrs Tag
- CE: ce.euag.org:2119/jobmanager-lcgpbs-gilda
  - Tag                 GLITE-3_0_0
                        GLITE-3_1_0                   
                        LCG-2                         
                        LCG-2_1_0                     
                        LCG-2_1_1                     
                        LCG-2_2_0                     
                        LCG-2_3_0                     
                        LCG-2_3_1                     
                        LCG-2_4_0                     
                        LCG-2_5_0                     
                        LCG-2_6_0                     
                        LCG-2_7_0                     
                        R-GMA                         

- CE: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda
  - Tag                 POVRAY-3.5
                        DEMTOOLS-1.0                  
                        CSOUND-4.13                   
                        GEANT4-6                      
                        VLC-0.7.2                     
                        EGEODE-1.0                    
                        RASTER3D                      
                        G95-3.5.0                     
                        MAGIC-6.19                    
                        CODESA3D-1.0                  
                        CYCAS-3.20                    
                        MM5-3.7                       
                        MOLSCRIPT-1.0.2               
                        ASTRA-STK                     
                        GRELC_DAS_2_2_0               
                        MPICH                         
                        MPI-START                     
                        GATE-1.0.0-3                  
                        MPICH                         
                        MPICH-1.2.7p1                 
                        MPI-START                     
                        MPICH                         
                        MPI_NO_SHARED_HOME            
                        VO-gilda-scons-0.96.1         
                        VO-gilda-OCTAVE-2.9.10        
                        VO-gilda-ASTRA                
                        VO-gilda-PARAVIEW-2.4         
                        VO-gilda-MMCS-6.19-1          
                        VO-gilda-FALIGN               
                        VO-gilda-OCTAVE_3_2_0         
                        VO-gilda-GNUPLOT_4_2_4        
                        VO-gilda-MPEG2_V12            
                        VO-gilda-OCTAVE_3_2_0         
                        VO-gilda-GNUPLOT_4_2_4        
                        VO-gilda-MPEG2_V12            
                        VO-gilda-GATE-4.0.0           
                        VO-gilda-root_v5.20.00g       

Note that each value is a string and hence numerical tests on version numbers are not possible when applying matching conditions.

Next a more complex example usage of "lcg-info" is given. Suppose you wish to find all sites that support the MPICH packageand display how many CPU's are available plus the other software supported on those sites. It also makes sense to filter out CE's which are not available to the belle VO. This may be accomplished using:

# lcg-info --vo belle --list-ce --query 'Tag=MPICH' --attrs 'FreeCPUs,Tag'
- CE: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda
  - FreeCPUs            26
  - Tag                 POVRAY-3.5
                        DEMTOOLS-1.0                  
                        CSOUND-4.13                   
                        GEANT4-6                      
                        VLC-0.7.2                     
                        EGEODE-1.0                    
                        RASTER3D                      
                        G95-3.5.0                     
                        MAGIC-6.19                    
                        CODESA3D-1.0                  
                        CYCAS-3.20                    
                        MM5-3.7                       
                        MOLSCRIPT-1.0.2               
                        ASTRA-STK                     
                        GRELC_DAS_2_2_0               
                        MPICH                         
                        MPI-START                     
                        GATE-1.0.0-3                  
                        MPICH                         
                        MPICH-1.2.7p1                 
                        MPI-START                     
                        MPICH                         
                        MPI_NO_SHARED_HOME            
                        VO-gilda-scons-0.96.1         
                        VO-gilda-OCTAVE-2.9.10        
                        VO-gilda-ASTRA                
                        VO-gilda-PARAVIEW-2.4         
                        VO-gilda-MMCS-6.19-1          
                        VO-gilda-FALIGN               
                        VO-gilda-OCTAVE_3_2_0         
                        VO-gilda-GNUPLOT_4_2_4        
                        VO-gilda-MPEG2_V12            
                        VO-gilda-OCTAVE_3_2_0         
                        VO-gilda-GNUPLOT_4_2_4        
                        VO-gilda-MPEG2_V12            
                        VO-gilda-GATE-4.0.0           
                        VO-gilda-root_v5.20.00g

The final stage of this tutorial is to show that "lcg-info" can also query the attributes of CE's as is shown in the below that reverses the earlier search and searches for the closest CE to the SE " aliserv6.ct.infn.it". Type

# lcg-info --list-se --query 'SE=kek2-se.cc.kek.jp' --attrs CloseCE
- SE: kek2-se.cc.kek.jp
  - CloseCE             kek2-ce03.cc.kek.jp:2119/jobmanager-lcgpbs-ppj
                        kek2-ce03.cc.kek.jp:2119/jobmanager-lcgpbs-ops
                        kek2-ce06.cc.kek.jp:2119/jobmanager-lcgpbs-ilc
                        kek2-ce05.cc.kek.jp:2119/jobmanager-lcgpbs-ops
                        kek2-ce01.cc.kek.jp:2119/jobmanager-lcgpbs-ops
                        ....

 

If you have time left

Job Management

Login to User Interface

Please login to the user interface to do this practical training about job submission.

The address of the interface is :

Username :

Password :

 

Create VOMS proxy

Users, if already belonging to a group, or already assigned to a Role, can apply the request while creating the proxy with voms-proxy-init command. In this way, the information will be signed by the VOMS server and inserted in the proxy AC; resources will be able to parse them assigning to the user the expected rights.

# voms-proxy-init --voms belle
Your identity: /C=AU/O=APACGrid/OU=The University of Melbourne/CN=Tom Fifield
Creating temporary proxy ............................................................................................ Done
Contacting voms.cc.kek.jp:15020 [/C=JP/O=KEK/OU=CRC/CN=host/voms.cc.kek.jp] "belle" Done
Creating proxy ....................... Done
Your proxy is valid until Mon Mar 15 13:24:10 2010

 

Simple JDL

  • To submit a job to the Workload management System, a text file containing Job Description Language is used. The JDL describes the job and its requirements.

Here is the simplest example of a JDL file, to run a simple job on the grid:

[ui ~]$ cat hostname.jdl
Type = "Job";
JobType = "Normal";
Executable = "/bin/hostname";
StdOutput = "hostname.out";
StdError = "hostname.err";
OutputSandbox = {"hostname.err","hostname.out"};
Arguments = "-f";
RetryCount = 0;
ShallowRetryCount = 3;

The Executable attribute specifies the command to be run on the Worker Node. The OutputSandbox attribute indicates the files you want to be copied back after job execution; normally these are files where output and error streams are redirected;their names are determined by the StdOutput and StdError attributes respectively. Also the number of retries is specified, in case of failures.

JDL is more fully described here

The examples below assume that you have this hostname JDL file - please copy the listing above to create your own hostname.jdl.

Credentials Delegation

The commands shown on this page use WMProxy, a service that interacts with the WMS on your behalf. The authentication model within WM Proxy is as follows. In addition to a valid proxy it is necessary to delegate credentials to the WM Proxy server.

The user can either specify the delegationId to be associated with the delegated proxy by using the --delegationid option (shortly -d):

glite-wms-job-delegate-proxy -d myfirstdelegationid

Using -d option, the delegation is created, and its name is hold, so that subsequent invocations of glite-wms-job-submit and glite-wms-job-list-match can be given that delegation name, bypassing the delegation of a new proxy. So, when calling glite-wms-job-submit and glite-wms-job-list-match the delegation name is given with the -d option.

Instead of creating a delegation, it could be used -a option, which causes a delegated proxy to be established automatically.
glite-wms-job-delegate-proxy -a
But you have to specify -a option for each use of glite-wms-job-submit -a and glite-wms-job-list-match -a.

However massive use of this option it's not recommended, since it delegates a new proxy for each command issued, and delegation is a time-consuming operation, so it's better to do it once with glite-wms-job-delegate-proxy and reuse it.

To continue this tutorial, create a delegation towards WMProxy using as identifier your username, that you can get from the environment variable $USER.

[ui ~]$ echo $USER
user01
[ui ~]$ glite-wms-job-delegate-proxy -d $USER

Connecting to the service https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server


================== glite-wms-job-delegate-proxy Success ==================

Your proxy has been successfully delegated to the WMProxy:
https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server

with the delegation identifier: UPM04

==========================================================================

 

Job List Match

A JDL (Job Description Language) file describes a job that can be run. Before running the job, it is useful to test which computing elements (CE's) are able to accept it. Do this with the command glite-wms-job-list-match. As you can see, with -d option allows you to specify the delegation identifier you have created. Since we did it using the username (as get from $USER), this is the value we give to the option.

[user01@ui ~]$ glite-wms-job-list-match -d $USER hostname.jdl

Connecting to the service https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
The following CE(s) matching your job requirements have been found:

   *CEId*
 - ce.euag.org:2119/jobmanager-lcgpbs-gilda

==========================================================================

The list of Computing Elements (CE) testifies that the JDL syntax is correct and that the job can run on one or more of the shown CEs.

 

Job Submission

A simple job can be submitted by the command: glite-wms-job-submit -d delegationId -o jobidfile jdlname

[user01@ui ~]$ glite-wms-job-submit -d $USER -o jobid hostname.jdl

Connecting to the service https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ

The job identifier has been saved in the following file:
/home/user01/jobid

==========================================================================

The file /home/account/jobid is the output of the submission process. It receives the jobID(s) returned by the submission process. If another job is submitted (by repeating the submission line) its jobID is appended to the same jobID file. Try it by yourself.


 

Job Status

In order to know about the job status another command is available: glite-wms-job-status; this command queries LB (Logging and Bookkeeping service) on the status of any job whose job id is present in the specified file, in this case, jobid :

[user01@ui ~]$ glite-wms-job-status -i jobid
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://kek2-wms.cc.kek.jp:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all:3

*************************************************************
BOOKKEEPING INFORMATION: 

Status info for the Job : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
Current Status:     Running 
Status Reason:      Job successfully submitted to Globus
Destination:        ce.euag.org:2119/jobmanager-lcgpbs-gilda
Submitted:          Fri Jul 31 05:20:54 2009 UTC
*************************************************************

The command, on the basis of the content of jobid file, shows a list of the corresponding submitted jobs, whose status can be queried for one or all of them. The -i option describes the file from which the command takes the jobID(s) to be inspected. Current job status is done for the first two jobs, scheduled for the third one. The selected CE is also shown. Alternatively the same command can be issued directly specifying the jobID(s), as in the following case:

[user01@ui ~]$ glite-wms-job-status https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ

*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
Current Status:     Running 
Status Reason:      Job successfully submitted to Globus
Destination:        ce.euag.org:2119/jobmanager-lcgpbs-gilda
Submitted:          Fri Jul 31 05:20:54 2009 UTC
*************************************************************

Note that this command doesn't require a delegation identifier to be specified.

 

Job Logging

A complete history of a job is permanently stored in the Logging & Bookkeeping service and can be retrieved using the command:
glite-wms-job-logging-info <jobID> -v option is used for the level of verbosity, 3 is the highest.

[user01@ui ~] glite-wms-job-logging-info https://wms104.cern.ch:9000/hk0VSbNhe59j0Buo24G_qw
*********************************************************************************************
LOGGING INFORMATION

Printing info for the Job:https://wms104.cern.ch:9000/

 

Job Output

When the result of a glite-job-status is that a job has been succesfully completed, the result can be retrieved by the command: glite-wms-job-output. You don't need to specify a delegation identifier.

[user01@ui ~]$ glite-wms-job-output -i jobid
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://wms.euag.org:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://wms.euag.org:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all (use , as separator or - for a range): 3 

Connecting to the service https://wms.euag.org:7443/glite_wms_wmproxy_server 

================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/user01_tcl6gKsrMJm8TReyXcc7tQ

================================================================================

In order to inspect the job output, list the files in the indicated directory and show the content of the output file(s).

[UPM04@ui ~]$ cd /tmp/jobOutput/UPM04_tcl6gKsrMJm8TReyXcc7tQ
[UPM04@ui UPM04_tcl6gKsrMJm8TReyXcc7tQ]$ ls -la
total 12
drwxr-xr-x  2 UPM04 UPM04 4096 Jul 31 05:51 .
drwxrwxrwt  8 root  root  4096 Jul 31 05:51 ..
-rw-rw-r--  1 UPM04 UPM04    0 Jul 31 05:51 hostname.err
-rw-rw-r--  1 UPM04 UPM04   13 Jul 31 05:51 hostname.out
[UPM04@ui UPM04_tcl6gKsrMJm8TReyXcc7tQ]$ cat hostname.out
wnc.euag.org
[UPM04@ui UPM04_tcl6gKsrMJm8TReyXcc7tQ]$ 

The output directory can be choosen by the user by the --dir option:

[user01@ui ~]$ glite-wms-job-output -i jobid --dir jobdir
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://kek2-wms.cc.kek.jp:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all (use , as separator or - for a range): 3

Connecting to the service https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
have been successfully retrieved and stored in the directory:
/home/user01/jobdir

================================================================================
$ ls -la jobdir/
total 12
drwxr-xr-x   2 giorgio users  112 Jun 16 09:49 .
drwx------  87 giorgio users 8536 Jun 16 09:49 ..
-rw-r--r--   1 giorgio users    0 Jun 16 09:49 testsandbox.err
-rw-r--r--   1 giorgio users  405 Jun 16 09:49 testsandbox.out

 

Job cancel

If anything goes wrong a job can be cancelled by the command: glite-wms-job-cancel Again the -i option is also available (especially useful in order to cancel more files with a single command):

[user01@ui ~]$ glite-wms-job-cancel -i jobid
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://kek2-wms.cc.kek.jp:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all (use , as separator or - for a range): 3

Are you sure you want to remove specified job(s) [y/n]y : y

Error - Cancel not allowed
Current Job Status is Cleared

In this case the job was not cancelled because it has been already succesfully completed - the status "cleared" shows that we have already retrieved the output.

Data Management

LFC and LCG commands

 

Listing file and directory

For each of the supported VO a separate "top level" directory exists under the "/grid" directory. You can see all the files that are stored for the belle VO. First ensure you have a running Voms Proxy and then type:

 

[ui]$ lfc-ls -l /grid/belle

You will see a listing of the contents of the /grid/belle directory.

Rather than having to type an absolute path for every file and directory you use, it is instead possible to define a HOME directory from which you may use relative file/directory paths. Set this now by setting the environment variable LFC_HOME as follows:

 

[ui]$ export LFC_HOME=/grid/belle/tutorials 

 

Create directory

Before creating and uploading any of your own files create a personal directory for storage by typing (of course, here and in the next examples, you have to replace userdirectory with your account name; so, don't just cut and paste !):

 

[ui]$ lfc-mkdir /grid/belle/tutorials/userdirectory 

To check that you have created your directory type:

 

[ui]$ lfc-ls -l /grid/belle/tutorials

and you should see your directory (plus possibly those of other attendees).

 

Upload file into SE

The next step is to upload a file into the directory you just created. First create locally a simple text file:

 

[ui]$ echo "Put something here" > text_file.txt 

The command used for this is lcg-cr (LCG copy and register). Type the following to store this file on the kek2-se.cc.kek.jp Storage Element : (use lcg-infosites in order to find which the available SE are)

 

[ui]$ lcg-cr --vo belle  -l lfn:/grid/belle/tutorials/userdirectory/text_file.txt -d kek2-se.cc.kek.jp file://$HOME/text_file.txt 

The output should be something like this:

 

[ui]$ lcg-cr --vo belle -l lfn:/grid/belle/tutorials/userdirectory/text_file.txt  -d kek2-se.cc.kek.jp file://$HOME/text_file.txt   guid:c8da3f42-9a84-47de-9f32-f10ef93c8daa 

Of course, the guid you will have is different, since it is an unique identifier for each file (except for the case when you are replicating a file, as we will see). Check that the file is there by listing the contents of your directory.

Before continuing it is worth noting the difference between the command used to store the file and the creation of the directory in previous. The directory created is just a virtual directory and only exists within the catalog of lfn's. On the other hand the file physically exists on an SE but has an additional "virtual" filename in the catalog. This is connected to why the commands just handling the lfn namespace tend to start with "lfc" whilst the commands manipulating the file directly tend to start with "lcg".

 

Get the file SURL

For several purpose, FTS for instance, is useful to know the file SURL (they can be many if the file has replicas somewhere). The appropriate command is lcg-lr (list-replicas) [lfn | guid]

[ui]$ lcg-lr --vo belle lfn:/grid/belle/tutorials/userdirectory/text_file.txt
srm://kek2-se.cc.kek.jp/dpm/cc.kek.jp/home/belle/generated/2009-07-19/filef42b726f-d9dc-45f4-b3b5-f4d55a6d9c44 

 

Replicate file between SE

gLite supports file replication. A file can be stored on multiple SE's and then a running job can access the closest SE with the file on it, thus giving faster access times to the data. This also helps protect against failures/access difficulties with a particular SE. To find the list of SE'S available to you see the tutorial on lcg-infosites which can be found here. We will replicate the file just created to the SE dcache.ijs.si with the command

 

[ui]$ lcg-rep --vo belle -d dcache.ijs.si lfn:/grid/belle/tutorials/userdirectory/text_file.txt 

There is no output from this command on success, but you can check that the replica was created by listing all the replicas of your file, that is done by using the LCG list replicas command:

 

[ui]$ lcg-lr --vo belle lfn:/grid/belle/tutorials/userdirectory/text_file.txt 

You should get two replicas listed, as here:

 

[fifieldt@kek2-ui01 ~]$ lcg-lr --vo belle lfn:/grid/belle/tutorials/fifieldt/text_file.txt
srm://dcache.ijs.si/pnfs/ijs.si/belle/generated/2010-03-22/file7d204403-a6f1-4f71-bb49-01274473325d
srm://kek2-se.cc.kek.jp/dpm/cc.kek.jp/home/belle/generated/2010-03-22/filee920b934-e391-4980-8777-6cdc8c0feb46

Note how the path to where each file is stored is different. This demonstrates how the use of a "lfn" avoids the need to understand the local filesystem where the replica is actually stored.

 

As an example of some of some of the other functionality available a second lfn for the file you uploaded is created. This is very similar to the Unix symbolic link. A second lfn is created with the command:

 

[ui]$ lfc-ln -s $HOME/text_file.txt /grid/belle/tutorials/userdirectory/text_file_symlink.txt 

Note the file path for the source file was used so as to avoid problems with relative links being used from the wrong directory. If you now list the contents of your directory you should see both names, but the symbolic link has 0 size and also the link target is shown:

 

[ui]$ lfc-ls -l  /grid/belle/tutorials/userdirectory  
-rw-rw-r--   1 554      102              19 Jul 19 15:36 text_file.txt  
lrwxrwxrwx  1 554      102                0 Jul 19 15:37 text_file_symlink.txt -> /grid/belle/tutorial/catania41/text_file.txt 

 

Download a file from SE to UI

Having already uploaded a file the next step is to show downloading a file. To download the file you already uploaded using the new lfn you have just created use the command:

 

[ui]$ lcg-cp --vo belle lfn:/grid/belle/tutorials/userdirectory/text_file.txt file://$HOME/text_file_copy.txt 

You can check that the file you just downloaded is the same as the file you uploaded with cat command, you should find the same text you entered below.

 

[ui]$ cat text_file_copy.txt 

 

Clean all !

You can delete a file from SE with lcg-del

[ui]$ lcg-del -a --vo belle lfn:/grid/belle/tutorials/userdirectory/text_file.txt

This will remove File Catalog entries as well

 

[ui]$ lfc-ls /grid/belle/tutorials/userdirectory 
[ui]$ 

complete by removing the working directory on file catalog

 

[ui]$ lfc-rm -r /grid/belle/tutorials/userdirectory
[ui]$ lfc-ls /grid/belle/tutorials | grep userdirectory 

 

If you still have time

Advanced: Recursive Deletion Tools

Advanced: FTS Usage

Advanced: Uber FTP Advanced: LFC API

Advanced: Permissions

 

Metadata

Setup the .mdclient.config configuration file

  • Before starting the AMGA client application, it is necessary to copy a configuration file into the home directory. There is a template file provided by the AMGA client installation that can be customised and used to access your own AMGA server.
  • Copy template configuration file to the HOME directory
    • Default configuration file is in the path, "/opt/glite/etc/"
    • copy the template for this training, /tmp/mdclient.config

The following fields should be modified according to your login name, password and home directory:

# Connection options
Host = amga2.atlas.unimelb.edu.au
Port = 8822

# User settings
#Login = 
#Password = 
PermissionMask = rwx
GroupMask = r-x
#Home =

# Security options
UseSSL = no # Values: require, try, no. If off,  all options below are ignored

AuthenticateWithCertificate = 0 # Use certificate to authenticate
TrustedCertDir = /etc/grid-security/certificates

 

Connect to the AMGA server

It is possible to get help anytime on the client just using the 'help' command.

[user01@ui ~]$ mdclient -c ~/.mdclient.config 
Connecting to ui.euag.org:8822...
ARDA Metadata Server 1.9.0
Query> help createdir
>> createdir /parentdir/dir [option]
>>   Creates a new directory..
Query> help rmdir
>> rmdir directory
>>   Removes a directory.
Query>

Make sure you have logined with the right account

Query> whoami
>> user01

 

General Operations on AMGA

AMGA provides a directory-like structure of the collections of entries

Query> ls -l 
>> drwxrwxr-x user01  /trainer/taipei01/
Query> createdir tutorial ( create the directory under "UPM04" directory )
Query> ls -l  
>> drwxrwxr-x taipei01   /trainer/taipei01/tutorial
Query> cd tutorial
Query> pwd           
>> /trainer/taipei01/tutorial/
Query>

 

Let us start with an example

  • create cities directory which has longitde and latitude as attributes.
    • the data types of longitde and latitude are int.
Query> pwd
/trainer/user01/  
Query> createdir cities
Query> addattr cities longitude int
Query> addattr cities latitude int
Query> listattr cities
>> longitude
>> int
>> latitude
>> int
  • create weather directory having city, temp_hi, temp_lo, humidity, time as attributes.
    • the data type of city attribute is varchar with 30 bytes.
    • temp_hi means the high temperature and its data type is float.
    • temp_lo means the low temperature and its data type is float.
    • the data type of humidity is also float.
    • time means time stamp of inserting entry.
Query> createdir weather
Query> addattr weather city varchar(30) temp_hi float temp_lo float humidity float time timestamp
Query> listattr weather
>> city
>> varchar(30)
>> temp_hi
>> float
>> temp_lo
>> float
>> humidity
>> int
>> time
>> timestamp

Valid datatypes are summarized in the following table where the corresponding AMGA DB backend datatype is also shown.

AMGAPostgreSQLMySQLOracleSQLitePython
intintegerintnumber(38)intint
floatdouble precisiondouble precisionfloatfloatfloat
varchar(n)character varying(n)character varying(n)varchar2(n)varchar(n)string
timestamptimestamp w/o TZdatetimetimestamp(6)unsupportedtime(unsupported)
texttexttextlongtextstring
numeric(p,s)numeric(p.s)numeric(p.s)numeric(p.s)numeric(p.s)float

 

  • Fill the tables with entries
    • ingest the city name as entry ID in cities directory
Query> addentry cities/KualaLumpur latitude 3  longitude 101 
Query> addentry cities/Taipei latitude 25 longitude 121 
Query> addentry cities/Tokyo latitude 35  longitude 139
Query> addentry cities/Hongkong latitude  22 longitude 110 
Query> ls -l cities  
>> -rwxr-xr-x root  KualaLumpur
>> -rwxr-xr-x root  Taipei
>> -rwxr-xr-x root  Tokyo
>> -rwxr-xr-x root  Hongkong

Query> addentry weather/record_hk01 city 'Hongkong' temp_hi 30 temp_lo 23 humidity 65 time '2009-07-28 11:00:06'
Query> addentry weather/record_hk02 city 'Hongkong' temp_hi 32 temp_lo 24 humidity 70 time '2009-07-29 11:03:17'
Query> addentry weather/record_tk01 city 'Tokyo' temp_hi 24 temp_lo 20 humidity 61 time '2009-07-27 11:03:17'
Query> addentry weather/record_tk02 city 'Tokyo' temp_hi 17 temp_lo 11 humidity 52 time '2009-07-29 09:03:17'
Query> addentry weather/record_tp01 city 'Taipei' temp_hi 25 temp_lo 21 humidity 68 time '2009-07-28 11:03:17'
Query> addentry weather/record_tp02 city 'Taipei' temp_hi 28 temp_lo 22 humidity 65 time '2009-07-29 12:03:17'
Query> addentry weather/record_kl01 city 'KualaLumpur' temp_hi 32 temp_lo 28 humidity 66 time '2009-07-27 12:03:17'
Query> addentry weather/record_kl02 city 'KualaLumpur' temp_hi 30 temp_lo 24 humidity 65 time '2009-07-27 12:03:17'
Query> addentry weather/record_kl03 city 'KualaLumpur' temp_hi 33 temp_lo 26 humidity 67 time '2009-07-27 12:03:17'
Query>

Now, we can get some information back

Which are the cities further north then 30 degrees north, and how high are they?

Query> selectattr cities:FILE longitude 'latitude > 30'
>> Tokyo
>> 139 

Show all cities and their latitude

Query> selectattr cities:FILE latitude ' '
>> KualaLumpur
>> 3
>> Taipei
>> 25
>> Tokyo
>> 35
>> Hongkong
>> 22

Which city has low temperatures higher than 25 degrees?

Query> selectattr weather:city weather:temp_lo 'weather:temp_lo > 25'
>> KualaLumpur
>> 28
>> KualaLumpur
>> 26

Which city has low temperatures smaller than 15 degrees?

Query> selectattr weather:city weather:temp_lo 'weather:temp_lo < 15'
>> Tokyo
>> 11

[edit]

Exercises

Ex1. In the weather directory, we would like to retain the data about cities in ASIA. So, we have to remove the record_ma01entry. What could we do?

Hint: Manipulating Entry Command
Answer: 
Query> rm  /trainer/user01/weather/record_01

Ex2. Which city has high temperatures higher than 25 degrees and the city name starts with "K"?

Hint: use query condition with "like"
Answer: 
Query> selectattr weather:city  ' weather:temp_hi > 25 and like (weather:city, "K%") '
>> KualaLumpur

Ex3. Which city has the humidity smaller than 60% and shows its latitude, longitude, temp_lo, temp_hi?

Hint: one of the join condition is "cities:FILE=weather:city"
Answer:
Query> selectattr weather:city cities:longitude cities:latitude weather:temp_hi weather:temp_lo ' cities:FILE=weather:city and weather:humidity < 60 '
>> Tokyo
>> 139
>> 35
>> 17
>> 11

 

If you still have time

https://grid.ct.infn.it/twiki/bin/view/GILDA/AMGAAdv

 

Advanced Job Management

Objective

This advanced exercises aim to take you through the more advanced operations for job submission. Including specification for the long-running jobs, advanced sandbox management, multiple requirements and real-time output perusal. This practical will leads you step by step how to perform the above tasks with a series of glite commands and JDL settings.


 

MyProxy Server for Long-running job submission

  • To list MyProxy server of your UI machine
[user01@ui ~]$ echo $MYPROXY_SERVER
px.euag.org
  • To initiate your MyProxy server, -d option is to use the proxy certificate subject(DN) as the default username, instead of the LOGNAME env. var, -n option is for not prompt for passphrase.
[user01@ui ~]$ myproxy-init -d -n
  • To show your status of myproxy, use the command, myproxy-info. -d option is as the previous command.
[user01@ui ~]$ myproxy-info -d

 

Advanced Sandbox Management

This feature enables the possibility to indicate input sandbox files stored not on the UI, but on a GridFTP server, and similarly, to specify that output files should be transferred to a GridFTP server when the job finishes. This has several advantages:
1. the input files do not have to be on the host from which the job is submitted.
2. the output files are immediately available when the job ends, without having to issue a command to retrieve them.
3. the sandbox files do not have to go through the WMS host, which otherwise can easily become a bottleneck.

  • You need to specify the gsiftp location in jdl file for transferring files directly from a remote data storages.
  • To putting files on GridFTP through output sandbox, you need to have the authority of the GridFTP.
  • To look up the gsiftp location of the file you store in the SE, you need to use commands of Data Management to get TURL for gsiftp location. Here's an example of gsiftp location of the file stored in teh SE
InputSandbox = {"gsiftp://dpm01.grid.sinica.edu.tw/dpm01.grid.sinica.edu.tw:/path1/euasia/
2010-03-04/fileafff04d0-f85d-40e1-8041-b963cfc87505.1535017.0"}

 

Real Time Output Perusal

The user can enable the job perusal by setting the attribute PerusalFileEnable to true in JDL file. This makes the WN to upload, at regular time intervals (defined by the PerusalTimeInterval attribute and expressed in seconds), a copy of the output files specified using the glite-wms-job-perusal command to the WMS machine (by default), or to a GridFTP server specified by the attribute PerusalFileDestURI. The following JDL description shows how to set up job perusal in JDL file.

 

JDL Description

Specify your JDL file that including the above requirements. The JDL example describes as below. Here is the example of the JDL file including

  • MyProxy server for a long-running job on the grid
  • Set RetryCount=0 for disable deep resubmission and ShallowRetryCount=3 to enable shallow resubmission as 3
  • Real Time Output Perusal File: progress.inf
  • Time Interval of Real Time Output Perusal: 40 secs
[user01@ui ~]$ cat longrun.jdl
Type = "Job";
JobType = "Normal";
Executable = "longrun.sh";
Arguments = "120";
StdOutput = "longrun.out";
StdError = "longrun.err";
InputSandbox = {"longrun.sh"};
OutputSandbox = {"longrun.err","longrun.out", "progress.inf"};
#MyProxyServer = "px.euag.org";
RetryCount =0;
ShallowRetryCount = 3;
PerusalFileEnable = true;
PerusalTimeInterval = 40;

This JDL will execute longrun.sh.

  • longrun.sh is the script to specify an argument of a time period(minutes), and print out the progress percentage of how many minutes goes by in progress.inf. Please copy longrun.sh from /tmp/longrun.sh.
#!/bin/bash

for (( c=1; c<=$1; ++c ))
do
  sleep 1m
  echo "$c/$1" >> progress.inf
done

echo "done"

For example, the following command will start a task for running 120 mins(2hrs)

[user01@ui ~]$ longrun.sh 120

Please copy and create your own longrun.sh and longrun.jdl for a long-running job practice.

 

Credentials Delegation

Same step as in Basic session

 

Job List Match

A JDL (Job Description Language) file describes a job that can be run. Before running the job, it is useful to test which computing elements (CE's) are able to accept it. Do this with the command glite-wms-job-list-match. As you can see, with -d option allows you to specify the delegation identifier you have created. Since we did it using the username (as get from $USER), this is the value we give to the option.

[user01@ui ~]$ glite-wms-job-list-match -d $USER longrun.jdl

Connecting to the service https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server

==========================================================================

           COMPUTING ELEMENT IDs LIST 
The following CE(s) matching your job requirements have been found:

   *CEId*
 - ce.euag.org:2119/jobmanager-lcgpbs-gilda

==========================================================================

The list of Computing Elements (CE) testifies that the JDL syntax is correct and that the job can run on one or more of the shown CEs.

 

Job Submission

The job can be submitted by the command:
glite-wms-job-submit -d delegationId -o jobidfile jdlname or glite-wms-job-submit -a -o jobidfile jdlname

[user01@ui ~]$ glite-wms-job-submit -a $USER -o jobid longrun.jdl

Connecting to the service https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server


====================== glite-wms-job-submit Success ======================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ

The job identifier has been saved in the following file:
/home/user01/jobid

==========================================================================

The file /home/user01/jobid is the output of the submission process. It receives the jobID(s) returned by the submission process. If another job is submitted (by repeating the submission line) its jobID is appended to the same jobID file. Try it by yourself.


 

Job Status

In order to know about the job status another command is available: glite-wms-job-status; this command queries LB (Logging and Bookkeeping service) on the status of any job whose job id is present in the specified file, in this case, jobid :

[user01@ui ~]$ glite-wms-job-status -i jobid
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://kek2-wms.cc.kek.jp:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all:3

*************************************************************
BOOKKEEPING INFORMATION: 

Status info for the Job : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
Current Status:     Running 
Status Reason:      Job successfully submitted to Globus
Destination:        ce.euag.org:2119/jobmanager-lcgpbs-gilda
Submitted:          Fri Jul 31 05:20:54 2009 UTC
*************************************************************

The command, on the basis of the content of jobid file, shows a list of the corresponding submitted jobs, whose status can be queried for one or all of them. The -i option describes the file from which the command takes the jobID(s) to be inspected. Current job status is done for the first two jobs, scheduled for the third one. The selected CE is also shown. Alternatively the same command can be issued directly specifying the jobID(s), as in the following case:

[user01@ui ~]$ glite-wms-job-status https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ

*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
Current Status:     Running 
Status Reason:      Job successfully submitted to Globus
Destination:        ce.euag.org:2119/jobmanager-lcgpbs-gilda
Submitted:          Fri Jul 31 05:20:54 2009 UTC
*************************************************************

Note that this command doesn't require a delegation identifier to be specified.

 

Job Real Time Output Perusal

  • After a job starts, you can use set command for the perusal to the specific file.
[user01@ui ~]$ glite-wms-job-perusal --set -f progress.inf
  • After setting the perusal to the file, you can get the
[user01@ui ~]$ glite-wms-job-perusal --get -f progress.inf

 

Job Logging

A complete history of a job is permanently stored in the Logging & Bookkeeping service and can be retrieved using the command:
glite-wms-job-logging-info <jobID> -v option is used for the level of verbosity, 3 is the highest.

[user01@ui ~]$ glite-wms-job-logging-info -v 3 <jobID>

 

Job Output

When the result of a glite-job-status is that a job has been succesfully completed, the result can be retrieved by the command: glite-wms-job-output. You don't need to specify a delegation identifier.

[user01@ui ~]$ glite-wms-job-output -i jobid
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://wms.euag.org:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://wms.euag.org:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all (use , as separator or - for a range): 3 

Connecting to the service https://wms.euag.org:7443/glite_wms_wmproxy_server 

================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/user01_tcl6gKsrMJm8TReyXcc7tQ

================================================================================

In order to inspect the job output, list the files in the indicated directory and show the content of the output file(s).

[user01@ui ~]$ cd /tmp/jobOutput/taipei01_tcl6gKsrMJm8TReyXcc7tQ
[taipei01@ui taipei01_tcl6gKsrMJm8TReyXcc7tQ]$ ls -la
total 12
drwxr-xr-x  2 taipei01 taipei01 4096 Jul 31 05:51 .
drwxrwxrwt  8 root     root     4096 Jul 31 05:51 ..
-rw-rw-r--  1 taipei01 taipei01    0 Jul 31 05:51 hostname.err
-rw-rw-r--  1 taipei01 taipei01   13 Jul 31 05:51 hostname.out
[taipei01@ui taipei01_tcl6gKsrMJm8TReyXcc7tQ]$ cat hostname.out
wnc.euag.org
[user01@ui user01_tcl6gKsrMJm8TReyXcc7tQ]$ 

The output directory can be choosen by the user by the --dir option:

[user01@ui ~]$ glite-wms-job-output -i jobid --dir jobdir
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://kek2-wms.cc.kek.jp:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all (use , as separator or - for a range): 3

Connecting to the service https://kek2-wms.cc.kek.jp:7443/glite_wms_wmproxy_server


================================================================================

         JOB GET OUTPUT OUTCOME

Output sandbox files for the job:
https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
have been successfully retrieved and stored in the directory:
/home/user01/jobdir

================================================================================
$ ls -la jobdir/
total 12
drwxr-xr-x   2 giorgio users  112 Jun 16 09:49 .
drwx------  87 giorgio users 8536 Jun 16 09:49 ..
-rw-r--r--   1 giorgio users    0 Jun 16 09:49 testsandbox.err
-rw-r--r--   1 giorgio users  405 Jun 16 09:49 testsandbox.out

 

Job cancel

If anything goes wrong a job can be cancelled by the command: glite-wms-job-cancel Again the -i option is also available (especially useful in order to cancel more files with a single command):

[user01@ui ~]$ glite-wms-job-cancel -i jobid
------------------------------------------------------------------
1 : https://kek2-wms.cc.kek.jp:9000/0GSFe8gObmvvyOJOvWx2-A
2 : https://kek2-wms.cc.kek.jp:9000/DNfFrL9cxnTTHzl9edVCJg
3 : https://kek2-wms.cc.kek.jp:9000/tcl6gKsrMJm8TReyXcc7tQ
a : all
q : quit
------------------------------------------------------------------

Choose one or more jobId(s) in the list - [1-3]all (use , as separator or - for a range): 3

Are you sure you want to remove specified job(s) [y/n]y : y

Error - Cancel not allowed
Current Job Status is Cleared

In this case the job was not cancelled because it has been already succesfully completed - the status "cleared" shows that we have already retrieved the output.

If you still have time

Medium: More on JDL

Medium: Moving Data

 

-- BelleII - 19 Jan 2010

  • No labels