sherpaTarCreator.slurm_taurus.batchJob Class Reference

Inheritance diagram for sherpaTarCreator.slurm_taurus.batchJob:

Collaboration diagram for sherpaTarCreator.slurm_taurus.batchJob:

Public Member Functions
def	submit (self, dryRun=False)
def	write (self, useSingularity=True, extraDirs=[])

Public Attributes
	memMB
	id
	name
	cmds
	hours
	nCores
	env
	basedir
	dependsOnOk
	dependsOnAny

Detailed Description

Definition at line 13 of file slurm_taurus.py.

Member Function Documentation

submit()

def sherpaTarCreator.slurm_taurus.batchJob.submit	(		self,
			dryRun = False
	)

Definition at line 15 of file slurm_taurus.py.

    def submit(self, dryRun=False):
        cpt = 1
        if self.memMB == 1499:
            self.memMB = 1970
        elif self.memMB == 2499:
            #ram = 5250
            self.memMB = 2583
            cpt = 2
        cmd = "sbatch -e "+self.basedir+"/"+self.name+".log -o "+self.basedir+"/"+self.name+".log --time="+str(self.hours)+":00:00 --mem-per-cpu="+str(self.memMB)+" -p haswell,romeo --cpus-per-task="+str(cpt)+" -N 1 --ntasks "+str(self.nCores)
        if os.environ.get('SLURM_MAIL_USER'):
            cmd +=  " --mail-type='END,FAIL,TIME_LIMIT' --mail-user='"+os.environ.get('SLURM_MAIL_USER')+"'"
 
        if len(self.dependsOnOk)>0 or len(self.dependsOnAny)>0:
            cmd += " -d "
            if len(self.dependsOnOk)>0:
                okdeps = "afterok:"+",afterok:".join(self.dependsOnOk)
                cmd += okdeps+","
            if len(self.dependsOnAny)>0:
                anydeps = "afterany:"+",afterany:".join(self.dependsOnAny)
                cmd += anydeps+","
            cmd = cmd[:-1]
 
        if "ecm" in self.basedir:
            jobname = os.path.relpath(self.basedir+"/"+self.name+".log", self.basedir+"/../..")
        else:
            jobname = os.path.relpath(self.basedir+"/"+self.name+".log", self.basedir+"/..")
        cmd += " -J "+jobname+" "+self.basedir+"/"+self.name+".sh >> qsub.log \n"
 
        if dryRun:
            print (cmd)
            self.id = "-1"
        else:
            print(cmd)
            p = subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
            retcode = p.communicate()
            if len(retcode[0]):
                print (retcode[0])
            if len(retcode[1]):
                print (retcode[1])
            if p.returncode:
                print ("ERROR: error while submitting job")
                print ("return code: " + str(p.returncode))
                sys.exit(11)
 
            p.wait()
 
            self.id = getJobIDfromlastJob()
 
        print("Submitted "+str(self.id)+" ("+self.name+")\n\n")
 

write()

def sherpaTarCreator.batchJobBase.batchJobBase.write (   self,   useSingularity = True,   extraDirs = []  )

inherited

Definition at line 24 of file batchJobBase.py.

  def write(self, useSingularity=True, extraDirs=[]):
    executable =  "#!/bin/sh -\n"
 
    # COMPILER_PATH
    if useSingularity:
      platform = str(os.environ['COMPILER_PATH']).split('/')[-1].replace('el9', 'almalinux9')
      executable += 'if [ "$1" != "--really" ]; then \n'
      executable += '  exec singularity exec -e --no-home'
      for dir in ["/cvmfs", "/var", self.basedir, "$(pwd | cut -d '/' -f 1-2)"] + extraDirs:
        executable += ' -B '+dir
      executable += ' /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/'
      executable += platform + ' /bin/bash -- "$0" --really "$@";\n'
      executable += 'fi\n'
      executable += "shift;\n\n"
      executable += "export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase\n"
      executable += "source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh\n"
 
    executable += "ulimit -f 1000000;\n"
    executable += "cd "+self.basedir+"\n\n"
    executable += "echo 'ncores="+str(self.nCores)+" nhours="+str(self.hours)+" "+self.basedir+"/"+self.name+".sh';\n"
    for cmd in self.cmds:
      executable += cmd+"\n"
    executable += "exit 0\n"
 
 
    filename = self.basedir+"/"+self.name+".sh"
    with open(filename, 'w') as f:
        f.write(executable)
 
    #make shell-files executable
    st = os.stat(filename)
    os.chmod(filename, st.st_mode | stat.S_IEXEC)
 
 

Member Data Documentation

basedir

sherpaTarCreator.batchJobBase.batchJobBase.basedir

inherited

Definition at line 19 of file batchJobBase.py.

cmds

sherpaTarCreator.batchJobBase.batchJobBase.cmds

inherited

Definition at line 10 of file batchJobBase.py.

dependsOnAny

sherpaTarCreator.batchJobBase.batchJobBase.dependsOnAny

inherited

Definition at line 22 of file batchJobBase.py.

dependsOnOk

sherpaTarCreator.batchJobBase.batchJobBase.dependsOnOk

inherited

Definition at line 21 of file batchJobBase.py.

env

sherpaTarCreator.batchJobBase.batchJobBase.env

inherited

Definition at line 18 of file batchJobBase.py.

hours

sherpaTarCreator.batchJobBase.batchJobBase.hours

inherited

Definition at line 11 of file batchJobBase.py.

id

sherpaTarCreator.slurm_taurus.batchJob.id

Definition at line 45 of file slurm_taurus.py.

memMB

sherpaTarCreator.slurm_taurus.batchJob.memMB

Definition at line 17 of file slurm_taurus.py.

name

sherpaTarCreator.batchJobBase.batchJobBase.name

inherited

Definition at line 9 of file batchJobBase.py.

nCores

sherpaTarCreator.batchJobBase.batchJobBase.nCores

inherited

Definition at line 12 of file batchJobBase.py.

---

The documentation for this class was generated from the following file:

• slurm_taurus.py