ATLAS Offline Software
Public Member Functions | Public Attributes | Private Member Functions | Private Attributes | List of all members
python.trfExe.bsMergeExecutor Class Reference

Specalise the script executor to deal with the BS merge oddity of excluding empty DRAWs. More...

Inheritance diagram for python.trfExe.bsMergeExecutor:
Collaboration diagram for python.trfExe.bsMergeExecutor:

Public Member Functions

def preExecute (self, input=set(), output=set())
 
def execute (self)
 
def postExecute (self)
 
def exe (self)
 
def exe (self, value)
 
def exeArgs (self)
 
def exeArgs (self, value)
 
def validate (self)
 
def inData (self)
 
def inData (self, value)
 
def outData (self)
 
def outData (self, value)
 
def myMerger (self)
 Now define properties for these data members. More...
 
def name (self)
 
def name (self, value)
 
def substep (self)
 
def trf (self)
 
def trf (self, value)
 
def inDataUpdate (self, value)
 
def outDataUpdate (self, value)
 
def input (self)
 
def output (self)
 
def extraMetadata (self)
 
def hasExecuted (self)
 
def rc (self)
 
def errMsg (self)
 
def validation (self)
 
def validation (self, value)
 
def hasValidated (self)
 
def isValidated (self)
 
def first (self)
 
def preExeStartTimes (self)
 
def exeStartTimes (self)
 
def exeStopTimes (self)
 
def valStartTimes (self)
 
def valStopTimes (self)
 
def preExeCpuTime (self)
 
def preExeWallTime (self)
 
def cpuTime (self)
 
def usrTime (self)
 
def sysTime (self)
 
def wallTime (self)
 
def memStats (self)
 
def memAnalysis (self)
 
def postExeCpuTime (self)
 
def postExeWallTime (self)
 
def validationCpuTime (self)
 
def validationWallTime (self)
 
def cpuTimeTotal (self)
 
def wallTimeTotal (self)
 
def eventCount (self)
 
def reSimEvent (self)
 
def athenaMP (self)
 
def dbMonitor (self)
 
def setPreExeStart (self)
 
def setValStart (self)
 
def doAll (self, input=set(), output=set())
 Convenience function. More...
 

Public Attributes

 conf
 Executor configuration: More...
 
 inData
 
 outData
 

Private Member Functions

def _buildStandardCommand (self)
 

Private Attributes

 _inputBS
 
 _outputBS
 
 _maskedFiles
 
 _useStubFile
 
 _mergeBSFileList
 
 _mergeBSLogfile
 
 _outputFilename
 
 _doRename
 
 _cmd
 
 _exeStart
 
 _memMonitor
 
 _hasExecuted
 
 _rc
 
 _exeStop
 
 _expectedOutput
 
 _exe
 
 _exeArgs
 
 _echoOutput
 
 _input
 
 _output
 
 _logFileName
 
 _echologger
 
 _exeLogFile
 
 _echostream
 
 _memSummaryFile
 
 _memFullFile
 
 _memStats
 
 _valStart
 
 _hasValidated
 
 _isValidated
 
 _errMsg
 
 _eventCount
 Check event counts (always do this by default) Do this here so that all script executors have this by default (covers most use cases with events) More...
 
 _valStop
 
 _name
 
 _inData
 
 _outData
 
 _extraMetadata
 
 _preExeStart
 
 _memLeakResult
 
 _athenaMP
 
 _athenaMT
 
 _athenaConcurrentEvents
 
 _dbMonitor
 
 _resimevents
 
 _alreadyInContainer
 
 _containerSetup
 
 _myMerger
 
 _trf
 
 _validation
 

Detailed Description

Specalise the script executor to deal with the BS merge oddity of excluding empty DRAWs.

Definition at line 2025 of file trfExe.py.

Member Function Documentation

◆ _buildStandardCommand()

def python.trfExe.scriptExecutor._buildStandardCommand (   self)
privateinherited

Definition at line 705 of file trfExe.py.

705  def _buildStandardCommand(self):
706  if self._exe:
707  self._cmd = [self.exe, ]
708  else:
709  raise trfExceptions.TransformExecutionException(trfExit.nameToCode('TRF_EXEC_SETUP_FAIL'),
710  'No executor set in {0}'.format(self.__class__.__name__))
711  for arg in self.exeArgs:
712  if arg in self.conf.argdict:
713  # If we have a list then add each element to our list, else just str() the argument value
714  # Note if there are arguments which need more complex transformations then
715  # consider introducing a special toExeArg() method.
716  if isinstance(self.conf.argdict[arg].value, list):
717  self._cmd.extend([ str(v) for v in self.conf.argdict[arg].value])
718  else:
719  self._cmd.append(str(self.conf.argdict[arg].value))
720 
721 

◆ athenaMP()

def python.trfExe.transformExecutor.athenaMP (   self)
inherited

Definition at line 451 of file trfExe.py.

451  def athenaMP(self):
452  return self._athenaMP
453 

◆ cpuTime()

def python.trfExe.transformExecutor.cpuTime (   self)
inherited

Definition at line 365 of file trfExe.py.

365  def cpuTime(self):
366  if self._exeStart and self._exeStop:
367  return calcCpuTime(self._exeStart, self._exeStop)
368  else:
369  return None
370 

◆ cpuTimeTotal()

def python.trfExe.transformExecutor.cpuTimeTotal (   self)
inherited

Definition at line 429 of file trfExe.py.

429  def cpuTimeTotal(self):
430  if self._preExeStart and self._valStop:
431  return calcCpuTime(self._preExeStart, self._valStop)
432  else:
433  return None
434 

◆ dbMonitor()

def python.trfExe.transformExecutor.dbMonitor (   self)
inherited

Definition at line 455 of file trfExe.py.

455  def dbMonitor(self):
456  return self._dbMonitor
457 
458 

◆ doAll()

def python.trfExe.transformExecutor.doAll (   self,
  input = set(),
  output = set() 
)
inherited

Convenience function.

Definition at line 498 of file trfExe.py.

498  def doAll(self, input=set(), output=set()):
499  self.preExecute(input, output)
500  self.execute()
501  self.postExecute()
502  self.validate()
503 

◆ errMsg()

def python.trfExe.transformExecutor.errMsg (   self)
inherited

Definition at line 303 of file trfExe.py.

303  def errMsg(self):
304  return self._errMsg
305 

◆ eventCount()

def python.trfExe.transformExecutor.eventCount (   self)
inherited

Definition at line 443 of file trfExe.py.

443  def eventCount(self):
444  return self._eventCount
445 

◆ exe() [1/2]

def python.trfExe.scriptExecutor.exe (   self)
inherited

Definition at line 640 of file trfExe.py.

640  def exe(self):
641  return self._exe
642 

◆ exe() [2/2]

def python.trfExe.scriptExecutor.exe (   self,
  value 
)
inherited

Definition at line 644 of file trfExe.py.

644  def exe(self, value):
645  self._exe = value
646  self._extraMetadata['script'] = value
647 

◆ exeArgs() [1/2]

def python.trfExe.scriptExecutor.exeArgs (   self)
inherited

Definition at line 649 of file trfExe.py.

649  def exeArgs(self):
650  return self._exeArgs
651 

◆ exeArgs() [2/2]

def python.trfExe.scriptExecutor.exeArgs (   self,
  value 
)
inherited

Definition at line 653 of file trfExe.py.

653  def exeArgs(self, value):
654  self._exeArgs = value
655 # self._extraMetadata['scriptArgs'] = value
656 

◆ execute()

def python.trfExe.bsMergeExecutor.execute (   self)

Reimplemented from python.trfExe.scriptExecutor.

Definition at line 2083 of file trfExe.py.

2083  def execute(self):
2084  if self._useStubFile:
2085  # Need to fake execution!
2086  self._exeStart = os.times()
2087  msg.debug('exeStart time is {0}'.format(self._exeStart))
2088  msg.info("Using stub file for empty BS output - execution is fake")
2089  if self._outputFilename != self.conf.argdict['emptyStubFile'].value:
2090  os.rename(self.conf.argdict['emptyStubFile'].value, self._outputFilename)
2091  self._memMonitor = False
2092  self._hasExecuted = True
2093  self._rc = 0
2094  self._exeStop = os.times()
2095  msg.debug('exeStop time is {0}'.format(self._exeStop))
2096  else:
2097  super(bsMergeExecutor, self).execute()
2098 

◆ exeStartTimes()

def python.trfExe.transformExecutor.exeStartTimes (   self)
inherited

Definition at line 335 of file trfExe.py.

335  def exeStartTimes(self):
336  return self._exeStart
337 

◆ exeStopTimes()

def python.trfExe.transformExecutor.exeStopTimes (   self)
inherited

Definition at line 339 of file trfExe.py.

339  def exeStopTimes(self):
340  return self._exeStop
341 

◆ extraMetadata()

def python.trfExe.transformExecutor.extraMetadata (   self)
inherited

Definition at line 291 of file trfExe.py.

291  def extraMetadata(self):
292  return self._extraMetadata
293 

◆ first()

def python.trfExe.transformExecutor.first (   self)
inherited
Note
At the moment only athenaExecutor sets this property, but that might be changed...

Definition at line 324 of file trfExe.py.

324  def first(self):
325  if hasattr(self, '_first'):
326  return self._first
327  else:
328  return None
329 

◆ hasExecuted()

def python.trfExe.transformExecutor.hasExecuted (   self)
inherited

Definition at line 295 of file trfExe.py.

295  def hasExecuted(self):
296  return self._hasExecuted
297 

◆ hasValidated()

def python.trfExe.transformExecutor.hasValidated (   self)
inherited

Definition at line 315 of file trfExe.py.

315  def hasValidated(self):
316  return self._hasValidated
317 

◆ inData() [1/2]

def python.trfExe.transformExecutor.inData (   self)
inherited

Definition at line 234 of file trfExe.py.

234  def inData(self):
235 
236  if '_inData' in dir(self):
237  return self._inData
238  return None
239 

◆ inData() [2/2]

def python.trfExe.transformExecutor.inData (   self,
  value 
)
inherited

Definition at line 241 of file trfExe.py.

241  def inData(self, value):
242  self._inData = set(value)
243 

◆ inDataUpdate()

def python.trfExe.transformExecutor.inDataUpdate (   self,
  value 
)
inherited

Definition at line 244 of file trfExe.py.

244  def inDataUpdate(self, value):
245 
246  if '_inData' in dir(self):
247  self._inData.update(value)
248  else:
249 
250  self.inData = value
251 
252 

◆ input()

def python.trfExe.transformExecutor.input (   self)
inherited
Note
This returns the actual input data with which this executor ran (c.f. inData which returns all the possible data types this executor could run with)

Definition at line 275 of file trfExe.py.

275  def input(self):
276 
277  if '_input' in dir(self):
278  return self._input
279  return None
280 

◆ isValidated()

def python.trfExe.transformExecutor.isValidated (   self)
inherited

Definition at line 319 of file trfExe.py.

319  def isValidated(self):
320  return self._isValidated
321 

◆ memAnalysis()

def python.trfExe.transformExecutor.memAnalysis (   self)
inherited

Definition at line 397 of file trfExe.py.

397  def memAnalysis(self):
398  return self._memLeakResult
399 

◆ memStats()

def python.trfExe.transformExecutor.memStats (   self)
inherited

Definition at line 393 of file trfExe.py.

393  def memStats(self):
394  return self._memStats
395 

◆ myMerger()

def python.trfExe.transformExecutor.myMerger (   self)
inherited

Now define properties for these data members.

Definition at line 206 of file trfExe.py.

206  def myMerger(self):
207  return self._myMerger
208 

◆ name() [1/2]

def python.trfExe.transformExecutor.name (   self)
inherited

Definition at line 210 of file trfExe.py.

210  def name(self):
211  return self._name
212 

◆ name() [2/2]

def python.trfExe.transformExecutor.name (   self,
  value 
)
inherited

Definition at line 214 of file trfExe.py.

214  def name(self, value):
215  self._name = value
216 

◆ outData() [1/2]

def python.trfExe.transformExecutor.outData (   self)
inherited

Definition at line 254 of file trfExe.py.

254  def outData(self):
255 
256  if '_outData' in dir(self):
257  return self._outData
258  return None
259 

◆ outData() [2/2]

def python.trfExe.transformExecutor.outData (   self,
  value 
)
inherited

Definition at line 261 of file trfExe.py.

261  def outData(self, value):
262  self._outData = set(value)
263 

◆ outDataUpdate()

def python.trfExe.transformExecutor.outDataUpdate (   self,
  value 
)
inherited

Definition at line 264 of file trfExe.py.

264  def outDataUpdate(self, value):
265 
266  if '_outData' in dir(self):
267  self._outData.update(value)
268  else:
269 
270  self.outData = value
271 

◆ output()

def python.trfExe.transformExecutor.output (   self)
inherited
Note
This returns the actual output data with which this executor ran (c.f. outData which returns all the possible data types this executor could run with)

Definition at line 284 of file trfExe.py.

284  def output(self):
285 
286  if '_output' in dir(self):
287  return self._output
288  return None
289 

◆ postExeCpuTime()

def python.trfExe.transformExecutor.postExeCpuTime (   self)
inherited

Definition at line 401 of file trfExe.py.

401  def postExeCpuTime(self):
402  if self._exeStop and self._valStart:
403  return calcCpuTime(self._exeStop, self._valStart)
404  else:
405  return None
406 

◆ postExecute()

def python.trfExe.bsMergeExecutor.postExecute (   self)

Reimplemented from python.trfExe.scriptExecutor.

Definition at line 2099 of file trfExe.py.

2099  def postExecute(self):
2100  if self._useStubFile:
2101  pass
2102  elif self._doRename:
2103  self._expectedOutput = self._outputFilename + '._0001.data'
2104  msg.info('Renaming {0} to {1}'.format(self._expectedOutput, self.conf.dataDictionary[self._outputBS].value[0]))
2105  try:
2106  os.rename(self._outputFilename + '._0001.data', self.conf.dataDictionary[self._outputBS].value[0])
2107  except OSError as e:
2108  raise trfExceptions.TransformExecutionException(trfExit.nameToCode('TRF_OUTPUT_FILE_ERROR'),
2109  'Exception raised when renaming {0} to {1}: {2}'.format(self._outputFilename, self.conf.dataDictionary[self._outputBS].value[0], e))
2110  super(bsMergeExecutor, self).postExecute()
2111 
2112 

◆ postExeWallTime()

def python.trfExe.transformExecutor.postExeWallTime (   self)
inherited

Definition at line 408 of file trfExe.py.

408  def postExeWallTime(self):
409  if self._exeStop and self._valStart:
410  return calcWallTime(self._exeStop, self._valStart)
411  else:
412  return None
413 

◆ preExeCpuTime()

def python.trfExe.transformExecutor.preExeCpuTime (   self)
inherited

Definition at line 351 of file trfExe.py.

351  def preExeCpuTime(self):
352  if self._preExeStart and self._exeStart:
353  return calcCpuTime(self._preExeStart, self._exeStart)
354  else:
355  return None
356 

◆ preExecute()

def python.trfExe.bsMergeExecutor.preExecute (   self,
  input = set(),
  output = set() 
)

Reimplemented from python.trfExe.scriptExecutor.

Definition at line 2027 of file trfExe.py.

2027  def preExecute(self, input = set(), output = set()):
2028  self.setPreExeStart()
2029  self._inputBS = list(input)[0]
2030  self._outputBS = list(output)[0]
2031  self._maskedFiles = []
2032  self._useStubFile = False
2033  if 'maskEmptyInputs' in self.conf.argdict and self.conf.argdict['maskEmptyInputs'].value is True:
2034  eventfullFiles = []
2035  for fname in self.conf.dataDictionary[self._inputBS].value:
2036  nEvents = self.conf.dataDictionary[self._inputBS].getSingleMetadata(fname, 'nentries')
2037  msg.debug('Found {0} events in file {1}'.format(nEvents, fname))
2038  if isinstance(nEvents, int) and nEvents > 0:
2039  eventfullFiles.append(fname)
2040  self._maskedFiles = list(set(self.conf.dataDictionary[self._inputBS].value) - set(eventfullFiles))
2041  if len(self._maskedFiles) > 0:
2042  msg.info('The following input files are masked because they have 0 events: {0}'.format(' '.join(self._maskedFiles)))
2043  if len(eventfullFiles) == 0:
2044  if 'emptyStubFile' in self.conf.argdict and path.exists(self.conf.argdict['emptyStubFile'].value):
2045  self._useStubFile = True
2046  msg.info("All input files are empty - will use stub file {0} as output".format(self.conf.argdict['emptyStubFile'].value))
2047  else:
2048  raise trfExceptions.TransformExecutionException(trfExit.nameToCode('TRF_INPUT_FILE_ERROR'),
2049  'All input files had zero events - aborting BS merge')
2050 
2051  # Write the list of input files to a text file, so that testMergedFiles can swallow it
2052  self._mergeBSFileList = '{0}.list'.format(self._name)
2053  self._mergeBSLogfile = '{0}.out'.format(self._name)
2054  try:
2055  with open(self._mergeBSFileList, 'w') as BSFileList:
2056  for fname in self.conf.dataDictionary[self._inputBS].value:
2057  if fname not in self._maskedFiles:
2058  print(fname, file=BSFileList)
2059  except OSError as e:
2060  errMsg = 'Got an error when writing list of BS files to {0}: {1}'.format(self._mergeBSFileList, e)
2061  msg.error(errMsg)
2062  raise trfExceptions.TransformExecutionException(trfExit.nameToCode('TRF_EXEC_SETUP_WRAPPER'), errMsg)
2063 
2064  # Hope that we were given a correct filename...
2065  self._outputFilename = self.conf.dataDictionary[self._outputBS].value[0]
2066  if self._outputFilename.endswith('._0001.data'):
2067  self._doRename = False
2068  self._outputFilename = self._outputFilename.split('._0001.data')[0]
2069  elif self.conf.argdict['allowRename'].value is True:
2070  # OK, non-fatal, we go for a renaming
2071  msg.info('Output filename does not end in "._0001.data" will proceed, but be aware that the internal filename metadata will be wrong')
2072  self._doRename = True
2073  else:
2074  # No rename allowed, so we are dead...
2075  errmsg = 'Output filename for outputBS_MRGFile must end in "._0001.data" or infile metadata will be wrong'
2076  raise trfExceptions.TransformExecutionException(trfExit.nameToCode('TRF_OUTPUT_FILE_ERROR'), errmsg)
2077 
2078  # Set the correct command for execution
2079  self._cmd = [self._exe, self._mergeBSFileList, '0', self._outputFilename]
2080 
2081  super(bsMergeExecutor, self).preExecute(input=input, output=output)
2082 

◆ preExeStartTimes()

def python.trfExe.transformExecutor.preExeStartTimes (   self)
inherited

Definition at line 331 of file trfExe.py.

331  def preExeStartTimes(self):
332  return self._preExeStart
333 

◆ preExeWallTime()

def python.trfExe.transformExecutor.preExeWallTime (   self)
inherited

Definition at line 358 of file trfExe.py.

358  def preExeWallTime(self):
359  if self._preExeStart and self._exeStart:
360  return calcWallTime(self._preExeStart, self._exeStart)
361  else:
362  return None
363 

◆ rc()

def python.trfExe.transformExecutor.rc (   self)
inherited

Definition at line 299 of file trfExe.py.

299  def rc(self):
300  return self._rc
301 

◆ reSimEvent()

def python.trfExe.transformExecutor.reSimEvent (   self)
inherited

Definition at line 447 of file trfExe.py.

447  def reSimEvent(self):
448  return self._resimevents
449 

◆ setPreExeStart()

def python.trfExe.transformExecutor.setPreExeStart (   self)
inherited

Definition at line 460 of file trfExe.py.

460  def setPreExeStart(self):
461  if self._preExeStart is None:
462  self._preExeStart = os.times()
463  msg.debug('preExeStart time is {0}'.format(self._preExeStart))
464 

◆ setValStart()

def python.trfExe.transformExecutor.setValStart (   self)
inherited

Definition at line 465 of file trfExe.py.

465  def setValStart(self):
466  if self._valStart is None:
467  self._valStart = os.times()
468  msg.debug('valStart time is {0}'.format(self._valStart))
469 

◆ substep()

def python.trfExe.transformExecutor.substep (   self)
inherited

Reimplemented in python.trfExe.athenaExecutor.

Definition at line 218 of file trfExe.py.

218  def substep(self):
219  if '_substep' in dir(self):
220  return self._substep
221  return None
222 

◆ sysTime()

def python.trfExe.transformExecutor.sysTime (   self)
inherited

Definition at line 379 of file trfExe.py.

379  def sysTime(self):
380  if self._exeStart and self._exeStop:
381  return self._exeStop[3] - self._exeStart[3]
382  else:
383  return None
384 

◆ trf() [1/2]

def python.trfExe.transformExecutor.trf (   self)
inherited

Definition at line 224 of file trfExe.py.

224  def trf(self):
225  if '_trf' in dir(self):
226  return self._trf
227  return None
228 

◆ trf() [2/2]

def python.trfExe.transformExecutor.trf (   self,
  value 
)
inherited

Definition at line 230 of file trfExe.py.

230  def trf(self, value):
231  self._trf = value
232 

◆ usrTime()

def python.trfExe.transformExecutor.usrTime (   self)
inherited

Definition at line 372 of file trfExe.py.

372  def usrTime(self):
373  if self._exeStart and self._exeStop:
374  return self._exeStop[2] - self._exeStart[2]
375  else:
376  return None
377 

◆ validate()

def python.trfExe.scriptExecutor.validate (   self)
inherited

Reimplemented from python.trfExe.transformExecutor.

Reimplemented in python.trfExe.DQMergeExecutor, python.trfExe.optionalAthenaExecutor, and python.trfExe.athenaExecutor.

Definition at line 800 of file trfExe.py.

800  def validate(self):
801  if self._valStart is None:
802  self._valStart = os.times()
803  msg.debug('valStart time is {0}'.format(self._valStart))
804  self._hasValidated = True
805 
806 
807  if self._rc == 0:
808  msg.info('Executor {0} validated successfully (return code {1})'.format(self._name, self._rc))
809  self._isValidated = True
810  self._errMsg = ''
811  else:
812  # Want to learn as much as possible from the non-zero code
813  # this is a bit hard in general, although one can do signals.
814  # Probably need to be more specific per exe, i.e., athena non-zero codes
815  self._isValidated = False
816  if self._rc < 0:
817  # Map return codes to what the shell gives (128 + SIGNUM)
818  self._rc = 128 - self._rc
819  if trfExit.codeToSignalname(self._rc) != "":
820  self._errMsg = '{0} got a {1} signal (exit code {2})'.format(self._name, trfExit.codeToSignalname(self._rc), self._rc)
821  else:
822  self._errMsg = 'Non-zero return code from %s (%d)' % (self._name, self._rc)
823  raise trfExceptions.TransformValidationException(trfExit.nameToCode('TRF_EXEC_FAIL'), self._errMsg)
824 
825 
827  if 'checkEventCount' in self.conf.argdict and self.conf.argdict['checkEventCount'].returnMyValue(exe=self) is False:
828  msg.info('Event counting for substep {0} is skipped'.format(self.name))
829  else:
830  checkcount=trfValidation.eventMatch(self)
831  checkcount.decide()
832  self._eventCount = checkcount.eventCount
833  msg.info('Event counting for substep {0} passed'.format(self.name))
834 
835  self._valStop = os.times()
836  msg.debug('valStop time is {0}'.format(self._valStop))
837 
838 
839 

◆ validation() [1/2]

def python.trfExe.transformExecutor.validation (   self)
inherited

Definition at line 307 of file trfExe.py.

307  def validation(self):
308  return self._validation
309 

◆ validation() [2/2]

def python.trfExe.transformExecutor.validation (   self,
  value 
)
inherited

Definition at line 311 of file trfExe.py.

311  def validation(self, value):
312  self._validation = value
313 

◆ validationCpuTime()

def python.trfExe.transformExecutor.validationCpuTime (   self)
inherited

Definition at line 415 of file trfExe.py.

415  def validationCpuTime(self):
416  if self._valStart and self._valStop:
417  return calcCpuTime(self._valStart, self._valStop)
418  else:
419  return None
420 

◆ validationWallTime()

def python.trfExe.transformExecutor.validationWallTime (   self)
inherited

Definition at line 422 of file trfExe.py.

422  def validationWallTime(self):
423  if self._valStart and self._valStop:
424  return calcWallTime(self._valStart, self._valStop)
425  else:
426  return None
427 

◆ valStartTimes()

def python.trfExe.transformExecutor.valStartTimes (   self)
inherited

Definition at line 343 of file trfExe.py.

343  def valStartTimes(self):
344  return self._valStart
345 

◆ valStopTimes()

def python.trfExe.transformExecutor.valStopTimes (   self)
inherited

Definition at line 347 of file trfExe.py.

347  def valStopTimes(self):
348  return self._valStop
349 

◆ wallTime()

def python.trfExe.transformExecutor.wallTime (   self)
inherited

Definition at line 386 of file trfExe.py.

386  def wallTime(self):
387  if self._exeStart and self._exeStop:
388  return calcWallTime(self._exeStart, self._exeStop)
389  else:
390  return None
391 

◆ wallTimeTotal()

def python.trfExe.transformExecutor.wallTimeTotal (   self)
inherited

Definition at line 436 of file trfExe.py.

436  def wallTimeTotal(self):
437  if self._preExeStart and self._valStop:
438  return calcWallTime(self._preExeStart, self._valStop)
439  else:
440  return None
441 

Member Data Documentation

◆ _alreadyInContainer

python.trfExe.transformExecutor._alreadyInContainer
privateinherited

Definition at line 197 of file trfExe.py.

◆ _athenaConcurrentEvents

python.trfExe.transformExecutor._athenaConcurrentEvents
privateinherited

Definition at line 194 of file trfExe.py.

◆ _athenaMP

python.trfExe.transformExecutor._athenaMP
privateinherited

Definition at line 192 of file trfExe.py.

◆ _athenaMT

python.trfExe.transformExecutor._athenaMT
privateinherited

Definition at line 193 of file trfExe.py.

◆ _cmd

python.trfExe.bsMergeExecutor._cmd
private

Definition at line 2079 of file trfExe.py.

◆ _containerSetup

python.trfExe.transformExecutor._containerSetup
privateinherited

Definition at line 198 of file trfExe.py.

◆ _dbMonitor

python.trfExe.transformExecutor._dbMonitor
privateinherited

Definition at line 195 of file trfExe.py.

◆ _doRename

python.trfExe.bsMergeExecutor._doRename
private

Definition at line 2067 of file trfExe.py.

◆ _echologger

python.trfExe.scriptExecutor._echologger
privateinherited

Definition at line 691 of file trfExe.py.

◆ _echoOutput

python.trfExe.scriptExecutor._echoOutput
privateinherited
Note
Query the environment for echo configuration Let the manual envars always win over auto-detected settings

Definition at line 631 of file trfExe.py.

◆ _echostream

python.trfExe.scriptExecutor._echostream
privateinherited

Definition at line 701 of file trfExe.py.

◆ _errMsg

python.trfExe.scriptExecutor._errMsg
privateinherited

Definition at line 810 of file trfExe.py.

◆ _eventCount

python.trfExe.scriptExecutor._eventCount
privateinherited

Check event counts (always do this by default) Do this here so that all script executors have this by default (covers most use cases with events)

Definition at line 832 of file trfExe.py.

◆ _exe

python.trfExe.scriptExecutor._exe
privateinherited

Definition at line 621 of file trfExe.py.

◆ _exeArgs

python.trfExe.scriptExecutor._exeArgs
privateinherited

Definition at line 624 of file trfExe.py.

◆ _exeLogFile

python.trfExe.scriptExecutor._exeLogFile
privateinherited

Definition at line 696 of file trfExe.py.

◆ _exeStart

python.trfExe.bsMergeExecutor._exeStart
private

Definition at line 2086 of file trfExe.py.

◆ _exeStop

python.trfExe.bsMergeExecutor._exeStop
private

Definition at line 2094 of file trfExe.py.

◆ _expectedOutput

python.trfExe.bsMergeExecutor._expectedOutput
private

Definition at line 2103 of file trfExe.py.

◆ _extraMetadata

python.trfExe.transformExecutor._extraMetadata
privateinherited

Definition at line 180 of file trfExe.py.

◆ _hasExecuted

python.trfExe.bsMergeExecutor._hasExecuted
private

Definition at line 2092 of file trfExe.py.

◆ _hasValidated

python.trfExe.scriptExecutor._hasValidated
privateinherited

Definition at line 804 of file trfExe.py.

◆ _inData

python.trfExe.transformExecutor._inData
privateinherited

Definition at line 143 of file trfExe.py.

◆ _input

python.trfExe.scriptExecutor._input
privateinherited

Definition at line 661 of file trfExe.py.

◆ _inputBS

python.trfExe.bsMergeExecutor._inputBS
private

Definition at line 2029 of file trfExe.py.

◆ _isValidated

python.trfExe.scriptExecutor._isValidated
privateinherited

Definition at line 809 of file trfExe.py.

◆ _logFileName

python.trfExe.scriptExecutor._logFileName
privateinherited
Note
If an inherited class has set self._cmd leave it alone

Definition at line 670 of file trfExe.py.

◆ _maskedFiles

python.trfExe.bsMergeExecutor._maskedFiles
private

Definition at line 2031 of file trfExe.py.

◆ _memFullFile

python.trfExe.scriptExecutor._memFullFile
privateinherited

Definition at line 749 of file trfExe.py.

◆ _memLeakResult

python.trfExe.transformExecutor._memLeakResult
privateinherited

Definition at line 189 of file trfExe.py.

◆ _memMonitor

python.trfExe.bsMergeExecutor._memMonitor
private

Definition at line 2091 of file trfExe.py.

◆ _memStats

python.trfExe.scriptExecutor._memStats
privateinherited

Definition at line 793 of file trfExe.py.

◆ _memSummaryFile

python.trfExe.scriptExecutor._memSummaryFile
privateinherited

Definition at line 748 of file trfExe.py.

◆ _mergeBSFileList

python.trfExe.bsMergeExecutor._mergeBSFileList
private

Definition at line 2052 of file trfExe.py.

◆ _mergeBSLogfile

python.trfExe.bsMergeExecutor._mergeBSLogfile
private

Definition at line 2053 of file trfExe.py.

◆ _myMerger

python.trfExe.transformExecutor._myMerger
privateinherited

Definition at line 201 of file trfExe.py.

◆ _name

python.trfExe.transformExecutor._name
privateinherited

Definition at line 140 of file trfExe.py.

◆ _outData

python.trfExe.transformExecutor._outData
privateinherited

Definition at line 144 of file trfExe.py.

◆ _output

python.trfExe.scriptExecutor._output
privateinherited

Definition at line 662 of file trfExe.py.

◆ _outputBS

python.trfExe.bsMergeExecutor._outputBS
private

Definition at line 2030 of file trfExe.py.

◆ _outputFilename

python.trfExe.bsMergeExecutor._outputFilename
private

Definition at line 2065 of file trfExe.py.

◆ _preExeStart

python.trfExe.transformExecutor._preExeStart
privateinherited
Note
Place holders for resource consumption. CPU and walltime are available for all executors but currently only athena is instrumented to fill in memory stats (and then only if PerfMonSD is enabled).

Definition at line 185 of file trfExe.py.

◆ _rc

python.trfExe.bsMergeExecutor._rc
private

Definition at line 2093 of file trfExe.py.

◆ _resimevents

python.trfExe.transformExecutor._resimevents
privateinherited

Definition at line 196 of file trfExe.py.

◆ _trf

python.trfExe.transformExecutor._trf
privateinherited

Definition at line 231 of file trfExe.py.

◆ _useStubFile

python.trfExe.bsMergeExecutor._useStubFile
private

Definition at line 2032 of file trfExe.py.

◆ _validation

python.trfExe.transformExecutor._validation
privateinherited

Definition at line 312 of file trfExe.py.

◆ _valStart

python.trfExe.scriptExecutor._valStart
privateinherited

Definition at line 802 of file trfExe.py.

◆ _valStop

python.trfExe.scriptExecutor._valStop
privateinherited

Definition at line 835 of file trfExe.py.

◆ conf

python.trfExe.transformExecutor.conf
inherited

Executor configuration:

Note
that if conf and trf are None then we'll probably set the conf up later (this is allowed and expected to be done once the master transform has figured out what it's doing for this job)

Definition at line 162 of file trfExe.py.

◆ inData

python.trfExe.transformExecutor.inData
inherited
Note
Protect against _inData not yet being defined
Use normal setter

Definition at line 250 of file trfExe.py.

◆ outData

python.trfExe.transformExecutor.outData
inherited
Note
Protect against _outData not yet being defined
Use normal setter

Definition at line 270 of file trfExe.py.


The documentation for this class was generated from the following file:
validation
Definition: validation.py:1
validation.validate
def validate(testSampleDir, thisSampleName, testSamplePath, weight_database, outputSamples)
Definition: validation.py:26
vtune_athena.format
format
Definition: vtune_athena.py:14
python.trfUtils.calcCpuTime
def calcCpuTime(start, stop)
Definition: trfUtils.py:1683
dumpHVPathFromNtuple.append
bool append
Definition: dumpHVPathFromNtuple.py:91
LArG4FSStartPointFilterLegacy.execute
execute
Definition: LArG4FSStartPointFilterLegacy.py:20
PlotPulseshapeFromCool.input
input
Definition: PlotPulseshapeFromCool.py:106
histSizes.list
def list(name, path='/')
Definition: histSizes.py:38
beamspotman.dir
string dir
Definition: beamspotman.py:623
CxxUtils::set
constexpr std::enable_if_t< is_bitmask_v< E >, E & > set(E &lhs, E rhs)
Convenience function to set bits in a class enum bitmask.
Definition: bitmask.h:224
merge.output
output
Definition: merge.py:17
TCS::join
std::string join(const std::vector< std::string > &v, const char c=',')
Definition: Trigger/TrigT1/L1Topo/L1TopoCommon/Root/StringUtils.cxx:10
name
std::string name
Definition: Control/AthContainers/Root/debug.cxx:195
python.trfUtils.calcWallTime
def calcWallTime(start, stop)
Definition: trfUtils.py:1691
Trk::open
@ open
Definition: BinningType.h:40
dqt_zlumi_pandas.update
update
Definition: dqt_zlumi_pandas.py:42
DeMoScan.first
bool first
Definition: DeMoScan.py:534
Muon::print
std::string print(const MuPatSegment &)
Definition: MuonTrackSteering.cxx:28
str
Definition: BTagTrackIpAccessor.cxx:11
CaloLCW_tf.trf
trf
Definition: CaloLCW_tf.py:20
Trk::split
@ split
Definition: LayerMaterialProperties.h:38