ATLAS Offline Software
Public Member Functions | Public Attributes | List of all members
python.StandardTests.DerivationTest Class Reference
Inheritance diagram for python.StandardTests.DerivationTest:
Collaboration diagram for python.StandardTests.DerivationTest:

Public Member Functions

None __init__ (self, str ID, WorkflowRun run, WorkflowType type, List[str] steps, TestSetup setup, str extra_args="")
 
None run_reference (self)
 
None run_validation (self)
 
bool run_checks (self, List[WorkflowCheck] performance_checks)
 

Public Attributes

 command
 
 skip_performance_checks
 
 output_checks
 
 ID
 
 tag
 
 steps
 
 digest_checks
 
 run
 
 type
 
 setup
 
 logger
 

Detailed Description

Derivations test.

Definition at line 212 of file StandardTests.py.

Constructor & Destructor Documentation

◆ __init__()

None python.StandardTests.DerivationTest.__init__ (   self,
str  ID,
WorkflowRun  run,
WorkflowType  type,
List[str steps,
TestSetup  setup,
str   extra_args = "" 
)

Definition at line 215 of file StandardTests.py.

215  def __init__(self, ID: str, run: WorkflowRun, type: WorkflowType, steps: List[str], setup: TestSetup, extra_args: str = "") -> None:
216  test_def = ID.split("_")
217  data_type = test_def[0].lower()
218  formats = [format.upper() for format in test_def[1:-1]]
219 
220  threads = 0
221  if setup.custom_threads is not None:
222  threads = setup.custom_threads
223 
224  if "maxEvents" not in extra_args:
225  base_events = 100
226  events = threads * base_events + 1
227  flush = 80
228 
229  extra_args += f" --maxEvents {events}"
230  format_flush = ", ".join([f"\"DAOD_{format}\": {flush}" for format in formats])
231  extra_args += f" --preExec 'flags.Output.TreeAutoFlush={{{format_flush}}}'"
232 
233  if "inputAODFile" not in extra_args:
234  extra_args += f" --inputAODFile {input_AOD[run][data_type]}"
235 
236  # could also use p5503
237  self.command = \
238  (f"ATHENA_CORE_NUMBER={threads} Derivation_tf.py"
239  f" --formats {' '.join(formats)}"
240  " --multiprocess --multithreadedFileValidation True"
241  " --athenaMPMergeTargetSize 'DAOD_*:0'"
242  " --sharedWriter True"
243  " --outputDAODFile myOutput.pool.root"
244  f" --imf False {extra_args}")
245 
246  # skip performance checks for now
247  self.skip_performance_checks = True
248 
249  self.output_checks = []
250  for format in formats:
251  self.output_checks.append(FrozenTier0PolicyCheck(setup, f"DAOD_{format}", 10))
252  self.output_checks.append(MetadataCheck(setup, f"DAOD_{format}"))
253 
254  super().__init__(ID, run, type, steps, setup)
255 
256 

Member Function Documentation

◆ run_checks()

bool python.Test.WorkflowTest.run_checks (   self,
List[WorkflowCheck performance_checks 
)
inherited

Definition at line 200 of file Tools/WorkflowTestRunner/python/Test.py.

200  def run_checks(self, performance_checks: List[WorkflowCheck]) -> bool:
201  self.logger.info("-----------------------------------------------------")
202  self.logger.info(f"----------- Post-processing of {self.ID} Test -----------")
203  result = True
204 
205  # digest checks
206  for check in self.digest_checks:
207  result = check.run(self) and result
208 
209  # output checks
210  if not self.setup.disable_output_checks:
211  for check in self.output_checks:
212  result = check.run(self) and result
213 
214  if self.setup.validation_only or self.skip_performance_checks:
215  return result # Performance checks against static references not possible
216 
217  # performance checks
218  for check in performance_checks:
219  result = check.run(self) and result
220 
221  return result

◆ run_reference()

None python.Test.WorkflowTest.run_reference (   self)
inherited

Definition at line 163 of file Tools/WorkflowTestRunner/python/Test.py.

163  def run_reference(self) -> None:
164  self.logger.info(f"Running reference in rel {self.setup.release_reference}")
165  self.logger.info(f"\"{self.command}\"")
166 
167  self.reference_path.mkdir(parents=True, exist_ok=True)
168 
169  cmd = (f"cd {self.reference_path};"
170  f"source $AtlasSetup/scripts/asetup.sh {self.setup.release_reference} >& /dev/null;")
171  cmd += f"TRF_NOECHO=1 {self.command} > {self.ID}.log 2>&1"
172 
173  subprocess.call(cmd, shell=True)
174 
175  self.logger.info(f"Finished clean in rel {self.setup.release_reference}")
176  self.logger.info(f"\"{self.command}\"")
177 

◆ run_validation()

None python.Test.WorkflowTest.run_validation (   self)
inherited

Definition at line 178 of file Tools/WorkflowTestRunner/python/Test.py.

178  def run_validation(self) -> None:
179  self.logger.info(f"Running validation in rel {self.setup.release_validation}")
180  self.logger.info(f"\"{self.command}\"")
181 
182  self.validation_path.mkdir(parents=True, exist_ok=True)
183 
184  cmd = f"cd {self.validation_path};"
185  if self.setup.disable_release_setup or not self.setup.release_validation:
186  pass
187  elif "WorkDir_DIR" in environ:
188  cmake_build_dir = environ["WorkDir_DIR"]
189  cmd += (f"source $AtlasSetup/scripts/asetup.sh {self.setup.release_validation} >& /dev/null;"
190  f"source {cmake_build_dir}/setup.sh;")
191  else:
192  cmd += f"source $AtlasSetup/scripts/asetup.sh {self.setup.release_validation} >& /dev/null;"
193  cmd += f"TRF_NOECHO=1 {self.command} > {self.ID}.log 2>&1"
194 
195  subprocess.call(cmd, shell=True)
196 
197  self.logger.info(f"Finished validation in rel {self.setup.release_validation}")
198  self.logger.info(f"\"{self.command}\"")
199 

Member Data Documentation

◆ command

python.StandardTests.DerivationTest.command

Definition at line 237 of file StandardTests.py.

◆ digest_checks

python.Test.WorkflowTest.digest_checks
inherited

Definition at line 151 of file Tools/WorkflowTestRunner/python/Test.py.

◆ ID

python.Test.WorkflowTest.ID
inherited

Definition at line 136 of file Tools/WorkflowTestRunner/python/Test.py.

◆ logger

python.Test.WorkflowTest.logger
inherited

Definition at line 159 of file Tools/WorkflowTestRunner/python/Test.py.

◆ output_checks

python.StandardTests.DerivationTest.output_checks

Definition at line 249 of file StandardTests.py.

◆ run

python.Test.WorkflowTest.run
inherited

Definition at line 156 of file Tools/WorkflowTestRunner/python/Test.py.

◆ setup

python.Test.WorkflowTest.setup
inherited

Definition at line 158 of file Tools/WorkflowTestRunner/python/Test.py.

◆ skip_performance_checks

python.StandardTests.DerivationTest.skip_performance_checks

Definition at line 247 of file StandardTests.py.

◆ steps

python.Test.WorkflowTest.steps
inherited

Definition at line 142 of file Tools/WorkflowTestRunner/python/Test.py.

◆ tag

python.Test.WorkflowTest.tag
inherited

Definition at line 139 of file Tools/WorkflowTestRunner/python/Test.py.

◆ type

python.Test.WorkflowTest.type
inherited

Definition at line 157 of file Tools/WorkflowTestRunner/python/Test.py.


The documentation for this class was generated from the following file:
plot_material.mkdir
def mkdir(path, recursive=True)
Definition: plot_material.py:15
dumpHVPathFromNtuple.append
bool append
Definition: dumpHVPathFromNtuple.py:91
TCS::join
std::string join(const std::vector< std::string > &v, const char c=',')
Definition: Trigger/TrigT1/L1Topo/L1TopoCommon/Root/StringUtils.cxx:10
python.processes.powheg.ZZ.ZZ.__init__
def __init__(self, base_directory, **kwargs)
Constructor: all process options are set here.
Definition: ZZ.py:18
python.ParticleTypeUtil.info
def info
Definition: ParticleTypeUtil.py:87
python.ScriptUtils.run_checks
bool run_checks(TestSetup setup, List[WorkflowTest] tests, List[WorkflowCheck] performance_checks)
Definition: ScriptUtils.py:269