python.TrigValSteering.CheckSteps.LogMergeStep Class Reference

Inheritance diagram for python.TrigValSteering.CheckSteps.LogMergeStep:

Collaboration diagram for python.TrigValSteering.CheckSteps.LogMergeStep:

Public Member Functions
def	__init__ (self, name='LogMerge')
def	configure (self, test)
def	process_extra_regex (self)
def	merge_logs (self)
def	run (self, dry_run=False)

Public Attributes
	log_files
	extra_log_regex
	merged_name
	warn_if_missing
	result

Detailed Description

Merge several log files into one for post-processing

Definition at line 121 of file CheckSteps.py.

Constructor & Destructor Documentation

__init__()

def python.TrigValSteering.CheckSteps.LogMergeStep.__init__	(		self,
			name = 'LogMerge'
	)

Definition at line 124 of file CheckSteps.py.

    def __init__(self, name='LogMerge'):
        super(LogMergeStep, self).__init__(name)
        self.log_files = None
        self.extra_log_regex = None
        self.merged_name = 'athena.merged.log'
        self.warn_if_missing = True
 

Member Function Documentation

configure()

def python.TrigValSteering.CheckSteps.LogMergeStep.configure	(		self,
			test
	)

Definition at line 131 of file CheckSteps.py.

    def configure(self, test):
        if self.log_files is None:
            self.log_files = []
            for step in test.exec_steps:
                self.log_files.append(step.name)
        # Protect against infinite loop
        if self.merged_name in self.log_files:
            self.misconfig_abort(
                'output log name %s is same as one of the input log names.'
                ' This will lead to infinite loop, aborting.', self.merged_name)
        super(LogMergeStep, self).configure(test)
 

merge_logs()

def python.TrigValSteering.CheckSteps.LogMergeStep.merge_logs

(

self

)

Definition at line 151 of file CheckSteps.py.

    def merge_logs(self):
        try:
            with open(self.merged_name, 'w', encoding='utf-8') as merged_file:
                for log_name in self.log_files:
                    if not os.path.isfile(log_name):
                        if self.warn_if_missing:
                            self.log.warning('Cannot open %s', log_name)
                            merged_file.write(
                                '### WARNING Missing {} ###\n'.format(log_name))
                        continue
                    with open(log_name, encoding='utf-8') as log_file:
                        merged_file.write('### {} ###\n'.format(log_name))
                        # temporary workaround to ignore false positives in AOD->DAOD log parsing
                        # FIXME: drop AODtoDAOD once test_trigAna_AODtoDAOD_run2_build.py is migrated to Derivation_tf
                        if log_name == 'log.Derivation' or log_name == 'log.AODtoDAOD':
                            for line in log_file:
                                merged_file.write(line.replace('Selected dynamic Aux', 'Selected Dynamic Aux'))
                        else:
                            for line in log_file:
                                merged_file.write(line)
            return 0
        except OSError as e:
            self.log.error('%s merging failed due to OSError: %s',
                           self.name, e.strerror)
            return 1
 

process_extra_regex()

def python.TrigValSteering.CheckSteps.LogMergeStep.process_extra_regex

(

self

)

Definition at line 143 of file CheckSteps.py.

    def process_extra_regex(self):
        if self.extra_log_regex:
            files = os.listdir('.')
            r = re.compile(self.extra_log_regex)
            match_files = filter(r.match, files)
            for f in match_files:
                self.log_files.append(f)
 

run()

def python.TrigValSteering.CheckSteps.LogMergeStep.run	(		self,
			dry_run = False
	)

Definition at line 177 of file CheckSteps.py.

    def run(self, dry_run=False):
        self.process_extra_regex()
        # Sort log files by modification time
        self.log_files.sort(key=lambda f : os.path.getmtime(f) if os.path.isfile(f) else 0)
        self.log.info('Running %s merging logs %s into %s',
                      self.name, self.log_files, self.merged_name)
        if dry_run:
            self.result = 0
        else:
            self.result = self.merge_logs()
        return self.result, '# (internal) {} in={} out={}'.format(self.name, self.log_files, self.merged_name)
 
 

Member Data Documentation

extra_log_regex

python.TrigValSteering.CheckSteps.LogMergeStep.extra_log_regex

Definition at line 127 of file CheckSteps.py.

log_files

python.TrigValSteering.CheckSteps.LogMergeStep.log_files

Definition at line 126 of file CheckSteps.py.

merged_name

python.TrigValSteering.CheckSteps.LogMergeStep.merged_name

Definition at line 128 of file CheckSteps.py.

result

python.TrigValSteering.CheckSteps.LogMergeStep.result

Definition at line 184 of file CheckSteps.py.

warn_if_missing

python.TrigValSteering.CheckSteps.LogMergeStep.warn_if_missing

Definition at line 129 of file CheckSteps.py.

---

The documentation for this class was generated from the following file:

• CheckSteps.py