EL::Detail::AlgorithmStateModule Class Reference

a Module managing the state of the algorithms on the workers More...

#include <AlgorithmStateModule.h>

Inheritance diagram for EL::Detail::AlgorithmStateModule:

Collaboration diagram for EL::Detail::AlgorithmStateModule:

Public Member Functions
virtual StatusCode	onInitialize (ModuleData &data) override
	action just before algorithms are initialized
virtual StatusCode	onFinalize (ModuleData &data) override
	actions just before algorithms are finalized
virtual StatusCode	onCloseInputFile (ModuleData &data) override
	actions before closing an input file
virtual StatusCode	onNewInputFile (ModuleData &data) override
	actions after opening a new input file
virtual StatusCode	onFileExecute (ModuleData &data) override
	actions just before fileExecute is called on algorithms
virtual StatusCode	onExecute (ModuleData &data) override
	actions just before execute is called on algorithms
virtual StatusCode	firstInitialize (ModuleData &data)
	action at the the very beginning of the worker job
virtual StatusCode	preFileInitialize (ModuleData &data)
	action before opening the first file in the worker job
virtual StatusCode	processInputs (ModuleData &data, IInputModuleActions &actions)
	process all input files
virtual StatusCode	postFirstEvent (ModuleData &data)
	action after processing first event
virtual StatusCode	onFirstInputFile (ModuleData &data)
	actions after opening a the first input file
virtual StatusCode	onNextInputFile (ModuleData &data)
	actions after opening an input file after the first one
virtual StatusCode	postCloseInputFile (ModuleData &data)
	actions after CloseInputFile is called on the algorithms
virtual StatusCode	postFinalize (ModuleData &data)
	actions after algorithms have been finalized
virtual StatusCode	onWorkerEnd (ModuleData &data)
	action at the end of the worker job
virtual StatusCode	postFileClose (ModuleData &data)
	action at end of the worker job
virtual void	reportInputFailure (ModuleData &data)
	report that we failed to open our input file

Private Attributes
bool	m_initialized {false}
	whether Algorithm::initialize has been called

Detailed Description

a Module managing the state of the algorithms on the workers

\warn This module ought to be the last module loaded. That ensures that the algorithms get to do their thing after the modules. If a module needs to act after the algorithms, a new callback should be added for that.

The algorithms require actually a fairly complicated state machine to maintain. Mostly because the functions of the Algorithm class have to be called in a specific order, and each needs to have a specific set of pre-conditions fulfilled. This logic used to be (before 19 Jan 19) in the Worker class itself, but I decided it is better to factor it out into a separate module.

Definition at line 36 of file AlgorithmStateModule.h.

Member Function Documentation

firstInitialize()

StatusCode EL::Detail::Module::firstInitialize ( ModuleData & data )

virtualinherited

action at the the very beginning of the worker job

This gets called as early as possible in the worker initialization. Essentially all that should happen before this is to load all the modules. The main purpose is to start any benchmarks that are meant to capture the initialization process as well.

Reimplemented in EL::Detail::AlgorithmMemoryModule, EL::Detail::AlgorithmTimerModule, EL::Detail::MemoryMonitorModule, and EL::Detail::StopwatchModule.

Definition at line 25 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

onCloseInputFile()

StatusCode EL::Detail::AlgorithmStateModule::onCloseInputFile ( ModuleData & data )

overridevirtual

actions before closing an input file

Reimplemented from EL::Detail::Module.

Definition at line 96 of file AlgorithmStateModule.cxx.

    {
      return forAllAlgorithms (msg(), data, "endInputFile", [&] (AlgorithmData& alg) {
          return alg->endInputFile ();});
    }

onExecute()

StatusCode EL::Detail::AlgorithmStateModule::onExecute ( ModuleData & data )

overridevirtual

actions just before execute is called on algorithms

For now that is mostly used to point input modules to the right event

Reimplemented from EL::Detail::Module.

Definition at line 134 of file AlgorithmStateModule.cxx.

    {
      data.m_skipEvent = false;
      bool sequenceSkip = false;
      for (auto& algData : data.m_algs)
      {
        try
        {
          if (algData.m_sequenceStart)
            sequenceSkip = false;
          else if (sequenceSkip)
          {
            algData.m_wasSkipped = true;
            continue;
          }
 
          algData.m_executeCount += 1;
          if (algData.m_algorithm->execute() == StatusCode::FAILURE)
          {
            ANA_MSG_ERROR ("while calling execute() on algorithm " << algData.m_algorithm->getName());
            return StatusCode::FAILURE;
          }
 
          if (data.m_skipEvent)
          {
            algData.m_skipCount += 1;
            algData.m_wasSkipped = true;
            sequenceSkip = true;
            data.m_skipEvent = false;
          }
        } catch (...)
        {
          Detail::report_exception (std::current_exception());
          ANA_MSG_ERROR ("while calling execute() on algorithm " << algData.m_algorithm->getName());
          return StatusCode::FAILURE;
        }
      }
 
      for (auto& algData : data.m_algs)
      {
        try
        {
          // This will skip `postExecute` for all algorithms that called
          // `setFilterPassed(false)` or that were skipped because of a
          // prior algorithm calling `setFilterPassed(false)`.
          if (algData.m_wasSkipped)
          {
            algData.m_wasSkipped = false;
            continue;
          }
          if (algData.m_algorithm->postExecute() == StatusCode::FAILURE)
          {
            ANA_MSG_ERROR ("while calling postExecute() on algorithm " << algData.m_algorithm->getName());
            return StatusCode::FAILURE;
          }
        } catch (...)
        {
          Detail::report_exception (std::current_exception());
          ANA_MSG_ERROR ("while calling postExecute() on algorithm " << algData.m_algorithm->getName());
          return StatusCode::FAILURE;
        }
      }
 
      return StatusCode::SUCCESS;
    }

onFileExecute()

StatusCode EL::Detail::AlgorithmStateModule::onFileExecute ( ModuleData & data )

overridevirtual

actions just before fileExecute is called on algorithms

Reimplemented from EL::Detail::Module.

Definition at line 125 of file AlgorithmStateModule.cxx.

    {
      return forAllAlgorithms (msg(), data, "fileExecute", [&] (AlgorithmData& alg) {
          return alg->fileExecute ();});
    }

onFinalize()

StatusCode EL::Detail::AlgorithmStateModule::onFinalize ( ModuleData & data )

overridevirtual

actions just before algorithms are finalized

Reimplemented from EL::Detail::Module.

Definition at line 83 of file AlgorithmStateModule.cxx.

    {
      if (!m_initialized)
        return StatusCode::SUCCESS;
      if (forAllAlgorithms (msg(), data, "finalize", [&] (AlgorithmData& alg) {
            return alg->finalize ();}).isFailure())
        return StatusCode::FAILURE;
      return StatusCode::SUCCESS;
    }

onFirstInputFile()

StatusCode EL::Detail::Module::onFirstInputFile ( ModuleData & data )

virtualinherited

actions after opening a the first input file

Reimplemented in EL::Detail::EventModule.

Definition at line 37 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

onInitialize()

StatusCode EL::Detail::AlgorithmStateModule::onInitialize ( ModuleData & data )

overridevirtual

action just before algorithms are initialized

This is typically used for any setup that this module needs to do.

Reimplemented from EL::Detail::Module.

Definition at line 62 of file AlgorithmStateModule.cxx.

    {
      if (m_initialized)
      {
        ANA_MSG_ERROR ("getting second initialize call");
        return StatusCode::FAILURE;
      }
      m_initialized = true;
      AlgorithmWorkerData workerData;
      workerData.m_histogramWorker = data.m_worker;
      workerData.m_treeWorker = data.m_worker;
      workerData.m_filterWorker = data.m_worker;
      workerData.m_wk = data.m_worker;
      workerData.m_evtStore = data.m_evtStore;
      return forAllAlgorithms (msg(), data, "initialize", [&] (AlgorithmData& alg) {
        return alg->initialize (workerData);});
    }

onNewInputFile()

StatusCode EL::Detail::AlgorithmStateModule::onNewInputFile ( ModuleData & data )

overridevirtual

actions after opening a new input file

Reimplemented from EL::Detail::Module.

Definition at line 105 of file AlgorithmStateModule.cxx.

    {
      if (!m_initialized)
      {
        ANA_MSG_ERROR ("algorithms have not been initialized yet");
        return StatusCode::FAILURE;
      }
 
      // Check that there are events on input
      if (!data.m_hasInputEvents) return StatusCode::SUCCESS;
 
      if (forAllAlgorithms (msg(), data, "changeInput", [&] (AlgorithmData& alg) {
            return alg->beginInputFile ();}).isFailure())
        return StatusCode::FAILURE;
      return StatusCode::SUCCESS;
    }

onNextInputFile()

StatusCode EL::Detail::Module::onNextInputFile ( ModuleData & data )

virtualinherited

actions after opening an input file after the first one

Reimplemented in EL::Detail::EventModule.

Definition at line 43 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

onWorkerEnd()

StatusCode EL::Detail::Module::onWorkerEnd ( ModuleData & data )

virtualinherited

action at the end of the worker job

This is mostly meant/used to print job summary statements at the very end of worker job, and actually have them show up at or at least near the end of any log file.

Reimplemented in EL::Detail::LeakCheckModule, EL::Detail::MemoryMonitorModule, and EL::Detail::StopwatchModule.

Definition at line 113 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

postCloseInputFile()

StatusCode EL::Detail::Module::postCloseInputFile ( ModuleData & data )

virtualinherited

actions after CloseInputFile is called on the algorithms

Right now that is only used to disconnect the Event object from the input file.

Reimplemented in EL::Detail::EventModule.

Definition at line 61 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

postFileClose()

StatusCode EL::Detail::Module::postFileClose ( ModuleData & data )

virtualinherited

action at end of the worker job

Executed just right before worker exit successfully at this stage all outputs have been created

Reimplemented in EL::Detail::GridReportingModule, and EL::Detail::PostClosedOutputsModule.

Definition at line 119 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

postFinalize()

StatusCode EL::Detail::Module::postFinalize ( ModuleData & data )

virtualinherited

actions after algorithms have been finalized

This is usually used to calculate some job summary information that is to be stored it in the histogram output file.

Reimplemented in EL::Detail::EventCountModule, EL::Detail::EventModule, EL::Detail::FileExecutedModule, EL::Detail::LeakCheckModule, and EL::Detail::StopwatchModule.

Definition at line 107 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

postFirstEvent()

StatusCode EL::Detail::Module::postFirstEvent ( ModuleData & data )

virtualinherited

action after processing first event

This is mostly meant to set up benchmarks that record per-event performance. While a lot of initialization happens during initialize() there is a fair amount of initialization that happens on the first event, so when recording per-event performance the first event is sort of "special" and may need to be omitted.

Reimplemented in EL::Detail::LeakCheckModule, and EL::Detail::MemoryMonitorModule.

Definition at line 67 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

preFileInitialize()

StatusCode EL::Detail::Module::preFileInitialize ( ModuleData & data )

virtualinherited

action before opening the first file in the worker job

This is mostly meant to allow loading the dictionaries before any files and associated information is loaded.

Definition at line 31 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

processInputs()

StatusCode EL::Detail::Module::processInputs ( ModuleData & data, IInputModuleActions & actions )

virtualinherited

process all input files

This deviates slightly from the usual pattern for module functions in that I pass in the possible actions as an argument. See IInputModuleActions for details.

Reimplemented in EL::Detail::BatchInputModule, and EL::Detail::DirectInputModule.

Definition at line 95 of file Module.cxx.

    {
      return ::StatusCode::SUCCESS;
    }

reportInputFailure()

void EL::Detail::Module::reportInputFailure ( ModuleData & data )

virtualinherited

report that we failed to open our input file

Reimplemented in EL::Detail::GridReportingModule.

Definition at line 73 of file Module.cxx.

    {}

Member Data Documentation

m_initialized

bool EL::Detail::AlgorithmStateModule::m_initialized {false}

private

whether Algorithm::initialize has been called

Definition at line 61 of file AlgorithmStateModule.h.

{false};

---

The documentation for this class was generated from the following files:

• AlgorithmStateModule.h
• AlgorithmStateModule.cxx