ATLAS Offline Software
Public Member Functions | Protected Member Functions | Private Types | Private Member Functions | Static Private Member Functions | Private Attributes | List of all members
EL::Worker Class Referencefinal

#include <Worker.h>

Inheritance diagram for EL::Worker:
Collaboration diagram for EL::Worker:

Public Member Functions

void testInvariant () const
 effects: test the invariant of this object guarantee: no-fail More...
 
virtual ~Worker ()
 effects: standard destructor guarantee: no-fail More...
 
void addOutput (TObject *output_swallow) final override
 effects: add an object to the output. More...
 
void addOutputList (const std::string &name, TObject *output_swallow) override
 effects: add a given object to the output. More...
 
TObject * getOutputHist (const std::string &name) const final override
 get the output histogram with the given name More...
 
TFile * getOutputFile (const std::string &label) const override
 effects: get the output file that goes into the dataset with the given label. More...
 
TFile * getOutputFileNull (const std::string &label) const override
 effects: get the output file that goes into the dataset with the given label. More...
 
::StatusCode addTree (const TTree &tree, const std::string &stream) final override
 effects: adds a tree to an output file specified by the stream/label failures: Incorrect stream/label specified, called at the wrong time note: See getOutputFile for failure types... More...
 
TTree * getOutputTree (const std::string &name, const std::string &stream) const final override
 effects: get the tree that was added to an output file earlier failures: Tree doesn't exist More...
 
const SH::MetaObjectmetaData () const override
 description: the sample meta-data we are working on guarantee: no-fail invariant: metaData != 0 rationale: this can be used for accessing sample meta-data More...
 
TTree * tree () const override
 description: the tree we are running on guarantee: no-fail More...
 
Long64_t treeEntry () const override
 description: the entry in the tree we are reading guarantee: no-fail More...
 
TFile * inputFile () const override
 description: the file we are reading the current tree from guarantee: no-fail More...
 
std::string inputFileName () const override
 the name of the file we are reading the current tree from, without the path component More...
 
TTree * triggerConfig () const override
 description: the trigger config tree from the input file, or NULL if we did not find it guarantee: strong failures: i/o errors More...
 
xAOD::TEventxaodEvent () const override
 description: the xAOD event and store guarantee: strong failures: out of memory I failures: TEventSvc not configured postcondition: result != 0 More...
 
xAOD::TStorexaodStore () const override
 
EL::AlgorithmgetAlg (const std::string &name) const override
 effects: returns the algorithms with the given name or NULL if there is none guarantee: strong failures: out of memory II More...
 
void skipEvent () override
 effects: skip the current event, i.e. More...
 
virtual bool filterPassed () const noexcept final override
 whether the current algorithm passed its filter criterion for the current event More...
 
virtual void setFilterPassed (bool val_filterPassed) noexcept final override
 set the value of filterPassed More...
 
 Worker ()
 standard constructor More...
 
::StatusCode directExecute (const SH::SamplePtr &sample, const Job &job, const std::string &location, const SH::MetaObject &options)
 run the job More...
 
::StatusCode batchExecute (unsigned job_id, const char *confFile)
 effects: do what is needed to execute the given job segment guarantee: basic failures: job specific More...
 
::StatusCode gridExecute (const std::string &sampleName, Long64_t SkipEvents, Long64_t nEventsPerJob)
 

Protected Member Functions

void setMetaData (const SH::MetaObject *val_metaData)
 set the metaData More...
 
void setOutputHist (const std::string &val_outputTarget)
 set the histogram output list More...
 
void setSegmentName (const std::string &val_segmentName)
 set the segment name More...
 
void setJobConfig (JobConfig &&jobConfig)
 set the JobConfig More...
 
::StatusCode initialize ()
 initialize the worker More...
 
::StatusCode processInputs ()
 process all the inputs More...
 
::StatusCode finalize ()
 finalize the worker More...
 
::StatusCode processEvents (EventRange &eventRange) override
 process the given event range More...
 
::StatusCode openInputFile (const std::string &inputFileUrl) override
 open the given input file without processing it More...
 
::StatusCode addOutputStream (const std::string &label, Detail::OutputStreamData output)
 effects: add another output file guarantee: strong failures: low level errors II failures: label already used More...
 
Long64_t inputFileNumEntries () const override
 the number of events in the input file More...
 
uint64_t eventsProcessed () const noexcept
 the number of events that have been processed More...
 

Private Types

enum  GridErrorCodes { EC_FAIL = 220, EC_ABORT = 221, EC_NOTFINISHED = 222, EC_BADINPUT = 223 }
 
typedef std::map< std::string, TH1 * >::const_iterator OutputHistMapIter
 the output map More...
 
typedef std::map< std::pair< std::string, std::string >, TTree * >::const_iterator OutputTreeMapIter
 description: the list of output trees More...
 

Private Member Functions

void gridCreateJobSummary (uint64_t eventsProcessed)
 
void addOutput (std::unique_ptr< TObject > output)
 add the given output object to the histogram output stream More...
 

Static Private Member Functions

static bool fileOpenErrorFilter (int level, bool, const char *, const char *)
 Error handler for file opening. More...
 

Private Attributes

std::map< std::string, TH1 * > m_outputHistMap
 
std::map< std::pair< std::string, std::string >, TTree * > m_outputTreeMap
 
std::vector< std::unique_ptr< Detail::Module > > m_modules
 the list of modules we hold More...
 
bool m_newInputFile {false}
 whether this is a new input file (i.e. More...
 
std::string m_outputTarget
 the target file to which we will write the histogram output More...
 
std::string m_segmentName
 the name of the segment we are processing More...
 
bool m_algorithmsInitialized {false}
 whether the algorithms are initialized More...
 
bool m_firstEvent {true}
 whether we are still to process the first event More...
 
std::vector< asg::AsgComponentConfigm_moduleConfig
 the module configurations we use More...
 
std::vector< Detail::AlgorithmDatam_algs
 the list of algorithms More...
 
std::string m_inputFileUrl
 the input file url of the currently opened file More...
 
std::unique_ptr< TFile > m_inputFile
 the input file pointer of the currently opened filed More...
 
TTree * m_inputTree {nullptr}
 the (main) tree in the input file More...
 
uint64_t m_inputTreeEntry {0}
 the entry in the input tree we are currently looking at More...
 
bool m_skipEvent = false
 whether we are skipping the current event More...
 
const SH::MetaObjectm_metaData {nullptr}
 the meta-data we use More...
 
uint64_t m_eventsProcessed {0}
 the number of events that have been processed More...
 
OutputStreamData * m_histOutput {nullptr}
 the histogram output stream More...
 
std::unique_ptr< TTree > m_jobStats
 Tree saving per-job statistics information. More...
 
xAOD::TEventm_tevent {nullptr}
 the TEvent structure, if we use one More...
 
xAOD::TStorem_tstore {nullptr}
 the TStore structure, if we use one More...
 
asg::SgTEventm_evtStore {nullptr}
 the SgTEvent structure, if we use one More...
 
Workerm_worker {nullptr}
 the worker (to pass on to the algorithms) More...
 
std::map< std::string, std::shared_ptr< Detail::OutputStreamData > > m_outputs
 the list of output files More...
 
BatchJobm_batchJob = nullptr
 the BatchJob configuration (if used) More...
 

Detailed Description

Definition at line 25 of file Worker.h.

Member Typedef Documentation

◆ OutputHistMapIter

typedef std::map<std::string,TH1*>::const_iterator EL::Worker::OutputHistMapIter
private

the output map

Definition at line 402 of file Worker.h.

◆ OutputTreeMapIter

typedef std::map<std::pair<std::string,std::string>,TTree*>::const_iterator EL::Worker::OutputTreeMapIter
private

description: the list of output trees

Definition at line 409 of file Worker.h.

Member Enumeration Documentation

◆ GridErrorCodes

Enumerator
EC_FAIL 
EC_ABORT 
EC_NOTFINISHED 
EC_BADINPUT 

Definition at line 250 of file Worker.h.

250  {
251  EC_FAIL = 220,
252  EC_ABORT = 221,
253  EC_NOTFINISHED = 222,
254  EC_BADINPUT = 223
255  };

Constructor & Destructor Documentation

◆ ~Worker()

EL::Worker::~Worker ( )
virtual

effects: standard destructor guarantee: no-fail

Definition at line 80 of file Worker.cxx.

82  {
83  RCU_DESTROY_INVARIANT (this);
84  }

◆ Worker()

EL::Worker::Worker ( )

standard constructor

Guarantee
strong
Failures
out of memory I

Definition at line 324 of file Worker.cxx.

326  {
327  m_worker = this;
328 
329  RCU_NEW_INVARIANT (this);
330  }

Member Function Documentation

◆ addOutput() [1/2]

void EL::Detail::ModuleData::addOutput ( std::unique_ptr< TObject >  output)
inherited

add the given output object to the histogram output stream

Guarantee
basic
Failures
out of memory II

Definition at line 33 of file ModuleData.cxx.

35  {
36  RCU_ASSERT (m_histOutput != nullptr);
37  m_histOutput->addOutput (std::move (output));
38  }

◆ addOutput() [2/2]

void EL::Worker::addOutput ( TObject *  output_swallow)
finaloverridevirtual

effects: add an object to the output.

the worker takes over ownership of the object, but the caller may maintain a reference to it guarantee: basic, argument is always swallowed failures: out of memory I requires: output_swallow != 0 warning: so far I placed no requirements on the output objects. I may do that at a later stage though, possibly breaking existing code.

Implements EL::IHistogramWorker.

Definition at line 88 of file Worker.cxx.

90  {
91  std::unique_ptr<TObject> output (output_swallow);
92 
93  RCU_CHANGE_INVARIANT (this);
94  RCU_REQUIRE_SOFT (output_swallow != 0);
95 
96  RCU::SetDirectory (output_swallow, 0);
97  ModuleData::addOutput (std::move (output));
98  }

◆ addOutputList()

void EL::Worker::addOutputList ( const std::string &  name,
TObject *  output_swallow 
)
overridevirtual

effects: add a given object to the output.

instead of trying to merge the outputs from multiple jobs the output file will contain a TList of the given name with the objects from all the output. guarantee: basic, argument is always swallowed failures: out of memory II requires: output_swallow != 0 rationale: This was primarily build to allow the GRL output to be collected from EventLoop jobs, but it can be used for any kind of output that can not or should not be merged.

Implements EL::IWorker.

Definition at line 102 of file Worker.cxx.

104  {
105  std::unique_ptr<TObject> output (output_swallow);
106 
107  RCU_CHANGE_INVARIANT (this);
108  RCU_REQUIRE_SOFT (output_swallow != 0);
109 
110  RCU::SetDirectory (output_swallow, 0);
111  std::unique_ptr<TList> list (new TList);
112  list->SetName (name.c_str());
113  list->Add (output.release());
114  addOutput (list.release());
115  }

◆ addOutputStream()

StatusCode EL::Worker::addOutputStream ( const std::string &  label,
Detail::OutputStreamData  output 
)
protected

effects: add another output file guarantee: strong failures: low level errors II failures: label already used

Definition at line 738 of file Worker.cxx.

741  {
742  using namespace msgEventLoop;
743  RCU_CHANGE_INVARIANT (this);
744 
745  if (m_outputs.find (label) != m_outputs.end())
746  {
747  ANA_MSG_ERROR ("output file already defined for label: " + label);
748  return ::StatusCode::FAILURE;
749  }
750  if (data.file() == nullptr)
751  {
752  ANA_MSG_ERROR ("output stream does not have a file attached");
753  return ::StatusCode::FAILURE;
754  }
755  if (data.mainStreamName().empty())
756  data.setMainStreamName (label);
757  m_outputs.insert (std::make_pair (label, std::make_shared<Detail::OutputStreamData>(std::move (data))));
758  return ::StatusCode::SUCCESS;
759  }

◆ addTree()

StatusCode EL::Worker::addTree ( const TTree &  tree,
const std::string &  stream 
)
finaloverridevirtual

effects: adds a tree to an output file specified by the stream/label failures: Incorrect stream/label specified, called at the wrong time note: See getOutputFile for failure types...

Implements EL::ITreeWorker.

Definition at line 155 of file Worker.cxx.

157  {
158  using namespace msgEventLoop;
159  RCU_READ_INVARIANT( this );
160 
161  auto outputIter = m_outputs.find (stream);
162  if (outputIter == m_outputs.end())
163  {
164  ANA_MSG_ERROR ( "No output file with stream name \"" + stream +
165  "\" found" );
166  return ::StatusCode::FAILURE;
167  }
168 
169  outputIter->second->addClone (tree);
170 
171  // Return gracefully:
172  return ::StatusCode::SUCCESS;
173  }

◆ batchExecute()

StatusCode EL::Worker::batchExecute ( unsigned  job_id,
const char *  confFile 
)

effects: do what is needed to execute the given job segment guarantee: basic failures: job specific

Definition at line 831 of file Worker.cxx.

833  {
834  using namespace msgEventLoop;
835  RCU_CHANGE_INVARIANT (this);
836 
837  try
838  {
839  std::unique_ptr<TFile> file (TFile::Open (confFile, "READ"));
840  if (file.get() == nullptr || file->IsZombie())
841  {
842  ANA_MSG_ERROR ("failed to open file: " << confFile);
843  return ::StatusCode::FAILURE;
844  }
845 
846  std::unique_ptr<BatchJob> job (dynamic_cast<BatchJob*>(file->Get ("job")));
847  m_batchJob = job.get();
848  if (job.get() == nullptr)
849  {
850  ANA_MSG_ERROR ("failed to retrieve BatchJob object");
851  return ::StatusCode::FAILURE;
852  }
853 
854  if (job_id >= job->segments.size())
855  {
856  ANA_MSG_ERROR ("invalid job-id " << job_id << ", max is " << job->segments.size());
857  return ::StatusCode::FAILURE;
858  }
859  BatchSegment *segment = &job->segments[job_id];
860  RCU_ASSERT (segment->job_id == job_id);
861  RCU_ASSERT (segment->sample < job->samples.size());
862  BatchSample *sample = &job->samples[segment->sample];
863 
864  gSystem->Exec ("pwd");
865  gSystem->MakeDirectory ("output");
866 
867  setMetaData (&sample->meta);
868  setOutputHist (job->location + "/fetch");
869  setSegmentName (segment->fullName);
870 
871  setJobConfig (JobConfig (job->job.jobConfig()));
872 
873  for (Job::outputIter out = job->job.outputBegin(),
874  end = job->job.outputEnd(); out != end; ++ out)
875  {
876  Detail::OutputStreamData data {
877  out->output()->makeWriter (segment->sampleName, segment->segmentName, ".root")};
878  ANA_CHECK (addOutputStream (out->label(), std::move (data)));
879  }
880 
881  {
882  m_moduleConfig.emplace_back ("EL::Detail::BatchInputModule/BatchInputModule");
883  ANA_CHECK (m_moduleConfig.back().setProperty ("jobId", job_id));
884  Long64_t maxEvents = metaData()->castDouble (Job::optMaxEvents, -1);
885  if (maxEvents != -1)
886  ANA_CHECK (m_moduleConfig.back().setProperty ("maxEvents", maxEvents));
887  }
888 
889  ANA_CHECK (initialize ());
891  ANA_CHECK (finalize ());
892 
893  std::ostringstream job_name;
894  job_name << job_id;
895  std::ofstream completed ((job->location + "/status/completed-" + job_name.str()).c_str());
896  return ::StatusCode::SUCCESS;
897  } catch (...)
898  {
899  Detail::report_exception (std::current_exception());
900  return ::StatusCode::FAILURE;
901  }
902  }

◆ directExecute()

StatusCode EL::Worker::directExecute ( const SH::SamplePtr sample,
const Job job,
const std::string &  location,
const SH::MetaObject options 
)

run the job

Guarantee
basic

Definition at line 786 of file Worker.cxx.

789  {
790  using namespace msgEventLoop;
791  RCU_CHANGE_INVARIANT (this);
792 
793  SH::MetaObject meta (*sample->meta());
794  meta.fetchDefaults (options);
795 
796  setMetaData (&meta);
797  setOutputHist (location);
798  setSegmentName (sample->name());
799 
800  ANA_MSG_INFO ("Running sample: " << sample->name());
801 
802  setJobConfig (JobConfig (job.jobConfig()));
803 
804  for (Job::outputIter out = job.outputBegin(),
805  end = job.outputEnd(); out != end; ++ out)
806  {
807  Detail::OutputStreamData data {
808  out->output()->makeWriter (sample->name(), "", ".root")};
809  ANA_CHECK (addOutputStream (out->label(), std::move (data)));
810  }
811 
812  {
813  m_moduleConfig.emplace_back ("EL::Detail::DirectInputModule/DirectInputModule");
814  ANA_CHECK (m_moduleConfig.back().setProperty ("fileList", sample->makeFileList()));
815  Long64_t maxEvents = metaData()->castDouble (Job::optMaxEvents, -1);
816  if (maxEvents != -1)
817  ANA_CHECK (m_moduleConfig.back().setProperty ("maxEvents", maxEvents));
819  if (skipEvents != 0)
820  ANA_CHECK (m_moduleConfig.back().setProperty ("skipEvents", skipEvents));
821  }
822 
823  ANA_CHECK (initialize ());
825  ANA_CHECK (finalize ());
826  return ::StatusCode::SUCCESS;
827  }

◆ eventsProcessed()

uint64_t EL::Worker::eventsProcessed ( ) const
protectednoexcept

the number of events that have been processed

Guarantee
no-fail

Definition at line 777 of file Worker.cxx.

779  {
780  RCU_READ_INVARIANT (this);
781  return m_eventsProcessed;
782  }

◆ fileOpenErrorFilter()

bool EL::Worker::fileOpenErrorFilter ( int  level,
bool  ,
const char *  s1,
const char *  s2 
)
staticprivate

Error handler for file opening.

Definition at line 637 of file Worker.cxx.

639  {
640  // Don't fail on missing dictionary messages.
641  if (strstr (s2, "no streamer or dictionary") != nullptr) {
642  return true;
643  }
644 
645  // For messages above warning level (SysError, Error, Fatal)
646  if( level > kWarning ) {
647  // We won't output further; ROOT should have already put something in the log file
648  std::string msg = "ROOT error detected in Worker.cxx: ";
649  msg += s1;
650  msg += " ";
651  msg += s2;
652  throw std::runtime_error(msg);
653 
654  // No need for further error handling
655  return false;
656  }
657 
658  // Pass to the default error handlers
659  return true;
660  }

◆ filterPassed()

bool EL::Worker::filterPassed ( ) const
finaloverridevirtualnoexcept

whether the current algorithm passed its filter criterion for the current event

Guarantee
no-fail

Implements EL::IFilterWorker.

Definition at line 306 of file Worker.cxx.

308  {
309  RCU_READ_INVARIANT (this);
310  return !m_skipEvent;
311  }

◆ finalize()

StatusCode EL::Worker::finalize ( )
protected

finalize the worker

This method ought to be called after all events have been processed. It is meant to ensure that the job is ended properly and all outputs are written out and files are closed.

Guarantee
basic
Failures
finalization failures

Definition at line 506 of file Worker.cxx.

508  {
509  using namespace msgEventLoop;
510 
511  RCU_CHANGE_INVARIANT (this);
512 
513  if (m_algorithmsInitialized == false)
514  {
515  ANA_MSG_ERROR ("algorithms never got initialized");
516  return StatusCode::FAILURE;
517  }
518 
519  ANA_CHECK (openInputFile (""));
520  for (auto& module : m_modules)
521  ANA_CHECK (module->onFinalize (*this));
522  for (auto& output : m_outputs)
523  {
524  if (output.first != Job::histogramStreamName && output.second->mainStreamName() == output.first)
525  {
526  output.second->saveOutput ();
527  output.second->close ();
528  std::string path = output.second->finalFileName ();
529  if (!path.empty())
530  addOutputList ("EventLoop_OutputStream_" + output.first, new TObjString (path.c_str()));
531  }
532  }
533  for (auto& module : m_modules)
534  ANA_CHECK (module->postFinalize (*this));
535  if (m_jobStats->GetListOfBranches()->GetEntries() > 0)
536  {
537  if (m_jobStats->Fill() <= 0)
538  {
539  ANA_MSG_ERROR ("failed to fill the job statistics tree");
540  return ::StatusCode::FAILURE;
541  }
542  ModuleData::addOutput (std::move (m_jobStats));
543  }
545  for (auto& module : m_modules)
546  ANA_CHECK (module->onWorkerEnd (*this));
548  m_histOutput->close ();
549 
550  for (auto& module : m_modules){
551  ANA_CHECK (module->postFileClose(*this));
552  }
553  ANA_MSG_INFO ("worker finished successfully");
554  return ::StatusCode::SUCCESS;
555  }

◆ getAlg()

Algorithm * EL::Worker::getAlg ( const std::string &  name) const
overridevirtual

effects: returns the algorithms with the given name or NULL if there is none guarantee: strong failures: out of memory II

Implements EL::IWorker.

Definition at line 283 of file Worker.cxx.

285  {
286  RCU_READ_INVARIANT (this);
287  for (auto& alg : m_algs)
288  {
289  if (alg->hasName (name))
290  return alg.m_algorithm->getLegacyAlg();
291  }
292  return 0;
293  }

◆ getOutputFile()

TFile * EL::Worker::getOutputFile ( const std::string &  label) const
overridevirtual

effects: get the output file that goes into the dataset with the given label.

this dataset has to be registered before the job is submitted. typically that happens in the doSetupJob method. guarantee: strong failures: no dataset with the given label postcondition: result != 0 note: the default value for the argument corresponds to the default label value in the OutputInfo class.

Implements EL::IWorker.

Definition at line 131 of file Worker.cxx.

133  {
134  RCU_READ_INVARIANT (this);
135  TFile *result = getOutputFileNull (label);
136  if (result == 0)
137  RCU_THROW_MSG ("no output dataset defined with label: " + label);
138  return result;
139  }

◆ getOutputFileNull()

TFile * EL::Worker::getOutputFileNull ( const std::string &  label) const
overridevirtual

effects: get the output file that goes into the dataset with the given label.

this dataset has to be registered before the job is submitted. typically that happens in the doSetupJob method. guarantee: strong failures: internal errors note: the default value for the argument corresponds to the default label value in the OutputInfo class. rationale: this method was added to support optional output files, i.e.: the algorithm leaves it to the user whether or not the output file gets configured. then the algorithm just checks whether the output file is there. if so it fills it, otherwise it ignores it.

Implements EL::IWorker.

Definition at line 143 of file Worker.cxx.

145  {
146  RCU_READ_INVARIANT (this);
147  auto iter = m_outputs.find (label);
148  if (iter == m_outputs.end())
149  return 0;
150  return iter->second->file();
151  }

◆ getOutputHist()

TObject * EL::Worker::getOutputHist ( const std::string &  name) const
finaloverridevirtual

get the output histogram with the given name

This is mostly meant, so that I can emulate the Athena histogram mechanism.

Guarantee
strong
Failures
object not found
Postcondition
result != 0

Implements EL::IHistogramWorker.

Definition at line 119 of file Worker.cxx.

121  {
122  RCU_READ_INVARIANT (this);
123 
124  TObject *result = m_histOutput->getOutputHist (name);
125  if (result == nullptr) RCU_THROW_MSG ("unknown output histogram: " + name);
126  return result;
127  }

◆ getOutputTree()

TTree * EL::Worker::getOutputTree ( const std::string &  name,
const std::string &  stream 
) const
finaloverridevirtual

effects: get the tree that was added to an output file earlier failures: Tree doesn't exist

Implements EL::ITreeWorker.

Definition at line 177 of file Worker.cxx.

179  {
180  using namespace msgEventLoop;
181  RCU_READ_INVARIANT( this );
182 
183  auto outputIter = m_outputs.find (stream);
184  if (outputIter == m_outputs.end())
185  {
186  RCU_THROW_MSG ( "No output file with stream name \"" + stream
187  + "\" found" );
188  }
189 
190  TTree *result = outputIter->second->getOutputTree( name );
191  if( result == nullptr ) {
192  RCU_THROW_MSG ( "No tree with name \"" + name + "\" in stream \"" +
193  stream + "\"" );
194  }
195  return result;
196  }

◆ gridCreateJobSummary()

void EL::Worker::gridCreateJobSummary ( uint64_t  eventsProcessed)
private

◆ gridExecute()

StatusCode EL::Worker::gridExecute ( const std::string &  sampleName,
Long64_t  SkipEvents,
Long64_t  nEventsPerJob 
)

Definition at line 906 of file Worker.cxx.

908  {
909  using namespace msgEventLoop;
910  RCU_CHANGE_INVARIANT (this);
911 
912  ANA_MSG_INFO ("Running with ROOT version " << gROOT->GetVersion()
913  << " (" << gROOT->GetVersionDate() << ")");
914 
915  ANA_MSG_INFO ("Loading EventLoop grid job");
916 
917 
918  TList bigOutputs;
919  std::unique_ptr<JobConfig> jobConfig;
920  SH::MetaObject *mo = 0;
921 
922  std::unique_ptr<TFile> f (TFile::Open("jobdef.root"));
923  if (f == nullptr || f->IsZombie()) {
924  ANA_MSG_ERROR ("Could not read jobdef");
925  return ::StatusCode::FAILURE;
926  }
927 
928  mo = dynamic_cast<SH::MetaObject*>(f->Get(sampleName.c_str()));
929  if (!mo)
930  mo = dynamic_cast<SH::MetaObject*>(f->Get("defaultMetaObject"));
931  if (!mo) {
932  ANA_MSG_ERROR ("Could not read in sample meta object");
933  return ::StatusCode::FAILURE;
934  }
935 
936  jobConfig.reset (dynamic_cast<JobConfig*>(f->Get("jobConfig")));
937  if (jobConfig == nullptr)
938  {
939  ANA_MSG_ERROR ("failed to read jobConfig object");
940  return ::StatusCode::FAILURE;
941  }
942 
943  {
944  std::unique_ptr<TList> outs ((TList*)f->Get("outputs"));
945  if (outs == nullptr)
946  {
947  ANA_MSG_ERROR ("Could not read list of outputs");
948  return ::StatusCode::FAILURE;
949  }
950 
951  TIter itr(outs.get());
952  TObject *obj = 0;
953  while ((obj = itr())) {
954  EL::OutputStream * out = dynamic_cast<EL::OutputStream*>(obj);
955  if (out) {
956  bigOutputs.Add(out);
957  }
958  else {
959  ANA_MSG_ERROR ("Encountered unexpected entry in list of outputs");
960  return ::StatusCode::FAILURE;
961  }
962  }
963  }
964 
965  f->Close();
966  f.reset ();
967 
968  const std::string location = ".";
969 
970  mo->setBool (Job::optGridReporting, true);
971  setMetaData (mo);
972  setOutputHist (location);
973  setSegmentName ("output");
974 
975  ANA_MSG_INFO ("Starting EventLoop Grid worker");
976 
977  {//Create and register the "big" output files with base class
978  TIter itr(&bigOutputs);
979  TObject *obj = 0;
980  while ((obj = itr())) {
981  EL::OutputStream *os = dynamic_cast<EL::OutputStream*>(obj);
982  if (os == nullptr)
983  {
984  ANA_MSG_ERROR ("Bad input");
985  return ::StatusCode::FAILURE;
986  }
987  {
988  Detail::OutputStreamData data {
989  location + "/" + os->label() + ".root", "RECREATE"};
990  ANA_CHECK (addOutputStream (os->label(), std::move (data)));
991  }
992  }
993  }
994 
995  setJobConfig (std::move (*jobConfig));
996 
997  {
998  std::vector<std::string> fileList;
999  std::ifstream infile("input.txt");
1000  while (infile) {
1001  std::string sLine;
1002  if (!getline(infile, sLine)) break;
1003  std::istringstream ssLine(sLine);
1004  while (ssLine) {
1005  std::string sFile;
1006  if (!getline(ssLine, sFile, ',')) break;
1007  fileList.push_back(sFile);
1008  }
1009  }
1010  if (fileList.size() == 0) {
1011  ANA_MSG_ERROR ("no input files provided");
1012  //User was expecting input after all.
1013  gSystem->Exit(EC_BADINPUT);
1014  }
1015  m_moduleConfig.emplace_back ("EL::Detail::DirectInputModule/DirectInputModule");
1016  ANA_CHECK (m_moduleConfig.back().setProperty ("fileList", fileList));
1017 
1018  if (nEventsPerJob != -1)
1019  ANA_CHECK (m_moduleConfig.back().setProperty ("maxEvents", nEventsPerJob));
1020  if (SkipEvents != 0)
1021  ANA_CHECK (m_moduleConfig.back().setProperty ("skipEvents", SkipEvents));
1022  }
1023 
1024  ANA_CHECK (initialize());
1025  ANA_CHECK (processInputs ());
1026  ANA_CHECK (finalize ());
1027 
1028  int nEvents = eventsProcessed();
1029  ANA_MSG_INFO ("Loop finished.");
1030  ANA_MSG_INFO ("Read/processed " << nEvents << " events.");
1031 
1032  ANA_MSG_INFO ("EventLoop Grid worker finished");
1033  ANA_MSG_INFO ("Saving output");
1034  return ::StatusCode::SUCCESS;
1035  }

◆ initialize()

StatusCode EL::Worker::initialize ( )
protected

initialize the worker

This method ought to be called after the options on the worker are set and before any events are processed. It is meant to make sure everything is ready and set up for event processing.

Guarantee
basic
Failures
initialization failures

Definition at line 377 of file Worker.cxx.

379  {
380  using namespace msgEventLoop;
381  RCU_CHANGE_INVARIANT (this);
382 
383  const bool xAODInput = m_metaData->castBool (Job::optXAODInput, false);
384 
385  ANA_MSG_INFO ("xAODInput = " << xAODInput);
386 
387  if (metaData()->castBool (Job::optAlgorithmMemoryMonitor, false))
388  m_moduleConfig.emplace_back ("EL::Detail::MemoryMonitorModule/EarlyMemoryMonitorModule");
389  if (auto cacheSize = metaData()->castDouble (Job::optCacheSize, 0); cacheSize > 0)
390  {
391  m_moduleConfig.emplace_back ("EL::Detail::TreeCacheModule/TreeCacheModule");
392  ANA_CHECK (m_moduleConfig.back().setProperty ("cacheSize", Long64_t (cacheSize)));
393  ANA_CHECK (m_moduleConfig.back().setProperty ("cacheLearnEntries", Long64_t (metaData()->castInteger (Job::optCacheLearnEntries, 0))));
394  ANA_CHECK (m_moduleConfig.back().setProperty ("printPerFileStats", metaData()->castBool (Job::optPrintPerFileStats, false)));
395  }
396  if (xAODInput)
397  {
398  m_moduleConfig.emplace_back ("EL::Detail::TEventModule/TEventModule");
399  ANA_CHECK (m_moduleConfig.back().setProperty ("accessMode", metaData()->castString (Job::optXaodAccessMode)));
400  if (metaData()->castDouble (Job::optXAODSummaryReport, 1) == 0)
401  ANA_CHECK (m_moduleConfig.back().setProperty ("summaryReport", false));
402  ANA_CHECK (m_moduleConfig.back().setProperty ("useStats", metaData()->castBool (Job::optXAODPerfStats, false)));
403  }
404  auto factoryPreload = metaData()->castString (Job::optFactoryPreload, "");
405  if (!factoryPreload.empty())
406  {
407  m_moduleConfig.emplace_back ("EL::Detail::FactoryPreloadModule/FactoryPreloadModule");
408  ANA_CHECK (m_moduleConfig.back().setProperty ("preloader", factoryPreload));
409  }
410  m_moduleConfig.emplace_back ("EL::Detail::LeakCheckModule/LeakCheckModule");
411  ANA_CHECK (m_moduleConfig.back().setProperty ("failOnLeak", metaData()->castBool (Job::optMemFailOnLeak, false)));
412  ANA_CHECK (m_moduleConfig.back().setProperty ("absResidentLimit", metaData()->castInteger (Job::optMemResidentIncreaseLimit, 10000)));
413  ANA_CHECK (m_moduleConfig.back().setProperty ("absVirtualLimit", metaData()->castInteger (Job::optMemVirtualIncreaseLimit, 0)));
414  ANA_CHECK (m_moduleConfig.back().setProperty ("perEvResidentLimit", metaData()->castInteger (Job::optMemResidentPerEventIncreaseLimit, 10)));
415  ANA_CHECK (m_moduleConfig.back().setProperty ("perEvVirtualLimit", metaData()->castInteger (Job::optMemVirtualPerEventIncreaseLimit, 0)));
416  m_moduleConfig.emplace_back ("EL::Detail::StopwatchModule/StopwatchModule");
417  if (metaData()->castBool (Job::optGridReporting, false))
418  m_moduleConfig.emplace_back ("EL::Detail::GridReportingModule/GridReportingModule");
419  if (metaData()->castBool (Job::optAlgorithmTimer, false))
420  m_moduleConfig.emplace_back ("EL::Detail::AlgorithmTimerModule/AlgorithmTimerModule");
421  if (metaData()->castBool (Job::optAlgorithmMemoryMonitor, false))
422  m_moduleConfig.emplace_back ("EL::Detail::AlgorithmMemoryModule/AlgorithmMemoryModule");
423  m_moduleConfig.emplace_back ("EL::Detail::FileExecutedModule/FileExecutedModule");
424  m_moduleConfig.emplace_back ("EL::Detail::EventCountModule/EventCountModule");
425  m_moduleConfig.emplace_back ("EL::Detail::WorkerConfigModule/WorkerConfigModule");
426  m_moduleConfig.emplace_back ("EL::Detail::AlgorithmStateModule/AlgorithmStateModule");
427  m_moduleConfig.emplace_back ("EL::Detail::PostClosedOutputsModule/PostClosedOutputsModule");
428  if (metaData()->castBool (Job::optAlgorithmMemoryMonitor, false))
429  m_moduleConfig.emplace_back ("EL::Detail::MemoryMonitorModule/LateMemoryMonitorModule");
430 
431  for (const auto& config : m_moduleConfig)
432  {
433  std::unique_ptr<Detail::Module> module;
434  ANA_CHECK (make_module (module, config));
435  m_modules.push_back (std::move (module));
436  }
437 
438  if (m_outputs.find (Job::histogramStreamName) == m_outputs.end())
439  {
440  Detail::OutputStreamData data {
441  m_outputTarget + "/hist-" + m_segmentName + ".root", "RECREATE"};
443  }
446  if (auto aliases = metaData()->castString (Job::optStreamAliases, ""); !aliases.empty())
447  {
448  // the format of aliases is "alias1=realname1,alias2=realname2"
449  std::istringstream iss (aliases);
450  std::string alias;
451  while (std::getline (iss, alias, ','))
452  {
453  auto pos = alias.find ('=');
454  if (pos == std::string::npos)
455  {
456  ANA_MSG_ERROR ("Invalid alias format: " << alias);
457  return ::StatusCode::FAILURE;
458  }
459  auto aliasName = alias.substr (0, pos);
460  auto realName = alias.substr (pos + 1);
461  auto realOutput = m_outputs.find (realName);
462  if (realOutput == m_outputs.end())
463  {
464  ANA_MSG_ERROR ("output stream " << realName << " not found for alias " << aliasName);
465  return ::StatusCode::FAILURE;
466  }
467  auto [aliasOutput, success] = m_outputs.emplace (aliasName, realOutput->second);
468  if (!success)
469  {
470  ANA_MSG_ERROR ("output stream " << aliasName << " already exists, can't make alias");
471  return ::StatusCode::FAILURE;
472  }
473  }
474  }
475 
476  m_jobStats = std::make_unique<TTree>
477  ("EventLoop_JobStats", "EventLoop job statistics");
478  m_jobStats->SetDirectory (nullptr);
479 
480  ANA_MSG_INFO ("calling firstInitialize on all modules");
481  for (auto& module : m_modules)
482  ANA_CHECK (module->firstInitialize (*this));
483  ANA_MSG_INFO ("calling preFileInitialize on all modules");
484  for (auto& module : m_modules)
485  ANA_CHECK (module->preFileInitialize (*this));
486 
487  return ::StatusCode::SUCCESS;
488  }

◆ inputFile()

TFile * EL::Worker::inputFile ( ) const
overridevirtual

description: the file we are reading the current tree from guarantee: no-fail

Implements EL::IWorker.

Definition at line 227 of file Worker.cxx.

229  {
230  RCU_READ_INVARIANT (this);
231  return m_inputFile.get();
232  }

◆ inputFileName()

std::string EL::Worker::inputFileName ( ) const
overridevirtual

the name of the file we are reading the current tree from, without the path component

Guarantee
no-fail

Implements EL::IWorker.

Definition at line 236 of file Worker.cxx.

238  {
239  // no invariant used
240  std::string path = inputFile()->GetName();
241  auto split = path.rfind ('/');
242  if (split != std::string::npos)
243  return path.substr (split + 1);
244  else
245  return path;
246  }

◆ inputFileNumEntries()

Long64_t EL::Worker::inputFileNumEntries ( ) const
overrideprotectedvirtual

the number of events in the input file

Guarantee
no-fail
Precondition
inputFile() != 0

Implements EL::Detail::IInputModuleActions.

Definition at line 763 of file Worker.cxx.

765  {
766  RCU_READ_INVARIANT (this);
767  RCU_REQUIRE (inputFile() != 0);
768 
769  if (m_inputTree != 0)
770  return m_inputTree->GetEntries();
771  else
772  return 0;
773  }

◆ metaData()

const SH::MetaObject * EL::Worker::metaData ( ) const
overridevirtual

description: the sample meta-data we are working on guarantee: no-fail invariant: metaData != 0 rationale: this can be used for accessing sample meta-data

Implements EL::IWorker.

Definition at line 200 of file Worker.cxx.

202  {
203  RCU_READ_INVARIANT (this);
204  return m_metaData;
205  }

◆ openInputFile()

StatusCode EL::Worker::openInputFile ( const std::string &  inputFileUrl)
overrideprotectedvirtual

open the given input file without processing it

This is mostly to allow the driver to query the number of events in the input file without processing it, usually to determine the range of events to process.

Guarantee
basic
Failures
file can't be opened

Implements EL::Detail::IInputModuleActions.

Definition at line 662 of file Worker.cxx.

664  {
665  using namespace msgEventLoop;
666 
667  // Enable custom error handling in a nice way
669 
670  RCU_CHANGE_INVARIANT (this);
671 
672  if (m_inputFileUrl == inputFileUrl)
673  return ::StatusCode::SUCCESS;
674 
675  if (!m_inputFileUrl.empty())
676  {
677  if (m_newInputFile == false)
678  {
679  for (auto& module : m_modules)
680  ANA_CHECK (module->onCloseInputFile (*this));
681  for (auto& module : m_modules)
682  ANA_CHECK (module->postCloseInputFile (*this));
683  }
684  m_newInputFile = false;
685  m_inputTree = nullptr;
686  m_inputFile.reset ();
687  m_inputFileUrl.clear ();
688  }
689 
690  if (inputFileUrl.empty())
691  return ::StatusCode::SUCCESS;
692 
693  ANA_MSG_INFO ("Opening file " << inputFileUrl);
694  std::unique_ptr<TFile> inputFile;
695  try
696  {
697  inputFile = SH::openFile (inputFileUrl, *metaData());
698  } catch (...)
699  {
700  Detail::report_exception (std::current_exception());
701  }
702  if (inputFile.get() == 0)
703  {
704  ANA_MSG_ERROR ("failed to open file " << inputFileUrl);
705  for (auto& module : m_modules)
706  module->reportInputFailure (*this);
707  return ::StatusCode::FAILURE;
708  }
709  if (inputFile->IsZombie())
710  {
711  ANA_MSG_ERROR ("input file is a zombie: " << inputFileUrl);
712  for (auto& module : m_modules)
713  module->reportInputFailure (*this);
714  return ::StatusCode::FAILURE;
715  }
716 
717  TTree *tree = 0;
718  const std::string treeName
720  tree = dynamic_cast<TTree*>(inputFile->Get (treeName.c_str()));
721  if (tree == nullptr)
722  {
723  ANA_MSG_INFO ("tree " << treeName << " not found in input file: " << inputFileUrl);
724  ANA_MSG_INFO ("treating this like a tree with no events");
725  }
726 
727  m_newInputFile = true;
728  m_inputTree = tree;
729  m_inputTreeEntry = 0;
730  m_inputFile = std::move (inputFile);
731  m_inputFileUrl = std::move (inputFileUrl);
732 
733  return ::StatusCode::SUCCESS;
734  }

◆ processEvents()

StatusCode EL::Worker::processEvents ( EventRange eventRange)
overrideprotectedvirtual

process the given event range

This will update eventRange if the end is set to eof

Guarantee
basic
Failures
file can't be opened
event range exceeds length of file
processing failures

Implements EL::Detail::IInputModuleActions.

Definition at line 559 of file Worker.cxx.

561  {
562  using namespace msgEventLoop;
563 
564  RCU_CHANGE_INVARIANT (this);
565  RCU_REQUIRE (!eventRange.m_url.empty());
566  RCU_REQUIRE (eventRange.m_beginEvent >= 0);
567  RCU_REQUIRE (eventRange.m_endEvent == EventRange::eof || eventRange.m_endEvent >= eventRange.m_beginEvent);
568 
569  ANA_CHECK (openInputFile (eventRange.m_url));
570 
571  if (eventRange.m_beginEvent > inputFileNumEntries())
572  {
573  ANA_MSG_ERROR ("first event (" << eventRange.m_beginEvent << ") points beyond last event in file (" << inputFileNumEntries() << ")");
574  return ::StatusCode::FAILURE;
575  }
576  if (eventRange.m_endEvent == EventRange::eof)
577  {
578  eventRange.m_endEvent = inputFileNumEntries();
579  } else if (eventRange.m_endEvent > inputFileNumEntries())
580  {
581  ANA_MSG_ERROR ("end event (" << eventRange.m_endEvent << ") points beyond last event in file (" << inputFileNumEntries() << ")");
582  return ::StatusCode::FAILURE;
583  }
584 
585  m_inputTreeEntry = eventRange.m_beginEvent;
586 
587  if (m_algorithmsInitialized == false)
588  {
589  for (auto& module : m_modules)
590  ANA_CHECK (module->onInitialize (*this));
592  }
593 
594  if (m_newInputFile)
595  {
596  m_newInputFile = false;
597  for (auto& module : m_modules)
598  ANA_CHECK (module->onNewInputFile (*this));
599  }
600 
601  if (eventRange.m_beginEvent == 0)
602  {
603  for (auto& module : m_modules)
604  ANA_CHECK (module->onFileExecute (*this));
605  }
606 
607  ANA_MSG_INFO ("Processing events " << eventRange.m_beginEvent << "-" << eventRange.m_endEvent << " in file " << eventRange.m_url);
608 
609  for (uint64_t event = eventRange.m_beginEvent;
610  event != uint64_t (eventRange.m_endEvent);
611  ++ event)
612  {
614  for (auto& module : m_modules)
615  {
616  if (module->onExecute (*this).isFailure())
617  {
618  ANA_MSG_ERROR ("processing event " << treeEntry() << " on file " << inputFileName());
619  return ::StatusCode::FAILURE;
620  }
621  }
622  if (m_firstEvent)
623  {
624  m_firstEvent = false;
625  for (auto& module : m_modules)
626  ANA_CHECK (module->postFirstEvent (*this));
627  }
628  m_eventsProcessed += 1;
629  if (m_eventsProcessed % 10000 == 0)
630  ANA_MSG_INFO ("Processed " << m_eventsProcessed << " events");
631  }
632  return ::StatusCode::SUCCESS;
633  }

◆ processInputs()

StatusCode EL::Worker::processInputs ( )
protected

process all the inputs

This method ought to be called after initialize and before finalize. It will rely on the defined modules to steer it to the files and events it ought to process.

Definition at line 492 of file Worker.cxx.

494  {
495  using namespace msgEventLoop;
496 
497  RCU_CHANGE_INVARIANT (this);
498 
499  for (auto& module : m_modules)
500  ANA_CHECK (module->processInputs (*this, *this));
501  return ::StatusCode::SUCCESS;
502  }

◆ setFilterPassed()

void EL::Worker::setFilterPassed ( bool  val_filterPassed)
finaloverridevirtualnoexcept

set the value of filterPassed

Guarantee
no-fail

Implements EL::IFilterWorker.

Definition at line 315 of file Worker.cxx.

317  {
318  RCU_CHANGE_INVARIANT (this);
319  m_skipEvent = !val_filterPassed;
320  }

◆ setJobConfig()

void EL::Worker::setJobConfig ( JobConfig &&  jobConfig)
protected

set the JobConfig

This takes care of adding the algorithms, etc. (only algorithms for now, 03 Feb 17).

Note the rvalue calling convention here: Algorithms are objects that get modified, so if you use them more than once you need to copy/clone them. However, in practice no driver should need that (though many do for now, 03 Feb 17), as drivers generally stream the JobConfig in for one-time use.

Guarantee
basic
Failures
job configuration/streaming errors

Definition at line 365 of file Worker.cxx.

367  {
368  RCU_CHANGE_INVARIANT (this);
369  for (std::unique_ptr<IAlgorithmWrapper>& alg : jobConfig.extractAlgorithms())
370  {
371  m_algs.push_back (std::move (alg));
372  }
373  }

◆ setMetaData()

void EL::Worker::setMetaData ( const SH::MetaObject val_metaData)
protected

set the metaData

Guarantee
no-fail

Definition at line 334 of file Worker.cxx.

336  {
337  RCU_CHANGE_INVARIANT (this);
338  RCU_REQUIRE (val_metaData != 0);
339 
340  m_metaData = val_metaData;
341  }

◆ setOutputHist()

void EL::Worker::setOutputHist ( const std::string &  val_outputTarget)
protected

set the histogram output list

Guarantee
no-fail

Definition at line 345 of file Worker.cxx.

347  {
348  RCU_CHANGE_INVARIANT (this);
349 
350  m_outputTarget = val_outputTarget;
351  }

◆ setSegmentName()

void EL::Worker::setSegmentName ( const std::string &  val_segmentName)
protected

set the segment name

Guarantee
no-fail

Definition at line 355 of file Worker.cxx.

357  {
358  RCU_CHANGE_INVARIANT (this);
359 
360  m_segmentName = val_segmentName;
361  }

◆ skipEvent()

void EL::Worker::skipEvent ( )
overridevirtual

effects: skip the current event, i.e.

skip the rest of the algorithms for this event guarantee: no-fail rationale: if you have an analysis strategy in which you divide work into multiple algorithms you may want to have dedicated algorithms for event selection that then skip later algorithms that fill histograms

Implements EL::IWorker.

Definition at line 297 of file Worker.cxx.

299  {
300  RCU_CHANGE_INVARIANT (this);
301  m_skipEvent = true;
302  }

◆ testInvariant()

void EL::Worker::testInvariant ( ) const

effects: test the invariant of this object guarantee: no-fail

Definition at line 68 of file Worker.cxx.

70  {
71  RCU_INVARIANT (this != nullptr);
72  for (std::size_t iter = 0, end = m_algs.size(); iter != end; ++ iter)
73  {
74  RCU_INVARIANT (m_algs[iter].m_algorithm != nullptr);
75  }
76  }

◆ tree()

TTree * EL::Worker::tree ( ) const
overridevirtual

description: the tree we are running on guarantee: no-fail

Implements EL::IWorker.

Definition at line 209 of file Worker.cxx.

211  {
212  RCU_READ_INVARIANT (this);
213  return m_inputTree;
214  }

◆ treeEntry()

Long64_t EL::Worker::treeEntry ( ) const
overridevirtual

description: the entry in the tree we are reading guarantee: no-fail

Implements EL::IWorker.

Definition at line 218 of file Worker.cxx.

220  {
221  RCU_READ_INVARIANT (this);
222  return m_inputTreeEntry;
223  }

◆ triggerConfig()

TTree * EL::Worker::triggerConfig ( ) const
overridevirtual

description: the trigger config tree from the input file, or NULL if we did not find it guarantee: strong failures: i/o errors

Implements EL::IWorker.

Definition at line 250 of file Worker.cxx.

252  {
253  RCU_READ_INVARIANT (this);
254  return dynamic_cast<TTree*>(inputFile()->Get("physicsMeta/TrigConfTree"));
255  }

◆ xaodEvent()

xAOD::TEvent * EL::Worker::xaodEvent ( ) const
overridevirtual

description: the xAOD event and store guarantee: strong failures: out of memory I failures: TEventSvc not configured postcondition: result != 0

Implements EL::IWorker.

Definition at line 259 of file Worker.cxx.

261  {
262  RCU_READ_INVARIANT (this);
263 
264  if (m_tevent == nullptr)
265  RCU_THROW_MSG ("Job not configured for xAOD support");
266  return m_tevent;
267  }

◆ xaodStore()

xAOD::TStore * EL::Worker::xaodStore ( ) const
overridevirtual

Implements EL::IWorker.

Definition at line 271 of file Worker.cxx.

273  {
274  RCU_READ_INVARIANT (this);
275 
276  if (m_tstore == nullptr)
277  RCU_THROW_MSG ("Job not configured for xAOD support");
278  return m_tstore;
279  }

Member Data Documentation

◆ m_algorithmsInitialized

bool EL::Worker::m_algorithmsInitialized {false}
private

whether the algorithms are initialized

Definition at line 436 of file Worker.h.

◆ m_algs

std::vector<Detail::AlgorithmData> EL::Detail::ModuleData::m_algs
inherited

the list of algorithms

Definition at line 66 of file ModuleData.h.

◆ m_batchJob

BatchJob* EL::Detail::ModuleData::m_batchJob = nullptr
inherited

the BatchJob configuration (if used)

Definition at line 112 of file ModuleData.h.

◆ m_eventsProcessed

uint64_t EL::Detail::ModuleData::m_eventsProcessed {0}
inherited

the number of events that have been processed

Definition at line 88 of file ModuleData.h.

◆ m_evtStore

asg::SgTEvent* EL::Detail::ModuleData::m_evtStore {nullptr}
inherited

the SgTEvent structure, if we use one

Definition at line 103 of file ModuleData.h.

◆ m_firstEvent

bool EL::Worker::m_firstEvent {true}
private

whether we are still to process the first event

Definition at line 441 of file Worker.h.

◆ m_histOutput

OutputStreamData* EL::Detail::ModuleData::m_histOutput {nullptr}
inherited

the histogram output stream

Definition at line 91 of file ModuleData.h.

◆ m_inputFile

std::unique_ptr<TFile> EL::Detail::ModuleData::m_inputFile
inherited

the input file pointer of the currently opened filed

Definition at line 72 of file ModuleData.h.

◆ m_inputFileUrl

std::string EL::Detail::ModuleData::m_inputFileUrl
inherited

the input file url of the currently opened file

Definition at line 69 of file ModuleData.h.

◆ m_inputTree

TTree* EL::Detail::ModuleData::m_inputTree {nullptr}
inherited

the (main) tree in the input file

Definition at line 75 of file ModuleData.h.

◆ m_inputTreeEntry

uint64_t EL::Detail::ModuleData::m_inputTreeEntry {0}
inherited

the entry in the input tree we are currently looking at

Definition at line 79 of file ModuleData.h.

◆ m_jobStats

std::unique_ptr<TTree> EL::Detail::ModuleData::m_jobStats
inherited

Tree saving per-job statistics information.

Definition at line 94 of file ModuleData.h.

◆ m_metaData

const SH::MetaObject* EL::Detail::ModuleData::m_metaData {nullptr}
inherited

the meta-data we use

Definition at line 85 of file ModuleData.h.

◆ m_moduleConfig

std::vector<asg::AsgComponentConfig> EL::Worker::m_moduleConfig
private

the module configurations we use

Definition at line 446 of file Worker.h.

◆ m_modules

std::vector<std::unique_ptr<Detail::Module> > EL::Worker::m_modules
private

the list of modules we hold

Definition at line 415 of file Worker.h.

◆ m_newInputFile

bool EL::Worker::m_newInputFile {false}
private

whether this is a new input file (i.e.

one that has not yet been connected to the algorithms)

Definition at line 421 of file Worker.h.

◆ m_outputHistMap

std::map<std::string,TH1*> EL::Worker::m_outputHistMap
private

Definition at line 403 of file Worker.h.

◆ m_outputs

std::map<std::string,std::shared_ptr<Detail::OutputStreamData> > EL::Detail::ModuleData::m_outputs
inherited

the list of output files

Definition at line 109 of file ModuleData.h.

◆ m_outputTarget

std::string EL::Worker::m_outputTarget
private

the target file to which we will write the histogram output

Definition at line 426 of file Worker.h.

◆ m_outputTreeMap

std::map<std::pair<std::string,std::string>,TTree*> EL::Worker::m_outputTreeMap
private

Definition at line 410 of file Worker.h.

◆ m_segmentName

std::string EL::Worker::m_segmentName
private

the name of the segment we are processing

Definition at line 431 of file Worker.h.

◆ m_skipEvent

bool EL::Detail::ModuleData::m_skipEvent = false
inherited

whether we are skipping the current event

Definition at line 82 of file ModuleData.h.

◆ m_tevent

xAOD::TEvent* EL::Detail::ModuleData::m_tevent {nullptr}
inherited

the TEvent structure, if we use one

Definition at line 97 of file ModuleData.h.

◆ m_tstore

xAOD::TStore* EL::Detail::ModuleData::m_tstore {nullptr}
inherited

the TStore structure, if we use one

Definition at line 100 of file ModuleData.h.

◆ m_worker

Worker* EL::Detail::ModuleData::m_worker {nullptr}
inherited

the worker (to pass on to the algorithms)

Definition at line 106 of file ModuleData.h.


The documentation for this class was generated from the following files:
nEvents
const int nEvents
Definition: fbtTestBasics.cxx:78
EL::Worker::finalize
::StatusCode finalize()
finalize the worker
Definition: Worker.cxx:507
EL::Job::optMaxEvents
static const std::string optMaxEvents
description: the name of the option used for setting the maximum number of events to process per samp...
Definition: Job.h:225
EL::Detail::OutputStreamData::addOutput
void addOutput(std::unique_ptr< TObject > outputObject)
add the given output object to this stream
Definition: OutputStreamData.cxx:224
EL::Worker::m_newInputFile
bool m_newInputFile
whether this is a new input file (i.e.
Definition: Worker.h:421
EL::Worker::addOutput
void addOutput(TObject *output_swallow) final override
effects: add an object to the output.
Definition: Worker.cxx:89
EL::Worker::m_firstEvent
bool m_firstEvent
whether we are still to process the first event
Definition: Worker.h:441
EL::Worker::m_modules
std::vector< std::unique_ptr< Detail::Module > > m_modules
the list of modules we hold
Definition: Worker.h:415
EL::Worker::m_outputTarget
std::string m_outputTarget
the target file to which we will write the histogram output
Definition: Worker.h:426
HelloWorldOptions.job
job
Definition: HelloWorldOptions.py:18
data
char data[hepevt_bytes_allocation_ATLAS]
Definition: HepEvt.cxx:11
ReadCellNoiseFromCoolCompare.s1
s1
Definition: ReadCellNoiseFromCoolCompare.py:378
createLinkingScheme.iter
iter
Definition: createLinkingScheme.py:62
EL::Worker::getOutputFileNull
TFile * getOutputFileNull(const std::string &label) const override
effects: get the output file that goes into the dataset with the given label.
Definition: Worker.cxx:144
EL::Detail::OutputStreamData::getOutputHist
TObject * getOutputHist(const std::string &name) const noexcept
get the output histogram with the given name, or nullptr if there is no histogam with such a name
Definition: OutputStreamData.cxx:313
EL::Detail::ModuleData::m_inputTree
TTree * m_inputTree
the (main) tree in the input file
Definition: ModuleData.h:75
get_generator_info.result
result
Definition: get_generator_info.py:21
athena.path
path
python interpreter configuration --------------------------------------—
Definition: athena.py:128
run.infile
string infile
Definition: run.py:13
EL::Worker::eventsProcessed
uint64_t eventsProcessed() const noexcept
the number of events that have been processed
Definition: Worker.cxx:778
EL::Job::optXaodAccessMode
static const std::string optXaodAccessMode
description: the option to select the access mode for xAODs.
Definition: Job.h:400
SH::MetaObject
A class that manages meta-data to be associated with an object.
Definition: MetaObject.h:56
EL::Worker::inputFileName
std::string inputFileName() const override
the name of the file we are reading the current tree from, without the path component
Definition: Worker.cxx:237
SH::MetaObject::castBool
bool castBool(const std::string &name, bool def_val=false, CastMode mode=CAST_ERROR_THROW) const
the meta-data boolean with the given name
EL::Job::optPrintPerFileStats
static const std::string optPrintPerFileStats
description: the option to turn on printing of i/o statistics at the end of each file rationale: whil...
Definition: Job.h:419
EL::OutputStream
Definition: OutputStream.h:34
EL::Worker::inputFile
TFile * inputFile() const override
description: the file we are reading the current tree from guarantee: no-fail
Definition: Worker.cxx:228
Execution.SkipEvents
SkipEvents
Definition: Execution.py:91
EL::Worker::EC_FAIL
@ EC_FAIL
Definition: Worker.h:251
RCU_REQUIRE
#define RCU_REQUIRE(x)
Definition: Assert.h:208
EL::Worker::openInputFile
::StatusCode openInputFile(const std::string &inputFileUrl) override
open the given input file without processing it
Definition: Worker.cxx:663
EL::Job::optMemResidentPerEventIncreaseLimit
static const std::string optMemResidentPerEventIncreaseLimit
The minimal per-event resident memory increase for triggering an error.
Definition: Job.h:554
EL::Worker::EC_NOTFINISHED
@ EC_NOTFINISHED
Definition: Worker.h:253
python.AthDsoLogger.out
out
Definition: AthDsoLogger.py:70
CscCalibQuery.fileList
fileList
Definition: CscCalibQuery.py:329
EL::Worker::m_algorithmsInitialized
bool m_algorithmsInitialized
whether the algorithms are initialized
Definition: Worker.h:436
EL::Detail::ModuleData::m_batchJob
BatchJob * m_batchJob
the BatchJob configuration (if used)
Definition: ModuleData.h:112
ANA_MSG_ERROR
#define ANA_MSG_ERROR(xmsg)
Macro printing error messages.
Definition: Control/AthToolSupport/AsgMessaging/AsgMessaging/MessageCheck.h:294
EL::Worker::EC_ABORT
@ EC_ABORT
Definition: Worker.h:252
ANA_CHECK
#define ANA_CHECK(EXP)
check whether the given expression was successful
Definition: Control/AthToolSupport/AsgMessaging/AsgMessaging/MessageCheck.h:324
EL::Detail::OutputStreamData::close
void close()
close this file
Definition: OutputStreamData.cxx:173
EL::Job::optFactoryPreload
static const std::string optFactoryPreload
a boolean flag for whether to perform a component factory preload
Definition: Job.h:214
SH::MetaObject::setBool
void setBool(const std::string &name, bool value)
set the meta-data boolean with the given name
skel.nEventsPerJob
nEventsPerJob
Definition: skel.ABtoEVGEN.py:552
EL::Detail::ModuleData::m_inputFile
std::unique_ptr< TFile > m_inputFile
the input file pointer of the currently opened filed
Definition: ModuleData.h:72
AthenaPoolTestWrite.stream
string stream
Definition: AthenaPoolTestWrite.py:12
EL::Detail::ModuleData::m_jobStats
std::unique_ptr< TTree > m_jobStats
Tree saving per-job statistics information.
Definition: ModuleData.h:94
EL::Job::optXAODSummaryReport
static const std::string optXAODSummaryReport
the option to turn on/off the xAOD summary reporting at the end of the job
Definition: Job.h:409
EL::Job::optXAODInput
static const std::string optXAODInput
the option to select whether our input is xAODs
Definition: Job.h:393
config
Definition: PhysicsAnalysis/AnalysisCommon/AssociationUtils/python/config.py:1
mergePhysValFiles.end
end
Definition: DataQuality/DataQualityUtils/scripts/mergePhysValFiles.py:92
python.iconfTool.models.loaders.level
level
Definition: loaders.py:20
jetMakeRefSamples.skipEvents
int skipEvents
Definition: jetMakeRefSamples.py:55
python.SCT_ByteStreamErrorsTestAlgConfig.maxEvents
maxEvents
Definition: SCT_ByteStreamErrorsTestAlgConfig.py:43
EL::Detail::ModuleData::m_outputs
std::map< std::string, std::shared_ptr< Detail::OutputStreamData > > m_outputs
the list of output files
Definition: ModuleData.h:109
EL::Worker::fileOpenErrorFilter
static bool fileOpenErrorFilter(int level, bool, const char *, const char *)
Error handler for file opening.
Definition: Worker.cxx:638
EL::Job::optStreamAliases
static const std::string optStreamAliases
an option for stream aliases
Definition: Job.h:217
python.PyAthena.module
module
Definition: PyAthena.py:131
JobConfig
Definition: CoolFix.cxx:83
EL::Worker::metaData
const SH::MetaObject * metaData() const override
description: the sample meta-data we are working on guarantee: no-fail invariant: metaData !...
Definition: Worker.cxx:201
event
POOL::TEvent event(POOL::TEvent::kClassAccess)
FullCPAlgorithmsTest_eljob.sample
sample
Definition: FullCPAlgorithmsTest_eljob.py:116
RCU_REQUIRE_SOFT
#define RCU_REQUIRE_SOFT(x)
Definition: Assert.h:153
EL::Worker::setSegmentName
void setSegmentName(const std::string &val_segmentName)
set the segment name
Definition: Worker.cxx:356
EL::Worker::setOutputHist
void setOutputHist(const std::string &val_outputTarget)
set the histogram output list
Definition: Worker.cxx:346
EL::Job::optCacheSize
static const std::string optCacheSize
description: this option allows to configure the TTreeCache size for this job.
Definition: Job.h:312
EL::Detail::ModuleData::m_tstore
xAOD::TStore * m_tstore
the TStore structure, if we use one
Definition: ModuleData.h:100
EL::Job::optXAODPerfStats
static const std::string optXAODPerfStats
description: the name of the option for turning on XAODPerfStats.
Definition: Job.h:356
COOLRates.alias
alias
Definition: COOLRates.py:1172
ANA_MSG_INFO
#define ANA_MSG_INFO(xmsg)
Macro printing info messages.
Definition: Control/AthToolSupport/AsgMessaging/AsgMessaging/MessageCheck.h:290
add-xsec-uncert-quadrature-N.label
label
Definition: add-xsec-uncert-quadrature-N.py:104
file
TFile * file
Definition: tile_monitor.h:29
dumpFileToPlots.treeName
string treeName
Definition: dumpFileToPlots.py:19
xAOD::uint64_t
uint64_t
Definition: EventInfo_v1.cxx:123
SH::openFile
std::unique_ptr< TFile > openFile(const std::string &name, const MetaObject &options)
open a file with the given options
Definition: ToolsOther.cxx:35
hist_file_dump.f
f
Definition: hist_file_dump.py:140
EL::EventRange::eof
static constexpr Long64_t eof
the special value to indicate that the range includes all events until the end of the file
Definition: EventRange.h:34
SH::MetaObject::castString
std::string castString(const std::string &name, const std::string &def_val="", CastMode mode=CAST_ERROR_THROW) const
the meta-data string with the given name
histSizes.list
def list(name, path='/')
Definition: histSizes.py:38
EL::Worker::m_moduleConfig
std::vector< asg::AsgComponentConfig > m_moduleConfig
the module configurations we use
Definition: Worker.h:446
python.AtlRunQueryLib.options
options
Definition: AtlRunQueryLib.py:378
EL::Job::optCacheLearnEntries
static const std::string optCacheLearnEntries
description: this option allows to configure the number of tree entries used for learning cache behav...
Definition: Job.h:327
EL::Job::optMemFailOnLeak
static const std::string optMemFailOnLeak
Failure behaviour of the code when a "significant memory leak" is found.
Definition: Job.h:594
EL::Worker::inputFileNumEntries
Long64_t inputFileNumEntries() const override
the number of events in the input file
Definition: Worker.cxx:764
SH::MetaFields::treeName
static const std::string treeName
the name of the tree in the sample
Definition: MetaFields.h:52
EL::Worker::setJobConfig
void setJobConfig(JobConfig &&jobConfig)
set the JobConfig
Definition: Worker.cxx:366
RCU_INVARIANT
#define RCU_INVARIANT(x)
Definition: Assert.h:201
beamspotman.jobConfig
dictionary jobConfig
Definition: beamspotman.py:1067
EL::Job::optAlgorithmTimer
static const std::string optAlgorithmTimer
a boolean flag for whether to add a timer for the algorithms
Definition: Job.h:205
ReadFromCoolCompare.os
os
Definition: ReadFromCoolCompare.py:231
EL::Job::optMemResidentIncreaseLimit
static const std::string optMemResidentIncreaseLimit
The minimal resident memory increase necessary to trigger an error.
Definition: Job.h:572
merge.output
output
Definition: merge.py:16
EL::Job::histogramStreamName
static const std::string histogramStreamName
the name of the histogram output stream
Definition: Job.h:605
EL::Worker::EC_BADINPUT
@ EC_BADINPUT
Definition: Worker.h:254
EL::Job::optAlgorithmMemoryMonitor
static const std::string optAlgorithmMemoryMonitor
a boolean flag for whether to add a memory monitor for the algorithms
Definition: Job.h:210
EL::Worker::addOutputStream
::StatusCode addOutputStream(const std::string &label, Detail::OutputStreamData output)
effects: add another output file guarantee: strong failures: low level errors II failures: label alre...
Definition: Worker.cxx:739
name
std::string name
Definition: Control/AthContainers/Root/debug.cxx:240
RCU::SetDirectory
bool SetDirectory(TObject *object, TDirectory *directory)
effects: set the directory this object is associated with returns: whether the object type actively k...
Definition: RootUtils.cxx:28
RootUtils::WithRootErrorHandler
Run a MT piece of code with an alternate root error handler.
Definition: WithRootErrorHandler.h:56
RegSelToolConfig.alg
alg
Definition: RegSelToolConfig.py:332
EL::Detail::ModuleData::m_inputTreeEntry
uint64_t m_inputTreeEntry
the entry in the input tree we are currently looking at
Definition: ModuleData.h:79
EL::Detail::ModuleData::m_algs
std::vector< Detail::AlgorithmData > m_algs
the list of algorithms
Definition: ModuleData.h:66
python.LumiBlobConversion.pos
pos
Definition: LumiBlobConversion.py:16
EL::Detail::ModuleData::m_inputFileUrl
std::string m_inputFileUrl
the input file url of the currently opened file
Definition: ModuleData.h:69
EL::Detail::ModuleData::m_histOutput
OutputStreamData * m_histOutput
the histogram output stream
Definition: ModuleData.h:91
EL::Detail::ModuleData::m_eventsProcessed
uint64_t m_eventsProcessed
the number of events that have been processed
Definition: ModuleData.h:88
RCU_DESTROY_INVARIANT
#define RCU_DESTROY_INVARIANT(x)
Definition: Assert.h:235
EL::Detail::ModuleData::m_tevent
xAOD::TEvent * m_tevent
the TEvent structure, if we use one
Definition: ModuleData.h:97
EL::Job::outputIter
const OutputStream * outputIter
Definition: Job.h:143
ReadCellNoiseFromCoolCompare.s2
s2
Definition: ReadCellNoiseFromCoolCompare.py:379
SH::MetaObject::castDouble
double castDouble(const std::string &name, double def_val=0, CastMode mode=CAST_ERROR_THROW) const
the meta-data double with the given name
EL::Detail::ModuleData::m_skipEvent
bool m_skipEvent
whether we are skipping the current event
Definition: ModuleData.h:82
RCU_CHANGE_INVARIANT
#define RCU_CHANGE_INVARIANT(x)
Definition: Assert.h:231
EL::Worker::setMetaData
void setMetaData(const SH::MetaObject *val_metaData)
set the metaData
Definition: Worker.cxx:335
EL::Worker::treeEntry
Long64_t treeEntry() const override
description: the entry in the tree we are reading guarantee: no-fail
Definition: Worker.cxx:219
RCU_THROW_MSG
#define RCU_THROW_MSG(message)
Definition: PrintMsg.h:58
EL::Job::optSkipEvents
static const std::string optSkipEvents
description: the name of the option used for skipping a certain number of events in the beginning rat...
Definition: Job.h:233
EL::Worker::initialize
::StatusCode initialize()
initialize the worker
Definition: Worker.cxx:378
SH::MetaFields::treeName_default
static const std::string treeName_default
the default value of treeName
Definition: MetaFields.h:55
EL::Detail::OutputStreamData::saveOutput
void saveOutput()
write the list of output objects to disk and clear it
Definition: OutputStreamData.cxx:192
EL::Detail::ModuleData::m_worker
Worker * m_worker
the worker (to pass on to the algorithms)
Definition: ModuleData.h:106
python.PyAthena.obj
obj
Definition: PyAthena.py:132
RCU_ASSERT
#define RCU_ASSERT(x)
Definition: Assert.h:222
checkJobs.completed
completed
Definition: checkJobs.py:23
EL::Job::optMemVirtualPerEventIncreaseLimit
static const std::string optMemVirtualPerEventIncreaseLimit
The minimal per-event virtual memory increase for triggering an error.
Definition: Job.h:564
EL::Detail::ModuleData::m_metaData
const SH::MetaObject * m_metaData
the meta-data we use
Definition: ModuleData.h:85
RCU_READ_INVARIANT
#define RCU_READ_INVARIANT(x)
Definition: Assert.h:229
EL::Detail::report_exception
void report_exception(std::exception_ptr eptr)
print out the currently evaluated exception
Definition: PhysicsAnalysis/D3PDTools/EventLoop/Root/MessageCheck.cxx:27
python.AutoConfigFlags.msg
msg
Definition: AutoConfigFlags.py:7
EL::Worker::tree
TTree * tree() const override
description: the tree we are running on guarantee: no-fail
Definition: Worker.cxx:210
Trk::split
@ split
Definition: LayerMaterialProperties.h:38
getEFTrackSample.sampleName
sampleName
Definition: getEFTrackSample.py:13
EL::Job::optMemVirtualIncreaseLimit
static const std::string optMemVirtualIncreaseLimit
The minimal virtual memory increase necessary to trigger an error.
Definition: Job.h:580
EL::Worker::addOutputList
void addOutputList(const std::string &name, TObject *output_swallow) override
effects: add a given object to the output.
Definition: Worker.cxx:103
EL::Worker::processInputs
::StatusCode processInputs()
process all the inputs
Definition: Worker.cxx:493
EL::Job::optGridReporting
static const std::string optGridReporting
whether to use grid reporting even when not running on the grid
Definition: Job.h:497
EL::Worker::m_segmentName
std::string m_segmentName
the name of the segment we are processing
Definition: Worker.h:431
NSWL1::PadTriggerAdapter::segment
Muon::NSW_PadTriggerSegment segment(const NSWL1::PadTrigger &data)
Definition: PadTriggerAdapter.cxx:5
RCU_NEW_INVARIANT
#define RCU_NEW_INVARIANT(x)
Definition: Assert.h:233