EL Namespace Reference

This module defines the arguments passed from the BATCH driver to the BATCH worker. More...

Namespaces
	Detail

Classes
class	Algorithm
class	AlgorithmMemoryWrapper
	an IAlgorithmWrapper that adds a memory monitor to an algorithm More...
class	AlgorithmTimerWrapper
	an IAlgorithmWrapper that adds a timer to an algorithm More...
struct	AlgorithmWorkerData
	all the external components an algorithm needs before initialization (in EventLoop) More...
class	AlgorithmWrapper
	an IAlgorithmWrapper for AnaAlgorithm More...
class	AlgSelect
class	AnaAlgorithm
	the (new) base class for EventLoop algorithms More...
class	AnaAlgorithmConfig
	an object that can create a AnaAlgorithm More...
class	AnaAlgorithmWrapper
	an IAlgorithmWrapper for AnaAlgorithm More...
class	AnaReentrantAlgorithm
	the base class for EventLoop reentrant algorithms More...
class	AnaReentrantAlgorithmConfig
	an object that can create a AnaReentrantAlgorithm More...
class	AnaReentrantAlgorithmWrapper
	an IAlgorithmWrapper for AnaReentrantAlgorithm More...
class	AsgServiceWrapper
	an IAlgorithmWrapper for AsgService More...
class	AsgToolWrapper
	an IAlgorithmWrapper for AsgTool More...
class	BatchDriver
	the base class for all drivers running on batch systems More...
struct	BatchJob
struct	BatchSample
struct	BatchSegment
class	CondorDriver
	a Driver to run on Condor batch systems More...
class	DirectDriver
	a Driver that runs directly inside the submission job itself More...
class	Driver
	the base class for the various EventLoop drivers that allow to run jobs on different backends More...
class	DuplicateChecker
struct	EventRange
	a range of events in a given file More...
class	ExecDriver
	a Driver for running batch jobs locally as a new process More...
class	GEDriver
	a Driver for running on GE batch systems More...
class	IAlgorithmWrapper
	a wrapper around a generic algorithm for use within EventLoop More...
class	IFilterWorker
	the interface to the filter functions in the algorithm sequence More...
class	IHistogramWorker
	the interface to histogram storage on the worker More...
class	ITreeWorker
	The interface to TTree storage on the worker. More...
class	IUnitTestTool
	the interface for UnitTestTool More...
class	IWorker
	the interface for algorithms to access IWorker More...
class	Job
class	JobConfig
	the job configuration that is independent of driver and dataset More...
class	KubernetesDriver
	a Driver to run on Kubernetes More...
class	LLDriver
	a Driver for running on IBM Load Leveler batch systems More...
class	LocalDriver
	a Driver for running batch jobs locally for testing purposes More...
class	LSFDriver
	a Driver for running on LSF batch systems More...
class	MetricsSvc
class	NTupleSvc
class	OutputStream
class	PrunDriver
	a Driver to submit jobs via prun More...
class	PythonConfigBase
	the base class for the python configuration of any asg::AsgComponent More...
class	SlurmDriver
	a Driver for running on SLURM batch systems More...
class	SoGEDriver
	a Driver for running on SoGE(?) batch systems More...
class	TorqueDriver
	a Driver for running on Torque batch systems More...
struct	UnitTest
class	UnitTestAlg
class	UnitTestAlg1
class	UnitTestAlg2
class	UnitTestAlg3
	Algorithm capable of leaking a set amount of memory per event. More...
class	UnitTestAlg4
	Algorithm testing file metadata access. More...
class	UnitTestAlg5
class	UnitTestAlg6
class	UnitTestAlg7
	a AnaAlgorithm for testing the configuration on the worker node More...
class	UnitTestAlgXAOD
struct	UnitTestConfig
	a class describing how to run the jobs for driver unit tests More...
class	UnitTestFixture
class	UnitTestSkim
struct	UnitTestTool
	a tool used to unit test AnaToolHandle More...
class	VomsProxySvc
class	WhiteBoardSvc
class	Worker
class	WorkerConfig

Typedefs
typedef ::StatusCode	StatusCode
	StatusCode definition for legacy code. More...

Functions
void	processEmptyFiles (const std::string &submitdir, const Job &job)
	check the output of the given (completed) job, rerun all (empty) files that did not get a fileExecute() and merge in the outputs More...
void	swap (Job &a, Job &b)
	effects: standard swap guarantee: no-fail More...
NTupleSvc *	getNTupleSvc (IWorker *worker, const std::string &outputStream, const std::string &treeName="")
	effects: get the skimming algorithm for the given output for this worker guarantee: strong failures: formula service not configured requires: worker != 0 requires: !outputStream.empty() postcondition: result != 0 More...
WhiteBoardSvc *	getWhiteBoardSvc (IWorker *worker)
	effects: get the whiteboard service for this worker guarantee: strong failures: formula service not configured requires: worker != 0 requires: !outputStream.empty() postcondition: result != 0 More...
	TEST_P (UnitTestFixture, empty_eventCount)
	TEST_P (UnitTestFixture, empty_callbacks)
	TEST_P (UnitTestFixture, empty_fileExecuted)
	TEST_P (UnitTestFixture, single_eventCount)
	TEST_P (UnitTestFixture, single_callbacks)
	TEST_P (UnitTestFixture, single_fileExecuted)
	TEST_P (UnitTestFixture, multi_eventCount)
	TEST_P (UnitTestFixture, multi_callbacks)
	TEST_P (UnitTestFixture, multi_fileExecuted)
	TEST_P (UnitTestFixture, multi_out_empty)

Detailed Description

This module defines the arguments passed from the BATCH driver to the BATCH worker.

This module defines a base class for classes that implement an interface to a white board service.

This module defines an algorithm used for unit tests of the skimming algorithm.

This module provides a lot of global definitions, forward declarations and includes that are used by all modules.

This module defines a class that describes everything that needs to be known about an output dataset.

Author

Nils Krumnack

The interface provided in this class is intended for experts only. The module is considered to be in the pre-alpha stage.

Author

Nils Krumnack

Attila Krasznahorkay

Nils Krumnack

Morten Dam Joergensen mdj@n.nosp@m.bi.d.nosp@m.k 2012

The interface provided in this module is intended for the general user. The module is considered to be in the pre-alpha stage.

Author

Romain Bouquet

As such it doesn't fall into the user vs. expert classification.

The interface provided in this module is intended for the experts only. The module is considered to be in the pre-alpha stage.

Typedef Documentation

StatusCode

typedef ::StatusCode EL::StatusCode

StatusCode definition for legacy code.

This typedef is solely here to be picked up by all the code that follows the old recommendations. At some point we may deprecate using this, but certainly not before release 22, as this has been used in all of our instructions for user code.

Definition at line 22 of file PhysicsAnalysis/D3PDTools/EventLoop/EventLoop/StatusCode.h.

Function Documentation

getNTupleSvc()

NTupleSvc* EL::getNTupleSvc	(	IWorker *	worker,
		const std::string &	outputStream,
		const std::string &	treeName = ""
	)

effects: get the skimming algorithm for the given output for this worker guarantee: strong failures: formula service not configured requires: worker != 0 requires: !outputStream.empty() postcondition: result != 0

getWhiteBoardSvc()

WhiteBoardSvc* EL::getWhiteBoardSvc

(

IWorker *

worker

)

effects: get the whiteboard service for this worker guarantee: strong failures: formula service not configured requires: worker != 0 requires: !outputStream.empty() postcondition: result != 0

processEmptyFiles()

void EL::processEmptyFiles	(	const std::string &	submitdir,
		const Job &	job
	)

check the output of the given (completed) job, rerun all (empty) files that did not get a fileExecute() and merge in the outputs

This is primarily for the PROOF driver, which can't process empty files as part of the normal operation.

Guarantee

basic

Failures

i/o errors
job errors
events found in empty files

Definition at line 182 of file EmptyFileHandler.cxx.

  {
    SH::SampleHandler shReprocess = makeReprocessSamples (submitdir);
 
    if (shReprocess.size() > 0)
    {
      std::string mysubmitdir = submitdir + "/emptyFiles";
      Job myjob = job;
      myjob.sampleHandler (shReprocess);
 
      DirectDriver driver;
      driver.submit (myjob, mysubmitdir);
 
      mergeHists (submitdir, mysubmitdir, shReprocess);
      for (auto output = job.outputBegin(),
         end = job.outputEnd(); output != end; ++ output)
      {
    mergeNtuple (submitdir + "/output-" + output->label(),
             mysubmitdir + "/output-" + output->label());
      }
    }
  }

swap()

void EL::swap	(	Job &	a,
		Job &	b
	)

effects: standard swap guarantee: no-fail

Definition at line 132 of file Job.cxx.

  {
    swap (a.m_sampleHandler, b.m_sampleHandler);
    a.m_jobConfig.swap (b.m_jobConfig);
    swap (a.m_output, b.m_output);
  }

TEST_P() [1/10]

EL::TEST_P	(	UnitTestFixture	,
		empty_callbacks
	)

Definition at line 276 of file UnitTestFixture.cxx.

  {
    TH1 *callbacks = getCallbacks ("empty");
    EXPECT_EQ (0, callbacks->GetBinContent (1 + UnitTestAlg1::CB_CHANGE_INPUT_FIRST));
    EXPECT_EQ (0, callbacks->GetBinContent (1 + UnitTestAlg1::CB_CHANGE_INPUT_OTHER));
    EXPECT_EQ (0, callbacks->GetBinContent (1 + UnitTestAlg1::CB_INITIALIZE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_HIST_INITIALIZE));
    EXPECT_EQ (0, callbacks->GetBinContent (1 + UnitTestAlg1::CB_EXECUTE));
    EXPECT_EQ (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_FILE_EXECUTE));
    EXPECT_EQ (0, callbacks->GetBinContent (1 + UnitTestAlg1::CB_FINALIZE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_HIST_FINALIZE));
  }

TEST_P() [2/10]

EL::TEST_P	(	UnitTestFixture	,
		empty_eventCount
	)

Definition at line 269 of file UnitTestFixture.cxx.

  {
    EXPECT_EQ (eventCount ("empty"), 0u);
  }

TEST_P() [3/10]

EL::TEST_P	(	UnitTestFixture	,
		empty_fileExecuted
	)

Definition at line 291 of file UnitTestFixture.cxx.

  {
    checkFileExecuted ("empty");
  }

TEST_P() [4/10]

EL::TEST_P	(	UnitTestFixture	,
		multi_callbacks
	)

Definition at line 334 of file UnitTestFixture.cxx.

  {
    TH1 *callbacks = getCallbacks ("multi");
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_CHANGE_INPUT_FIRST));
    EXPECT_LE (0, callbacks->GetBinContent (1 + UnitTestAlg1::CB_CHANGE_INPUT_OTHER));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_INITIALIZE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_HIST_INITIALIZE));
    EXPECT_EQ (100000, callbacks->GetBinContent (1 + UnitTestAlg1::CB_EXECUTE));
    EXPECT_EQ (10, callbacks->GetBinContent (1 + UnitTestAlg1::CB_FILE_EXECUTE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_FINALIZE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_HIST_FINALIZE));
  }

TEST_P() [5/10]

EL::TEST_P	(	UnitTestFixture	,
		multi_eventCount
	)

Definition at line 327 of file UnitTestFixture.cxx.

  {
    EXPECT_EQ (eventCount ("multi"), 100000u);
  }

TEST_P() [6/10]

EL::TEST_P	(	UnitTestFixture	,
		multi_fileExecuted
	)

Definition at line 349 of file UnitTestFixture.cxx.

  {
    checkFileExecuted ("multi");
  }

TEST_P() [7/10]

EL::TEST_P	(	UnitTestFixture	,
		multi_out_empty
	)

Definition at line 356 of file UnitTestFixture.cxx.

  {
    SH::SampleHandler sh;
    sh.load (getJob() + "/output-out_empty");
    SH::Sample *sample = sh.get ("multi");
    ASSERT_TRUE (sample != nullptr);
 
    for (auto fileName : sample->makeFileList())
    {
      std::unique_ptr<TFile> file (TFile::Open (fileName.c_str(), "READ"));
    }
  }

TEST_P() [8/10]

EL::TEST_P	(	UnitTestFixture	,
		single_callbacks
	)

Definition at line 305 of file UnitTestFixture.cxx.

  {
    TH1 *callbacks = getCallbacks ("single");
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_CHANGE_INPUT_FIRST));
    EXPECT_LE (0, callbacks->GetBinContent (1 + UnitTestAlg1::CB_CHANGE_INPUT_OTHER));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_INITIALIZE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_HIST_INITIALIZE));
    EXPECT_EQ (10000, callbacks->GetBinContent (1 + UnitTestAlg1::CB_EXECUTE));
    EXPECT_EQ (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_FILE_EXECUTE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_FINALIZE));
    EXPECT_LE (1, callbacks->GetBinContent (1 + UnitTestAlg1::CB_HIST_FINALIZE));
  }

TEST_P() [9/10]

EL::TEST_P	(	UnitTestFixture	,
		single_eventCount
	)

Definition at line 298 of file UnitTestFixture.cxx.

  {
    EXPECT_EQ (eventCount ("single"), 10000u);
  }

TEST_P() [10/10]

EL::TEST_P	(	UnitTestFixture	,
		single_fileExecuted
	)

Definition at line 320 of file UnitTestFixture.cxx.

  {
    checkFileExecuted ("single");
  }