Base class for configurable objects in the jobOptions. More...

Inheritance diagram for python.ProphecyConfig.ProphecyConfig:

Collaboration diagram for python.ProphecyConfig.ProphecyConfig:

Public Types
typedef HLT::TypeInformation::for_each_type_c< typenameEDMLIST::map, my_functor, my_result<>, my_arg< HLT::TypeInformation::get_cont, CONTAINER > >::type	result

Public Member Functions
	__init__ (self, runArgs=None, opts=None)
	generate (self, filter_name='', filter_args='')
	Run normal event generation.
	generateRunCard (self)
	Initialise runcard with generic options.
	generateEvents (self)
	Run normal event generation.
	add_parameter (self, configurable_name, value, desc='', parameter=None)
	Register configurable parameter.
	add_parameter_set (self, parameter_set, **kwargs)
	Alias to DecoratorFactory.decorate.
	emit_heartbeat (self, duration)
	Output a heartbeat message.
	fix_parameter (self, parameter, value=None, desc='')
	Register non-configurable parameter.
	runProphecy (configurator)
	run_directory (self)
	Get run directory.
	configurable_parameters (self)
	Get dictionary of configurable parameters.
	fixed_parameters (self)
	Get list of non-configurable parameters.
	logger (self)
	Get handle to logger.
	output_events_file_name (self)
	Get output file name.
	output_events_file_name (self, value)
	Set output file name.
	prophecy_directory (self)
	Get Prophecy directory.
	run_card_decorators (self)
	Get list of enabled run card decorators.
	run_card_path (self)
	Get full path to runcard.

Public Attributes
	nEvents = int( 1.1 * runArgs.maxEvents + 0.5 )
	Add universal functionality.
	random_seed = runArgs.randomSeed
int	nEvents_weighted
	Print final preparation message.
	run_card_decorators
	Finalise registered decorators.
	run_card_path
	Write out final runcard.
	fixed_parameters
	Write out final runcard.
list	running_process = []
	Remove any existing .lhe files to avoid repeated events.
	output_events_file_name
	Move output to correctly named file.
	run_directory

Static Protected Attributes
str	_prophecy_executable = 'Prophecy4f'
	This must be defined by each derived class - don't change it in the jobOptions!

Private Attributes
str	__output_events_file_name = 'ProphecyOTF._1.events'
	This needs to be set so that Generate_trf finds an appropriate file format for showering.
list	__fixed_parameters = []
	Set up lists of parameters and decorators.
dict	__configurable_parameters = {}
list	__run_card_decorators = []
bool	__enable_reweighting = True

Static Private Attributes
	__run_directory = os.environ['PWD']
	__logger = Logging.logging.getLogger('ProphecyControl')
	Setup athena-compatible logger.

Detailed Description

Base class for configurable objects in the jobOptions.

All subprocesses inherit from this class

Set up run directory and path to Prophecy

Definition at line 12 of file ProphecyConfig.py.

Member Typedef Documentation

◆ result

typedef HLT::TypeInformation::for_each_type_c<typenameEDMLIST::map,my_functor,my_result<>,my_arg<HLT::TypeInformation::get_cont,CONTAINER>>::type master_search< typenameEDMLIST::map, HLT::TypeInformation::get_cont, CONTAINER >::result

inherited

Definition at line 90 of file EDM_MasterSearch.h.

Constructor & Destructor Documentation

◆ init()

python.ProphecyConfig.ProphecyConfig.__init__	(	self,
		runArgs = None,
		opts = None )

Definition at line 22 of file ProphecyConfig.py.

 
  def __init__( self, runArgs=None, opts=None ) :
 
    
    self.__output_events_file_name = 'ProphecyOTF._1.events'
 
    
    self.__fixed_parameters = []
    self.__configurable_parameters = {}
    self.__run_card_decorators = []
 
    
    self.add_parameter_set( 'fromDefault' )
 
    
    if runArgs is None :
      self.logger.warning( 'No run arguments found! Using defaults.' )
    else :
      # Read values from runArgs
      if hasattr(runArgs,'maxEvents') and runArgs.maxEvents > 0 :
        self.nEvents = int( 1.1 * runArgs.maxEvents + 0.5 )
      if hasattr(runArgs,'randomSeed') :
        self.random_seed = runArgs.randomSeed
 
      if 100 * self.nEvents > self.nEvents_weighted :
        self.logger.warning( 'There are {0} events requested with {1} weighted ones.'.format( self.nEvents, self.nEvents_weighted ) )
        self.nEvents_weighted = 100 * self.nEvents
        self.logger.warning( 'Forcing weighted events to be {0}.'.format( self.nEvents_weighted ) )
 

Member Function Documentation

◆ add_parameter()

python.ProphecyConfig.ProphecyConfig.add_parameter	(	self,
		configurable_name,
		value,
		desc = '',
		parameter = None )

Register configurable parameter.

Definition at line 147 of file ProphecyConfig.py.

  
  def add_parameter( self, configurable_name, value, desc='', parameter=None ) :
    setattr( self, configurable_name, value ) # add new attribute
    prophecy_parameter = parameter if parameter is not None else configurable_name
    self.configurable_parameters[prophecy_parameter] = ( configurable_name, desc )
 

◆ add_parameter_set()

python.ProphecyConfig.ProphecyConfig.add_parameter_set	(		self,
			parameter_set,
		**	kwargs )

Alias to DecoratorFactory.decorate.

Definition at line 154 of file ProphecyConfig.py.

  
  def add_parameter_set( self, parameter_set, **kwargs ) :
    return decorate( self, parameter_set, **kwargs )

◆ configurable_parameters()

python.ProphecyConfig.ProphecyConfig.configurable_parameters ( self )

Get dictionary of configurable parameters.

Definition at line 207 of file ProphecyConfig.py.

  @property
  def configurable_parameters(self) :
    return self.__configurable_parameters
 

◆ emit_heartbeat()

python.ProphecyConfig.ProphecyConfig.emit_heartbeat	(		self,
			duration )

Output a heartbeat message.

Definition at line 159 of file ProphecyConfig.py.

  
  def emit_heartbeat(self, duration) :
    message = 'Heartbeat: Prophecy generation has been running for {0} in total'.format( RepeatingTimer.human_readable_time_interval(duration) ) # noqa: F821
    self.logger.info( message )
    with open( '{0}/eventLoopHeartBeat.txt'.format( self.__run_directory ), 'w' ) as f : f.write( message )

◆ fix_parameter()

python.ProphecyConfig.ProphecyConfig.fix_parameter	(	self,
		parameter,
		value = None,
		desc = '' )

Register non-configurable parameter.

Definition at line 165 of file ProphecyConfig.py.

  
  def fix_parameter( self, parameter, value=None, desc='' ) :
    # Get previously set value if not overwriting
    if value is None : value = getattr( self, parameter )
    # Remove it from the configurable list if it was there
    for prophecy_parameter, configurable_name_tuple in self.configurable_parameters.items() :
      # Retrieve Prophecy parameter name and description if there is a match
      if parameter == configurable_name_tuple[0] :
        parameter, desc = prophecy_parameter, configurable_name_tuple[1]
        self.configurable_parameters.pop(prophecy_parameter)
        break
    self.fixed_parameters.append( (parameter, value, desc) )

◆ fixed_parameters()

python.ProphecyConfig.ProphecyConfig.fixed_parameters ( self )

Get list of non-configurable parameters.

Definition at line 213 of file ProphecyConfig.py.

  @property
  def fixed_parameters(self) :
    return self.__fixed_parameters
 

◆ generate()

python.ProphecyConfig.ProphecyConfig.generate	(	self,
		filter_name = '',
		filter_args = '' )

Run normal event generation.

Definition at line 52 of file ProphecyConfig.py.

  
  def generate( self, filter_name='', filter_args='' ) :
    self.generateRunCard()
    self.generateEvents()

◆ generateEvents()

python.ProphecyConfig.ProphecyConfig.generateEvents ( self )

Run normal event generation.

Definition at line 91 of file ProphecyConfig.py.

  
  def generateEvents(self) :
    
    time_start = time.time()
    self.logger.info( 'Starting Prophecy LHEF event generation at {0}'.format( time.ctime( time_start ) ) )
 
    
    heartbeat = HeartbeatTimer(600., "{}/eventLoopHeartBeat.txt".format(self.__run_directory))
    heartbeat.setName("heartbeat thread")
    heartbeat.daemon = True  # Allow program to exit if this is the only live thread
    heartbeat.start()
 
    
    self.logger.info( 'Removing old Prophecy LHE files' )
    try :
      os.remove( 'plot*.lhe plot*.ev*ts' )
    except OSError :
      pass
 
    
    if not os.path.exists('UNWEIGHTEDEVENTS'):
      os.makedirs('UNWEIGHTEDEVENTS')
      self.logger.info( 'Created directory UNWEIGHTEDEVENTS' )
 
    
    self.running_process = []
 
    self.runProphecy()
 
    
    heartbeat.cancel()
 
    
    generation_end = time.time()
    elapsed_time = generation_end - time_start
    self.logger.info( 'Running nominal Prophecy took {0} for {1} events => {2:6.3f} Hz'.format( HeartbeatTimer.readable_duration(elapsed_time), self.nEvents, self.nEvents / elapsed_time ) )
 
    self.logger.info( 'Removing Prophecy born LHE file' )
    try :
      os.remove( 'plot_unweighted_born.lhe' )
    except OSError :
      pass
 
    
    try :
      os.rename( 'UNWEIGHTEDEVENTS/plot_unweighted.lhe', self.output_events_file_name )
      self.logger.info( 'Moved plot_unweighted.lhe to {0}'.format(self.output_events_file_name) )
    except OSError :
      self.logger.warning( 'No output LHEF file found! Probably because the Prophecy process was killed before finishing.' )
 
    
    self.logger.info( 'Finished at {0}'.format( time.asctime() ) )
    return
 

◆ generateRunCard()

python.ProphecyConfig.ProphecyConfig.generateRunCard ( self )

Initialise runcard with generic options.

Definition at line 57 of file ProphecyConfig.py.

  
  def generateRunCard(self) :
 
    
    for run_card_decorator in self.run_card_decorators :
      if hasattr( run_card_decorator, 'finalise' ) : run_card_decorator.finalise()
 
    
    self.logger.info( '** User configurable parameters for this process **' )
    self.logger.info( ': Configurable parameter :   Current value   : Description' )
    for value_tuple in sorted( self.configurable_parameters.values(), key=lambda x: x[0].lower() ) :
      self.logger.info( ': {0:<22} : {1:>17} : {2}'.format( value_tuple[0], getattr(self, value_tuple[0]), value_tuple[1] ) )
 
    
    [ self.fix_parameter( parameter=value_tuple[0], desc=value_tuple[1] ) for value_tuple in self.configurable_parameters.values() ]
 
    
    self.logger.info( 'Writing Prophecy runcard to {0}'.format( self.run_card_path ) )
    with open( self.run_card_path, 'w' ) as f :
      for parameter_tuple in sorted( self.fixed_parameters, key=lambda x: x[0].lower() ) :
        name, value, desc = parameter_tuple
        # Set starting value to first in list when multiple values are provided
        if isinstance(value,list) :
          value = value[0]
          self.__enable_reweighting = True
        output_line = '{0:<30}! {1}'.format( '{0}={1}'.format( name, value ), desc )
        f.write( '{0}\n'.format(output_line) )
        self.logger.info( 'Wrote {0}'.format( output_line ) )
 
    
    self.logger.info( 'Using executable: {0}'.format( self._prophecy_executable ) )
    self.logger.info( 'Number of events and weight events: {0} {1}.'.format( self.nEvents, self.nEvents_weighted ) )
 

◆ logger()

python.ProphecyConfig.ProphecyConfig.logger ( self )

Get handle to logger.

Definition at line 219 of file ProphecyConfig.py.

  @property
  def logger(self) :
    return self.__logger
 

◆ output_events_file_name() [1/2]

python.ProphecyConfig.ProphecyConfig.output_events_file_name ( self )

Get output file name.

Definition at line 225 of file ProphecyConfig.py.

  @property
  def output_events_file_name(self) :
    return self.__output_events_file_name

◆ output_events_file_name() [2/2]

python.ProphecyConfig.ProphecyConfig.output_events_file_name	(		self,
			value )

Set output file name.

Definition at line 230 of file ProphecyConfig.py.

  @output_events_file_name.setter
  def output_events_file_name(self, value) :
    self.__output_events_file_name = value
 

◆ prophecy_directory()

python.ProphecyConfig.ProphecyConfig.prophecy_directory ( self )

Get Prophecy directory.

Definition at line 236 of file ProphecyConfig.py.

  @property
  def prophecy_directory(self) :
    return self.__prophecy_directory
 

◆ run_card_decorators()

python.ProphecyConfig.ProphecyConfig.run_card_decorators ( self )

Get list of enabled run card decorators.

Definition at line 242 of file ProphecyConfig.py.

  @property
  def run_card_decorators(self) :
    return self.__run_card_decorators
 

◆ run_card_path()

python.ProphecyConfig.ProphecyConfig.run_card_path ( self )

Get full path to runcard.

Definition at line 248 of file ProphecyConfig.py.

  @property
  def run_card_path(self) :
    return '{0}/prophecy.input'.format( self.run_directory )
 

◆ run_directory()

python.ProphecyConfig.ProphecyConfig.run_directory ( self )

Get run directory.

Definition at line 201 of file ProphecyConfig.py.

  @property
  def run_directory(self) :
    return self.__run_directory
 

◆ runProphecy()

python.ProphecyConfig.ProphecyConfig.runProphecy ( configurator )

Definition at line 177 of file ProphecyConfig.py.

 
  def runProphecy(configurator) :
    configurator.running_process.append(subprocess.Popen( [configurator._prophecy_executable,''], stdout=subprocess.PIPE, stdin=open(configurator.run_card_path), stderr=subprocess.STDOUT ) )
 
    while configurator.running_process :
     # Write output buffer if any
     for process in configurator.running_process :
       while True :
         output = process.stdout.readline().rstrip()
         if len(output) == 0 : break
         configurator.logger.info( '{0}'.format(output) )
       if process.poll() is not None : # process has ended
         # Flush buffer and print final output (if any) to screen
         process.stdout.flush()
         while True :
           output = process.stdout.readline().rstrip()
           if len(output) == 0 : break
           configurator.logger.info( '{0}'.format(output) )
         # Close output stream and remove process from list
         process.stdout.close()
         configurator.running_process.remove( process )
         configurator.logger.info( 'Prophecy finished - all done.' )

Member Data Documentation

◆ __configurable_parameters

dict python.ProphecyConfig.ProphecyConfig.__configurable_parameters = {}

private

Definition at line 29 of file ProphecyConfig.py.

◆ __enable_reweighting

bool python.ProphecyConfig.ProphecyConfig.__enable_reweighting = True

private

Definition at line 80 of file ProphecyConfig.py.

◆ __fixed_parameters

list python.ProphecyConfig.ProphecyConfig.__fixed_parameters = []

private

Set up lists of parameters and decorators.

Definition at line 28 of file ProphecyConfig.py.

◆ __logger

python.ProphecyConfig.ProphecyConfig.__logger = Logging.logging.getLogger('ProphecyControl')

staticprivate

Setup athena-compatible logger.

Definition at line 16 of file ProphecyConfig.py.

◆ __output_events_file_name

str python.ProphecyConfig.ProphecyConfig.__output_events_file_name = 'ProphecyOTF._1.events'

private

This needs to be set so that Generate_trf finds an appropriate file format for showering.

Definition at line 25 of file ProphecyConfig.py.

◆ __run_card_decorators

list python.ProphecyConfig.ProphecyConfig.__run_card_decorators = []

private

Definition at line 30 of file ProphecyConfig.py.

◆ __run_directory

python.ProphecyConfig.ProphecyConfig.__run_directory = os.environ['PWD']

staticprivate

Definition at line 13 of file ProphecyConfig.py.

◆ _prophecy_executable

python.ProphecyConfig.ProphecyConfig._prophecy_executable = 'Prophecy4f'

staticprotected

This must be defined by each derived class - don't change it in the jobOptions!

Print final preparation message.

For rel 21.6, no longer need to set explicit path because prophecy4f is in $PATH

Definition at line 20 of file ProphecyConfig.py.

◆ fixed_parameters

python.ProphecyConfig.ProphecyConfig.fixed_parameters

Write out final runcard.

Definition at line 75 of file ProphecyConfig.py.

◆ nEvents

python.ProphecyConfig.ProphecyConfig.nEvents = int( 1.1 * runArgs.maxEvents + 0.5 )

Add universal functionality.

Print final preparation message.

Initialise values from runArgs

Definition at line 41 of file ProphecyConfig.py.

◆ nEvents_weighted

python.ProphecyConfig.ProphecyConfig.nEvents_weighted

Print final preparation message.

Definition at line 45 of file ProphecyConfig.py.

◆ output_events_file_name

python.ProphecyConfig.ProphecyConfig.output_events_file_name

Move output to correctly named file.

Definition at line 136 of file ProphecyConfig.py.

◆ random_seed

python.ProphecyConfig.ProphecyConfig.random_seed = runArgs.randomSeed

Definition at line 43 of file ProphecyConfig.py.

◆ run_card_decorators

python.ProphecyConfig.ProphecyConfig.run_card_decorators

Finalise registered decorators.

Definition at line 60 of file ProphecyConfig.py.

◆ run_card_path

python.ProphecyConfig.ProphecyConfig.run_card_path

Write out final runcard.

Definition at line 73 of file ProphecyConfig.py.

◆ run_directory

python.ProphecyConfig.ProphecyConfig.run_directory

Definition at line 249 of file ProphecyConfig.py.

◆ running_process

list python.ProphecyConfig.ProphecyConfig.running_process = []

Remove any existing .lhe files to avoid repeated events.

The following is only needed for newer installation of prophecy?? RDS 2019/07 Create dirs UNWEIGHTEDEVENTS and HISTUNWEIGHTED

Initialise generation process tracker

Definition at line 116 of file ProphecyConfig.py.

The documentation for this class was generated from the following file:

ProphecyConfig.py

Public Types

Public Member Functions

Public Attributes

Static Protected Attributes

Private Attributes

Static Private Attributes

Detailed Description

Member Typedef Documentation

◆ result

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ add_parameter()

◆ add_parameter_set()

◆ configurable_parameters()

◆ emit_heartbeat()

◆ fix_parameter()

◆ fixed_parameters()

◆ generate()

◆ generateEvents()

◆ generateRunCard()

◆ logger()

◆ output_events_file_name() [1/2]

◆ output_events_file_name() [2/2]

◆ prophecy_directory()

◆ run_card_decorators()

◆ run_card_path()

◆ run_directory()

◆ runProphecy()

Member Data Documentation

◆ __configurable_parameters

◆ __enable_reweighting

◆ __fixed_parameters

◆ __logger

◆ __output_events_file_name

◆ __run_card_decorators

◆ __run_directory

◆ _prophecy_executable

◆ fixed_parameters

◆ nEvents

◆ nEvents_weighted

◆ output_events_file_name

◆ random_seed

◆ run_card_decorators

◆ run_card_path

◆ run_directory

◆ running_process

◆ init()