ATLAS Offline Software
Public Member Functions | Static Public Member Functions | Public Attributes | List of all members
python.OutputAnalysisConfig.OutputAnalysisConfig Class Reference
Inheritance diagram for python.OutputAnalysisConfig.OutputAnalysisConfig:
Collaboration diagram for python.OutputAnalysisConfig.OutputAnalysisConfig:

Public Member Functions

def __init__ (self)
 
def instanceName (self)
 
def createOutputAlgs (self, config, name, vars, isMet=False)
 
def makeAlgs (self, config)
 
def createSelectionFlagBranches (self, config)
 
def makeSelectionSummaryAlg (self, config, containerName, selectionName)
 

Static Public Member Functions

def branchSortOrder (rule)
 

Public Attributes

 validated
 
 containers
 
 vars
 
 varsOnlyForMC
 
 metVars
 
 truthMetVars
 

Detailed Description

the ConfigBlock for the MET configuration

Definition at line 10 of file OutputAnalysisConfig.py.

Constructor & Destructor Documentation

◆ __init__()

def python.OutputAnalysisConfig.OutputAnalysisConfig.__init__ (   self)

Definition at line 13 of file OutputAnalysisConfig.py.

13  def __init__ (self) :
14  super (OutputAnalysisConfig, self).__init__ ()
15  self.addOption ('postfix', '', type=str,
16  info="a postfix to apply to decorations and algorithm names. "
17  "Typically not needed here.")
18  self.addOption ('vars', [], type=None,
19  info="a list of mappings (list of strings) between containers and "
20  "decorations to output branches. The default is [] (empty list).")
21  self.addOption ('varsOnlyForMC', [], type=None,
22  info="same as vars, but for MC-only variables so as to avoid a "
23  "crash when running on data. The default is [] (empty list).")
24  self.addOption ('metVars', [], type=None,
25  info="a list of mappings (list of strings) between containers "
26  "and decorations to output branches. Specficially for MET "
27  "variables, where only the final MET term is retained. "
28  "The default is [] (empty list).")
29  self.addOption ('truthMetVars', [], type=None,
30  info="a list of mappings (list of strings) between containers "
31  "and decorations to output branches for truth MET. "
32  "The default is [] (empty list).")
33  self.addOption ('containers', {}, type=None,
34  info="a dictionary mapping prefixes (key) to container names "
35  "(values) to be used when saving to the output tree. Branches "
36  "are then of the form prefix_decoration.")
37  self.addOption ('containersFullMET', {}, type=None,
38  info="same as containers, but for MET containers that should be "
39  "saved with all terms (as opposed to just the final term). This "
40  "is useful for special studies. A container can appear both here and "
41  "in containers (with different prefixes).")
42  self.addOption ('containersOnlyForMC', {}, type=None,
43  info="same as containers, but for MC-only containers so as to avoid "
44  "a crash when running on data.")
45  self.addOption ('containersOnlyForDSIDs', {}, type=None,
46  info="specify which DSIDs are allowed to produce a given container. "
47  "This works like 'onlyForDSIDs': pass a list of DSIDs or regexps.")
48  self.addOption ('treeName', 'analysis', type=str,
49  info="name of the output TTree to save. The default is analysis.")
50  self.addOption ('streamName', 'ANALYSIS', type=str,
51  info="name of the output stream to save the tree in. "
52  "The default is ANALYSIS.")
53  self.addOption ('metTermName', 'Final', type=str,
54  info="the name (string) of the MET term to save, turning the MET "
55  "container into a single object. The default is 'Final'.")
56  self.addOption ('truthMetTermName', 'NonInt', type=str,
57  info="the name (string) of the truth MET term to save, turning the MET "
58  "container into a single object. The default is 'NonInt'.")
59  # TODO: add info strng
60  self.addOption ('storeSelectionFlags', True, type=bool,
61  info="")
62  # TODO: add info strng
63  self.addOption ('selectionFlagPrefix', 'select', type=str,
64  info="")
65  self.addOption ('commands', [], type=None,
66  info="a list of strings containing commands (regexp strings "
67  "prefaced by the keywords enable or disable) to turn on/off the "
68  "writing of branches to the output ntuple. The default is None "
69  "(no modification to the scheduled output branches).")
70  self.addOption ('commandsOnlyForDSIDs', {}, type=None,
71  info="a dictionary with individual DSIDs as keys, and a list of strings "
72  "like for the 'commands' option as items. These 'commands' will only be run "
73  "for the corresponding DSID.")
74  self.addOption ('alwaysAddNosys', False, type=bool,
75  info="If set to True, all branches will be given a systematics suffix, "
76  "even if they have no systematics (beyond the nominal).")
77  self.addOption ('skipRedundantSelectionFlags', True, type=bool,
78  info="remove the redundant 'outputSelect' branches created by the Thinning step. "
79  "These could however be used to simplify downstream workflows, as in Easyjet. "
80  "The default is True.")
81  # helper to protect for second pass
82  self.validated = False
83 

Member Function Documentation

◆ branchSortOrder()

def python.OutputAnalysisConfig.OutputAnalysisConfig.branchSortOrder (   rule)
static

Definition at line 91 of file OutputAnalysisConfig.py.

91  def branchSortOrder (rule):
92  return rule.split('->')[1].strip()
93 

◆ createOutputAlgs()

def python.OutputAnalysisConfig.OutputAnalysisConfig.createOutputAlgs (   self,
  config,
  name,
  vars,
  isMet = False 
) 
A helper function to create output algorithm

Definition at line 94 of file OutputAnalysisConfig.py.

94  def createOutputAlgs (self, config, name, vars, isMet=False):
95  """A helper function to create output algorithm"""
96  alg = config.createAlgorithm('CP::AsgxAODMetNTupleMakerAlg' if isMet else 'CP::AsgxAODNTupleMakerAlg', name)
97  alg.TreeName = self.treeName
98  alg.RootStreamName = self.streamName
99  branchList = list(vars)
100  branchList.sort(key=self.branchSortOrder)
101  branchList_nosys = [branch for branch in branchList if "%SYS%" not in branch]
102  branchList_sys = [branch for branch in branchList if "%SYS%" in branch]
103  alg.Branches = branchList_nosys + branchList_sys
104  return alg
105 

◆ createSelectionFlagBranches()

def python.OutputAnalysisConfig.OutputAnalysisConfig.createSelectionFlagBranches (   self,
  config 
) 
For each container and for each selection, create a single pass variable in output NTuple,
which aggregates all the selections flag of the given selection. For example, this can include
pT, eta selections, some object ID selection, overlap removal, etc.
The goal is to have only one flag per object and working point in the output NTuple.

Definition at line 284 of file OutputAnalysisConfig.py.

284  def createSelectionFlagBranches(self, config):
285  """
286  For each container and for each selection, create a single pass variable in output NTuple,
287  which aggregates all the selections flag of the given selection. For example, this can include
288  pT, eta selections, some object ID selection, overlap removal, etc.
289  The goal is to have only one flag per object and working point in the output NTuple.
290  """
291  originalContainersSeen = []
292  for prefix in self.containers.keys() :
293  outputContainerName = self.containers[prefix]
294  containerName = config.getOutputContainerOrigin(outputContainerName)
295  if containerName in originalContainersSeen:
296  continue
297  else:
298  originalContainersSeen.append(containerName)
299 
300  # EventInfo is one obvious example of a container that has no object selections
301  if containerName == 'EventInfo':
302  continue
303 
304  selectionNames = config.getSelectionNames(containerName)
305  for selectionName in selectionNames:
306  # skip default selection
307  if selectionName == '':
308  continue
309  # skip selection coming from the Thinning block
310  if self.skipRedundantSelectionFlags and "outputSelect" in selectionName:
311  continue
312  self.makeSelectionSummaryAlg(config, containerName, selectionName)
313 

◆ instanceName()

def python.OutputAnalysisConfig.OutputAnalysisConfig.instanceName (   self) 
Return the instance name for this block

Definition at line 84 of file OutputAnalysisConfig.py.

84  def instanceName (self) :
85  """Return the instance name for this block"""
86  if self.postfix is not None and self.postfix != '':
87  return self.postfix
88  return self.treeName
89 

◆ makeAlgs()

def python.OutputAnalysisConfig.OutputAnalysisConfig.makeAlgs (   self,
  config 
)

Definition at line 106 of file OutputAnalysisConfig.py.

106  def makeAlgs (self, config) :
107 
108  log = logging.getLogger('OutputAnalysisConfig')
109 
110  # do some transformations of the options we should only do once
111  if not self.validated:
112 
113  self.containers = dict(self.containers)
114  self.vars = set(self.vars)
115  self.varsOnlyForMC = set(self.varsOnlyForMC)
116  self.metVars = set(self.metVars)
117  self.truthMetVars = set(self.truthMetVars)
118 
119  # check for overlaps between containers and containersFullMET
120  overlapping_keys = set(self.containers.keys()).intersection(self.containersFullMET.keys())
121  if overlapping_keys:
122  # convert the set of overlapping keys to a list of strings for the message (represents the empty string too!)
123  keys_message = [repr(key) for key in overlapping_keys]
124  raise KeyError(f"containersFullMET would overwrite the following container keys: {', '.join(keys_message)}")
125  # move items in self.containersFullMET to containers
126  self.containers.update(self.containersFullMET)
127 
128  # merge the MC-specific branches and containers into the main list/dictionary only if we are not running on data
129  if config.dataType() is not DataType.Data:
130  self.vars |= self.varsOnlyForMC
131 
132  # protect 'containers' against being overwritten
133  # find overlapping keys
134  overlapping_keys = set(self.containers.keys()).intersection(self.containersOnlyForMC.keys())
135  if overlapping_keys:
136  # convert the set of overlapping keys to a list of strings for the message (represents the empty string too!)
137  keys_message = [repr(key) for key in overlapping_keys]
138  raise KeyError(f"containersOnlyForMC would overwrite the following container keys: {', '.join(keys_message)}")
139 
140  # move items in self.containersOnlyForMC to self.containers
141  self.containers.update(self.containersOnlyForMC)
142  # clear the dictionary to avoid overlapping key error during the second pass
143  self.containersOnlyForMC.clear()
144 
145  # now filter the containers depending on DSIDs
146  if self.containersOnlyForDSIDs:
147  for container, dsid_filters in self.containersOnlyForDSIDs.items():
148  if container not in self.containers:
149  log.warning("Skipping unrecognised container prefix '%s' for DSID-filtering in OutputAnalysisConfig...", container)
150  continue
151  if not filter_dsids (dsid_filters, config):
152  # if current DSID is not allowed for this container, remove it
153  log.info("Skipping container prefix '%s' due to DSID filtering...", container)
154  # filter branches for validated containers
155  for var in set(self.vars): # make a copy of the list to avoid modifying it while iterating
156  var_container = var.split('.')[0].replace('_NOSYS', '').replace('_%SYS%', '')
157  if var_container == self.containers[container]:
158  self.vars.remove(var)
159  log.info("Skipping branch definition '%s' for excluded container %s...", var, var_container)
160  # remove the container from the list at the end
161  self.containers.pop (container)
162  # clear the dictionary to avoid warnings during the second pass
163  self.containersOnlyForDSIDs.clear()
164 
165  # at this point we are OK
166  self.validated = True
167 
168  if self.storeSelectionFlags:
169  self.createSelectionFlagBranches(config)
170 
171  outputConfigs = {}
172  for prefix in self.containers.keys() :
173  containerName = self.containers[prefix]
174  outputDict = config.getOutputVars (containerName)
175  for outputName in outputDict :
176  outputConfig = copy.deepcopy (outputDict[outputName])
177  if containerName != outputConfig.origContainerName or config.checkOutputContainer(containerName):
178  outputConfig.outputContainerName = containerName + '_%SYS%'
179  else:
180  outputConfig.outputContainerName = config.readName(containerName)
181  outputConfig.prefix = prefix
182  # if the container is a MET container with all terms, we
183  # also need to write out the name of each MET term
184  if prefix in self.containersFullMET and outputConfig.variableName == 'name':
185  outputConfig.enabled = True
186  outputConfigs[prefix + outputName] = outputConfig
187 
188  # check for DSID-specific commands
189  for dsid, dsid_commands in self.commandsOnlyForDSIDs.items():
190  if filter_dsids([dsid], config):
191  self.commands += dsid_commands
192 
193  outputConfigsRename = {}
194  for command in self.commands :
195  words = command.split (' ')
196  if len (words) == 0 :
197  raise ValueError ('received empty command for "commands" option')
198  optional = words[0] == 'optional'
199  if optional :
200  words = words[1:] # remove the 'optional' keyword
201  if words[0] == 'enable' :
202  if len (words) != 2 :
203  raise ValueError ('enable takes exactly one argument: ' + command)
204  used = False
205  for name in outputConfigs :
206  if re.match (words[1], name) :
207  outputConfigs[name].enabled = True
208  used = True
209  if not used and not optional and config.dataType() is not DataType.Data:
210  raise KeyError ('unknown branch pattern for enable: ' + words[1])
211  elif words[0] == 'disable' :
212  if len (words) != 2 :
213  raise ValueError ('disable takes exactly one argument: ' + command)
214  used = False
215  for name in outputConfigs :
216  if re.match (words[1], name) :
217  outputConfigs[name].enabled = False
218  used = True
219  if not used and not optional and config.dataType() is not DataType.Data:
220  raise KeyError ('unknown branch pattern for disable: ' + words[1])
221  elif words[0] == 'rename' :
222  if len (words) != 3 :
223  raise ValueError ('rename takes exactly two arguments: ' + command)
224  used = False
225  for name in outputConfigs :
226  if re.match (words[1], name) :
227  new_name = re.sub (words[1], words[2], name)
228  outputConfigsRename[new_name] = copy.deepcopy(outputConfigs[name])
229  outputConfigs[name].enabled = False
230  used = True
231  if not used and not optional and config.dataType() is not DataType.Data:
232  raise KeyError ('unknown branch pattern for rename: ' + words[1])
233  else :
234  raise KeyError ('unknown command for "commands" option: ' + words[0])
235 
236  # update the outputConfigs with renamed branches
237  outputConfigs.update(outputConfigsRename)
238 
239  autoVars = set()
240  autoMetVars = set()
241  autoTruthMetVars = set()
242  for outputName, outputConfig in outputConfigs.items():
243  if outputConfig.enabled :
244  if config.isMetContainer (outputConfig.origContainerName) and outputConfig.prefix not in self.containersFullMET:
245  if "Truth" in outputConfig.origContainerName:
246  myVars = autoTruthMetVars
247  else:
248  myVars = autoMetVars
249  else :
250  myVars = autoVars
251  if outputConfig.noSys :
252  outputConfig.outputContainerName = outputConfig.outputContainerName.replace ('%SYS%', 'NOSYS')
253  outputConfig.variableName = outputConfig.variableName.replace ('%SYS%', 'NOSYS')
254  if self.alwaysAddNosys :
255  outputName += "_NOSYS"
256  else :
257  outputName += '_%SYS%'
258  myVars.add(f"{outputConfig.outputContainerName}.{outputConfig.variableName} -> {outputName}")
259 
260  # Add an ntuple dumper algorithm:
261  treeMaker = config.createAlgorithm( 'CP::TreeMakerAlg', 'TreeMaker' )
262  treeMaker.TreeName = self.treeName
263  treeMaker.RootStreamName = self.streamName
264  # the auto-flush setting still needs to be figured out
265  #treeMaker.TreeAutoFlush = 0
266 
267  if self.vars or autoVars:
268  ntupleMaker = self.createOutputAlgs(config, 'NTupleMaker', self.vars | autoVars)
269 
270  if self.metVars or autoMetVars:
271  ntupleMaker = self.createOutputAlgs(config, 'MetNTupleMaker', self.metVars | autoMetVars, isMet=True)
272  ntupleMaker.termName = self.metTermName
273 
274  if config.dataType() is not DataType.Data and (self.truthMetVars or autoTruthMetVars):
275  ntupleMaker = self.createOutputAlgs(config, 'TruthMetNTupleMaker', self.truthMetVars | autoTruthMetVars, isMet=True)
276  ntupleMaker.termName = self.truthMetTermName
277 
278  treeFiller = config.createAlgorithm( 'CP::TreeFillerAlg', 'TreeFiller' )
279  treeFiller.TreeName = self.treeName
280  treeFiller.RootStreamName = self.streamName
281 
282 
283 

◆ makeSelectionSummaryAlg()

def python.OutputAnalysisConfig.OutputAnalysisConfig.makeSelectionSummaryAlg (   self,
  config,
  containerName,
  selectionName 
) 
Schedule an algorithm to pick up all cut flags for a given selectionName.
The summary selection flag is written to output as selectionFlagPrefix_selectionName.

Definition at line 314 of file OutputAnalysisConfig.py.

314  def makeSelectionSummaryAlg(self, config, containerName, selectionName):
315  """
316  Schedule an algorithm to pick up all cut flags for a given selectionName.
317  The summary selection flag is written to output as selectionFlagPrefix_selectionName.
318  """
319  alg = config.createAlgorithm( 'CP::AsgSelectionAlg',
320  f'ObjectSelectionSummary_{containerName}_{selectionName}')
321  selectionDecoration = f'baselineSelection_{selectionName}_%SYS%'
322  alg.selectionDecoration = f'{selectionDecoration},as_char'
323  alg.particles = config.readName (containerName)
324  alg.preselection = config.getFullSelection (containerName, selectionName)
325  config.addOutputVar (containerName, selectionDecoration, self.selectionFlagPrefix + '_' + selectionName)

Member Data Documentation

◆ containers

python.OutputAnalysisConfig.OutputAnalysisConfig.containers

Definition at line 113 of file OutputAnalysisConfig.py.

◆ metVars

python.OutputAnalysisConfig.OutputAnalysisConfig.metVars

Definition at line 116 of file OutputAnalysisConfig.py.

◆ truthMetVars

python.OutputAnalysisConfig.OutputAnalysisConfig.truthMetVars

Definition at line 117 of file OutputAnalysisConfig.py.

◆ validated

python.OutputAnalysisConfig.OutputAnalysisConfig.validated

Definition at line 82 of file OutputAnalysisConfig.py.

◆ vars

python.OutputAnalysisConfig.OutputAnalysisConfig.vars

Definition at line 114 of file OutputAnalysisConfig.py.

◆ varsOnlyForMC

python.OutputAnalysisConfig.OutputAnalysisConfig.varsOnlyForMC

Definition at line 115 of file OutputAnalysisConfig.py.

The documentation for this class was generated from the following file:
replace
std::string replace(std::string s, const std::string &s2, const std::string &s3)
Definition: hcg.cxx:307
intersection
std::vector< std::string > intersection(std::vector< std::string > &v1, std::vector< std::string > &v2)
Definition: compareFlatTrees.cxx:25
PixelModuleFeMask_create_db.remove
string remove
Definition: PixelModuleFeMask_create_db.py:83
PyAthena::repr
std::string repr(PyObject *o)
returns the string representation of a python object equivalent of calling repr(o) in python
Definition: PyAthenaUtils.cxx:106
histSizes.list
def list(name, path='/')
Definition: histSizes.py:38
CxxUtils::set
constexpr std::enable_if_t< is_bitmask_v< E >, E & > set(E &lhs, E rhs)
Convenience function to set bits in a class enum bitmask.
Definition: bitmask.h:232
TrigJetMonitorAlgorithm.items
items
Definition: TrigJetMonitorAlgorithm.py:71
python.processes.powheg.ZZ.ZZ.__init__
def __init__(self, base_directory, **kwargs)
Constructor: all process options are set here.
Definition: ZZ.py:18
python.ConfigBlock.filter_dsids
def filter_dsids(filterList, config)
Definition: ConfigBlock.py:13
VKalVrtAthena::varHolder_detail::clear
void clear(T &var)
Definition: NtupleVars.h:48
python.Bindings.keys
keys
Definition: Control/AthenaPython/python/Bindings.py:801
Generated on Tue Sep 2 2025 22:22:00 for ATLAS Offline Software by   doxygen 1.8.18