ATLAS Offline Software
Public Member Functions | Private Member Functions | Private Attributes | List of all members
python.ConfigText.TextConfig Class Reference
Inheritance diagram for python.ConfigText.TextConfig:
Collaboration diagram for python.ConfigText.TextConfig:

Public Member Functions

def __init__ (self, yamlPath=None, *addDefaultBlocks=True)
 
def setConfig (self, config)
 
def preprocessConfig (self, config, algs)
 
def cleanupPlaceholders (self, config)
 
def loadConfig (self, yamlPath)
 
def printConfig (self, sort=False, jsonFormat=False)
 
def saveYaml (self, filePath='config.yaml', default_flow_style=False, **kwargs)
 
def addBlock (self, name, **kwargs)
 
def setOptions (self, **kwargs)
 
def configure (self)
 

Private Member Functions

def _addNewConfigBlocks (self, modulePath, functionName, algName, defaults=None, pos=None, superBlocks=None)
 
def _configureAlg (self, block, blockConfig, configSeq=None, containerName=None, extraOptions=None)
 

Private Attributes

 _config
 
 __loadedYaml
 
 _last
 

Detailed Description

Definition at line 33 of file ConfigText.py.

Constructor & Destructor Documentation

◆ __init__()

def python.ConfigText.TextConfig.__init__ (   self,
  yamlPath = None,
addDefaultBlocks = True 
)

Definition at line 34 of file ConfigText.py.

34  def __init__(self, yamlPath=None, *, addDefaultBlocks=True):
35  super().__init__(addDefaultBlocks=False)
36 
37  # Block to add new blocks to this object
38  self.addAlgConfigBlock(algName="AddConfigBlocks", alg=self._addNewConfigBlocks,
39  defaults={'self': self})
40  # add default blocks
41  if addDefaultBlocks:
42  self.addDefaultAlgs()
43  # load yaml
44  self._config = {}
45  # do not allow for loading multiple yaml files
46  self.__loadedYaml = False
47  if yamlPath is not None:
48  self.loadConfig(yamlPath)
49  # last is used for setOptionValue when using addBlock
50  self._last = None
51 
52 

Member Function Documentation

◆ _addNewConfigBlocks()

def python.ConfigText.TextConfig._addNewConfigBlocks (   self,
  modulePath,
  functionName,
  algName,
  defaults = None,
  pos = None,
  superBlocks = None 
)
private
Load <functionName> from <modulePath>

Definition at line 228 of file ConfigText.py.

228  def _addNewConfigBlocks(self, modulePath, functionName,
229  algName, defaults=None, pos=None, superBlocks=None):
230  """
231  Load <functionName> from <modulePath>
232  """
233  try:
234  module = importlib.import_module(modulePath)
235  fxn = getattr(module, functionName)
236  except ModuleNotFoundError as e:
237  raise ModuleNotFoundError(f"{e}\nFailed to load {functionName} from {modulePath}")
238  else:
239  sys.modules[functionName] = fxn
240  # add new algorithm to available algorithms
241  self.addAlgConfigBlock(algName=algName, alg=fxn,
242  defaults=defaults,
243  superBlocks=superBlocks,
244  pos=pos)
245  return
246 
247 

◆ _configureAlg()

def python.ConfigText.TextConfig._configureAlg (   self,
  block,
  blockConfig,
  configSeq = None,
  containerName = None,
  extraOptions = None 
)
private

Definition at line 248 of file ConfigText.py.

248  def _configureAlg(self, block, blockConfig, configSeq=None, containerName=None,
249  extraOptions=None):
250  if not isinstance(blockConfig, list):
251  blockConfig = [blockConfig]
252 
253  for options in blockConfig:
254  # Special case: propogate containerName down to subAlgs
255  if 'containerName' in options:
256  containerName = options['containerName']
257  elif containerName is not None and 'containerName' not in options:
258  options['containerName'] = containerName
259  # will check which options are associated alg and not options
260  logCPAlgTextCfg.info(f"Configuring {block.algName}")
261  seq, funcOpts = block.makeConfig(options)
262  if not seq._blocks:
263  continue
264  algOpts = seq.setOptions(options)
265  # If containerName was not set explicitly, we can now retrieve
266  # its default value
267  if containerName is None:
268  for opt in algOpts:
269  if 'name' in opt and opt['name'] == 'containerName':
270  containerName = opt.get('value', None)
271  break # Exit the loop as we've found the key
272 
273  if configSeq is not None:
274  configSeq += seq
275 
276  # propagate special extra options to subalgs
277  if extraOptions is None:
278  extraOptionsList = ["skipOnData", "skipOnMC", "onlyForDSIDs"]
279  for i in algOpts:
280  if i['name'] in extraOptionsList and i['defaultValue'] != i['value']:
281  if extraOptions is None:
282  extraOptions = {}
283  extraOptions[i['name']] = i['value']
284  else:
285  algOpts = seq.setOptions(extraOptions)
286 
287  # check to see if there are unused parameters
288  algOpts = [i['name'] for i in algOpts]
289  expectedOptions = set(funcOpts)
290  expectedOptions |= set(algOpts)
291  expectedOptions |= set(block.subAlgs)
292 
293  difference = set(options.keys()) - expectedOptions
294  difference.discard('__placeholder__')
295  if difference:
296  difference = "\n".join(difference)
297  raise ValueError(f"There are options set that are not used for "
298  f"{block.algName}:\n{difference}\n"
299  "Please check your configuration.")
300 
301  # check for sub-blocks and call this function recursively
302  for alg in self._order.get(block.algName, []):
303  if alg in options:
304  subAlg = block.subAlgs[alg]
305  self._configureAlg(subAlg, options[alg], configSeq, containerName, extraOptions)
306  return configSeq
307 
308 

◆ addBlock()

def python.ConfigText.TextConfig.addBlock (   self,
  name,
**  kwargs 
)
Create entry into dictionary representing the text configuration

Definition at line 165 of file ConfigText.py.

165  def addBlock(self, name, **kwargs):
166  """
167  Create entry into dictionary representing the text configuration
168  """
169  def setEntry(name, config, opts):
170  if '.' not in name:
171  if name not in config:
172  config[name] = opts
173  elif isinstance(config[name], list):
174  config[name].append(opts)
175  else:
176  config[name] = [config[name], opts]
177  # set last added block for setOptionValue
178  self._last = opts
179  else:
180  name, rest = name[:name.index('.')], name[name.index('.') + 1:]
181  config = config[name]
182  if isinstance(config, list):
183  config = config[-1]
184  setEntry(rest, config, opts)
185  return
186  setEntry(name, self._config, dict(kwargs))
187  return
188 
189 

◆ cleanupPlaceholders()

def python.ConfigText.TextConfig.cleanupPlaceholders (   self,
  config 
)
Remove placeholder markers after initialization.

Definition at line 85 of file ConfigText.py.

85  def cleanupPlaceholders(self, config):
86  """
87  Remove placeholder markers after initialization.
88  """
89  if not isinstance(config, dict):
90  return
91  if "__placeholder__" in config:
92  del config["__placeholder__"]
93  for key, value in config.items():
94  self.cleanupPlaceholders(value)
95 

◆ configure()

def python.ConfigText.TextConfig.configure (   self)
Process YAML configuration file and confgure added algorithms.

Definition at line 201 of file ConfigText.py.

201  def configure(self):
202  """Process YAML configuration file and confgure added algorithms."""
203  # make sure all blocks in yaml file are added (otherwise they would be ignored)
204  for blockName in self._config:
205  if blockName not in self._order[self.ROOTNAME]:
206  if not blockName:
207  blockName = list(self._config[blockName].keys())[0]
208  raise ValueError(f"Unkown block {blockName} in yaml file")
209 
210  # configure blocks
211  configSeq = ConfigSequence()
212  for blockName in self._order[self.ROOTNAME]:
213  if blockName == "AddConfigBlocks":
214  continue
215 
216  assert blockName in self._algs
217 
218  # order only applies to root blocks
219  if blockName in self._config:
220  blockConfig = self._config[blockName]
221  alg = self._algs[blockName]
222  self._configureAlg(alg, blockConfig, configSeq)
223  else:
224  continue
225  return configSeq
226 
227 

◆ loadConfig()

def python.ConfigText.TextConfig.loadConfig (   self,
  yamlPath 
)
read a YAML file. Will combine with any config blocks added using python

Definition at line 96 of file ConfigText.py.

96  def loadConfig(self, yamlPath):
97  """
98  read a YAML file. Will combine with any config blocks added using python
99  """
100  if self.__loadedYaml or isinstance(yamlPath, list):
101  raise NotImplementedError("Mering multiple yaml files is not implemented.")
102  self.__loadedYaml = True
103 
104  def merge(config, algs, path=''):
105  """Add to config block-by-block"""
106  if not isinstance(config, list):
107  config = [config]
108  # loop over list of blocks with same block name
109  for blocks in config:
110  # deal with case where empty dict is config
111  if blocks == {} and path:
112  self.addBlock(path)
113  return
114  # remove any subBlocks from block config
115  subBlocks = {}
116  for blockName in algs:
117  if blockName in blocks:
118  subBlocks[blockName] = blocks.pop(blockName)
119  # anything left should be a block and it's configuration
120  if blocks:
121  self.addBlock(path, **blocks)
122  # add in any subBlocks
123  for subName, subBlock in subBlocks.items():
124  newPath = f'{path}.{subName}' if path else subName
125  merge(subBlock, algs[subName].subAlgs, newPath)
126  return
127 
128  logCPAlgTextCfg.info(f'loading {yamlPath}')
129  config = readYaml(yamlPath)
130  # check if blocks are defined in yaml file
131  if "AddConfigBlocks" in config:
132  self._configureAlg(self._algs["AddConfigBlocks"], config["AddConfigBlocks"])
133 
134  # Preprocess the configuration dictionary (see !76767)
135  self.preprocessConfig(config, self._algs)
136 
137  merge(config, self._algs)
138 
139  # Cleanup placeholders (see !76767)
140  self.cleanupPlaceholders(config)
141 
142  return
143 
144 

◆ preprocessConfig()

def python.ConfigText.TextConfig.preprocessConfig (   self,
  config,
  algs 
)
Preprocess the configuration dictionary.
Ensure blocks with only sub-blocks are initialized with an empty dictionary.

Definition at line 61 of file ConfigText.py.

61  def preprocessConfig(self, config, algs):
62  """
63  Preprocess the configuration dictionary.
64  Ensure blocks with only sub-blocks are initialized with an empty dictionary.
65  """
66  def processNode(node, algs):
67  if not isinstance(node, dict):
68  return # Base case: not a dictionary
69  for blockName, blockContent in list(node.items()):
70  # If the block name is recognized in algs
71  if blockName in algs:
72  # If the block only defines sub-blocks, initialize it
73  if isinstance(blockContent, dict) and not any(
74  key in algs[blockName].options for key in blockContent
75  ):
76  # Ensure parent block is initialized as an empty dictionary
77  node[blockName] = {'__placeholder__': True, **blockContent}
78  # Recurse into sub-blocks
79  processNode(node[blockName], algs[blockName].subAlgs)
80 
81  # Start processing from the root of the configuration
82  processNode(config, algs)
83 

◆ printConfig()

def python.ConfigText.TextConfig.printConfig (   self,
  sort = False,
  jsonFormat = False 
)
Print YAML configuration file.

Definition at line 145 of file ConfigText.py.

145  def printConfig(self, sort=False, jsonFormat=False):
146  """Print YAML configuration file."""
147  if self._config is None:
148  raise ValueError("No configuration has been loaded.")
149  printYaml(self._config, sort, jsonFormat)
150  return
151 
152 

◆ saveYaml()

def python.ConfigText.TextConfig.saveYaml (   self,
  filePath = 'config.yaml',
  default_flow_style = False,
**  kwargs 
)
Convert dictionary representation to yaml and save

Definition at line 153 of file ConfigText.py.

153  def saveYaml(self, filePath='config.yaml', default_flow_style=False,
154  **kwargs):
155  """
156  Convert dictionary representation to yaml and save
157  """
158  logCPAlgTextCfg.info(f"Saving configuration to {filePath}")
159  config = self._config
160  with open(filePath, 'w') as outfile:
161  yaml.dump(config, outfile, default_flow_style=False, **kwargs)
162  return
163 
164 

◆ setConfig()

def python.ConfigText.TextConfig.setConfig (   self,
  config 
)
Print YAML configuration file.

Definition at line 53 of file ConfigText.py.

53  def setConfig(self, config):
54  """Print YAML configuration file."""
55  if self._config:
56  raise ValueError("Configuration has already been loaded.")
57  self._config = config
58  return
59 

◆ setOptions()

def python.ConfigText.TextConfig.setOptions (   self,
**  kwargs 
)
Set option(s) for the lsat block that was added. If an option
was added previously, will update value

Definition at line 190 of file ConfigText.py.

190  def setOptions(self, **kwargs):
191  """
192  Set option(s) for the lsat block that was added. If an option
193  was added previously, will update value
194  """
195  if self._last is None:
196  raise TypeError("Cannot set options before adding a block")
197  # points to dict with opts for last added block
198  self._last.update(**kwargs)
199 
200 

Member Data Documentation

◆ __loadedYaml

python.ConfigText.TextConfig.__loadedYaml
private

Definition at line 46 of file ConfigText.py.

◆ _config

python.ConfigText.TextConfig._config
private

Definition at line 44 of file ConfigText.py.

◆ _last

python.ConfigText.TextConfig._last
private

Definition at line 50 of file ConfigText.py.


The documentation for this class was generated from the following file:
configure
bool configure(asg::AnaToolHandle< ITrigGlobalEfficiencyCorrectionTool > &tool, ToolHandleArray< IAsgElectronEfficiencyCorrectionTool > &electronEffToolsHandles, ToolHandleArray< IAsgElectronEfficiencyCorrectionTool > &electronSFToolsHandles, ToolHandleArray< CP::IMuonTriggerScaleFactors > &muonToolsHandles, ToolHandleArray< IAsgPhotonEfficiencyCorrectionTool > &photonEffToolsHandles, ToolHandleArray< IAsgPhotonEfficiencyCorrectionTool > &photonSFToolsHandles, const std::string &triggers, const std::map< std::string, std::string > &legsPerTool, unsigned long nToys, bool debug)
Definition: TrigGlobEffCorrValidation.cxx:514
dumpHVPathFromNtuple.append
bool append
Definition: dumpHVPathFromNtuple.py:91
python.ConfigText.readYaml
def readYaml(yamlPath)
Definition: ConfigText.py:19
python.ConfigText.printYaml
def printYaml(d, sort=False, jsonFormat=False)
Definition: ConfigText.py:28
histSizes.list
def list(name, path='/')
Definition: histSizes.py:38
CxxUtils::set
constexpr std::enable_if_t< is_bitmask_v< E >, E & > set(E &lhs, E rhs)
Convenience function to set bits in a class enum bitmask.
Definition: bitmask.h:232
TCS::join
std::string join(const std::vector< std::string > &v, const char c=',')
Definition: Trigger/TrigT1/L1Topo/L1TopoCommon/Root/StringUtils.cxx:10
python.processes.powheg.ZZ.ZZ.__init__
def __init__(self, base_directory, **kwargs)
Constructor: all process options are set here.
Definition: ZZ.py:18
Trk::open
@ open
Definition: BinningType.h:40
get
T * get(TKey *tobj)
get a TObject* from a TKey* (why can't a TObject be a TKey?)
Definition: hcg.cxx:127
python.utility.LHE.merge
def merge(input_file_pattern, output_file)
Merge many input LHE files into a single output file.
Definition: LHE.py:29
python.Bindings.keys
keys
Definition: Control/AthenaPython/python/Bindings.py:798
WriteBchToCool.update
update
Definition: WriteBchToCool.py:67
merge
Definition: merge.py:1