IOVDbMetaDataTool Class Reference

This is an interface to a tool used to manage the IOV Meta Data for a given object in the Meta Data Store. More...

#include <IOVDbMetaDataTool.h>

Inheritance diagram for IOVDbMetaDataTool:

Collaboration diagram for IOVDbMetaDataTool:

Public Member Functions
	IOVDbMetaDataTool (const std::string &type, const std::string &name, const IInterface *parent)
virtual	~IOVDbMetaDataTool ()
virtual StatusCode	initialize () override
	Initialize AlgTool.
virtual StatusCode	finalize () override
	Finalize AlgTool.
virtual StatusCode	beginInputFile (const SG::SourceID &) override
	Function called when a new input file is opened.
virtual StatusCode	endInputFile (const SG::SourceID &) override
	Function called when the currently open input file got completely processed.
virtual StatusCode	metaDataStop () override
	Function called when the tool should write out its metadata.
virtual void	handle (const Incident &incident) override
	Incident service handle listening for BeginInputFile and EndInputFile.
virtual StatusCode	registerFolder (const std::string &folderName, const std::string &folderDescription) const override
	Register folder in the IOV Db MetaData - done once at initialize.
virtual StatusCode	registerFolder (const std::string &folderName) const override
	Register folder in the IOV Db MetaData without folder description - uses default folder description for CondAttrListCollection.
virtual StatusCode	addPayload (const std::string &folderName, CondAttrListCollection *payload) const override
	Add an IOV and Payload for a particular folder - replaces payloads if there is an IOV overlap.
virtual StatusCode	processInputFileMetaData (const std::string &fileName) override
	Explicit call to process IOV meta data from the input meta data store, transferring it to the main meta data store.
virtual IOVMetaDataContainer *	findMetaDataContainer (const std::string &folderName) const override final
virtual void	lock_shared () const override final
virtual void	unlock_shared () const override final

Private Types
typedef ServiceHandle< StoreGateSvc >	StoreGateSvc_t
	Handles to the meta data stores.

Private Member Functions
IOVMetaDataContainer *	getMetaDataContainer (const std::string &folderName, const std::string &folderDescription) const
	return meta data container from the meta data store
void	checkOverrideRunNumber ()
	check if we should override the run number in the incoming meta data
StatusCode	overrideIOV (CondAttrListCollection *&coll) const
	override IOV with new run number
StatusCode	modifyPayload (const std::string &folderName, CondAttrListCollection *&payload) const
	Modify a Payload for a particular folder - replaces one of the internal attributes.
std::string	serializeContainerToJSON (const IOVMetaDataContainer *container) const
	Serialize IOVMetaDataContainer to JSON format for ByteStream metadata.
StatusCode	serializeIOVMetadataToBSMetadata ()
	Serialize IOV metadata to ByteStream metadata (called from processInputFileMetaData).

Private Attributes
StoreGateSvc_t	m_metaDataStore
StoreGateSvc_t	m_inputStore
StoreGateSvc_t	m_condStore
bool	m_overrideRunNumber
bool	m_overrideMinMaxRunNumber
unsigned int	m_newRunNumber
unsigned int	m_oldRunNumber
unsigned int	m_minRunNumber
unsigned int	m_maxRunNumber
std::set< std::string >	m_filesProcessed
UnsignedIntegerArrayProperty	m_minMaxRunNumbers {this, "MinMaxRunNumbers", {}, "Min/max run numbers for overriding meta data IOV"}
StringArrayProperty	m_foldersToBeModified {this, "FoldersToBeModified", {"/Simulation/Parameters"}}
StringArrayProperty	m_attributesToBeRemoved {this, "AttributesToBeRemoved", {"RandomSeedOffset"}}
bool	m_modifyFolders
Gaudi::Property< std::map< std::string, std::string > >	m_payloads
Gaudi::Property< std::vector< std::string > >	m_foldersToSerializeToBSMetadata
std::shared_mutex	m_mutex

Detailed Description

This is an interface to a tool used to manage the IOV Meta Data for a given object in the Meta Data Store.

Definition at line 39 of file IOVDbMetaDataTool.h.

Member Typedef Documentation

StoreGateSvc_t

typedef ServiceHandle<StoreGateSvc> IOVDbMetaDataTool::StoreGateSvc_t

private

Handles to the meta data stores.

Definition at line 119 of file IOVDbMetaDataTool.h.

Constructor & Destructor Documentation

IOVDbMetaDataTool()

IOVDbMetaDataTool::IOVDbMetaDataTool	(	const std::string &	type,
		const std::string &	name,
		const IInterface *	parent )

Definition at line 36 of file IOVDbMetaDataTool.cxx.

  : base_class(type, name, parent)
  , m_metaDataStore ("StoreGateSvc/MetaDataStore",      name)
  , m_inputStore    ("StoreGateSvc/InputMetaDataStore", name)
  , m_condStore     ("StoreGateSvc/ConditionStore",     name)
  , m_overrideRunNumber(false)
  , m_overrideMinMaxRunNumber(false)
  , m_newRunNumber(0)
  , m_oldRunNumber(0)
  , m_minRunNumber(0)
  , m_maxRunNumber(0)
  , m_modifyFolders(false)
{
}

~IOVDbMetaDataTool()

IOVDbMetaDataTool::~IOVDbMetaDataTool ( )

virtual

Definition at line 54 of file IOVDbMetaDataTool.cxx.

{}

Member Function Documentation

addPayload()

StatusCode IOVDbMetaDataTool::addPayload ( const std::string & folderName, CondAttrListCollection * payload ) const

overridevirtual

Add an IOV and Payload for a particular folder - replaces payloads if there is an IOV overlap.

Definition at line 372 of file IOVDbMetaDataTool.cxx.

{
  // lock the tool while it is modifying the folder
  std::scoped_lock  guard( m_mutex );
 
  ATH_MSG_DEBUG("begin addPayload ");
  
  // Check if the  folder has already been found
  IOVMetaDataContainer* cont = m_metaDataStore->tryRetrieve<IOVMetaDataContainer>(folderName);
  if(cont) {
    ATH_MSG_DEBUG("Retrieved IOVMetaDataContainer from MetaDataStore for folder " 
          << folderName);
  }
  else {
    ATH_MSG_ERROR("addPayload: Could not find IOVMetaDataContainer in MetaDataStore for folder " 
           << folderName 
           << ". One must have previously called registerFolder. ");
     return StatusCode::FAILURE;
  }
 
  // Override run number if requested
  if (m_overrideRunNumber || m_overrideMinMaxRunNumber) {
    ATH_CHECK( overrideIOV(payload) );
  }
 
  // Add payload to container
  bool success = cont->merge(payload);
  if (success) {
    ATH_MSG_DEBUG("Added new payload for folder " << folderName);
  }
  else {
    ATH_MSG_DEBUG("Could not add new payload for folder " 
          << folderName 
          << " (may be duplicate payload).");
 
    // To Do: the function implicitly assumes ownership on the payload pointer
    delete payload;
    payload = nullptr;
  }
 
  // Debug printout
  if(payload && msgLvl(MSG::DEBUG)) {
    std::ostringstream stream;
    payload->dump(stream);
    ATH_MSG_DEBUG(stream.str());
  }
 
  return StatusCode::SUCCESS;
}

beginInputFile()

StatusCode IOVDbMetaDataTool::beginInputFile ( const SG::SourceID & sid )

overridevirtual

Function called when a new input file is opened.

Definition at line 189 of file IOVDbMetaDataTool.cxx.

{
  if (!m_filesProcessed.contains(sid)) {
    ATH_CHECK(processInputFileMetaData(sid));
    m_filesProcessed.insert(sid);
  }
  return StatusCode::SUCCESS;
}

checkOverrideRunNumber()

void IOVDbMetaDataTool::checkOverrideRunNumber ( )

private

check if we should override the run number in the incoming meta data

Definition at line 258 of file IOVDbMetaDataTool.cxx.

{
    ATH_MSG_DEBUG("begin checkOverrideRunNumber");
 
    // Check if override run numbers have been set by properties or by
    // the EventSelector
 
    if (m_minMaxRunNumbers.value().size() > 0) {
        m_overrideMinMaxRunNumber = true;
        m_minRunNumber = m_minMaxRunNumbers.value()[0];
        if (m_minMaxRunNumbers.value().size() > 1) m_maxRunNumber = m_minMaxRunNumbers.value()[1];
        else m_maxRunNumber = m_minRunNumber;
 
        if (m_maxRunNumber > IOVTime::MAXRUN) m_maxRunNumber = IOVTime::MAXRUN;
        
        ATH_MSG_INFO("checkOverrideRunNumber: overriding IOV for range - min: " << m_minRunNumber 
                      << " max: " << m_maxRunNumber);
        return;
    }
 
    ATH_MSG_DEBUG("checkOverrideRunNumber: check if tag is set in jobOpts");
 
    // Get name of event selector from the application manager to
    // make sure we get the one for MC signal events
    SmartIF<IProperty> appMgr{serviceLocator()->service("ApplicationMgr")};
    if (!appMgr) {
        ATH_MSG_ERROR("checkOverrideRunNumber: Cannot get ApplicationMgr "); 
        return;
    }
    StringProperty property("EvtSel", "");
    StatusCode sc = appMgr->getProperty(&property);
    if (!sc.isSuccess()) {
        ATH_MSG_ERROR("checkOverrideRunNumber: unable to get EvtSel: found " << property.value());
        return;
    }
    // Get EventSelector for ApplicationMgr
    const std::string eventSelector = property.value();
    SmartIF<IProperty> evtSel{serviceLocator()->service(eventSelector)};
    if (!evtSel) {
        ATH_MSG_ERROR("checkOverrideRunNumber: Cannot get EventSelector " << eventSelector); 
        return;
    }
 
    // Is flag set to override the run number? 
    BooleanProperty overrideRunNumber("OverrideRunNumberFromInput", false);
    sc = evtSel->getProperty(&overrideRunNumber);
    if (!sc.isSuccess()) {
        // Not all EventSelectors have this property, so we must be tolerant
        ATH_MSG_DEBUG("resetRunNumber: unable to get OverrideRunNumberFromInput property from EventSelector ");
        return;
    }
    m_overrideRunNumber = overrideRunNumber.value();
    if (m_overrideRunNumber) {
        // New run number
        IntegerProperty runNumber("RunNumber", 0);
        sc = evtSel->getProperty(&runNumber);
        if (!sc.isSuccess()) {
            ATH_MSG_ERROR("checkOverrideRunNumber: unable to get RunNumber from EventSelector: found "
                          << runNumber.value());
            return;
        }
        m_newRunNumber = runNumber.value();
        // Old run number
        IntegerProperty oldRunNumber("OldRunNumber", 0);
        sc = evtSel->getProperty(&oldRunNumber);
        if (!sc.isSuccess()) {
            ATH_MSG_ERROR("checkOverrideRunNumber: unable to get OldRunNumber from EventSelector: found "
                          << oldRunNumber.value());
            return;
        }
        m_oldRunNumber = oldRunNumber.value();
 
        ATH_MSG_DEBUG("checkOverrideRunNumber: Changing old to new run number:  " << m_oldRunNumber
                      << " " << m_newRunNumber << " obtained from " << eventSelector);
    }
    else ATH_MSG_DEBUG("checkOverrideRunNumber: OverrideRunNumberFromInput not set for " << eventSelector);
}

endInputFile()

StatusCode IOVDbMetaDataTool::endInputFile ( const SG::SourceID & )

overridevirtual

Function called when the currently open input file got completely processed.

Definition at line 198 of file IOVDbMetaDataTool.cxx.

{
  return StatusCode::SUCCESS;
}

finalize()

StatusCode IOVDbMetaDataTool::finalize ( )

overridevirtual

Finalize AlgTool.

Definition at line 166 of file IOVDbMetaDataTool.cxx.

{
    return StatusCode::SUCCESS;
}

findMetaDataContainer()

IOVMetaDataContainer * IOVDbMetaDataTool::findMetaDataContainer ( const std::string & folderName ) const

finaloverridevirtual

Definition at line 513 of file IOVDbMetaDataTool.cxx.

{
  // lock the tool before this call
  // Return the folder if it is in the meta data store
  return m_metaDataStore->tryRetrieve<IOVMetaDataContainer>(folderName);
}

getMetaDataContainer()

IOVMetaDataContainer * IOVDbMetaDataTool::getMetaDataContainer ( const std::string & folderName, const std::string & folderDescription ) const

private

return meta data container from the meta data store

Definition at line 522 of file IOVDbMetaDataTool.cxx.

{
  // protected by locks in addPayload() and registerFolder()
  ATH_MSG_DEBUG("begin getMetaDataContainer ");
 
  IOVMetaDataContainer* cont{nullptr};
  // See if it is in the meta data store
  if (!m_metaDataStore->contains<IOVMetaDataContainer>(folderName)) {
    // Container not found, add in new one
    cont = new IOVMetaDataContainer(folderName, folderDescription);
    ATH_MSG_DEBUG("No IOVMetaDataContainer in MetaDataStore for folder " << folderName
          << ". Created a new instance");
    StatusCode sc = m_metaDataStore->record(cont, folderName);
    if (!sc.isSuccess()) {
      ATH_MSG_ERROR("Could not record IOVMetaDataContainer in MetaDataStore for folder " << folderName);
      delete cont;
      cont = nullptr;
    }
  } 
  else {
    ATH_MSG_DEBUG("IOVMetaDataContainer already in MetaDataStore for folder " << folderName);
    StatusCode sc = m_metaDataStore->retrieve(cont, folderName);
    if (!sc.isSuccess()) {
      ATH_MSG_ERROR("Could not retrieve IOVMetaDataContainer in MetaDataStore for folder " << folderName);
      cont = nullptr;
    }
  }
  return cont;
}

handle()

void IOVDbMetaDataTool::handle ( const Incident & incident )

overridevirtual

Incident service handle listening for BeginInputFile and EndInputFile.

Definition at line 173 of file IOVDbMetaDataTool.cxx.

{
  const FileIncident* fileInc  = dynamic_cast<const FileIncident*>(&inc);
  if(!fileInc) throw std::runtime_error("Unable to get FileName from FirstInputFile incident");
 
  const std::string fileName = fileInc->fileName();
  ATH_MSG_DEBUG("handle() " << inc.type() << " for " << fileName);
 
  // Check if we need to override run number - only needed for simulation
  checkOverrideRunNumber();
 
  StatusCode sc = processInputFileMetaData(fileName);
  if(!sc.isSuccess()) throw std::runtime_error("Could not process input file meta data");
  m_filesProcessed.insert(std::move(fileName));
}

initialize()

StatusCode IOVDbMetaDataTool::initialize ( )

overridevirtual

Initialize AlgTool.

Definition at line 60 of file IOVDbMetaDataTool.cxx.

{
  ATH_MSG_DEBUG("in initialize()");
 
  // Set to be listener for FirstInputFile
  ServiceHandle<IIncidentSvc> incSvc("IncidentSvc", this->name());
  ATH_CHECK(incSvc.retrieve());
  incSvc->addListener(this, "FirstInputFile", 60); // pri has to be < 100 to be after MetaDataSvc.
 
  // locate the meta data stores
  ATH_CHECK(m_metaDataStore.retrieve());
  ATH_CHECK(m_inputStore.retrieve());
 
  // Check whether folders need to be modified
  m_modifyFolders = (m_foldersToBeModified.value().size()>0);
  ATH_MSG_DEBUG("initialize(): " << (m_modifyFolders ? "" : "No ") << "need to modify folders");
 
  // Process direct payloads if configured
  if (!m_payloads.value().empty()) {
    // Group by folder: parse "folder:key" format from flat map
    std::map<std::string, std::map<std::string, std::string>> folderPayloads;
 
    for (const auto& [key, value] : m_payloads.value()) {
      // Split key on ':' to get folder and parameter name
      size_t colonPos = key.find(':');
      if (colonPos == std::string::npos) {
        ATH_MSG_ERROR("Invalid payload key format: " << key << " (expected 'folder:key')");
        return StatusCode::FAILURE;
      }
 
      std::string folderName = key.substr(0, colonPos);
      std::string paramName = key.substr(colonPos + 1);
      folderPayloads[folderName][paramName] = value;
    }
    // before going into loop, prepare these strings
    static const std::string beginRunKey{"beginRun"};
    static const std::string endRunKey{"endRun"};
    static const std::string specType{"string"};
    //
    //..and this predicate when building an AttributeListSpecification
    const auto isIOVKey = [](const std::string& key) {
      return key == beginRunKey || key == endRunKey;
    };
    ATH_MSG_DEBUG("Processing " << folderPayloads.size() << " folder(s) for direct payload registration");
    for (const auto& [folderName, parameters] : folderPayloads) {
      //will use these twice, so only search once and cache them
      auto beginRunItr = parameters.find(beginRunKey);
      auto endRunItr = parameters.find(endRunKey);
      if ( (beginRunItr == parameters.end()) || (endRunItr == parameters.end()) ) {
        ATH_MSG_ERROR("Payload for folder " << folderName << " missing beginRun or endRun");
        return StatusCode::FAILURE;
      }
      // Extract beginRun and endRun from parameters
      unsigned int beginRun = std::stoul(beginRunItr->second);
      unsigned int endRun = std::stoul(endRunItr->second);
 
      ATH_MSG_DEBUG("Registering folder " << folderName << " with " << parameters.size() - 2
                    << " parameters, IOV [" << beginRun << ", " << endRun << "]");
 
      // Parameter folders are read back as a single AthenaAttributeList, so use that
      // description (clid 40774348) rather than the CondAttrListCollection default.
      static const std::string attrListDescr{"<timeStamp>run-event</timeStamp><addrHeader><address_header service_type=\"71\" clid=\"40774348\" /></addrHeader><typeName>AthenaAttributeList</typeName>"};
      ATH_CHECK(registerFolder(folderName, attrListDescr));
 
      // Build payload for MetaDataStore
      // Note: AttributeListSpecification has protected destructor, must use new
      // The AttributeList constructor with 'true' takes ownership and will call release()
      coral::AttributeListSpecification* spec = new coral::AttributeListSpecification();
      for (const auto& [key, value] : parameters) {
        if (!isIOVKey(key)) {
          spec->extend(key, specType);
        }
      }
 
      coral::AttributeList attrList(*spec, true);
      for (const auto& [key, value] : parameters) {
        if (!isIOVKey(key)) {
          attrList[key].setValue(value);
        }
      }
 
      // Magic channel 0xFFFF + per-channel IOV: read back as AthenaAttributeList,
      // matching IOVDbFolder::addMetaAttrList.
      auto payload = std::make_unique<CondAttrListCollection>(true);
      const IOVRange range(IOVTime(beginRun, 0), IOVTime(endRun, IOVTime::MAXEVENT));
      payload->add(0xFFFF, attrList);
      payload->add(0xFFFF, range);
 
      ATH_MSG_DEBUG("Created payload with IOV [" << beginRun << ", " << endRun << "]");
 
      // Add to MetaDataStore
      ATH_CHECK(addPayload(folderName, payload.release()));
    }
 
    // Note: ConditionStore population is NOT done here because during initialize()
    // there is no valid EventContext, which is required for CondCont::insert().
    // Instead, use MetaDataToCondAlg (configured via MetaDataToCondAlgCfg) to populate
    // ConditionStore during event processing when a valid EventContext is available.
  }
 
  return(StatusCode::SUCCESS);
}

lock_shared()

virtual void IOVDbMetaDataTool::lock_shared ( ) const

inlinefinaloverridevirtual

Definition at line 92 of file IOVDbMetaDataTool.h.

{ m_mutex.lock_shared(); }

metaDataStop()

StatusCode IOVDbMetaDataTool::metaDataStop ( )

overridevirtual

Function called when the tool should write out its metadata.

Definition at line 250 of file IOVDbMetaDataTool.cxx.

{
  return StatusCode::SUCCESS;
}

modifyPayload()

StatusCode IOVDbMetaDataTool::modifyPayload ( const std::string & folderName, CondAttrListCollection *& payload ) const

private

Modify a Payload for a particular folder - replaces one of the internal attributes.

Modify a Payload for a particular folder - replaces one of the internal attributes

Definition at line 426 of file IOVDbMetaDataTool.cxx.

{
    // protected by lock in processInputFileMetaData()

    ATH_MSG_DEBUG("begin modifyPayload for folder " << folderName);
 
    // check if this folder needs to be modified
    bool modifyAttr = false;
    std::string attributeName;
    const std::vector<std::string>& folders = m_foldersToBeModified.value();
    const std::vector<std::string>& attrs   = m_attributesToBeRemoved.value();
    for (unsigned int i = 0; i < folders.size(); ++i) {
        if (folderName == folders[i]) {
            if (attrs.size() > i) {
                attributeName = attrs[i];
                modifyAttr    = true;
                ATH_MSG_DEBUG("modifyPayload: remove attribute " << attributeName);
                break;
            }
        }
    }
 
    if (!modifyAttr) {
        ATH_MSG_DEBUG("modifyPayload: folder " << folderName << " OK ");
        return StatusCode::SUCCESS;
    }
 
    bool iovSizeIsZero = coll->iov_size() == 0;
    IOVRange testIOV = coll->minRange();
    IOVTime  start   = testIOV.start();
    IOVTime  stop    = testIOV.stop();
    // Set the IOV
    CondAttrListCollection* coll1 = new CondAttrListCollection(true);
    if (iovSizeIsZero) { 
        // Only add in overall range if channels do not have
        // IOVs - otherwise this is automatically calculated
        coll1->addNewStart(start);
        coll1->addNewStop (stop);
    }
    // Add in channels
    unsigned int nchans = coll->size();
    bool hasChanNames = (coll->name_size() == nchans);
    for (unsigned int ichan = 0; ichan < nchans; ++ichan) {
        CondAttrListCollection::ChanNum chan = coll->chanNum(ichan);
        // Now filter out the unwanted attribute
        CondAttrListCollection::AttributeList  newAttrList;
        const CondAttrListCollection::AttributeList& oldAttrList = coll->attributeList(chan);
        for (unsigned int iatt = 0; iatt < oldAttrList.size(); ++iatt) {
            // skip the unwanted attribute
            if (attributeName == oldAttrList[iatt].specification().name()) {
                ATH_MSG_DEBUG("modifyPayload: skipping attribute name " << oldAttrList[iatt].specification().name());
                continue;
            }
                
            // copy the rest
            newAttrList.extend(oldAttrList[iatt].specification().name(),
                               oldAttrList[iatt].specification().type());
            const coral::Attribute&  oldAttr = oldAttrList[iatt];
            coral::Attribute&        newAttr = newAttrList[oldAttrList[iatt].specification().name()];
            newAttr = oldAttr;
            // newAttr.setValue(oldAttr.data());
            ATH_MSG_DEBUG("modifyPayload: copying attr name " 
                          << oldAttrList[iatt].specification().name() << " " 
                          /*<< newAttr*/);
        }
        coll1->add(chan, newAttrList);
        if (!iovSizeIsZero) coll1->add(chan, coll->iovRange(chan));
        if(hasChanNames)coll1->add(chan, coll->chanName(chan));
        ATH_MSG_DEBUG("modifyPayload: copied attribute list for channel " << chan);
    }
    delete coll;
    coll = coll1;
    if (msgLvl(MSG::DEBUG)) {
      std::ostringstream stream;
      coll->dump(stream);
      ATH_MSG_DEBUG(stream.str());
    }
 
    return StatusCode::SUCCESS;
}

overrideIOV()

StatusCode IOVDbMetaDataTool::overrideIOV ( CondAttrListCollection *& coll ) const

private

override IOV with new run number

Definition at line 827 of file IOVDbMetaDataTool.cxx.

{
    ATH_MSG_DEBUG("overrideIOV ");
 
    // Override the IOV for run/event IOVs
 
    //   (ONLY TRUE FOR OVERRIDE COMING IN VIA EVENTSELECTOR:)
    //   NOTE: we require that the old run number falls within the
    //   IOVRange of the incoming collection. We override ALL IOVs for
    //   ALL channels forcing the IOVRange to be (newRunNumber,1) to
    //   (newRunNumber+1,1)
 
    bool iovSizeIsZero = coll->iov_size() == 0;
    IOVRange testIOV = coll->minRange();
    IOVTime  start   = testIOV.start();
    IOVTime  stop    = testIOV.stop();
    IOVTime  oldRun(m_oldRunNumber, 0);
    if (start.isRunEvent() && stop.isRunEvent()) { // only for run/event
        IOVRange newRange;
        // Two ways of resetting 
        if (m_overrideMinMaxRunNumber) newRange = IOVRange(IOVTime(m_minRunNumber, 0), IOVTime(m_maxRunNumber, IOVTime::MAXEVENT));
        else if (m_overrideRunNumber)  newRange = IOVRange(IOVTime(m_newRunNumber, 0), IOVTime(m_newRunNumber + 1, 0));
 
        if (m_overrideRunNumber && !testIOV.isInRange(oldRun)) { 
            // old run must be in the range
            ATH_MSG_ERROR("overrideIOV: old run number does not match. Old run number " << m_oldRunNumber << " IOVRange: " << testIOV);
            return StatusCode::SUCCESS;
        }
 
        ATH_MSG_DEBUG("overrideIOV: overrideMinMaxRunNumber: " << (int)m_overrideMinMaxRunNumber
                      << " overrideRunNumber " << (int)m_overrideRunNumber
                      << " iovSizeIsZero: " << (int)iovSizeIsZero
                      << " newRange " << newRange);
        
        // Now over ride IOVs - two cases: 1) single IOV for full collection, 2) IOVs for individual channels.
        // Must treat the reset of collection IOV differently
        if (iovSizeIsZero) {
            // Only add in overall range if channels do not have
            // IOVs - otherwise this is automatically calculated
            coll->resetMinRange(); // must first reset to 'full range' and then reduce the IOVRange accordingly
            coll->addNewStart(newRange.start());
            coll->addNewStop (newRange.stop());
        }
        else {
            // Add in channels
            unsigned int nchans = coll->size();
            ATH_MSG_DEBUG("overrideIOV: nchans " << nchans);
            for (unsigned int ichan = 0; ichan < nchans; ++ichan) {
                // FIXME: O(N^2)!
                CondAttrListCollection::ChanNum chan = coll->chanNum(ichan);
                coll->add(chan, newRange);
                ATH_MSG_DEBUG("overrideIOV: overriding the IOV of collection chan " << chan);
            }
            // must reset the collection range AFTER the channels, because the collection range will be
            // 'narrowed' to that of the channels
            coll->resetMinRange();
        }
        if (msgLvl(MSG::DEBUG)) {
            ATH_MSG_DEBUG("overrideIOV: after  overriding the IOV of collection");
            std::ostringstream stream;
            coll->dump(stream);
            ATH_MSG_DEBUG(stream.str());
        }
    }
    else ATH_MSG_DEBUG("overrideIOV: IOV is not run/event ");
 
    return StatusCode::SUCCESS;
}

processInputFileMetaData()

StatusCode IOVDbMetaDataTool::processInputFileMetaData ( const std::string & fileName )

overridevirtual

Explicit call to process IOV meta data from the input meta data store, transferring it to the main meta data store.

Normally, this method is called at the BeginInputFile incident. However, this explicit call allows the transfer to occur during the initialization phase, which occurs before BeginInputFile incident.

Definition at line 555 of file IOVDbMetaDataTool.cxx.

{
  // lock the tool while it is processing input metadata 
  std::scoped_lock  guard( m_mutex );
 
  ATH_MSG_DEBUG("processInputFileMetaData: file name " << fileName);
 
  // Retrieve all meta data containers from InputMetaDataStore
  SG::ConstIterator<IOVMetaDataContainer> cont;
  SG::ConstIterator<IOVMetaDataContainer> contEnd;
 
  StatusCode sc = m_inputStore->retrieve(cont, contEnd);
  if (!sc.isSuccess()) {
    ATH_MSG_DEBUG("processInputFileMetaData: Could not retrieve IOVMetaDataContainer objects from InputMetaDataStore - cannot process input file meta data");
    return StatusCode::SUCCESS;
  }
 
  ATH_MSG_DEBUG("processInputFileMetaData: Retrieved from IOVMetaDataContainer(s) from InputMetaDataStore");
 
  // For each container, merge its contents into the MetaDataStore 
  unsigned int ncolls    = 0;
  unsigned int ndupColls = 0;
  for (; cont != contEnd; ++cont) {
    IOVMetaDataContainer* contMaster = getMetaDataContainer(cont->folderName()
                                , cont->folderDescription());
 
    // We assume that the folder is the same for all versions, and
    // now we loop over versions for the payloads
    std::list<SG::ObjectWithVersion<IOVMetaDataContainer> > allVersions;
    sc = m_inputStore->retrieveAllVersions(allVersions, cont.key());
    if (!sc.isSuccess()) {
      ATH_MSG_ERROR("Could not retrieve all versions for " << cont.key());
      return sc;
    }
 
    for (SG::ObjectWithVersion<IOVMetaDataContainer>& obj : allVersions) {
      const IOVPayloadContainer*  payload = obj.dataObject->payloadContainer();
 
      ATH_MSG_DEBUG("processInputFileMetaData: New container: payload size " << payload->size() << " version key " << obj.versionedKey);
 
      // detailed printout before merge
      if (msgLvl(MSG::VERBOSE)) {
    const IOVPayloadContainer*  payloadMaster = contMaster->payloadContainer();
    ATH_MSG_VERBOSE("Before merge, payload minRange for folder " << cont->folderName());
    if (payloadMaster && payloadMaster->size()) {
      // Loop over AttrColls and print out minRange
      IOVPayloadContainer::const_iterator itColl    = payloadMaster->begin();
      IOVPayloadContainer::const_iterator itCollEnd = payloadMaster->end();
      unsigned int iPayload = 0;
      for (; itColl != itCollEnd; ++itColl, ++iPayload) {
        ATH_MSG_VERBOSE(iPayload << " " << (*itColl)->minRange() << " " 
                << (*itColl)->size());
      }
    }
    else { 
      ATH_MSG_VERBOSE("  no payloads yet!"); 
    }
      }
    }
        
    // Detailed printout
    if (msgLvl(MSG::DEBUG)) {
      ATH_MSG_DEBUG("processInputFileMetaData: Current payload before merge " << contMaster->folderName());
      IOVPayloadContainer::const_iterator itColl1    = contMaster->payloadContainer()->begin();
      IOVPayloadContainer::const_iterator itCollEnd1 = contMaster->payloadContainer()->end();
      std::ostringstream stream;
      for (; itColl1 != itCollEnd1; ++itColl1) (*itColl1)->dump(stream);
      ATH_MSG_DEBUG(stream.str());
    }
 
    //
    // Loop over CondAttrListCollections and do merge
    //
    for (SG::ObjectWithVersion<IOVMetaDataContainer>& obj : allVersions) {
      const IOVPayloadContainer*  payload = obj.dataObject->payloadContainer();
      IOVPayloadContainer::const_iterator itColl    = payload->begin();
      IOVPayloadContainer::const_iterator itCollEnd = payload->end();
      for (; itColl != itCollEnd; ++itColl) {
 
    // Make a copy of the collection and merge it into
    // master container in meta data store 
    CondAttrListCollection* coll = new CondAttrListCollection(**itColl);
    // Override run number if requested
    if (m_overrideRunNumber || m_overrideMinMaxRunNumber) {
          ATH_CHECK( overrideIOV(coll) );
        }
 
    // first check if we need to modify the incoming payload
    if (!modifyPayload (contMaster->folderName(), coll).isSuccess()) {
          ATH_MSG_ERROR("processInputFileMetaData: Could not modify the payload for folder " << contMaster->folderName());
          return StatusCode::FAILURE;
        }
 
    ATH_MSG_VERBOSE("processInputFileMetaData: merge minRange: " << coll->minRange());
    if (!contMaster->merge(coll)) {
      // Did not merge it in - was a duplicate, so we need to delete it 
      delete coll;
      ++ndupColls;
      ATH_MSG_VERBOSE(" => not merged ");
    }
    else {
      ++ncolls;
      ATH_MSG_VERBOSE(" => merged ");
    }
 
      }
      ATH_MSG_DEBUG("processInputFileMetaData: Merged together containers for folder " << cont->folderName() << " ncoll/ndup " 
            << ncolls << " " << ndupColls);
 
      // Check for consistency after merge
      const IOVPayloadContainer*  payloadMaster = contMaster->payloadContainer();
      if (payloadMaster && payloadMaster->size()) {
    // Loop over AttrColls and print out minRange
    IOVPayloadContainer::const_iterator itColl    = payloadMaster->begin();
    IOVPayloadContainer::const_iterator itCollEnd = payloadMaster->end();
    IOVTime lastStop;
    if ((*itColl)->minRange().start().isTimestamp()) lastStop = IOVTime(0);
    else lastStop = IOVTime(0,0);
    bool hasError = false;
    for (; itColl != itCollEnd; ++itColl) {
      if ((*itColl)->minRange().start() < lastStop) hasError = true;
      lastStop = (*itColl)->minRange().stop();
    }
    if (hasError) {
      ATH_MSG_ERROR("processInputFileMetaData: error after merge of file meta data. " );
      ATH_MSG_ERROR("processInputFileMetaData: Filename " << fileName);
      ATH_MSG_ERROR("processInputFileMetaData: folder " << contMaster->folderName());
      ATH_MSG_ERROR("processInputFileMetaData: MinRange for meta data folders ");
      unsigned int iPayload = 0;
      itColl    = payloadMaster->begin();
      for (; itColl != itCollEnd; ++itColl, ++iPayload) {
        ATH_MSG_ERROR(iPayload << " " << (*itColl)->minRange() << " " << (*itColl)->size());
      }
    }
      }
 
      // detailed printout after merge
      if (msgLvl(MSG::VERBOSE)) {
    const IOVPayloadContainer*  payloadMaster = contMaster->payloadContainer();
    ATH_MSG_VERBOSE("processInputFileMetaData: After merge, payload minRange ");
    if (payloadMaster) {
      // Loop over AttrColls and print out minRange
      IOVPayloadContainer::const_iterator itColl    = payloadMaster->begin();
      IOVPayloadContainer::const_iterator itCollEnd = payloadMaster->end();
      unsigned int iPayload = 0;
      for (; itColl != itCollEnd; ++itColl, ++iPayload) {
        ATH_MSG_VERBOSE(iPayload << " " << (*itColl)->minRange() << " " 
                << (*itColl)->size());
      }
    }
    else { ATH_MSG_ERROR("  no payloads yet!"); }
      }
      
      // Detailed printout
      if (msgLvl(MSG::DEBUG)) {
    ATH_MSG_DEBUG("processInputFileMetaData: Input payload " << cont->folderName());
    std::ostringstream streamInp;
    itColl    = payload->begin();
    itCollEnd = payload->end();
    for (; itColl != itCollEnd; ++itColl) (*itColl)->dump(streamInp);
    ATH_MSG_DEBUG(streamInp.str());
    ATH_MSG_DEBUG("processInputFileMetaData: Output payload " << contMaster->folderName());
    std::ostringstream streamOut;
    itColl    = contMaster->payloadContainer()->begin();
    itCollEnd = contMaster->payloadContainer()->end();
    for (; itColl != itCollEnd; ++itColl) (*itColl)->dump(streamOut);
    ATH_MSG_DEBUG(streamOut.str());
      }
    }
  }
 
  ATH_MSG_DEBUG("processInputFileMetaData: Total number of attribute collections  merged together " << ncolls
        << " Number of duplicate collections " << ndupColls);
 
  // Note: ConditionStore population for parameter folders (direct in-file metadata mode)
  // is now handled by MetaDataToCondAlg, which runs during event processing when
  // a valid EventContext is available.
 
  // Serialize requested IOV folders to ByteStream metadata
  ATH_CHECK(serializeIOVMetadataToBSMetadata());
 
  return StatusCode::SUCCESS;
}

registerFolder() [1/2]

StatusCode IOVDbMetaDataTool::registerFolder ( const std::string & folderName ) const

overridevirtual

Register folder in the IOV Db MetaData without folder description - uses default folder description for CondAttrListCollection.

Definition at line 339 of file IOVDbMetaDataTool.cxx.

{
    // Set the default folder description for a CondAttrListCollection
    // which will be read back via IOVDbSvc
    std::string folderDescr = "<timeStamp>run-event</timeStamp><addrHeader><address_header service_type=\"256\" clid=\"1238547719\" /> </addrHeader><typeName>CondAttrListCollection</typeName>" ;
 
    return registerFolder(folderName, folderDescr);
}

registerFolder() [2/2]

StatusCode IOVDbMetaDataTool::registerFolder ( const std::string & folderName, const std::string & folderDescription ) const

overridevirtual

Register folder in the IOV Db MetaData - done once at initialize.

Definition at line 351 of file IOVDbMetaDataTool.cxx.

{
    // lock the tool before getMetaDataContainer() call
    std::scoped_lock  guard( m_mutex );
 
    ATH_MSG_DEBUG("begin registerFolder ");
 
    if( ! getMetaDataContainer(folderName, folderDescription) ) {
        ATH_MSG_ERROR("Unable to register folder " << folderName);
        return(StatusCode::FAILURE);
    } 
    else {
        ATH_MSG_DEBUG("IOVMetaDataContainer  for folder " << folderName << " has been registered ");
    }
 
    return StatusCode::SUCCESS;
}

serializeContainerToJSON()

std::string IOVDbMetaDataTool::serializeContainerToJSON ( const IOVMetaDataContainer * container ) const

private

Serialize IOVMetaDataContainer to JSON format for ByteStream metadata.

Definition at line 742 of file IOVDbMetaDataTool.cxx.

{
  const IOVPayloadContainer* payloads = container->payloadContainer();
  if (!payloads || payloads->size() == 0) {
    ATH_MSG_WARNING("No payloads for folder " << container->folderName());
    return "";
  }
 
  using json = nlohmann::json;
  json jsonData;
 
  jsonData["folder"] = container->folderName();
  jsonData["description"] = container->folderDescription();
  jsonData["iovs"] = json::array();
 
  // Serialize each IOV payload
  for (const CondAttrListCollection* coll : *payloads) {
    json iov;
 
    // Get IOV range
    IOVRange range = coll->minRange();
    IOVTime start = range.start();
    IOVTime stop = range.stop();
 
    // IOV range
    if (start.isRunEvent()) {
      iov["range"]["start"] = {{"run", start.run()}, {"event", start.event()}};
      iov["range"]["stop"] = {{"run", stop.run()}, {"event", stop.event()}};
    } else {
      iov["range"]["start"] = {{"timestamp", start.timestamp()}};
      iov["range"]["stop"] = {{"timestamp", stop.timestamp()}};
    }
 
    // Attributes (serialize each channel)
    iov["attrs"] = json::object();
    for (const auto& chanAttrPair : *coll) {
      CondAttrListCollection::ChanNum chan = chanAttrPair.first;
      const coral::AttributeList& attrList = chanAttrPair.second;
 
      std::string chanKey = "chan" + std::to_string(chan);
      iov["attrs"][chanKey] = json::object();
 
      for (const auto& attr : attrList) {
        auto & thisAttribute = iov["attrs"][chanKey][attr.specification().name()];
        // Serialize attribute value based on type
        const std::type_info& type = attr.specification().type();
        if (type == typeid(std::string)) {
          thisAttribute = attr.data<std::string>();
        } else if (type == typeid(int)) {
          thisAttribute = attr.data<int>();
        } else if (type == typeid(unsigned int)) {
          thisAttribute = attr.data<unsigned int>();
        } else if (type == typeid(long)) {
          thisAttribute = attr.data<long>();
        } else if (type == typeid(unsigned long)) {
          thisAttribute = attr.data<unsigned long>();
        } else if (type == typeid(long long)) {
          thisAttribute = attr.data<long long>();
        } else if (type == typeid(unsigned long long)) {
          thisAttribute = attr.data<unsigned long long>();
        } else if (type == typeid(float)) {
          thisAttribute = attr.data<float>();
        } else if (type == typeid(double)) {
          thisAttribute = attr.data<double>();
        } else if (type == typeid(bool)) {
          thisAttribute = attr.data<bool>();
        } else {
          // For other types, convert to string representation
          std::ostringstream oss;
          attr.toOutputStream(oss);
          thisAttribute = oss.str();
          ATH_MSG_DEBUG("Attribute " << attr.specification().name() << " has unsupported type, converted to string: " << oss.str());
        }
      }
    }
 
    jsonData["iovs"].push_back(iov);
  }
 
  return jsonData.dump();
}

serializeIOVMetadataToBSMetadata()

StatusCode IOVDbMetaDataTool::serializeIOVMetadataToBSMetadata ( )

private

Serialize IOV metadata to ByteStream metadata (called from processInputFileMetaData).

Definition at line 203 of file IOVDbMetaDataTool.cxx.

{
  // Check if we have folders to serialize to ByteStream metadata
  if (m_foldersToSerializeToBSMetadata.value().empty()) {
    ATH_MSG_DEBUG("No folders configured for serialization");
    return StatusCode::SUCCESS;
  }
 
  // Collect IOV metadata strings first
  std::vector<std::string> iovMetaStrings;
  for (const std::string& folderName : m_foldersToSerializeToBSMetadata.value()) {
    IOVMetaDataContainer* container = findMetaDataContainer(folderName);
    if (!container) {
      ATH_MSG_WARNING("Could not find IOVMetaDataContainer for folder " << folderName << ", skipping");
      continue;
    }
 
    std::string jsonStr = serializeContainerToJSON(container);
    if (!jsonStr.empty()) {
      iovMetaStrings.push_back("IOVMeta." + folderName + "=" + jsonStr);
      ATH_MSG_DEBUG("Serialized folder " << folderName << " (" << jsonStr.size() << " bytes JSON)");
    }
  }
 
  if (iovMetaStrings.empty()) {
    ATH_MSG_DEBUG("No IOV metadata serialized");
    return StatusCode::SUCCESS;
  }
 
  // Store the IOV metadata strings in MetaDataStore for ByteStreamCnvSvc to retrieve
  // Check if the object already exists and update it, or create a new one
  if (m_metaDataStore->contains<std::vector<std::string>>("IOVMetaDataStrings")) {
    // Retrieve existing and append
    std::vector<std::string>* existingStrings = nullptr;
    ATH_CHECK(m_metaDataStore->retrieve(existingStrings, "IOVMetaDataStrings"));
    existingStrings->insert(existingStrings->end(), iovMetaStrings.begin(), iovMetaStrings.end());
    ATH_MSG_DEBUG("Appended " << iovMetaStrings.size() << " IOV metadata strings to existing collection");
  } else {
    // Create new
    auto iovMetaData = std::make_unique<std::vector<std::string>>(std::move(iovMetaStrings));
    ATH_CHECK(m_metaDataStore->record(std::move(iovMetaData), "IOVMetaDataStrings"));
    ATH_MSG_DEBUG("Stored " << iovMetaStrings.size() << " IOV metadata strings in MetaDataStore");
  }
 
  return StatusCode::SUCCESS;
}

unlock_shared()

virtual void IOVDbMetaDataTool::unlock_shared ( ) const

inlinefinaloverridevirtual

Definition at line 93 of file IOVDbMetaDataTool.h.

{ m_mutex.unlock_shared(); }

Member Data Documentation

m_attributesToBeRemoved

StringArrayProperty IOVDbMetaDataTool::m_attributesToBeRemoved {this, "AttributesToBeRemoved", {"RandomSeedOffset"}}

private

Definition at line 142 of file IOVDbMetaDataTool.h.

{this, "AttributesToBeRemoved", {"RandomSeedOffset"}};

m_condStore

StoreGateSvc_t IOVDbMetaDataTool::m_condStore

private

Definition at line 122 of file IOVDbMetaDataTool.h.

m_filesProcessed

std::set<std::string> IOVDbMetaDataTool::m_filesProcessed

private

Definition at line 138 of file IOVDbMetaDataTool.h.

m_foldersToBeModified

StringArrayProperty IOVDbMetaDataTool::m_foldersToBeModified {this, "FoldersToBeModified", {"/Simulation/Parameters"}}

private

Definition at line 141 of file IOVDbMetaDataTool.h.

{this, "FoldersToBeModified", {"/Simulation/Parameters"}};

m_foldersToSerializeToBSMetadata

Gaudi::Property<std::vector<std::string> > IOVDbMetaDataTool::m_foldersToSerializeToBSMetadata

private

Initial value:

{
        this, "FoldersToSerializeToBSMetadata", {},
        "List of folder names to serialize to ByteStream metadata as JSON"}

Definition at line 152 of file IOVDbMetaDataTool.h.

                                                                        {
        this, "FoldersToSerializeToBSMetadata", {},
        "List of folder names to serialize to ByteStream metadata as JSON"};

m_inputStore

StoreGateSvc_t IOVDbMetaDataTool::m_inputStore

private

Definition at line 121 of file IOVDbMetaDataTool.h.

m_maxRunNumber

unsigned int IOVDbMetaDataTool::m_maxRunNumber

private

Definition at line 135 of file IOVDbMetaDataTool.h.

m_metaDataStore

StoreGateSvc_t IOVDbMetaDataTool::m_metaDataStore

private

Definition at line 120 of file IOVDbMetaDataTool.h.

m_minMaxRunNumbers

UnsignedIntegerArrayProperty IOVDbMetaDataTool::m_minMaxRunNumbers {this, "MinMaxRunNumbers", {}, "Min/max run numbers for overriding meta data IOV"}

private

Definition at line 140 of file IOVDbMetaDataTool.h.

{this, "MinMaxRunNumbers", {}, "Min/max run numbers for overriding meta data IOV"};

m_minRunNumber

unsigned int IOVDbMetaDataTool::m_minRunNumber

private

Definition at line 134 of file IOVDbMetaDataTool.h.

m_modifyFolders

bool IOVDbMetaDataTool::m_modifyFolders

private

Definition at line 143 of file IOVDbMetaDataTool.h.

m_mutex

std::shared_mutex IOVDbMetaDataTool::m_mutex

mutableprivate

Definition at line 157 of file IOVDbMetaDataTool.h.

m_newRunNumber

unsigned int IOVDbMetaDataTool::m_newRunNumber

private

Definition at line 131 of file IOVDbMetaDataTool.h.

m_oldRunNumber

unsigned int IOVDbMetaDataTool::m_oldRunNumber

private

Definition at line 132 of file IOVDbMetaDataTool.h.

m_overrideMinMaxRunNumber

bool IOVDbMetaDataTool::m_overrideMinMaxRunNumber

private

Definition at line 128 of file IOVDbMetaDataTool.h.

m_overrideRunNumber

bool IOVDbMetaDataTool::m_overrideRunNumber

private

Definition at line 127 of file IOVDbMetaDataTool.h.

m_payloads

Gaudi::Property<std::map<std::string, std::string> > IOVDbMetaDataTool::m_payloads

private

Initial value:

{
        this, "Payloads", {}, "Direct payload registration (format: {folder:key: value})"}

Definition at line 148 of file IOVDbMetaDataTool.h.

                                                          {
        this, "Payloads", {}, "Direct payload registration (format: {folder:key: value})"};

---

The documentation for this class was generated from the following files:

• IOVDbMetaDataTool.h
• IOVDbMetaDataTool.cxx