Algorithm demonstrating the usage of the ONNX Runtime C++ API. More...

#include <EvaluateModelWithAthInfer.h>

Inheritance diagram for AthOnnx::EvaluateModelWithAthInfer:

Collaboration diagram for AthOnnx::EvaluateModelWithAthInfer:

Public Member Functions
virtual StatusCode	sysInitialize () override
	Override sysInitialize.
virtual bool	isClonable () const override
	Specify if the algorithm is clonable.
virtual StatusCode	sysExecute (const EventContext &ctx) override
	Execute an algorithm.
virtual const DataObjIDColl &	extraOutputDeps () const override
	Return the list of extra output dependencies.
virtual bool	filterPassed (const EventContext &ctx) const
	Get filter decision:
virtual void	setFilterPassed (bool state, const EventContext &ctx) const
	Set filter decision:
ServiceHandle< StoreGateSvc > &	evtStore ()
	The standard `StoreGateSvc` (event store) Returns (kind of) a pointer to the `StoreGateSvc`.
const ServiceHandle< StoreGateSvc > &	detStore () const
	The standard `StoreGateSvc/DetectorStore` Returns (kind of) a pointer to the `StoreGateSvc`.
virtual StatusCode	sysStart () override
	Handle START transition.
virtual std::vector< Gaudi::DataHandle * >	inputHandles () const override
	Return this algorithm's input handles.
virtual std::vector< Gaudi::DataHandle * >	outputHandles () const override
	Return this algorithm's output handles.
Gaudi::Details::PropertyBase &	declareProperty (Gaudi::Property< T, V, H > &t)
void	updateVHKA (Gaudi::Details::PropertyBase &)
MsgStream &	msg () const
bool	msgLvl (const MSG::Level lvl) const
Function(s) inherited from @c AthAlgorithm
virtual StatusCode	initialize () override
	Function initialising the algorithm.
virtual StatusCode	execute (const EventContext &ctx) const override
	Function executing the algorithm for a single event.

Protected Member Functions
void	renounceArray (SG::VarHandleKeyArray &handlesArray)
	remove all handles from I/O resolution
std::enable_if_t< std::is_void_v< std::result_of_t< decltype(&T::renounce)(T)> > &&!std::is_base_of_v< SG::VarHandleKeyArray, T > &&std::is_base_of_v< Gaudi::DataHandle, T >, void >	renounce (T &h)
void	extraDeps_update_handler (Gaudi::Details::PropertyBase &ExtraDeps)
	Add StoreName to extra input/output deps as needed.

Private Types
typedef ServiceHandle< StoreGateSvc >	StoreGateSvc_t

Private Member Functions
Gaudi::Details::PropertyBase &	declareGaudiProperty (Gaudi::Property< T, V, H > &hndl, const SG::VarHandleKeyType &)
	specialization for handling Gaudi::Property<SG::VarHandleKey>

Private Attributes
DataObjIDColl	m_extendedExtraObjects
	Extra output dependency collection, extended by AthAlgorithmDHUpdate to add symlinks.
StoreGateSvc_t	m_evtStore
	Pointer to StoreGate (event store by default).
StoreGateSvc_t	m_detStore
	Pointer to StoreGate (detector store by default).
std::vector< SG::VarHandleKeyArray * >	m_vhka
bool	m_varHandleArraysDeclared
Algorithm properties
Gaudi::Property< std::string >	m_pixelFileName
	Name of the model file to load.
Gaudi::Property< int >	m_batchSize {this, "BatchSize", 1, "No. of elements/example in a batch"}
	Following properties needed to be consdered if the .onnx model is evaluated in batch mode.
ToolHandle< AthInfer::IAthInferenceTool >	m_onnxTool
	Tool handler for onnx inference session.
std::vector< std::vector< std::vector< float > > >	m_input_tensor_values_notFlat

Detailed Description

Algorithm demonstrating the usage of the ONNX Runtime C++ API.

In most cases this should be preferred over the C API...

Author: Debottam Bakshi Gupta Debot.nosp@m.tam..nosp@m.Baksh.nosp@m.i.Gu.nosp@m.pta@c.nosp@m.ern..nosp@m.ch; Attila Krasznahorkay Attil.nosp@m.a.Kr.nosp@m.aszna.nosp@m.hork.nosp@m.ay@ce.nosp@m.rn.c.nosp@m.h

Definition at line 29 of file EvaluateModelWithAthInfer.h.

Member Typedef Documentation

◆ StoreGateSvc_t

typedef ServiceHandle<StoreGateSvc> AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::StoreGateSvc_t

privateinherited

Definition at line 388 of file AthCommonDataStore.h.

Member Function Documentation

◆ declareGaudiProperty()

Gaudi::Details::PropertyBase & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::declareGaudiProperty	(	Gaudi::Property< T, V, H > &	hndl,
		const SG::VarHandleKeyType &	)

inlineprivateinherited

specialization for handling Gaudi::Property<SG::VarHandleKey>

Definition at line 156 of file AthCommonDataStore.h.

  {
    return *AthCommonDataStore<PBASE>::declareProperty(hndl.name(),
                                                       hndl.value(), 
                                                       hndl.documentation());
 
  }

◆ declareProperty()

Gaudi::Details::PropertyBase & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::declareProperty ( Gaudi::Property< T, V, H > & t )

inlineinherited

Definition at line 145 of file AthCommonDataStore.h.

                                                                     {
    typedef typename SG::HandleClassifier<T>::type htype;
    return AthCommonDataStore<PBASE>::declareGaudiProperty(t, htype());
  }

◆ detStore()

const ServiceHandle< StoreGateSvc > & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::detStore ( ) const

inlineinherited

The standard StoreGateSvc/DetectorStore Returns (kind of) a pointer to the StoreGateSvc.

Definition at line 95 of file AthCommonDataStore.h.

95{ return m_detStore; }

AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_detStore

StoreGateSvc_t m_detStore

Definition AthCommonDataStore.h:393

◆ evtStore()

ServiceHandle< StoreGateSvc > & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::evtStore ( )

inlineinherited

The standard StoreGateSvc (event store) Returns (kind of) a pointer to the StoreGateSvc.

Definition at line 85 of file AthCommonDataStore.h.

85{ return m_evtStore; }

AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_evtStore

StoreGateSvc_t m_evtStore

Definition AthCommonDataStore.h:390

◆ execute()

StatusCode AthOnnx::EvaluateModelWithAthInfer::execute ( const EventContext & ctx ) const

overridevirtual

Function executing the algorithm for a single event.

Definition at line 61 of file EvaluateModelWithAthInfer.cxx.

                                                    {
  // We know we have at least one image, otherwise we would have errored out
  // earlier
  const std::size_t n_batches =
      m_input_tensor_values_notFlat.size() / m_batchSize.value();
  const auto n_rows = std::int64_t(m_input_tensor_values_notFlat[0].size());
  const auto n_cols = std::int64_t(m_input_tensor_values_notFlat[0][0].size());
 
  for (std::size_t batch_idx = 0; batch_idx < n_batches; ++batch_idx) {
    // prepare inputs
    std::vector<float> inputDataVector;
    inputDataVector.reserve(m_batchSize.value() * n_rows * n_cols);
    for (const std::vector<std::vector<float>>& imageData :
         m_input_tensor_values_notFlat |
             std::views::drop(batch_idx * m_batchSize.value()) |
             std::views::take(m_batchSize.value())) {
      std::vector<float> flatten =
          AthOnnxUtils::flattenNestedVectors(imageData);
      inputDataVector.insert(inputDataVector.end(), flatten.begin(),
                             flatten.end());
    }
 
    std::vector<int64_t> inputShape = {m_batchSize.value(), n_rows, n_cols};
 
    AthInfer::InputDataMap inputData;
    inputData["flatten_input:0"] =
        std::make_pair(inputShape, std::move(inputDataVector));
 
    const std::int64_t n_scores = 10;
    AthInfer::OutputDataMap outputData;
    outputData["dense_1/Softmax:0"] = std::make_pair(
        std::vector<int64_t>{m_batchSize, n_scores}, std::vector<float>{});
 
    ATH_CHECK(m_onnxTool->inference(inputData, outputData));
 
    auto const& outputScores =
        std::get<std::vector<float>>(outputData["dense_1/Softmax:0"].second);
 
    if (outputScores.size() != std::size_t(n_scores * m_batchSize.value())) {
      ATH_MSG_ERROR("Got back " << outputScores.size()
                                << " scores when it should have been "
                                << n_scores << " * " << m_batchSize.value()
                                << " = " << n_scores * m_batchSize.value());
      return StatusCode::FAILURE;
    }
 
    for (int img_idx = 0; img_idx < m_batchSize.value(); img_idx++) {
      std::span scores(outputScores.begin() + img_idx * n_scores,
                       outputScores.begin() + (img_idx + 1) * n_scores);
      ATH_MSG_DEBUG("Scores for img " << img_idx << " of batch " << batch_idx
                                      << ": "
                                      << EvaluateUtils::spanToString(scores));
      const auto max_elem = std::ranges::max_element(scores);
      ATH_MSG_DEBUG("Class: " << max_elem - scores.begin()
                              << " has the highest score: " << *max_elem
                              << " in img " << img_idx << " of batch "
                              << batch_idx);
    }
  }
  return StatusCode::SUCCESS;
}

◆ extraDeps_update_handler()

void AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::extraDeps_update_handler ( Gaudi::Details::PropertyBase & ExtraDeps )

protectedinherited

Add StoreName to extra input/output deps as needed.

use the logic of the VarHandleKey to parse the DataObjID keys supplied via the ExtraInputs and ExtraOuputs Properties to add the StoreName if it's not explicitly given

◆ extraOutputDeps()

const DataObjIDColl & AthCommonAlgorithm< Gaudi::Algorithm >::extraOutputDeps ( ) const

overridevirtualinherited

Return the list of extra output dependencies.

This list is extended to include symlinks implied by inheritance relations.

Definition at line 89 of file AthCommonAlgorithm.cxx.

{
  // If we didn't find any symlinks to add, just return the collection
  // from the base class.  Otherwise, return the extended collection.
  if (!m_extendedExtraObjects.empty()) {
    return m_extendedExtraObjects;
  }
  return BaseAlg::extraOutputDeps();
}

◆ filterPassed()

virtual bool AthCommonAlgorithm< Gaudi::Algorithm >::filterPassed ( const EventContext & ctx ) const

inlinevirtualinherited

Get filter decision:

Definition at line 93 of file AthCommonAlgorithm.h.

                                                           {
    return execState( ctx ).filterPassed();
  }

◆ initialize()

StatusCode AthOnnx::EvaluateModelWithAthInfer::initialize ( )

overridevirtual

Function initialising the algorithm.

Definition at line 18 of file EvaluateModelWithAthInfer.cxx.

                                                 {
  if (m_batchSize.value() < 1) {
    ATH_MSG_ERROR("Requested an invalid batch size: " << m_batchSize.value());
    return StatusCode::FAILURE;
  }
 
  // Fetch tools
  ATH_CHECK(m_onnxTool.retrieve());
 
  // read input file, and the target file for comparison.
  std::string pixelFilePath =
      PathResolver::find_calib_file(m_pixelFileName.value());
  ATH_MSG_INFO("Using pixel file: " << pixelFilePath);
 
  try {
    m_input_tensor_values_notFlat =
        EvaluateUtils::read_mnist_pixel_notFlat(pixelFilePath);
    ATH_MSG_INFO(
        "Total no. of samples: " << m_input_tensor_values_notFlat.size());
  } catch (const std::exception& e) {
    ATH_MSG_ERROR(e.what());
    return StatusCode::FAILURE;
  }
 
  if (std::size_t(m_batchSize.value()) > m_input_tensor_values_notFlat.size()) {
    ATH_MSG_ERROR("The batch size requested ("
                  << m_batchSize.value()
                  << ") is greater than the number of available "
                     "samples ("
                  << m_input_tensor_values_notFlat.size() << ")");
    return StatusCode::FAILURE;
  }
 
  if (m_input_tensor_values_notFlat.size() % m_batchSize.value() != 0) {
    ATH_MSG_ERROR("The number of samples ("
                  << m_input_tensor_values_notFlat.size()
                  << ") is not a multiple of the requested batch size ("
                  << m_batchSize.value() << ")");
    return StatusCode::FAILURE;
  }
  return StatusCode::SUCCESS;
}

◆ inputHandles()

virtual std::vector< Gaudi::DataHandle * > AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::inputHandles ( ) const

overridevirtualinherited

Return this algorithm's input handles.

We override this to include handle instances from key arrays if they have not yet been declared. See comments on updateVHKA.

◆ isClonable()

virtual bool AthCommonAlgorithm< Gaudi::Algorithm >::isClonable ( ) const

inlineoverridevirtualinherited

Specify if the algorithm is clonable.

Only relevant for non-reentrant algorithms. Actual number of clones needs to be set via the "Cardinality" property.

Reimplemented in AFP_DigiTop, AlgB, AlgT, BCM_Digitization, CscDigitBuilder, CscDigitToCscRDO, G4AtlasAlg, G4RunAlg, HGTD_Digitization, HiveAlgBase, InDet::GNNSeedingTrackMaker, InDet::SCT_Clusterization, InDet::SiSPGNNTrackMaker, InDet::SiSPSeededTrackFinder, InDet::SiTrackerSpacePointFinder, ISF::SimKernelMT, ITk::StripDigitization, ITkPixelCablingAlg, ITkStripCablingAlg, LArHitEMapMaker, LArTTL1Maker, LUCID_DigiTop, LVL1::L1TopoSimulation, MergeCalibHits, MergeGenericMuonSimHitColl, MergeHijingPars, MergeMcEventCollection, MergeTrackRecordCollection, MergeTruthJets, MergeTruthParticles, MuonDigitizer, PileUpMTAlg, PixelDigitization, RoIBResultToxAOD, SCT_ByteStreamErrorsTestAlg, SCT_CablingCondAlgFromCoraCool, SCT_CablingCondAlgFromText, SCT_ConditionsParameterTestAlg, SCT_ConditionsSummaryTestAlg, SCT_ConfigurationConditionsTestAlg, SCT_Digitization, SCT_FlaggedConditionTestAlg, SCT_LinkMaskingTestAlg, SCT_MajorityConditionsTestAlg, SCT_ModuleVetoTestAlg, SCT_MonitorConditionsTestAlg, SCT_PrepDataToxAOD, SCT_RawDataToxAOD, SCT_ReadCalibChipDataTestAlg, SCT_ReadCalibDataTestAlg, SCT_RODVetoTestAlg, SCT_SensorsTestAlg, SCT_SiliconConditionsTestAlg, SCT_StripVetoTestAlg, SCT_TdaqEnabledTestAlg, SCT_TestCablingAlg, SCTEventFlagWriter, SCTRawDataProvider, SCTSiLorentzAngleTestAlg, SCTSiPropertiesTestAlg, SGInputLoader, Simulation::BeamEffectsAlg, TileHitVecToCnt, TileMuonFitter, TilePulseForTileMuonReceiver, TileRawChannelMaker, TRTDigitization, and ZDC_DigiTop.

Definition at line 68 of file AthCommonAlgorithm.h.

                                           {
    return true;
  }

◆ msg()

MsgStream & AthCommonMsg< Gaudi::Algorithm >::msg ( ) const

inlineinherited

Definition at line 24 of file AthCommonMsg.h.

                                {
    return this->msgStream();
  }

◆ msgLvl()

bool AthCommonMsg< Gaudi::Algorithm >::msgLvl ( const MSG::Level lvl ) const

inlineinherited

Definition at line 30 of file AthCommonMsg.h.

                                               {
    return this->msgLevel(lvl);
  }

◆ outputHandles()

virtual std::vector< Gaudi::DataHandle * > AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::outputHandles ( ) const

overridevirtualinherited

Return this algorithm's output handles.

We override this to include handle instances from key arrays if they have not yet been declared. See comments on updateVHKA.

◆ renounce()

std::enable_if_t< std::is_void_v< std::result_of_t< decltype(&T::renounce)(T)> > &&!std::is_base_of_v< SG::VarHandleKeyArray, T > &&std::is_base_of_v< Gaudi::DataHandle, T >, void > AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::renounce ( T & h )

inlineprotectedinherited

Definition at line 380 of file AthCommonDataStore.h.

  {
    h.renounce();
    PBASE::renounce (h);
  }

◆ renounceArray()

void AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::renounceArray ( SG::VarHandleKeyArray & handlesArray )

inlineprotectedinherited

remove all handles from I/O resolution

Definition at line 364 of file AthCommonDataStore.h.

                                                          {
    handlesArray.renounce();
  }

◆ setFilterPassed()

virtual void AthCommonAlgorithm< Gaudi::Algorithm >::setFilterPassed	(	bool	state,
		const EventContext &	ctx ) const

inlinevirtualinherited

Set filter decision:

Reimplemented in AthFilterAlgorithm.

Definition at line 99 of file AthCommonAlgorithm.h.

                                                                            {
    execState( ctx ).setFilterPassed( state );
  }

◆ sysExecute()

StatusCode AthCommonAlgorithm< Gaudi::Algorithm >::sysExecute ( const EventContext & ctx )

overridevirtualinherited

Execute an algorithm.

We override this in order to work around an issue with the Algorithm base class storing the event context in a member variable that can cause crashes in MT jobs.

Reimplemented in AthAnalysisAlgorithm.

Definition at line 80 of file AthCommonAlgorithm.cxx.

{
  return BaseAlg::sysExecute (ctx);
}

◆ sysInitialize()

StatusCode AthCommonAlgorithm< Gaudi::Algorithm >::sysInitialize ( )

overridevirtualinherited

Override sysInitialize.

Override sysInitialize from the base class.

Loop through all output handles, and if they're WriteCondHandles, automatically register them and this Algorithm with the CondSvc

Scan through all outputHandles, and if they're WriteCondHandles, register them with the CondSvc

Reimplemented from AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >.

Reimplemented in AthAnalysisAlgorithm, AthFilterAlgorithm, AthHistogramAlgorithm, HypoBase, InputMakerBase, and PyAthena::Alg.

Definition at line 60 of file AthCommonAlgorithm.cxx.

                                                      {
  StatusCode sc=AthCommonDataStore<AthCommonMsg<BaseAlg>>::sysInitialize();
 
  if (sc.isFailure()) {
    return sc;
  }
  
  ServiceHandle<ICondSvc> cs("CondSvc",name());
  for (auto h : outputHandles()) {
    if (h->isCondition() && h->mode() == Gaudi::DataHandle::Writer) {
      // do this inside the loop so we don't create the CondSvc until needed
      if ( cs.retrieve().isFailure() ) {
        ATH_MSG_WARNING("no CondSvc found: won't autoreg WriteCondHandles");
        return StatusCode::SUCCESS;
      }      
      if (cs->regHandle(this,*h).isFailure()) {
        sc = StatusCode::FAILURE;
        ATH_MSG_ERROR("unable to register WriteCondHandle " << h->fullKey()
                      << " with CondSvc");
      }
    }
  }
  return sc;  
}

◆ sysStart()

virtual StatusCode AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::sysStart ( )

overridevirtualinherited

Handle START transition.

We override this in order to make sure that conditions handle keys can cache a pointer to the conditions container.

◆ updateVHKA()

void AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::updateVHKA ( Gaudi::Details::PropertyBase & )

inlineinherited

Definition at line 308 of file AthCommonDataStore.h.

                                               {
    // debug() << "updateVHKA for property " << p.name() << " " << p.toString() 
    //         << "  size: " << m_vhka.size() << endmsg;
    for (auto &a : m_vhka) {
      std::vector<SG::VarHandleKey*> keys = a->keys();
      for (auto k : keys) {
        k->setOwner(this);
      }
    }
  }

Member Data Documentation

◆ m_batchSize

Gaudi::Property<int> AthOnnx::EvaluateModelWithAthInfer::m_batchSize {this, "BatchSize", 1, "No. of elements/example in a batch"}

private

Following properties needed to be consdered if the .onnx model is evaluated in batch mode.

Definition at line 55 of file EvaluateModelWithAthInfer.h.

55{this, "BatchSize", 1, "No. of elements/example in a batch"};

◆ m_detStore

StoreGateSvc_t AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_detStore

privateinherited

Pointer to StoreGate (detector store by default).

Definition at line 393 of file AthCommonDataStore.h.

◆ m_evtStore

StoreGateSvc_t AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_evtStore

privateinherited

Pointer to StoreGate (event store by default).

Definition at line 390 of file AthCommonDataStore.h.

◆ m_extendedExtraObjects

DataObjIDColl AthCommonAlgorithm< Gaudi::Algorithm >::m_extendedExtraObjects

privateinherited

Extra output dependency collection, extended by AthAlgorithmDHUpdate to add symlinks.

Empty if no symlinks were found.

Definition at line 108 of file AthCommonAlgorithm.h.

◆ m_input_tensor_values_notFlat

std::vector<std::vector<std::vector<float> > > AthOnnx::EvaluateModelWithAthInfer::m_input_tensor_values_notFlat

private

Definition at line 62 of file EvaluateModelWithAthInfer.h.

◆ m_onnxTool

ToolHandle< AthInfer::IAthInferenceTool > AthOnnx::EvaluateModelWithAthInfer::m_onnxTool

private

Initial value:

{
         this, "ORTInferenceTool", "AthOnnx::OnnxRuntimeInferenceTool"
      }

Tool handler for onnx inference session.

Definition at line 58 of file EvaluateModelWithAthInfer.h.

                                                         {
         this, "ORTInferenceTool", "AthOnnx::OnnxRuntimeInferenceTool"
      };

◆ m_pixelFileName

Gaudi::Property< std::string > AthOnnx::EvaluateModelWithAthInfer::m_pixelFileName

private

Initial value:

{ this, "InputDataPixel",
         "dev/MLTest/2020-03-31/t10k-images-idx3-ubyte",
         "Name of the input pixel file to load" }

Name of the model file to load.

Definition at line 50 of file EvaluateModelWithAthInfer.h.

                                                { this, "InputDataPixel",
         "dev/MLTest/2020-03-31/t10k-images-idx3-ubyte",
         "Name of the input pixel file to load" };

◆ m_varHandleArraysDeclared

bool AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_varHandleArraysDeclared

privateinherited

Definition at line 399 of file AthCommonDataStore.h.

◆ m_vhka

std::vector<SG::VarHandleKeyArray*> AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_vhka

privateinherited

Definition at line 398 of file AthCommonDataStore.h.

The documentation for this class was generated from the following files:

Public Member Functions

Protected Member Functions

Private Types

Private Member Functions

Private Attributes

Detailed Description

Member Typedef Documentation

◆ StoreGateSvc_t

Member Function Documentation

◆ declareGaudiProperty()

◆ declareProperty()

◆ detStore()

◆ evtStore()

◆ execute()

◆ extraDeps_update_handler()

◆ extraOutputDeps()

◆ filterPassed()

◆ initialize()

◆ inputHandles()

◆ isClonable()

◆ msg()

◆ msgLvl()

◆ outputHandles()

◆ renounce()

◆ renounceArray()

◆ setFilterPassed()

◆ sysExecute()

◆ sysInitialize()

◆ sysStart()

◆ updateVHKA()

Member Data Documentation

◆ m_batchSize

◆ m_detStore

◆ m_evtStore

◆ m_extendedExtraObjects

◆ m_input_tensor_values_notFlat

◆ m_onnxTool

◆ m_pixelFileName

◆ m_varHandleArraysDeclared

◆ m_vhka