BasicGPUToAthenaImporter Class Reference

Standard tool to convert the GPU data representation back to CPU. More...

#include <BasicGPUToAthenaImporter.h>

Inheritance diagram for BasicGPUToAthenaImporter:

Collaboration diagram for BasicGPUToAthenaImporter:

Public Member Functions
	BasicGPUToAthenaImporter (const std::string &type, const std::string &name, const IInterface *parent)
virtual StatusCode	initialize () override
virtual StatusCode	convert (const EventContext &ctx, const CaloRecGPU::ConstantDataHolder &constant_data, CaloRecGPU::EventDataHolder &event_data, xAOD::CaloClusterContainer *cluster_collection) const override
virtual StatusCode	finalize () override
virtual	~BasicGPUToAthenaImporter ()=default

Protected Member Functions
void	record_times (const size_t event_num, const std::vector< size_t > &times) const
template<class ... Args>
void	record_times (const size_t event_num, const size_t &value) const
template<class ... Args>
void	record_times (const size_t event_num, const size_t &value, Args &&... args) const
void	print_times (const std::string &header, const size_t time_size) const

Protected Attributes
std::shared_mutex	m_timeMutex
	Mutex that is locked when recording times.
std::vector< size_t > m_times	ATLAS_THREAD_SAFE
	Vector to hold execution times to be recorded if necessary.
std::vector< size_t > m_eventNumbers	ATLAS_THREAD_SAFE
	Vector to hold the event numbers to be recorded if necessary.
Gaudi::Property< bool >	m_measureTimes
	If true, times are recorded to the file given by m_timeFileName.
Gaudi::Property< std::string >	m_timeFileName
	File to which times should be saved.

Private Member Functions
void	record_times_helper (const size_t) const
template<class Arg>
void	record_times_helper (const size_t index, Arg &&arg) const
template<class ... Args>
void	record_times_helper (size_t index, Args &&... args) const

Private Attributes
Gaudi::Property< bool >	m_useCPUPropertiesCalculation {this, "UseCPUClusterPropertiesCalculation", false, "Use CaloClusterKineHelper::calculateKine instead of GPU-calculated cluster properties"}
	if set to true, cluster properties are (re-)calculated using CaloClusterKineHelper::calculateKine.
SG::ReadHandleKey< CaloCellContainer >	m_cellsKey {this, "CellsName", "", "Name(s) of Cell Containers"}
	vector of names of the cell containers to use as input.
Gaudi::Property< std::string >	m_clusterSizeString {this, "ClusterSize", "Topo_420", "The size/type of the clusters"}
	Cluster size. Should be set accordingly to the threshold.
xAOD::CaloCluster::ClusterSize	m_clusterSize
Gaudi::Property< std::vector< int > >	m_missingCellsToFill {this, "MissingCellsToFill", {}, "Force fill these cells as disabled on empty containers."}
	Cell indices to fill as disabled cells (useful if the cell vector is always missing the same cells).
const CaloCell_ID *	m_calo_id {nullptr}
	Pointer to Calo ID Helper.
Gaudi::Property< bool >	m_saveUncalibrated {this, "SaveUncalibratedSignalState", true, "Use CaloClusterKineHelper::calculateKine instead of GPU-calculated cluster properties"}
	if set to true, the uncalibrated state is saved when importing the clusters.
ToolHandle< AthCUDA::IStreamTool >	m_streamTool {this, "StreamTool", "", "CUDA stream tool."}
	Tool providing the CUDA stream to use.

Detailed Description

Standard tool to convert the GPU data representation back to CPU.

Author

Nuno Fernandes nuno..nosp@m.dos..nosp@m.santo.nosp@m.s.fe.nosp@m.rnand.nosp@m.es@c.nosp@m.ern.c.nosp@m.h

Date

30 May 2022

Definition at line 31 of file BasicGPUToAthenaImporter.h.

Constructor & Destructor Documentation

BasicGPUToAthenaImporter()

BasicGPUToAthenaImporter::BasicGPUToAthenaImporter	(	const std::string &	type,
		const std::string &	name,
		const IInterface *	parent )

Definition at line 25 of file BasicGPUToAthenaImporter.cxx.

                                                                                                                           :
  base_class(type, name, parent),
  CaloGPUTimed(this)
{
}

~BasicGPUToAthenaImporter()

virtual BasicGPUToAthenaImporter::~BasicGPUToAthenaImporter ( )

virtualdefault

Member Function Documentation

convert()

StatusCode BasicGPUToAthenaImporter::convert ( const EventContext & ctx, const CaloRecGPU::ConstantDataHolder & constant_data, CaloRecGPU::EventDataHolder & event_data, xAOD::CaloClusterContainer * cluster_collection ) const

overridevirtual

Definition at line 91 of file BasicGPUToAthenaImporter.cxx.

{
  using clock_type = boost::chrono::thread_clock;
  auto time_cast = [](const auto & before, const auto & after)
  {
    return boost::chrono::duration_cast<boost::chrono::microseconds>(after - before).count();
  };
 
  cluster_container->clear();
 
  const auto start = clock_type::now();
 
  SG::ReadHandle<CaloCellContainer> cell_collection(m_cellsKey, ctx);
  if ( !cell_collection.isValid() )
    {
      ATH_MSG_ERROR( " Cannot retrieve CaloCellContainer: " << cell_collection.name()  );
      return StatusCode::FAILURE;
    }
  const DataLink<CaloCellContainer> cell_collection_link (cell_collection.name(), ctx);
 
  ed.returnToCPU(MomentsOptionsArray::all(), m_streamTool->stream(ctx), false, false, true);
 
  const auto after_send = clock_type::now();
 
  size_t extra_times[5];
 
  ed.exportClusters(cluster_container,
                    cell_collection_link,
                    MomentsOptionsArray::all(),
                    true,
                    m_saveUncalibrated,
                    false,
                    m_missingCellsToFill,
                    m_measureTimes ? extra_times : nullptr);
 
  const auto after_export = clock_type::now();
 
  for (auto && cluster : *cluster_container)
    {
      cluster->setClusterSize(m_clusterSize);
    }
 
  const auto after_size = clock_type::now();
 
  if (m_measureTimes)
    {
      record_times(ctx.evt(),
                  time_cast(start, after_send),
                  extra_times[0],
                  extra_times[1],
                  extra_times[2],
                  extra_times[3],
                  extra_times[4],
                  time_cast(after_export, after_size)
                  );
    }
 
  return StatusCode::SUCCESS;
 
}

finalize()

StatusCode BasicGPUToAthenaImporter::finalize ( )

overridevirtual

Definition at line 156 of file BasicGPUToAthenaImporter.cxx.

{
 
  if (m_measureTimes)
    {
      print_times("Transfer_from_GPU Cell_Link_Creation Cell_Adding Sorting Filling_Collection Moments Cluster_Size", 7);
    }
  return StatusCode::SUCCESS;
}

initialize()

StatusCode BasicGPUToAthenaImporter::initialize ( )

overridevirtual

Definition at line 33 of file BasicGPUToAthenaImporter.cxx.

{
  ATH_CHECK( m_cellsKey.initialize() );
 
  ATH_CHECK( detStore()->retrieve(m_calo_id, "CaloCell_ID") );
 
  auto get_option_from_string = [](const std::string & str, bool & failed)
  {
    failed = false;
    //cppcheck-suppress syntaxError
    CRGPU_RECURSIVE_MACRO(
            CRGPU_CHEAP_STRING_TO_ENUM( str, xAOD::CaloCluster,
                                        SW_55ele,
                                        SW_35ele,
                                        SW_37ele,
                                        SW_55gam,
                                        SW_35gam,
                                        SW_37gam,
                                        SW_55Econv,
                                        SW_35Econv,
                                        SW_37Econv,
                                        SW_softe,
                                        Topo_420,
                                        Topo_633,
                                        SW_7_11,
                                        SuperCluster,
                                        Tower_01_01,
                                        Tower_005_005,
                                        Tower_fixed_area
                                      )
    )
    //I know Topological Clustering only supports a subset of those,
    //but this is supposed to be a general data exporting tool...
    else
      {
        //failed = true;
        return xAOD::CaloCluster::CSize_Unknown;
      }
  };
 
  bool size_failed = false;
  m_clusterSize = get_option_from_string(m_clusterSizeString, size_failed);
 
  if (m_clusterSize == xAOD::CaloCluster::CSize_Unknown)
    {
      ATH_MSG_ERROR("Invalid Cluster Size: " << m_clusterSizeString);
    }
 
  if (size_failed)
    {
      return StatusCode::FAILURE;
    }
 
  ATH_CHECK( m_streamTool.retrieve() );
 
  return StatusCode::SUCCESS;
}

print_times()

void CaloGPUTimed::print_times ( const std::string & header, const size_t time_size ) const

inlineprotectedinherited

Definition at line 143 of file CaloGPUTimed.h.

  {
    std::shared_lock<std::shared_mutex> lock(m_timeMutex);
    
    if (m_timeFileName.size() == 0)
      {
        return;
      }
    
    std::vector<size_t> indices(m_eventNumbers.size());
    
    std::iota(indices.begin(), indices.end(), 0);
    std::sort(indices.begin(), indices.end(), [&](size_t a, size_t b)
    {
      return m_eventNumbers[a] < m_eventNumbers[b];
    }
             );
    std::ofstream out(m_timeFileName);
 
    out << "Event_Number Total " << header << "\n";
 
    for (const size_t idx : indices)
      {
        out << m_eventNumbers[idx] << " ";
 
        size_t total = 0;
 
        for (size_t i = 0; i < time_size; ++i)
          {
            total += m_times[idx * time_size + i];
          }
 
        out << total << " ";
 
        for (size_t i = 0; i < time_size; ++i)
          {
            out << m_times[idx * time_size + i] << " ";
          }
        out << "\n";
      }
 
    out << std::endl;
 
    out.close();
  }

record_times() [1/3]

template<class ... Args>

void CaloGPUTimed::record_times ( const size_t event_num, const size_t & value ) const

inlineprotectedinherited

Definition at line 105 of file CaloGPUTimed.h.

  {
    const size_t time_size = 1;
 
    size_t old_size;
 
    {
      std::unique_lock<std::shared_mutex> lock(m_timeMutex);
      old_size = m_times.size();
      m_times.resize(old_size + time_size);
      m_eventNumbers.push_back(event_num);
    }
    {
      std::shared_lock<std::shared_mutex> lock(m_timeMutex);
      record_times_helper(old_size, value);
    }
  }

record_times() [2/3]

template<class ... Args>

void CaloGPUTimed::record_times ( const size_t event_num, const size_t & value, Args &&... args ) const

inlineprotectedinherited

Definition at line 124 of file CaloGPUTimed.h.

  {
    const size_t time_size = sizeof...(args) + 1;
 
    size_t old_size;
 
    {
      std::unique_lock<std::shared_mutex> lock(m_timeMutex);
      old_size = m_times.size();
      m_times.resize(old_size + time_size);
      m_eventNumbers.push_back(event_num);
    }
    {
      std::shared_lock<std::shared_mutex> lock(m_timeMutex);
      record_times_helper(old_size, value, std::forward<Args>(args)...);
    }
 
  }

record_times() [3/3]

void CaloGPUTimed::record_times ( const size_t event_num, const std::vector< size_t > & times ) const

inlineprotectedinherited

Definition at line 86 of file CaloGPUTimed.h.

  {
    size_t old_size;
    {
      std::unique_lock<std::shared_mutex> lock(m_timeMutex);
      old_size = m_times.size();
      m_times.resize(old_size + times.size());
      m_eventNumbers.push_back(event_num);
    }
    {
      std::shared_lock<std::shared_mutex> lock(m_timeMutex);
      for (size_t i = 0; i < times.size(); ++i)
        {
          m_times[old_size + i] = times[i];
        }
    }
  }

record_times_helper() [1/3]

template<class Arg>

void CaloGPUTimed::record_times_helper ( const size_t index, Arg && arg ) const

inlineprivateinherited

Definition at line 70 of file CaloGPUTimed.h.

  {
    // coverity[missing_lock]
    m_times[index] = std::forward<Arg>(arg);
    
    //This is called within a function that holds the lock itself.
  }

record_times_helper() [2/3]

void CaloGPUTimed::record_times_helper ( const size_t ) const

inlineprivateinherited

Definition at line 64 of file CaloGPUTimed.h.

  {
    //Do nothing
  }

record_times_helper() [3/3]

template<class ... Args>

void CaloGPUTimed::record_times_helper ( size_t index, Args &&... args ) const

inlineprivateinherited

Definition at line 79 of file CaloGPUTimed.h.

  {
    (record_times_helper(index++, std::forward<Args>(args)), ...);
  }

Member Data Documentation

ATLAS_THREAD_SAFE [1/2]

std::vector<size_t> m_eventNumbers CaloGPUTimed::ATLAS_THREAD_SAFE

mutableprotectedinherited

Vector to hold the event numbers to be recorded if necessary.

Definition at line 40 of file CaloGPUTimed.h.

ATLAS_THREAD_SAFE [2/2]

std::vector<size_t> m_times CaloGPUTimed::ATLAS_THREAD_SAFE

mutableprotectedinherited

Vector to hold execution times to be recorded if necessary.

Definition at line 35 of file CaloGPUTimed.h.

m_calo_id

const CaloCell_ID* BasicGPUToAthenaImporter::m_calo_id {nullptr}

private

Pointer to Calo ID Helper.

Definition at line 72 of file BasicGPUToAthenaImporter.h.

{nullptr};

m_cellsKey

SG::ReadHandleKey<CaloCellContainer> BasicGPUToAthenaImporter::m_cellsKey {this, "CellsName", "", "Name(s) of Cell Containers"}

private

vector of names of the cell containers to use as input.

Definition at line 58 of file BasicGPUToAthenaImporter.h.

{this, "CellsName", "", "Name(s) of Cell Containers"};

m_clusterSize

xAOD::CaloCluster::ClusterSize BasicGPUToAthenaImporter::m_clusterSize

private

Definition at line 63 of file BasicGPUToAthenaImporter.h.

m_clusterSizeString

Gaudi::Property<std::string> BasicGPUToAthenaImporter::m_clusterSizeString {this, "ClusterSize", "Topo_420", "The size/type of the clusters"}

private

Cluster size. Should be set accordingly to the threshold.

Definition at line 61 of file BasicGPUToAthenaImporter.h.

{this, "ClusterSize", "Topo_420", "The size/type of the clusters"};

m_measureTimes

Gaudi::Property<bool> CaloGPUTimed::m_measureTimes

protectedinherited

If true, times are recorded to the file given by m_timeFileName.

Defaults to false.

Definition at line 46 of file CaloGPUTimed.h.

m_missingCellsToFill

Gaudi::Property<std::vector<int> > BasicGPUToAthenaImporter::m_missingCellsToFill {this, "MissingCellsToFill", {}, "Force fill these cells as disabled on empty containers."}

private

Cell indices to fill as disabled cells (useful if the cell vector is always missing the same cells).

Definition at line 67 of file BasicGPUToAthenaImporter.h.

{this, "MissingCellsToFill", {}, "Force fill these cells as disabled on empty containers."};

m_saveUncalibrated

Gaudi::Property<bool> BasicGPUToAthenaImporter::m_saveUncalibrated {this, "SaveUncalibratedSignalState", true, "Use CaloClusterKineHelper::calculateKine instead of GPU-calculated cluster properties"}

private

if set to true, the uncalibrated state is saved when importing the clusters.

Default is true.

Definition at line 77 of file BasicGPUToAthenaImporter.h.

{this, "SaveUncalibratedSignalState", true, "Use CaloClusterKineHelper::calculateKine instead of GPU-calculated cluster properties"};

m_streamTool

ToolHandle<AthCUDA::IStreamTool> BasicGPUToAthenaImporter::m_streamTool {this, "StreamTool", "", "CUDA stream tool."}

private

Tool providing the CUDA stream to use.

Definition at line 80 of file BasicGPUToAthenaImporter.h.

{this, "StreamTool", "", "CUDA stream tool."};

m_timeFileName

Gaudi::Property<std::string> CaloGPUTimed::m_timeFileName

protectedinherited

File to which times should be saved.

Definition at line 50 of file CaloGPUTimed.h.

m_timeMutex

std::shared_mutex CaloGPUTimed::m_timeMutex

mutableprotectedinherited

Mutex that is locked when recording times.

Definition at line 32 of file CaloGPUTimed.h.

m_useCPUPropertiesCalculation

Gaudi::Property<bool> BasicGPUToAthenaImporter::m_useCPUPropertiesCalculation {this, "UseCPUClusterPropertiesCalculation", false, "Use CaloClusterKineHelper::calculateKine instead of GPU-calculated cluster properties"}

private

if set to true, cluster properties are (re-)calculated using CaloClusterKineHelper::calculateKine.

Else, the GPU-calculated values are used. Default is false.

Definition at line 53 of file BasicGPUToAthenaImporter.h.

{this, "UseCPUClusterPropertiesCalculation", false, "Use CaloClusterKineHelper::calculateKine instead of GPU-calculated cluster properties"};

---

The documentation for this class was generated from the following files:

• BasicGPUToAthenaImporter.h
• BasicGPUToAthenaImporter.cxx