PixelClustering Class Reference

Class for the pixel clustering kernel. More...

#include <PixelClustering.h>

Inheritance diagram for PixelClustering:

Collaboration diagram for PixelClustering:

Public Member Functions
StatusCode	initialize () override
	Detect the OpenCL devices and prepare OpenCL context.
StatusCode	execute (const EventContext &ctx) const override
	Should be overriden by derived classes to perform meaningful work.
StatusCode	loadProgram (const std::string &xclbin)
	Find the xclbin file and load it into the OpenCL program object.
StatusCode	precheck (const std::vector< Gaudi::Property< std::string > > &inputs) const
	Check if the the desired Gaudi properties are set.
virtual StatusCode	sysInitialize () override
	Override sysInitialize.
virtual bool	isClonable () const override
	Specify if the algorithm is clonable.
virtual unsigned int	cardinality () const override
	Cardinality (Maximum number of clones that can exist) special value 0 means that algorithm is reentrant.
virtual StatusCode	sysExecute (const EventContext &ctx) override
	Execute an algorithm.
virtual const DataObjIDColl &	extraOutputDeps () const override
	Return the list of extra output dependencies.
virtual bool	filterPassed (const EventContext &ctx) const
virtual void	setFilterPassed (bool state, const EventContext &ctx) const
ServiceHandle< StoreGateSvc > &	evtStore ()
	The standard StoreGateSvc (event store) Returns (kind of) a pointer to the StoreGateSvc.
const ServiceHandle< StoreGateSvc > &	detStore () const
	The standard StoreGateSvc/DetectorStore Returns (kind of) a pointer to the StoreGateSvc.
virtual StatusCode	sysStart () override
	Handle START transition.
virtual std::vector< Gaudi::DataHandle * >	inputHandles () const override
	Return this algorithm's input handles.
virtual std::vector< Gaudi::DataHandle * >	outputHandles () const override
	Return this algorithm's output handles.
Gaudi::Details::PropertyBase &	declareProperty (Gaudi::Property< T, V, H > &t)
void	updateVHKA (Gaudi::Details::PropertyBase &)
MsgStream &	msg () const
bool	msgLvl (const MSG::Level lvl) const

Protected Member Functions
void	renounceArray (SG::VarHandleKeyArray &handlesArray)
	remove all handles from I/O resolution
std::enable_if_t< std::is_void_v< std::result_of_t< decltype(&T::renounce)(T)> > &&!std::is_base_of_v< SG::VarHandleKeyArray, T > &&std::is_base_of_v< Gaudi::DataHandle, T >, void >	renounce (T &h)
void	extraDeps_update_handler (Gaudi::Details::PropertyBase &ExtraDeps)
	Add StoreName to extra input/output deps as needed.

Protected Attributes
cl::Device	m_accelerator
	Device object for the accelerator card.
cl::Context	m_context
	Context object for the application.
cl::Program	m_program
	Program object containing the kernel.
Gaudi::Property< std::string >	m_deviceBDF {this, "bdfID", "", "BDF ID of the accelerator card"}
	BDF ID of the accelerator card.
Gaudi::Property< bool >	m_doEmulation {this, "doEmulation", false, "If software or hardware emulation is being used for debugging"}

Private Types
typedef ServiceHandle< StoreGateSvc >	StoreGateSvc_t

Private Member Functions
Gaudi::Details::PropertyBase &	declareGaudiProperty (Gaudi::Property< T, V, H > &hndl, const SG::VarHandleKeyType &)
	specialization for handling Gaudi::Property<SG::VarHandleKey>

Private Attributes
Gaudi::Property< std::string >	m_xclbin {this, "xclbin", "", "xclbin path and name"}
	Path and name of the xclbin file.
Gaudi::Property< std::string >	m_kernelName {this, "KernelName", "", "Kernel name"}
	Kernel name.
Gaudi::Property< std::string >	m_inputTV {this, "InputTV", "", "Input TestVector"}
	Input TestVector.
Gaudi::Property< std::string >	m_refTV {this, "RefTV", "", "Reference TestVector"}
	Reference TestVector.
SG::ReadHandleKey< PixelRDO_Container >	m_pixelRDOKey { this, "PixelRDO", "ITkPixelRDOs" }
ToolHandle< FPGADataFormatTool >	m_FPGADataFormatTool {this, "FPGADataFormatTool", "FPGADataFormatTool", "Tool to convert RDOs into FPGA data format"}
ToolHandle< xAODClusterMaker >	m_xAODClusterMaker {this, "xAODClusterMaker", "xAODClusterMaker", "Tool for creating xAOD containers"}
DataObjIDColl	m_extendedExtraObjects
	Extra output dependency collection, extended by AthAlgorithmDHUpdate to add symlinks.
StoreGateSvc_t	m_evtStore
	Pointer to StoreGate (event store by default)
StoreGateSvc_t	m_detStore
	Pointer to StoreGate (detector store by default)
std::vector< SG::VarHandleKeyArray * >	m_vhka
bool	m_varHandleArraysDeclared

Detailed Description

Class for the pixel clustering kernel.

Author

zhaoy.nosp@m.uan..nosp@m.cui@c.nosp@m.ern..nosp@m.ch

Date

Sep. 10, 2024

Definition at line 24 of file PixelClustering.h.

Member Typedef Documentation

StoreGateSvc_t

typedef ServiceHandle<StoreGateSvc> AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::StoreGateSvc_t

privateinherited

Definition at line 388 of file AthCommonDataStore.h.

Member Function Documentation

cardinality()

unsigned int AthCommonReentrantAlgorithm< Gaudi::Algorithm >::cardinality ( ) const

overridevirtualinherited

Cardinality (Maximum number of clones that can exist) special value 0 means that algorithm is reentrant.

Override this to return 0 for reentrant algorithms.

Definition at line 75 of file AthCommonReentrantAlgorithm.cxx.

{
  return 0;
}

declareGaudiProperty()

Gaudi::Details::PropertyBase & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::declareGaudiProperty ( Gaudi::Property< T, V, H > & hndl, const SG::VarHandleKeyType &  )

inlineprivateinherited

specialization for handling Gaudi::Property<SG::VarHandleKey>

Definition at line 156 of file AthCommonDataStore.h.

  {
    return *AthCommonDataStore<PBASE>::declareProperty(hndl.name(),
                                                       hndl.value(), 
                                                       hndl.documentation());
 
  }

declareProperty()

Gaudi::Details::PropertyBase & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::declareProperty ( Gaudi::Property< T, V, H > & t )

inlineinherited

Definition at line 145 of file AthCommonDataStore.h.

                                                                     {
    typedef typename SG::HandleClassifier<T>::type htype;
    return AthCommonDataStore<PBASE>::declareGaudiProperty(t, htype());
  }

detStore()

const ServiceHandle< StoreGateSvc > & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::detStore ( ) const

inlineinherited

The standard StoreGateSvc/DetectorStore Returns (kind of) a pointer to the StoreGateSvc.

Definition at line 95 of file AthCommonDataStore.h.

{ return m_detStore; }

evtStore()

ServiceHandle< StoreGateSvc > & AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::evtStore ( )

inlineinherited

The standard StoreGateSvc (event store) Returns (kind of) a pointer to the StoreGateSvc.

Definition at line 85 of file AthCommonDataStore.h.

{ return m_evtStore; }

execute()

StatusCode PixelClustering::execute ( const EventContext & ctx ) const

overridevirtual

Should be overriden by derived classes to perform meaningful work.

Reimplemented from IntegrationBase.

Definition at line 32 of file PixelClustering.cxx.

{
    ATH_MSG_DEBUG("In execute(), event slot: " << ctx.slot());
 
    auto pixelRDOHandle = SG::makeHandle(m_pixelRDOKey, ctx);
 
    std::vector<uint64_t> inbufPixClustVec;
    std::vector<IdentifierHash> listOfIds;
    if(!m_FPGADataFormatTool->convertPixelHitsToFPGADataFormat(*pixelRDOHandle, inbufPixClustVec, listOfIds, ctx)) {
        return StatusCode::FAILURE;
    }
 
    unsigned int inbufPixClustSize = inbufPixClustVec.size();
 
    cl_int err;
 
    cl::CommandQueue queuePixClust{};
    OCL_CHECK(err, queuePixClust = cl::CommandQueue(m_context, m_accelerator, CL_QUEUE_PROFILING_ENABLE, &err));
 
    cl::Kernel kernelPixClust{};
    OCL_CHECK(err, kernelPixClust = cl::Kernel(m_program, "pixel_clustering_tool", &err));
 
    cl::Buffer clInbufPixClust{};
    std::unique_ptr<uint64_t, decltype(std::free) *> inbufPixClust(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * inbufPixClustSize)), std::free);
    for (unsigned int i = 0; i < inbufPixClustSize; ++i) {
        inbufPixClust.get()[i] = inbufPixClustVec.at(i);
        ATH_MSG_DEBUG("inbufPixClust[" << std::setw(6) << i << "] = 0x" << std::hex << std::setfill('0') << std::setw(16) << inbufPixClust.get()[i] << std::setfill(' ') << std::dec);
    }
    // Clean inbufPixClustVec to save memory
    inbufPixClustVec.resize(1);
    inbufPixClustVec.clear();
    OCL_CHECK(err, clInbufPixClust = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_READ_ONLY, sizeof(uint64_t) * inbufPixClustSize, inbufPixClust.get(), &err));
    OCL_CHECK(err, err = kernelPixClust.setArg(0, clInbufPixClust));
    cl::Buffer clOutbufPixClust{};
    unsigned int outbufPixClustSize = inbufPixClustSize + 9164 + 6;
    std::unique_ptr<uint64_t, decltype(std::free) *> outbufPixClust(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * outbufPixClustSize)), std::free);
    OCL_CHECK(err, clOutbufPixClust = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_WRITE_ONLY, sizeof(uint64_t) * outbufPixClustSize, outbufPixClust.get(), &err));
    OCL_CHECK(err, err = kernelPixClust.setArg(1, clOutbufPixClust));
    cl::Buffer clOutbufEDMPixClust{};
    unsigned int outbufEDMPixClustSize = inbufPixClustSize * 10 + 6;
    std::unique_ptr<uint64_t, decltype(std::free) *> outbufEDMPixClust(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * outbufEDMPixClustSize)), std::free);
    OCL_CHECK(err, clOutbufEDMPixClust = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_WRITE_ONLY, sizeof(uint64_t) * outbufEDMPixClustSize, outbufEDMPixClust.get(), &err));
    OCL_CHECK(err, err = kernelPixClust.setArg(2, clOutbufEDMPixClust));
 
    ATH_MSG_INFO("Making clusters out of " << inbufPixClustSize - 6 << " channels over threshold");
    // Write data to accelerator
    auto write_start = std::chrono::high_resolution_clock::now();
    OCL_CHECK(err, err = queuePixClust.enqueueMigrateMemObjects({clInbufPixClust}, 0));
    OCL_CHECK(err, err = queuePixClust.finish());
    auto write_end = std::chrono::high_resolution_clock::now();
 
    // Run algorithm on accelerator
    OCL_CHECK(err, err = queuePixClust.enqueueTask(kernelPixClust));
    OCL_CHECK(err, err = queuePixClust.finish());
    auto compute_end = std::chrono::high_resolution_clock::now();
 
    // Read back result
    OCL_CHECK(err, err = queuePixClust.enqueueMigrateMemObjects({clOutbufPixClust}, CL_MIGRATE_MEM_OBJECT_HOST));
    OCL_CHECK(err, err = queuePixClust.enqueueMigrateMemObjects({clOutbufEDMPixClust}, CL_MIGRATE_MEM_OBJECT_HOST));
    OCL_CHECK(err, err = queuePixClust.finish());
    auto read_end = std::chrono::high_resolution_clock::now();
 
    auto total_latency = std::chrono::duration<float, std::micro>(read_end - write_start).count();
    auto write_latency = std::chrono::duration<float, std::micro>(write_end - write_start).count();
    auto compute_latency = std::chrono::duration<float, std::micro>(compute_end - write_end).count();
    auto read_latency = std::chrono::duration<float, std::micro>(read_end - compute_end).count();
    ATH_MSG_INFO("Total latency:" << total_latency << " us,  " << "Compute latency: " << compute_latency << " us");
    ATH_MSG_INFO("Write latency:" << write_latency << " us,  " << "Read latency: " << read_latency << " us");
    ATH_MSG_INFO("Clusters found, calculate coordinates");
 
    cl::CommandQueue queuePixCoords{};
    OCL_CHECK(err, queuePixCoords = cl::CommandQueue(m_context, m_accelerator, CL_QUEUE_PROFILING_ENABLE, &err));
    cl::Kernel kernelPixCoords{};
    OCL_CHECK(err, kernelPixCoords = cl::Kernel(m_program, "l2g_pixel_tool", &err));
 
    cl::Buffer clInbufPixCoords{};
    std::unique_ptr<uint64_t, decltype(std::free) *> inbufPixCoords(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * outbufPixClustSize)), std::free);
    for (unsigned int i = 0; i < outbufPixClustSize; ++i) {
        ATH_MSG_DEBUG("outbufPixClust[" << std::setw(6) << i << "] = 0x" << std::hex << std::setfill('0') << std::setw(16) << outbufPixClust.get()[i] << std::setfill(' ') << std::dec);
        inbufPixCoords.get()[i] = outbufPixClust.get()[i];
    }
    OCL_CHECK(err, clInbufPixCoords = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_READ_ONLY, sizeof(uint64_t) * outbufPixClustSize, inbufPixCoords.get(), &err));
    OCL_CHECK(err, err = kernelPixCoords.setArg(0, clInbufPixCoords));
 
    cl::Buffer clInbufEDMPixCoords{};
    std::unique_ptr<uint64_t, decltype(std::free) *> inbufEDMPixCoords(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * outbufEDMPixClustSize)), std::free);
    for (unsigned int i = 0; i < outbufEDMPixClustSize; ++i) {
        ATH_MSG_DEBUG("outbufEDMPixClust[" << std::setw(6) << i << "] = 0x" << std::hex << std::setfill('0') << std::setw(16) << outbufEDMPixClust.get()[i] << std::setfill(' ') << std::dec);
        inbufEDMPixCoords.get()[i] = outbufEDMPixClust.get()[i];
        // Resize EDM container to be not larger than neccessary
        if ((i > 2) && ((i - 3) % 10 == 0) && (((outbufEDMPixClust.get()[i] & (((1ULL << 8) - 1ULL) << 56)) >> 56) == 0xcd) && (((outbufEDMPixClust.get()[i - 1] & (((1ULL << 1) - 1ULL) << 25)) >> 25))) {
            outbufEDMPixClustSize = i + 3;
        }
    }
    OCL_CHECK(err, clInbufEDMPixCoords = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_READ_ONLY, sizeof(uint64_t) * outbufEDMPixClustSize, inbufEDMPixCoords.get(), &err));
    OCL_CHECK(err, err = kernelPixCoords.setArg(1, clInbufEDMPixCoords));
 
    cl::Buffer clOutbufPixCoords{};
    unsigned int outbufPixCoordsSize = inbufPixClustSize * 2 + 9164 + 6;
    std::unique_ptr<uint64_t, decltype(std::free) *> outbufPixCoords(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * outbufPixCoordsSize)), std::free);
    OCL_CHECK(err, clOutbufPixCoords = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_WRITE_ONLY, sizeof(uint64_t) * outbufPixCoordsSize, outbufPixCoords.get(), &err));
    OCL_CHECK(err, err = kernelPixCoords.setArg(2, clOutbufPixCoords));
    cl::Buffer clOutbufEDMPixCoords{};
    std::unique_ptr<uint64_t, decltype(std::free) *> outbufEDMPixCoords(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * outbufEDMPixClustSize)), std::free);
    OCL_CHECK(err, clOutbufEDMPixCoords = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_WRITE_ONLY, sizeof(uint64_t) * outbufEDMPixClustSize, outbufEDMPixCoords.get(), &err));
    OCL_CHECK(err, err = kernelPixCoords.setArg(3, clOutbufEDMPixCoords));
 
    // Write data to accelerator
    write_start = std::chrono::high_resolution_clock::now();
    OCL_CHECK(err, err = queuePixCoords.enqueueMigrateMemObjects({clInbufPixCoords}, 0));
    OCL_CHECK(err, err = queuePixCoords.enqueueMigrateMemObjects({clInbufEDMPixCoords}, 0));
    OCL_CHECK(err, err = queuePixCoords.finish());
    write_end = std::chrono::high_resolution_clock::now();
 
    // Run algorithm on accelerator
    OCL_CHECK(err, err = queuePixCoords.enqueueTask(kernelPixCoords));
    OCL_CHECK(err, err = queuePixCoords.finish());
    compute_end = std::chrono::high_resolution_clock::now();
 
    // Read back result
    OCL_CHECK(err, err = queuePixCoords.enqueueMigrateMemObjects({clOutbufPixCoords}, CL_MIGRATE_MEM_OBJECT_HOST));
    OCL_CHECK(err, err = queuePixCoords.enqueueMigrateMemObjects({clOutbufEDMPixCoords}, CL_MIGRATE_MEM_OBJECT_HOST));
    OCL_CHECK(err, err = queuePixCoords.finish());
    read_end = std::chrono::high_resolution_clock::now();
 
    total_latency = std::chrono::duration<float, std::micro>(read_end - write_start).count();
    write_latency = std::chrono::duration<float, std::micro>(write_end - write_start).count();
    compute_latency = std::chrono::duration<float, std::micro>(compute_end - write_end).count();
    read_latency = std::chrono::duration<float, std::micro>(read_end - compute_end).count();
    ATH_MSG_INFO("Total latency:" << total_latency << " us,  " << "Compute latency: " << compute_latency << " us");
    ATH_MSG_INFO("Write latency:" << write_latency << " us,  " << "Read latency: " << read_latency << " us");
    ATH_MSG_INFO("Coordinates calculated, transform to xAOD::PixelCluster compatible formats");
 
    for (unsigned int i = 0; i < outbufPixCoordsSize; ++i) {
        ATH_MSG_DEBUG("outbufPixCoords[" << std::setw(6) << i << "] = 0x" << std::hex << std::setfill('0') << std::setw(16) << outbufPixCoords.get()[i] << std::setfill(' ') << std::dec);
    }
 
    cl::CommandQueue queuePixEDMPrep{};
    OCL_CHECK(err, queuePixEDMPrep = cl::CommandQueue(m_context, m_accelerator, CL_QUEUE_PROFILING_ENABLE, &err));
    cl::Kernel kernelPixEDMPrep{};
    OCL_CHECK(err, kernelPixEDMPrep = cl::Kernel(m_program, "EDMPrep", &err));
 
    cl::Buffer clInbufEDMPixEDMPrep{};
    std::unique_ptr<uint64_t, decltype(std::free) *> inbufEDMPixEDMPrep(static_cast<uint64_t*>(aligned_alloc(64, sizeof(uint64_t) * outbufEDMPixClustSize)), std::free);
    for (unsigned int i = 0; i < outbufEDMPixClustSize; ++i) {
        inbufEDMPixEDMPrep.get()[i] = outbufEDMPixCoords.get()[i];
        ATH_MSG_DEBUG("inbufEDMPixEDMPrep[" << std::setw(6) << i << "] = 0x" << std::hex << std::setfill('0') << std::setw(16) << inbufEDMPixEDMPrep.get()[i] << std::setfill(' ') << std::dec);
    }
    OCL_CHECK(err, clInbufEDMPixEDMPrep = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_READ_ONLY, sizeof(uint64_t) * outbufEDMPixClustSize, inbufEDMPixEDMPrep.get(), &err));
    OCL_CHECK(err, err = kernelPixEDMPrep.setArg(0, clInbufEDMPixEDMPrep));
 
    cl::Buffer clOutbufEDMPixEDMPrep{};
    std::unique_ptr<uint64_t, decltype(std::free) *> outbufEDMPixEDMPrep(static_cast<uint64_t*>(aligned_alloc(512, sizeof(uint64_t) * EFTrackingTransient::PIXEL_CONTAINER_BUF_SIZE)), std::free);
    OCL_CHECK(err, clOutbufEDMPixEDMPrep = cl::Buffer(m_context, CL_MEM_USE_HOST_PTR | CL_MEM_WRITE_ONLY, sizeof(uint64_t) * EFTrackingTransient::PIXEL_CONTAINER_BUF_SIZE, outbufEDMPixEDMPrep.get(), &err));
    OCL_CHECK(err, err = kernelPixEDMPrep.setArg(1, clOutbufEDMPixEDMPrep));
 
    // Write data to accelerator
    write_start = std::chrono::high_resolution_clock::now();
    OCL_CHECK(err, err = queuePixEDMPrep.enqueueMigrateMemObjects({clInbufEDMPixEDMPrep}, 0));
    OCL_CHECK(err, err = queuePixEDMPrep.finish());
    write_end = std::chrono::high_resolution_clock::now();
 
    // Run algorithm on accelerator
    OCL_CHECK(err, err = queuePixEDMPrep.enqueueTask(kernelPixEDMPrep));
    OCL_CHECK(err, err = queuePixEDMPrep.finish());
    compute_end = std::chrono::high_resolution_clock::now();
 
    // Read back result
    OCL_CHECK(err, err = queuePixEDMPrep.enqueueMigrateMemObjects({clOutbufEDMPixEDMPrep}, CL_MIGRATE_MEM_OBJECT_HOST));
    OCL_CHECK(err, err = queuePixEDMPrep.finish());
    read_end = std::chrono::high_resolution_clock::now();
 
    for (unsigned int i = 0; i < EFTrackingTransient::PIXEL_CONTAINER_BUF_SIZE; ++i)
        ATH_MSG_DEBUG("outbufEDMPixEDMPrep[" << std::setw(6) << i << "] = 0x" << std::hex << std::setfill('0') << std::setw(16) << outbufEDMPixEDMPrep.get()[i] << std::setfill(' ') << std::dec);
 
    total_latency = std::chrono::duration<float, std::micro>(read_end - write_start).count();
    write_latency = std::chrono::duration<float, std::micro>(write_end - write_start).count();
    compute_latency = std::chrono::duration<float, std::micro>(compute_end - write_end).count();
    read_latency = std::chrono::duration<float, std::micro>(read_end - compute_end).count();
    ATH_MSG_INFO("Total latency:" << total_latency << " us,  " << "Compute latency: " << compute_latency << " us");
    ATH_MSG_INFO("Write latency:" << write_latency << " us,  " << "Read latency: " << read_latency << " us");
    ATH_MSG_INFO("Received data for making xAOD::PixelCluster's");
 
    uint64_t* pixelClusters = (uint64_t*)outbufEDMPixEDMPrep.get();
    unsigned int numClusters = pixelClusters[0];
    ATH_MSG_INFO("Received " << numClusters << " clusters");
 
    write_start = std::chrono::high_resolution_clock::now();
    std::unique_ptr<EFTrackingTransient::Metadata> metadata = std::make_unique<EFTrackingTransient::Metadata>();
 
    metadata->numOfPixelClusters = numClusters;
    metadata->pcRdoIndexSize = numClusters;
 
    EFTrackingTransient::PixelClusterAuxInput pcAux;
 
    int rdoCounter, row;
    uint64_t rdo;
    for (unsigned int i = 0; i < numClusters; ++i) {
        ATH_MSG_DEBUG("New Cluster number " << i);
        rdoCounter = 0;
        row = 0;
        pcAux.idHash.push_back(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]);
        ATH_MSG_DEBUG("idHash = " << pcAux.idHash.back());
 
        row++;
        pcAux.id.push_back(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]);
        ATH_MSG_DEBUG("id = " << pcAux.id.back());
 
        row++;
        rdo = pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8];
        if (rdo) {
            pcAux.rdoList.push_back(rdo);
            rdoCounter++;
            ATH_MSG_DEBUG("rdo = " << pcAux.rdoList.back());
        }
 
        row++;
        rdo = pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8];
        if (rdo) {
            pcAux.rdoList.push_back(rdo);
            rdoCounter++;
            ATH_MSG_DEBUG("rdo = " << pcAux.rdoList.back());
        }
 
        row++;
        rdo = pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8];
        if (rdo) {
            pcAux.rdoList.push_back(rdo);
            rdoCounter++;
            ATH_MSG_DEBUG("rdo = " << pcAux.rdoList.back());
        }
 
        row++;
        rdo = pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8];
        if (rdo) {
            pcAux.rdoList.push_back(rdo);
            rdoCounter++;
            ATH_MSG_DEBUG("rdo = " << pcAux.rdoList.back());
        }
 
        row++;
        pcAux.localPosition.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("local x = " << pcAux.localPosition.back());
 
        row++;
        pcAux.localPosition.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("local y = " << pcAux.localPosition.back());
 
        row++;
        pcAux.localCovariance.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("local covariance [0, 0] = " << pcAux.localCovariance.back());
 
        row++;
        pcAux.localCovariance.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("local covariance [1, 1] = " << pcAux.localCovariance.back());
 
        row++;
        pcAux.globalPosition.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("global x = " << pcAux.globalPosition.back());
 
        row++;
        pcAux.globalPosition.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("global y = " << pcAux.globalPosition.back());
 
        row++;
        pcAux.globalPosition.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("global z = " << pcAux.globalPosition.back());
 
        row++;
        pcAux.channelsInPhi.push_back(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]);
        ATH_MSG_DEBUG("channels in phi = " << pcAux.channelsInPhi.back());
 
        row++;
        pcAux.channelsInEta.push_back(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]);
        ATH_MSG_DEBUG("chanels in eta = " << pcAux.channelsInEta.back());
 
        row++;
        pcAux.widthInEta.push_back(std::bit_cast<double>(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]));
        ATH_MSG_DEBUG("width in eta = " << pcAux.widthInEta.back());
 
        row++;
        row++;
        row++;
        pcAux.totalToT.push_back(pixelClusters[row * EFTrackingTransient::MAX_NUM_CLUSTERS + i + 8]);
        ATH_MSG_DEBUG("total ToT = " << pcAux.totalToT.back());
 
        metadata->pcRdoIndex[i] = rdoCounter;
    }
 
    ATH_CHECK(m_xAODClusterMaker->makePixelClusterContainer(pcAux, metadata.get(), ctx));
    write_end = std::chrono::high_resolution_clock::now();
    write_latency = std::chrono::duration<float, std::micro>(write_end - write_start).count();
    ATH_MSG_INFO("xAOD::PixelClusterContainer made in " << write_latency << " us");
 
    return StatusCode::SUCCESS;
}

extraDeps_update_handler()

void AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::extraDeps_update_handler ( Gaudi::Details::PropertyBase & ExtraDeps )

protectedinherited

Add StoreName to extra input/output deps as needed.

use the logic of the VarHandleKey to parse the DataObjID keys supplied via the ExtraInputs and ExtraOuputs Properties to add the StoreName if it's not explicitly given

extraOutputDeps()

const DataObjIDColl & AthCommonReentrantAlgorithm< Gaudi::Algorithm >::extraOutputDeps ( ) const

overridevirtualinherited

Return the list of extra output dependencies.

This list is extended to include symlinks implied by inheritance relations.

Definition at line 94 of file AthCommonReentrantAlgorithm.cxx.

{
  // If we didn't find any symlinks to add, just return the collection
  // from the base class.  Otherwise, return the extended collection.
  if (!m_extendedExtraObjects.empty()) {
    return m_extendedExtraObjects;
  }
  return BaseAlg::extraOutputDeps();
}

filterPassed()

virtual bool AthCommonReentrantAlgorithm< Gaudi::Algorithm >::filterPassed ( const EventContext & ctx ) const

inlinevirtualinherited

Definition at line 96 of file AthCommonReentrantAlgorithm.h.

                                                           {
    return execState( ctx ).filterPassed();
  }

initialize()

StatusCode PixelClustering::initialize ( )

overridevirtual

Detect the OpenCL devices and prepare OpenCL context.

This should always be called by derived classes when running on the FPGA accelerator.

Reimplemented from IntegrationBase.

Definition at line 18 of file PixelClustering.cxx.

{
    ATH_CHECK(m_pixelRDOKey.initialize());
 
    ATH_CHECK(m_FPGADataFormatTool.retrieve());
    ATH_CHECK(m_xAODClusterMaker.retrieve());
 
    ATH_CHECK(IntegrationBase::precheck({m_xclbin, m_kernelName, m_inputTV, m_refTV}));
    ATH_CHECK(IntegrationBase::initialize());
    ATH_CHECK(IntegrationBase::loadProgram(m_xclbin));
 
    return StatusCode::SUCCESS;
}

inputHandles()

virtual std::vector< Gaudi::DataHandle * > AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::inputHandles ( ) const

overridevirtualinherited

Return this algorithm's input handles.

We override this to include handle instances from key arrays if they have not yet been declared. See comments on updateVHKA.

isClonable()

bool AthCommonReentrantAlgorithm< Gaudi::Algorithm >::isClonable ( ) const

overridevirtualinherited

Specify if the algorithm is clonable.

Reentrant algorithms are clonable.

Reimplemented in InDet::GNNSeedingTrackMaker, InDet::SCT_Clusterization, InDet::SiSPGNNTrackMaker, InDet::SiSPSeededTrackFinder, InDet::SiTrackerSpacePointFinder, ITkPixelCablingAlg, ITkStripCablingAlg, RoIBResultToxAOD, SCT_ByteStreamErrorsTestAlg, SCT_CablingCondAlgFromCoraCool, SCT_CablingCondAlgFromText, SCT_ConditionsParameterTestAlg, SCT_ConditionsSummaryTestAlg, SCT_ConfigurationConditionsTestAlg, SCT_FlaggedConditionTestAlg, SCT_LinkMaskingTestAlg, SCT_MajorityConditionsTestAlg, SCT_ModuleVetoTestAlg, SCT_MonitorConditionsTestAlg, SCT_PrepDataToxAOD, SCT_RawDataToxAOD, SCT_ReadCalibChipDataTestAlg, SCT_ReadCalibDataTestAlg, SCT_RODVetoTestAlg, SCT_SensorsTestAlg, SCT_SiliconConditionsTestAlg, SCT_StripVetoTestAlg, SCT_TdaqEnabledTestAlg, SCT_TestCablingAlg, SCTEventFlagWriter, SCTRawDataProvider, SCTSiLorentzAngleTestAlg, SCTSiPropertiesTestAlg, and Simulation::BeamEffectsAlg.

Definition at line 68 of file AthCommonReentrantAlgorithm.cxx.

{
  // Reentrant algorithms are clonable.
  return true;
}

loadProgram()

StatusCode IntegrationBase::loadProgram ( const std::string & xclbin )

inherited

Find the xclbin file and load it into the OpenCL program object.

Definition at line 115 of file IntegrationBase.cxx.

{
    // Open binary object in binary mode
    std::ifstream bin_file(xclbin, std::ios_base::binary);
    if (!bin_file)
    {
        ATH_MSG_ERROR("Couldn't find the xclbin file: " << xclbin);
        return StatusCode::FAILURE;
    }
    // Get the size of the binary file
    bin_file.seekg(0, bin_file.end);
    unsigned bin_size = bin_file.tellg();
    // Reset the reference point back to the beginning
    bin_file.seekg(0, bin_file.beg);
    // Create a new pointer for the binary buffer and get the set a pointer to the binary buffer
    std::vector<char> buf(bin_size);
    bin_file.read(buf.data(), bin_size);
 
    // Create binary object and program object
    cl_int err = 0;
    std::vector<cl_int> binaryStatus;
    cl::Program::Binaries bins{{buf.data(), bin_size}};
    m_program = cl::Program(m_context, {m_accelerator}, bins, &binaryStatus, &err);
 
    bin_file.close();
 
    if (err == CL_SUCCESS && binaryStatus.at(0) == CL_SUCCESS)
    {
        ATH_MSG_INFO("Successfully loaded xclbin file into " << m_accelerator.getInfo<CL_DEVICE_NAME>());
    }
    else
    {
        ATH_MSG_ERROR("Error loading xclbin file (" << xclbin << ") into " << m_accelerator.getInfo<CL_DEVICE_NAME>() <<". Error code: " << err);
        return StatusCode::FAILURE;
    }
 
    return StatusCode::SUCCESS;
}

msg()

MsgStream & AthCommonMsg< Gaudi::Algorithm >::msg ( ) const

inlineinherited

Definition at line 24 of file AthCommonMsg.h.

                                {
    return this->msgStream();
  }

msgLvl()

bool AthCommonMsg< Gaudi::Algorithm >::msgLvl ( const MSG::Level lvl ) const

inlineinherited

Definition at line 30 of file AthCommonMsg.h.

                                               {
    return this->msgLevel(lvl);
  }

outputHandles()

virtual std::vector< Gaudi::DataHandle * > AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::outputHandles ( ) const

overridevirtualinherited

Return this algorithm's output handles.

We override this to include handle instances from key arrays if they have not yet been declared. See comments on updateVHKA.

precheck()

StatusCode IntegrationBase::precheck ( const std::vector< Gaudi::Property< std::string > > & inputs ) const

inherited

Check if the the desired Gaudi properties are set.

Definition at line 154 of file IntegrationBase.cxx.

{
    for(const auto &item : inputs)
    {
        if(item.empty())
        {
            ATH_MSG_FATAL(item.documentation()<<" is empty. Please set it to a valid value");
            return StatusCode::FAILURE;
        }
    }
 
    // Always check if bdf is set
    if (m_deviceBDF.empty())
    {
        ATH_MSG_WARNING("Device BDF is not set. Using the first found accelerator card. Set property 'bdfID' to specify the BDF of the device.");
    }
 
    return StatusCode::SUCCESS;
}

renounce()

std::enable_if_t< std::is_void_v< std::result_of_t< decltype(&T::renounce)(T)> > &&!std::is_base_of_v< SG::VarHandleKeyArray, T > &&std::is_base_of_v< Gaudi::DataHandle, T >, void > AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::renounce ( T & h )

inlineprotectedinherited

Definition at line 380 of file AthCommonDataStore.h.

  {
    h.renounce();
    PBASE::renounce (h);
  }

renounceArray()

void AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::renounceArray ( SG::VarHandleKeyArray & handlesArray )

inlineprotectedinherited

remove all handles from I/O resolution

Definition at line 364 of file AthCommonDataStore.h.

                                                          {
    handlesArray.renounce();
  }

setFilterPassed()

virtual void AthCommonReentrantAlgorithm< Gaudi::Algorithm >::setFilterPassed ( bool state, const EventContext & ctx ) const

inlinevirtualinherited

Definition at line 100 of file AthCommonReentrantAlgorithm.h.

                                                                            {
    execState( ctx ).setFilterPassed( state );
  }

sysExecute()

StatusCode AthCommonReentrantAlgorithm< Gaudi::Algorithm >::sysExecute ( const EventContext & ctx )

overridevirtualinherited

Execute an algorithm.

We override this in order to work around an issue with the Algorithm base class storing the event context in a member variable that can cause crashes in MT jobs.

Definition at line 85 of file AthCommonReentrantAlgorithm.cxx.

{
  return BaseAlg::sysExecute (ctx);
}

sysInitialize()

StatusCode AthCommonReentrantAlgorithm< Gaudi::Algorithm >::sysInitialize ( )

overridevirtualinherited

Override sysInitialize.

Override sysInitialize from the base class.

Loop through all output handles, and if they're WriteCondHandles, automatically register them and this Algorithm with the CondSvc

Scan through all outputHandles, and if they're WriteCondHandles, register them with the CondSvc

Reimplemented from AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >.

Reimplemented in HypoBase, and InputMakerBase.

Definition at line 61 of file AthCommonReentrantAlgorithm.cxx.

                                                               {
  StatusCode sc=AthCommonDataStore<AthCommonMsg<BaseAlg>>::sysInitialize();
 
  if (sc.isFailure()) {
    return sc;
  }
  
  ServiceHandle<ICondSvc> cs("CondSvc",name());
  for (auto h : outputHandles()) {
    if (h->isCondition() && h->mode() == Gaudi::DataHandle::Writer) {
      // do this inside the loop so we don't create the CondSvc until needed
      if ( cs.retrieve().isFailure() ) {
        ATH_MSG_WARNING("no CondSvc found: won't autoreg WriteCondHandles");
        return StatusCode::SUCCESS;
      }      
      if (cs->regHandle(this,*h).isFailure()) {
        sc = StatusCode::FAILURE;
        ATH_MSG_ERROR("unable to register WriteCondHandle " << h->fullKey()
                      << " with CondSvc");
      }
    }
  }
  return sc;  
}

sysStart()

virtual StatusCode AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::sysStart ( )

overridevirtualinherited

Handle START transition.

We override this in order to make sure that conditions handle keys can cache a pointer to the conditions container.

updateVHKA()

void AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::updateVHKA ( Gaudi::Details::PropertyBase & )

inlineinherited

Definition at line 308 of file AthCommonDataStore.h.

                                               {
    // debug() << "updateVHKA for property " << p.name() << " " << p.toString() 
    //         << "  size: " << m_vhka.size() << endmsg;
    for (auto &a : m_vhka) {
      std::vector<SG::VarHandleKey*> keys = a->keys();
      for (auto k : keys) {
        k->setOwner(this);
      }
    }
  }

Member Data Documentation

m_accelerator

cl::Device IntegrationBase::m_accelerator

protectedinherited

Device object for the accelerator card.

Definition at line 66 of file IntegrationBase.h.

m_context

cl::Context IntegrationBase::m_context

protectedinherited

Context object for the application.

Definition at line 67 of file IntegrationBase.h.

m_detStore

StoreGateSvc_t AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_detStore

privateinherited

Pointer to StoreGate (detector store by default)

Definition at line 393 of file AthCommonDataStore.h.

m_deviceBDF

Gaudi::Property<std::string> IntegrationBase::m_deviceBDF {this, "bdfID", "", "BDF ID of the accelerator card"}

protectedinherited

BDF ID of the accelerator card.

Definition at line 69 of file IntegrationBase.h.

{this, "bdfID", "", "BDF ID of the accelerator card"}; 

m_doEmulation

Gaudi::Property<bool> IntegrationBase::m_doEmulation {this, "doEmulation", false, "If software or hardware emulation is being used for debugging"}

protectedinherited

Definition at line 70 of file IntegrationBase.h.

{this, "doEmulation", false, "If software or hardware emulation is being used for debugging"}; 

m_evtStore

StoreGateSvc_t AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_evtStore

privateinherited

Pointer to StoreGate (event store by default)

Definition at line 390 of file AthCommonDataStore.h.

m_extendedExtraObjects

DataObjIDColl AthCommonReentrantAlgorithm< Gaudi::Algorithm >::m_extendedExtraObjects

privateinherited

Extra output dependency collection, extended by AthAlgorithmDHUpdate to add symlinks.

Empty if no symlinks were found.

Definition at line 114 of file AthCommonReentrantAlgorithm.h.

m_FPGADataFormatTool

ToolHandle<FPGADataFormatTool> PixelClustering::m_FPGADataFormatTool {this, "FPGADataFormatTool", "FPGADataFormatTool", "Tool to convert RDOs into FPGA data format"}

private

Definition at line 39 of file PixelClustering.h.

{this, "FPGADataFormatTool", "FPGADataFormatTool", "Tool to convert RDOs into FPGA data format"};

m_inputTV

Gaudi::Property<std::string> PixelClustering::m_inputTV {this, "InputTV", "", "Input TestVector"}

private

Input TestVector.

Definition at line 34 of file PixelClustering.h.

{this, "InputTV", "", "Input TestVector"};   

m_kernelName

Gaudi::Property<std::string> PixelClustering::m_kernelName {this, "KernelName", "", "Kernel name"}

private

Kernel name.

Definition at line 33 of file PixelClustering.h.

{this, "KernelName", "", "Kernel name"};  

m_pixelRDOKey

SG::ReadHandleKey<PixelRDO_Container> PixelClustering::m_pixelRDOKey { this, "PixelRDO", "ITkPixelRDOs" }

private

Definition at line 37 of file PixelClustering.h.

{ this, "PixelRDO", "ITkPixelRDOs" };

m_program

cl::Program IntegrationBase::m_program

protectedinherited

Program object containing the kernel.

Definition at line 68 of file IntegrationBase.h.

m_refTV

Gaudi::Property<std::string> PixelClustering::m_refTV {this, "RefTV", "", "Reference TestVector"}

private

Reference TestVector.

Definition at line 35 of file PixelClustering.h.

{this, "RefTV", "", "Reference TestVector"};   

m_varHandleArraysDeclared

bool AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_varHandleArraysDeclared

privateinherited

Definition at line 399 of file AthCommonDataStore.h.

m_vhka

std::vector<SG::VarHandleKeyArray*> AthCommonDataStore< AthCommonMsg< Gaudi::Algorithm > >::m_vhka

privateinherited

Definition at line 398 of file AthCommonDataStore.h.

m_xAODClusterMaker

ToolHandle<xAODClusterMaker> PixelClustering::m_xAODClusterMaker {this, "xAODClusterMaker", "xAODClusterMaker", "Tool for creating xAOD containers"}

private

Definition at line 40 of file PixelClustering.h.

{this, "xAODClusterMaker", "xAODClusterMaker", "Tool for creating xAOD containers"};

m_xclbin

Gaudi::Property<std::string> PixelClustering::m_xclbin {this, "xclbin", "", "xclbin path and name"}

private

Path and name of the xclbin file.

Definition at line 32 of file PixelClustering.h.

{this, "xclbin", "", "xclbin path and name"}; 

---

The documentation for this class was generated from the following files:

• PixelClustering.h
• PixelClustering.cxx