#include <DefaultMerger.h>

Inheritance diagram for H5Utils::DefaultMerger:

Collaboration diagram for H5Utils::DefaultMerger:

Public Member Functions
	DefaultMerger (hsize_t mergeAxis=0, int chunkSize=-1, bool requireSameFormat=true, std::size_t bufferSize=-1, bool bufferInRows=false)
	Create the merger. More...

	~DefaultMerger ()

void	merge (H5::Group &target, const H5::Group &source) override
	Merge a source group into a target group. More...

void	merge (H5::DataSet &target, const H5::DataSet &source) override
	Merge a source dataset into a target dataset. More...

H5::DataSet	createFrom (H5::H5Location &targetLocation, const H5::DataSet &source) override
	Make a new dataset from information in a source dataset. More...

virtual void	merge (H5::H5File &target, const H5::H5File &source)
	Merge a source file into a target file. More...

virtual void	merge (H5::Group &target, const H5::Group &source)=0
	Merge a source group into a target group. More...

virtual void	merge (H5::DataSet &target, const H5::DataSet &source)=0
	Merge a source dataset into a target dataset. More...

virtual H5::Group	createFrom (H5::H5Location &targetLocation, const H5::Group &source)
	Make a new group from information in a source group. More...

virtual H5::DataSet	createFrom (H5::H5Location &targetLocation, const H5::DataSet &source)=0
	Make a new dataset from information in a source dataset. More...

virtual void	merge (H5::H5File &target, const H5::H5File &source)
	Merge a source file into a target file. More...

virtual H5::Group	createFrom (H5::H5Location &targetLocation, const H5::Group &source)
	Make a new group from information in a source group. More...

Protected Attributes
hsize_t	m_mergeAxis
	The axis to merge along. More...

int	m_chunkSize
	The chunk size to apply. More...

bool	m_requireSameFormat
	Whether to require the same group structure. More...

std::size_t	m_bufferSize
	The size of the buffer. More...

bool	m_measureBufferInRows
	Whether to measure the buffer in bytes or rows. More...

Detailed Description

Definition at line 21 of file DefaultMerger.h.

Constructor & Destructor Documentation

◆ DefaultMerger()

H5Utils::DefaultMerger::DefaultMerger	(	hsize_t	mergeAxis = `0`,
		int	chunkSize = `-1`,
		bool	requireSameFormat = `true`,
		std::size_t	bufferSize = `-1`,
		bool	bufferInRows = `false`
	)

Create the merger.

Parameters

mergeAxis	The axis to merge along
chunkSize	The chunk size to apply. If negative then the value found in the input datasets will be used.
requireSameFormat	Require all input files to have the same groups and datasets.
bufferSize	The maximum size of the buffer to use while merging datasets
bufferInRows	Whether the buffer size is specified in rows or bytes

Definition at line 12 of file DefaultMerger.cxx.

                          :
     m_mergeAxis(mergeAxis),
     m_chunkSize(chunkSize),
     m_requireSameFormat(requireSameFormat),
     m_bufferSize(bufferSize),
     m_measureBufferInRows(bufferInRows) {}

◆ ~DefaultMerger()

H5Utils::DefaultMerger::~DefaultMerger ( )

Definition at line 24 of file DefaultMerger.cxx.

24 {}

Member Function Documentation

◆ createFrom() [1/4]

H5::DataSet H5Utils::DefaultMerger::createFrom	(	H5::H5Location &	targetLocation,
		const H5::DataSet &	source
	)

overridevirtual

Make a new dataset from information in a source dataset.

Parameters

targetLocation	Where the new dataset will be created
source	The dataset to use to create the new dataset

Implements H5Utils::IH5Merger.

Definition at line 113 of file DefaultMerger.cxx.

   {
     return createDataSet(targetLocation, source, m_mergeAxis, m_chunkSize);
   }

◆ createFrom() [2/4]

virtual H5::DataSet H5Utils::IH5Merger::createFrom

Make a new dataset from information in a source dataset.

Parameters

targetLocation	Where the new dataset will be created
source	The dataset to use to create the new dataset

◆ createFrom() [3/4]

H5::Group H5Utils::IH5Merger::createFrom

Make a new group from information in a source group.

Parameters

targetLocation	Where the new group will be created
source	The group to use to create the new group

The default implementation provided here just copies the source group's name then uses the merge function.

Definition at line 61 of file IH5Merger.cxx.

   {
     H5::Group newGroup = targetLocation.createGroup(source.getObjName() );
     merge(newGroup, source);
     return newGroup;
   }

◆ createFrom() [4/4]

H5::Group H5Utils::IH5Merger::createFrom	(	H5::H5Location &	targetLocation,
		const H5::Group &	source
	)

virtualinherited

Make a new group from information in a source group.

Parameters

targetLocation	Where the new group will be created
source	The group to use to create the new group

The default implementation provided here just copies the source group's name then uses the merge function.

Definition at line 18 of file IH5Merger.cxx.

   {
     H5::Group newGroup = targetLocation.createGroup(source.getObjName() );
     merge(newGroup, source);
     return newGroup;
   }

◆ merge() [1/6]

void H5Utils::DefaultMerger::merge	(	H5::DataSet &	target,
		const H5::DataSet &	source
	)

overridevirtual

Merge a source dataset into a target dataset.

Parameters

target	The dataset to merge into
source	The dataset to merge from

Implements H5Utils::IH5Merger.

Definition at line 98 of file DefaultMerger.cxx.

   {
     std::size_t bufferSize = m_bufferSize;
     if (m_measureBufferInRows) {
       // Need to calculate the actual buffer size
       std::size_t rowSize = getRowSize(source, m_mergeAxis);
       if (std::size_t(-1) / m_bufferSize < rowSize)
         std::overflow_error("Requested buffer would overflow the register!");
       bufferSize = rowSize * m_bufferSize;
     }
     mergeDatasets(target, source, m_mergeAxis, bufferSize);
   }

◆ merge() [2/6]

virtual void H5Utils::IH5Merger::merge

Merge a source dataset into a target dataset.

Parameters

target	The dataset to merge into
source	The dataset to merge from

◆ merge() [3/6]

void H5Utils::DefaultMerger::merge	(	H5::Group &	target,
		const H5::Group &	source
	)

overridevirtual

Merge a source group into a target group.

Parameters

target	The group to merge into
source	The group to merge from

Implements H5Utils::IH5Merger.

Definition at line 26 of file DefaultMerger.cxx.

   {
     // Check if this group was empty before we started
     bool isEmpty = target.getNumObjs() == 0;
  
     // Iterate through each child of the source group
     for (hsize_t ii = 0; ii < source.getNumObjs(); ++ii) {
       H5G_obj_t childType = source.getObjTypeByIdx(ii);
       std::string childName = source.getObjnameByIdx(ii);
       // Find the correct index in the target
       hsize_t targetIdx = 0;
       for (; targetIdx < target.getNumObjs(); ++targetIdx) 
         if (target.getObjnameByIdx(targetIdx) == childName)
           break;
       bool found = targetIdx != target.getNumObjs();
       if (found) {
         // Make sure these are the same type!
         if (target.getObjTypeByIdx(targetIdx) != childType)
           throw std::invalid_argument(
               "Both target and source contain " + childName +
               " but they have different types!");
       }
       else if (m_requireSameFormat && !isEmpty) {
         throw std::invalid_argument(
             "Target and source have different formats!");
       }
       switch (childType) {
         case H5G_GROUP:
           {
             H5::Group sg = source.openGroup(childName);
             H5::Group tg = found ? 
               target.openGroup(childName) : 
               createFrom(target, sg);
             try {
               merge(tg, sg);
             }
             catch (...) {
               std::cerr << "Encountered an error merging child " << childName << std::endl;
               throw;
             }
           }
           break;
         case H5G_DATASET:
           {
             H5::DataSet sd = source.openDataSet(childName);
             if (sd.getSpace().getSimpleExtentNdims() == 0) {
               std::cerr << "WARNING: skipping scalar '"
                         << childName << "'" << std::endl;
               break;
             }
             H5::DataSet td = found ? 
               target.openDataSet(childName) : 
               createFrom(target, sd);
             try {
               merge(td, sd);
             }
             catch (...) {
               std::cerr << "Encountered an error merging child " << childName << std::endl;
               throw;
             }
           }
           break;
         default:
           break; 
       }
     } //> end loop over children
     // TODO - this did no check to see if target contained something source
     // didn't, this is probably fine though.
   } //> end function merge(group)

◆ merge() [4/6]

virtual void H5Utils::IH5Merger::merge

Merge a source group into a target group.

Parameters

target	The group to merge into
source	The group to merge from

◆ merge() [5/6]

void H5Utils::IH5Merger::merge	(	H5::H5File &	target,
		const H5::H5File &	source
	)

virtualinherited

Merge a source file into a target file.

Parameters

target	The file to merge into
source	The file to merge from

The default implementation provided here just forwards this to the group function.

Definition at line 11 of file IH5Merger.cxx.

   {
     merge(
         static_cast<H5::Group&>(target),
         static_cast<const H5::Group&>(source) );
   }

◆ merge() [6/6]

void H5Utils::IH5Merger::merge

Merge a source file into a target file.

Parameters

target	The file to merge into
source	The file to merge from

The default implementation provided here just forwards this to the group function.

Definition at line 37 of file IH5Merger.cxx.

   {
     merge(
         static_cast<H5::Group&>(target),
         static_cast<const H5::Group&>(source) );
   }

Member Data Documentation

◆ m_bufferSize

std::size_t H5Utils::DefaultMerger::m_bufferSize

protected

The size of the buffer.

Definition at line 79 of file DefaultMerger.h.

◆ m_chunkSize

int H5Utils::DefaultMerger::m_chunkSize

protected

The chunk size to apply.

Definition at line 75 of file DefaultMerger.h.

◆ m_measureBufferInRows

bool H5Utils::DefaultMerger::m_measureBufferInRows

protected

Whether to measure the buffer in bytes or rows.

Definition at line 81 of file DefaultMerger.h.

◆ m_mergeAxis

hsize_t H5Utils::DefaultMerger::m_mergeAxis

protected

The axis to merge along.

Definition at line 73 of file DefaultMerger.h.

◆ m_requireSameFormat

bool H5Utils::DefaultMerger::m_requireSameFormat

protected

Whether to require the same group structure.

Definition at line 77 of file DefaultMerger.h.

The documentation for this class was generated from the following files:

Public Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ DefaultMerger()

◆ ~DefaultMerger()

Member Function Documentation

◆ createFrom() [1/4]

◆ createFrom() [2/4]

◆ createFrom() [3/4]

◆ createFrom() [4/4]

◆ merge() [1/6]

◆ merge() [2/6]

◆ merge() [3/6]

◆ merge() [4/6]

◆ merge() [5/6]

◆ merge() [6/6]

Member Data Documentation

◆ m_bufferSize

◆ m_chunkSize

◆ m_measureBufferInRows

◆ m_mergeAxis

◆ m_requireSameFormat