columnar::ColumnVectorHeader Class Reference

the header information for the entire columnar data vector More...

#include <ColumnVectorWrapper.h>

Collaboration diagram for columnar::ColumnVectorHeader:

Public Member Functions
	ColumnVectorHeader ()
	standard contructor
std::size_t	addColumn (const ColumnInfo &columnInfo)
	add a column for the given ColumnInfo, returning its index
void	setOffsetColumn (std::size_t columnIndex, std::size_t offsetIndex)
	set the index of the offset column for the given column
std::size_t	numColumns () const noexcept
	the number of columns in the columnar data vector
const ColumnVectorElementHeader &	getColumn (std::size_t index) const
	get the column for the given index
std::size_t	getColumnIndex (const std::string &name) const noexcept
	get the column index for the given name, or nullIndex if not found
std::unordered_map< std::string, ColumnInfo >	getAllColumnInfo () const
	get all columns as a map of ColumnInfo for use with IColumnData::connect
void	checkSelf () const
	check the self-consistency of the header
void	checkData (std::span< const void *const > data) const
	do a basic check of the data vector

Static Public Attributes
static constexpr std::size_t	nullIndex = 0u
	the index used for an invalid index (always has to be 0)
static constexpr std::size_t	sizeIndex = 1u
	the index used for the column size column
static constexpr std::size_t	unsetIndex = static_cast<std::size_t>(-1)
	the number used for an unset but non-null index
static constexpr std::size_t	numFixedColumns = 2u
	the number of fix elements in the columnar data vector

Private Attributes
std::vector< ColumnVectorElementHeader >	m_elements
	the elements in the columnar data vector
std::unordered_map< std::string, std::size_t >	m_nameToIndex
	map from column name to index for deduplication

Detailed Description

the header information for the entire columnar data vector

At its core this is just a vector of ColumnVectorElementHeader, with some extra helper functions. Note that this is completely independent of the tool used, and can indeed be used with multiple tools.

Definition at line 124 of file ColumnVectorWrapper.h.

Constructor & Destructor Documentation

ColumnVectorHeader()

columnar::ColumnVectorHeader::ColumnVectorHeader

(

)

standard contructor

Definition at line 29 of file ColumnVectorWrapper.cxx.

  {
    m_elements.resize (numFixedColumns);
 
    // we always reserve the first column as having a null value
    m_elements.at(nullIndex).debugName = "<null>";
    m_elements.at(nullIndex).isOptional = true;
 
    // we always reserve the second column as the size column
    m_elements.at(sizeIndex).debugName = "<size>";
    m_elements.at(sizeIndex).type = &typeid(std::size_t);
    m_elements.at(sizeIndex).isOptional = false;
    m_elements.at(sizeIndex).readOnly = true;
  }

Member Function Documentation

addColumn()

std::size_t columnar::ColumnVectorHeader::addColumn ( const ColumnInfo & columnInfo )

nodiscard

add a column for the given ColumnInfo, returning its index

Definition at line 47 of file ColumnVectorWrapper.cxx.

  {
    // Check for existing column with the same name (deduplication for multi-tool support)
    auto iter = m_nameToIndex.find(columnInfo.name);
    if (iter != m_nameToIndex.end())
    {
      auto& existingHeader = m_elements.at(iter->second);
 
      // Variant link key columns with different targets get auto-renamed
      if (!columnInfo.keyColumnForVariantLink.empty() && !existingHeader.keyColumnForVariantLink.empty() && existingHeader.variantLinkTargetNames != columnInfo.variantLinkTargetNames)
      {
        std::string uniqueName;
        for (unsigned suffix = 1; ; ++suffix)
        {
          uniqueName = columnInfo.name + "." + std::to_string(suffix);
          if (m_nameToIndex.find(uniqueName) == m_nameToIndex.end())
            break;
        }
        ColumnInfo renamedInfo = columnInfo;
        renamedInfo.name = std::move(uniqueName);
        return addColumn(renamedInfo);
      }
 
      // Check that relevant fields match
      if (existingHeader.type != columnInfo.type)
        throw std::runtime_error("column " + columnInfo.name + " type mismatch");
      if (existingHeader.offsetName != columnInfo.offsetName)
        throw std::runtime_error("column " + columnInfo.name + " offset name mismatch: " + existingHeader.offsetName + " vs " + columnInfo.offsetName);
      if (existingHeader.isOffset != columnInfo.isOffset)
        throw std::runtime_error("column " + columnInfo.name + " isOffset mismatch");
      if (existingHeader.fixedDimensions != columnInfo.fixedDimensions)
        throw std::runtime_error("column " + columnInfo.name + " fixed dimensions mismatch");
      if (existingHeader.soleLinkTargetName != columnInfo.soleLinkTargetName)
        throw std::runtime_error("column " + columnInfo.name + " sole link target name mismatch");
      if (existingHeader.soleLinkTargetClid != columnInfo.soleLinkTargetClid)
        throw std::runtime_error("column " + columnInfo.name + " sole link target clid mismatch");
      if (existingHeader.isVariantLink != columnInfo.isVariantLink)
        throw std::runtime_error("column " + columnInfo.name + " isVariantLink mismatch");
      if (existingHeader.variantLinkTargetNames != columnInfo.variantLinkTargetNames)
        throw std::runtime_error("column " + columnInfo.name + " variant link target names mismatch");
      if (existingHeader.keyColumnForVariantLink != columnInfo.keyColumnForVariantLink)
        throw std::runtime_error("column " + columnInfo.name + " key column for variant link mismatch");
 
      // Handle access mode conflicts
      if (columnInfo.accessMode == ColumnAccessMode::output && existingHeader.readOnly)
        throw std::runtime_error("column " + columnInfo.name + " already registered as input, cannot register as output");
      // Promote to read-write if new column needs write access (update mode)
      if (columnInfo.accessMode == ColumnAccessMode::update)
        existingHeader.readOnly = false;
      // If existing is already non-readOnly (output or update), keep it that way
      return iter->second;
    }
 
    m_elements.emplace_back();
    m_elements.at(sizeIndex).arraySize = m_elements.size();
 
    auto& header = m_elements.back();
    header.debugName = columnInfo.name;
    header.type = columnInfo.type;
    header.accessMode = columnInfo.accessMode;
    header.offsetName = columnInfo.offsetName;
    header.soleLinkTargetName = columnInfo.soleLinkTargetName;
    header.soleLinkTargetClid = columnInfo.soleLinkTargetClid;
    header.isVariantLink = columnInfo.isVariantLink;
    header.variantLinkTargetNames = columnInfo.variantLinkTargetNames;
    header.keyColumnForVariantLink = columnInfo.keyColumnForVariantLink;
    header.fixedDimensions = columnInfo.fixedDimensions;
 
    switch (columnInfo.accessMode)
    {
    case ColumnAccessMode::input:
      break;
    case ColumnAccessMode::output:
      header.readOnly = false;
      break;
    case ColumnAccessMode::update:
      header.readOnly = false;
      break;
    }
    for (unsigned dim : columnInfo.fixedDimensions)
      header.arraySize *= dim;
    header.isOffset = columnInfo.isOffset;
    if (!columnInfo.isOptional)
      header.isOptional = false;
    if (!columnInfo.offsetName.empty())
      header.offsetIndex = unsetIndex;
 
    const std::size_t newIndex = m_elements.size() - 1;
    m_nameToIndex.emplace(columnInfo.name, newIndex);
    return newIndex;
  }

checkData()

void columnar::ColumnVectorHeader::checkData

(

std::span< const void *const >

data

)

const

do a basic check of the data vector

Definition at line 199 of file ColumnVectorWrapper.cxx.

  {
    if (data.size() != m_elements.size())
      throw std::logic_error ("data vector has wrong size, expected " + std::to_string(m_elements.size()) + " but got " + std::to_string(data.size()));
    if (data[nullIndex] != nullptr)
      throw std::logic_error ("null column is not set to a nullptr value");
    const auto* sizeVector = static_cast<const std::size_t*>(data[sizeIndex]);
    for (std::size_t columnIndex = 0u; columnIndex != m_elements.size(); ++ columnIndex)
    {
      const auto& column = m_elements[columnIndex];
      if (data[columnIndex] == nullptr)
      {
        if (column.isOptional)
          continue;
        if (column.offsetIndex != nullIndex && data[column.offsetIndex] == nullptr)
          continue;
        throw std::logic_error ("column " + column.debugName + " was not set");
      }
      if (column.isOffset)
      {
        const auto size = sizeVector[columnIndex];
        if (size < 1)
          throw std::runtime_error ("offset column " + column.debugName + " has size " + std::to_string(size) + ", but needs at least 1 element");
        auto *offsets = static_cast<const ColumnarOffsetType*>(data[columnIndex]);
        if (offsets[0] != 0)
          throw std::runtime_error ("offset column doesn't start with 0: " + column.debugName);
        for (std::size_t i = 1u; i != size; ++ i)
        {
          if (offsets[i] < offsets[i-1])
            throw std::runtime_error ("offset column " + column.debugName + " is not monotonically increasing at index " + std::to_string(i) + ": " + std::to_string(offsets[i-1]) + " > " + std::to_string(offsets[i]));
        }
      }
    }
    for (std::size_t columnIndex = 0u; columnIndex != m_elements.size(); ++ columnIndex)
    {
      if (data[columnIndex] == nullptr)
        continue;
      const auto& column = m_elements[columnIndex];
      std::size_t expectedSize = 1u;
      if (column.offsetIndex != ColumnVectorHeader::nullIndex)
      {
        if (data[column.offsetIndex] == nullptr)
          throw std::runtime_error ("column " + column.debugName + " uses offset column " + m_elements[column.offsetIndex].debugName + " that is not set");
        const auto offsetIndex = column.offsetIndex;
        auto *offsetsPtr = static_cast<const ColumnarOffsetType*>(data[offsetIndex]);
        expectedSize = offsetsPtr[sizeVector[offsetIndex]-1];
      }
      expectedSize *= column.arraySize;
 
      if (column.isOffset)
        expectedSize += 1u;
 
      if (sizeVector[columnIndex] != expectedSize)
        throw std::runtime_error ("column size doesn't match expected size: " + column.debugName + ", found " + std::to_string (sizeVector[columnIndex]) + " vs exptected=" + std::to_string (expectedSize) + " isOffset=" + std::to_string (column.isOffset));
    }
  }

checkSelf()

void columnar::ColumnVectorHeader::checkSelf

(

)

const

check the self-consistency of the header

Definition at line 160 of file ColumnVectorWrapper.cxx.

  {
    if (m_elements.size() < numFixedColumns)
      throw std::logic_error ("header has too few m_elements, expected at least " + std::to_string(numFixedColumns) + " but got " + std::to_string(m_elements.size()));
    if (m_elements[sizeIndex].arraySize != m_elements.size())
      throw std::logic_error ("size column has wrong array size, expected " + std::to_string(m_elements.size()) + " but got " + std::to_string(m_elements[sizeIndex].arraySize));
 
    // skipping the first column, which is always the null column
    // and behaves in a funny manner
    for (std::size_t columnIndex = 1u; columnIndex != m_elements.size(); ++ columnIndex)
    {
      const auto& column = m_elements[columnIndex];
      if (column.debugName.empty())
        throw std::logic_error ("column " + std::to_string(columnIndex) + " has no name");
      if (column.type == nullptr)
        throw std::logic_error ("column " + column.debugName + " has no type");
      if (column.isOffset && *column.type != typeid(ColumnarOffsetType))
        throw std::logic_error ("column " + column.debugName + " is an offset column that is of type " + boost::core::demangle(column.type->name()) + " instead of ColumnarOffsetType");
    }
 
    for (std::size_t columnIndex = 1u; columnIndex != m_elements.size(); ++ columnIndex)
    {
      const auto& column = m_elements[columnIndex];
      if (column.offsetIndex != nullIndex)
      {
        if (column.offsetIndex == unsetIndex)
          throw std::logic_error ("column " + column.debugName + " has an offset index that was never set");
        if (column.offsetIndex >= m_elements.size())
          throw std::logic_error ("column " + column.debugName + " has invalid offset index " + std::to_string(column.offsetIndex) + " (max is " + std::to_string(m_elements.size()-1) + ")");
        const auto& offsetElement = m_elements[column.offsetIndex];
        if (!offsetElement.isOffset)
          throw std::logic_error ("column " + column.debugName + " has offset index that is not marked as offset");
      }
    }
  }

getAllColumnInfo()

std::unordered_map< std::string, ColumnInfo > columnar::ColumnVectorHeader::getAllColumnInfo ( ) const

nodiscard

get all columns as a map of ColumnInfo for use with IColumnData::connect

Definition at line 331 of file ColumnVectorWrapper.cxx.

  {
    std::unordered_map<std::string, ColumnInfo> result;
    // Skip the fixed columns (null and size)
    for (std::size_t i = numFixedColumns; i < m_elements.size(); ++i)
    {
      const auto& header = m_elements[i];
      ColumnInfo info;
      info.name = header.debugName;
      info.index = static_cast<unsigned>(i);
      info.type = header.type;
      info.accessMode = header.accessMode;
      info.offsetName = header.offsetName;
      info.fixedDimensions = header.fixedDimensions;
      info.isOffset = header.isOffset;
      info.isOptional = header.isOptional;
      info.soleLinkTargetName = header.soleLinkTargetName;
      info.soleLinkTargetClid = header.soleLinkTargetClid;
      info.isVariantLink = header.isVariantLink;
      info.variantLinkTargetNames = header.variantLinkTargetNames;
      info.keyColumnForVariantLink = header.keyColumnForVariantLink;
      result.emplace(info.name, std::move(info));
    }
    return result;
  }

getColumn()

const ColumnVectorElementHeader & columnar::ColumnVectorHeader::getColumn ( std::size_t index ) const

inlinenodiscard

get the column for the given index

Definition at line 159 of file ColumnVectorWrapper.h.

                                      {
      return m_elements.at (index); }

getColumnIndex()

std::size_t columnar::ColumnVectorHeader::getColumnIndex ( const std::string & name ) const

inlinenodiscardnoexcept

get the column index for the given name, or nullIndex if not found

Definition at line 164 of file ColumnVectorWrapper.h.

                                                          {
      auto iter = m_nameToIndex.find(name);
      return (iter != m_nameToIndex.end()) ? iter->second : nullIndex; }

numColumns()

std::size_t columnar::ColumnVectorHeader::numColumns ( ) const

inlinenodiscardnoexcept

the number of columns in the columnar data vector

Definition at line 154 of file ColumnVectorWrapper.h.

                                                         {
      return m_elements.size(); }

setOffsetColumn()

void columnar::ColumnVectorHeader::setOffsetColumn	(	std::size_t	columnIndex,
		std::size_t	offsetIndex )

set the index of the offset column for the given column

Definition at line 142 of file ColumnVectorWrapper.cxx.

  {
    if (columnIndex >= m_elements.size())
      throw std::logic_error ("column index out of range");
    if (offsetIndex >= m_elements.size())
      throw std::logic_error ("offset index out of range");
    auto& column = m_elements.at(columnIndex);
    auto& offset = m_elements.at(offsetIndex);
    if (!offset.isOffset)
      throw std::runtime_error ("trying to set " + offset.debugName + " as offset column for " + column.debugName + ", but it is not marked as offset column");
    if (column.offsetIndex != unsetIndex && column.offsetIndex != offsetIndex)
      throw std::runtime_error ("trying to set " + offset.debugName + " as offset column for " + column.debugName + ", but it is already set to " + m_elements.at(column.offsetIndex).debugName);
    column.offsetIndex = offsetIndex;
  }

Member Data Documentation

m_elements

std::vector<ColumnVectorElementHeader> columnar::ColumnVectorHeader::m_elements

private

the elements in the columnar data vector

Private Members

Definition at line 184 of file ColumnVectorWrapper.h.

m_nameToIndex

std::unordered_map<std::string, std::size_t> columnar::ColumnVectorHeader::m_nameToIndex

private

map from column name to index for deduplication

Definition at line 187 of file ColumnVectorWrapper.h.

nullIndex

std::size_t columnar::ColumnVectorHeader::nullIndex = 0u

staticconstexpr

the index used for an invalid index (always has to be 0)

Public Members

Definition at line 131 of file ColumnVectorWrapper.h.

numFixedColumns

std::size_t columnar::ColumnVectorHeader::numFixedColumns = 2u

staticconstexpr

the number of fix elements in the columnar data vector

Definition at line 140 of file ColumnVectorWrapper.h.

sizeIndex

std::size_t columnar::ColumnVectorHeader::sizeIndex = 1u

staticconstexpr

the index used for the column size column

Definition at line 134 of file ColumnVectorWrapper.h.

unsetIndex

std::size_t columnar::ColumnVectorHeader::unsetIndex = static_cast<std::size_t>(-1)

staticconstexpr

the number used for an unset but non-null index

Definition at line 137 of file ColumnVectorWrapper.h.

---

The documentation for this class was generated from the following files:

• ColumnVectorWrapper.h
• ColumnVectorWrapper.cxx