ATLAS Offline Software
Loading...
Searching...
No Matches
columnar::ColumnVectorHeader Class Referencefinal

the header information for the entire columnar data vector More...

#include <ColumnVectorWrapper.h>

Collaboration diagram for columnar::ColumnVectorHeader:

Public Member Functions

 ColumnVectorHeader ()
 standard contructor
std::size_t addColumn (const ColumnInfo &columnInfo)
 add a column for the given ColumnInfo, returning its index
void setOffsetColumn (std::size_t columnIndex, std::size_t offsetIndex)
 set the index of the offset column for the given column
std::size_t numColumns () const noexcept
 the number of columns in the columnar data vector
const ColumnVectorElementHeader & getColumn (std::size_t index) const
 get the column for the given index
std::size_t getColumnIndex (const std::string &name) const noexcept
 get the column index for the given name, or nullIndex if not found
std::unordered_map< std::string, ColumnInfo > getAllColumnInfo () const
 get all columns as a map of ColumnInfo for use with IColumnData::connect
void checkSelf () const
 check the self-consistency of the header
void checkData (std::span< const void *const > data) const
 do a basic check of the data vector

Static Public Attributes

static constexpr std::size_t nullIndex = 0u
 the index used for an invalid index (always has to be 0)
static constexpr std::size_t sizeIndex = 1u
 the index used for the column size column
static constexpr std::size_t unsetIndex = static_cast<std::size_t>(-1)
 the number used for an unset but non-null index
static constexpr std::size_t numFixedColumns = 2u
 the number of fix elements in the columnar data vector

Private Attributes

std::vector< ColumnVectorElementHeader > m_elements
 the elements in the columnar data vector
std::unordered_map< std::string, std::size_t > m_nameToIndex
 map from column name to index for deduplication

Detailed Description

the header information for the entire columnar data vector

At its core this is just a vector of ColumnVectorElementHeader, with some extra helper functions. Note that this is completely independent of the tool used, and can indeed be used with multiple tools.

Definition at line 124 of file ColumnVectorWrapper.h.

Constructor & Destructor Documentation

◆ ColumnVectorHeader()

columnar::ColumnVectorHeader::ColumnVectorHeader ( )

standard contructor

Definition at line 30 of file ColumnVectorWrapper.cxx.

32 {
34
35 // we always reserve the first column as having a null value
36 m_elements.at(nullIndex).debugName = "<null>";
37 m_elements.at(nullIndex).isOptional = true;
38
39 // we always reserve the second column as the size column
40 m_elements.at(sizeIndex).debugName = "<size>";
41 m_elements.at(sizeIndex).type = &typeid(std::size_t);
42 m_elements.at(sizeIndex).sizeOfType = sizeof(std::size_t);
44 m_elements.at(sizeIndex).isOptional = false;
45 m_elements.at(sizeIndex).readOnly = true;
46 }
static constexpr std::size_t nullIndex
the index used for an invalid index (always has to be 0)
static constexpr std::size_t numFixedColumns
the number of fix elements in the columnar data vector
std::vector< ColumnVectorElementHeader > m_elements
the elements in the columnar data vector
static constexpr std::size_t sizeIndex
the index used for the column size column
constexpr ColumnTypeCategory columnTypeCategoryFor() noexcept
the default ColumnTypeCategory for the given C++ type
Definition ColumnInfo.h:208

Member Function Documentation

◆ addColumn()

std::size_t columnar::ColumnVectorHeader::addColumn ( const ColumnInfo & columnInfo)
nodiscard

add a column for the given ColumnInfo, returning its index

Definition at line 50 of file ColumnVectorWrapper.cxx.

52 {
53 // Check for existing column with the same name (deduplication for multi-tool support)
54 auto iter = m_nameToIndex.find(columnInfo.name);
55 if (iter != m_nameToIndex.end())
56 {
57 auto& existingHeader = m_elements.at(iter->second);
58
59 // Check that relevant fields match
60 if (existingHeader.type != columnInfo.type)
61 throw std::runtime_error("column " + columnInfo.name + " type mismatch");
62 // for the "relaxed type matching" the strictest tool wins. note
63 // that size missmatches can't happen, due to the type check above.
64 existingHeader.typeCategory = mergeColumnTypeCategory (existingHeader.typeCategory, columnInfo.typeCategory);
65 if (existingHeader.offsetName != columnInfo.offsetName)
66 throw std::runtime_error("column " + columnInfo.name + " offset name mismatch: " + existingHeader.offsetName + " vs " + columnInfo.offsetName);
67 if (existingHeader.isOffset != columnInfo.isOffset)
68 throw std::runtime_error("column " + columnInfo.name + " isOffset mismatch");
69 if (existingHeader.fixedDimensions != columnInfo.fixedDimensions)
70 throw std::runtime_error("column " + columnInfo.name + " fixed dimensions mismatch");
71 if (existingHeader.isEventIndex != columnInfo.isEventIndex)
72 throw std::runtime_error("column " + columnInfo.name + " isEventIndex mismatch");
73
74 // Handle access mode conflicts
75 if (columnInfo.accessMode == ColumnAccessMode::output && existingHeader.readOnly)
76 throw std::runtime_error("column " + columnInfo.name + " already registered as input, cannot register as output");
77 // Promote to read-write if new column needs write access (update mode)
78 if (columnInfo.accessMode == ColumnAccessMode::update)
79 existingHeader.readOnly = false;
80 // If existing is already non-readOnly (output or update), keep it that way
81
82 // the column is required if any tool requires it
83 if (!columnInfo.isOptional)
84 existingHeader.isOptional = false;
85 return iter->second;
86 }
87
88 m_elements.emplace_back();
89 m_elements.at(sizeIndex).arraySize = m_elements.size();
90
91 auto& header = m_elements.back();
92 header.debugName = columnInfo.name;
93 header.type = columnInfo.type;
94 header.sizeOfType = columnInfo.sizeOfType;
95 header.typeCategory = columnInfo.typeCategory;
96 header.accessMode = columnInfo.accessMode;
97 header.offsetName = columnInfo.offsetName;
98 header.isEventIndex = columnInfo.isEventIndex;
99 header.fixedDimensions = columnInfo.fixedDimensions;
100
101 switch (columnInfo.accessMode)
102 {
104 break;
106 header.readOnly = false;
107 break;
109 header.readOnly = false;
110 break;
111 }
112 for (unsigned dim : columnInfo.fixedDimensions)
113 header.arraySize *= dim;
114 header.isOffset = columnInfo.isOffset;
115 if (!columnInfo.isOptional)
116 header.isOptional = false;
117 if (!columnInfo.offsetName.empty())
118 header.offsetIndex = unsetIndex;
119
120 const std::size_t newIndex = m_elements.size() - 1;
121 m_nameToIndex.emplace(columnInfo.name, newIndex);
122 return newIndex;
123 }
std::unordered_map< std::string, std::size_t > m_nameToIndex
map from column name to index for deduplication
static constexpr std::size_t unsetIndex
the number used for an unset but non-null index
constexpr ColumnTypeCategory mergeColumnTypeCategory(ColumnTypeCategory a, ColumnTypeCategory b) noexcept
merge the type categories of two users of the same column (with the same type), the strictest one win...
@ update
an updateable column
Definition ColumnInfo.h:28
@ output
an output column
Definition ColumnInfo.h:25
@ input
an input column
Definition ColumnInfo.h:22

◆ checkData()

void columnar::ColumnVectorHeader::checkData ( std::span< const void *const > data) const

do a basic check of the data vector

Definition at line 184 of file ColumnVectorWrapper.cxx.

186 {
187 if (data.size() != m_elements.size())
188 throw std::logic_error ("data vector has wrong size, expected " + std::to_string(m_elements.size()) + " but got " + std::to_string(data.size()));
189 if (data[nullIndex] != nullptr)
190 throw std::logic_error ("null column is not set to a nullptr value");
191 const auto* sizeVector = static_cast<const std::size_t*>(data[sizeIndex]);
192 for (std::size_t columnIndex = 0u; columnIndex != m_elements.size(); ++ columnIndex)
193 {
194 const auto& column = m_elements[columnIndex];
195 if (data[columnIndex] == nullptr)
196 {
197 if (column.isOptional)
198 continue;
199 if (column.offsetIndex != nullIndex && data[column.offsetIndex] == nullptr)
200 continue;
201 throw std::logic_error ("column " + column.debugName + " was not set");
202 }
203 if (column.isOffset)
204 {
205 const auto size = sizeVector[columnIndex];
206 if (size < 1)
207 throw std::runtime_error ("offset column " + column.debugName + " has size " + std::to_string(size) + ", but needs at least 1 element");
208 auto *offsets = static_cast<const ColumnarOffsetType*>(data[columnIndex]);
209 if (offsets[0] != 0)
210 throw std::runtime_error ("offset column doesn't start with 0: " + column.debugName);
211 for (std::size_t i = 1u; i != size; ++ i)
212 {
213 if (offsets[i] < offsets[i-1])
214 throw std::runtime_error ("offset column " + column.debugName + " is not monotonically increasing at index " + std::to_string(i) + ": " + std::to_string(offsets[i-1]) + " > " + std::to_string(offsets[i]));
215 }
216 }
217 }
218 for (std::size_t columnIndex = 0u; columnIndex != m_elements.size(); ++ columnIndex)
219 {
220 if (data[columnIndex] == nullptr)
221 continue;
222 const auto& column = m_elements[columnIndex];
223 std::size_t expectedSize = 1u;
224 if (column.offsetIndex != ColumnVectorHeader::nullIndex)
225 {
226 if (data[column.offsetIndex] == nullptr)
227 throw std::runtime_error ("column " + column.debugName + " uses offset column " + m_elements[column.offsetIndex].debugName + " that is not set");
228 const auto offsetIndex = column.offsetIndex;
229 auto *offsetsPtr = static_cast<const ColumnarOffsetType*>(data[offsetIndex]);
230 expectedSize = offsetsPtr[sizeVector[offsetIndex]-1];
231 }
232 expectedSize *= column.arraySize;
233
234 if (column.isOffset)
235 expectedSize += 1u;
236
237 if (sizeVector[columnIndex] != expectedSize)
238 throw std::runtime_error ("column size doesn't match expected size: " + column.debugName + ", found " + std::to_string (sizeVector[columnIndex]) + " vs exptected=" + std::to_string (expectedSize) + " isOffset=" + std::to_string (column.isOffset));
239 }
240 }
size_t size() const
Number of registered mappings.
std::size_t ColumnarOffsetType
the type used for the size and offsets in the columnar data

◆ checkSelf()

void columnar::ColumnVectorHeader::checkSelf ( ) const

check the self-consistency of the header

Definition at line 145 of file ColumnVectorWrapper.cxx.

147 {
148 if (m_elements.size() < numFixedColumns)
149 throw std::logic_error ("header has too few m_elements, expected at least " + std::to_string(numFixedColumns) + " but got " + std::to_string(m_elements.size()));
150 if (m_elements[sizeIndex].arraySize != m_elements.size())
151 throw std::logic_error ("size column has wrong array size, expected " + std::to_string(m_elements.size()) + " but got " + std::to_string(m_elements[sizeIndex].arraySize));
152
153 // skipping the first column, which is always the null column
154 // and behaves in a funny manner
155 for (std::size_t columnIndex = 1u; columnIndex != m_elements.size(); ++ columnIndex)
156 {
157 const auto& column = m_elements[columnIndex];
158 if (column.debugName.empty())
159 throw std::logic_error ("column " + std::to_string(columnIndex) + " has no name");
160 if (column.type == nullptr)
161 throw std::logic_error ("column " + column.debugName + " has no type");
162 if (column.isOffset && *column.type != typeid(ColumnarOffsetType))
163 throw std::logic_error ("column " + column.debugName + " is an offset column that is of type " + boost::core::demangle(column.type->name()) + " instead of ColumnarOffsetType");
164 }
165
166 for (std::size_t columnIndex = 1u; columnIndex != m_elements.size(); ++ columnIndex)
167 {
168 const auto& column = m_elements[columnIndex];
169 if (column.offsetIndex != nullIndex)
170 {
171 if (column.offsetIndex == unsetIndex)
172 throw std::logic_error ("column " + column.debugName + " has an offset index that was never set");
173 if (column.offsetIndex >= m_elements.size())
174 throw std::logic_error ("column " + column.debugName + " has invalid offset index " + std::to_string(column.offsetIndex) + " (max is " + std::to_string(m_elements.size()-1) + ")");
175 const auto& offsetElement = m_elements[column.offsetIndex];
176 if (!offsetElement.isOffset)
177 throw std::logic_error ("column " + column.debugName + " has offset index that is not marked as offset");
178 }
179 }
180 }

◆ getAllColumnInfo()

std::unordered_map< std::string, ColumnInfo > columnar::ColumnVectorHeader::getAllColumnInfo ( ) const
nodiscard

get all columns as a map of ColumnInfo for use with IColumnData::connect

Definition at line 345 of file ColumnVectorWrapper.cxx.

347 {
348 std::unordered_map<std::string, ColumnInfo> result;
349 // Skip the fixed columns (null and size)
350 for (std::size_t i = numFixedColumns; i < m_elements.size(); ++i)
351 {
352 const auto& header = m_elements[i];
353 ColumnInfo info;
354 info.name = header.debugName;
355 info.index = static_cast<unsigned>(i);
356 info.type = header.type;
357 info.sizeOfType = header.sizeOfType;
358 info.typeCategory = header.typeCategory;
359 info.accessMode = header.accessMode;
360 info.offsetName = header.offsetName;
361 info.fixedDimensions = header.fixedDimensions;
362 info.isOffset = header.isOffset;
363 info.isOptional = header.isOptional;
364 info.isEventIndex = header.isEventIndex;
365 result.emplace(info.name, std::move(info));
366 }
367 return result;
368 }

◆ getColumn()

const ColumnVectorElementHeader & columnar::ColumnVectorHeader::getColumn ( std::size_t index) const
inlinenodiscard

get the column for the given index

Definition at line 159 of file ColumnVectorWrapper.h.

159 {
160 return m_elements.at (index); }

◆ getColumnIndex()

std::size_t columnar::ColumnVectorHeader::getColumnIndex ( const std::string & name) const
inlinenodiscardnoexcept

get the column index for the given name, or nullIndex if not found

Definition at line 164 of file ColumnVectorWrapper.h.

164 {
165 auto iter = m_nameToIndex.find(name);
166 return (iter != m_nameToIndex.end()) ? iter->second : nullIndex; }

◆ numColumns()

std::size_t columnar::ColumnVectorHeader::numColumns ( ) const
inlinenodiscardnoexcept

the number of columns in the columnar data vector

Definition at line 154 of file ColumnVectorWrapper.h.

154 {
155 return m_elements.size(); }

◆ setOffsetColumn()

void columnar::ColumnVectorHeader::setOffsetColumn ( std::size_t columnIndex,
std::size_t offsetIndex )

set the index of the offset column for the given column

Definition at line 127 of file ColumnVectorWrapper.cxx.

129 {
130 if (columnIndex >= m_elements.size())
131 throw std::logic_error ("column index out of range");
132 if (offsetIndex >= m_elements.size())
133 throw std::logic_error ("offset index out of range");
134 auto& column = m_elements.at(columnIndex);
135 auto& offset = m_elements.at(offsetIndex);
136 if (!offset.isOffset)
137 throw std::runtime_error ("trying to set " + offset.debugName + " as offset column for " + column.debugName + ", but it is not marked as offset column");
138 if (column.offsetIndex != unsetIndex && column.offsetIndex != offsetIndex)
139 throw std::runtime_error ("trying to set " + offset.debugName + " as offset column for " + column.debugName + ", but it is already set to " + m_elements.at(column.offsetIndex).debugName);
140 column.offsetIndex = offsetIndex;
141 }

Member Data Documentation

◆ m_elements

std::vector<ColumnVectorElementHeader> columnar::ColumnVectorHeader::m_elements
private

the elements in the columnar data vector

Private Members

Definition at line 184 of file ColumnVectorWrapper.h.

◆ m_nameToIndex

std::unordered_map<std::string, std::size_t> columnar::ColumnVectorHeader::m_nameToIndex
private

map from column name to index for deduplication

Definition at line 187 of file ColumnVectorWrapper.h.

◆ nullIndex

std::size_t columnar::ColumnVectorHeader::nullIndex = 0u
staticconstexpr

the index used for an invalid index (always has to be 0)

Public Members

Definition at line 131 of file ColumnVectorWrapper.h.

◆ numFixedColumns

std::size_t columnar::ColumnVectorHeader::numFixedColumns = 2u
staticconstexpr

the number of fix elements in the columnar data vector

Definition at line 140 of file ColumnVectorWrapper.h.

◆ sizeIndex

std::size_t columnar::ColumnVectorHeader::sizeIndex = 1u
staticconstexpr

the index used for the column size column

Definition at line 134 of file ColumnVectorWrapper.h.

◆ unsetIndex

std::size_t columnar::ColumnVectorHeader::unsetIndex = static_cast<std::size_t>(-1)
staticconstexpr

the number used for an unset but non-null index

Definition at line 137 of file ColumnVectorWrapper.h.


The documentation for this class was generated from the following files: