ATLAS Offline Software
Public Member Functions | Private Member Functions | Private Attributes | Static Private Attributes | List of all members
Crest::CrestFsClient Class Reference

#include <CrestApiFs.h>

Inheritance diagram for Crest::CrestFsClient:
Collaboration diagram for Crest::CrestFsClient:

Public Member Functions

 CrestFsClient (bool rewriteIfExists, const std::string &root_folder)
 CrestFsClient constructor. More...
 
 ~CrestFsClient ()
 CrestClient destructor. More...
 
void createGlobalTag (GlobalTagDto &globalTag) override
 This method creates a global tag on the file storage. More...
 
GlobalTagDto findGlobalTag (const std::string &name) override
 This method finds a global tag by name on the file storage. More...
 
GlobalTagSetDto listGlobalTags (const std::string &name, int size, int page, const std::string &sort) override
 This method finds the global tags on the file storage. More...
 
void removeGlobalTag (const std::string &name) override
 This method removes a global tag. More...
 
void createTag (TagDto &tag) override
 This method creates a tag on the file storage. More...
 
TagDto findTag (const std::string &name) override
 This method finds a tag by the name on the file storage. More...
 
TagSetDto listTags (const std::string &name, int size, int page, const std::string &sort) override
 This method returns the tag list as TagSetDto. More...
 
void removeTag (const std::string &name) override
 This method removes a tag from the file storage. More...
 
int getSize (const std::string &tagname) override
 This method gets the number of IOVs for the given tag. More...
 
void createTagMeta (TagMetaDto &tag) override
 This method creates a tag meta info on the file storage. More...
 
void updateTagMeta (TagMetaDto &tag) override
 This method updates a tag meta info on the file storage. More...
 
TagMetaDto findTagMeta (const std::string &name) override
 This method gets a tag meta info by the tag name from the file storage. More...
 
IovSetDto selectIovs (const std::string &name, uint64_t since, uint64_t until, long snapshot, int size, int page, const std::string &sort) override
 This method selects IOVs for a given tagname from the file storage. More...
 
IovSetDto selectGroups (const std::string &name, long snapshot, int size, int page, const std::string &sort) override
 This method returns IOV groups for a given tagname. More...
 
void createGlobalTagMap (GlobalTagMapDto &globalTagMap) override
 This method creates a global tag map on the file storage. More...
 
GlobalTagMapSetDto findGlobalTagMap (const std::string &name, const std::string &xCrestMapMode) override
 This method searches for tag mappings using the global tag name or tag name on the file storage. More...
 
void removeGlobalTagMap (const std::string &name, const std::string &record, const std::string &label, const std::string &tagname) override
 This method removes a global tag map from the file storage. More...
 
void storeData (const std::string &tag, const StoreSetDto &storeSetJson, const std::string &payloadFormat="JSON", const std::string &objectType="none", const std::string &compressionType="none", const std::string &version="1.0", uint64_t endTime=-1) override
 This method stores several payloads in batch mode on the file storage. More...
 
std::string getPayload (const std::string &hash) override
 This method finds a payload resource associated to the hash on the file storage. More...
 
PayloadDto getPayloadMeta (const std::string &hash) override
 This method finds a payload meta info for the hash on the file storage. More...
 

Private Member Functions

void checkFsException (const char *method_name)
 Auxiliary method to check if the CrestApi library initialized to work with the file system. More...
 
std::string getFileString (const std::string &path)
 Auxiliary method to get a file as a string. More...
 
void getFileList (const std::string &path)
 Auxiliary method to get a file list in the given directory. More...
 
void flush ()
 Auxiliary method flush the data to the file system. More...
 
std::string getDateAndTime ()
 
std::string getFirstLetters (const std::string &str)
 This is an auxillary method extract first letters from the string (hash). More...
 
std::string getHash (std::string_view str)
 This method method calculates the hash for string. More...
 
std::string getHashForFile (const std::string &file)
 This method method calculates the hash for a file. More...
 
nlohmann::json getPage (nlohmann::json data, int size, int page)
 
std::vector< std::string > getVectorPage (std::vector< std::string > data, int size, int page)
 
std::vector< std::string > nameList (std::string &folder, bool ascending=true)
 
std::string buildPath (const std::string &path, const std::string &file)
 
nlohmann::json sortJson (nlohmann::json js, const std::string &par, bool order)
 
nlohmann::json sortIOVJson (nlohmann::json js, bool order)
 
bool isMatch (std::string word, long unsigned int n, std::string pattern, long unsigned int m)
 
bool isMatch (std::string word, std::string pattern)
 
nlohmann::json findAllIovs (const std::string &tagname)
 This auxiliary method finds all iovs for a given tag name. More...
 
void storePayloadDump (const std::string &tag, uint64_t since, const std::string &js, const std::string &payloadFormat, const std::string &objectType, const std::string &compressionType, const std::string &version, const std::string &streamerInfo)
 This auxiliary method stores a single payload (with since time) on the file system. More...
 

Private Attributes

std::string m_root_folder {}
 
std::string m_data_folder {}
 
std::map< std::string, nlohmann::jsonm_data {}
 
bool m_isRewrite {}
 
std::string m_currentTag {}
 

Static Private Attributes

static const std::string s_FS_TAG_PATH = "/tags"
 
static const std::string s_FS_GLOBALTAG_PATH = "/globaltags"
 
static const std::string s_FS_DATA_PATH = "/data"
 
static const std::string s_FS_TAG_FILE = "/tag.json"
 
static const std::string s_FS_IOV_FILE = "/iovs.json"
 
static const std::string s_FS_TAGMETAINFO_FILE = "/tagmetainfo.json"
 
static const std::string s_FS_META_FILE = "/meta.json"
 
static const std::string s_FS_PALOAD_FILE = "/payload.json"
 
static const std::string s_FS_GLOBALTAG_FILE = "/globaltag.json"
 
static const std::string s_FS_MAP_FILE = "/maps.json"
 
static const std::string s_FS_PATH = ""
 
static const int s_FS_PREFIX_LENGTH = 3
 

Detailed Description

Definition at line 31 of file CrestApiFs.h.

Constructor & Destructor Documentation

◆ CrestFsClient()

Crest::CrestFsClient::CrestFsClient ( bool  rewriteIfExists,
const std::string &  root_folder 
)

CrestFsClient constructor.

CrestFsClient constructor for Internet mode.

Parameters
rewriteIfExist- this parameter sets if the data can be rewritten.
root_folder- the folder where the data have to be stored.

If CrestClient is created with this method the data will be sent to the CREST Server.

Parameters
rewriteIfExists- Boolean flag to reinitialize the file system.
root_folder- root directory of the CREST storage.

Definition at line 35 of file CrestApiFs.cxx.

36  : m_root_folder(root_folder), m_isRewrite(rewriteIfExists)
37  {
38  if (m_root_folder == "") {
39  m_root_folder = std::filesystem::current_path();
40  m_root_folder += "/crest";
41  }
43  {
44  std::filesystem::create_directory(std::filesystem::path(m_root_folder));
45  }
46  m_data_folder = m_root_folder + "/data";
48  {
49  std::filesystem::create_directory(std::filesystem::path(m_data_folder));
50  }
51  }

◆ ~CrestFsClient()

Crest::CrestFsClient::~CrestFsClient ( )

CrestClient destructor.

Definition at line 56 of file CrestApiFs.cxx.

56 {}

Member Function Documentation

◆ buildPath()

std::string Crest::CrestFsClient::buildPath ( const std::string &  path,
const std::string &  file 
)
private

Definition at line 67 of file CrestApiFs.cxx.

68  {
69  std::string p = m_root_folder;
71  {
72  std::filesystem::create_directory(std::filesystem::path(m_root_folder));
73  }
74  p += path;
76  {
77  std::filesystem::create_directory(std::filesystem::path(p));
78  }
79  p += '/';
80  p += name;
82  {
83  std::filesystem::create_directory(std::filesystem::path(p));
84  }
85  return p;
86  }

◆ checkFsException()

void Crest::CrestFsClient::checkFsException ( const char *  method_name)
private

Auxiliary method to check if the CrestApi library initialized to work with the file system.

It was used to throw an exception if the library method not implemented for file system.

Parameters
method_name- method name.

Definition at line 523 of file CrestApiFs.cxx.

524  {
525  throw CrestException("ERROR in " + std::string(method_name) + " This methods is unsupported for FILESYSTEM mode");
526  }

◆ createGlobalTag()

void Crest::CrestFsClient::createGlobalTag ( GlobalTagDto globalTag)
overridevirtual

This method creates a global tag on the file storage.

Parameters
globalTag- global tag as GlobalTagDto.
Example:
 nlohmann::json js =
     {
         {"name", tagname},
         {"validity", 0},
         {"description", "test"},
         {"release", "1"},
         {"insertionTime", "2018-12-18T11:32:58.081+0000"},
         {"snapshotTime", "2018-12-18T11:32:57.952+0000"},
         {"scenario", "test"},
         {"workflow", "M"},
         {"type", "t"},
     };
 GlobalTagDto dto = GlobalTagDto();
 dto = dto.from_json(js);
 myCrestClient.createGlobalTag(dto);

Implements Crest::CrestApiBase.

Definition at line 106 of file CrestApiFs.cxx.

107  {
108  std::string name = "";
109  if (globalTag.name != "")
110  {
111  name = globalTag.name;
112  }
113  else
114  {
115  throw CrestException("ERROR in CrestFsClient::createGlobalTag: global tag name is not set.");
116  }
117 
118  std::string workDir = buildPath(s_FS_GLOBALTAG_PATH, name);
119  std::string globalTagFile = workDir + s_FS_GLOBALTAG_FILE;
120 
121  if (m_isRewrite)
122  {
124  {
126  }
127 
128  std::ofstream outFile;
129 
130  outFile.open(globalTagFile.c_str());
131  outFile << globalTag.to_json();
132  outFile.close();
133  }
134 
135  return;
136  }

◆ createGlobalTagMap()

void Crest::CrestFsClient::createGlobalTagMap ( GlobalTagMapDto globalTagMap)
overridevirtual

This method creates a global tag map on the file storage.

Parameters
globalTagMap- the global tag map as GlobalTagMapDto.
Example:
 nlohmann::json js =
 {
   {"globalTagName", globaltag},
   {"record", "testing2"},
   {"label", "test2"},
   {"tagName", tagname}
 };
 GlobalTagMapDto globalTagMap = GlobalTagMapDto();
 globalTagMap = globalTagMap.from_json(js);
 myCrestClient.createGlobalTagMap(globalTagMap);

Implements Crest::CrestApiBase.

Definition at line 431 of file CrestApiFs.cxx.

432  {
433  nlohmann::json js = globalTagMap.to_json();
434 
435  // global tag name:
436  std::string name = "";
437  try
438  {
439  name = static_cast<std::string>(js["globalTagName"]);
440  }
441  catch (...)
442  {
443  throw CrestException("ERROR in CrestClient::createGlobalTagMap: cannot get the global tag name from JSON.");
444  }
445 
446  // tag name:
447  std::string tagname = "";
448  try
449  {
450  tagname = static_cast<std::string>(js["tagName"]);
451  }
452  catch (...)
453  {
454  throw CrestException("ERROR in CrestClient::createGlobalTagMap: cannot get the tag name from JSON.");
455  }
456 
457  std::string workDir = buildPath(s_FS_GLOBALTAG_PATH, name);
458  std::string catalogFile = workDir + s_FS_MAP_FILE;
459 
461  {
462  // cathalogue file exists:
463 
464  std::string array_lst = getFileString(catalogFile);
465 
466  nlohmann::json cathalogue;
467  try
468  {
469  cathalogue = nlohmann::json::parse(array_lst);
470  }
471  catch (...)
472  {
473  throw CrestException("ERROR in CrestFsClient::createGlobalTagMap: global tag map file corrupted.");
474  }
475 
477  {
478  // the file storage contains the record of the global tag map:
479  int m = cathalogue.size();
480  for (int i = 0; i < m; i++)
481  {
482  const std::string &tn = cathalogue[i]["tagName"];
483  if (tn == tagname)
484  {
485  cathalogue.erase(i);
486  }
487  }
488  }
489  else
490  {
491  // the file storage does not contain the record of the global tag map:
492  }
494  cathalogue.push_back(js);
495  std::ofstream outFile;
496 
497  outFile.open(catalogFile.c_str());
498  outFile << cathalogue;
499  outFile.close();
500  }
501  else
502  {
503  // cathalogue file does not exist (creation):
504 
505  nlohmann::json cathalogue = nlohmann::json::array();
506  cathalogue.push_back(js);
507 
508  std::ofstream outFile;
509 
510  outFile.open(catalogFile.c_str());
511  outFile << cathalogue;
512  outFile.close();
513  }
514 
515  return;
516  }

◆ createTag()

void Crest::CrestFsClient::createTag ( TagDto tag)
overridevirtual

This method creates a tag on the file storage.

Parameters
tag- tag as TagDto.
 Example:
 nlohmann::json js =
 {
   {"name", tagname},
   {"timeType", "time"},
   {"description", "test"},
   {"synchronization", "none"},
   {"insertionTime", "2018-12-18T11:32:58.081+0000"},
   {"modificationTime", "2018-12-18T11:32:57.952+0000"},
   {"payloadSpec", "JSON"},
   {"lastValidatedTime", 0.},
   {"endOfValidity", 0.},
 };
 TagDto dto = TagDto();
 dto = dto.from_json(js);
 myCrestClient.createTag(dto);
 

Implements Crest::CrestApiBase.

Definition at line 233 of file CrestApiFs.cxx.

234  {
235  std::string name = tag.name;
236  std::string workDir = buildPath(s_FS_TAG_PATH, name);
237 
238  if (name.compare(m_currentTag) != 0 && m_isRewrite)
239  {
240  flush();
241  }
242  m_currentTag = name;
243  std::string tagFile = workDir + s_FS_TAG_FILE;
244  std::string iovFile = workDir + s_FS_IOV_FILE;
245 
246  if (m_isRewrite)
247  {
249  {
251  }
253  {
255  }
256  std::ofstream outFile;
257  outFile.open(tagFile.c_str());
258  outFile << tag.to_json();
259  outFile.close();
260  }
261  if (m_data.find(name) == m_data.end())
262  {
263  m_data.insert(std::pair<std::string, nlohmann::json>(name, nlohmann::json(nlohmann::json::value_t::array)));
264  }
265  }

◆ createTagMeta()

void Crest::CrestFsClient::createTagMeta ( TagMetaDto tag)
overridevirtual

This method creates a tag meta info on the file storage.

Parameters
tag- tag meta info as TagMetaDto.
Example:
 nlohmann::json channel = {{"0", "ATLAS_PREFERRED"}};
 nlohmann::json chanList = nlohmann::json::array({channel});
 nlohmann::json tagInfo =
     {
         {"channel_list", chanList},
         {"node_description", "description of the node"},
         {"payload_spec", "stave:Int32, eta:Int32, mag:Float, base:Float, free:Float"}};
 nlohmann::json js =
     {
         {"tagName", tagname},
         {"description", "none"},
         {"chansize", 1},
         {"colsize", 6},
         {"tagInfo", tagInfo.dump()},
         {"insertionTime", "2020-12-04"}};
 TagMetaDto dto = TagMetaDto();
 dto = dto.from_json(js);
 myCrestClient.createTagMeta(dto);

Implements Crest::CrestApiBase.

Definition at line 379 of file CrestApiFs.cxx.

380  {
381  std::string name = tag.tagName;
382 
383  std::string workDir = buildPath(s_FS_TAG_PATH, name);
384  std::string tagMetaFile = workDir + s_FS_TAGMETAINFO_FILE;
385 
386  if (m_isRewrite)
387  {
389  {
391  }
392 
393  std::ofstream outFile;
394 
395  outFile.open(tagMetaFile.c_str());
396  outFile << tag.to_json();
397  outFile.close();
398  }
399  }

◆ findAllIovs()

nlohmann::json Crest::CrestFsClient::findAllIovs ( const std::string &  tagname)
private

This auxiliary method finds all iovs for a given tag name.

The result is a JSON object.

Parameters
tagname- tag name.
Returns
JSON array with the IOV list. Example:
IOV list =
[
   {
      "tagName": "CaloOflHadDMCorr2-R123-FTFP-BERT-IOVDEP-01",
      "since": 0,
      "insertionTime": "2023-09-13T12:53:46+0000",
      "payloadHash": "ba26ca6b4e17b0c33b008045c5e703bdf049a1546e731ec7c3d2d39789b1ce1f"
  },
  ...
]

IOV JSON parameters:

 tagName          string              tag name
 since            number              since time parameter
 insertionTime    string($date-time)  insertion time
 payloadHash      string              payload hash

Definition at line 621 of file CrestApiFs.cxx.

622  {
623  nlohmann::json js = nullptr;
624  std::string file_path = m_root_folder;
625  file_path += '/';
627  file_path += '/';
628  file_path += tagname;
630 
631  try
632  {
633  std::string tag = getFileString(file_path);
635  }
636  catch (const std::exception &e)
637  {
638  throw CrestException("ERROR in CrestFsClient::findAllIovs : cannot get the iov information form file storage ");
639  }
640 
641  return js;
642  }

◆ findGlobalTag()

GlobalTagDto Crest::CrestFsClient::findGlobalTag ( const std::string &  name)
overridevirtual

This method finds a global tag by name on the file storage.

Only one global tag should be returned. (This method is an analogue of the find_global_tag method in Python)

Parameters
name- global tag name,
Returns
global tag as GlobalTagDto.

Implements Crest::CrestApiBase.

Definition at line 138 of file CrestApiFs.cxx.

139  {
140  nlohmann::json js = nullptr;
141  GlobalTagDto dto;
142 
143  std::string workDir = buildPath(s_FS_GLOBALTAG_PATH, name);
144  std::string file_path = workDir + s_FS_GLOBALTAG_FILE;
145 
146  try
147  {
148  std::string tag = getFileString(file_path);
150  dto = GlobalTagDto::from_json(js);
151  }
152  catch (...)
153  {
154  throw CrestException(
155  "ERROR in CrestFsClient::findGlobalTag: cannot get the global tag " + name + " form the file storage.");
156  }
157 
158  return dto;
159  }

◆ findGlobalTagMap()

GlobalTagMapSetDto Crest::CrestFsClient::findGlobalTagMap ( const std::string &  name,
const std::string &  xCrestMapMode 
)
overridevirtual

This method searches for tag mappings using the global tag name or tag name on the file storage.

(This method is an analogue of the find_global_tag_map method in Python)

Parameters
name- name of a global tag or a tag
xCrestMapMode- search mode (Trace or BackTrace). If it is set as "Trace" the global tag name will be used for searching, otherwise - the tag name.
Returns
global tag map list as GlobalTagMapSetDto.

Implements Crest::CrestApiBase.

Definition at line 528 of file CrestApiFs.cxx.

529  {
530 
531  nlohmann::json js = nullptr;
532 
533  if (xCrestMapMode != "Trace")
534  {
535  throw CrestException(
536  "ERROR in CrestFsClient::getGlobalTagMap: not supported value for the parameter xCrestMapMode = " + xCrestMapMode);
537  }
538 
539  std::string workDir = m_root_folder + s_FS_GLOBALTAG_PATH;
540  std::string file_path = workDir;
541  file_path += '/';
542  file_path += name;
544 
545  try
546  {
547  std::string tag = getFileString(file_path);
549  }
550  catch (...)
551  {
552  throw CrestException(
553  "ERROR in CrestFsClient::getGlobalTagMap: cannot get the global tag map " + name +
554  " form the file storage.");
555  }
556 
557  GlobalTagMapSetDto dto;
559  return dto;
560  }

◆ findTag()

TagDto Crest::CrestFsClient::findTag ( const std::string &  name)
overridevirtual

This method finds a tag by the name on the file storage.

(This method is an analogue of the find_tag method in Python)

Parameters
name- tag name
Returns
tag as TagDto object.

Implements Crest::CrestApiBase.

Definition at line 267 of file CrestApiFs.cxx.

268  {
269  nlohmann::json js = nullptr;
270  TagDto dto;
271 
272  std::string workDir = buildPath(s_FS_TAG_PATH, name);
273  std::string file_path = workDir + s_FS_TAG_FILE;
274 
275  try
276  {
277  std::string tag = getFileString(file_path);
279  dto = TagDto::from_json(js);
280  }
281  catch (...)
282  {
283  throw CrestException(
284  "ERROR in CrestFsClient::findTag: cannot get the tag " + name + " form the file storage.");
285  }
286 
287  return dto;
288  }

◆ findTagMeta()

TagMetaDto Crest::CrestFsClient::findTagMeta ( const std::string &  name)
overridevirtual

This method gets a tag meta info by the tag name from the file storage.

Parameters
name- tag name
Returns
tag meta info as a TagMetaDto.

Implements Crest::CrestApiBase.

Definition at line 406 of file CrestApiFs.cxx.

407  {
408  nlohmann::json js = nullptr;
409  TagMetaDto dto;
410 
411  std::string workDir = buildPath(s_FS_TAG_PATH, name);
412  std::string file_path = workDir + s_FS_TAGMETAINFO_FILE;
413 
414  try
415  {
416  std::string tag = getFileString(file_path);
418  dto = TagMetaDto::from_json(js);
419  }
420  catch (...)
421  {
422  throw CrestException(
423  "ERROR in CrestFsClient::findTagMeta: cannot get the tag " + name + " form the file storage.");
424  }
425 
426  return dto;
427  }

◆ flush()

void Crest::CrestFsClient::flush ( )
private

Auxiliary method flush the data to the file system.

Definition at line 360 of file CrestApiFs.cxx.

361  {
362 
363  for (auto &item : m_data)
364  {
365  nlohmann::json iov = item.second;
366  std::string name = item.first;
367  std::string workDir = m_root_folder + s_FS_TAG_PATH + '/' + name;
368  std::ofstream outFile;
369  std::string tagFile = workDir + s_FS_IOV_FILE;
370  outFile.open(tagFile.c_str());
371  outFile << iov;
372  outFile.close();
373  }
374 
375  m_data.clear();
376  }

◆ getDateAndTime()

std::string Crest::CrestFsClient::getDateAndTime ( )
private

Definition at line 711 of file CrestApiFs.cxx.

712  {
713  time_t now = time(0);
714  struct tm tstruct;
715  char buf[80];
716 
717  localtime_r(&now, &tstruct);
718  strftime(buf, sizeof(buf), "%Y-%m-%d %X", &tstruct);
719  return buf;
720  }

◆ getFileList()

void Crest::CrestFsClient::getFileList ( const std::string &  path)
private

Auxiliary method to get a file list in the given directory.

(The old method name - get_file_list)

Parameters
path- path to the directory.

Definition at line 88 of file CrestApiFs.cxx.

89  {
91  for (auto i = std::filesystem::directory_iterator(p); i != std::filesystem::directory_iterator(); i++)
92  {
93  std::string file = i->path().filename().string();
94  if (file != "data")
95  {
96  std::cout << file << std::endl;
97  }
98  else{
99  continue;
100  }
101  }
102  }

◆ getFileString()

std::string Crest::CrestFsClient::getFileString ( const std::string &  path)
private

Auxiliary method to get a file as a string.

(The old method name - get_file_string)

Parameters
path- path to the file.

Definition at line 60 of file CrestApiFs.cxx.

60  {
61  std::ifstream ifs(path);
62  std::stringstream buf;
63  buf << ifs.rdbuf();
64  return buf.str();
65  }

◆ getFirstLetters()

std::string Crest::CrestFsClient::getFirstLetters ( const std::string &  str)
private

This is an auxillary method extract first letters from the string (hash).

The letter number is in the s_FS_PREFIX_LENGTH variable. This method is used to create a catalogue sub directory in the file storage.

Parameters
str- string to extract first letters.
Returns
- string with first letters

Definition at line 728 of file CrestApiFs.cxx.

729  {
730  std::string result = str.substr(0, s_FS_PREFIX_LENGTH);
731  return result;
732  }

◆ getHash()

std::string Crest::CrestFsClient::getHash ( std::string_view  str)
private

This method method calculates the hash for string.

Parameters
str- string.
Returns
hash for a given string.

Definition at line 722 of file CrestApiFs.cxx.

723  {
724  std::string hash_hex_str = picosha2::hash256_hex_string(str.begin(), str.end());
725  return hash_hex_str;
726  }

◆ getHashForFile()

std::string Crest::CrestFsClient::getHashForFile ( const std::string &  file)
private

This method method calculates the hash for a file.

Parameters
file- path to the file with the file name..
Returns
hash for a given file.

Definition at line 856 of file CrestApiFs.cxx.

857  {
858  std::ifstream ifs(file);
859  if (!ifs)
860  {
861  throw CrestException(
862  "ERROR in CrestFsClient::getHashForFile cannot open file \"" + file + "\".");
863  }
864 
866  std::vector<char> buffer(1024 * 1024); // use 1M memory
867  while (ifs.read(buffer.data(), static_cast<std::streamsize>(buffer.size())))
868  {
869  hasher.process(buffer.begin(), buffer.end());
870  }
871  // process remains
872  hasher.process(buffer.begin(), buffer.begin() + static_cast<int>(ifs.gcount()));
873  hasher.finish();
874 
875  std::string hash = picosha2::get_hash_hex_string(hasher);
876  return hash;
877  }

◆ getPage()

nlohmann::json Crest::CrestFsClient::getPage ( nlohmann::json  data,
int  size,
int  page 
)
private

Definition at line 949 of file CrestApiFs.cxx.

950  {
952  int dataSize = data.size();
953 
954  if (dataSize == 0)
955  return js; // the data is absent
956 
957  // index interval to load the data from JSON array:
958  int kmin = size * page;
959  int kmax = size * (page + 1);
960 
961  // check if the interval is correct:
962  if (kmin > dataSize - 1)
963  return js; // out of range
964 
965  if (kmax > dataSize - 1)
966  { // this step is not full
967  kmax = dataSize;
968  }
969 
970  for (int i = kmin; i < kmax; i++)
971  {
972  js.push_back(data[i]);
973  }
974  return js;
975  }

◆ getPayload()

std::string Crest::CrestFsClient::getPayload ( const std::string &  hash)
overridevirtual

This method finds a payload resource associated to the hash on the file storage.

The payload returns as a string. (This method is an analogue of the get_payload method in Python)

Parameters
hash- hash.
Returns
payload as a std::string

Implements Crest::CrestApiBase.

Definition at line 881 of file CrestApiFs.cxx.

882  {
883  std::string workDir = m_data_folder;
884  workDir += '/';
885  workDir += getFirstLetters(hash);
886  workDir += '/';
887  workDir += hash;
888  std::string filePath = workDir + "/payload.json";
889  std::string res = "";
890 
891  try
892  {
894  {
896  }
897  else
898  {
899  throw CrestException("payload with hash " + hash + " does not exist.");
900  }
901  }
902  catch (const std::exception &e)
903  {
904  std::string message = e.what();
905  throw CrestException("ERROR in CrestFsClient::getPayload cannot get the payload form file storage, " + message);
906  }
907 
908  return res;
909  }

◆ getPayloadMeta()

PayloadDto Crest::CrestFsClient::getPayloadMeta ( const std::string &  hash)
overridevirtual

This method finds a payload meta info for the hash on the file storage.

(This method is an analogue of the get_payload_meta method in Python)

Parameters
hash- hash.
Returns
payload meta info as PayloadDto.

Implements Crest::CrestApiBase.

Definition at line 911 of file CrestApiFs.cxx.

912  {
913  nlohmann::json js = nullptr;
914  PayloadDto dto;
915 
916  std::string workDir = m_data_folder;
917  workDir += '/';
918  workDir += getFirstLetters(hash);
919  workDir += '/';
920  workDir += hash;
921 
922  std::string filePath = workDir;
923  filePath += "/meta.json";
924 
925  std::string res = "";
926 
927  try
928  {
930  {
933  dto = PayloadDto::from_json(js);
934  }
935  else
936  {
937  throw CrestException("payload meta info with hash " + hash + " does not exist.");
938  }
939  }
940  catch (const std::exception &e)
941  {
942  std::string message = e.what();
943  throw CrestException("ERROR in CrestClient::getPayloadMeta cannot get the payload meta info form file storage, " + message);
944  }
945 
946  return dto;
947  }

◆ getSize()

int Crest::CrestFsClient::getSize ( const std::string &  tagname)
overridevirtual

This method gets the number of IOVs for the given tag.

(This method is an analogue of the get_size method in Python)

Parameters
tagname- tag name.
Returns
IOV number.

Implements Crest::CrestApiBase.

Definition at line 644 of file CrestApiFs.cxx.

645  {
646  int res = 0;
647 
648  try
649  {
651  int length = iovs.size();
652  return length;
653  }
654  catch (...)
655  {
656  return res;
657  }
658 
659  return res;
660  }

◆ getVectorPage()

std::vector< std::string > Crest::CrestFsClient::getVectorPage ( std::vector< std::string >  data,
int  size,
int  page 
)
private

Definition at line 977 of file CrestApiFs.cxx.

978  {
979  std::vector<std::string> res;
980  int dataSize = data.size();
981 
982  if (dataSize == 0)
983  return res; // the data is absent
984 
985  // index interval to load the data from JSON array:
986  int kmin = size * page;
987  int kmax = size * (page + 1);
988 
989  // check if the interval is correct:
990  if (kmin > dataSize - 1)
991  return res; // out of range
992 
993  if (kmax > dataSize - 1)
994  { // this step is not full
995  kmax = dataSize;
996  }
997 
998  for (int i = kmin; i < kmax; i++)
999  {
1000  res.push_back(data[i]);
1001  }
1002  return res;
1003  }

◆ isMatch() [1/2]

bool Crest::CrestFsClient::isMatch ( std::string  word,
long unsigned int  n,
std::string  pattern,
long unsigned int  m 
)
private

Definition at line 1101 of file CrestApiFs.cxx.

1102  {
1103  if (m == pattern.size())
1104  {
1105  return n == word.size();
1106  }
1107 
1108  if (n == word.size())
1109  {
1110  for (long unsigned int i = m; i < pattern.size(); i++)
1111  {
1112  if (pattern[i] != '%')
1113  {
1114  return false;
1115  }
1116  }
1117 
1118  return true;
1119  }
1120 
1121  if (pattern[m] == '?' || pattern[m] == word[n])
1122  {
1123  return isMatch(word, n + 1, pattern, m + 1);
1124  }
1125 
1126  if (pattern[m] == '%')
1127  {
1128  return isMatch(word, n + 1, pattern, m) || isMatch(word, n, pattern, m + 1);
1129  }
1130 
1131  return false;
1132  }

◆ isMatch() [2/2]

bool Crest::CrestFsClient::isMatch ( std::string  word,
std::string  pattern 
)
private

Definition at line 1134 of file CrestApiFs.cxx.

1135  {
1136  return isMatch(word, 0, pattern, 0);
1137  }

◆ listGlobalTags()

GlobalTagSetDto Crest::CrestFsClient::listGlobalTags ( const std::string &  name,
int  size,
int  page,
const std::string &  sort 
)
overridevirtual

This method finds the global tags on the file storage.

Parameters
name- global tag name pattern, "%" can be used for any symbols,
size- page size,
page- page number,
sort- sorting order (name:ASC or name:DESC),
Returns
global tag list as GlobalTagSetDto.

Implements Crest::CrestApiBase.

Definition at line 166 of file CrestApiFs.cxx.

167  {
168 
169  std::string folder = m_root_folder + s_FS_GLOBALTAG_PATH;
170 
171  GlobalTagSetDto tagSet;
172 
173  bool ascending = true;
174  if (sort == "name:ASC")
175  {
176  ascending = true;
177  }
178  else if (sort == "name:DESC")
179  {
180  ascending = false;
181  }
182  else
183  {
184  throw CrestException(
185  "ERROR in CrestFsClient::listTags: wrong sort parameter." + sort);
186  }
187 
188  try
189  {
190  std::vector<std::string> taglist = nameList(folder, ascending);
191  std::vector<std::string> clearedTaglist;
192 
193  for (std::string tag : taglist)
194  {
195  std::string file_name = folder + "/" + tag + "/" + s_FS_GLOBALTAG_FILE;
196 
198  {
199  if (name != "")
200  {
201  if (isMatch(tag, name))
202  {
203  clearedTaglist.push_back(tag);
204  }
205  }
206  else
207  {
208  clearedTaglist.push_back(tag);
209  }
210  }
211  }
212 
213  taglist = getVectorPage(clearedTaglist, size, page);
214  for (std::string tag : taglist)
215  {
216  std::string file_name = folder + "/" + tag + "/" + s_FS_GLOBALTAG_FILE;
218  tagSet.resources.push_back(dto);
219  }
220  tagSet.datatype = "";
221  }
222  catch (const std::exception &e)
223  {
224  throw CrestException(
225  "ERROR in CrestFsClient::listGlobalTags: cannot get the tag list.");
226  }
227 
228  return tagSet;
229  }

◆ listTags()

TagSetDto Crest::CrestFsClient::listTags ( const std::string &  name,
int  size,
int  page,
const std::string &  sort 
)
overridevirtual

This method returns the tag list as TagSetDto.

(This method is an analogue of the list_tags method in Python)

Parameters
name- tag name pattern,
size- page size,
page- page number,
sort- sorting order (name:ASC or name:DESC).
Returns
tag list as TagSetDto object.

Implements Crest::CrestApiBase.

Definition at line 295 of file CrestApiFs.cxx.

296  {
297 
298  std::string folder = m_root_folder + s_FS_TAG_PATH;
299 
300  TagSetDto tagSet;
301 
302  bool ascending = true;
303  if (sort == "name:ASC")
304  {
305  ascending = true;
306  }
307  else if (sort == "name:DESC")
308  {
309  ascending = false;
310  }
311  else
312  {
313  throw CrestException(
314  "ERROR in CrestFsClient::listTags: wrong sort parameter." + sort);
315  }
316 
317  try
318  {
319  std::vector<std::string> taglist = nameList(folder, ascending);
320  std::vector<std::string> clearedTaglist;
321 
322  for (std::string tag : taglist)
323  {
324  std::string file_name = folder + "/" + tag + "/" + s_FS_TAG_FILE;
325 
327  {
328  if (name != "")
329  {
330  if (isMatch(tag, name))
331  {
332  clearedTaglist.push_back(tag);
333  }
334  }
335  else
336  {
337  clearedTaglist.push_back(tag);
338  }
339  }
340  }
341 
342  taglist = getVectorPage(clearedTaglist, size, page);
343  for (std::string tag : taglist)
344  {
345  std::string file_name = folder + "/" + tag + "/" + s_FS_TAG_FILE;
346  TagDto dto = findTag(tag);
347  tagSet.resources.push_back(dto);
348  }
349  tagSet.datatype = "";
350  }
351  catch (const std::exception &e)
352  {
353  throw CrestException(
354  "ERROR in CrestFsClient::listTags: cannot get the tag list.");
355  }
356 
357  return tagSet;
358  }

◆ nameList()

std::vector< std::string > Crest::CrestFsClient::nameList ( std::string &  folder,
bool  ascending = true 
)
private

Definition at line 1005 of file CrestApiFs.cxx.

1006  {
1007  std::vector<std::string> tag_list;
1009 
1010  for (auto i = std::filesystem::directory_iterator(p); i != std::filesystem::directory_iterator(); i++)
1011  {
1012  std::string file = i->path().filename().string();
1013  tag_list.push_back(file);
1014  }
1015 
1016  std::sort(tag_list.begin(), tag_list.end());
1017 
1018  if (ascending == false)
1019  {
1020  std::reverse(tag_list.begin(), tag_list.end());
1021  }
1022 
1023  return tag_list;
1024  }

◆ removeGlobalTag()

void Crest::CrestFsClient::removeGlobalTag ( const std::string &  name)
overridevirtual

This method removes a global tag.

(This method is an analogue of the remove_global_tag method in Python)

Parameters
name- global tag name

Implements Crest::CrestApiBase.

Definition at line 161 of file CrestApiFs.cxx.

162  {
163  checkFsException("CrestFsClient::removeGlobalTag");
164  }

◆ removeGlobalTagMap()

void Crest::CrestFsClient::removeGlobalTagMap ( const std::string &  name,
const std::string &  record,
const std::string &  label,
const std::string &  tagname 
)
overridevirtual

This method removes a global tag map from the file storage.

Parameters
name- the global tag name,
label- label,
tagname- tag name.

Implements Crest::CrestApiBase.

Definition at line 518 of file CrestApiFs.cxx.

519  {
520  checkFsException("CrestFsClient::removeGlobalTagMap");
521  }

◆ removeTag()

void Crest::CrestFsClient::removeTag ( const std::string &  name)
overridevirtual

This method removes a tag from the file storage.

(This method is an analogue of the remove_tag method in Python)

Parameters
tagName- tag name

Implements Crest::CrestApiBase.

Definition at line 290 of file CrestApiFs.cxx.

291  {
292  checkFsException("CrestFsClient::removeTag");
293  }

◆ selectGroups()

IovSetDto Crest::CrestFsClient::selectGroups ( const std::string &  name,
long  snapshot,
int  size,
int  page,
const std::string &  sort 
)
overridevirtual

This method returns IOV groups for a given tagname.

The result is an IOV list. (This method is an analogue of the select_groups method in Python)

Parameters
name- tag name,
snapshot- snapshot,
size- page size,
page- page number,
sort- sorting order (id.since:ASC or id.since:DESC)
Returns
an IOV groups as IovSetDto.

Implements Crest::CrestApiBase.

Definition at line 662 of file CrestApiFs.cxx.

663  {
664  checkFsException("CrestFsClient::selectGroups");
665  return IovSetDto();
666  }

◆ selectIovs()

IovSetDto Crest::CrestFsClient::selectIovs ( const std::string &  name,
uint64_t  since,
uint64_t  until,
long  snapshot,
int  size,
int  page,
const std::string &  sort 
)
overridevirtual

This method selects IOVs for a given tagname from the file storage.

The result is an IOV list. (This method is an analogue of the select_iovs method in Python)

Parameters
name- tag name,
since- since time (the beginning of the time interval),
until- until time (end of the time interval),
snapshot- snapshot,
size- page size,
page- page number,
sort- sorting order (id.since:ASC or id.since:DESC)
Returns
an IOV list as IovSetDto.

Implements Crest::CrestApiBase.

Definition at line 564 of file CrestApiFs.cxx.

565  {
566  IovSetDto dto;
567 
569 
570  try
571  {
573  int niovs = iovList.size();
574 
575  for (int i = 0; i < niovs; i++)
576  {
577  if (iovList[i].find("since") != iovList[i].end())
578  {
579  uint64_t currentS = iovList[i]["since"];
580 
581  if (until != static_cast<uint64_t>(-1))
582  {
583  if (currentS >= since && currentS <= until)
584  {
585  js.push_back(iovList[i]);
586  }
587  }
588  else
589  { // until == -1, Infinity
590  if (currentS >= since)
591  {
592  js.push_back(iovList[i]);
593  }
594  } // until == -1
595  }
596  } // for
597  }
598  catch (...)
599  {
600  throw CrestException("ERROR in CrestClient::selectIovsFS : cannot get the iov list form file storage");
601  }
602 
603  bool ascending = true;
604  if (sort == "id.since:ASC")
605  ascending = true;
606  else if (sort == "id.since:DESC")
607  ascending = false;
608  else
609  {
610  throw CrestException(
611  "ERROR in CrestFsClient::selectIovs: wrong sort parameter." + sort);
612  }
613 
617 
618  return dto;
619  }

◆ sortIOVJson()

nlohmann::json Crest::CrestFsClient::sortIOVJson ( nlohmann::json  js,
bool  order 
)
private

Definition at line 1064 of file CrestApiFs.cxx.

1065  {
1066  std::string par = "since";
1068  std::vector<double> parlist;
1069  std::map<double, nlohmann::json> m;
1070 
1071  int size = js.size();
1072  for (int i = 0; i < size; i++)
1073  {
1074  nlohmann::json elem = js[i];
1075 
1076  auto res = elem.find(par);
1077 
1078  if (res != elem.end())
1079  {
1080  double par_val = elem[par];
1081  parlist.push_back(par_val);
1082  m[par_val] = elem;
1083  }
1084  }
1085 
1086  std::sort(parlist.begin(), parlist.end());
1087 
1088  if (order == false)
1089  {
1090  std::reverse(parlist.begin(), parlist.end());
1091  }
1092 
1093  for (double item : parlist)
1094  {
1095  respond.push_back(m[item]);
1096  }
1097 
1098  return respond;
1099  }

◆ sortJson()

nlohmann::json Crest::CrestFsClient::sortJson ( nlohmann::json  js,
const std::string &  par,
bool  order 
)
private

Definition at line 1027 of file CrestApiFs.cxx.

1028  {
1030  std::vector<std::string> parlist;
1031  std::map<std::string, nlohmann::json> m;
1032 
1033  int size = js.size();
1034  for (int i = 0; i < size; i++)
1035  {
1036  nlohmann::json elem = js[i];
1037 
1038  auto res = elem.find(par);
1039 
1040  if (res != elem.end())
1041  {
1042  std::string par_val = elem[par];
1043  parlist.push_back(par_val);
1044  m[par_val] = elem;
1045  }
1046  }
1047 
1048  std::sort(parlist.begin(), parlist.end());
1049 
1050  if (order == false)
1051  {
1052  std::reverse(parlist.begin(), parlist.end());
1053  }
1054 
1055  for (std::string item : parlist)
1056  {
1057  respond.push_back(m[item]);
1058  }
1059 
1060  return respond;
1061  }

◆ storeData()

void Crest::CrestFsClient::storeData ( const std::string &  tag,
const StoreSetDto storeSetJson,
const std::string &  payloadFormat = "JSON",
const std::string &  objectType = "none",
const std::string &  compressionType = "none",
const std::string &  version = "1.0",
uint64_t  endTime = -1 
)
overridevirtual

This method stores several payloads in batch mode on the file storage.

(This method is an analogue of the store_data method in Python)

Parameters
tag- tag name.
storeSetJson- iov and payload data as a JSON object.
payloadFormat- payload format [FILE | JSON].
objectType- object type.
compressionType- compression type.
version- version.
endTime- end time, if endtime = 0, the server does not use this parameter in the internal check.
Example how to use these parameters:
   std::string tag = "test_MvG3a";
   uint64_t endtime = 200;
   std::string objectType = "test";
   std::string compressionType = "none";
   std::string version = "1.0";
   std::string payloadFormat = "JSON";
   std::string jsonString = R"({"size": 1, "datatype": "data", "format": "StoreSetDto", "resources": [{"since": 1000,"data": "Sample data","streamerInfo": "Info123"}]})";
   StoreSetDto storeSetJson = StoreSetDto::from_json(jsonString);

Implements Crest::CrestApiBase.

Definition at line 669 of file CrestApiFs.cxx.

676  {
677  std::string name = tag;
678 
679  std::string workDir = buildPath(s_FS_TAG_PATH, name);
680  std::string tagFile = workDir + s_FS_TAG_FILE;
681  std::string iovFile = workDir + s_FS_IOV_FILE;
682 
683  nlohmann::json js_data = storeSetJson.to_json();
684 
686  auto it = js_data.find("resources");
687  if (it != js_data.end())
688  {
689  res = js_data["resources"];
690  }
691 
692  try
693  {
694  for (auto &kvp : res)
695  {
696  std::string payload = kvp.value("data", "");
697  int since = kvp.value("since", 0);
698  std::string streamer = kvp.value("streamerInfo", "");
699  // Register everything on the file system.
700  storePayloadDump(tag, since, payload, payloadFormat,
701  objectType, compressionType, version, streamer);
702  }
703  } // end of try
704  catch (...)
705  {
706  throw CrestException("ERROR in CrestFsClient::storeData cannot store the data in a file");
707  } // end of catch
708  flush();
709  }

◆ storePayloadDump()

void Crest::CrestFsClient::storePayloadDump ( const std::string &  tag,
uint64_t  since,
const std::string &  js,
const std::string &  payloadFormat,
const std::string &  objectType,
const std::string &  compressionType,
const std::string &  version,
const std::string &  streamerInfo 
)
private

This auxiliary method stores a single payload (with since time) on the file system.

Parameters
tag- tag name.
since- since parameter,
js- a string with a payload or file (the path and the file name) with payload.
payloadFormat- payload format [FILE | JSON].
objectType- object type.
compressionType- compression type.
version- version.
endTime- end time, if endtime = 0, the server does not use this parameter in the internal check.

Definition at line 734 of file CrestApiFs.cxx.

742  {
743 
744  std::ofstream outFile;
745 
746  std::string hashCode;
747  std::string payloadLocalFile;
748 
749  // payload file:
750  if (payloadFormat == "JSON")
751  {
752  hashCode = getHash(js);
753  }
754  else
755  {
756 
757  int found_dots = js.find_first_of(':');
758  int word_size = js.size();
759  payloadLocalFile = js.substr(found_dots + 3, word_size);
760 
761  hashCode = getHashForFile(payloadLocalFile);
762  }
763 
764  std::string workDir = m_data_folder;
765  workDir += '/';
766  workDir += getFirstLetters(hashCode);
768  {
769  std::filesystem::create_directory(std::filesystem::path(workDir));
770  }
771  workDir += '/';
772  workDir += hashCode;
774  {
775  std::filesystem::create_directory(std::filesystem::path(workDir));
776  }
777  std::string tagFile = workDir + "/payload.json";
778 
779  if (payloadFormat == "JSON")
780  {
781  outFile.open(tagFile);
782  outFile << js;
783  outFile.close();
784  }
785  else
786  {
787  try
788  {
789  std::filesystem::copy_file(payloadLocalFile, tagFile);
790  }
791  catch (std::filesystem::filesystem_error &e)
792  {
793  throw CrestException("ERROR in CrestFsClient::storePayloadDump cannot not save payload file: " + tagFile + e.what());
794  }
795  }
796 
797  // Define the meta info for the payload:
798  nlohmann::json jsn =
799  {
800  {"hash", hashCode},
801  {"checkSum", "SHA-256"},
802  {"objectType", objectType},
803  {"version", version},
804  {"size", js.size()},
805  {"streamerInfo", streamerInfo},
806  {"compressionType", compressionType},
807  {"insertionTime", getDateAndTime()}};
808 
809  // payload meta info file:
810  std::string metaFile = workDir + "/meta.json";
811 
812  outFile.open(metaFile);
813  outFile << jsn.dump();
814  outFile.close();
815 
816  // check if data exists
817 
818  if (m_data.find(tag) == m_data.end())
819  {
820  try
821  {
823  m_data.insert(std::pair<std::string, nlohmann::json>(tag, jsi));
824  }
825  catch (...)
826  {
827  try
828  { // tag exists, but there are no IOVs
829  nlohmann::json tagJS = findTag(tag).to_json();
831  m_data.insert(std::pair<std::string, nlohmann::json>(tag, jsFree));
832  }
833  catch (...)
834  {
835  throw CrestException(
836  "ERROR in CrestFsClient::storePayloadDump cannot get data for tag \"" + tag + "\" from file storage.");
837  }
838  }
839  }
840 
842  if (it != m_data.end())
843  {
844  std::string link = hashCode;
845  nlohmann::json iovs = it->second;
847  obj["tagName"] = tag;
848  obj["since"] = since;
849  obj["insertionTime"] = getDateAndTime();
850  obj["payloadHash"] = link;
851  iovs.push_back(obj);
852  m_data[it->first] = iovs;
853  }
854  }

◆ updateTagMeta()

void Crest::CrestFsClient::updateTagMeta ( TagMetaDto tag)
overridevirtual

This method updates a tag meta info on the file storage.

Parameters
tag- tag meta info as TagMetaDto.

Implements Crest::CrestApiBase.

Definition at line 401 of file CrestApiFs.cxx.

402  {
403  checkFsException("CrestFsClient::updateTagMeta");
404  }

Member Data Documentation

◆ m_currentTag

std::string Crest::CrestFsClient::m_currentTag {}
private

Definition at line 58 of file CrestApiFs.h.

◆ m_data

std::map<std::string, nlohmann::json> Crest::CrestFsClient::m_data {}
private

Definition at line 36 of file CrestApiFs.h.

◆ m_data_folder

std::string Crest::CrestFsClient::m_data_folder {}
private

Definition at line 35 of file CrestApiFs.h.

◆ m_isRewrite

bool Crest::CrestFsClient::m_isRewrite {}
private

Definition at line 37 of file CrestApiFs.h.

◆ m_root_folder

std::string Crest::CrestFsClient::m_root_folder {}
private

Definition at line 34 of file CrestApiFs.h.

◆ s_FS_DATA_PATH

const std::string Crest::CrestFsClient::s_FS_DATA_PATH = "/data"
inlinestaticprivate

Definition at line 43 of file CrestApiFs.h.

◆ s_FS_GLOBALTAG_FILE

const std::string Crest::CrestFsClient::s_FS_GLOBALTAG_FILE = "/globaltag.json"
inlinestaticprivate

Definition at line 52 of file CrestApiFs.h.

◆ s_FS_GLOBALTAG_PATH

const std::string Crest::CrestFsClient::s_FS_GLOBALTAG_PATH = "/globaltags"
inlinestaticprivate

Definition at line 42 of file CrestApiFs.h.

◆ s_FS_IOV_FILE

const std::string Crest::CrestFsClient::s_FS_IOV_FILE = "/iovs.json"
inlinestaticprivate

Definition at line 46 of file CrestApiFs.h.

◆ s_FS_MAP_FILE

const std::string Crest::CrestFsClient::s_FS_MAP_FILE = "/maps.json"
inlinestaticprivate

Definition at line 53 of file CrestApiFs.h.

◆ s_FS_META_FILE

const std::string Crest::CrestFsClient::s_FS_META_FILE = "/meta.json"
inlinestaticprivate

Definition at line 49 of file CrestApiFs.h.

◆ s_FS_PALOAD_FILE

const std::string Crest::CrestFsClient::s_FS_PALOAD_FILE = "/payload.json"
inlinestaticprivate

Definition at line 50 of file CrestApiFs.h.

◆ s_FS_PATH

const std::string Crest::CrestFsClient::s_FS_PATH = ""
inlinestaticprivate

Definition at line 55 of file CrestApiFs.h.

◆ s_FS_PREFIX_LENGTH

const int Crest::CrestFsClient::s_FS_PREFIX_LENGTH = 3
inlinestaticprivate

Definition at line 57 of file CrestApiFs.h.

◆ s_FS_TAG_FILE

const std::string Crest::CrestFsClient::s_FS_TAG_FILE = "/tag.json"
inlinestaticprivate

Definition at line 45 of file CrestApiFs.h.

◆ s_FS_TAG_PATH

const std::string Crest::CrestFsClient::s_FS_TAG_PATH = "/tags"
inlinestaticprivate

Definition at line 41 of file CrestApiFs.h.

◆ s_FS_TAGMETAINFO_FILE

const std::string Crest::CrestFsClient::s_FS_TAGMETAINFO_FILE = "/tagmetainfo.json"
inlinestaticprivate

Definition at line 47 of file CrestApiFs.h.


The documentation for this class was generated from the following files:
xAOD::iterator
JetConstituentVector::iterator iterator
Definition: JetConstituentVector.cxx:68
mergePhysValFiles.pattern
pattern
Definition: DataQuality/DataQualityUtils/scripts/mergePhysValFiles.py:26
AllowedVariables::e
e
Definition: AsgElectronSelectorTool.cxx:37
Crest::CrestFsClient::s_FS_GLOBALTAG_PATH
static const std::string s_FS_GLOBALTAG_PATH
Definition: CrestApiFs.h:42
Crest::CrestFsClient::s_FS_TAG_PATH
static const std::string s_FS_TAG_PATH
Definition: CrestApiFs.h:41
IovSetDto::from_fs_json
static IovSetDto from_fs_json(const json &j)
Definition: CrestModel.cxx:531
GlobalTagMapSetDto::from_fs_json
static GlobalTagMapSetDto from_fs_json(const json &j)
Definition: CrestModel.cxx:277
data
char data[hepevt_bytes_allocation_ATLAS]
Definition: HepEvt.cxx:11
GlobalTagMapDto::to_json
json to_json() const
Definition: CrestModel.cxx:235
picosha2::get_hash_hex_string
void get_hash_hex_string(const hash256_one_by_one &hasher, std::string &hex_str)
Definition: picosha2.h:281
get_generator_info.result
result
Definition: get_generator_info.py:21
dqt_zlumi_alleff_HIST.iov
iov
Definition: dqt_zlumi_alleff_HIST.py:119
python.SystemOfUnits.m
int m
Definition: SystemOfUnits.py:91
athena.path
path
python interpreter configuration --------------------------------------—
Definition: athena.py:128
Crest::CrestFsClient::m_data_folder
std::string m_data_folder
Definition: CrestApiFs.h:35
find
std::string find(const std::string &s)
return a remapped string
Definition: hcg.cxx:135
json
nlohmann::json json
Definition: HistogramDef.cxx:9
Crest::CrestFsClient::getFirstLetters
std::string getFirstLetters(const std::string &str)
This is an auxillary method extract first letters from the string (hash).
Definition: CrestApiFs.cxx:728
GlobalTagSetDto::resources
std::vector< GlobalTagDto > resources
Definition: CrestModel.h:83
find_tgc_unfilled_channelids.iovs
iovs
Definition: find_tgc_unfilled_channelids.py:12
TagDto::to_json
json to_json() const
Definition: CrestModel.cxx:172
Crest::CrestFsClient::sortIOVJson
nlohmann::json sortIOVJson(nlohmann::json js, bool order)
Definition: CrestApiFs.cxx:1064
parse
std::map< std::string, std::string > parse(const std::string &list)
Definition: egammaLayerRecalibTool.cxx:1054
picosha2::hash256_hex_string
void hash256_hex_string(InIter first, InIter last, std::string &hex_str)
Definition: picosha2.h:353
skel.it
it
Definition: skel.GENtoEVGEN.py:396
picosha2::hash256_one_by_one::finish
void finish()
Definition: picosha2.h:207
Crest::CrestFsClient::s_FS_IOV_FILE
static const std::string s_FS_IOV_FILE
Definition: CrestApiFs.h:46
TagSetDto
Definition: CrestModel.h:126
CheckTagAssociation.taglist
taglist
Definition: CheckTagAssociation.py:103
LArConditionsTestConfig.catalogFile
catalogFile
Definition: LArConditionsTestConfig.py:83
WriteCaloSwCorrections.tag_list
tag_list
Definition: WriteCaloSwCorrections.py:28
TagDto::from_json
static TagDto from_json(const json &j)
Definition: CrestModel.cxx:188
Crest::CrestFsClient::flush
void flush()
Auxiliary method flush the data to the file system.
Definition: CrestApiFs.cxx:360
GlobalTagSetDto
Definition: CrestModel.h:81
ReweightUtils.message
message
Definition: ReweightUtils.py:15
makeDTCalibBlob_pickPhase.globalTag
globalTag
Definition: makeDTCalibBlob_pickPhase.py:398
TagMetaDto
Definition: CrestModel.h:238
mergePhysValFiles.end
end
Definition: DataQuality/DataQualityUtils/scripts/mergePhysValFiles.py:93
hotSpotInTAG.objectType
objectType
Definition: hotSpotInTAG.py:107
DeMoUpdate.reverse
reverse
Definition: DeMoUpdate.py:563
Crest::CrestFsClient::getHash
std::string getHash(std::string_view str)
This method method calculates the hash for string.
Definition: CrestApiFs.cxx:722
PayloadDto
Definition: CrestModel.h:345
dq_defect_copy_defect_database.since
def since
Definition: dq_defect_copy_defect_database.py:54
dq_defect_copy_defect_database.until
def until
Definition: dq_defect_copy_defect_database.py:55
Crest::CrestFsClient::s_FS_MAP_FILE
static const std::string s_FS_MAP_FILE
Definition: CrestApiFs.h:53
PixelModuleFeMask_create_db.remove
string remove
Definition: PixelModuleFeMask_create_db.py:83
Crest::CrestFsClient::nameList
std::vector< std::string > nameList(std::string &folder, bool ascending=true)
Definition: CrestApiFs.cxx:1005
python.setupRTTAlg.size
int size
Definition: setupRTTAlg.py:39
physics_parameters.file_name
string file_name
Definition: physics_parameters.py:32
TagSetDto::resources
std::vector< TagDto > resources
Definition: CrestModel.h:128
Crest::CrestFsClient::getHashForFile
std::string getHashForFile(const std::string &file)
This method method calculates the hash for a file.
Definition: CrestApiFs.cxx:856
GlobalTagDto::from_json
static GlobalTagDto from_json(const json &j)
Definition: CrestModel.cxx:155
Crest::CrestFsClient::getFileString
std::string getFileString(const std::string &path)
Auxiliary method to get a file as a string.
Definition: CrestApiFs.cxx:60
Crest::CrestFsClient::m_root_folder
std::string m_root_folder
Definition: CrestApiFs.h:34
python.handimod.now
now
Definition: handimod.py:675
python.utils.AtlRunQueryDQUtils.p
p
Definition: AtlRunQueryDQUtils.py:210
createCoolChannelIdFile.buffer
buffer
Definition: createCoolChannelIdFile.py:12
defineDB.tagname
string tagname
Definition: JetTagCalibration/share/defineDB.py:19
Crest::CrestFsClient::s_FS_TAG_FILE
static const std::string s_FS_TAG_FILE
Definition: CrestApiFs.h:45
lumiFormat.i
int i
Definition: lumiFormat.py:85
GlobalTagDto
Definition: CrestModel.h:55
python.DecayParser.buf
buf
print ("=> [%s]"cmd)
Definition: DecayParser.py:27
Crest::CrestFsClient::getDateAndTime
std::string getDateAndTime()
Definition: CrestApiFs.cxx:711
beamspotman.n
n
Definition: beamspotman.py:731
PayloadDto::from_json
static PayloadDto from_json(const json &j)
Definition: CrestModel.cxx:627
TagDto
The TagDto class It contains all fields of the TagDto class from the CREST API.
Definition: CrestModel.h:98
mc.order
order
Configure Herwig7.
Definition: mc.Herwig7_Dijet.py:12
calibdata.exception
exception
Definition: calibdata.py:496
res
std::pair< std::vector< unsigned int >, bool > res
Definition: JetGroupProductTest.cxx:14
athena.file_path
file_path
Definition: athena.py:94
file
TFile * file
Definition: tile_monitor.h:29
Crest::CrestFsClient::s_FS_PREFIX_LENGTH
static const int s_FS_PREFIX_LENGTH
Definition: CrestApiFs.h:57
xAOD::uint64_t
uint64_t
Definition: EventInfo_v1.cxx:123
Crest::CrestFsClient::checkFsException
void checkFsException(const char *method_name)
Auxiliary method to check if the CrestApi library initialized to work with the file system.
Definition: CrestApiFs.cxx:523
MakeFileForMJB.ext
string ext
Definition: Moriond2016/MakeFileForMJB.py:41
python.TriggerAPI.TriggerAPISession.ascending
ascending
Definition: TriggerAPISession.py:435
GlobalTagMapSetDto
Definition: CrestModel.h:160
CrestBaseResponse::datatype
std::optional< std::string > datatype
Definition: CrestModel.h:45
DerivationFramework::TriggerMatchingUtils::sorted
std::vector< typename T::value_type > sorted(T begin, T end)
Helper function to create a sorted vector from an unsorted one.
picosha2::hash256_one_by_one::process
void process(RaIter first, RaIter last)
Definition: picosha2.h:196
DQPostProcessTest.outFile
outFile
Comment Out Those You do not wish to run.
Definition: DQPostProcessTest.py:37
Crest::CrestFsClient::findGlobalTag
GlobalTagDto findGlobalTag(const std::string &name) override
This method finds a global tag by name on the file storage.
Definition: CrestApiFs.cxx:138
lumiFormat.array
array
Definition: lumiFormat.py:91
Crest::CrestFsClient::getVectorPage
std::vector< std::string > getVectorPage(std::vector< std::string > data, int size, int page)
Definition: CrestApiFs.cxx:977
Crest::CrestFsClient::findTag
TagDto findTag(const std::string &name) override
This method finds a tag by the name on the file storage.
Definition: CrestApiFs.cxx:267
Crest::CrestFsClient::s_FS_TAGMETAINFO_FILE
static const std::string s_FS_TAGMETAINFO_FILE
Definition: CrestApiFs.h:47
picosha2::hash256_one_by_one
Definition: picosha2.h:184
hancool.filePath
string filePath
Definition: hancool.py:28
name
std::string name
Definition: Control/AthContainers/Root/debug.cxx:221
createCoolChannelIdFile.par
par
Definition: createCoolChannelIdFile.py:29
PayloadHelpers::dataSize
size_t dataSize(TDA::PayloadIterator start)
Size in bytes of the buffer that is needed to decode next fragment data content.
Definition: TriggerEDMDeserialiserAlg.cxx:188
item
Definition: ItemListSvc.h:43
Crest::CrestFsClient::s_FS_GLOBALTAG_FILE
static const std::string s_FS_GLOBALTAG_FILE
Definition: CrestApiFs.h:52
PixelModuleFeMask_create_db.payload
string payload
Definition: PixelModuleFeMask_create_db.py:69
Crest::CrestFsClient::isMatch
bool isMatch(std::string word, long unsigned int n, std::string pattern, long unsigned int m)
Definition: CrestApiFs.cxx:1101
Crest::CrestFsClient::getPage
nlohmann::json getPage(nlohmann::json data, int size, int page)
Definition: CrestApiFs.cxx:949
MyPlots.page
page
Definition: MyPlots.py:234
get_generator_info.version
version
Definition: get_generator_info.py:33
TagMetaDto::from_json
static TagMetaDto from_json(const json &j)
Definition: CrestModel.cxx:401
Crest::CrestFsClient::m_data
std::map< std::string, nlohmann::json > m_data
Definition: CrestApiFs.h:36
Crest::CrestFsClient::m_currentTag
std::string m_currentTag
Definition: CrestApiFs.h:58
CaloCondBlobAlgs_fillNoiseFromASCII.hash
dictionary hash
Definition: CaloCondBlobAlgs_fillNoiseFromASCII.py:109
CaloSwCorrections.time
def time(flags, cells_name, *args, **kw)
Definition: CaloSwCorrections.py:242
Crest::CrestFsClient::buildPath
std::string buildPath(const std::string &path, const std::string &file)
Definition: CrestApiFs.cxx:67
std::sort
void sort(typename std::reverse_iterator< DataModel_detail::iterator< DVL > > beg, typename std::reverse_iterator< DataModel_detail::iterator< DVL > > end, const Compare &comp)
Specialization of sort for DataVector/List.
Definition: DVL_algorithms.h:623
StoreSetDto::to_json
json to_json() const
Definition: CrestModel.cxx:576
IovSetDto
Definition: CrestModel.h:287
Crest::CrestFsClient::storePayloadDump
void storePayloadDump(const std::string &tag, uint64_t since, const std::string &js, const std::string &payloadFormat, const std::string &objectType, const std::string &compressionType, const std::string &version, const std::string &streamerInfo)
This auxiliary method stores a single payload (with since time) on the file system.
Definition: CrestApiFs.cxx:734
tagname
Definition: tagname.h:29
CaloCondBlobAlgs_fillNoiseFromASCII.folder
folder
Definition: CaloCondBlobAlgs_fillNoiseFromASCII.py:56
pickleTool.object
object
Definition: pickleTool.py:30
PlotCalibFromCool.iovList
iovList
Definition: PlotCalibFromCool.py:351
str
Definition: BTagTrackIpAccessor.cxx:11
CaloCondBlobAlgs_fillNoiseFromASCII.tag
string tag
Definition: CaloCondBlobAlgs_fillNoiseFromASCII.py:24
python.dummyaccess.exists
def exists(filename)
Definition: dummyaccess.py:9
python.PyAthena.obj
obj
Definition: PyAthena.py:132
Crest::CrestFsClient::m_isRewrite
bool m_isRewrite
Definition: CrestApiFs.h:37
length
double length(const pvec &v)
Definition: FPGATrackSimLLPDoubletHoughTransformTool.cxx:26
Crest::CrestFsClient::findAllIovs
nlohmann::json findAllIovs(const std::string &tagname)
This auxiliary method finds all iovs for a given tag name.
Definition: CrestApiFs.cxx:621