ATLAS Offline Software
Public Member Functions | Public Attributes | List of all members
python.TrigConfFrontier.FrontierCursor Class Reference
Inheritance diagram for python.TrigConfFrontier.FrontierCursor:
Collaboration diagram for python.TrigConfFrontier.FrontierCursor:

Public Member Functions

def __init__ (self, urls, schema, refreshFlag=False, doDecode=True, retrieveZiplevel="zip", encoding="utf-8")
 
def __str__ (self)
 
def execute (self, query, bindvars={})
 
def fetchall (self)
 
def checkResultForErrors (self)
 
def decodeResult (self)
 

Public Attributes

 urls
 
 schema
 
 refreshFlag
 
 retrieveZiplevel
 
 doDecode
 
 encoding
 
 result
 

Detailed Description

Definition at line 62 of file TrigConfFrontier.py.

Constructor & Destructor Documentation

◆ __init__()

def python.TrigConfFrontier.FrontierCursor.__init__ (   self,
  urls,
  schema,
  refreshFlag = False,
  doDecode = True,
  retrieveZiplevel = "zip",
  encoding = "utf-8" 
)

Definition at line 63 of file TrigConfFrontier.py.

63  def __init__(self, urls, schema, refreshFlag=False, doDecode=True, retrieveZiplevel="zip", encoding="utf-8"):
64  self.urls = [str(x) + "/Frontier" for x in urls]#Add /Frontier to each URL
65  self.schema = schema
66  self.refreshFlag = refreshFlag
67  self.retrieveZiplevel = retrieveZiplevel
68  self.doDecode = doDecode
69  self.encoding = encoding
70 

Member Function Documentation

◆ __str__()

def python.TrigConfFrontier.FrontierCursor.__str__ (   self)

Definition at line 71 of file TrigConfFrontier.py.

71  def __str__(self):
72  s = "Using Frontier URL: %s\n" % self.urls
73  s += "Schema: %s\n" % self.schema
74  s += "Refresh cache: %s" % self.refreshFlag
75  return s
76 

◆ checkResultForErrors()

def python.TrigConfFrontier.FrontierCursor.checkResultForErrors (   self) 
Parse the response, looking for errors

Definition at line 130 of file TrigConfFrontier.py.

130  def checkResultForErrors(self):
131  ''' Parse the response, looking for errors '''
132  from xml.dom.minidom import parseString
133  dom = parseString(self.result)
134 
135  globalError = dom.getElementsByTagName("global_error")
136  for node in globalError:
137  raise Exception(node.getAttribute("msg"))
138 
139  qualityList = dom.getElementsByTagName("quality")
140  for node in qualityList:
141  if int(node.getAttribute("error")) > 0:
142  raise Exception(node.getAttribute("message"))
143 

◆ decodeResult()

def python.TrigConfFrontier.FrontierCursor.decodeResult (   self)

Definition at line 144 of file TrigConfFrontier.py.

144  def decodeResult(self):
145  from xml.dom.minidom import parseString
146  import base64, zlib, curses.ascii, re
147  #print ("Query result:\n", self.result)
148  dom = parseString(self.result)
149  dataList = dom.getElementsByTagName("data")
150  keepalives = 0
151  result = []
152  # Control characters represent records, but I won't bother with that now,
153  # and will simply replace those by space.
154  for data in dataList:
155  for node in data.childNodes:
156  # <keepalive /> elements may be present, combined with whitespace text
157  if node.nodeName == "keepalive":
158  # this is of type Element
159  keepalives += 1
160  continue
161  # else assume of type Text
162  if node.data.strip() == "":
163  continue
164  if keepalives > 0:
165  print (keepalives, "keepalives received\n")
166  keepalives = 0
167 
168  row = base64.decodebytes(node.data.encode())
169  if self.retrieveZiplevel != "":
170  row = zlib.decompress(row).decode(self.encoding)
171 
172  #Hack to get these lines to work in python 2
173  if sys.version_info[0] < 3:
174  row = row.encode('ascii', 'xmlcharrefreplace')
175 
176  endFirstRow = row.find('\x07')
177  firstRow = row[:endFirstRow]
178  for c in firstRow:
179  if curses.ascii.isctrl(c):
180  firstRow = firstRow.replace(c, ' ')
181  fields = [x for i,x in enumerate(firstRow.split()) if i%2==0]
182  types = [x for i,x in enumerate(firstRow.split()) if i%2==1]
183  ptypes = []
184  for t in types:
185  if t.startswith("NUMBER"):
186  if ",0" in t:
187  ptypes.append(int)
188  else:
189  ptypes.append(float)
190  else:
191  ptypes.append(str)
192 
193 
194  log.debug("Fields : %r", fields)
195  log.debug("DB Types : %r", types)
196  log.debug("Python Types: %r", ptypes)
197 
198  row = row[endFirstRow+1:]
199 
200  row_h = row.rstrip('\x07')
201 
202  if 'BLOB' in types:
203  row_h = re.sub("^.*?{","{",row_h)
204 
205  row_h = row_h.replace("\x07\x06",'.nNn.\x06')
206 
207  # pattern = re.compile("\x06\x00\x00\x00.",flags=re.S)
208  #replace pattern above more restrictive version, as longerstrings in the results
209  #have a size variable in the column separate that becomes visible if the string
210  #is large enough - this then broke the prevous decoding
211  pattern = re.compile("\x06\x00\x00..",flags=re.S)
212  row_h = pattern.sub('.xXx.',row_h)
213  row_h = row_h.replace("\x86", '.xXx.')
214 
215  row_h = row_h.split('.nNn.')
216  row_h = [r.split('.xXx.') for r in row_h]
217 
218  result = []
219  for r in row_h:
220  if r[0]=='': r[0:1]=[]
221  r = tuple([t(v) for t,v in zip(ptypes,r)])
222  result.append( r )
223 
224  self.result = result
225 
226 
227 

◆ execute()

def python.TrigConfFrontier.FrontierCursor.execute (   self,
  query,
  bindvars = {} 
)

Definition at line 77 of file TrigConfFrontier.py.

77  def execute(self, query, bindvars={}):
78  if len(bindvars)>0:
79  query = replacebindvars(query,bindvars)
80 
81 
82  log.debug("Frontier URLs : %s", self.urls)
83  log.debug("Refresh cache : %s", self.refreshFlag)
84  log.debug("Query : %s", query)
85 
86  import base64, zlib, urllib.request, urllib.error, urllib.parse
87 
88  self.result = None
89 
90  for url in self.urls:
91  try:
92  compQuery = zlib.compress(query.encode("utf-8"),9)
93  base64Query = base64.binascii.b2a_base64(compQuery).decode("utf-8")
94  encQuery = base64Query.replace("+", ".").replace("\n","").replace("/","-").replace("=","_")
95  log.debug("Frontier Request : %s", encQuery)
96  frontierRequest="%s/type=frontier_request:1:DEFAULT&encoding=BLOB%s&p1=%s" % (url, self.retrieveZiplevel, encQuery)
97  request = urllib.request.Request(frontierRequest)
98  if self.refreshFlag:
99  request.add_header("pragma", "no-cache")
100 
101  frontierId = "TrigConfFrontier 1.0"
102  request.add_header("X-Frontier-Id", frontierId)
103 
104  queryStart = time.localtime()
105  log.debug("Query started: %s", time.strftime("%m/%d/%y %H:%M:%S %Z", queryStart))
106 
107  t1 = time.time()
108  result = urllib.request.urlopen(request,timeout=10).read().decode('utf-8')
109  t2 = time.time()
110 
111  queryEnd = time.localtime()
112  log.debug("Query ended: %s", time.strftime("%m/%d/%y %H:%M:%S %Z", queryEnd))
113  log.debug("Query time: %s [seconds]", (t2-t1))
114  log.debug("Result size: %i [seconds]", len(result))
115  self.result = result
116  self.checkResultForErrors()
117  return
118  except urllib.error.HTTPError:
119  log.warning("Problem with Frontier connection to %s trying next server", url)
120  except Exception as err:
121  log.warning("Problem with the request {0}".format(err))
122 
123  raise Exception("All servers failed")
124 
125 

◆ fetchall()

def python.TrigConfFrontier.FrontierCursor.fetchall (   self)

Definition at line 126 of file TrigConfFrontier.py.

126  def fetchall(self):
127  if self.doDecode: self.decodeResult()
128  return self.result
129 

Member Data Documentation

◆ doDecode

python.TrigConfFrontier.FrontierCursor.doDecode

Definition at line 68 of file TrigConfFrontier.py.

◆ encoding

python.TrigConfFrontier.FrontierCursor.encoding

Definition at line 69 of file TrigConfFrontier.py.

◆ refreshFlag

python.TrigConfFrontier.FrontierCursor.refreshFlag

Definition at line 66 of file TrigConfFrontier.py.

◆ result

python.TrigConfFrontier.FrontierCursor.result

Definition at line 88 of file TrigConfFrontier.py.

◆ retrieveZiplevel

python.TrigConfFrontier.FrontierCursor.retrieveZiplevel

Definition at line 67 of file TrigConfFrontier.py.

◆ schema

python.TrigConfFrontier.FrontierCursor.schema

Definition at line 65 of file TrigConfFrontier.py.

◆ urls

python.TrigConfFrontier.FrontierCursor.urls

Definition at line 64 of file TrigConfFrontier.py.

The documentation for this class was generated from the following file:
read
IovVectorMap_t read(const Folder &theFolder, const SelectionCriterion &choice, const unsigned int limit=10)
Definition: openCoraCool.cxx:569
replace
std::string replace(std::string s, const std::string &s2, const std::string &s3)
Definition: hcg.cxx:307
vtune_athena.format
format
Definition: vtune_athena.py:14
CaloCellPos2Ntuple.int
int
Definition: CaloCellPos2Ntuple.py:24
read_hist_ntuple.t
t
Definition: read_hist_ntuple.py:5
LArG4FSStartPointFilterLegacy.execute
execute
Definition: LArG4FSStartPointFilterLegacy.py:20
python.PerfMonSerializer.decode
def decode(s)
Definition: PerfMonSerializer.py:388
DiTauRecTools::parseString
std::vector< TString > parseString(const TString &str, const TString &delim=",")
Definition: Reconstruction/DiTauRecTools/Root/HelperFunctions.cxx:19
python.processes.powheg.ZZ.ZZ.__init__
def __init__(self, base_directory, **kwargs)
Constructor: all process options are set here.
Definition: ZZ.py:18
str
Definition: BTagTrackIpAccessor.cxx:11
python.TrigConfFrontier.replacebindvars
def replacebindvars(query, bindvars)
Definition: TrigConfFrontier.py:46
Generated on Thu Jul 4 2024 22:25:46 for ATLAS Offline Software by   doxygen 1.8.18