COOLRates.COOLQueryHandler Class Reference

Inheritance diagram for COOLRates.COOLQueryHandler:

Collaboration diagram for COOLRates.COOLQueryHandler:

Public Member Functions
	__init__ (self, mySignal, numProc=None, timeConverter={}, verbose=False)
	createProcesses (self)
	addQueryBundle (self, queryBundle)
	beginQuery (self)
	processResults (self, resultBundle)
	terminateProcesses (self)
	collectResults (self)

Public Attributes
	troughQueue = Queue()
	resultQueue = Queue()
list	QueryProcesses = []
dict	result = {}
int	counter = 0
	verbose = verbose
	timeConverter = timeConverter
	mySignal = mySignal
	numProc = numProc

Detailed Description

Definition at line 155 of file COOLRates.py.

Constructor & Destructor Documentation

__init__()

COOLRates.COOLQueryHandler.__init__	(		self,
			mySignal,
			numProc = None,
			timeConverter = {},
			verbose = False )

Definition at line 156 of file COOLRates.py.

    def __init__(self,mySignal,numProc=None,timeConverter={},verbose=False):
        self.troughQueue = Queue()
        self.resultQueue = Queue()
        self.QueryProcesses = []
        self.result = {}
        self.counter=0
        self.verbose = verbose
        self.timeConverter = timeConverter
        self.mySignal = mySignal
        if numProc is None: numProc = cpu_count()
        self.numProc = numProc
 

Member Function Documentation

addQueryBundle()

COOLRates.COOLQueryHandler.addQueryBundle	(		self,
			queryBundle )

Definition at line 174 of file COOLRates.py.

    def addQueryBundle(self,queryBundle):
        self.counter+=1
        self.troughQueue.put(queryBundle)
    

beginQuery()

COOLRates.COOLQueryHandler.beginQuery

(

self

)

Definition at line 178 of file COOLRates.py.

    def beginQuery(self):
        self.createProcesses()
        for i in range(self.numProc):
            self.troughQueue.put(True)
        logger.info('{0} queries distributed over {1} workers'.format(self.counter,len(self.QueryProcesses)))
        for process in self.QueryProcesses: process.start()
        try:
            self.collectResults()
        except KeyboardInterrupt:
            logger.critical('Caught signal, terminating processes')
            self.terminateProcesses()
            return
        for process in self.QueryProcesses: process.join()
        return
    

collectResults()

COOLRates.COOLQueryHandler.collectResults

(

self

)

Definition at line 211 of file COOLRates.py.

    def collectResults(self):
        finished = 0
        startingQueries = self.counter
        target=0.
        while True:
            if not self.mySignal.is_alive():
                raise KeyboardInterrupt
                break
                #sys.exit(0)
            try: resultBundle = self.resultQueue.get(False,0.001)
            except Empty:
                sleep(.001)
                continue
            if resultBundle is True:
                finished+=1
                if finished == self.numProc: break
                continue
            self.counter-=1
            percentComplete = min(100,int(round(float(startingQueries-self.counter)/startingQueries*100+.5)))
            if percentComplete>=target:
                target+=5.
                logger.info('{0}% complete'.format(percentComplete))
            self.processResults(resultBundle)
#       except KeyboardInterrupt:
#           logger.critical('Caught signal, terminating processes')
#           self.terminateProcesses()
        return
 
#====================================================================================================================
 

createProcesses()

COOLRates.COOLQueryHandler.createProcesses

(

self

)

Reimplemented in COOLRates.RateQueryHandler.

Definition at line 168 of file COOLRates.py.

    def createProcesses(self):
        self.QueryProcesses = []
        for i in range(self.numProc):       
            process = Process(target = COOLQueryWorker(timeConverter=self.timeConverter), args = (self.troughQueue,self.resultQueue,))
            self.QueryProcesses.append(process)
            

processResults()

COOLRates.COOLQueryHandler.processResults	(		self,
			resultBundle )

Definition at line 193 of file COOLRates.py.

    def processResults(self,resultBundle):
        for resultAlias in resultBundle:
            if resultAlias not in self.result: self.result[resultAlias]={}
            self.result[resultAlias].update(resultBundle[resultAlias])  
    

terminateProcesses()

COOLRates.COOLQueryHandler.terminateProcesses

(

self

)

Definition at line 198 of file COOLRates.py.

    def terminateProcesses(self):
        while 1:
            try:self.troughQueue.get(False)
            except Empty: break
        logger.critical('Queue is empty')
        for i in range(self.numProc): self.troughQueue.put(True)
        time.sleep(1)
        for process in self.QueryProcesses:
            process.terminate()
            process.join()
        logger.critical('Processes are joined')
        raise KeyboardInterrupt
 

Member Data Documentation

counter

COOLRates.COOLQueryHandler.counter = 0

Definition at line 161 of file COOLRates.py.

mySignal

COOLRates.COOLQueryHandler.mySignal = mySignal

Definition at line 164 of file COOLRates.py.

numProc

COOLRates.COOLQueryHandler.numProc = numProc

Definition at line 166 of file COOLRates.py.

QueryProcesses

COOLRates.COOLQueryHandler.QueryProcesses = []

Definition at line 159 of file COOLRates.py.

result

dict COOLRates.COOLQueryHandler.result = {}

Definition at line 160 of file COOLRates.py.

resultQueue

COOLRates.COOLQueryHandler.resultQueue = Queue()

Definition at line 158 of file COOLRates.py.

timeConverter

COOLRates.COOLQueryHandler.timeConverter = timeConverter

Definition at line 163 of file COOLRates.py.

troughQueue

COOLRates.COOLQueryHandler.troughQueue = Queue()

Definition at line 157 of file COOLRates.py.

verbose

COOLRates.COOLQueryHandler.verbose = verbose

Definition at line 162 of file COOLRates.py.

---

The documentation for this class was generated from the following file:

• COOLRates.py