python.MpUtils.SubProcessIterator Class Reference

Inheritance diagram for python.MpUtils.SubProcessIterator:

Collaboration diagram for python.MpUtils.SubProcessIterator:

Public Member Functions
def	__init__ (self, itertask, eoi='__eoi__')
def	__iter__ (self)
def	next (self)

Static Public Member Functions
def	work (iterator, master, eoi)

Public Attributes
	master
	end_of_input
	process
	started
	next

Private Member Functions
def	_start (self)
def	_empty (self, *args, **params)

Detailed Description

Instances of this class process iterators in separate processes.

Definition at line 27 of file MpUtils.py.

Constructor & Destructor Documentation

__init__()

def python.MpUtils.SubProcessIterator.__init__	(		self,
			itertask,
			eoi = '__eoi__'
	)

Create a new subprocess iterator.

itertask : some iterable task to execute in a subprocess
eoi : an end-of-iteration marker - returned from the subprocess
      to signal that iteration is complete.

Definition at line 29 of file MpUtils.py.

    def __init__(self, itertask, eoi='__eoi__'):
        """Create a new subprocess iterator.
 
        itertask : some iterable task to execute in a subprocess
        eoi : an end-of-iteration marker - returned from the subprocess
              to signal that iteration is complete.
        """
        self.client, self.master = Pipe()
        self.end_of_input = eoi
        pargs = [itertask, self.master, eoi]
        self.process = Process(target=self.work, args=pargs)
        self.started = False
 

Member Function Documentation

__iter__()

def python.MpUtils.SubProcessIterator.__iter__

(

self

)

Definition at line 53 of file MpUtils.py.

    def __iter__(self):
        if not self.started:
            self._start()
        return self
 

_empty()

def python.MpUtils.SubProcessIterator._empty (   self, *  args, **  params  )

private

Definition at line 66 of file MpUtils.py.

    def _empty(self, *args, **params):
        raise StopIteration
 

_start()

def python.MpUtils.SubProcessIterator._start (   self )

private

Definition at line 42 of file MpUtils.py.

    def _start(self):
        self.started = True
        self.process.start()
 

next()

def python.MpUtils.SubProcessIterator.next

(

self

)

Definition at line 58 of file MpUtils.py.

    def next(self):
        item = self.client.recv()
        if item != self.end_of_input:
            return item
        else:
            self.next = self._empty
            raise StopIteration
 

work()

def python.MpUtils.SubProcessIterator.work (   iterator,   master,   eoi  )

static

The actual callable that is executed in the subprocess.

Definition at line 47 of file MpUtils.py.

    def work(iterator, master, eoi):
        """The actual callable that is executed in the subprocess."""
        for chunk in iterator:
            master.send(chunk)
        master.send(eoi)
 

Member Data Documentation

end_of_input

python.MpUtils.SubProcessIterator.end_of_input

Definition at line 37 of file MpUtils.py.

master

python.MpUtils.SubProcessIterator.master

Definition at line 36 of file MpUtils.py.

next

python.MpUtils.SubProcessIterator.next

Definition at line 63 of file MpUtils.py.

process

python.MpUtils.SubProcessIterator.process

Definition at line 39 of file MpUtils.py.

started

python.MpUtils.SubProcessIterator.started

Definition at line 40 of file MpUtils.py.

---

The documentation for this class was generated from the following file:

• MpUtils.py