Inheritance diagram for python.virtual_mixin.DefectsDBVirtualDefectsMixin:

Collaboration diagram for python.virtual_mixin.DefectsDBVirtualDefectsMixin:

Public Types
typedef HLT::TypeInformation::for_each_type_c< typenameEDMLIST::map, my_functor, my_result<>, my_arg< HLT::TypeInformation::get_cont, CONTAINER > >::type	result

Public Member Functions
None	__init__ (self)
None	validate_clauses (self, str clauses)
None	update_virtual_defect (self, str defect_name, str clauses, Optional[str] comment=None)
None	rename_defect (self, str defect_name, str new_defect_name)
int	new_virtual_defect (self, str defect_name, Optional[str] comment, str clauses)
Set[str]	resolve_primary_defects (self, Iterable[DefectLogic] defect_logics)
Mapping[str, DefectLogic]	virtual_defect_logics (self)
	get_intolerable_defects (self, bool old_primary_only=True, Sequence[str] exclude=['TIGHT', 'IDTIGHT', 'PHYS_.*'])

Public Attributes
str	logics_tag = "HEAD", NONHEAD_MODIFICATION_MSG
	logics_tags

Protected Member Functions
None	_update_virtual_defect (self, str defect_name, str clauses, Optional[str] comment=None, Optional[str] tag=None)
List[DefectLogic]	_resolve_evaluation_order (self, Optional[Iterable[str]] defects=None)
Mapping[str, DefectLogic]	_get_virtual_defect_logics (self, str tag)
bool	_virtual_defect_consistency_check (self)

Protected Attributes
Mapping[str, DefectLogic]	_virtual_defect_logics = None
	_read_only
	_defect_logic_payload
	_virtual_defect_names = set()
dict	_virtual_defect_id_map = {}
bool	_virtual_initialized = False
	_defect_names = set()
dict	_defect_id_map = {}
bool	_initialized = False

Detailed Description

A DefectsDB mixin for managing virtual defects

Definition at line 39 of file virtual_mixin.py.

Member Typedef Documentation

◆ result

typedef HLT::TypeInformation::for_each_type_c<typenameEDMLIST::map,my_functor,my_result<>,my_arg<HLT::TypeInformation::get_cont,CONTAINER>>::type master_search< typenameEDMLIST::map, HLT::TypeInformation::get_cont, CONTAINER >::result

inherited

Definition at line 90 of file EDM_MasterSearch.h.

Constructor & Destructor Documentation

◆ init()

None python.virtual_mixin.DefectsDBVirtualDefectsMixin.__init__ ( self )

Definition at line 44 of file virtual_mixin.py.

    def __init__(self) -> None:
        self._virtual_defect_logics = None
        super(DefectsDBVirtualDefectsMixin, self).__init__()
 

Member Function Documentation

◆ _get_virtual_defect_logics()

Mapping[str, DefectLogic] python.virtual_mixin.DefectsDBVirtualDefectsMixin._get_virtual_defect_logics	(		self,
		str	tag )

protected

Definition at line 232 of file virtual_mixin.py.

    def _get_virtual_defect_logics(self, tag: str) -> Mapping[str, DefectLogic]:
        if tag != "HEAD" and not self.defect_logic_folder.existsUserTag(tag.encode('ascii')):
            # The tag doesn't exist, so there is nothing to fetch.
            return {}
        
        logics = fetch_iovs(self.defect_logic_folder, tag=tag.encode('ascii'), 
                            named_channels=True)
 
        logics = {l.channel: DefectLogic(l) for l in logics}
        
        for _, defect_logic in logics.items():
            defect_logic._populate(logics)
            
        return logics
 

◆ _resolve_evaluation_order()

List[DefectLogic] python.virtual_mixin.DefectsDBVirtualDefectsMixin._resolve_evaluation_order	(		self,
		Optional[Iterable[str]]	defects = None )

protected

Returns a list of DefectLogic objects that need to be evaluated, 
in the correct order for them to be consistent.

`defects` should be a list of names

Definition at line 183 of file virtual_mixin.py.

    def _resolve_evaluation_order(self, defects: Optional[Iterable[str]] = None) -> List[DefectLogic]:
        """
        Returns a list of DefectLogic objects that need to be evaluated, 
        in the correct order for them to be consistent.
        
        `defects` should be a list of names
        """
        if defects is None:
            logics = self.virtual_defect_logics.items()
        else:
            # flags begins as a list of names.
            # Make it a list of logic objects instead
            # If it is not in the logics map, the logic is not defined for the
            # query interval and we shouldn't try to resolve it.
            
            
            logics = [self.virtual_defect_logics[d] for d in defects]
        
        class MasterNode(object):
            dependencies = logics
            
        resolved = _resolve_dependencies(MasterNode)
        resolved.remove(MasterNode)
        return resolved
    

◆ _update_virtual_defect()

None python.virtual_mixin.DefectsDBVirtualDefectsMixin._update_virtual_defect	(		self,
		str	defect_name,
		str	clauses,
		Optional[str]	comment = None,
		Optional[str]	tag = None )

protected

Definition at line 62 of file virtual_mixin.py.

                               comment: Optional[str] = None, tag: Optional[str] = None) -> None:
        defect_id = self.defect_chan_as_id(defect_name)
        
        assert self.defect_is_virtual(defect_id), ("Tried to update nonvirtual"
            " defect with update_virtual_defect()")
 
        self.validate_clauses(clauses)
 
        tag = tag if tag is not None else 'HEAD'
        ucomment = comment.encode('utf-8') if comment is not None else None
 
        self._defect_logic_payload["clauses"] = clauses.encode('ascii')
        
        store = self.defect_logic_folder.storeObject
        store(0, 2**63-1, self._defect_logic_payload, defect_id, tag.encode('ascii'),
              (True if tag != 'HEAD' else False))
        
        if comment is not None:
            self.defect_logic_folder.setChannelDescription(defect_id, ucomment)
 

◆ _virtual_defect_consistency_check()

bool python.virtual_mixin.DefectsDBVirtualDefectsMixin._virtual_defect_consistency_check ( self )

protected

When called, uses an assertion to check that there are no missing 
defects. This is a database consistency check which should never be
violated (but was when DQDefects was new).

Definition at line 265 of file virtual_mixin.py.

    def _virtual_defect_consistency_check(self) -> bool:
        """
        When called, uses an assertion to check that there are no missing 
        defects. This is a database consistency check which should never be
        violated (but was when DQDefects was new).
        """
        all_defects = self.defect_names | self.virtual_defect_names
        used_defects = set()
        
        used_by = {}
        for defect, logic in self.virtual_defect_logics.items():
            used_defects.update(logic.realclauses)
            for clause in logic.realclauses:
                used_by.setdefault(clause, []).append(defect)
                
        missing_defects = used_defects - all_defects
        
        if missing_defects:
            log.error("-- The missing defects are used by the following virtual defects")
            log.error("<missing> : <used by>")
            for missing in missing_defects:
                log.error(missing + ":" + ", ".join(used_by[missing]))
            
        assert not missing_defects, ("The following defects are used but not "
                                     "defined anywhere: %r" % missing_defects)
        return True

◆ get_intolerable_defects()

python.virtual_mixin.DefectsDBVirtualDefectsMixin.get_intolerable_defects	(		self,
		bool	old_primary_only = True,
		Sequence[str]	exclude = ['TIGHT', 'IDTIGHT', 'PHYS_.*'] )

Returns primary defects that are depended on by a virtual defect
if old_primary_only == True, only return those depended on by a
virtual defect with no _ in the name
ignores virtual defects in exclude list
accepts regexes in exclude list

Definition at line 247 of file virtual_mixin.py.

                                exclude: Sequence[str] = ['TIGHT', 'IDTIGHT', 'PHYS_.*']):
        """
        Returns primary defects that are depended on by a virtual defect
        if old_primary_only == True, only return those depended on by a
        virtual defect with no _ in the name
        ignores virtual defects in exclude list
        accepts regexes in exclude list
        """
        vdl = self.virtual_defect_logics
        if old_primary_only:
            vdl = dict(l for l in vdl.items() if '_' not in l[0])
        if len(exclude) > 0:
            import re
            rex = re.compile('(%s)' % '|'.join(exclude))
            vdl = dict(l for l in vdl.items() if not rex.match(l[0]))
        return self.resolve_primary_defects(vdl.values())
        

◆ new_virtual_defect()

int python.virtual_mixin.DefectsDBVirtualDefectsMixin.new_virtual_defect	(		self,
		str	defect_name,
		Optional[str]	comment,
		str	clauses )

Create a new virtual defect

Definition at line 148 of file virtual_mixin.py.

    def new_virtual_defect(self, defect_name: str, comment: Optional[str], clauses: str) -> int:
        """
        Create a new virtual defect
        """
        assert self.logics_tag == "HEAD", NONHEAD_MODIFICATION_MSG
        assert not self._read_only, "Insertion on read-only database"
        from DQUtils.channel_mapping import get_channel_ids_names
        
        # Force load of defects_folder to populate _defect_payload
        store = self.defect_logic_folder.storeObject
        p = self._defect_logic_payload
        
        self.validate_clauses(clauses)
 
        p["clauses"] = clauses.encode('ascii')
 
        # need to get true mapping of folder, not just what is visible
        # from this tag
        ids, _, mapping = get_channel_ids_names(self.defect_logic_folder)
        defect_id = choose_new_defect_id(mapping, defect_name, True)
        log.debug("Creating new defect %s: 0x%08x", defect_name, defect_id)
        
        try:
            oldname = self.normalize_defect_names(defect_name)
            raise DefectExistsError('Defect %s already exists' % oldname)
        except DefectUnknownError:
            pass
 
        if (defect_id in ids):
            raise DefectExistsError(defect_name)
        self.defect_logic_folder.createChannel(defect_id, defect_name.encode('ascii'), comment.encode('utf-8'))
        store(0, 2**63-1, p, defect_id)
        self._new_virtual_defect(defect_id, defect_name)
        return defect_id
    

◆ rename_defect()

None python.virtual_mixin.DefectsDBVirtualDefectsMixin.rename_defect	(		self,
		str	defect_name,
		str	new_defect_name )

Rename a defect (primary or virtual).  Will keep data and channel ID.
Will fix up all virtual defect dependencies in all tags.

Definition at line 83 of file virtual_mixin.py.

    def rename_defect(self, defect_name: str, new_defect_name: str) -> None:
        """
        Rename a defect (primary or virtual).  Will keep data and channel ID.
        Will fix up all virtual defect dependencies in all tags.
        """
        assert not self._read_only, "Channel rename on read-only database"
        
        try:
            oldname = self.normalize_defect_names(new_defect_name)
            raise DefectExistsError('Defect %s already exists' % oldname)
        except DefectUnknownError:
            pass
        
        defect_id = self.defect_chan_as_id(defect_name)
        
        anew_defect_name = new_defect_name.encode('ascii')
        if self.defect_is_virtual(defect_id):
            self.defect_logic_folder.setChannelName(defect_id, anew_defect_name)
            self._virtual_defect_names = set()
            self._virtual_defect_id_map = {}
            self._virtual_defect_logics = None
            self._virtual_initialized = False
        else:
            self.defects_folder.setChannelName(defect_id, anew_defect_name)
            self._defect_names = set()
            self._defect_id_map = {}
            self._initialized = False
        
        import contextlib
        @contextlib.contextmanager
        def logics_unlocking(tag):
            atag = tag.encode('ascii')
            log.info('Unlocking tag %s', tag)
            folder = self.defect_logic_folder
            orig_status = folder.tagLockStatus(atag)
            if orig_status != 0:
                folder.setTagLockStatus(atag, 0)
            yield
            if orig_status != 0:
                folder.setTagLockStatus(atag, orig_status)
            log.info('Done with tag %s', tag)
 
        def logic_substitute(clause, oldname, newname):
            oclause = clause
            prefix = ''
            if clause[0] in ('!',):
                prefix = clause[0]
                clause = clause[1:]
            if clause != oldname:
                return oclause
            else:
                return prefix + newname
        
        for tag in ['HEAD'] + self.logics_tags:
            logics = self._get_virtual_defect_logics(tag)
            for vd, logic in logics.items():
                if defect_name in logic.realclauses:
                    newclauses = ' '.join([logic_substitute(x, defect_name,
                                                            new_defect_name)
                                           for x in logic.clauses
                                           ])
                    with logics_unlocking(tag):
                        self._update_virtual_defect(vd, newclauses, tag=tag)
        self._virtual_defect_logics = None
    

◆ resolve_primary_defects()

Set[str] python.virtual_mixin.DefectsDBVirtualDefectsMixin.resolve_primary_defects	(		self,
		Iterable[DefectLogic]	defect_logics )

Determine which primary flags are used for 
a given list of input `virtual_defect_names`.

Definition at line 208 of file virtual_mixin.py.

    def resolve_primary_defects(self, defect_logics: Iterable[DefectLogic]) -> Set[str]:
        """
        Determine which primary flags are used for 
        a given list of input `virtual_defect_names`.
        """
        primary_defects = set()
        for defect_logic in defect_logics:
            primary_defects |= defect_logic.primary_defects
            
        return primary_defects
    

◆ update_virtual_defect()

None python.virtual_mixin.DefectsDBVirtualDefectsMixin.update_virtual_defect	(		self,
		str	defect_name,
		str	clauses,
		Optional[str]	comment = None )

Definition at line 55 of file virtual_mixin.py.

    def update_virtual_defect(self, defect_name: str, clauses: str, comment: Optional[str] = None) -> None:
        # HEAD protection is here, to allow other functions to
        # change other tags in a known manner with _update_virtual_defect
        assert self.logics_tag == "HEAD", NONHEAD_MODIFICATION_MSG
        assert not self._read_only, "Insertion on read-only database"
        self._update_virtual_defect(defect_name, clauses, comment)
    

◆ validate_clauses()

None python.virtual_mixin.DefectsDBVirtualDefectsMixin.validate_clauses	(		self,
		str	clauses )

Definition at line 48 of file virtual_mixin.py.

    def validate_clauses(self, clauses: str) -> None:
        all_defects = self.defect_names | self.virtual_defect_names
        for clause in clauses.split():
            if clause[0] in ('!', '-'):
                clause = clause[1:]
            assert clause in all_defects, (clause + " is not a known defect")
    

◆ virtual_defect_logics()

Mapping[str, DefectLogic] python.virtual_mixin.DefectsDBVirtualDefectsMixin.virtual_defect_logics ( self )

Returns all virtual defects in the form {"defect_name" : DefectLogic()}
for the tag DefectDB was constructed on.

Definition at line 220 of file virtual_mixin.py.

    def virtual_defect_logics(self) -> Mapping[str, DefectLogic]:
        """
        Returns all virtual defects in the form {"defect_name" : DefectLogic()}
        for the tag DefectDB was constructed on.
        """
        if self._virtual_defect_logics is not None:
            return self._virtual_defect_logics
                
        self._virtual_defect_logics = self._get_virtual_defect_logics(self.logics_tag)
        
        return self._virtual_defect_logics
 

Member Data Documentation

◆ _defect_id_map

dict python.virtual_mixin.DefectsDBVirtualDefectsMixin._defect_id_map = {}

protected

Definition at line 108 of file virtual_mixin.py.

◆ _defect_logic_payload

python.virtual_mixin.DefectsDBVirtualDefectsMixin._defect_logic_payload

protected

Definition at line 77 of file virtual_mixin.py.

◆ _defect_names

python.virtual_mixin.DefectsDBVirtualDefectsMixin._defect_names = set()

protected

Definition at line 107 of file virtual_mixin.py.

◆ _initialized

bool python.virtual_mixin.DefectsDBVirtualDefectsMixin._initialized = False

protected

Definition at line 109 of file virtual_mixin.py.

◆ _read_only

python.virtual_mixin.DefectsDBVirtualDefectsMixin._read_only

protected

Definition at line 59 of file virtual_mixin.py.

◆ _virtual_defect_id_map

dict python.virtual_mixin.DefectsDBVirtualDefectsMixin._virtual_defect_id_map = {}

protected

Definition at line 102 of file virtual_mixin.py.

◆ _virtual_defect_logics

Mapping[str, DefectLogic] python.virtual_mixin.DefectsDBVirtualDefectsMixin._virtual_defect_logics = None

protected

Definition at line 45 of file virtual_mixin.py.

◆ _virtual_defect_names

python.virtual_mixin.DefectsDBVirtualDefectsMixin._virtual_defect_names = set()

protected

Definition at line 101 of file virtual_mixin.py.

◆ _virtual_initialized

bool python.virtual_mixin.DefectsDBVirtualDefectsMixin._virtual_initialized = False

protected

Definition at line 104 of file virtual_mixin.py.

◆ logics_tag

str python.virtual_mixin.DefectsDBVirtualDefectsMixin.logics_tag = "HEAD", NONHEAD_MODIFICATION_MSG

Definition at line 58 of file virtual_mixin.py.

◆ logics_tags

python.virtual_mixin.DefectsDBVirtualDefectsMixin.logics_tags

Definition at line 136 of file virtual_mixin.py.

The documentation for this class was generated from the following file:

virtual_mixin.py

Public Types

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Detailed Description

Member Typedef Documentation

◆ result

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ _get_virtual_defect_logics()

◆ _resolve_evaluation_order()

◆ _update_virtual_defect()

◆ _virtual_defect_consistency_check()

◆ get_intolerable_defects()

◆ new_virtual_defect()

◆ rename_defect()

◆ resolve_primary_defects()

◆ update_virtual_defect()

◆ validate_clauses()

◆ virtual_defect_logics()

Member Data Documentation

◆ _defect_id_map

◆ _defect_logic_payload

◆ _defect_names

◆ _initialized

◆ _read_only

◆ _virtual_defect_id_map

◆ _virtual_defect_logics

◆ _virtual_defect_names

◆ _virtual_initialized

◆ logics_tag

◆ logics_tags

◆ init()