1*cda5da8dSAndroid Build Coastguard Workerfrom types import GenericAlias 2*cda5da8dSAndroid Build Coastguard Worker 3*cda5da8dSAndroid Build Coastguard Worker__all__ = ["TopologicalSorter", "CycleError"] 4*cda5da8dSAndroid Build Coastguard Worker 5*cda5da8dSAndroid Build Coastguard Worker_NODE_OUT = -1 6*cda5da8dSAndroid Build Coastguard Worker_NODE_DONE = -2 7*cda5da8dSAndroid Build Coastguard Worker 8*cda5da8dSAndroid Build Coastguard Worker 9*cda5da8dSAndroid Build Coastguard Workerclass _NodeInfo: 10*cda5da8dSAndroid Build Coastguard Worker __slots__ = "node", "npredecessors", "successors" 11*cda5da8dSAndroid Build Coastguard Worker 12*cda5da8dSAndroid Build Coastguard Worker def __init__(self, node): 13*cda5da8dSAndroid Build Coastguard Worker # The node this class is augmenting. 14*cda5da8dSAndroid Build Coastguard Worker self.node = node 15*cda5da8dSAndroid Build Coastguard Worker 16*cda5da8dSAndroid Build Coastguard Worker # Number of predecessors, generally >= 0. When this value falls to 0, 17*cda5da8dSAndroid Build Coastguard Worker # and is returned by get_ready(), this is set to _NODE_OUT and when the 18*cda5da8dSAndroid Build Coastguard Worker # node is marked done by a call to done(), set to _NODE_DONE. 19*cda5da8dSAndroid Build Coastguard Worker self.npredecessors = 0 20*cda5da8dSAndroid Build Coastguard Worker 21*cda5da8dSAndroid Build Coastguard Worker # List of successor nodes. The list can contain duplicated elements as 22*cda5da8dSAndroid Build Coastguard Worker # long as they're all reflected in the successor's npredecessors attribute. 23*cda5da8dSAndroid Build Coastguard Worker self.successors = [] 24*cda5da8dSAndroid Build Coastguard Worker 25*cda5da8dSAndroid Build Coastguard Worker 26*cda5da8dSAndroid Build Coastguard Workerclass CycleError(ValueError): 27*cda5da8dSAndroid Build Coastguard Worker """Subclass of ValueError raised by TopologicalSorter.prepare if cycles 28*cda5da8dSAndroid Build Coastguard Worker exist in the working graph. 29*cda5da8dSAndroid Build Coastguard Worker 30*cda5da8dSAndroid Build Coastguard Worker If multiple cycles exist, only one undefined choice among them will be reported 31*cda5da8dSAndroid Build Coastguard Worker and included in the exception. The detected cycle can be accessed via the second 32*cda5da8dSAndroid Build Coastguard Worker element in the *args* attribute of the exception instance and consists in a list 33*cda5da8dSAndroid Build Coastguard Worker of nodes, such that each node is, in the graph, an immediate predecessor of the 34*cda5da8dSAndroid Build Coastguard Worker next node in the list. In the reported list, the first and the last node will be 35*cda5da8dSAndroid Build Coastguard Worker the same, to make it clear that it is cyclic. 36*cda5da8dSAndroid Build Coastguard Worker """ 37*cda5da8dSAndroid Build Coastguard Worker 38*cda5da8dSAndroid Build Coastguard Worker pass 39*cda5da8dSAndroid Build Coastguard Worker 40*cda5da8dSAndroid Build Coastguard Worker 41*cda5da8dSAndroid Build Coastguard Workerclass TopologicalSorter: 42*cda5da8dSAndroid Build Coastguard Worker """Provides functionality to topologically sort a graph of hashable nodes""" 43*cda5da8dSAndroid Build Coastguard Worker 44*cda5da8dSAndroid Build Coastguard Worker def __init__(self, graph=None): 45*cda5da8dSAndroid Build Coastguard Worker self._node2info = {} 46*cda5da8dSAndroid Build Coastguard Worker self._ready_nodes = None 47*cda5da8dSAndroid Build Coastguard Worker self._npassedout = 0 48*cda5da8dSAndroid Build Coastguard Worker self._nfinished = 0 49*cda5da8dSAndroid Build Coastguard Worker 50*cda5da8dSAndroid Build Coastguard Worker if graph is not None: 51*cda5da8dSAndroid Build Coastguard Worker for node, predecessors in graph.items(): 52*cda5da8dSAndroid Build Coastguard Worker self.add(node, *predecessors) 53*cda5da8dSAndroid Build Coastguard Worker 54*cda5da8dSAndroid Build Coastguard Worker def _get_nodeinfo(self, node): 55*cda5da8dSAndroid Build Coastguard Worker if (result := self._node2info.get(node)) is None: 56*cda5da8dSAndroid Build Coastguard Worker self._node2info[node] = result = _NodeInfo(node) 57*cda5da8dSAndroid Build Coastguard Worker return result 58*cda5da8dSAndroid Build Coastguard Worker 59*cda5da8dSAndroid Build Coastguard Worker def add(self, node, *predecessors): 60*cda5da8dSAndroid Build Coastguard Worker """Add a new node and its predecessors to the graph. 61*cda5da8dSAndroid Build Coastguard Worker 62*cda5da8dSAndroid Build Coastguard Worker Both the *node* and all elements in *predecessors* must be hashable. 63*cda5da8dSAndroid Build Coastguard Worker 64*cda5da8dSAndroid Build Coastguard Worker If called multiple times with the same node argument, the set of dependencies 65*cda5da8dSAndroid Build Coastguard Worker will be the union of all dependencies passed in. 66*cda5da8dSAndroid Build Coastguard Worker 67*cda5da8dSAndroid Build Coastguard Worker It is possible to add a node with no dependencies (*predecessors* is not provided) 68*cda5da8dSAndroid Build Coastguard Worker as well as provide a dependency twice. If a node that has not been provided before 69*cda5da8dSAndroid Build Coastguard Worker is included among *predecessors* it will be automatically added to the graph with 70*cda5da8dSAndroid Build Coastguard Worker no predecessors of its own. 71*cda5da8dSAndroid Build Coastguard Worker 72*cda5da8dSAndroid Build Coastguard Worker Raises ValueError if called after "prepare". 73*cda5da8dSAndroid Build Coastguard Worker """ 74*cda5da8dSAndroid Build Coastguard Worker if self._ready_nodes is not None: 75*cda5da8dSAndroid Build Coastguard Worker raise ValueError("Nodes cannot be added after a call to prepare()") 76*cda5da8dSAndroid Build Coastguard Worker 77*cda5da8dSAndroid Build Coastguard Worker # Create the node -> predecessor edges 78*cda5da8dSAndroid Build Coastguard Worker nodeinfo = self._get_nodeinfo(node) 79*cda5da8dSAndroid Build Coastguard Worker nodeinfo.npredecessors += len(predecessors) 80*cda5da8dSAndroid Build Coastguard Worker 81*cda5da8dSAndroid Build Coastguard Worker # Create the predecessor -> node edges 82*cda5da8dSAndroid Build Coastguard Worker for pred in predecessors: 83*cda5da8dSAndroid Build Coastguard Worker pred_info = self._get_nodeinfo(pred) 84*cda5da8dSAndroid Build Coastguard Worker pred_info.successors.append(node) 85*cda5da8dSAndroid Build Coastguard Worker 86*cda5da8dSAndroid Build Coastguard Worker def prepare(self): 87*cda5da8dSAndroid Build Coastguard Worker """Mark the graph as finished and check for cycles in the graph. 88*cda5da8dSAndroid Build Coastguard Worker 89*cda5da8dSAndroid Build Coastguard Worker If any cycle is detected, "CycleError" will be raised, but "get_ready" can 90*cda5da8dSAndroid Build Coastguard Worker still be used to obtain as many nodes as possible until cycles block more 91*cda5da8dSAndroid Build Coastguard Worker progress. After a call to this function, the graph cannot be modified and 92*cda5da8dSAndroid Build Coastguard Worker therefore no more nodes can be added using "add". 93*cda5da8dSAndroid Build Coastguard Worker """ 94*cda5da8dSAndroid Build Coastguard Worker if self._ready_nodes is not None: 95*cda5da8dSAndroid Build Coastguard Worker raise ValueError("cannot prepare() more than once") 96*cda5da8dSAndroid Build Coastguard Worker 97*cda5da8dSAndroid Build Coastguard Worker self._ready_nodes = [ 98*cda5da8dSAndroid Build Coastguard Worker i.node for i in self._node2info.values() if i.npredecessors == 0 99*cda5da8dSAndroid Build Coastguard Worker ] 100*cda5da8dSAndroid Build Coastguard Worker # ready_nodes is set before we look for cycles on purpose: 101*cda5da8dSAndroid Build Coastguard Worker # if the user wants to catch the CycleError, that's fine, 102*cda5da8dSAndroid Build Coastguard Worker # they can continue using the instance to grab as many 103*cda5da8dSAndroid Build Coastguard Worker # nodes as possible before cycles block more progress 104*cda5da8dSAndroid Build Coastguard Worker cycle = self._find_cycle() 105*cda5da8dSAndroid Build Coastguard Worker if cycle: 106*cda5da8dSAndroid Build Coastguard Worker raise CycleError(f"nodes are in a cycle", cycle) 107*cda5da8dSAndroid Build Coastguard Worker 108*cda5da8dSAndroid Build Coastguard Worker def get_ready(self): 109*cda5da8dSAndroid Build Coastguard Worker """Return a tuple of all the nodes that are ready. 110*cda5da8dSAndroid Build Coastguard Worker 111*cda5da8dSAndroid Build Coastguard Worker Initially it returns all nodes with no predecessors; once those are marked 112*cda5da8dSAndroid Build Coastguard Worker as processed by calling "done", further calls will return all new nodes that 113*cda5da8dSAndroid Build Coastguard Worker have all their predecessors already processed. Once no more progress can be made, 114*cda5da8dSAndroid Build Coastguard Worker empty tuples are returned. 115*cda5da8dSAndroid Build Coastguard Worker 116*cda5da8dSAndroid Build Coastguard Worker Raises ValueError if called without calling "prepare" previously. 117*cda5da8dSAndroid Build Coastguard Worker """ 118*cda5da8dSAndroid Build Coastguard Worker if self._ready_nodes is None: 119*cda5da8dSAndroid Build Coastguard Worker raise ValueError("prepare() must be called first") 120*cda5da8dSAndroid Build Coastguard Worker 121*cda5da8dSAndroid Build Coastguard Worker # Get the nodes that are ready and mark them 122*cda5da8dSAndroid Build Coastguard Worker result = tuple(self._ready_nodes) 123*cda5da8dSAndroid Build Coastguard Worker n2i = self._node2info 124*cda5da8dSAndroid Build Coastguard Worker for node in result: 125*cda5da8dSAndroid Build Coastguard Worker n2i[node].npredecessors = _NODE_OUT 126*cda5da8dSAndroid Build Coastguard Worker 127*cda5da8dSAndroid Build Coastguard Worker # Clean the list of nodes that are ready and update 128*cda5da8dSAndroid Build Coastguard Worker # the counter of nodes that we have returned. 129*cda5da8dSAndroid Build Coastguard Worker self._ready_nodes.clear() 130*cda5da8dSAndroid Build Coastguard Worker self._npassedout += len(result) 131*cda5da8dSAndroid Build Coastguard Worker 132*cda5da8dSAndroid Build Coastguard Worker return result 133*cda5da8dSAndroid Build Coastguard Worker 134*cda5da8dSAndroid Build Coastguard Worker def is_active(self): 135*cda5da8dSAndroid Build Coastguard Worker """Return ``True`` if more progress can be made and ``False`` otherwise. 136*cda5da8dSAndroid Build Coastguard Worker 137*cda5da8dSAndroid Build Coastguard Worker Progress can be made if cycles do not block the resolution and either there 138*cda5da8dSAndroid Build Coastguard Worker are still nodes ready that haven't yet been returned by "get_ready" or the 139*cda5da8dSAndroid Build Coastguard Worker number of nodes marked "done" is less than the number that have been returned 140*cda5da8dSAndroid Build Coastguard Worker by "get_ready". 141*cda5da8dSAndroid Build Coastguard Worker 142*cda5da8dSAndroid Build Coastguard Worker Raises ValueError if called without calling "prepare" previously. 143*cda5da8dSAndroid Build Coastguard Worker """ 144*cda5da8dSAndroid Build Coastguard Worker if self._ready_nodes is None: 145*cda5da8dSAndroid Build Coastguard Worker raise ValueError("prepare() must be called first") 146*cda5da8dSAndroid Build Coastguard Worker return self._nfinished < self._npassedout or bool(self._ready_nodes) 147*cda5da8dSAndroid Build Coastguard Worker 148*cda5da8dSAndroid Build Coastguard Worker def __bool__(self): 149*cda5da8dSAndroid Build Coastguard Worker return self.is_active() 150*cda5da8dSAndroid Build Coastguard Worker 151*cda5da8dSAndroid Build Coastguard Worker def done(self, *nodes): 152*cda5da8dSAndroid Build Coastguard Worker """Marks a set of nodes returned by "get_ready" as processed. 153*cda5da8dSAndroid Build Coastguard Worker 154*cda5da8dSAndroid Build Coastguard Worker This method unblocks any successor of each node in *nodes* for being returned 155*cda5da8dSAndroid Build Coastguard Worker in the future by a call to "get_ready". 156*cda5da8dSAndroid Build Coastguard Worker 157*cda5da8dSAndroid Build Coastguard Worker Raises :exec:`ValueError` if any node in *nodes* has already been marked as 158*cda5da8dSAndroid Build Coastguard Worker processed by a previous call to this method, if a node was not added to the 159*cda5da8dSAndroid Build Coastguard Worker graph by using "add" or if called without calling "prepare" previously or if 160*cda5da8dSAndroid Build Coastguard Worker node has not yet been returned by "get_ready". 161*cda5da8dSAndroid Build Coastguard Worker """ 162*cda5da8dSAndroid Build Coastguard Worker 163*cda5da8dSAndroid Build Coastguard Worker if self._ready_nodes is None: 164*cda5da8dSAndroid Build Coastguard Worker raise ValueError("prepare() must be called first") 165*cda5da8dSAndroid Build Coastguard Worker 166*cda5da8dSAndroid Build Coastguard Worker n2i = self._node2info 167*cda5da8dSAndroid Build Coastguard Worker 168*cda5da8dSAndroid Build Coastguard Worker for node in nodes: 169*cda5da8dSAndroid Build Coastguard Worker 170*cda5da8dSAndroid Build Coastguard Worker # Check if we know about this node (it was added previously using add() 171*cda5da8dSAndroid Build Coastguard Worker if (nodeinfo := n2i.get(node)) is None: 172*cda5da8dSAndroid Build Coastguard Worker raise ValueError(f"node {node!r} was not added using add()") 173*cda5da8dSAndroid Build Coastguard Worker 174*cda5da8dSAndroid Build Coastguard Worker # If the node has not being returned (marked as ready) previously, inform the user. 175*cda5da8dSAndroid Build Coastguard Worker stat = nodeinfo.npredecessors 176*cda5da8dSAndroid Build Coastguard Worker if stat != _NODE_OUT: 177*cda5da8dSAndroid Build Coastguard Worker if stat >= 0: 178*cda5da8dSAndroid Build Coastguard Worker raise ValueError( 179*cda5da8dSAndroid Build Coastguard Worker f"node {node!r} was not passed out (still not ready)" 180*cda5da8dSAndroid Build Coastguard Worker ) 181*cda5da8dSAndroid Build Coastguard Worker elif stat == _NODE_DONE: 182*cda5da8dSAndroid Build Coastguard Worker raise ValueError(f"node {node!r} was already marked done") 183*cda5da8dSAndroid Build Coastguard Worker else: 184*cda5da8dSAndroid Build Coastguard Worker assert False, f"node {node!r}: unknown status {stat}" 185*cda5da8dSAndroid Build Coastguard Worker 186*cda5da8dSAndroid Build Coastguard Worker # Mark the node as processed 187*cda5da8dSAndroid Build Coastguard Worker nodeinfo.npredecessors = _NODE_DONE 188*cda5da8dSAndroid Build Coastguard Worker 189*cda5da8dSAndroid Build Coastguard Worker # Go to all the successors and reduce the number of predecessors, collecting all the ones 190*cda5da8dSAndroid Build Coastguard Worker # that are ready to be returned in the next get_ready() call. 191*cda5da8dSAndroid Build Coastguard Worker for successor in nodeinfo.successors: 192*cda5da8dSAndroid Build Coastguard Worker successor_info = n2i[successor] 193*cda5da8dSAndroid Build Coastguard Worker successor_info.npredecessors -= 1 194*cda5da8dSAndroid Build Coastguard Worker if successor_info.npredecessors == 0: 195*cda5da8dSAndroid Build Coastguard Worker self._ready_nodes.append(successor) 196*cda5da8dSAndroid Build Coastguard Worker self._nfinished += 1 197*cda5da8dSAndroid Build Coastguard Worker 198*cda5da8dSAndroid Build Coastguard Worker def _find_cycle(self): 199*cda5da8dSAndroid Build Coastguard Worker n2i = self._node2info 200*cda5da8dSAndroid Build Coastguard Worker stack = [] 201*cda5da8dSAndroid Build Coastguard Worker itstack = [] 202*cda5da8dSAndroid Build Coastguard Worker seen = set() 203*cda5da8dSAndroid Build Coastguard Worker node2stacki = {} 204*cda5da8dSAndroid Build Coastguard Worker 205*cda5da8dSAndroid Build Coastguard Worker for node in n2i: 206*cda5da8dSAndroid Build Coastguard Worker if node in seen: 207*cda5da8dSAndroid Build Coastguard Worker continue 208*cda5da8dSAndroid Build Coastguard Worker 209*cda5da8dSAndroid Build Coastguard Worker while True: 210*cda5da8dSAndroid Build Coastguard Worker if node in seen: 211*cda5da8dSAndroid Build Coastguard Worker # If we have seen already the node and is in the 212*cda5da8dSAndroid Build Coastguard Worker # current stack we have found a cycle. 213*cda5da8dSAndroid Build Coastguard Worker if node in node2stacki: 214*cda5da8dSAndroid Build Coastguard Worker return stack[node2stacki[node] :] + [node] 215*cda5da8dSAndroid Build Coastguard Worker # else go on to get next successor 216*cda5da8dSAndroid Build Coastguard Worker else: 217*cda5da8dSAndroid Build Coastguard Worker seen.add(node) 218*cda5da8dSAndroid Build Coastguard Worker itstack.append(iter(n2i[node].successors).__next__) 219*cda5da8dSAndroid Build Coastguard Worker node2stacki[node] = len(stack) 220*cda5da8dSAndroid Build Coastguard Worker stack.append(node) 221*cda5da8dSAndroid Build Coastguard Worker 222*cda5da8dSAndroid Build Coastguard Worker # Backtrack to the topmost stack entry with 223*cda5da8dSAndroid Build Coastguard Worker # at least another successor. 224*cda5da8dSAndroid Build Coastguard Worker while stack: 225*cda5da8dSAndroid Build Coastguard Worker try: 226*cda5da8dSAndroid Build Coastguard Worker node = itstack[-1]() 227*cda5da8dSAndroid Build Coastguard Worker break 228*cda5da8dSAndroid Build Coastguard Worker except StopIteration: 229*cda5da8dSAndroid Build Coastguard Worker del node2stacki[stack.pop()] 230*cda5da8dSAndroid Build Coastguard Worker itstack.pop() 231*cda5da8dSAndroid Build Coastguard Worker else: 232*cda5da8dSAndroid Build Coastguard Worker break 233*cda5da8dSAndroid Build Coastguard Worker return None 234*cda5da8dSAndroid Build Coastguard Worker 235*cda5da8dSAndroid Build Coastguard Worker def static_order(self): 236*cda5da8dSAndroid Build Coastguard Worker """Returns an iterable of nodes in a topological order. 237*cda5da8dSAndroid Build Coastguard Worker 238*cda5da8dSAndroid Build Coastguard Worker The particular order that is returned may depend on the specific 239*cda5da8dSAndroid Build Coastguard Worker order in which the items were inserted in the graph. 240*cda5da8dSAndroid Build Coastguard Worker 241*cda5da8dSAndroid Build Coastguard Worker Using this method does not require to call "prepare" or "done". If any 242*cda5da8dSAndroid Build Coastguard Worker cycle is detected, :exc:`CycleError` will be raised. 243*cda5da8dSAndroid Build Coastguard Worker """ 244*cda5da8dSAndroid Build Coastguard Worker self.prepare() 245*cda5da8dSAndroid Build Coastguard Worker while self.is_active(): 246*cda5da8dSAndroid Build Coastguard Worker node_group = self.get_ready() 247*cda5da8dSAndroid Build Coastguard Worker yield from node_group 248*cda5da8dSAndroid Build Coastguard Worker self.done(*node_group) 249*cda5da8dSAndroid Build Coastguard Worker 250*cda5da8dSAndroid Build Coastguard Worker __class_getitem__ = classmethod(GenericAlias) 251