Runtime API: package grammarinator.runtime

class grammarinator.runtime.Annotations(root)

Bases: object

Class for calculating and managing additional metadata needed by the mutators, particularly to enforce size constraints and facilitate node filtering by rule types.

Parameters:

root (Rule) – Root of the tree to be annotated.

property nodes: list[Rule]

Get all the nodes of the tree.

Returns:

List of Rule nodes.

property quants: list[UnparserRuleQuantifier]

Get nodes created from quantified expressions.

Returns:

List of quantifier nodes.

property rules: list[UnlexerRule | UnparserRule]

Get nodes created from rule nodes.

Returns:

List of rule nodes.

class grammarinator.runtime.DefaultModel

Bases: Model

Default decision model implementation.

charset(node, idx, chars)

A single character is chosen randomly from the set of possible options (chars).

Parameters node and idx are unused.

Return type:

str

choice(node, idx, weights)

The decision is solely based upon the provided weights.

Parameters node and idx are unused.

Return type:

int

quantify(node, idx, cnt, start, stop, prob=0.5)

After generating the minimum expected items (start) and before reaching the maximum expected items (stop), quantify decides about the expansion of the optional items based on a random binary decision.

Parameters node, idx, cnt, start, and stop are unused.

Return type:

bool

class grammarinator.runtime.DispatchingListener

Bases: Listener

Base class of custom listeners that aim to override the enter and exit actions for specific rules. Subclassing DispatchingListener enables to define the enter and exit methods of a rule in the form of [enter|exit]_{rule_name}.

enter_rule(node)

Trampoline to the enter_{node.name} method of the subclassed listener, if it is defined.

Return type:

None

exit_rule(node)

Trampoline to the exit_{node.name} method of the subclassed listener, if it is defined.

Return type:

None

class grammarinator.runtime.DispatchingModel

Bases: DefaultModel

Base class of custom models that aim to override the decisions in specific rules. To override a decision point, the subclass must define methods in the form of choice_{rule_name}, quantify_{rule_name} or charset_{rule_name} - with the same signature as their counterparts in DefaultModel - in case of overriding an alternation, quantifier or charset decision, respectively.

charset(node, idx, chars)

Trampoline to the charset_{node.name} method of the subclassed model, if it exists. Otherwise, it calls the default implementation (DefaultModel.charset()).

Return type:

str

choice(node, idx, weights)

Trampoline to the choice_{node.name} method of the subclassed model, if it exists. Otherwise, it calls the default implementation (DefaultModel.choice()).

Return type:

int

quantify(node, idx, cnt, start, stop, prob=0.5)

Trampoline to the quantify_{node.name} method of the subclassed model, if it exists. Otherwise, it calls the default implementation (DefaultModel.quantify()).

Return type:

bool

class grammarinator.runtime.Generator(*, model=None, listeners=None, limit=None)

Bases: object

Base class of the generated Generators. Stores the decision model, the listeners, and additional internal state used during generation.

Parameters:

• model (Model | None) – Model object responsible for every decision during the generation. (default: DefaultModel).

• listeners (list[Listener] | None) – Listeners that get notified whenever a rule is entered or exited.

• limit (RuleSize | None) – The limit on the depth of the trees and on the number of tokens (number of unlexer rule calls), i.e., it must be possible to finish generation from the selected node so that the overall depth and token count of the tree does not exceed these limits (default: RuleSize. max).

class grammarinator.runtime.Individual(root=None)

Bases: object

Abstract base class of population individuals.

Parameters:

root (Rule | None) – Root of the tree of the individual.

property annotations: Annotations

Return the associated annotations if available, otherwise compute them immediately.

Returns:

The annotations associated with the tree.

property root: Rule

Return the root node of the tree of the individual.

Returns:

Root of the tree.

class grammarinator.runtime.Listener

Bases: object

Base class of listeners that get notified by generators whenever a rule is entered or exited. which needs to be subclassed.

enter_rule(node)

Actions to take when a rule is entered, i.e., before creating the derivation of node.

No-op by default.

Parameters:

node (Rule) – Empty node (it has no children yet, but node.name and node.parent are already valid) that is about to have a subtree generated.

Return type:

None

exit_rule(node)

Actions to take when a rule is exited, i.e., after creating the derivation of node.

No-op by default.

Parameters:

node (Rule) – Node with its subtree just generated.

Return type:

None

class grammarinator.runtime.Model

Bases: object

Abstract base class of models that make decisions for generators at alternations, quantifiers, and charsets.

charset(node, idx, chars)

Choose a character from a charset.

Raises NotImplementedError by default.

Parameters:

• node (Rule) – The current node. Its name field identifies the corresponding grammar rule, which contains the charset.

• idx (int) – Index of the charset inside the current rule.

• chars (tuple[int, ...]) – List of character codes (Unicode code points as integers) to choose a single character from.

Return type:

str

Returns:

The chosen character.

choice(node, idx, weights)

Choose an alternative from an alternation.

Raises NotImplementedError by default.

Parameters:

• node (Rule) – The current node. Its name field identifies the corresponding grammar rule, which contains the alternation to choose an alternative from.

• idx (int) – Index of the alternation inside the current rule.

• weights (list[float]) – Weights assigned to alternatives of the selected alternation.

Return type:

int

Returns:

The index of the chosen alternative.

quantify(node, idx, cnt, start, stop, prob=0.5)

Guide the loop of subtree quantification. This has to make a binary decision to tell whether to enable the next iteration or stop the loop.

Raises NotImplementedError by default.

Parameters:

• node (Rule) – The current node. Its name field identifies the corresponding grammar rule, which contains the quantified subtree.

• idx (int) – Index of the quantified subtree inside the current rule.

• cnt (int) – Number of the already generated subtrees, guaranteed to be between start (inclusive) and stop (exclusive).

• start (int) – Lower bound of the quantification range.

• stop (int | float) – Upper bound of the quantification range.

• prob (float) – Predefined probability of enabling the next iteration (between 0 and 1).

Return type:

bool

Returns:

Boolean value enabling the next iteration or stopping it.

class grammarinator.runtime.ParentRule(*, name, children=None)

Bases: Rule

Abstract base class of tree nodes that can have children.

Parameters:

• name (str) – Name of the corresponding parser rule in the grammar.

• children (list[Rule] | None) – Children of the rule (default: no children).

__iadd__(item)

Support for += operation to add one or more children to the current node. An alias to add_child() or add_children() depending on the type of child.

Parameters:

item (Rule | list[Rule]) – The node(s) to be added as child.

Return type:

ParentRule

Returns:

The current node with extended children.

add_child(node)

Add node to the end of the list of the children.

Parameters:

node (Rule) – Node to be added to children.

Return type:

None

add_children(nodes)

Add mulitple nodes to the end of the list of the children.

Parameters:

nodes (list[Rule]) – List of nodes to be added to children.

Return type:

None

children: list[Rule]

Children of the rule.

equals(other)

Compare two nodes (potentially including any children) for equality. The comparison is not implemented within __eq__ to ensure that nodes can be added to collections based on identity.

Parameters:

other (Any) – The node to compare the current node to.

Return type:

bool

Returns:

Whether the two nodes are equal.

insert_child(idx, node)

Insert node as child at position.

Parameters:

• idx (int) – Index of position to insert node at.

• node (Rule) – Node to be inserted.

Return type:

None

property last_child: Rule | None

Get the last child of the current node if any. Return None if the node has no children.

tokens()

Generator method to iterate over the (non-empty) tokens (i.e., strings) of the sub-tree of the node.

Return type:

Iterator[str]

Returns:

Iterator over token string contents.

class grammarinator.runtime.Population

Bases: object

Abstract base class of populations that store test cases in tree form (i.e., individuals) and can select trees for mutation or recombination based on some strategy.

add_individual(root, path=None)

Add a tree to the population.

Raises NotImplementedError by default.

Parameters:

• root (Rule) – Root of the tree to be added.

• path (str | None) – The pathname of the test case corresponding to the tree, if it exists. May be used for debugging.

Return type:

None

empty()

Return whether the population is empty.

Raises NotImplementedError by default.

Return type:

bool

Returns:

True if the population is empty and False otherwise.

select_individual(recipient=None)

Select an individual of the population.

Raises NotImplementedError by default.

Parameters:

recipient (Individual | None) – If None, the caller looks for an individual that could be mutated or recombined (i.e., a recipient). If not None, the caller looks for an individual (i.e., a donor) that could be recombined with the given individual (i.e., with the recipient).

Return type:

Individual

Returns:

A single individual of the population.

class grammarinator.runtime.Rule(*, name)

Bases: object

Abstract base class of tree nodes.

Tree nodes support deep copying via copy.deepcopy() (but no shallow copying via copy.copy()). A deep copy of a node is the copy of the whole subtree rooted at that node. However, the parent of a copy node is always None, even if the original node had a parent.

Tree nodes support various string representations:

• The “informal” representation of a node consists of the concatenation of the string contents of all tokens found in the tree rooted at the node in depth-first order. No extra whitespace is inserted for separation.

• The “official” representation of a node is an (almost) valid Python expression to recreate the tree rooted at the node. If the node has any children, the representation consists of multiple lines.

• The “debug” representation of a node is a multi-line string with the most important attributes of the tree rooted at the node, using vertical bars for visual guidance.

The builtins str and repr() can be used to compute the “informal” and “official” representations, respectively, while format() can produce all of the above. When string formatting is used, the following format specifiers are recognized:

Specifier	Meaning
'' or 's'	“Informal” representation.
'r'	“Official” representation.
'|'	“Debug” representation.

Thus, if n is a node, the following expressions give equivalent results:

• str(n), f'{n}', f'{n!s}', f'{n:s}', '{}'.format(n), '{!s}'.format(n), and '{:s}'.format(n)

• repr(n), f'{n!r}', f'{n:r}', '{!r}'.format(n), and '{:r}'.format(n)

• f'{n:|}' and '{:|}'.format(n)

Parameters:

name (str) – Name of the node, i.e., name of the corresponding parser or lexer rule in the grammar.

equalTokens(other)

Compare the tokens in the sub-trees of two nodes.

Parameters:

other (Rule) – The node to compare the current node to.

Return type:

bool

Returns:

Whether the two nodes are equal.

equals(other)

Compare two nodes (potentially including any children) for equality. The comparison is not implemented within __eq__ to ensure that nodes can be added to collections based on identity.

Parameters:

other (Any) – The node to compare the current node to.

Return type:

bool

Returns:

Whether the two nodes are equal.

property left_sibling: Rule | None

Get the left sibling of the node if any. Return None if the node has no parent or is the leftmost child of its parent.

Returns:

The left sibling of the current node or None.

name: str

Name of the node, i.e., name of the corresponding parser or lexer rule in the grammar.

parent: ParentRule | None

Parent node object.

remove()

Remove the current node from its parent.

Return type:

None

replace(node)

Replace the current node with node among its siblings.

Parameters:

node (Rule) – The replacement node.

Return type:

Rule

Returns:

node

property right_sibling: Rule | None

Get the right sibling of the node if any. Return None if the node has no parent or is the rightmost child of its parent.

Returns:

The right sibling of the current node or None.

property root: Rule

Get the root of the node, i.e., the node at the top of the parent chain.

Returns:

The root of the current node.

tokens()

Generator method to iterate over the (non-empty) tokens (i.e., strings) of the sub-tree of the node.

Return type:

Iterator[str]

Returns:

Iterator over token string contents.

class grammarinator.runtime.RuleSize(depth=0, tokens=0)

Bases: object

Size information about a grammar rule or a (sub)tree (generated based on a grammar rule), expressed using different metrics (both vertical and horizontal).

This class can be used to describe various size-related concepts, e.g.:

• The minimum depth of derivation of a grammar rule and the minimum number of tokens generated by a grammar rule.

• The actual depth of derivation of a grammar rule and the actual number of tokens generated by a grammar rule.

• The maximum allowed depth of derivation in a tree and the maximum number of tokens allowed in a tree.

• The depth of derivation reaching a node from the root of the tree and the number of tokens in the tree outside the subtree of the node.

The class supports additive arithmetics (+, +=, -, -=) and comparisons (==, <=, <). Note, however, that the type is not totally but partially ordered, i.e., a size object is less than another size object iff all its metrics compare less than the corresponding metrics of the other size object.

Parameters:

• depth (int | float) – Derivation length (default: 0).

• tokens (int | float) – Token count (default: 0).

depth: int | float

Derivation length.

max: ClassVar[RuleSize] = RuleSize(depth=inf, tokens=inf)

Maxium possible size ((inf, inf)).

tokens: int | float

Token count.

class grammarinator.runtime.UnlexerRule(*, name, src='', size=None, immutable=False)

Bases: Rule

Tree node representing a lexer rule or token. It has a string constant set in its src field.

Parameters:

• name (str) – Name of the corresponding lexer rule in the grammar.

• src (str) – String content of the lexer rule (default: “”).

• size (RuleSize | None) – Size of the lexer rule (default: (1,1) if src is not empty, (0,0) otherwise).

• immutable (bool) – Boolean to mark literal Unlexer nodes as immutable.

equals(other)

Compare two nodes (potentially including any children) for equality. The comparison is not implemented within __eq__ to ensure that nodes can be added to collections based on identity.

Parameters:

other (Any) – The node to compare the current node to.

Return type:

bool

Returns:

Whether the two nodes are equal.

immutable: bool

Whether the lexer rule is immutable, i.e., its source is defined as a literal and cannot be changed.

size: RuleSize

Size of the lexer rule, aggregated from the token fragments it is composed of.

src: str

String content of the lexer rule.

tokens()

Generator method to iterate over the (non-empty) tokens (i.e., strings) of the sub-tree of the node.

Return type:

Iterator[str]

Returns:

Iterator over token string contents.

class grammarinator.runtime.UnparserRule(*, name, children=None)

Bases: ParentRule

Tree node representing a parser rule. It can have zero or more UnparserRule, UnlexerRule, UnparserRuleQuantifier or UnparserRuleAlternative children.

Parameters:

• name (str) – Name of the corresponding parser rule in the grammar.

• children (list[Rule] | None) – Children of the rule (default: no children).

children: list[Rule]

Children of the rule.

class grammarinator.runtime.UnparserRuleAlternative(*, alt_idx, idx, children=None)

Bases: ParentRule

Tree node representing a sub-tree of an alternative. It can have zero or more UnparserRule, UnlexerRule, UnparserRuleQuantifier or UnparserRuleAlternative children.

Parameters:

• alt_idx (int) – Index of the alternation in the parent rule.

• idx (int) – Index of the alternative in the parent alternation.

• children (list[Rule] | None) – Children of the alternative (default: no children).

alt_idx: int

Index of the alternation in the parent rule.

equals(other)

Compare two nodes (potentially including any children) for equality. The comparison is not implemented within __eq__ to ensure that nodes can be added to collections based on identity.

Parameters:

other (Any) – The node to compare the current node to.

Return type:

bool

Returns:

Whether the two nodes are equal.

idx: int

Index of the alternative in the parent alternation.

class grammarinator.runtime.UnparserRuleQuantified(*, children=None)

Bases: ParentRule

Tree node representing a single instance of quantified sub-tree. It can have one or more UnparserRule, UnlexerRule, UnparserRuleQuantifier or UnparserRuleAlternative children.

Parameters:

children (list[Rule] | None) – Children of the quantified rule (default: no children).

children: list[Rule]

Children of the rule.

class grammarinator.runtime.UnparserRuleQuantifier(*, idx, start, stop, children=None)

Bases: ParentRule

Tree node representing the root of a quantified sub-tree. It can have one or more UnparserRuleQuantified children.

Parameters:

• idx (int) – Index of the quantifier in the parent rule.

• start (int) – Minimum number of expected items in the sub-tree.

• stop (int | float) – Maximum number of expected items in the sub-tree.

• children (list[Rule] | None) – Children of the quantifier (default: no children).

equals(other)

Compare two nodes (potentially including any children) for equality. The comparison is not implemented within __eq__ to ensure that nodes can be added to collections based on identity.

Parameters:

other (Any) – The node to compare the current node to.

Return type:

bool

Returns:

Whether the two nodes are equal.

idx: int

Index of the quantifier in the parent rule.

start: int

Minimum number of expected items in the sub-tree.

stop: int | float

Maximum number of expected items in the sub-tree.

class grammarinator.runtime.WeightedModel(model, *, weights=None, probs=None)

Bases: Model

Custom model (or model wrapper) that pre-multiplies the weights of alternatives before calling the underlying model.

Parameters:

• model (Model) – The underlying model.

• weights (dict[tuple[str, int, int], float] | None) – Multipliers of alternatives. The keys of the dictionary are tuples in the form of (str, int, int), each denoting an alternative: the first element specifies the name of the rule that contains the alternative, the second element specifies the index of the alternation containing the alternative within the rule, and the third element specifies the index of the alternative within the alternation (both indices start counting from 0). The first and second elements correspond to the node and idx parameters of choice(), while the third element corresponds to the indices of the weights parameter.

• probs (dict[tuple[str, int], float] | None) – Custom probabilities for quantifiers. The keys of the dictionary are tuples in the form of (str, int), each denoting a quantifier: the first element specifies the name of the rule that contains the quantifier, and the second element specifies the index of the quantifier within the rule (index starts counting from 0). The first and second elements correspond to the node and idx parameters of quantify().

charset(node, idx, chars)

Trampoline to the charset method of the underlying model.

Return type:

str

choice(node, idx, weights)

Transitively calls the choice method of the underlying model with multipliers applied to weights first.

Return type:

int

quantify(node, idx, cnt, start, stop, prob=0.5)

Trampoline to the quantify method of the underlying model with possibly a custom probability.

Return type:

bool

grammarinator.runtime.simple_space_serializer(root)

Simple serializer concatenating the children of UnparserRule s with a single space.

Parameters:

root (Rule) – The root node of the tree or subtree to serialize.

Return type:

str

Returns:

The serialized tree as string.