Transformers

Transformers are optional components used to modify the output trees generated by the generator before they are serialized. They allow for various modifications to be performed, such as semantic improvements, variable resolution, checksum generation, and more. Multiple transformers can be defined, and they are typically used to enhance the generated output.

A transformer is implemented as a function that traverses the tree and performs the necessary operations on the nodes. The function takes a node parameter, which represents the root of the tree being processed, and it returns the modified root as the result.

The following snippet shows an example transformer that normalizes function names in trees generated by an hypothetical code generator. In this tree, funcName nodes represent function declarations, while funcRef nodes should refer to existing function names.

Example implementation to normalize function names

import random

def func_name_transformer(root):

     def _walk(node):
         nonlocal idx

         if node.name == 'funcName':
             node.src = f'func_{idx}'
             idx += 1
         elif node.name == 'funcRef':
             node.src = f'func_{random.randint(0, idx)}'

         for child in node.children:
             _walk(child)

     idx = 0
     _walk(root)
     return root

Transformers can be applied in two scenarios:

1. When creating new test cases using grammarinator-generate, transformers can be used to postprocess the generated output.

2. When creating an initial population using grammarinator-parse, transformers can be used to postprocess the items of the seed.

Both scripts expect transformers to be specified using the --transformer argument in the form of a fully qualified name. When using Grammarinator from the API, transformers can be defined in the constructor of grammarinator.tool.GeneratorTool or grammarinator.tool.ParserTool.