A Short Swift GYB Tutorial

Posted on February 9, 2016 中文版

Update 9/2019: Updated the post and added a detailed description of the line-directive tool.

The GYB (Generate Your Boilerplate) tool is a Swift tool intended for internal use that generates source files starting from a template.

It’s extremely useful when you have more than one struct/class/enum that share a common structure, and you are not eager to maintain multiple versions of what is, actually, the same code. Every time you are writing the same set of methods or properties for slightly different objects, your maintenance effort (and bugs resulting from careless copy/paste) could be reduced using GYB. The tool is used extensively throughout the Swift codebase and there are just a few things you need to know to use it in your projects.

As a diligent guinea-pig (AFAIK, the only other project that uses GYB at the moment is Swift-JNI, part of the Android Swift porting project), I’ve used GYB extensively in the first release of Bitter, a Swift library that simplifies working with bits and bitwise operations, where I had a lot of very similar code inside extensions for each one of the fixed size Swift Ints.

With this tool, I was able to define a single template that the tool was able to expand to the 10 separate extensions I initially coded by hand.

Let’s see what you need to know to start using GYB.

GYB Engine Elements

The GYB templating engine is quite simple but requires a minimal knowledge of Python. A template is composed by these elements:

  • Lines starting with % (leading whitespace is ignored) are followed by Python code and are used for control flow statements, as you would expect those statement are closed with an %end. Statements can be nested.

  • Lines that do not start with an % are treated as text and simply inserted into the output.

  • Elements with the form ${VARIABLE_NAME} are replaced with the value of VARIABLE_NAME

  • The % and $ characters are escaped respectively as %% and $$.

  • Blocks of Python code can be added to the template and are delimited by %{ and }% , the indentation outside the block is stripped, so, it’s irrelevant for your Python code.

Let’s see what we can do with these few simple rules with an example, taken from the Bitter template, that adds to all fixed size integers a computed property allOnes, that returns an Int/UInt initialized with a bit pattern with all ones:


%{
  intTypes = [8,16,32,64]
}%

% for intType in intTypes:
    % for sign in ['','U']:

/// Extension that adds a few additional functionalities to ${sign}Int${intType}
extension ${sign}Int${intType} {

        /// Returns a ${sign}Int${intType} with all ones
        %if sign == '':
    public static var allOnes:Int${intType}{return Int${intType}(bitPattern: UInt${intType}.max)}
        %else:
    public static var allOnes:UInt${intType}{return UInt${intType}.max}
        %end

}
    %end
%end

With a Python block we declare an array with all the fixed sizes of the Ints available in Swift and then iterate over it, using an internal loop to consider signed and unsigned integers too. We than output two different snippets depending on the value of the sign variable, we’ll output the first one if the sign variable is empty (signed integers) or the second one if it’s not (unsigned integers).

In this example we are using a simple if/else and foreach statements, but we could have used everything that is valid in Python like an elif or a variation of that for.

Running this through GYB we’ll get 8 extensions, one for each fixed size integer, from Int8/UInt8 to Int64/UInt64.

Generating the source

You can download GYB from the Swift repository:


wget https://github.com/apple/swift/raw/master/utils/gyb
wget https://github.com/apple/swift/raw/master/utils/gyb.py
chmod +x gyb

And parse your template this way:


./gyb --line-directive '' -o ../Sources/Bitter/Bitter.swift Bitter.swift.gyb

The -o option specifies the output file, while the last filename specifies the name of the file containing the template.

Without the --line-directive '' parameter, GYB outputs additional debugging comments, that for each section of the output describe which element in the original template was executed to generate it.

Useful while you are still in the process of writing your template but once you are done the debug comments can be disabled to obtain a clean output.

Better debugging of generated sources with line-directive

But what about debugging code that has been generated with GYB?

That’s where the comments that we purged from the output in the previous step come in handy, comments that will be parsed by an additional tool, line-directive, that will simplify what would otherwise be an excruciating process.

Download it from the main repository on GitHub:


wget https://github.com/apple/swift/raw/master/utils/line-directive
chmod +x line-directive

To explain how this works, here is a detailed explanation by GYB’s author, Dave Abrahams, directly from the line-directive usage documentation.

#sourceLocation is a directive used by tools like the Swift compiler and debugger to adjust the lines reported in diagnostics and to determine what source you see when you’re stepping.

#sourceLocation corresponds to #line in C/C++ which is inserted by code generators like Lex/Flex/Yacc/Bison so that you deal with the actual code you wrote and not the generated result. For dealing with errors in the Swift generated by your gyb source it’s important that your tools can take you to the right line in your gyb file rather than in generated .swift file. If you don’t have such a tool, manually indirecting through the generated code is tedious, but at least it’s possible since gyb leaves #sourceLocation information behind.

But Swift’s #sourceLocation directive is suboptimal for the purposes of the freeform code generation done with gyb because it can only appear between grammatically-complete declarations and statements. So instead of inserting #sourceLocation directives, gyb inserts //###sourceLocation comments (by default, it’s tunable).

This line-directive tool remaps file and line information in the output of your swift compiler (or whatever tool you are using to process generated source, gyb is not swift-specific) so that the error points to the right place in the gyb source.

You invoke it as follows:


  line-directive <generated-sources> -- <compilation command>

For example, if you have foo.swift.gyb, bar.swift.gyb, and baz.swift, instead of:


  gyb foo.swift.gyb -o foo.swift
  gyb bar.swift.gyb -o bar.swift
  swiftc foo.swift bar.swift baz.swift

You do this:


  gyb foo.swift.gyb -o foo.swift
  gyb bar.swift.gyb -o bar.swift
  line-directive foo.swift bar.swift -- swiftc foo.swift bar.swift baz.swift

Did you like this article? Let me know on Twitter!

I'm also on Twitter and GitHub.

Subscribe via RSS or email.