Smith manual: building, testing and releasing fonts

A file called wscript needs to be created to control the build process. The convention is that this file is placed at the root of your font project source tree. This file is in fact a small python program, but the way it is run is designed to hide that as much as possible from the unsuspecting user.

Smith is really a larger toolchain with many dependencies that you install from a Docker registry which means you don’t have to install any of its fairly large number of components manually. In this manual, we assume you are using that approach as described at SIL Font Development Guide. Remember that smith is only the director and a wide range of utilities do the actual work. Installing just smith by itself (the python program) will not get you very far.

Smith is a command-line program, there is no graphical interface with menus. But to make things easier for you, there is a completion file installed to suggest all the smith subcommands (or targets), use the Tab key and it will autocomplete the remaining letters, for example: smith co, Tab and it will complete to configure. Similarly smith b, Tab and it will complete to build. (Currently this is only available to bash users).

Besides taking inspiration from the way smith is used in public projects, like the ones available on github.com/silnrsi for example, you can use the following command to populate a brand new project with basic templates and standard folders:

Then you can start adjusting the various files to the specifics of your font project.

The heart of the build system is the wscript file that controls the build process. This is done by the python program creating a set of component objects. It then takes these objects and allows the user to run various build commands on them.

Waf, on which smith is built, works by creating a build directory into which all the results are stored. This is by design and a useful feature as it leaves the source directories pristine and makes for easy clearing up.

The build directory is created using the command:

This process creates the new results directory, checks that all the tools that smith needs to achieve the build as described in wscript are available, and sets up various internal environment variables. Thus, if any changes are made to the wscript that indicate what extra tools are needed, then the configure command needs to be rerun.

After configuration you can now launch a build. This is done using:

This creates development artifacts of the various components configured to be built. But it does not create any publishable releases - or packages that you can share with someone else - these need another command:

This creates a zip of all the generated files and the corresponding documentation. Since this zip is targeted at Windows, text files have their line endings changed to CR LF. This is tagged with development version numbers.

This does the same work as the zip command except it uses LF Linux/macOS line endings and creates a .tar.xz, a compressed tarball. This is tagged with development version numbers.

This command does two things. First, it builds the various components, but marks the build for release. So things like font version strings no longer contain any development information in the form of git revision numbers, etc. Secondly, it builds the various release packages (zip and tarball). It also provides checksums, cryptographic signatures to allow comparison against the zip and tarball. This separate checksum file will allow to verify that what is distributed is really what has been produced.

This creates a subdirectory called graide/ that contains one .cfg file per font. This config file will be read by Graide, the Graphite IDE to allow easier testing and development of smart font behaviours. If the font has no graphite smarts, no configuration file is created.

This creates font tests output by chaining all the available tests. The Tests section will go into more details.

This removes the various files created by smith build in the build directory.

This removes the build directory completely, including any temporary files at the root of the project folder.

The wscript file is a Python program, but the internal environment is set up to minimise the amount of actual programming that needs to be done. There is no setup needed in the file, and object registration is automatic. It is possible to add waf specific extensions to the file, see details in the waf manual.

The basic steps needed to describe an entire build process is to create writing system component objects. These objects are font() or designspace and package(). Specific details on what information each of these objects requires is given in the corresponding sections of this document. Likewise examples are given in the sections.

We start with a simple, single font.

In line 1, we create a new font object that we want the system to build. We specify the target filename. This file will be created in the build tree (results or what is set in the out variable). Line 2 specifies where to find the source file. Notice that the target file is a .ttf file while the source is a .ufo file. Smith will use the necessary commands to build from one format to the other.

With this as our wscript file, we can build our font:

This is the first step in building any project. This command tells smith to set up the build environment and search out all the programs that it may need for the various tasks. If a necessary program is missing smith will stop at that point and indicate an error. Some programs are not strictly necessary and smith can run with reduced functionality without them. It will just issue a warning. Such missing programs are listed in orange. All other programs that smith searches for and finds are listed, along with their locations, in green. So you can see exactly which program smith will use for any particular task. This is hepful especially in cases where you may have a locally installed self-compiled version: you can more easily see if smith has found the version in /usr/local/ (or other local paths) instead of the stock packaged version.

This command tells smith to go and build all the objects the wscript says to be built. In this case just the simple Example-Regular.ttf which will appear in results. Not very exciting, but a good start.

Most font packages consist of more than one font file and this project is no exception. Can we scale our project to handle more than one file?

This example shows the power of integrating a description with a full programming language. wscript files are python programs, albeit very enhanced ones. So we can use any python-type constructs we might need. Usually the need is slight, and we show a typical example here.

Line 1 is the start of a loop. The lines below that are indented within the loop will be repeated for each value in the list. The first value is -Regular and the second is -Bold. Each time around the loop, the variable weight is set to the appropriate string. We will then use that variable to help set the appropriate values in the two font objects we are creating.

Each time around the loop, we create a new font object. In line 2 we create a new font object whose target font filename is dependent on the weight variable which is set to the various strings from the list at the start of the loop. So we will end up creating two fonts. One called Example-Regular.ttf as before, and one called Example-Bold.ttf. Line 3 gives the source files for each of these fonts.

It may seem easier just to expand out the loop and have two font() object commands but, as the complexity of this font() grows, we will see the value of using a loop. The advantage of adding the loop early is that we can make appropriate use of weight.

Now when we come to build this project, we will get two fonts:

It’s good that we can create multiple fonts, but what do we do with them then? There are two typical products that people want from a font project: a .zip file containing the fonts and corresponding files (with Windows line-endings CR+LF) and a tarball (with Linux/macOS line-endings LF). Smith can create these two products from a wscript, but it needs just a little more information to do so:

Line 1 gives the base name of the products that will be created and line 2 gives the version of that product. Notice that the version variable is a string and does not have to be numeric. Case is important here, these are, in effect, magic variables we are setting that smith looks up.

To build this project, we do the same as before, but we can also use two extra commands:

smith zip will create Example-0.0.1-dev-(git-commit-id).zip in the releases folder inside the results folder. (The git-commit-id part, e.g. 66d16eM, will be the first 7 characters taken from the git revision id. We assume you are working from within a git repository.)

smith tarball will create Example-0.0.1-dev-(git-commit-id).tar.xz in the releases folder inside the results folder. (The git-commit-id part, e.g. 66d16eM, will be the first 7 characters taken from the git revision id. We assume you are working from within a git repository.)

Notice the -dev- suffix to indicate that these are development versions, which means they have not been tagged as a stable and tested release.

This zip, or tarball file, contains the four target fonts the build created, since we have now added Bold and Bold-Italic as extra weights in the loop.

We also added extra text files at the root of the project folder: README.txt, README.md, FONTLOG.txt, OFL-FAQ.txt. These are just text files and more documentation than font sources, but they are nice to have and will help users and other developers of your font. We will go into more detail on packaging in the dedicated section.

The part of the tutorial is being rewritten

The part of the tutorial is being rewritten

The part of the tutorial is being rewritten

The part of the tutorial is being rewritten

This final example shows how to use the designspace() object.

There are multiple ways of adding OpenType information to a font. One is to already have it in the source font. In this case, we need to indicate that we are working with an OpenType font, even if everything is internal to the font. The font builder needs to know for font testing purposes or if the font is generated from a legacy font.

This will generate tests pertinent to OpenType testing. See the section on font tests.

One approach sometimes used for FontForge based projects is to keep all the lookups in one font and then to share these lookups across all the fonts in a project. For this we simply specify a sfd_master attribute and the font builder will use sfdmeld to integrate the lookups in the master into each font as it is built. (You will have to install FontForge yourself as it is no longer part of the smith toolchain default dependencies)

Obviously, if the sfd_master attribute is the same as the source file then no merge occurs. This is an alternative way of specifying that the font is OpenType.

Another approach to adding OpenType tables to a font is to use an external tool or text file to create the lookups and then to have smith compile them into the font. Two formats for source files are supported: Microsoft’s VOLT (Visual OpenType Layout Tool) and Adobe’s Feature File.

Feax is a set of extensions to provide easier and more powerful ways to write fea code. It is a fea preprocessor. For the specification of the feax language see feaextensions.md.

makefea is the script to generate fea from a feax source file.

Adding Graphite tables to a font is much like adding VOLT information. The relevant files are declared either to the font or a gdl() object. For example:

Notice that the ap and classes attributes have the same values and meaning as for OpenType tables. This is because the information is used in the creation of both sets of tables. The Example.gdl is created by the make_gdl process and it pulls in mymaster.gdl during compilation.

The complete list of attributes to a gdl() object are:

Fonts can also be built from another font, either legacy-encoded or generated from a source font or fonts. This can be achieved by giving a legacy() object as the source attribute for the font. For example, for a font generated from a legacy font using ttfbuilder we might do:

The legacy object creates the source font that is then copied to the output and perhaps smarts are added too.

The complete set of attributes to a legacy() object is:

WOFF and WOFF2 files are TTF files in special compressed formats used for webfont delivery. Smith can generate both WOFF and WOFF2 files. For example:

The first parameter to woff() is the name of the woff file(s) to be generated. Filename extension, if present, is ignored.

The woff object takes these optional attributes:

By default, the font version is extracted from the input ttf and used as the version for the woff font. To override with a specific version use the params attribute:

To use a command other than psfwoffit to create woff files, the cmd attribute can provide the desired command and its options. For example, to use ttf2woff to create woff file:

Fret is a font reporting tool that generates a PDF report from a font file, giving information about all the glyphs in the font.

The fret object takes these attributes:

An alternative to the font object is the designspace object. A designspace specification normally defines a family of related fonts, and therefore the designspace object typically results in a number of fonts being generated — in essence the designspace object creates multiple font objects. Most of the attributes of a font object also apply to a designspace object, the differences are described below.

Instead of a source attribute, the designspace object uses a designspace file. Each instance described in the designspace file is treated as a source, and the designspace object iterates over all these instances and builds output from each.

Thus the minimum needed for the designspace object is a designspace file and target attribute:

Except for source and sfd_master, all other attributes of the font object can be used with the designspace object. Additionally the following attribute can be used:

Note, however, that in contrast to the simplest font object, the target attribute cannot be as simple as Example-Regular.ttf but must be an expression that yields an appropriate filename for each instance. This will also be true for some other attributes as well, for example the attachment point information specified by the ap attribute will need to be different for each instance.

To facilitate this, the designspace object provides a number of variables whose value is based on the particular instance being processed. To prevent possible name conflicts, the designspace object uses a DS: prefix for each of the variables it provides.

For a given instance, each attribute and each location introduce one or more variables. Consider the following instance definition:

Based on the corresponding instance attributes, the following variables will be defined:

Additionally, for each variable above, three additional variables are defined. Adding _DASH to the variable name results in a value where all spaces are replaced with a hyphen. Adding _NOSPC produces a value where all spaces are removed. Finally, adding _BASE provides a value which is the basename (without the extension) of the original value. For example:

Based on the location specified for the instance, the following variables are defined:

One additional variable provides the path from the build directory to the instance UFO, which for our example would be:

There are various source file types that can be used as the basis of many tests. These are:

Test source files are stored in a standard place in the tree. The global variable TESTDIR can be used to specify where that place is, but the default is tests/. There are a number of test targets already defined in smith that can make use of these and other test information. The TESTDIR may also be a list of paths. The list of test directories can be extended with those specified in EXTRATESTDIR which may also be a string or list. These extra directories may be overridden using a ;-separated list specified in the SMITH_EXTRATESTDIR environment variable. This may, in its turn, be overridden by a ;-separated list specified in the command line option --extratestdir. If any of the directories in the list of test directories does not exist, it is quietly ignored.

Each test target, whether standard or user defined, creates a .html file in the results/tests directory (or as specified by the TESTRESULTSDIR variable). The file name is target\index.html from which links to the actual test results can be found.

Sometimes you want to create test files as part of the build. This can be done using testFile(). It takes the same parameters as for a create() and it does create the file, but it also adds it to the list of source test files as if it was stored in the tests/ directory (or wherever you have specified that test files are stored).

It is possible to add your own tests to the smith test system. One can create a variant of one of the standard tests listed above, and associate it with a new target. Or one can run a separate command to execute the test. The test is specified using a testCommand() function that takes a single fixed parameter of the target the command is to be associated with and a list of named parameters. It is possible to specify more than one testCommand be associated with the same target, in which case all the testCommands will be run when that target is specified.