Dhall Usage Guide

Dhall overview

Introduction

Dhall is a programming language that is designed to be used for the specification of configuration data (i.e. a collection of parameters that is consumed by a library or application that can be used to determine its behavior) in place of file formats such as JSON, YAML, CSV, and so on. File formats such as these have a number of shortcomings for certain tasks: in particular they have limited or no facilities for DRY programming and to ensure that the data is in the correct form needed for the intended library or application. Dhall aims to solve these problems in a programming language that is safe to run on your system with untrusted data (in contrast to general-purpose languages). Perhaps the best summary of Dhall is given by one of its taglines:

Dhall is a programmable configuration language that you can think of as: JSON + functions + types + imports.
— The Dhall language website

The Dhall ecosystem

There are a number of Dhall applications including (a complete list can be found in the Dhall releases GitHub page):

  • A Dhall compiler

  • A Dhall-to-JSON compiler

  • A Dhall-to-YAML compiler

  • A Dhall-to-CSV compiler

There are also a number of general-purpose programming languages that have libraries that implement the Dhall language including Haskell and Rust (see a complete list in the Contributing to Dhall GitHub document). This permits a direct conversion between the host language objects and their corresponding Dhall representations.

Additionally, there are many available Dhall packages that target a specific form of a given output type. For example, the dhall-gitlab-ci package provides types and utilities that can be used to produce YAML that conforms to the GitLab CI configuration schema. Other popular examples include the dhall-kubernetes and the dhall-aws-cloudformation packages.

We use Dhall in a number of important places in our codebase including the definition of codelists in the codelist repository and the definition of events in the event-data-model repository.

Dhall application installation extended discussion

Below are instructions for installing Dhall applications. There are several approaches available which each have their own advantages and disadvantages. If no particular option stands out to you, then installation via compiled binary files is a good default.

Dhall application installation via a package manager

At this time of writing (July 2022), Homebrew hosts most of the core Dhall applications, while the Chocolatey, Debian, Ubuntu and Fedora repositories host a severely limited or nonexistent set of Dhall applications.

  • Homebrew users can use brew search dhall to see what is available, and brew install formula to install the desired package (replace formula with dhall, dhall-json, etc.).

Dhall application installation via compiled binary files

You can install a Dhall application by using one of the available compiled binary files. The steps to install a given application are roughly as follows:

  1. Navigate to the Dhall releases page and download the desired application and version number for the appropriate target platform.

  2. Decompress and extract the files contained in the downloaded file.

  3. Move the extracted files into a directory that is on your shell’s executable search path. For example, a common location to place such files on Unix-like systems is the /usr/local/bin directory.

Operating system-specific Dhall installation instructions can be found for the particular case of installing the dhall-to-json and dhall-to-yaml applications in the links provided below. Other applications can be installed in much the same way.

  • Windows installation of dhall-to-json and dhall-to-yaml

  • OS X installation of dhall-to-json and dhall-to-yaml

  • Linux installation of dhall-to-json and dhall-to-yaml

Dhall application installation via Cabal

The Dhall suite of applications are compiled from Haskell packages that are available on Hackage so you can use cabal install to install the applications. So for example you can use cabal install dhall-to-json to install dhall-to-json, and so on. If you want to install a particular version (rather than the latest), you can append the desired version number to the target name as in e.g. cabal install dhall-to-json-1.7.10.

The tricky part of this approach is figuring out the target name and available version numbers. One way to find the candidate package names is to look in the dhall-lang GitHub namespace. You can then navigate to a given package’s website on Hackage and click on the Package description link to see the Cabal package description file which lists all of the package components. Furthmore, the Hackage webpage for the package will list all of the package versions that are available for you to specify in the cabal install target command.

Dhall compilation

The dhall application can be used to evaluate Dhall expressions that can be provided either as a file or through standard input. Evaluating a given expression roughly amounts to resolving any imports in the expression (i.e. replacing any embedded URLs, file paths, and environment variables with the expressions that they refer to), and evaluating all possible sub expressions.

Applications like dhall-to-json and dhall-to-yaml perform the work of evaluating a Dhall expression, plus the additional step of compiling that expression to its corresponding JSON or YAML representation. Note that only a subset of Dhall expressions can be transformed into JSON or YAML since e.g. functions and types can’t be represented in those file formats.

Consider the following example demonstrating various methods of compiling Dhall data from the command line. The Dhall expression used for the following examples is defined between the single quotes in 1 (note that the single quotes are part of the shell syntax). The Dhall expression binds the variable dir to a string, and then creates a record value with keys f1 and f2 with corresponding values "src/Lib.hs" and "src/Utils.hs" that are each created by concatenating dir with another string.

  • In 1 we bind the expr variable to a string representing the Dhall expression, and then in 2 we write the string from 1 into a file.

  • Then in 3, 4, and 5 we see various ways to compile the expression. Recall that <<< in many shells has the effect of providing the right-hand side term to the command’s standard input.

  • In 6 we read from a file and then perform an operation on the result, which in this case is to select the value of the f1 element in the record type.

  • In 7 and 8 we compile the Dhall expression into a JSON and YAML representation, respectively.

# Store a Dhall expression both as a string and in a file
expr='let dir = "src/" in { f1 = dir ++ "Lib.hs", f2 = dir ++ "Utils.hs" }' (1)
echo "$expr" > expr.dhall (2)

# Provide the input via standard input
dhall <<< "$expr" (3)
##> { f1 = "src/Lib.hs", f2 = "src/Utils.hs" }

# Provide the input from a file
dhall --file expr.dhall (4)
##> { f1 = "src/Lib.hs", f2 = "src/Utils.hs" }

# An alternative way to provide the input from a file (technically, you are
# creating a new Dhall expression that uses the Dhall import facilities to
# insert the contents of the file). This is not useful by itself since we can do
# the same thing with the `--file` argument, but we'll see a better use of this
# technique in the next example
dhall <<< './expr.dhall' (5)
##> { f1 = "src/Lib.hs", f2 = "src/Utils.hs" }

# Sometimes it is useful to do something with a Dhall expression that is defined
# in a file, such as e.g. extracting something from it. The following example
# imports the expression defined in `expr.dhall` and extracts the value of the
# `f1` element of the record type by using the `.f1` syntax (the open and close
# parentheses around the import statement are needed to differentiate the import
# from the record field access)
dhall <<< '(./expr.dhall).f1' (6)
##> "src/Lib.hs"

# Convert the input to JSON
dhall-to-json <<< "$expr" (7)
##> {
##>   "f1": "src/Lib.hs",
##>   "f2": "src/Utils.hs"
##> }

# Convert the input to YAML
dhall-to-yaml <<< "$expr" (8)
##> f1: src/Lib.hs
##> f2: src/Utils.hs