Developer Guide

Getting Started with nix

For instructions on installing nix, see TODO.

To develop within this repository, we recommend using a development shell defined in flake.nix. In this way, you will use applications pinned to same version that other developers and the CI use.

For example, from the project’s root directory, launch the project’s default shell using:

nix develop .

To launch a shell with the rust toolchain loaded:

nix develop .#rust

Or the haskell toolchain loaded:

nix develop .#haskell

After launching a nix shell, you’re in a bash shell with the appropriate applications in your path.

As of 2022-04-12, cabal build all fails in the project’s nix Haskell shell on an M1 Mac (it works on the CI’s linux machine). For now, use your locally installed versions of ghc and cabal.

Running repo checks

Run the same checks that the CI uses locally with

nix build .#check.aarch64-darwin

replacing aarch64-darwin with your system as needed.

Notable Indirections

This sections tries to document where and why the structure of certain parts of the repository may not be straightfoward.

  • The Rust and Haskell versions of fact-models use different files for test data. Both libraries test that the examples defined in a fact can be marshalled into the language and that the predicate functions are implemented correctly given the examples. However, the Haskell library directly uses the Fact.dhall (or Model.dhall) files to get the example values, while the Rust create uses Fact-examples.dhallb for test data. This is for convenience and speed. Marshalling into Rust from an encoded dhall file is much faster than an unencoded file. Moreover, picking off the the examples a Fact.dhall file can done by calling an Dhall expression like .(Fact.dhall).examples. It is simple to write any such Dhall expression using the Dhall Haskell libraries, less so in Rust. The repo checks include a check that the fragments are up-to-date. See check-fragments in flake.nix.

  • Tests for models are not automatically created in the fact-models library. Model creators should create a Test module for their model, and add those tests to the main Test module.

  • Tests of models' examples are not currently checked by the the Rust fact-models library; there are run in Haskell. There were challenges to parametrizing the macro(s) that generate tests to generate tests both for facts and models. A simple solution would be to duplicate the macro(s) and make the necessary tweaks. But this is leads to a lot of code duplication, so it is left an exercise for future developers to wire up the model tests. Lack of model tests in Rust are hopefully made up in integration tests.

  • This repo uses ormolu for Haskell formatting. Other repos such as asclepias use brittany, but as of 2022-04-22 brittany yields errors likely due to incompatability with template haskell.

  • The eventline shape is defined for Rust in this repo, but as of 2022-04-22, the same shapes in Haskell are defined in the event-data-theory package in asclepias.

  • JSON schema documents are generated by Rust code (via the schemars crate). It is arguably more direct to generate a schema from Haskell or Dhall, but as of 2022-04-22, there is not dhall to JSONschema application and the available Haskell JSONschema libraries are out-of-date or unsupported.

  • The build-fragments application (see flake.nix in project root) creates a dhall file than be used to check the integrity of each Fact.dhall (or Model.dhall) file. This is checked by the check-fragments.sh check (see flake.nix) with dhall freeze $1 --check --all. This also serves to check that the fragments are up-to-date assuming that the fragments are only created by build-fragments and not edited otherwise.

  • The example facts in Fact-examples.json are a deduplicated version of the corresponding dhall in Fact-examples.dhall. There is no to deduplicate in Dhall.

Producing JSON schema for a model

For example, to produce JSON schema for the claims-model.

cargo run -p fact-models -- --model claims-model

Tests

An important part of this repository are the integration tests that ensure that application logic is correct and that data can be marshalled to/from applications as appropriate.

This page documents checks done in this repo and where they are found.

Table 1. Tests
Description Notes Implemented

Values of facts and models can be marshalled into Haskell

See fact-models/src/Utilities.hs module for code that generates tests. The doJSONFactTests and doJSONModelTests functions do the following:

1. Read the values from the facts element of Fact.dhall (or Model.dhall).

2. The dhall values are passed through the same function that the dhall-to-json application used to create json (Aeson) values

3. The test checks that decoding/encoding the Aeson value to/from a bytestring is isomorphic.

See fact-models/src/Models/ClaimsModel/Tests.hs for example usage.

Each model needs to call the test functions in a test module.

Yes, though see note.

Values of facts and models can be marshalled into Rust

See the make_json_tests macro in fact-models/src/macros.rs for code that generates tests for facts. Also, see this note for differences between how the corresponding tests are implemented in Haskell vs Rust.

Yes, though see note

Predicate logic is correctly implemented in Haskell fact-models

Predicate logic is correctly implemented in Rust fact-models

Yes

Event tagging is correctly implemented by concipio

Eventline values can be marshalled to/from Haskell and Rust applications

No

Eventlins values produced by upstream processes can be marshalled to/from Haskell and Rust applications

No

Versioning

Versioning of fact-models and other packages in this repository follow semantic versioning (MAJOR.MINOR.PATCH), where you increment:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards compatible manner
PATCH version when you make backwards compatible bug fixes

In particular, adding new facts or models typically only requires a MINOR update, while changing the shape of existing facts or models should trigger a MAJOR update. Bug fixes to the library functions should only trigger a PATCH update.