fact-models User Guide

Getting Started

Nix Setup

To develop within event-data-model, and to use the fact-models tool we recommend using a development shell defined in flake.nix. This guarantees applications are pinned to the same version as other developers and the CI.

For instructions on installing nix, see the nsBuild setup instructions.

For information on nix usage, see the nsBuild usage guide.

TODO add haskell, rust and dhall setup

Instructions

How to create a new Fact

Instructions for creating a new fact are below. Each new Fact will need to be hooked up to a new or existing Model. See the instructions for creating a new model for more details.

Please note that the subject identifier, and begin and end dates for an event are not defined in the Fact. That information is defined in the source-to-model ETL parsers.

Adding a new Fact

  1. If not previously done, pull down the event-data-model repo and create a new branch.

  2. In the new branch, add a new directory for your fact in fact-models/src/Facts. Use CamelCase for the fact’s name.

  3. In the new directory, create a file called Fact.dhall.

  4. In Fact.dhall define the new Fact as described in Anatomy of Fact.dhall.

  5. Do a dry run of the fact application. In the project’s root directory, run the following command, changing NameOfFact to the name of the new fact:

    cabal run fact-models:exe:fact -- build --fact=NameOfFact --dry-run

  6. Review the output from the dry-run, and make changes as needed.

  7. When ready, run the fact application. In the project’s root directory, run the following command, changing NameOfFact to the name of the new fact:

    cabal run fact-models:exe:fact -- build --fact=NameOfFact

    This command also creates the files derived from Fact.dhall such as: Fact, Fact.dhallType, Fact-examples.dhall, etc.

  8. Add entries for the new fact in:

    1. fact-models/src/Facts/package.dhall

    2. fact-models/src/package.dhall in the fact-srcs object

    3. docs/modules/models/pages/facts.adoc

  9. Update the documentation.

    1. in docs/modules/fact-models/facts.adoc add a new entry for your fact, by copy-pasting from a previous Fact and updating the name.

  10. Follow the directions to create a new model, to hook the new Fact up to a Model.

  11. Create a merge request from the new branch to master and assign a member of the stats-dev-team to review.

    1. This can be done at any point, and is a great way to ask questions during the process.

    2. The branch should not be merged until the stats-dev-team member has approved.

  12. Once approved, merge in the new branch.

  13. Follow the directions in asclepias to add the new Fact and Model to your cohort building application.

How to add a new model

Adding a model is quite similar to adding a new fact with the main differences being working within the Models directory and the creation of the model’s Haskell module in Model.hs.

  1. If not previously done, pull down the event-data-model repo and create a new branch.

  2. In the new branch, add a new directory for your fact in fact-models/src/Models. Use CamelCase for the model’s name.

  3. In this directory, create a Model.dhall file in which to define your new model, as describe in anatomy of Model.dhall

  4. To check the new model, run the following command from the same directory as the new Model.dhall file.

    cabal run fact-models:exe:fact -- build --model=NameOfModel

    This command also creates the files derived from Model.dhall such as: Model, Model.dhallType, Model-examples.dhall, etc.

  5. Implement the model in Haskell.

    1. Copy Model.hs from the ExampleModel directory, and paste in same directory as Model.dhall.

    2. Update the new Model.hs with the specific Fact types needed for your project. See fact-models/src/Utilties.hs for information on the construct* and derive* functions used to create the Model.

  6. In the cabal object in fact-models/src/package.dhall, add an entry for Models.MyNewModel in the library.exposed-modules field.

  7. Run nix run .#build-fact-models-manifests to update fact-models.cabal.

  8. Now run cabal build fact-models to check that the new Haskell module successfully builds.

  9. Add a Tests.hs module for the Model to add tests for the model. See an existing model’s Test.hs file for an example. Add tests model to fact-models test suite in fact-models/src/Tests.hs.

  10. Run cabal test all --test-show-details=always to be sure your tests run and succeed.

How to use the fact application

The fact-models packages provides an executable for working with fact files within the event-data-model directory. The executable builds the files derived from Fact.dhall and Model.dhall.

The files derived from Fact.dhall are:

  1. Fact-examples.dhall

    1. The examples from the fact for use in testing routines

  2. Fact-examples.dhallb

    1. An encoded version of Fact-examples.dhall

  3. Fact-examples.json

    1. The facts from the examples (deduplicated) in JSON format

  4. Fact-example.json

    1. The fact from the head example.

    2. Used to present example JSON in the documentation site.

  5. Fact.dhallType

    1. The Type field from Fact.dhall .

    2. Used to display the type in the documentation site.

  6. Fact

    1. A frozen version of Fact.dhall, which an be used to check the integrity of facts, as in dhall freeze --check.

The files derived from Model.dhall are

  1. Model

  2. Model-example.json

  3. Model-examples.dhall

  4. Model-examples.dhallb

  5. Model-examples.json

  6. Model.dhallType

TODO describe what above files do

Instructions for how to use the fact application follow.

Building the application

There are two ways to use the application:

  1. Run with cabal run

    1. This approach ensures that you’re using the latest version of the app without having to reinstall new updates.

    2. Use the following command in the root directory of event-data-model to access the help menu

      cabal run fact-models:exe:fact -- --help

    3. You can run build, check, or list as outlined in the help menu.

  2. Install via cabal

    1. Use the following command in the root directory of event-data-model

      cabal install fact-models:exe:fact

    2. This will install the application in your $PATH.

    3. Confirm this by running

      fact --help

Usage

The application currenlty has three commmands:

  • build: Build artifacts for fact(s)

  • check: Check that fact artifacts are up-to-date

  • list: List available facts

Commands are invoked at the command-line as in:

cabal run fact-models:exe:fact -- build --help
cabal run fact-models:exe:fact -- check --help
cabal run fact-models:exe:fact -- list --help

Examples

Dry-run build of a fact

The following command prints a preview of the files that will be created when the --dry-run flag is removed.

cabal run fact-models:exe:fact -- build --fact=ServiceLocation --dry-run

Check of a fact

You can check that a fact’s artifacts are up-to-date with the following:

cabal run fact-models:exe:fact -- check --fact=Claim

The command should produce no output if the files are up-to-date.

Producing JSON Schema for a Model

TODO Need to update from JSON Schema changes.