The asclepias User Guide

The purpose of the user guide is to provide guidance on using asclepias. This guide covers:

  • How to get started using asclepias

  • Background theory on asclepias

  • Instructions for using asclepias

  • References

Getting Started

To get started using asclepias, there are a few prerequisite tasks. This section covers instructions on the installation of necessary software tools and prepping data for use by asclepias functions.

Setting Up Software Tools

To use asclepias, you will need to install the Haskell tool chain. To install, follow the directions in the setup guide, with these special instructions:

  • Use ghc version 8.10.7.

  • Use cabal version 3.6.2.0.

If you are new to Haskell, review the Haskell usage guide for best practices.

Data Requirements

asclepias expects data to be in a particular format. The data must be in JSON file, and follow NoviSci’s standard EDM schema where each line in the file is a valid EventLine. See the EventLine type in event-data-model for more details. Any project that uses the event-data-model package will meet these requirements.

Background

TODO flesh this out a bit The Theory and design section contains background information on the event-data-model design.

Instructions

1. Initialize a Project

  1. Initialize a git repo for your project.

  2. In your terminal, navigate to the project folder.

  3. Execute the following code in the terminal, replacing myProj with your project name.

    PROJID=myProj
cabal init --libandexe --application-dir=apps --source-dir=plans --package-name=$PROJID -m -u https://gitlab.novisci.com/nsResearch/$PROJID -d hasklepias
cabal update

  4. Rename MyLib.hs to Cohorts.hs.

  5. Update myProj.cabal

    1. expose all modules

    2. set hasklepias to a 0.2.5

    3. set library default extensions:

       default-extensions:
   NoImplicitPrelude
   OverloadedStrings
   LambdaCase
   FlexibleContexts
   FlexibleInstances
   DeriveGeneric
   MultiParamTypeClasses
   DataKinds
   TypeApplications

  6. add a cabal.project file with:

    source-repository-package
  type: git
  location: https://github.com/novisci/asclepias.git
  tag: #SET TO DESIRED COMMIT#
  subdir: hasklepias-core hasklepias-main hasklepias-templates stype event-data-theory

packages: ./myProj.cabal

  7. Start coding.

2. Define Data Schema

TODO placeholder really - need to update with exact steps

3. Create Events

TODO write a procedures.adoc for creating an event.

As a best practice, the Concepts type c should be a sum type object. Meaning, each possible concept should be enumerated in the type (data MyProjectTags = Diabetes | BirthDay | InHospital | ...). By defining the concepts as a sum type, type safety is ensured. One cannot misspell a concept or use an undefined concept, for example.

The schema (m) type for an Event must an instance of Eq, Show, Generic, and FromJSON typeclasses. The DeriveGeneric language extension makes deriving the Generic instance trivial, as in the code above. At this time, users do need to provide the FromJSON instance, and the boilerplate in the example above should work in most cases.

The concept (c) type for an Event must an instance of Eq, Show, Typeable, and FromJSON typeclasses. Making c Generic will also make it Typeable, so in most cases simply deriving (Eq, Show, Generic) and a stock FromJSON instance is sufficient for the concept type.

The event-data-theory packages provides a few utilities for testing a new model. These can be found in the EventDataTheory.Test module, which is not included in the main set of exported modules.

The eventDecodeTests and eventDecodeFailTests functions, for example, test for successful parsing and successful failed parsing (respectively) of EventLine m c a into the corresponding Event c m a. These functions take a directory path as an argument. Each file ending .jsonl in that directory should contain a single EventLine as JSON to be tested. See the test directory and EventDataTheory.TheoryTest module in this package for examples.

4. Create Features

TODO write procedures/instructions for creating a feature.

5. Create Cohorts

TODO

  • decide input data shape

    • e.g. choose Event and a model from fact-models

  • decide index type

  • decide output data shape

  • code and test cohort

  • build application

  • run application

6. Read in Project Data

TODO write this up

The EventDataTheory.EventLines module provides several utilities for decoding events from eventlines. The parseEventLinesL function, for example, converts a ByteString of new-line delimed JSON into a pair of [String] (containing any parse error messages) and [(SubjectID, Event c m a)], a list of Subject ID/event pairs.

Examples

This section provides detailed examples of asclepias usage. TODO include examples of creating events?

Features

This section provides examples of feature creation.

Find the last event that occurs within a time window of other events

This example demonstrates:

  • the formMeetingSequence function from interval-algebra

  • handling a failure case

  • writing a function generic over both the concept and interval types

In this example, the goal is to write a function that, given a list of concepts, converts a list of events into a list of interval durations such that:

  • the events with any of the given concepts are combined into a "meeting sequence";

  • durations of events of the resulting sequence which have all of the given concepts are returned;

  • but an empty result is treated as a failure.

A function like this could be useful if you wanted to find the durations of time when a subject was both hospitalized and on some medication. 
durationsOf
  :: forall n m c a b
   . (KnownSymbol n, Eventable c m a, IntervalSizeable a b)
  => [c]
  -> [Event c m a]
  -> Feature n [b]
durationsOf cpts =
  filter (`hasAnyConcepts` cpts) (1)
    .> fmap (into @(ConceptsInterval c a)) (2) (3)
    .> formMeetingSequence (4)
    .> filter (`hasAllConcepts` cpts) (5)
    .> \x -> if null x (6)
         then makeFeature $ featureDataL $ Other "no cases"
         else makeFeature $ featureDataR (durations x)

Take the case that a subject has the following events, and we want to know the duration that a subject was both hospitalized and on antibiotics. Below, we walk through the function step-by-by using this case.

   --                          <- [Non-medication]
      ----                     <- [Hospitalized]
       --                      <- [Antibiotics]
            ----               <- [Antibiotics]
------------------------------
1 Filter events to those that contain at least one of the given concepts. 
      ----                     <- [Hospitalized]
       --                      <- [Antibiotics]
            ----               <- [Antibiotics]
------------------------------
2 Cast each event into a ConceptsInterval c a, which is a synonym for PairedInterval (Concepts c) a.
3 This step is important for the formMeetingSequence function, as it requires the "data" part of the paired interval to be a Monoid. Concepts are a Monoid by unioning the elements of two values.
4 Form a sequence of intervals where one meets the next. The data of the running example would look like: 
      -                        <- [Hospitalized]
       --                      <- [Hospitalized, Antibiotics]
         -                     <- [Hospitalized]
          --                   <- []
            ----               <- [Antibiotics]
------------------------------
5 Filter to those intervals that have both of the given concepts. Note that hasAllConcepts works here because PairedInterval (Concepts c) a is defined as an instance of the HasConcepts typeclass in event-data-theory. 
       --                      <- [Hospitalized, Antibiotics]
------------------------------
6 Lastly, if the result of the previous step is empty, we return a failure, i.e. a Left value of FeatureData. Otherwise, we return the durations of any intervals, as a successful Right value of FeatureData.

The durationsOf function can be lifted into a Definition using defineA:

def
  :: (KnownSymbol n1, KnownSymbol n2, Eventable c m a, IntervalSizeable a b)
  => [c] (1)
  -> Def (F n1 [Event c m a] -> F n2 [b]) (2)
def cpts = defineA (durationsOf cpts)
1 Create a function which takes a list of concepts and
2 Returns a Definition

Find durations of time that satisfy multiple conditions

This example demonstrates

  • reasoning with the interval algebra

  • manipulating intervals

  • using concepts to group events

In this example, the goal is to write a function that, given a pair of lists of concepts and an interval of time:

  • filters an input list of events to those that concur with the given interval. Note that concur, in this context, means that the intervals are not disjoint.

  • splits the events into those with the first concepts and those with the second

  • returns the start of the last event of the first set of concepts where it occurs within +/- 3 time units of an event of the second set of concepts.

A function like this is useful for defining an index event where the index needs to concur with a time window of other events. 
examplePairComparison
  :: (Eventable c m a, IntervalSizeable a b)
  => ([c], [c])
  -> Interval a
  -> [Event c m a]
  -> Maybe a
examplePairComparison (c1, c2) i =
  filterConcur i    --  (1)
    .> splitByConcepts c1 c2 (2)
    .> uncurry allPairs (3)
    .> filter (\pr -> fst pr `concur` expand 3 3 (snd pr)) (4)
    .> lastMay (5)
    .> fmap (begin . fst) (6)

Take the case that a subject has the following events, and we want to know the first time a diagnosis occurred within +/- 3 days of a procedure. Our given interval, called Baseline here, is (6, 15). Below, we walk through the function step-by-by using this case.

      ---------                <- Baseline
    -                          <- [pr]
      -                        <- [pr]
          -                    <- [dx]
            -                  <- [pr]
            ----               <- [foo]
------------------------------
1 Filter events to those concurring with the given interval. 
      ---------                <- Baseline
      -                        <- [pr]
          -                    <- [dx]
            -                  <- [pr]
            ----               <- [foo]
------------------------------
2 Form a pair of lists where the first element has c1 (dx in our example) event intervals and the second has c2 (pr in our example) event intervals. Any events without c1 or c2 concepts are dropped. In the running example, the intervals of the events would make the following pair: 
( [(10,11)] -- the dx event interval
, [(6,7), (12,13)] -- the pr event intervals
)
3 Form a list of all (c1, c2) pairs of event intervals from the previous step. 
[ ( (10,11), (6,7) )
, ( (10,11), (12,13) )
]
4 Expand the c2 (pr) event intervals by +/- 3 units of time. 
[ ( (10,11), (3,10) )
, ( (10,11), (9,16) )
]

Then, filter this list to include only instances where the c1 (dx) interval concurs with a 'c2' interval.

[ ( (10,11), (9,16) ) ]
5 Take Just the last element of the list, if it exists. Otherwise, Nothing.
6 If it exists, take the begin of the last c1 interval. In our example, this is Just 10.

Lastly, the example function can be lifted into a Definition using the define function:

def
  :: (Eventable c m a, IntervalSizeable a b)
  => ([c], [c])
  -> Def (F n1 (Interval a) -> F n2 [Event c m a] -> F n3 (Maybe a))
def cpts = define (examplePairComparison cpts)

Create a function for identifying whether a unit has a history of some event

This example demonstrates:

  • a simple feature

  • writing a function in order to create multiple Feature definitions

Epidemiologic studies often seek to determine whether and when some event occurred. In general, the event logic can be quite complicated, but this example demonstrates a simple feature. We wish to determine whether an event of some given concepts occurred, relative to a provided assessment interval.

The function is given here:

makeHx
  :: (Ord a)
  => [Text] (1)
  -> AssessmentInterval a
  -> [Event Text ExampleModel a]
  -> Maybe (Interval a) (2)
makeHx cpts i events =
  events
    |> filterEvents (containsConcepts cpts &&& Predicate (enclose i)) (3)
    |> lastMay (4)
    |> fmap getInterval (5)
1 The example events use Text as the type of concepts, so the first argument is a list of Text values that will be used to filter events.
2 The return type is Maybe (Interval a). A value of Nothing indicates that no event of interest occurred. If one or more events occur, a value of Just < some interval > is the interval of the last event.
3 The first step in the function is to filter events to those that contain at least one of the given concepts and satisfies an interval relation relative to assessment interval. For this example, we use the enclose relation, meaning the event must not overlap either end of the assessment interval.
4 The lastMay function returns the last element of a list, if the last is not empty.
5 Lastly, getInterval gets the interval component from the event. The fmap function is necessary to apply the function to a Maybe (Event Text ExampleModel a).

With the makeHx function, we can create feature definitions:

duckHxDef (1)
  :: (Ord a)
  => Definition
       (  Feature "index" (AssessmentInterval a)
       -> Feature "events" [Event Text ExampleModel a]
       -> Feature "duck history" (Maybe (Interval a))
       )
duckHxDef = define (makeHx ["wasBitByDuck", "wasStruckByDuck"])

macawHxDef (2)
  :: (Ord a)
  => Definition
       (  Feature "index" (AssessmentInterval a)
       -> Feature "events" [Event Text ExampleModel a]
       -> Feature "macaw history" (Maybe (Interval a))
       )
macawHxDef = define (makeHx ["wasBitByMacaw", "wasStruckByMacaw"])
1 Defines a feature that identifies whether a unit was hit by a duck or struck by a duck.
2 Defines a feature that identifies whether a unit was hit by a macaw or struck by a macaw.

Creating "Two outpatient or one inpatient"

This example demonstrates:

  • a common feature used in studies of medical claims data

  • using a template to define a feature building function

This example defines a feature that indicates either:

  • at least 1 event during the baseline interval has any of the cpts1 concepts

  • there are at least 2 events that have cpts2 concepts which have at least 7 days between them during the baseline interval

twoOutOneIn
  :: (IntervalSizeable a b)
  => [Text] -- ^ inpatientConcepts
  -> [Text] -- ^ outpatientConcepts 
  -> Definition (1)
       (  Feature "index" (Interval a)
       -> Feature "allEvents" [Event Text ExampleModel a]
       -> Feature name Bool
       )
twoOutOneIn inpatientConcepts outpatientConcepts = buildNofXOrMofYWithGapBool (2)
  1
  (containsConcepts inpatientConcepts) (3)
  1
  7
  (containsConcepts outpatientConcepts) (4)
  concur
  (makeBaselineMeetsIndex 10) (5)
1 The twoOutOneIn function returns a Definition.
2 We use the buildNofXOrMofYWithGapBool template function to build our definition. This function takes seven arguments.
3 The first two are passed to the buildNofX template. The given arguments say that we’re looking for at least 1 event that contains one or more of the inpatientConcepts.
4 The next three arguments are passed to the buildNofXWithGap template. The given arguments say that we’re looking for at least 1 gap between any pair of events (and thus at least 2 events) that contains one or more of the outpatientConcepts.
5 The last two arguments determine when the events must occur relative to the index event. Here, the events must concur with a baseline assessment interval.

Count number of events

This example demonstrates:

  • using the AssessmentInterval type

  • using the combineIntervals function

  • counting the number of events satifying a condition

This example defines a function that takes an AssessmentInterval and a list of ExampleModel events to return a pair: (count of hospitalization events, duration of the last hospitalization).

countOfHospitalEvents
  :: (IntervalSizeable a b)
  => AssessmentInterval a
  -> [Event Text ExampleModel a]
  -> (Int, Maybe b)
countOfHospitalEvents i =
  filterEvents (containsConcepts ["wasHospitalized"]) (1)
    .> combineIntervals (2)
    .> filterConcur i (3)
    .> (\x -> (length x, duration <$> lastMay x)) (4)

Consider the follow events as a working example:

     **********      <- [assessment]
 ---                 <- [wasHospitalized]
    --               <- [wasHospitalized]
        --           <- [notHospitalized]
          -----      <- [wasHospitalized]
====================
1 As a first step, events are filtered to those satisfying the predicate of interest, In this example, events are filtered to those that contain the concept wasHospitalized: 
     **********      <- [assessment]
 ---                 <- [wasHospitalized]
    --               <- [wasHospitalized]
          -----      <- [wasHospitalized]
====================
2 The combineIntervals function from the interval-algebra package combines intervals that are not before or after. As in our example, this step can be important to combine intervals that we consider to be a single event. In the example, the first and second events would be joined into one event. 
     **********      <- [assessment]
 -----               <- [wasHospitalized]
          -----      <- [wasHospitalized]
====================
3 After combining the intervals, then the intervals are filtered to those not disjoint from the assessment interval. This step includes all hospitalization intervals in our running example. 
     **********      <- [assessment]
 -----               <- [wasHospitalized]
          -----      <- [wasHospitalized]
====================
4 Lastly, the result is derived from remaining hospitalization intervals. The example result is (2, Just 5) since there are 2 intervals and the duration of the last one is 5.

The function presented here is one of many ways to filter and count intervals. For example, the current function includes hospitalizations that overlap the assessment interval. If one wanted to filter out such hospitalizations, the filterConcur i could be changed to filter (not . (disjoint <|> overlaps) i).

Another consideration is the duration measurement. The current function measurement the duration of the last hospitalization interval, disregarding the assessment interval. One may instead want to measure the duration that concurs with the assessment.

The countOfHospitalEvents function can be lifted into a Definition using define:

countOfHospitalEventsDef
  :: (IntervalSizeable a b)
  => Definition
       (  Feature "index" (AssessmentInterval a)
       -> Feature "events" [Event Text ExampleModel a]
       -> Feature "count of hospitalizations" (Int, Maybe b)
       )
countOfHospitalEventsDef = define countOfHospitalEvents

Discontinuation from a Drug

This example demonstrates:

  • complex interval-algebra functionality

  • use of the bind operator (>>=)

In this example, the goal is to write a function that, given an assessment interval and list of events:

  • filters to antibiotic events

  • allows for a gap of 5 days between antibiotic events

  • only allow for treatment sequences that are started or overlapped by the assessment interval

  • returns the time discontinuation begins and the time since the beginning of the assessment interval to discontinuation.

For this example, we walkthrough three cases.

Case 1
     **********      <- [assessment]
 ---                 <- [tookAntibiotics]
    --               <- [tookAntibiotics]
        --           <- [wasHopitalized]
          -----      <- [tookAntibiotics]
====================
Case 2
     **********      <- [assessment]
      --             <- [tookAntibiotics]
          -----      <- [tookAntibiotics]
====================
Case 3
     **********      <- [assessment]
     ---             <- [tookAntibiotics]
====================

The logic of the feature is defined in the discontinuation function:

discontinuation
  :: (IntervalSizeable a b)
  => AssessmentInterval a
  -> [Event Text ExampleModel a]
  -> Maybe (a, b)
discontinuation i events =
  events
    |> filterEvents (containsConcepts ["tookAntibiotics"]) (1)
    |> fmap (expandr 5) (2)
    |> combineIntervals (3)
    |> nothingIfNone (startedBy <|> overlappedBy $ i) (4)
    |> (>>= gapsWithin i) (5)
    |> (>>= headMay) (6)
    |> fmap (\x -> (begin x, diff (begin x) (begin i))) (7)
1 First, we filter to events that have the concept "tookAntibiotics". In Case 1, the third interval is filtered out: 
     **********      <- [assessment]
 ---                 <- [tookAntibiotics]
    --               <- [tookAntibiotics]
          -----      <- [tookAntibiotics]
====================

Cases 2 and 3 are unchanged.
2 To allow for a grace period of 5 days between antibiotic events, each antibiotic event is extended by 5 units using the expandr function: For Case 1, this results in: 
     **********      <- [assessment]
 --------            <- [tookAntibiotics]
    -------          <- [tookAntibiotics]
          ---------- <- [tookAntibiotics]
====================

And similarly for Cases 2 and 3.
3 Antibiotic intervals that concur are considered one treatment sequence, so combineIntervals is used to collapse these intervals. In all the example cases, this results in one interval; e.g. for Case 2: 
     **********      <- [assessment]
      -------------- <- [tookAntibiotics]
====================
4 With all the treatment intervals transformed to allow for a gap in treatment; now we handle the case where none of the intervals start or overlap the assessment interval. The nothingIfNone function takes a predicate and a list and returns Nothing if none of the list elements satisfy the predicate; otherwise, it returns Just the list.

In Cases 1 and 3, the assessment interval is overlappedBy and startedBy (respectively) the treatment interval. However in Case 2, since antibiotic treatment starts after the assessment interval starts, nothingIfNone yields Nothing. This is final result for Case 2

In interval-algebra terminology, the assessment interval in Case 2 overlaps the treatment interval; which is different than being overlappedBy the treatment interval.
5 So far, we have the treatment interval in hand. We’re interested, though, in discovering gaps in treatment which is considered discontinuation. The gapsWithin function find gaps in the input intervals clipped to the assessment, yielding Nothing if no such gaps exist and Just the gaps otherwise. (See note about >>= below)

Case 1 has no gaps, hence the final result is Nothing. For Case 3, however, there is a gap between the treatment interval and the end of assessment:

     **********      <- [assessment]
             --      <- [gap]
====================
6 If there are multiple gaps in treatment, the first one is the discontinuation of interest.
7 Finally, provided that a gap in treatment exists, the time of discontinuation is the begin of that gap. The time from the start of assessment to discontinutation is computed by diff (begin x) (begin i).

For Case 2, the final result is Just (13, 8).
As implemented, a Nothing result from discontinuation could either indicate that a subject did not discontinue or that they simply had no antibiotics records. If such a distinction is important, the function could be modified to disambiguate these case using a sum type for example.

The discontinuation function can be lifted into a Definition using define:

discontinuationDef
  :: (IntervalSizeable a b)
  => Definition
       (  Feature "index" (AssessmentInterval a)
       -> Feature "events" [Event Text ExampleModel a]
       -> Feature "discontinuation" (Maybe (a, b))
       )
discontinuationDef = define discontinuation
Using the >>= operator

The >>= comes from Haskell’s Monad typeclass. Sometimes called the bind operator, it has the following type signature:

(>>=) :: m a -> (a -> m b) -> m b

Consider these lines of discontinuation function:

  |> nothingIfNone ( startedBy <|> overlappedBy $ i)
  |> (>>= gapsWithin i)

  • The type coming out of the nothingIfNone is Maybe [Interval a].

  • The type for gapsWithin i is [Interval a] → Maybe [Interval a], and we want that to return a Maybe [Interval a].

If you put those pieces together, you have a concrete signature for >>=:

Maybe [Interval a] -> ([Interval a] -> Maybe [Interval a]) -> Maybe [Interval a]

Cohorts

This section provides examples for defining cohorts.

Defining an Index Set

This example demonstrates:

  • how to create an index set

In this example, index is defined as the first time that a subject was bitten by an Orca (ICD10 codes W56.21/W56.21XA).

defineIndexSet
  :: Ord a
  => [Event Text ExampleModel a] (1)
  -> IndexSet (Interval a) (2)
defineIndexSet events =
  events
    |> filterEvents (containsConcepts ["wasBitByOrca"]) (3)
    |> headMay (4)
    |> fmap getInterval (5)
    |> into (6)
1 The input type for this example is a list of events, where the concepts are Text, the data model is ExampleModel, and the interval time is a generic type a.
2 The return type is an Indexset of Interval. The IndexSet type is defined in hasklepias-core as either Nothing or a set of unique ordered values.
3 To determine whether a subject has an index, we filter to the events tagged with the concept "wasBitByOrca".
4 The headMay function gets the first event, if one exists. We’re assuming the input list has already been sorted.
5 Then we get the interval, if it exists.
6 The into function casts the output from <4> into a IndexSet (Interval a) type.

Defining Assessment Intervals

This example demonstrates:

  • how to create assessment intervals for baseline and followup

In this example, TODO

bline :: (IntervalSizeable a b) => Interval a -> AssessmentInterval a
bline = makeBaselineMeetsIndex 60
flwup :: (IntervalSizeable a b) => Interval a -> AssessmentInterval a
flwup = makeFollowupStartedByIndex 30

Create a minimal cohort

This examples demonstrates:

  • a bare bones cohort, created without using features or events.

  • reading csv formatted data

Create a cohort with calendar-based indices

This examples demonstrates:

  • specifying cohorts from calendar-based indices

  • using asclepias' cohort module without using its feature module

  • using an empty return type for the cohort data to just compute attrition information

Review the cohort building checklist TODO: create such a document
Goal

Tha goal in this example is to create a cohort for each quarter of 2017. The cohort should include subjects if they have an enrollment event concurring with the first day of a quarter. For this example,

Decide on the data model

In this example, we use the following data model in our events: TODO

Here we create a type synonym for the the event type in this example

type Evnt = Event Text ExampleModel Day
Create intervals for dates used for indices
indices :: [Interval Day]
indices = map (\(y, m) -> beginervalMoment (fromGregorian y m 1))
              (allPairs [2017] [1, 4, 7, 10])
Define criteria
isEnrollmentEvent :: Predicate Evnt
isEnrollmentEvent = Predicate
  (\x -> case getFacts (getContext x) of
    Enrollment -> True
    _          -> False
  )

Include the subject if she has an enrollment interval concurring with index.

enrolled :: Interval Day -> [Evnt] -> Status
enrolled i es = includeIf . not . null $ filterEvents
  (Predicate (concur i) &&& isEnrollmentEvent)
  es
Write Cohort Specification

A cohort is TODO: link to cohort definition

makeIndexRunner :: Interval Day -> [Evnt] -> IndexSet (Interval Day)
makeIndexRunner i _ = makeIndexSet [i]
makeCriteriaRunner :: Interval Day -> [Evnt] -> Criteria
makeCriteriaRunner index events = criteria [criterion "isEnrolled" crit1]
  where crit1 = enrolled index events
TODO: we could have done this a different way

Create a cohort with multiple indices

TODO

Collect attrition info across partitions

TODO

Create a cohort application and process in AWS batch

TODO

Templates

Features

This section includes description and usage guides for Definition templates. A Definiton is a function that returns a Feature.

buildNofXBase: Basis for N of X pattern

Use this function template to:

  • write another function that answers a question about some count N of events that satisfy a predicate X.

The buildNofXBase template is used a basis for creating new templates with the following pattern:

  1. Filter events to those satisfying two conditions:

    • an interval relation with an AssessmentInterval

    • a provided Predicate (such as containing certain concepts)

  2. Preprocess these events.

  3. Process the events.

  4. Postprocess the events, optionally in conjunction with the AssessmentInterval.

Usage and Examples
example = buildNofXBase combineIntervals (1)
                        (fmap end) (2)
                        (fmap . diff . begin) (3)

The example function above returns another definiton builder that performs this logic:

1 combine the intervals of the input events (collapsing concurring and meeting intervals);
2 get the end of each interval;
3 computes the difference from each end to the begin of the assessment interval.

To then be fully specified as a Definition and used in a project, the example function needs 3 additional inputs:

  1. a function mapping the index interval to an assessment interval.

  2. a predicate function comparing events to the assessment interval.

  3. another predicate function on the events.

For example, the defBaseline180Enrollment below is a Definition that performs the logic of example.

defBaseline180Enrollment = example (makeBaselineMeetsIndex 180) (1)
                                   concur (2)
                                   (containsConcepts ["enrollment"]) (3)
1 Create a baseline interval from the index to 180 units (e.g. days) back in time.
2 Filter to events that concur with the baseline interval and
3 contains the concept "enrollment".
This example assumes the Concept type is Text.
Source code
View source code 
buildNofXBase
  :: ( Intervallic i0 a
     , Intervallic i1 a
     , Witherable container0
     , Witherable container1
     )
  => (container0 (Event c m a) -> container1 (i1 a)) -- ^ function mapping a container of events to a container of intervallic intervals (which could be events!)
  -> (container1 (i1 a) -> t) -- ^ function mapping the processed events to an intermediate type
  -> (AssessmentInterval a -> t -> outputType) -- ^ function casting intermediate type to output type with the option to use the assessment interval
  -> (i0 a -> AssessmentInterval a) -- ^ function which maps index interval to interval in which to assess the feature
  -> ComparativePredicateOf2 (AssessmentInterval a) (Event c m a) -- ^ the interval relation of the input events to the assessment interval
  -> Predicate (Event c m a) -- ^ The predicate to filter to Enrollment events (e.g. 'FeatureEvents.isEnrollment')
  -> Definition
       (  Feature indexName (i0 a)
       -> Feature eventsName (container0 (Event c m a))
       -> Feature varName outputType
       )
buildNofXBase runPreProcess runProcess runPostProcess makeAssessmentInterval relation predicate
  = define
    (\index ->
      -- filter events to those satisfying both
      -- the given relation to the assessment interval
      -- AND the given predicate
      filterEvents
          (Predicate (relation (makeAssessmentInterval index)) &&& predicate)
      -- run the preprocessing function
        .> runPreProcess
      -- run the processing function
        .> runProcess
      -- run the postprocessing function
        .> runPostProcess (makeAssessmentInterval index)
    )

buildNofX: Do N events satisfy a predicate X?

Use this template to create a Definition for a Feature that answers the following question:

  • Do N events relating to the assessment interval in some way satisfy the given predicate?
Usage and Examples

TODO

Specialized Versions
buildNofXBool

specialized to return Bool.

buildNofXBinary

specialized to return a stype Binary value.

buildNofXBinaryConcurBaseline

specialized to filter to events that concur with an assessment interval. created by makeBaselineMeetsIndex of a specified duration and a provided predicate.

buildNofConceptsBinaryConcurBaseline

specialized to filter to events that concur with an assessment interval created by makeBaselineMeetsIndex of a specified duration and that have a given set of concepts.

Source code
View source code 
buildNofX
  :: (Intervallic i a, Witherable container)
  => (Bool -> outputType) -- ^ casting function
  -> Natural -- ^ minimum number of cases
  -> (i a -> AssessmentInterval a) -- ^ function to transform a 'Cohort.Index' to an 'Cohort.AssessmentInterval'
  -> ComparativePredicateOf2 (AssessmentInterval a) (Event c m a) -- ^ interval predicate
  -> Predicate (Event c m a) -- ^ a predicate on events
  -> Definition
       (  Feature indexName (i a)
       -> Feature eventsName (container (Event c m a))
       -> Feature varName outputType
       )
buildNofX f n = buildNofXBase id (\x -> length x >= naturalToInt n) (const f)

buildNofUniqueBegins: Find the begin of all unique N events

Use this template to create a Definition for a Feature that:

  • filters a list of events to those satisfying both a given predicate and a relate to the assessment interval in the given way;

  • returns the a list of pairs (b, i) where

    • b is the difference between the begin of each unique event and the given assessment interval

    • i is a counter starting from 1
Usage and Examples

TODO

Source code
View source code 
buildNofUniqueBegins
  :: (Intervallic i a, IntervalSizeable a b, Witherable container)
  => (i a -> AssessmentInterval a) -- ^ function to transform a 'Cohort.Index' to an 'Cohort.AssessmentInterval'
  -> ComparativePredicateOf2 (AssessmentInterval a) (Event c m a) -- ^ interval predicate
  -> Predicate (Event c m a) -- ^ a predicate on events
  -> Definition
       (  Feature indexName (i a)
       -> Feature eventsName (container (Event c m a))
       -> Feature varName [(b, Natural)]
       )
buildNofUniqueBegins = buildNofXBase
  (fmap (momentize . getInterval))
  (fmap (, 1 :: Natural) .> F.toList .> M.fromList .> M.toList .> \x ->
    uncurry zip (fmap (scanl1 (+)) (unzip x))
  )
  (\window -> fmap (\i -> (diff (begin (fst i)) (begin window), snd i)))

buildNofXWithGap: Do events have a certain gap between them?

Use this template to create a Definition for a Feature that answers:

  • Are there N gaps of at least the given duration between any pair of events that relate to the assessment interval by the given relation and the satisfy the given predicate?

Find two outpatient events separated by at least 7 days is an example.
Usage and Examples

TODO

Source code
View source code 
buildNofXWithGap
  :: ( Intervallic i a
     , IntervalSizeable a b
     , IntervalCombinable i a
     , Witherable container
     )
  => (Bool -> outputType)
  -> Natural -- ^ the minimum number of gaps
  -> b -- ^ the minimum duration of a gap
  -> (i a -> AssessmentInterval a)
  -> ComparativePredicateOf2 (AssessmentInterval a) (Event c m a)
  -> Predicate (Event c m a)
  -> Definition
       (  Feature indexName (i a)
       -> Feature eventsName (container (Event c m a))
       -> Feature varName outputType
       )
buildNofXWithGap cast nGaps allowableGap = buildNofXBase
  (-- just need the intervals  
   fmap getInterval
   -- pairGaps needs List input as the container type
                    .> toList)
  (-- get (Maybe) durations of interval gaps between all pairs
     pairGaps
   -- throw away any non-gaps
  .> catMaybes
   -- keep only those gap durations at least the allowableGap
  .> F.filter (>= allowableGap)
   -- are there at least as many events as desired?
  .> \x -> length x >= naturalToInt nGaps
  )
  (const cast)

buildNofXOrNofYWithGap: Is either buildNofX or buildNofXWithGap satisfied?

Use this template to create a Definition for a Feature that answers:

  • Do N events satisfy predicate X?

  • OR are there M gaps of at least the given duration between any pair of events that relate to the assessment interval by the given relation and the satisfy the given predicate Y?

Find two outpatient events separated by at least 7 days or one inpatient event is an example.
Usage and Examples

TODO

Specialized Versions

TODO

Source code
View source code 
buildNofXOrMofYWithGap
  :: ( Intervallic i a
     , IntervalSizeable a b
     , IntervalCombinable i a
     , Witherable container
     )
  => (outputType -> outputType -> outputType)
  -> (Bool -> outputType)
  -> Natural -- ^ count passed to 'buildNofX'
  -> Predicate (Event c m a) -- ^ predicate for 'buildNofX' 
  -> Natural -- ^ the minimum number of gaps passed to 'buildNofXWithGap'
  -> b -- ^ the minimum duration of a gap passed to 'buildNofXWithGap'
  -> Predicate (Event c m a) -- ^ predicate for 'buildNofXWithGap' 
  -> ComparativePredicateOf2
       (AssessmentInterval a)
       (Event c m a)
  -> (i a -> AssessmentInterval a)
  -> Definition
       (  Feature indexName (i a)
       -> Feature
            eventsName
            (container (Event c m a))
       -> Feature varName outputType
       )
buildNofXOrMofYWithGap f cast xCount xPred gapCount gapDuration yPred intervalPred assess
  = D2C f
        (buildNofX cast xCount assess intervalPred xPred)
        (buildNofXWithGap cast gapCount gapDuration assess intervalPred yPred)

buildIsEnrolled: Does an enrollment event concur with index?

Use this template to create a Definition of a Feature for a Status where:

  • you have a predicate for identifying enrollment events;

  • you want to know whether a subject was enrolled (in a health plan, e.g.) at an index time;

  • the result will be used as inclusion/exclusion status.
Usage and Examples

TODO

Source code
View source code 
buildIsEnrolled
  :: ( Intervallic i0 a
     , Monoid (container (Interval a))
     , Applicative container
     , Witherable container
     )
  => Predicate (Event c m a) -- ^ The predicate to filter to Enrollment events (e.g. 'FeatureEvents.isEnrollment')
  -> Definition
       (  Feature indexName (i0 a)
       -> Feature eventsName (container (Event c m a))
       -> Feature varName Status
       )
buildIsEnrolled predicate = define
  (\index ->
    F.filterEvents predicate
      .> combineIntervals
      .> any (concur index)
      .> includeIf
  )

buildContinuousEnrollment: Does a sequence of enrollment events continuously occur?

Use this template to create a Definition of a Feature for a "continuous enrollment" Status where:

  • you have a predicate for identifying enrollment events;

  • you want to know whether a subject was enrolled (in a health plan, e.g.) during some assessment interval with the possibility for an allowable gap in enrollment;

  • the result will be used as inclusion/exclusion status.
Usage and Examples

TODO

Source code
View source code 
buildContinuousEnrollment
  :: ( Monoid (container (Interval a))
     , Monoid (container (Maybe (Interval a)))
     , Applicative container
     , Witherable container
     , IntervalSizeable a b
     )
  => (i0 a -> AssessmentInterval a) -- ^ function which maps index interval to interval in which to assess enrollment
  -> Predicate (Event c m a)  -- ^ The predicate to filter to events (e.g. 'FeatureEvents.isEnrollment')
  -> b  -- ^ duration of allowable gap between intervals
  ->
    {- tag::templateDefSig0 [] -}
     Definition
       (  Feature indexName (i0 a)
       -> Feature eventsName (container (Event c m a))
       -> Feature prevName Status
       -> Feature varName Status
       )
    {- end::templateDefSig0 [] -}
buildContinuousEnrollment makeAssessmentInterval predicate allowableGap =
  define
    (\index events prevStatus -> case prevStatus of
      Exclude -> Exclude
      Include -> includeIf
        (makeGapsWithinPredicate
          all
          (<)
          allowableGap
          (makeAssessmentInterval index)
          (combineIntervals $ F.filterEvents predicate events)
        )
    )

Cohorts

This section includes description and usage guides for cohort specification templates.

References

Lawvere Theory: https://bartoszmilewski.com/2017/08/26/lawvere-theories/

asclepias GitLab: https://gitlab.novisci.com/nsStat/asclepias

Haskell Setup: nsBuild site

Event Data Model Documentation: https://docs.novisci.com/edm-sandbox/latest