CDISC CORE and Custom Rules and Standards

 

This is a follow up on my prior posts "Adding Custom Rules to CDISC CORE - first experiences" and "Generating custom CORE Rules starting from CDISC SDTM Dataset Specializations".

Introduction

CDISC Open Rules Engine (CORE) is now well established as the best validation software for validation of submission SDTM, SEND and ADaM files. Also FDA reviewers are more and more switching to CORE, as the existing classic validation software they used before is notoriously known for its false positives, forcing them to "dive" in the Reviewers Guide to find out whether these false positives are documented.

CORE has the advantage over the classic validation tools that it is completely open source, whereas the classic tool is "black box", i.e. one does not know how each of the rules has been implemented. One can only guess.
Also, all the rules implemented in CORE are completely open (as either YAML or JSON) and can be inspected by everyone.

One of the advantages of CORE that is not so well known yet, is that everyone, every sponsor, every CRO or service provider, can develop their own, additional rules, and execute them in the CORE engine. Such "custom" rules can be either written in YAML or JSON, usually YAML being the easier choice.

Custom Rules

Suppose we have a trial where subjects older than 65 years are not allowed to be enrolled. To check whether such a subject is present in our DM dataset, we can develop a simple rule in YAML as:

 

In the existing CORE implementation (in this example using the Windows executables), the rule can then be executed using the "-lr" ("local-rules") parameter, like:

 .\core.exe validate -s sdtmig -v 3-4 -lr D:\CDISC_Standards\CDISC_CORE_extensions\AGE_65.yaml
-d D:\MetaDataSubmissionGuide_2_0_SASXPTFiles ...

with the parameter value of "-lr" the path to the (here YAML) file with the "custom rule, and the value of the "-d" parameter the path to the directory with the files to be validated.

The report then e.g. looks like:

 

The "-lr" parameter can not only take simple file as parameter value, but also a directory containing "custom" rules, like:

.\core.exe validate -s sdtmig -v 3-4 -lr D:\CDISC_Standards\CDISC_CORE_extensions

allowing to execute several custom rules in one run.

This can be especially interesting for CROs or service providers that work with different sponsors, studies, and thus, different sets of rules.

Custom Standard Rules

Another possibility is to really "add" sets of custom rules to the CORE engine itself for reuse. We call them "custom standards". Even nicer, it is possible to version such "custom standards", thus making maintenance easy.

The CORE CLI reference explains this pretty well, but of course, an example is always nicer.

First of all, one must add all the rules for a single "custom standard" and "version" in a single directory, usually as YAML files (but also JSON can be used). You will also need a (small) JSON file just containing a list of the rule names, like:

 

stating that this is for "standard" with name "sponsor_1_rules" with version is "1-0" (best to use a dash here) followed by the list of rule names. The latter than correspond to files "Rule_1.yaml", "Rule_2.yaml" etc..
Of course, one can right a little program to automate this step (as we did ...).

These custom rules can then be added to the CORE engine "cache" using the command:

.\core.exe update-cache --custom-standard D:\sponsor1_CORE\sponsor_1_rules.json ...

with the last part of the command pointing to the JSON file with the list, as above.
At the end, you will also need to add your API key using the "--apikey" parameter.

To check whether the set of rules has really been added, use:

.\core.exe list-rule-sets --custom

which returns a list of all custom rule sets in uppercase. For using a set, one must however use the lower-case values as defined in the JSON file.

To use the "custom standard", one needs to provide the keyword "-cs" ("custom-standard") in the validation command, like:

.\core.exe validate -cs -s sponsor1_rules -v 1-0 -d ....

stating that version 1-0 of the "custom standard" "sponsor1_rules" must be executed on the files in the directory provided after the "-d" parameter.

Sometimes, it is also useful to get a list of the rules within a "custom standard", although these are also already listed in the validation report. To do so, one can use:

.\core.exe list-rules --custom-rules -s sponsor1_rules -v 1-0 

It is even possible to ask for a single rule within a "custom standard", like:

.\core.exe list-rules --custom-rules -s sponsor1_rules -v 1-0 -r Rule_1

where one can leave out "-s sponsor1_rules -v 1-0" when one wants to obtain all rules named "Rule_1" within any if the custom standards and versions.

Using the python version of CORE

All the above assumes that the Windows executables of CORE are used. One can of course also use the native Python (source) code, even a Mac or Linux version.
For the use with Python, just replace ".\core.exe" with "python core.py" 

Rules generated from the CDISC "SDTM Dataset Specializations"

CDISC "Biomedical Concepts" are used more and more, especially in the generation of a digital protocol as USDM together with the "SDTM Dataset Specializations". The latter are essentially "SDTM implementations" of the Biomedical Concepts. For example, the SDTM Dataset Specialization for DIABP states that it must be mapped to the VS domain, with VSORRES expected to be an integer, and VSORRESU expected to be either "mmHg" or "cmHg", and VSPOS being limited to "PRONE", "SEMI-RECUMBENT", "SITTING", "STANDING", and " SUPINE".

Essentially, one can also consider this a set of rules. As one sees, one "SDTM Dataset Specialization" can then lead to several rules.

We found a way to automatically create a large set of rules (over 2000) from the "SDTM Dataset Specializations", in YAML format, which we then loaded as a "custom standard", with the name "cdisc_dataset_specializations" version 1-0. Later, a new release of these SDTM Dataset Specializations was published by CDISC, so we did the exercise again, and thus created and loaded a version 1-1, which now has over 2600 rules.

If you would like to obtain a copy of these, with or without consultancy services, please let us know.

Implementation and integration in (existing) software

Many of us will not prefer to always do CORE validation using CLI (command line interface). The latter is of course ideal for "batch" implementation, e.g. in "pipelines", for example, when SDTM datasets regularly are received and then automatically and immediately validated without human intervention.

A great advantage of CORE is that one can easily integrate it into existing software, or build a Graphical User Interface (GUI) on top of it. For the first (integration), we did so in our well-known SDTM-ETL mapping software. This means that when using SDTM-ETL, one can immediately validate mapping results, not only against the CDISC rules and FDA business rules, but also against the rules derived from the "SDTM Dataset Specializations", and even against own custom rules that have not been loaded into the CORE engine.

We have also integrated CORE, together with the rules derived from the "SDTM Dataset Specializations" in our open source "Smart Submission Dataset Viewer". We have build the GUI in such a way that the user can choose between either the "CDISC rules" or the "SDTM Dataset Specializations rules" (doing both in one run is technically not possible) or "own rules". Below are a few screenshots with explanations.

Example 1:

This screenshot comes from the open source "Smart Submission Dataset Viewer", with integrated CORE engine, including the custom rules derived from the CDISC SDTM Dataset Specializations.

 

where violations are highlighted in the "table view". This is made possible by having the CORE engine generating the result output as JSON, and the GUI picking up this output and using it to render the cells that violate a rule with a separate background color, together with a tooltip explaining what the violation is.

Example 2:

Also a screenshot from the "Smart Submission Dataset Viewer" showing how the user can select rules, in this case the "custom rules" generated from the CDISC SDTM Dataset Specializations. This can be done just by selecting single rules, or using the "Filter" button to select sets of rules by specific criteria. For further details, one can download the documentation file "Smart_Submission_Dataset_Viewer_CDISC_CORE.pdf" from the SourceForge site.

 

Example 3:

Most of these features have also been implemented in our popular SDTM-ETL [] mapping tool.
In the newest version, which will also support the new SDTMIG-4.0, it is even possible to perform validate individual mapping scripts using CORE. Below is a screenshot demonstrating this.

 

Conclusions

As "open source", CORE is not only superior to existing validation tools for CDISC submissions, but also has the advantage that companies can easily add their own validation rules, bundle them into "custom standards" and add them to the CORE engine.
Furthermore, as we have shown here, CORE can rather easily be integrated into existing software, or a GUI being build on top of it.

If you want to know more about how we can help companies with the deployment and integration of CORE, or need help with the development of "custom rules", always feel free to contact us.

As always, comments to this article are highly appreciated!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

SDTMIG-4.0 Draft - The new LC (Lab Test Results in Conventional Units) domain

 

CDISC just published a new set of draft SDTM domains for the future SDTMIG-4.0 standard. One of these "new" domains is LC: Lab Test Results in Conventional Units.
This until now non-official domain was already mentioned in numerous FDA publications, including Technical Conformance Guides (TCGs). From the June 2026 publication:

  

LC was not present in SDTMIG-3.4, so it is clear that the new LC domain has been added on request of the FDA.

The CDISC draft specification of LC states:
"Not all regulatory agencies require the LC domain in submissions. Refer to the specific guidance from the relevant regulatory authority to confirm expectations and timing for inclusion". It does not specifically state that it is "FDA only", but it is suggested by a sentence in the examples: "This example illustrates the US FDA requirement to have both LB and LC domain datasets for the same data within a submission".

What is the LC domain and how does it differ from LB?

If one reads the "assumptions", it becomes clear that an LC dataset is expected to be identical to the LB dataset (exactly the same number of rows, exactly the same number of variables, only with "LB" replaced by "LC" in the variable names), but with all values for "standardized results" using "conventional units" instead of "SI units".

This means that LCSTRESC/LCSTRESN/LCSTRESU must be using "conventional units", whereas LCORRES and LCORRESU must be identical to LBORRES/LBORRESU in the corresponding LB dataset.

Rather surprisingly, the draft specification does not say anything explicitly about LCSTNRLO and LCSTNRHI (reference ranges in standardized units), but from the examples, it shows that these must then also be using the "conventional units".

What are "conventional units"?

So, what are "conventional units"?
A bit of research reveals that there is no formally defined specification of what a "conventional unit" is, nor is there an official list. Usually, "SI units" refer to "molar" units, like "mmol/L", i.e. molar concentrations, whereas "conventional units", refer to "mass concentrations". BUT: there are exceptions … 

For example, FDA regards "g/L" not as a "concentional unit", but requires "g/dL" instead as the "conventional" unit. It seems to consider "g/L" as "SI", although it is not a "molar concentration". This is also visible from the examples in the draft specification.
Another example is "L/L", which is often used for hematocrit measurements. This is regarded as an "SI unit", with "%" as the conventional unit.

As said, unfortunately, there is no formal list of what exactly is a "conventional unit", and what is not. Even the CDISC specification of LC does not provide a definition at all, nor a link to a definition or list.
So, it all seems to be based on "tradition" …

Why this "new" domain?

 The question then arises why CDISC wants to have this "new" LC domain, which is an almost exact copy of LB, with just other values and units for a few variables only.
It is clear that this is then an "FDA-specific" domain, and probably only added by request of the FDA. Did CDISC (once again) give in on the "requirements" of one single regulatory authority, whereas there are much more simple solutions?

The most plausible reason for this new domain, and the FDA requirement to submit a dataset for it according to the FDA-TCG, is that reviewers are not capable to do any unit conversions themselves, or that their software systems are not able to provide such features. So we must do it for them …

The NLM RESTful Web Services

This is a bit surprising, as the NLM (National Library of Medicine), a US institution (!) offers a RESTful Web Service (RWS) to automate such conversions. It can be found at https://ucum.nlm.nih.gov/ucum-service.html.
It uses the UCUM international standard for unit notation, but for concentrations, CDISC units are for over 95% identical to UCUM. The other way is not true, as CDISC has a "list" of units, whereas UCUM is a "system". So, for concentrations, one can say that the CDISC units are mostly a subset of UCUM.

Let us take an example of the LC draft specification, example 2. This is the only example where LB contains a "molar" concentration, example 1 has no such, though typical, rows:

We find:

  

 Using the NLM RESTful Web Service, this conversion is easy to automate. The RWS request simply is:

https://ucum.nlm.nih.gov/ucum-service/v1/ucumtransform/4.9/from/mmol/L/to/g/dL/MOLWEIGHT/64458

leading to the result:
 

with the molecular weight of human hemoglobin being 64458, which is the molecular weight of the tetramer.
If however a molecular weight of 16000 is used, the result is:  

When we look into example 2 however, we see that the conversion may not been done correctly, depending on which molecular weight was used.

 

From the information in the example itself, it is completely unclear which molecular weight was used for the conversion. We can just guess that 16000 was used. It is however not documented at all.

When one does have the LOINC code available, the RWS request is even easier. For example for glucose in blood, starting from the 5 mmol/L with the LOINC code being 14771-0 and we want to convert to g/dL, the request is:

https://ucum.nlm.nih.gov/ucum-service/v1/ucumtransform/5/from/mmol/L/to/g/dL/LOINC/14771-0

with the result being:

 

Essentially, it looks as FDA reviewers are not capable to do unit conversions in an automated way.
Can it be that they do not have internet access? Or do their tools not support (or do not allow to) use RESTful Web Services? Even within SAS, this is extremely easy to accomplish, like:

 

Remark that I generated this script using AI - I am not a SAS specialist at all ...

Can't FDA reviewers simply implement such a script in order to do the unit conversions themselves? Why do we need to do these for them? It is just another source of possible error ...

That even the CDISC example shows a result that depends on which value for the molecular weight was used (and which is not documented in the file), raises a lot of issues:
For example: how can FDA reviewers check that the value in e.g. LCSTRESN is correct when they can't even use RESTful web services? Will they just believe that everything the sponsor provides in LC is correct? This can easily lead to catastrophe ...

So, what I am expecting, is that once sponsors start really using LC (although I presume they already submit the "unofficial" LC dataset), and FDA reviewers somehow find out that some of the conversions have not been done correctly, they will ask CDISC for yet another domain, this time a non-subject domain, like TC, "Trial Conversion Units", where the sponsor must submit all the conversion factors used in the study to generate LC.

But then, how will or can the FDA reviewers then check that the conversion factors used are correct? Checking this will consume review time ...

In the mean time, patients are dying as it takes too long before their life-saving medication becomes available ...

Other issues with LC

The LC specification states the LCLOINC is a "permissible" variable. However, the FDA requires the LOINC code to be submitted, when available from the lab. So, as this is an "FDA-specific" domain, shouldn't LCLOINC be at least "expected"? Or did the authors of the specification just copy of what is in LB?

The other issue is that when "conventional units" are provided, it can be that the LOINC code also changes. For example for "concentration of glucose in urine", when "molar concentration" is provided, the LOINC code is 15076-3.

When however, "mass concentration" (i.e. "conventional unit") is provided, the LOINC code is 2350-7, with the typical unit being mg/dL. So, if we collected "concentration of glucose in urine" in molar units in LB with LBLOINC=15076-3, and then generate the LC dataset, what should we use for LCLOINC?
The draft specification does not say anything about this …
 

When we look at example 2 in the draft specification (Hemoglobin in blood) which has mmol/L in LB and g/dL in LC, we see that LBLOINC=59260-0 and LCLOINC is also … 59260-0, which is essentially wrong. The correct LOINC code for hemoglobin with units of g/dL is 718-7. So it seems that, just from this one example, it is expected that the LOINC code must be copied from LB to LC, although this leads to an incorrect LOINC code in LC, at least for the "standardized" test.
The draft specification hower does not say anything about this at all.

The specification also says nothing about LCSTNRLO and LCSTNRHI. Must the values also be converted or does not need to copy them from LB? The specification doesn't say it. When however looking at the examples, it becomes more or less clear that they also need to be converted, also this is never explicitly written.

Other minor, but not unimportant issues with the draft SDTMIG-4.0 specification

When studying this draft specification, I noticed a few things for which there were a lot of comments in the first draft publication, but seems not to have been handled at all yet.

- The specification does not explicitly what the "label" for the dataset is. As well for SAS-XPT as for Dataset-JSON submissions the "label" must be provided and also goes into the define.xml under ItemGroupDef/Description.

The specification provides:

 

Ok, we all know from the previous IGs that what is highlighted here in yellow goes into the "label", but it would of course be better if it was stated explicitly - I don't like "interpretation by tradition" …

- The second issue which had a lot of comments was about the designation of "lc.xpt". Essentially, it suggests that the only possible for datasets, whether they are submitted or not, is SAS Transport 5. We all know that it can be expected that the FDA may soon allow submissions in the modern Dataset-JSON format. If "lc.xpt" (and of course others) is kept, it could give the opponents of a modern submission format another argument, like "it is not allowed by the SDTMIG" …
Personally, I think this is a serious issue.

 

SDTM-ETL and R: Best of both Worlds?

Some time ago, there was an interesting discussion on LinkedIn (unfortunately I cannot find it anymore) about R versus SAS for generating SDTM datasets. My response was that I considered both inefficient and expensive (in training cost and/or license costs) as there are now software tools on the market that have a graphical user interface (GUI), use "drag-and-drop", combined with wizards, to automate the mappings and generate the SDTM dataset. Examples of such software are our own SDTM-ETL, but also Ryze by Formedix (now owned by Certara) and XClinical's Marvin-Tabulator. This triggered a discussion about advantages and disadvantages of such "specialized for SDTM" software versus general purpose software such as R. One of the arguments by a contributor (not me) was that R is more flexible, whereas SDTM-specialized software such as SDTM-ETL, due to the user-friendliness, will probably be considerable more efficient for 80% of the studies. For very complicated studies however, and especially for studies with low-quality source data, R would then be the better choice due to the flexibility. Another contributor then asked whether there is "SDTM-specialized" software, that has a user-friendly GUI, uses "drag-and-drop", has build-in SDTM knowledge, but uses R as the scripting language.

As far as I know and could find out, there is no such software available.

SDTM-ETL uses "EasyXSL" as the scripting language (80-90% of the scripts are generated automatically), which is an easy-to-learn language that is on top of XSLT, the very powerful XML transformation language. This means that upon execution of the mappings, the EasyXSL scripts are combined (per domain or dataset definition) and are translated into XSLT, which is then executed on the source data, with generation of the SDTM datasets in either SAS-XPT, modern CDISC Dataset-JSON-1.1 format, or CSV.

So the idea was born: is it possible to have a version of SDTM-ETL, with all its advantages, which uses R as the scripting language instead of EasyXSL? This would mean that the by SDTM-ETL generated R-scripts can be fed into an R engine, and then be executed by that R engine. Essentially it would mean that it would be possible to combine the user-friendliness (and build-in SDTM knowledge) of SDTM-ETL with the flexibility and extensibility of R.
This could then be a "best of both worlds".

So we decided to start developing a prototype (just as a "proof of concept") of such a software. For the moment, we call it "SDTM-ETL-R".

What is important for us is that all existing "drag-and-drop" features and "SDTM knowledge" and  "wizards" remain intact, and that only difference is that the final generated script is now in R instead of in EasyXSL. For the moment, the execution is then done in R-Studio.

We started with DM as the first domain, but this was pretty easy, as DM only has 1 record per subject. For example, the mapping for RACE, automatically generated by SDTM-ETL, is then found to be:

 

and when tested on a set of clinical data (in CDISC ODM format), as exported from an EDC system, using the button "Test - Run in R":

and providing a file with clinical data by a "file chooser" (not shown here), the script is extended and reorganized in order to run for each subject in the source, and then opens in R-Studio:

 

When then run, the "DataFrame" is generated and displayed in R-Studio:

 

As this was "too easy", we started working on LB (Laboratory), which is always more difficult. 
The output from the EDC system (in ODM) is organized as follows:

  

Do not mind the colors too much for the moment - it just means that the items have annotations attached, already stating where they are supposed to come in SDTM. 
We always first start with the "topic" variable, which is LBTESTCD in this case. What we need to accomplish is to obtain one record per test (described by LBTESTCD/LBTEST) per visit per subject. As one can easily deduce, there are two tests here: "Erythrocytes" and "Leukocytes" (this is very simple example study for demo purposes only).
The target SDTM variable is shown in the table on the right side of the screen:

So we drag either "Erythrocytes" or "Leukocytes" from the left to the cell "LB.LBTESTCD". When we drag-and-drop "Erythrocytes", a dialog appears:

It automatically suggests that we take the identifier (Item-OID) from the source, not the value.
But we also want to have an entry in the output for Leukocytes, not only for Erythrocytes, and we want to have that for each visit.
So we select the checkbox "Generalize for all StudyEvents", and also check the checkbox "Generalize for all items", followed by clicking "Only for":

 

A list is then displayed from which we need to choose - we select "Erythrocytes" and "Leukocytes":

 

We do not want to use any of the other items for now, we will need them later to e.g. populate LBSTAT, LNORNRLO, LBORNRHI, LBNAM, etc..

Clicking OK then leads to the next "wizard":

 

The system loads the SDTM Controlled Terminology (CT) for LBTESTCD from the underlying define.xml, and presents all the values in the comboboxes:

 

Again we are lazy and allow the system to find out what the mappings need to be based on word similarity, if necessary additionally supported by CDISC's "synonym list", or our company synonym list. The latter can e.g. be useful when the lab test names are obtained in another language than English. So, we click "Attempt 1:1 mappings". The system, in all its wisdom, then proposes:

 

which looks fine …

At this stage, we can also generate the subset codelist for LBTESTCD and have it stored in the underlying define.xml.
Reason is that in SDTM-ETL, everything is stored in an underlying define.xml, which also means that at the end of the process, we do already have a define.xml, which we only need to clean up a bit to come to a "submission-ready" define.xml.

Clicking "OK" then generates the mapping script completely automatically:

For those who understand "path-systems" a bit, in line 6, it selects the OIDs (identifiers) in the source data for "I_LB_RBC" "I_LB_WBC" in the subform with identifier "IG_LB_HEMATOLOGY" in the form "F_LAB" and that for all visits (StudyEvents).
It then executes the mapping from the original item identifier to SDTM-CT, stores it in a temporary value "NEWCODEDVALUE" and in the last line stores it in the variable LB.LBTESTCD.
Simple, isn't it? And all that without writing a single line of code.

If then the button "Test - Run in R" is clicked, the script is extended and slightly reorganized to be able to run "standalone" in R-Studio:

and when run, leads to the following DataFrame in R-studio:

 

It doesn't show any other information yet, like the visit number, the date/time of collection, etc.., as it is only "local" testing. It is only when the R-scripts for each of the variables are combined, the full picture will appear.

For LBORRES, we can do exactly the same drag-and-drop, and as the system remembers the previous settings, and the sytem knows it now need the value, not the item identifier, this just leads to a one-liner in the mappingscript editor. When then clicking "Test - Run in R, this leads to the following in R-studio:

Good observers have already noticed that the upper part is always (almost) identical, setting up things, and loading a set of functions. In the lower part, the mapping script from the "Mapping editor" is just embedded into an iteration over all subjects. The way we do this may however change in future.

For LBTEST (test name), we could follow the same procedure as for LBTESTCD, as the system already knows what the codelist for LBTEST is. However, in "regular" SDTM-ETL, there is a smarter way: it has a function "decode", which takes the value of a --TESTCD variable, and then "decodes" it to the value of LBTEST.
Although not everything is ready yet, we already made a function for that in R: 

It just accepts the location of the underlying define.xml, the OID of the codelist (which is in the underlying define.xml anyway) and the value of --TESTCD.

In the mapping for LBDTC (date/time of collection), there is a peculiarity that we need to concatenate the value of the visit date and the visit time:

 

So we need two drag-and-drops.
The peculiarity here is that at the second drop, the path needs to be relative to the first drop. Also this is something that we solved, leading to the "test" R-script in R-Studio:

 
Resulting in the DataFrame:

 

Some of you may already ask themselve: "What is the relationship with Oak", the CDISC COSA (CDISC Open Source Alliance) project?". 

Oak is a great library of R-functions for use in developing mappings to SDTM. Oak doesn't have any build-in knowledge of CDISC standards though. As far as I could find out, it doesn't have functions for ODM import (or selection of items from ODM for mapping), it is still up to the user to import data in any form (usually "flat" format) and make it suitable for mapping to SDTM. Neither does it have an export function to SAS-XPT (that can however be realized using the xportr package), nor for modern CDISC Dataset-JSON (which can however be done using the "datasetjson" package). It has also no functions for e.g. automated codelist-codelist mapping based on word similarity, synonyms or company-specific libraries. It also does not use or produce define.xml. In Oak, CDISC codelists are to be read in as CSV files. Oak also doesn't seem to use RESTful Web Services (RWS), doesn't know what the CDISC Library is or does, nor has it an automated way of generating LBTESTCD, LBTEST, LBMETHOD, etc. starting from the provided LOINC code using the famous "LOINC-CDISC Mapping" services. This is one of the functionalities that is most appreciated by our users in our "regular" SDTM-ETL. Such a function can however also easily be implemented in R starting from the following code:

and as an example result for LOINC code 34183-2 (Total Beta globulin/Protein in 24 hour Urine):

 

So one cannot see Oak as a "complete solution". It is a set of useful functions for R, i.e. provides some easy-to-use "building blocks". This of course also provides the flexibility I talked about before.
Some of these Oak functions can however also be used in SDTM-ETL-R, as Oak has a good amount of functions for "post-processing", such as for assigning --SEQ values, derive the --LOBXFL (Last Observation before first Exposure Flag), study day derivation (--DY values), a function to move "non-standard variables" (NSVs) from the original domain to a SUPPxx dataset (unfortunately the information about the NSV needs to be delivered as … CSV …), derive DM.RFXSTDTC (Date/Time of First Study Treatment) and DM.RFXENDTC (Date/Time of Last Study Treatment) from information in the EX dataset. Such "post-processing" steps ones are done by XSLT or streaming parsing (SAX) in "regular" SDTM-ETL. What I did miss in Oak however, and which is present in "regular" SDTM-ETL is a function for automated assignment of VISITNUM for unscheduled visits. Also there do not seem to be any functions for generating RELREC or SV (Subject Visits) in Oak. These are easily generated in "regular" SDTM-ETL though.

So yes, we will very probably use Oak in SDTM-ETL-R, especially for some of the "post-processing" steps such as --SEQ assignment, --DY generation, --LOBXFL assignment, and maybe generation of SUPPxx. For the latter however, we do not want to use CSV files, as in SDTM-ETL, the information of what an NSV is (and its properties) is already in the underlying define.xml. This makes more sense.

Even when we use these Oak functions, we do not need to change anything in the Graphical User Interface:

 

What we will need to do however is to replace the existing "post-processing" modules by (calls to) Oak R-functions.

What are the next steps?

First we need to further adapt our existing mapping functions (we have about 40 of them) to R. This will also enable us to extend our example R-mappings as a "test bed" for how to proceed.
We will also start working on assembling the individual mappings for the variables for a single domain into a single R-script. This can be a script that calls (or imports) other R-scripts, or assembles everything into a single script. We still need to find out what is best. Of course, this "domain" script (or scripts) will call other functions, our own ones as well as Oak ones.

We will regularly report about our progress.

Personally, I am still convinced that, due to its user-friendliness, build-in SDTM knowledge, and many functionalities for which there is no equivalent in R nor Oak, "regular" SDTM-ETL can be the first choice for 80% of the studies, if not for 90% of the studies. The use of R however has the advantage of more flexibility that can be of advantage for the remaining 10-20%, for example for very complicated studies, or for studies where data quality is low and need a lot of "rearrangement" of the data. The use of R allows to e.g. change the order in which scripts are executed, or insert "special" scripts for exceptional cases. This is also possible with the XSLT generated by SDTM-ETL (and which most of the users never see), but there are not so many people who have such a good knowledge of XSLT that they can "adapt" this XSLT. For R however, there is a much large "specialist" base, especially as R is used a lot in academia, and millions of people are educated in it (even my daughter who studied "international development studies" had to learn it).

So, a software tool with the user-friendliness and build-in SDTM knowledge such as SDTM-ETL, but that uses R and Oak for the scripts may have a place in the market for SDTM (but also SEND) mapping tools. As such, it may combine "best of both worlds".

Do you think we should further develop SDTM-ETL-R? Please submit a comment here, or send us an e-mail.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Adding Custom Rules to CDISC CORE - first experiences

 

Today, I started gaining experience with the CDISC CORE feature that one can add its own "custom" core rules to the CORE engine. Essentially, this is a great feature, not offered at all by e.g. Pinnacle21.

How to add custom rules to the CORE engine is explained on the CORE GitHub website, but I still had to experiment a bit to get it all right. That is also the reason for this blog, so that the reader has a clear path and explanation how this can be accomplished.

As I did already have some experience with the development of CDISC CORE rules as a volunteer (I was made responsible for the SEND rules, and some of the FDA Business Rules", I could make a jumpstart.

So, I first made a custom rule that all subjects in my studies may not be older than 65 years:

simply using NotePad++.

Important here (as I found out mostly by trial and error) are:
- You can and should define your own "Organization"
- Normally, you will define custom rules as part of your own (set of) standard(s), in this case, "XML4Pharma_Standard". Originally, I had "CDISC" here, which caused my rule not to run. In this screenshot, that is already corrected. CROs may have several such sets of standards, e.g. for each sponsor customer.
- Assign an own ID for the rule, which is different from any of the CDISC CORE IDs

I also added a second one (with ID XML4P2) stating that AEDECOD must always be populated.
Whereas the first rule is more meant for specific studies, the second is more a "Quality Assurance" rule for submissions to regulatory authorities.

I then added the YAML files (one per custom rule) to a directory where we will manage all your custom rules. In our case case, we used the directory "D:\CDISC_Standards\CDISC_CORE_extensions":

 

Then, using Windows Powershell and the CORE Windows distribution, I used the command:

.\core.exe update-cache --custom_rules_directory D:\CDISC_Standards\CDISC_CORE_extensions

which failed, as the engine wants to have the "api key", message = 

Error: Missing option '--apikey'.

So I added it:

.\core.exe update-cache --custom_rules_directory D:\CDISC_Standards\CDISC_CORE_extensions --apikey {apikey}

where {apikey} is replaced by my real API key.

This lead to the message:

Added 2 rules: XML4P2, XML4P1
Cache updated successfully.

One also needs to set up a file that links the custom rules with a "custom standard", which is done in a JSON file.
In our case, I defined a "custom standard" named "xml4pharma_standard", with the contents:

{ 
  "xml4pharma_standard/1-0": [ 
    "XML4P1", "XML4P2" ]
}

Remark that I used lowercase for "xml4pharma_standard": the value will later be used case-sensitive when starting a validation. It took me some time realizing that …
The value after the slash is the version of our custom standard. This will enable versioning of rules and standards.

This JSON file with links between the custom standard and custom rules is then submitted to the engine using:

.\core.exe update-cache --custom_standard D:\CDISC_Standards\CDISC_CORE_extensions\XML4Pharma_Standard.json --apikey {apikey}

Leading to the message:

Added 1 new standards
Cache updated successfully

One can then check whether these are found, using:

 .\core.exe list-rules --custom_rules

which shows both the custom rules I defined in JSON format (although I defined them in YAML):

or, leading to the same result, as we only defined one "standard":

.\core.exe list-rules --custom_rules -s xml4pharma_standard -v 1-0

One can however imagine that a CRO may want to define several "custom" standards, one for each sponsor customer.

However, the command:

.\core.exe list-rule-sets

does not list the newly defined standard - it only lists the rules defined by CDISC.

For running against the custom rules, I then used:

.\core.exe validate -cs -s xml4pharma_standard -v 1-0 -d D:\MetaDataSubmissionGuide_2_0_JSON_1-1_Files_testing\SDTM_Dataset-JSON

I found that two things are very important to get this to work:

* one really needs to add the "-cs" keyword
* for the standard name, provided immediately after "-s", this is case sensitive.
When one e.g. uses "-s XML4Pharma_Standard" validation executes, but in the result Excel report, the tab "Rules_Report" will not list any results. So the name of the standard in the ".\core.exe validate -s" must exactly match with what is in the file XML4Pharma_Standard.json:

{ 
  "xml4pharma_standard/1-0": ["XML4P1", "XML4P2" ]
}

I then noticed that, although my study has participants over 65 years old (DM.AGE, DM.AGEU=YEARS), no issues were reported for DM. Reason was that originally, my rule originally stated that this is an SDTMIG rule, instead of that it is a rule of my custom standard. So I corrected the YAML, and then used the "update-cache" "--update_custom_rule" command

.\core.exe update-cache --update_custom_rule D:\CDISC_Standards\CDISC_CORE_extensions\AGE_65.yaml --apikey {apikey}

leading to the message:

Updated rule: XML4P1
Cache updated successfully

I then ran:  

.\core.exe validate -cs -s xml4pharma_standard -v 1-0 …

again, and indeed, the engine reported issues for all the subjects in DM that are over 65 years old:

 

Other important remarks:

* I used the CORE Windows distribution 0.10.0 for adding these custom rules. This will of course also work in the same way when using the native Python distribution: one just needs to replace ".\core.exe" by "python core.py".
For Linux (and probably for Mac), one should just use "./core" instead of ".\core.exe".
* I haven't found out how one can run the CDISC/FDA rules together with the custom rules in just one run. When I do so (if it is possible) I will update the blog.

Future work:

We recently developed software that uses the CDISC Library and its API to query the CDISC "Dataset specializations".
Essentially, these describe combinations of variables and their properties that belong together for individual cases. For example, in SC (Subject Characteristics) , when SCTESTCD=MARISTAT (Marital Status), it is expected that the value of SCSTRESC (Standardized result) is one of:
[ANNULLED, DIVORCED, DOMESTIC PARTNER, INTERLOCUTORY, LEGALLY SEPARATED, MARRIED, NEVER MARRIED, POLYGAMOUS, SEPARATED, 'WIDOWED], 
which can be translated into a CORE check:

In the define.xml, this will usually be defined by a "ValueList" (yes, we also have already automated the generation of such valuelists using the CDISC Library API).

We are currently setting up a library (as a separate "standard") that we will add to the CORE engine as a set of custom rules. This will be explained in a separate blog.

Another interesting thought is about the use of RESTful web services (RWS) to have "on-the-fly" rules. For example, to test whether the values of LBTESTCD, LBTEST, LBMETHOD, LBRESTYP, LBRESSCL, …, are in accordance with the value in LBLOINC.
We do already have the mappings between LOINC and SDTM for about 10,000 LOINC codes and an API for it, so taken LBTESTCD, LBTEST, LBMETHOD, LBRESTYP, LBRESSCL, ... into account, this would lead to >40,000 rules.
Is that what we want? Probably not …
We will probably want have only 5-6 rules that take the value of LBLOINC, then uses the RWS, and check whether the value in LBTESTCD matches. The same for LBTEST, LBMETHOD, etc.

One could also think about having something similar for checking e.g. whether the value in AEDECOD (which usually takes the "preferred term" in MedDRA) and the value in AELLT (MedDRA "Lower Level Term") match. However, in such a case, we cannot use a public RESTful Web Services due to license restrictions, but we can think of companies having their own (licensed) implementation of MedDRA as a central database, and having CORE rules that query this database, e.g. using an RWS.

Essentially, we are only at the beginning of exploring the opportunities that CORE, as Open Source software brings us. We can even start thinking about the use of "Dataset Specializations" and "CORE on the fly" during the development of the mappings. Image for example someone coding that the standardized result (--STRESC) for LBTESTCD=WEIGHT, is "cm", then the system would immediately protest and stating that only "kg" and "LB" are allowed. 

The only real limitation is (as so often): lack of imagination ...