Skip to content

Tutorial14_ValidateDataAgainstSchemaDefinitions.md

Latest commit

 

History

History
287 lines (222 loc) · 12.7 KB

Tutorial14_ValidateDataAgainstSchemaDefinitions.md

File metadata and controls

287 lines (222 loc) · 12.7 KB

Validate data against schema definitions

The ModelParser class is used to determine whether one or more DTDL models are valid, to identify specific modeling errors, and to enable inspection of model contents. In addition, the object-model representation of model content can be used to validate instance data for conformance with the model. This tutorial walks through the validation process.

Understand instance data

In this context, the term "instance data" refers to JSON text that can serve as a Telemetry message payload, as a Property in a twin, or as the body of a Command request or response.

To be considered as valid, instance data must be syntactically correct JSON, and it must also conform to the defined schema type. For example, if a Telemetry schema is "long", the message payload for an instance of the Telemetry must be a JSON number value that satisfies the constraints of the XML Schema Datatype "long": It must be in the range [-9223372036854775808, 9223372036854775807] and contain no decimal point.

For DTDL complex schema, instances are either JSON objects (for DTDL Map and Object) or JSON arrays (for DTDL Array). The property values in JSON objects are valid if they conform to the schema specified for the Map "value" or Object "fields" property, and the element values in JSON arrays are valid if they conform to the schema specifed by the Array "elementSchema" property.

Create a ModelParser

To parse a DTDL model, you need to instantiate a ModelParser. No arguments are required.

var modelParser = new ModelParser();

Obtain the JSON text of a DTDL model

The DTDL language is syntactically JSON. The ModelParser expects a single string or an enumeration of strings. The single string or each value in the enumeration is JSON text of a DTDL model.

string jsonText =
@"{
  ""@context"": ""dtmi:dtdl:context;3"",
  ""@id"": ""dtmi:example:anInterface;1"",
  ""@type"": ""Interface"",
  ""contents"": [
    {
      ""@type"": ""Property"",
      ""name"": ""installation"",
      ""schema"": {
        ""@type"": ""Object"",
        ""fields"": [
          {
            ""name"": ""revision"",
            ""schema"": ""integer""
          },
          {
            ""name"": ""installed"",
            ""schema"": ""date""
          },
          {
            ""name"": ""note"",
            ""schema"": ""string""
          }
        ]
      }
    }
  ]
}";

Submit the JSON text to the ModelParser

The main synchronous method on the ModelParser is Parse(). One argument is required, which can be either a string or an enumeration of strings containing the JSON text to parse as DTDL. If the submitted model is complete and valid, no exception will be thrown. Proper code should catch and process exceptions as shown in other tutorials such as this one, but for simplicity the present tutorial omits exception handling.

IReadOnlyDictionary<Dtmi, DTEntityInfo> objectModel = modelParser.Parse(jsonText);

Display elements in object model and their types

The object model is a collection of objects in a class hierarchy rooted at DTEntityInfo. All DTDL elements derive from the DTDL abstract type Entity, and each DTDL type has a corresponding C# class whose name has a prefix of "DT" (for Digital Twins) and a suffix of "Info". The elements in the object model are indexed by their identifiers, which have type Dtmi. The following snippet displays the identifiers of all elements in the object model:

Console.WriteLine($"{objectModel.Count} elements in model:");
foreach (KeyValuePair<Dtmi, DTEntityInfo> modelElement in objectModel)
{
    Console.WriteLine(modelElement.Value.EntityKind.ToString().PadRight(12) + modelElement.Key);
}

For the JSON text above, this snippet displays:

9 elements in model:
Field       dtmi:example:anInterface:_contents:__installation:_schema:_fields:__revision;1
Field       dtmi:example:anInterface:_contents:__installation:_schema:_fields:__installed;1
Field       dtmi:example:anInterface:_contents:__installation:_schema:_fields:__note;1
Object      dtmi:example:anInterface:_contents:__installation:_schema;1
Property    dtmi:example:anInterface:_contents:__installation;1
Interface   dtmi:example:anInterface;1
Integer     dtmi:dtdl:instance:Schema:integer;2
Date        dtmi:dtdl:instance:Schema:date;2
String      dtmi:dtdl:instance:Schema:string;2

Only the Interface identifier dtmi:example:anInterface;1 is present in the DTDL source model. The identifiers for the Property, the Object, and the three Fields are auto-generated by the ModelParser following rules that guarantee their uniqueness. The last three identifiers represent elements in the DTDL language model for the three primitive schemas referenced by the Fields in the model.

Validate instance data against primitive schema

The DTEntityInfo class offers a method for validating instance data:

IReadOnlyCollection<string> ValidateInstance(string instanceText);

This method has different implementations for each subclass, most of which merely throw a ValidationException, which indicates that instance validation is not meaningful for these types. The only classes that support ValidateInstance() are subclasses of DTSchemaInfo, which includes both primitive and complex schema types.

The return value from ValidateInstance() is a collection of strings. Each string describes a specific violation of the instance text with respect to the schema type performing the validation. If the instance text conforms to the schema, the method returns an empty collection.

To make the tutorial code somewhat cleaner, we define a small function that calls the ValidateInstance() method and produces a single string:

Func<DTEntityInfo, string, string, string> schemaValidator = (s, t, v) =>
{
    IReadOnlyCollection<string> violations = s.ValidateInstance($"{v}");
    return violations.Any() ? string.Join(" AND ", violations) : $"{v} IS A VALID {t}";
};

We begin by validating tutorial data against a few of the primitive schema types defined by the DTDL language model. Because the DTDL model has references to the primitive schema types integer and string, the elements representing these schema types are included in the object model. Therefore, we can retrieve them from the object model using their identifiers:

var integerSchema = objectModel[new Dtmi("dtmi:dtdl:instance:Schema:integer;2")];
var stringSchema = objectModel[new Dtmi("dtmi:dtdl:instance:Schema:string;2")];

The following code shows whether various JSON text strings are valid intances of DTDL integer or DTDL string:

Console.WriteLine(schemaValidator(integerSchema, "INTEGER", "3"));
Console.WriteLine(schemaValidator(stringSchema, "STRING", "3"));
Console.WriteLine();

Console.WriteLine(schemaValidator(integerSchema, "INTEGER", "\"3\""));
Console.WriteLine(schemaValidator(stringSchema, "STRING", "\"3\""));
Console.WriteLine();

Console.WriteLine(schemaValidator(integerSchema, "INTEGER", "3.0"));
Console.WriteLine(schemaValidator(stringSchema, "STRING", "3.0"));
Console.WriteLine();

Console.WriteLine(schemaValidator(integerSchema, "INTEGER", "\"hello\""));
Console.WriteLine(schemaValidator(stringSchema, "STRING", "\"hello\""));

This snippet displays:

3 IS A VALID INTEGER
>>3<< is not a string value

>>"3"<< is not a numeric value
"3" IS A VALID STRING

3.0 does not conform to the XSD definition of 'int'
>>3.0<< is not a string value

>>"hello"<< is not a numeric value
"hello" IS A VALID STRING

Each of the following bullets explains a corresponding validation result above:

  • A JSON value without quotes is not a string value
  • A JSON value with quotes is not a numeric value
  • An integer should not have a decimal point
  • A quoted sequence of characters is a string

Validate instance data against specialized primitive schema

Several DTDL schema types are defined as JSON strings with additional constraints applied. For example, a DTDL date is a JSON string that follows the ISO 8601 date format, per RFC 3339.

As above, we use the identfier of the date schema element to retrieve it from the object model, which works because the DTDL model has a reference to this schema type.

var dateSchema = objectModel[new Dtmi("dtmi:dtdl:instance:Schema:date;2")];

The following code shows whether various JSON text strings are valid intances of DTDL date:

Console.WriteLine(schemaValidator(dateSchema, "DATE", "\"2017-05-29\""));
Console.WriteLine(schemaValidator(dateSchema, "DATE", "2017-05-29"));
Console.WriteLine(schemaValidator(dateSchema, "DATE", "\"17-05-29\""));
Console.WriteLine(schemaValidator(dateSchema, "DATE", "\"20170529\""));
Console.WriteLine(schemaValidator(dateSchema, "DATE", "\"2017-15-29\""));
Console.WriteLine(schemaValidator(dateSchema, "DATE", "\"2017-02-29\""));
Console.WriteLine(schemaValidator(dateSchema, "DATE", "\"2016-02-29\""));

This snippet displays:

"2017-05-29" IS A VALID DATE
>>2017-05-29<< is not valid JSON text
"17-05-29" does not conform to the RFC 3339 definition of 'date'
"20170529" does not conform to the RFC 3339 definition of 'date'
"2017-15-29" does not conform to the RFC 3339 definition of 'date'
"2017-02-29" does not conform to the RFC 3339 definition of 'date'
"2016-02-29" IS A VALID DATE

Each of the following bullets explains a corresponding validation result above:

  • The correct format for a date is YYYY-MM-DD
  • A date must be a string value, so it must be in quotes
  • Two-digit years are not acceptable
  • Hyphens are required
  • The month must be in the range 01 to 12
  • February 29 is not valid for non-leap years
  • February 29 is valid for leap years

Validate instance data against complex schema

The model above defines a Property named "installation" whose schema is a specific Object. To determine whether a JSON text is a valid value for an instance of the Property, We can call ValidateInstance() on the element that is the value of the "installation" Property's "schema" property. First, we retrieve the DTPropertyInfo element from the object model by name:

var anInterfaceId = new Dtmi("dtmi:example:anInterface;1");
var anInterface = (DTInterfaceInfo)objectModel[anInterfaceId];

string installationName = "installation";
var installation = (DTPropertyInfo)anInterface.Contents[installationName];

The following code shows whether various JSON text strings are valid intances of the "installation" Property's schema defined in the model:

Console.WriteLine(schemaValidator(installation.Schema, "INSTALLATION",
    "{ \"revision\": 3, \"installed\": \"2017-05-29\", \"note\": \"easy breezy\" }"));
Console.WriteLine(schemaValidator(installation.Schema, "INSTALLATION",
    "{ \"revision\": 3, \"installed\": \"2017-05-29\", \"note\": \"whoops\", \"foo\": \"bar\" }"));
Console.WriteLine(schemaValidator(installation.Schema, "INSTALLATION",
    "{ \"revision\": 3, \"installed\": \"2017-05-29\" }"));
Console.WriteLine(schemaValidator(installation.Schema, "INSTALLATION",
  "{ \"revision\": 3, \"installed\": \"17-05-29\" }"));
Console.WriteLine(schemaValidator(installation.Schema, "INSTALLATION", "{ }"));

This snippet displays:

{ "revision": 3, "installed": "2017-05-29", "note": "easy breezy" } IS A VALID INSTALLATION
"foo" does not match any name in schema
{ "revision": 3, "installed": "2017-05-29" } IS A VALID INSTALLATION
"17-05-29" does not conform to the RFC 3339 definition of 'date'
{ } IS A VALID INSTALLATION

Each of the following bullets explains a corresponding validation result above:

  • The object is valid if all fields are valid
  • No additional fields may be present
  • One or more fields may be missing
  • Any field that is present must have a valid value
  • An empty object is always valid

If there are multiple independent violations, ValidateInstance() returns a string for each one:

Console.WriteLine(schemaValidator(installation.Schema, "INSTALLATION",
    "{ \"revision\": 3, \"installed\": \"17-05-29\", \"note\": \"whoops\", \"foo\": \"bar\" }"));

This snippet displays:

"17-05-29" does not conform to the RFC 3339 definition of 'date' AND "foo" does not match any name in schema