Veracity Data Integrity Manager

The SIF specification enables applications to exchange data, such that once it is made available in one place, it can be put to good use everywhere. In the process, valuable information from all sorts of applications crosses the SIF "zone." We recognize this flow as an asset in and of itself, and tap into it with Veracity.

The Repository and Data Quality Management

Veracity is our unique repository of SIF objects. It leverages SIF's event and request/response facilities to keep an up to date snapshot of the current status of all objects in the SIF zone. Once in Veracity, it doesn't matter that the data came from a transportation system or a food services system, it is available from one place under one data schema.

One of the biggest challenges to overcome before you are at the place of making decisions based on your district's data is that of data integrity. Can you trust your data? Is it accurate and dependable?

One of the fundamental goals in data integrity management is to move the correction of data errors back to the source. In many districts, when census data is being prepared, IT staff collects it from the various systems in the school, matches up records, cleans it, formats the result and then submits it. This cycle is continually repeated and many times the same error is corrected over and over.

The Veracity approach is the following:

By moving the correction of the problem back to the source of the error:

See if you can rely on your data.  First, you need to know how close the quality of your data is to the standards defined by your organization.

Veracity’s rule set articulates your district’s data quality rules, applies them against data collected from the connected systems and shows you where your district’s data falls short.

These rules can be checked on a periodic basis or in real time as data is published.Veracity Home Page

This chart shows what types of errors you have and how they are distributed.  If you click on the chart and hover over one of the slices, it tells you how many errors of that type there are.  Clicking on the image will cause a full size image to open in a new window.

See How You Are Progressing

Veracity requires that errors be corrected at their source and does not allow changes to be made in the repository.

Instead, it provides a set of tools that help people track errors and correct them at the source, so that all other systems connected through the SIF interface can also benefit when they are corrected.

The graph to the right gives users a sense of how the district’s data is improving over time.  Clicking on the image will cause a full size image to open in a new window.

The different colors represent errors, warnings and other items of interest.  If you click on this graph, it scrolls from side to side and, like the other graph, if you click and hover over a bar, it shows you the specific number of errors in a specific category.

In this example, the data was checked several times. As the data was checked, errors in the its source were being corrected until near the last sample, when some new rules were introduced. These new rules caught additional errors in the source data.

Seeing Error Details

Seeing the details is also important.  This screen allows you to view and drill down into the error details (this screen shows a summary of each rule that has been violated by data in your organization, and how many records violate that rule):

Errors - Overview
(click on image to see full-size)

…clicking on a “View Records” link allows you to see student-level detail for a specific category…

Veracity - Error pick list
(click on image to see full-size)

…clicking on “Student Details” link allows you to drill down to the student detail level…

Veracity - Error detail
(click on image to see full-size)

This student’s record violated two rules, the birth date/school reasonableness and the school district age limit test rule, so at the bottom of the screen, both rules are summarized.  The specific wording for each of these rules can be modified by the Veracity administrator through this web interface, wording them to match local policy definitions.  NOTE: Student information used in these screen captures are for fictional student records randomly created from lists of popular last and first names. Any resemblance to real persons, living or dead, is purely coincidental.

At the bottom of this screen (not shown) is a summary of other rules that have also been violated by this record (there could be several problems with this record).

Defining Rules

Veracity helps you continuously monitor the quality of your organization's data by giving you an ability to simply define data integrity rules that will be run against the consolidated data in Veracity and point out to your staff where data needs to be corrected.  It provides a simple configuration form to define rules such as "make sure every elementary school child who lives more than a half mile from school is on a bus route" or "make sure that every child has been assigned to at least one but not more than two schools".

Sets of rules can be established for different purposes.  For example, you may want to run general student information checks on a daily basis while other, more detailed checks may only be appropriate once the "beginning of the school year crunch" has passed.

Rules are defined through a web interface (assuming that the user has administrative rights) using a process that typically takes only a few minutes per rule.

Using the Combined Data

At Visual Software we understand that a school's business is not the numbers and strings passed around from application to application, but what those data elements represent.  You have data that describes how a child is doing, from discipline to grades to attendance.  The problem is that there is just too much data for any human to keep track of and use effectively at the student level.SQL Server 2008 Report Builder 2.0

You may already have systems that give detailed reports to principals, guidance counselors and other staff members.  But what happens when one line from each of three different reports put together signal a problem when each of the lines independently don't raise any flags? 

For example, a particular student's GPA may be dropping at the same time as attendance is worsening and discipline incidents are increasing.  Any one of these factors viewed independently might not get much attention, but put together they might signal a child "dropping off the edge." If this information was maintained by different systems or if the information was not available in a consolidated report, that important indicator would be lost.

Veracity collects and combines information from whichever applications are publishing in a zone. Records from these different systems for common students, teachers, parents, and classes (groups) are automatically matched, allowing users to access its underlying database to create consolidated reports using tools such as SQL Reporting and Analysis Services.

For more information on SQL Server Reporting and Analysis Services, click here.