Visual Software Services

SIF Application Verification Testing Services

Much has been said about "out of the box interoperability" and many who want to implement SIF have wondered "if two SIF agents are certified, then shouldn’t they automatically work together and why wouldn’t we always be happy with the results?"

What SIF Certification Tests

The SIF certification process tests the behavior of the SIF agent that is associated with an application, and does it thoroughly. It makes sure that the agent implements security models correctly, registers in a zone correctly, formats messages correctly, sends acknowledgements correctly and does exactly what it advertizes it does in its Conformance Statement.

For providers, it tests to make sure that the agent can respond to requests, even if those requests are unusual and when it generates events, that those messages are formed correctly. The certification test also tests to make sure that the agent can provide every object specified in the Conformance Statement.

For subscribers, the process makes sure that the agent can receive and process event messages and make requests and properly handle the responses. The certification test also checks to make sure that the agent can handle every object specified in the Conformance Statement.
These tests are numerous and exhaustive, but they only go as far as testing the agent’s functionality and not the application itself.

What SIF Certification Doesn't Test

There are a few things that the certification test does not test:

The SIF certification process is intended to be generic – to thoroughly test all the way through the application would require a custom designed test for each certification.

What is a SIF Application Verification Test?

A verification test is different from the SIF certification test in that it tests pairs of applications that are connected through a SIF infrastructure. (A verification test is not something performed (or officially endorsed) by the SIFA organization - it is something we have taken on as a service to the community.)

SIFApplicationVerification

The end result of the test is to verify that the subscribing application continues to work properly when it receives its input through its SIF agent instead of its normal user interface.

The Verification Test Ledger

If the subscribing application successfully works and all functionality works correctly, the results of the tests will be made available on a web site for future reference by the two companies. This will include (if acceptable to the two companies):

The Verification Test Process

The testing process will typically involve one or more agent providers connecting to the ZIS testing facility located at our office. The connections can be HTTP and/or HTTPS – if HTTPS, we have a certificate server and can issue certificates for use in testing.

The Test Lab

A test lab has been up that contains a four server high-availability ZIServer ZIS cluster and two agent machines for testing, although the agents should mostly be connected from user’s sites.

The Tests

The tests would be determined by the functionality of the subscribing agent. If the two agents passed information back and forth, then each agent would construct tests to validate the information it receives from the other. In either case, the receiving party would be the controller of the test.

The determination of the success of the test is if the application behaved correctly given the information passed through the SIF interface.

The Test Results

If the test ends up in a successful test, the results are published on a public site, including,

  1. basic information about the test including which operations were being performed, the versions of the software being used and any environmental factors that may have influenced the test
  2. detailed information about the test, including the audit trail from the ZIS that was captured during the test session

Testing like this would give those who want to implement this application pair confidence that the two applications work together, given that the data that is needed by the subscriber has been entered into the provider.

These tests can be performed independently in our labs or we can set them up on-premise for larger organizations, as we have for a large US state Eepartment of Education.

Integration Services

We at Visual Software are committed to making sure that your SIF implementation is successful and that you end up with a reliable result that contains trustworthy data.

To accomplish this result, we've developed a methodology for implementing SIF for connecting applications horizontally (within a school, for example). This is very different than what is typically done in many US installations, but we feel that by taking early deliberate steps to ensure that privacy and data quality requirements have been met, then many difficult problems that may occur later in the process can be avoided entirely. The process is as follows:Visual Software SIF Implementation

  1. Set up a test environment. If funds are tight, we would suggest using virtual servers and test editions of software – the test editions can usually be obtained at a fraction of the full cost and the environment for virtual servers (at least for Microsoft products) can be downloaded free of charge from their web site and can be run from a desktop or laptop computer. If you’re running in a large organization that has hosted, shared facilities, you can ask to have test zones created on the shared servers but you should still set up test copies of your applications.

  2. Install the Zone Integration Server software, Envoy and Veracity.

  3. Create a “Raw” zone. Since very little data would pass validation in initial testing, we recommend turning validation off at this point – this will be the zone that connects the SIS (MIS) and Envoy. Other agents will never connect to this zone because this data will not be privacy protected nor will its data be passed through any validation testing.

  4. Either the MIS (SIS) or Mimic becomes provider in this zone – this “Raw” zone approach allows connection of a feeding agent by any SIF-enabled application – whether it is a supplier-provided SIF agent or Mimic generating events on behalf of the MIS (SIS). Then import MIS data into Envoy and let it analyze the MIS (SIS) data.

  5. Envoy will separate MIS (SIS) data into “good data” and “bad data”, depending if the data would be able to pass current SIF validation rules. “Good data” will be marked as “OK to publish” and “bad data” will be presented to school users through a Veracity-like web-based user interface. In this way, Envoy is behaving like a “quality gateway”, only letting through objects that meet basic quality tests.

  6. Have school users clean up data in MIS (SIS) – as school users clean up data in the MIS (SIS), the SIF agent for the MIS (SIS) will automatically send through SIF “Change” events and the data will move from the “bad data” to the “good data” categories and will no longer show up in the school user’s user interface (no further user action will be required other than simply correcting the mistake).

  7. “Clean” zones for other applications to subscribe to – two or three zones will be created, depending on if Envoy is implemented at the Local Authority or Regional level. Once the data for the school is sufficiently clean, the school’s Envoy connection can now register as a provider in its corresponding “Clean” zones.

  8. Before it registers as the provider to other applications, the School Information Risk Officer reviews the privacy settings for the school in Envoy to make sure that all  sensitive information is properly protected. Note that this is all done before any other applications are connected, even within the same school.

  9. After Envoy is connected as the provider, the School, LA and Regional copies of Veracity will be loaded with initial copies of the school’s data and initial Veracity rules for Type 2-5 errors will be run. For LAs and Regions, these will include data privacy checks to make sure that no  sensitive information has “leaked” into one of the higher level zones.

  10. Connect other applications, convert data into them and check to make sure all data has arrived successfully.

Compare this to a typical US installation:

  1. Install the ZIS.

  2. Install MIS (SIS) agent and subscribing agents, most likely in a test environment first.

  3. Convert the data into subscribing applications and use a lack of information in the subscribing application to detect problems in the quality of the data in the MIS (SIS).

Although the second approach sounds much simpler on the surface, it has a few problems:

For more information on this methodology, see Reliability and Privacy.  To find out more about integration services, call us at the number at the bottom of the page or ask someone to contact you.

Consultancy Quick Start

Visual Software is offering a limited program for consulting companies who are interested in entering the SIF integration marketplace.  The goal of this program is to enable integration partners to begin a SIF practice with reduced risk, accompanied by a pre-trained staff of experts.

In addition to accompanying you on your critical first implementations, we will provide you with the software tools you need to be successful and provide you the backup support you need to thrive.

Quick Start Program Overview

Because this program will require a great deal of support from us to make it work well, we will only be offering this to a limited number of partners and then discontinuing the program. If you would like more information on the program, please send us a request at Contact Us and put "Quick Start" in the Question section.