SIF Basics

What's the Problem?

In any educational organizational organization, many computer applications are used to support the process of educating students, in one way or another. The majority of these applications know about who attends the school, who teaches there, and who the teachers are. Some of them know the students' schedules while others know details such as the grades they received or detailed health-related information such as immunizations or records of injuries. The point is that much of this information is common to many of them.

So, what is the "best" approach to managing the information that is shared by these applications?

Most of the time, what ends up being adopted is either a "best of breed" or a mixture of the two: an organization starts off wanting to get an integrated system, but then realizes that some other systems are needed as well.

Regardless of what's in place, these organizations will have multiple systems that will, by necessity, contain the same data. The big question is: how does that data get into those systems and who keeps it up to date?

How can this problem be managed?

Before we introduce SIF, we'll speak to three other alternatives that have been used in the past and are being used to address this issue; one by default (manual data entry), another because it was a simple technology that was available (exports and imports), and a third that is being experimented with in other industries (enterprise service buses). These are only being presented here to give some perspective.

Manual data entry

Person at deskThe most obvious method (and what gets done by default) is to enter data manually into each system using the application's own user interface. Besides being the most obvious, this is also the most expensive and is the most error prone.

We had a customer district once follow some sample stacks of paperwork around their district to see how much time was spent entering data from those pieces of paper into various systems and how much of that data entry was redundant. With a student population of about 7,000 students, they estimated that they spent about two staff positions worth of time in a given year simply doing the redundant part of the data entry.

Exports and Imports

Import/ExportThis method of synchronizing data works by having the provider application of the data create an "export" file on a regular basis, store it in a pre-determined location for each type of data (student teacher, parent, school, enrollment, etc.). Then, each of the "receiving" applications look for files in that location, open them and read the information. If they see any changes in the files, they refresh their databases with the new information.

A setup like this can be set up by an IT department if all the applications have appropriate interfaces.  This assumes that:

Many organizations use this, but it is costly and error-prone. Costly because whether the IT staff is managing the CSV import/export process or the software suppliers are, the end user ends up paying the cost.  There are no standards for the layouts of these files and most application suppliers typically spend a considerable effort maintaining a library of CSV-interfaces.

These interfaces are error-prone because the CSV format doesn't have the capability to recover from errors like other transmission methods do. Typically, if there is an error in a field, the entire record is rejected - or perhaps even the remainder of the file.

Enterprise Service Bus

Enterprise Service BusThere appear to be many definitions of this term (we will not include the airport car rental company shuttles in this discussion), but most of them center around the offerings of particular companies, such as Microsoft, Oracle or IBM. They allow connections from different operating systems types, they support web services interfaces and use XML and SOAP to transfer data.

Because of the way they are structured, they have a better error handling capacity than the older CSV import and export and because of its design, the transfer of information will be right away, instead of in batches. This means when a student enrolls, his or her information is immediately sent to all connected systems and is available in a few seconds, not in the next day or two.

The problem with ESB is that it is a technology - it defines the transportation mechanism through which data can pass. It is a very good and well thought-out one, but it doesn't do anything for making life simpler or more cost-effective for a school organization. Those who develop around it will still need to re-do much of the work that all of the companies participating in SIF have done for the last 10 years. This is why these projects tend to be enormously expensive.

Then came SIF...

The biggest difference with SIF and these other ways of solving this problem is that it is an industry standard. It wasn't imposed by a government or invented by a single company, but was created by industry experts who amazingly agreed on a number of key concepts, including:

This didn't happen overnight; this has been an ongoing process since the late 1990's and has involved hundreds of companies and thousands of school organizations (people who use and support these applications every day). It has grown to support three regional specifications in the US, UK and Australia and is currently being used in tens of thousands of schools worldwide.

SIF is standards-based; it is fast and it is secure. It can economically meet the needs of a very small installation, yet scale to meet the needs of a large state with millions of students. 

How Does SIF Work?

In a SIF environment, applications share data through web connections that send and receive XML language messages in a standardized format. All of the applications communicating connect to a common hub (called a Zone Integration Server (ZIS)) that is responsible for making sure that all messages are delivered properly. So, at the highest level, the collection of systems looks like the picture to the right.

The Zone Integration Server is software that runs either at the school organization or at a hosted location.  

For all of these applications to be able to communicate properly, they must "use a common language" as follows:

The Zone Integration Server (ZIS)

The ZIS is a piece of software that runs as a web site that routes messages between applications. It does not hold permanent copies of school information; the only time it keeps much of anything is if one of the receiving applications is offline or slow - it will temporarily hold messages for it until the receiving application drains its message queue. (Another exception is with our ZIServer product - we store a complete audit trail of all messages that pass through the ZIS - these only maintained for auditing purposes)ZIS and Agents

A ZIS can either be hosted and shared between many organizations or can be installed locally at a school or school district.

Besides routing messages, it is also responsible for maintaining the infrastructure's security.

Visual Software's ZIS is ZIServer™. You can find more information about it here.

Applications and Agents

When SIF was first envisioned, it was assumed that most applications would be adapted to work with SIF and that they wouldn't be built from the ground-up knowing how to directly communicate with a Zone Integration Server.  These adapters are called "SIF Agents".

 When SIF was in its infancy, several companies set out to address the need to build SIF Agents. We at Visual Software went in one direction; everyone else went in the other.  AgentThe others began with the assumption that these agents were going to be different enough that each of them should be built mostly from the ground-up (with a little help from an ADK (a programmer's library).

We at Visual Software, however, had spent the previous eight years refining techniques and building technology allowing "new services to attached to existing technologies without disrupting them."  So, instead of starting fresh, we used these principles and used this code base to create the first version of our configurable SIF agent ZIAgent, that provides SIF services to existing applications without requiring programming and without requiring modifications to the target application.

In years since, we've continued to refine this model, making it more and more efficient, flexible, reliable and scalable. Moreover, we've made the agent do "the difficult stuff" by default; the things many leave out; the things that separate the good from the excellent agents. To learn more about our configurable agent, see ZIAgent.

How Software Suppliers See SIF

Ipseity is SIF-CertifiedAlmost universally, software suppliers would prefer to spend as much effort as possible focused on their own application, but instead find themselves getting sidetracked with multiple interfaces to other applications. They need to stay aware of when the other applications change, then change their interfaces to match the changing designs of the other applications. This is an expensive process that takes away from their ability to address issues within their own product.

If it isn't a waste of their time, then they're charging a significant price for these interfaces, because they take significant work to maintain.

What SIF allows them to do is to create a single interface that can be used to connect to many other applications. Unlike these Import/Export interfaces (which most of the existing ones are), the SIF interface can handle errors and properly recover from them, can protect private information at levels meet or that exceed government standards and can even simplify the entire process for their customers (if done properly).

How School Organizations See SIF

Schools need to solve a number of data-related problems:

SIF is a good way to solve those problems because it has good error handling characteristics, it scales well and data is passed between applications as soon as the data event happens.

 We have found in many installations that once the system has been set up, it doesn't need much care beyond normal backups. 

Because it is a standard, a school organization can look to an application supplier to provide such an interface without that request being considered unusual, nor seen by the vendor as something useful for only one customer.

Is it Perfect? (time to be honest...)

Well, like anything human-made, SIF has its issues. From our experience installing and supporting many SIF agents over the past ten years, we have found that if all participants use the specification the way it was intended to be used, then things work remarkably well and the users and suppliers are very pleased with the results.

Issues arise, however, whenever one party or another begins to compromise the specification. The certification process catches many of these errors, but it isn't perfect either.

A good rule for evaluation is if you ever hear anyone saying that a particular agent only works when paired with another agent or when paired with a specific Zone Integration Server, then even if it is SIF certified, don't even consider it to be a SIF agent. This means that they have compromised the SIF specification and found a way to fool the certification test harness.

Do you want to know more?

Here are some other links that might be of interest: