Communications & Security

Email This Author Comment On Article About The Author More Articles By This Author

Each segment of the electric network uses different communications methods and application models. It is not feasible to specify a uniform communications protocol or semantic for all aspects of utility operations because different segments have different requirements. However, for seamless operation of a smart grid, these differences must not impede the flow of data and control. Therefore, the various protocols and semantics need to interoperate among themselves and with others, possibly via gateways that translate between protocols. This paper defines interoperability and explains how to achieve it within the smart grid.

A number of industries in the past have attempted to achieve interoperability among providers of software systems that facilitate the exchange of information within an industry. Some industries have been very successful, and their hard won efforts provide considerable benefit to their users. However, many others have either not succeeded or have made slow progress in their efforts to achieve interoperability. This paper explores the key points that we have found are required to conceive and execute a successful conformance/interoperability testing methodology, and some of the problem areas to avoid, if one wishes to achieve interoperability. It will provide the reader with the high-level principles and guidelines for creating a business plan necessary to build a successful interoperability testing program, regardless of the specific standards or profiles under test. These principles are based on our finding conformance/interoperability programs in a number of industries over the past 10 years.

Every industry has different adoption drivers and subtle adjustments must be made to generic testing regimes and methodologies to reflect an industry's unique characteristics. This position paper will not discuss these adjustments.

W

e will look at six areas that encompass the key success factors for any conformance/interoperability program and discuss just a few of the more important aspects of each ignoring the excruciating detail involved in a successful program.

Stakeholder Identification

One of the most important aspects is to identify the stakeholders. Now, while it seems to be an obvious and straightforward endeavor, two of these stakeholders are often overlooked or not emphasized enough when the testing program is developed. The relevant stakeholders are normally:

• Standard authors
• Standard implementers (e.g., vendors)
• End-users
• Test program

All of these must have sufficiently successful outcomes from the test program in order for conformance and interoperability to take hold within an industry. They all must be involved in the conformance/interoperability process, some to lesser and some to greater degrees, so that conformance and interoperability become a reality. The two stakeholders which are often overlooked are the end-user and the test program itself. For the Smart Grid interoperability case, the end-users would be the electric utilities, buildings, homes, generation, transmission, etc. These are the users of the implementations which have been certified by the testing program.

They expect the implementation, which has been certified interoperable, to work "out of the box" without much additional effort. If this is not the case, and they end up spending resources to debug the implementations, in their view, the test program has failed. This brings into focus the other stakeholder that is often forgotten in the effort to achieve conformance and interoperability -- the test program itself. If the test program is certifying implementations that require additional debugging by the end-user, the industry will soon view the test program as meaningless. The vendors will stop participating in the testing program since the end-users will show no preference for implementations which have been certified by, in their view, a flawed test program. This compromises the effectiveness of the test program and results in non-interoperable implementations within the industry.

Interoperability as a Lifecycle Artifact

The conformance and interoperability program must be a staple for the standards implementation's full lifecycle and not just a series of tests that are executed once. It must anticipate the coordination of versioning of both the standard and the implementations. It must handle the introduction of new implementations and the exiting of previously tested implementations, and it must support several versions of a single implementation within the target deployment environment over time. This normally requires anticipating issues and possible issues resulting from experience in other testing programs.

Pervasive Industry Support

The interoperability testing program must be industry-wide and not be limited to partial acceptance and adoption within the industry. This aspect is fairly obvious, but if there is more than one test program within an industry covering the same standard for achieving conformance and interoperability, the competing programs may at times produce conflicting sets of conformant and interoperable implementations. This depends somewhat on the effective coordination between the programs. However, it is much more difficult to achieve and coordinate interoperability among products from separate programs, so it is very important to have a single conformance/interoperability test program covering the industry (or multiple industries) for a single standard. The failure to do so has very often caused problems for all stakeholders in other industries.

Marketplace Development

The testing program must help develop a marketplace for conformant and interoperable implementations, especially off-the-shelf vendor products. This is one of the most difficult of the six aspects to produce success. The testing of an implementation carries a cost to the implementer in both testing fees and manpower resources in order to participate in the test and to perform any modifications to the product that results of the test might require. Thus, the cost to produce a certified conformant and interoperable product will be more than the non-certified product that has not participated in the testing program. For the implementer, there is little reason to participate in conformance and interoperability testing unless the industry mandates it or the end-users (as a collective group) mandate it by only buying certified implementations. There must be marketing reasons in order for the implementers to incur the additional cost that comes with testing. The cost/benefit ratio must be sufficiently low to achieve success in this area, and this decision must happen early on at the executive level across the industry. Our most successful programs have developed this component, the least successful have not.

Conformance is not Interoperability

The program must clearly convey the different meanings between conformance of an implementation to a standard, and interoperability between two or more implementations of the standard. Confusion regarding this aspect is currently a major hindrance to the success of conformance and interoperability programs. This misunderstanding of the differences between conformance and interoperability in the marketplace, testing, and at times, the program authors themselves, results in confusion as to what is meant by successfully passing the testing program. Conformance means that an implementation adheres to the dictates of the standard. (I will not discuss profiling of standards at this time.)

While, theoretically, all programs that completely adhere to a standard should be interoperable, in practice they often are not. Interoperability means that implementations adhere to the dictates of the standard and intercommunicate appropriately with other implementations that adhere to that same standard. (I will forgo the discussion of gateway standards at this time.) Interoperability adds one more requirement over and above conformance.

The problem is that many testing programs test only for conformance and then unceremoniously presume and declare it interoperable. Stakeholders in the marketplace believe they are receiving interoperable implementations because they have been told so, but they are getting only conformant products. Conformant implementations may not be interoperable among themselves. This is especially the case in more complex software and hardware systems. This leads to the first aspect discussed above in which "certified" implementations now require debugging when they are installed by the end-user, thus damaging the credibility of the test program. Once the compromising of the testing program's credibility starts, it can take a couple of years to correct the perception by the marketplace of end-users. This is why the test program must be thought of as a stakeholder early on.

Interoperability Verified not Presumed

The program must verify, not just assume, interoperability among the various implementations of a standard. There are many different types of standards. Some are device oriented. Some are business-to-business. Some are written from the ground up detailing all the software and firmware with dependencies on other standards to achieve their purpose. Other standards are focused on communication protocols while others are focused on the semantic meaning of the data. Conformance testing any of these standards may achieve different levels of 'near' or 'actual' interoperability. Depending on a number of factors, including the standard, the testing regime, the software/firmware under test, and others, conformance testing may actually produce interoperable implementations. This is good in that no additional testing steps are required to achieve interoperability. However, there remains a problem. It is seldom actually known that a conformance test has produced interoperable implementations unless verification is performed with an additional test step to prove that the implementations are indeed interoperable. There are only two points in the timeline as a standard evolves from formation to implementation where implementations can be verified as actually interoperable:

1. The implementations may verify interoperability in concert with conformance testing; or
2. When the end-user is attempting to deploy the implementation in the field.

The first case represents the testing program and the stamp of approval of 'certified' by the program and demonstrates that products are both conformant and interoperable. In the second case, the conformant and presumed interoperable implementations are released to the marketplace where the end-user is expected to validate interoperability and correct any shortcomings in the testing program. It is well known from studies over the past 20 years that errors found in software products after field deployment may cost as much as 40 times the amount to correct than if those errors are found before the implementation is released to the marketplace. This additional cost does not include the original cost, frustrations and loss of good will by the end-users.

Not verifying that conformant implementations are interoperable when they are given a 'certified' grade in a conformance and interoperability testing program often cause the program to become irrelevant as we have seen in other industries. When this happens, interoperability often stalls for that standard in the industry -- sometimes for years.

Summary

Success of a conformance and interoperability program is about methodologies, market positioning and securing success for all the stakeholders. The program must be focused on supporting the implementations in the field for not only the product lifecycle, but also the lifecycle of the standard. The program must clearly identify what it is offering to the all the stakeholders as it identifies certified implementations. Are the products verified conformant or are the products verified conformant and interoperable? The program designers must anticipate its growth and demise as conformance and interoperability become institutionalized in the implementations over their lifetimes. All of these issues should be anticipated for a successful testing program irrespective of the standard. Not doing so may greatly reduce the introduction of conformant and interoperable implementations of the standard into the industry -- stalling interoperability.