|
|||||||||||||
A Meter Data Management System (MDMS) is a key software component of any utility's smart grid advanced metering infrastructure. It has many moving parts that need to work together and communicate well in order to operate effectively and provide error-free and efficient performance. An MDMS typically requires high levels of manual intervention for the installation, configuration, ongoing maintenance (service packs, hotfixes, patches) and monitoring. Some of these activities, such as installation and configuration typically require manual attention and are not worth automating. This is because these activities are typically "once only" activities that are not repeatable. However, other activities such as daily operational interactions, routine jobs and monitoring can and should be automated, in order to eliminate the chance of human error and to achieve consistency in operational results.
Monitoring Meter Read Collection
Smooth and uninterrupted meter read collection is a vital first step in the process. A manual approach to monitoring the read collection would require the MDMS operators to visually track the live graph of the read collection data for the current window of interrogation. Although this approach has its own advantages, it can lead to fatigue and human error, especially during the overnight hours. A better approach would be to automate this monitoring, and only alert us in case of anomalies. The database tables in the MDMS that store the incoming meter read data (interval data, register read data, etc.) can be queried on a predetermined scheduled (such as every 15 minutes or 30 minutes), and if the data doesn't meet certain thresholds, the operators and/or system administrators can be notified using an automated alerting or paging system. Corrective actions might include checking the health of the AMI (Advanced Metering Infrastructure) head-end system for any interruption in read collection, checking the communication channel between the head-end system and the MDMS, the health of the database system, etc.
Monitoring Log Files
Most MDMS components write errors and exceptions to log files in a standardized format. Depending on the logging level specified, some of these log files can grow large and numerous, quickly getting out of control. For example, an MDMS component may run of out of allocated memory space, or may be unable to make a connection to another component or server. Rather than manually scanning the log files for specific processing errors, a better approach would be to set up an automated search of the log files for specific strings of text. In a UNIX environment, a "cron" schedule can be setup to run a shell script or a PERL or Python script that can scan the last lines written to the log files (i.e. only those lines that were written to the log files since the previous run of the script). The script could then email the operators and/or system administrators, notifying them of the event, log file name, the MDMS component name, and the date and time of the log entry.
Automating Data Synchronization
The data in the AMI database of the MDMS needs periodic synchronization with the master customer data in the CIS (Customer Information System), which is usually a legacy, home grown system, or could be running on a vendor supplied system such as SAP for Utilities. This master customer data needs to be pushed to the MDMS periodically. It could be a "Full Sync" or an "Incremental Sync". The MDMS usually provides a trigger application to kick of the synchronization. But it requires the operator or system administrator to click the trigger button to initiate the sync, after the data files from the CIS become available. This process can and should be automated. For example, on a UNIX server, a "cron" schedule can be setup to run a shell script or a PERL or Python script to kick off the sync trigger, beginning the process of data synchronization, on a configurable schedule, after the minimum (configurable) number of data files become available from the CIS.
Monitoring the MDMS web services
The MDMS typically includes several web service components to support various types of functionality, such as Meter Data querying, Device Control, etc. It is possible for some of these components to become unavailable for various reasons, such as network issues, web server disk space and/or memory issues, firewall restrictions, etc. Rather than find out about this the hard way and scramble to resolve the issue, a better approach would be to proactively monitor these MDMS web services by "pinging" them from an automated script running on a periodic schedule. If the ping fails, the script could immediately notify the operators and / or system administrators, alerting them of the issue.
Automating the MDMS shutdowns and Restarts
Most MDMS vendors provide periodic upgrades to the software, in the form of service packs, patches and hotfixes. Some of these require a full shutdown and restart of all the servers involved. Shutting down a huge multi-server MDMS installation can be a non-trivial task, and requires coordination between various activities and a pre-determined order in which to shut down the system. This is a prime opportunity for automation. A centralized master automation program (such as a shell script, PERL or Python script) could orchestrate a coordinated shutdown of the various servers involved, resulting in a repeatable, time-tested process that eliminates the element of surprise. Upon successful installation of the service pack or patch or hotfix, the system could then be started back up using a similar automation script.
