 |
| Michael Lunsford, CEO Spectrum PACS Services |
Michael Lunsford – has been a practicing IT-based PACS administrator in a multi-hospital system with over 15 years of combined IT and PACS Administration, PACS network design and Nuclear Med, MR and CT field engineering experience with companies such as Siemens, Picker and Park Medical. He developed software, called DICOMetrix, which is a complete software platform for analyzing and monitoring radiology, cardiology, PET /CT, teleradiology any DICOM based images delivery system made by any manufacturer. He founded the DICOMetrix company in 2005 and remains its CEO. In 2013, he formed a consulting services company called Spectrum PACS Services that offers the expertise of his and other former PACS vendor support engineers and other IT gurus on his team to provide affordable expert-level help in all aspects of PACS systems performance.
Q. What do you see as a couple of key issues facing PACS administrators for 2015?
A. You probably have to say it depends on the type of PACS administrator you are talking about. There are three kinds, one is the application level administrator who is probably more centered on the software itself in support of users. Then you've got your systems admins, (where I was centered on), who are more concerned with supporting the technical envelopes of the vendors’ application software, it’s supporting hardware and storage systems. Then you have an hybrid of the two, which I did for many years.
There are going to be problems with any complex system in general, and they are always going to get more complex simply because the addition of more flexibilities through enhanced software and multi-platform user integrations with CIS and RIS systems that are always ongoing along with integration with more media and storage.
In my career with Sisters of Charity Health of Leavenworth in Denver, the servers and the storage systems were changed four or five times and each of these were major events with significant downtimes. These and normal everyday operations of a PACS system in all its facets can and always do cause application usage and image transfer slowdowns for users, which is not only very frustrating for all departments in a hospital but it also delays patient care. I met my own ‘blindness’ and the frustrations it caused and I decided to attack them by inventing some software to ‘see’ where and at what point things were going slow. You need to be able to identify in real time one location and then all locations in a multi-hospital system to be able to resolve the cause of the slow downs and the downtimes. Those could occur at the workstations, the servers, the storage nodes and outgoing / incoming image data from our business associates.
After a short time, it became apparent that we were still being ‘reactive.’ I then decided we could turn this into a proactive system to alert us via email or pager when anything in the PACS system dropped down below what users defined as ‘normal.’ So our development team added the web-based enterprise system.
Anywhere along the line of PACS systems and workstations, performance in my world was always a major concern. The DICOMetrix software was invaluable for both before and after system and software upgrades. Once you bring your upgrades or updates online in the early morning hours, you need some way to see if everything is still running as well as it did before. On occasion you get noteworthy performance improvements, neither of which without the software could you have objectively or scientifically gauged in anyway. Over the years, we have found that some people think ‘flying blind’ and ‘ignorance is bliss’ and not knowing truly how fast their systems are running at any point in time provides a certain level of comfort, until the angry radiologists call them up wondering what they just did to make it all go so slow.
PACS Systems are delicate and they can be a very complex thing to adjust for optimal performance. Using images that take up significant amounts of storage space and network bandwidth, PACS systems are also a bit more vulnerable to changes in power fluctuations, even on guarded and redundant power systems, and most especially to unknown and untested security or operational system policy pushes that seem to always happen even with the best in change control procedures. You have to be able to know how to get to the slowness. You have to be able to see what the complaining user or department is seeing and follow the trail back to the source of the problem and doing so visually is the most efficient method. That is predominantly in a nutshell what DICOMetrix does, so you can visually see in easy terms where and when things start to slow down and where you need to go to restore full system performance.
Q. What are the most common problems that cause system slowdowns?
A. Other than application and system software upgrades that are done with various levels of testing. I think that the most common problems occur with storage systems. Those are the most complex systems in the PACS environment and nowadays are commonly shared with many other mission critical applications. I think they are the main Achilles’ heel to the entire hospital network system, especially for the PACS because it uses more storage and more bandwidth than any other application and always will.
I think keeping a good eye on storage is probably the best overall recommendation. You have to have good visibility in how your system is functioning and having good relationships with your storage people is critical. In the last 10 years, storage technology has had many major changes. Now there are technical specialists out there that just deal with storage systems. I know that other applications become sensitive to slowing performance but nothing like PACS. For example, if any part of a PACS system slows down delivering studies to radiologists and ED staff, can delay patient care, and delays can be life threatening, so PACS needs to be fast. With a stroke victim waiting for the CT angiogram to be read before he or she can get the proper dose of TPS can be a life threatening situation.
Q. Is the movement to integrated systems causing more problems?
A. Yes. The more complex your system becomes through interfacing to other systems and their applications - the more difficult it will be to maintain and restore optimal performance. Or said another way, as the complexities increase there will also be more causes of slowdowns. More importantly, now the concept of shared storage is adding to the complexity.
Q. What are some other issues likely to be added to PACS administrators’ duties?
A. A significant one that is showing up on the radar for hospital PACS is that as relationships blossom with external physicians and imaging centers, an entity has now been created in which images from PACS are shared with everybody, and that added layer of complexity increases the load on the system naturally, but it also increases your own workload significantly. In the past, image studies were printed on film and sent by courier to your outside facilities. Now outside facilities are literally connected to your own networks and using your system resources directly. Training on PACS applications use for inside users has now expanded to all your outside users. That adds another layer of complexity. Years ago what was a one-person job just to handle that layer, now that layer is added onto your (PACS administrator’s) existing job duties.
Q. Specifically, how does the DICOMetrix software work?
A. I think I identified earlier you need to be able to identify, no matter who or what it is that is making your PACS system go slow. You need to be able to define what is slow. You need to be able to benchmark your current system and software so you know how things behave after you do upgrades. It is the PACS Systems admin person who is charged with finding out why the system is doing all of these things.
As all admins already know, when a PACS system slowdown occurs it could be reported first by humans and second by the system itself. The human users will always let you know somehow. The DICOMetrix software can not only provide everyone using it visual, real-time feedback but equally importantly, it can proactively respond and send out text messages or emails to PACS support staff about system slow-downs and outages so that when something slows down you can quickly jump on it and find out why and where. DICOMetrix does this by using a small extremely low impact software client that sniffs each node in the network where the user installs it for specific packets of DICOM image data that are the actual images.
DICOMetrix holds patents for the formula for doing that accurately. After detection, it basically uses some easy-to-see and understand visual-based widgets in a small footprint GUI to display it to the user in a format that looks like an automobile dashboard, something everyone is familiar with and understands. To keep it simple like your car’s speedometer, we measure and display two important parameters. We use kilobytes per second (kB/S) and something non-IT based people understand better, we also display and measure images per second (IPS).
We found that more people affected by PACS slowdowns better understand images per unit of time. So in a nutshell that is why and how the software was created. By displaying IT vernacular of kB/S and also IPS it effectively bridges communication gaps between IT and the users departments so you now have a common communication vernacular provided by DICOMetrix that allows you to say, ‘I have a problem with receiving my studies in my remote location and using the common language and graphical GUI you can now very easily tell others … here's why.
Once you have every major data node in your system monitored you can easily see where the data slows. Kind of like you would watch an artery in your heart, if you could. Similarly the PACS system is an arterial system feeding people information, that is mission critical patient images. When you are trying to restore a PACS system to full speed operation or bring it up from a complete downtime event, it is important for everyone that will be working on the PACS problem to understand and be able to visualize where, when, and what (is happening), in order to solve problems as fast as possible. To be able to inform your users and department administration what the problem is and in such a way that they understand clearly what is typically a complex technical problem, you can use DICOMetrix to take snapshots of the screens you have and send them to administration at all levels should you want or need to. Using this interface, now they get the best understanding of the situation along with the reassurance of knowing you are on top of the issue.
Q. Do you also provide consulting services?
A. Yes, I started another company called SPS, Spectrum PACS Services. In order to get past the obvious hurdles of what can be a long and drawn-out ‘capital-based’ purchase for facilities to obtain the benefit of the DICOMetrix software system, and turn it into an operationalized one, I found I needed to create a separate company based on a expert level service perspective. SPS was created to provide integrated services that add our own years of systems maintenance and performance analysis expertise to the DICOMetrix software product. In addition to bringing in the DICOMetrix software and installing it onto the customer’s systems, we also integrate our expert level experience as consultants. Through customer supported and authorized VPN access, we periodically access the servers that the software is running on to ensure the software itself is running and make sure everything else related to server performance of the system is fine.
While we provide the software to the customer that allows them to see into their own systems, we also offer and leverage our past experience to be that extra set of eyes to tease out problems and institute solutions that we have successfully used before. SPS offers our services separately or integrated with the DICOMetrix software on a quarterly, annual or multi-year basis. Our consulting services are designed to get past the impediment of a capital purchase for the DICOMetrix software. Coupled with our experience and packaged as a service, SPS offers everything as an operational cost.
No comments:
Post a Comment