"with some mechanism for retrieving, should it be required."
This is where the catch is. Have an explicit understanding with your PACS supplier/storage vendor: 1. What will be the process of retrieving data if I need it-even if it is 25years later? 2. Will they ensure the data will actually be retrievable?
Storage hardware is cheap, but "safe secure, managed, retrievable storage" --which is what you need for patient data is expensive.Remeber you will also need to migrate the data out of the storage when you change your vendor. Remember the LSPs CDS were quoted at costing 35million for NHS.
Roy, Trusts will need to make their own decisions--what works for them both in terms of cost & patient benefit.
I was going to start a new thread based on some of the comments coming out of EHI but this looks like a good place to continue this conversation. The idea of culling is predicated on the perspective that storage is expensive.
It *is* true that while 2 or 3 TB disks go for next to nothing nowadays - properly managed storage with SLAs and full redundancy does cost more. But it would be unwise (IMO) to judge the situation on the basis of costs/quotes made 7/8 years ago when SAN technology was relatively young.
I'd like to pose the question (note this is NOT market research :-) ) of what folk think is expensive (or value) for storage. As a benchmark I took a look at the Amazon S3 storage facilities which include multiple redundant silos and pathways. The actual cost depends on how much of a number of options (e.g. used bandwidth - anyone else fly RyanAir?) are included but I roughed up a ballpark figure of £50k p.a. for 30 TB of storage.
That, of course, isn't local storage - and I know the CDS/Cloud paradigm has lost a little of its lustre. But is that expensive? If not - would it be better to put pressure on vendors to find realistic costs rather than cull data.
posted on Wednesday, November 23, 2011 - 08:04 am
Martin, I agree with what you say. However, there are a few issues. 1. We cannot keep data for 200years or more can we? We have to cull data at some stage. 2. What robust evidence is there that keeping data beyond the previous tried & tested 8 years for films brings clinical benefits---which is statistically significant. 3. £50K for year for a Trust is significant savings. 3. Then you have the cost of migration--currently expensive.
OJEU taken by Trusts in 2013 will certainly bring the costs of storage down. OJEU is the way for NHS to flex its muscle as a customer.
The costs of storage the NHS are playing today are ‘high’ to say the least and 2013 changes are going to see a dramatic reduction of these costs, whatever happens. However, in 10 years time things will have moved on so my opinion is that ‘storage efficiency’ is at least as important as cost (though I wouldn’t want to have that argument with an accountant!)
To understand how to store something ‘efficiently’ you have to consider the data profile, how data is created, how it is consumed and what actions are required to maintain it. PACS data has a profile that sees a lot of initial access, then very little access thereafter, which would suggest that a total SAN or total offline solution would not be efficient. Organizations are looking to create multiple copies of data for protection reasons and it would seem wise that each of these copies is retained for different periods (e.g. SAN data is retained for shorter time periods) and each copy resides on a different storage technology (e.g. data to be retained for long periods keep on NAS or Offline).
Migration is a good example of measuring an efficient system. Choosing a system based just on price and not considering efficiency is likely to mean you have a migration headache all over again in the future. It is interesting to note that the OJEU’s that are in process thus far are considering protection as a must have feature but few have really laid out plans for efficient storage.
Public Cloud Storage (e.g. Amazon S3) appears to have one critical problem, contracts. SUN Microsystems, Concord and Bearings Bank all have one thing in common; they no longer exist. Creating a contract for sensitive medical data with a public company that guarantees what is going to happen with the company (and your data) has proved nearly impossible.
Private Clouds are going to have a place but the efficiency question still exists as ‘cloud’ is just an interface to a storage system; the physical storage systems haven’t changed.
@Jamie Agree fully with your point about public storage - applies equally to co-location and is just the point I made at the London meeting in March. I used Amazon purely as an easily accessible example of numbers for today's managed storage pricing.
1. We cannot keep data for 200years or more can we? We have to cull data at some stage. 2. What robust evidence is there that keeping data beyond the previous tried & tested 8 years for films brings clinical benefits---which is statistically significant.
True, but if storage cost is taken out of the equation, then culling can be a one-off operation at contract end - based on really basic rules like death + X years - as Roy suggested above. There shouldn't be a need otherwise
3. Then you have the cost of migration--currently expensive
In my mind it would be better to address why migration costs are expensive than to limit the amount of data migrated. Two factors that affect the amount of work needed to migrate are data quality and proprietary data structures. Data quality is something that should not be as big an issue as in the past now that the likes of OCM and MWL are the rule rather than the exception. Proprietary elements are a different story but if there is intention to impose limitations on which data is migrated - why not impose those limitations on the proprietary data only - say for the last 5 years of data. Images older than that 5 years can be migrated with DICOM Q/R - then at least you have the original data to work from (and anyway - if you are replacing PACS presumably its because the new one is *better* so are those proprietary elements worth keeping?). Again - taking the cost of storage out of the equation yields different ways to skin the cat.
OJEU is the way for NHS to flex its muscle as a customer
But in terms of storage costs the OJEU process is a two edged sword. Particularly in terms of contract length. If there is one thing that can be forecast with any certainty it is that storage costs in 5 years will be much lower than today, and in Birmingham contract periods of 10, 14 and even 20 (only partly seriously) years were in the air. Vendors will be limited in the amount of discount they can apply for future reductions - if any - so a contract negotiated today will inevitably be overpriced in the future. Ironically, the push towards VNAs exacerbates the situation as vendors can't be confident that any discount applied today can't be recovered in the future.
posted on Wednesday, November 23, 2011 - 05:21 pm
Thanks Martin & Jamie for your valuable contribution to a debate. This is vital for the user community to hear different points of view.
"culling can be a one-off operation at contract end" --I dont agree with this at all. As you yourself said that same PACS contract can last 15-20years. Rather than trying to address the culling issue head-on with a well managed culling process---we are leaving this unmanaged to the end of contract. The net result is we will have some views expressed "How can we lose this data. Let us just extend the contract with current PACS provider". Which has been the mindset on many PACS contracts extensions. I think this will be disasterous for NHS with continued high costs. I think we need to address this head-on with simple culling rules as suggested by Andrew. I think with more of us asking for this in our 2013 replacement contracts we will have PACS companies forced to deliver this. I dont think it is difficult. It is a chnage in mindset of PACS companies that is required.
Regarding future data migrations--I think we need to address this head-on as well. I think having a VNA as a back-up standard DICOM archive with ability to do do "tag-morphing" on the way out--will give customers control over their own data. Thus by having a VNA (ensure the VNA is specified correctly though) customers will be able to migrate the data to another PACS vendor at the end of contract (without needing support from existing PACS vendor)
If NHS Trusts do not look at addressing both issues in 2013 ---it will be a costly mistake for these Trusts.