UK Imaging Informatics Group
Archive through August 11, 2011 PreviousNext
UK Imaging Informatics Group > Questions & Answers > PACS Procurement & Business Cases > PACS Data Migration >
Message/Author
 Link to this message Marco Crispini  posted on Tuesday, August 02, 2011 - 02:43 pm Edit Post Delete Post Print Post
* Commercial Interest *

I think Kevin's comment, The Tortoise sometimes wins I understand (as long as he starts on time), hits the nail on the head, and it is also in Simon's recommendation as I understand it. From our experience of migrations, we are offering that approach (I refer early birds to http://cypherit.co.uk/pacs2013/)

I also note from the posts that there's a lot of fear of vendors being uncooperative, but experiences of migrations posted here have had co-operative vendors. Those vendors are the PACS vendors; I guess the fear is whether the LSPs will co-operate in the same spirit?

Finally, Simon when you say 'PACS secondary store on your own site(s)', are you saying a VNA for an interim period? Or a full PACS, as that would mean the migration would be done, ie without any fence-sitting?
 Link to this message Simon Waddington  posted on Tuesday, August 02, 2011 - 03:27 pm Edit Post Delete Post Print Post
Hi Marco. I meant reproducing the function, structure and supplier for the off-site secondary store on-site within the Trust but at a different physical IT location for resilience purposes i.e. as per when moving from one LSP (Fujitsu / CSC) to another. In this case it would be LSP to NHS Trust. This is so you can extend the contract without the LSP being then involved until there was the resources and technical availability to make a fully informed decision as to where to go. I know of one Trust who made an expensive mistake in copying long term data to migrate and effectively got a load of unusable information and had to start the process all over again. As James highlighted, understanding the contract and all of the issues is key before embarking on a process but having options / a Plan B is necessary due to the sheer size of the problem / number of Trusts co-terminating.
 Link to this message Neelam Dugar  posted on Wednesday, August 03, 2011 - 02:43 am Edit Post Delete Post Print Post
The debate here is a good one.

1. DICOM q/r migration
Standards based migration so PACS vendor HAS to oblige.
Due to slow speed in migration, it requires NHS as a customer to have new PACS/VNA vendor in place perhaps 1 year before LSP contracts expire. So 2 vendors to pay for 1year. (As you need somewhere to migrate data into.) If you choose a VNA instead of a PACS vendor -- you also need to migrate to new PACS from VNA. Old PACS to VNA to new PACS.
2. Bulk Migration
Faster migration. But will depend on support from LSP/PACS.
Overlap between old & new PACS vendor maybe as little as 1-3 months.

James is right. This is more of a political issue than a technical one. How & when will LSPs release data. The answers lie with CFH. Until we have clarity on how & when data will be released it is difficult for Trusts to go into OJEU. We need to know what to OJEU for. What kind of migration support we require.
 Link to this message Richard Mann  posted on Wednesday, August 03, 2011 - 10:11 am Edit Post Delete Post Print Post
Have any Trusts actually formally contacted their LSP provider and said:
1)they wish to (or are seriously considering) exit in 2013
2)asked how can they get their data back
3)if there are options - how much do they cost?

If you are serious about ojeu you probably ought to kick that off and not wait for CFH!
 Link to this message Neelam Dugar  posted on Wednesday, August 03, 2011 - 11:27 am Edit Post Delete Post Print Post
Yes. Asked those very questions. :-)

They say negotiations are ongoing with CFH.
 Link to this message Neelam Dugar  posted on Wednesday, August 03, 2011 - 07:42 pm Edit Post Delete Post Print Post
LSPs are offering extensions for Trusts who wish to take it.

However, any Trust who wants to come out of the contract, it is very difficult to plan or get any indicative costs from suppliers without the following information:
1. Method of data migration which will be allowed
2. Speed of data migration which will be allowed
3. Cost of data migration from LSP

Any commercial company will try to maximise their profits. Remember what protects customers is
1. Standards
2. Contracts

Just another point to make, although I am no expert, i have been told that Trusts will not be able to do direct deals with Existing PACS suppliers ( thus cutting out the middleman) as this will be challenged.

Most current PACS have come of age-- they are hardware independent. You can use recently refreshed workstations and also your Trust storage infrastructure ( Mathew Condron will be speaking on this at our next meeting).

Hence, Trusts need to compare software & service charges from LSPs and directly from major PACS vendors. Gosh it is an eye opener!!!

Moving ahead i would suggest writing this in your new contract.
"3 months before the end of contract the PACS supplier will be able to handover all the stored image data in DICOM part 10 format. Data stored will be generally for a period of 7 years ( as per DOH guidance). Supplier must support both DICOM push & bulk data transfer standard of DICOM (when it become available). If PACS supplier thinks data transfer to another PACS archive will take more than 3 months ( for 7 years of data stored), this must be explicit in their response to OJEU tender"

Any comments on the wording of the above statement is welcome.
 Link to this message Dave Harvey  posted on Friday, August 05, 2011 - 01:10 pm Edit Post Delete Post Print Post
Neelam & All,

=20

Sorry for coming so late into this discussion, but I think that it's worth clearing up a few potential issues:

=20

1) Having access to all the data in "Part 10" format is useful, but in practice other well documented and accessible formats could do the job when handled by a competent migration service, and being in Part 10 format does not in itself guarantee usefulness as below....

2) DICOM allows ANY transfer syntax, including private ones, to be used with the Part 10 "wrapper", and there is no point is having "part 10 format" files if the vendor has chosen to use some weird and "wonderful" transfer syntax/compression method. Of course, those that do this (and you know who you are!), claim that it is for performance/size reasons, but we all know that the result is that it effectively becomes a form of "encryption" - which can act as a major barrier to migration away from that particular vendor.

3) In practice, virtually no PACS handle demographic updates by making changes to the data stored on disk - and store the data instead in the database. Whilst a nuisance for migration, this is in fact done perfectly good reasons, including:

a. Performance (no need to change images which might never be read again anyway)

b. Consistency (backup tapes, remote duplication etc.)

c. Safety (1) - since the original data is always available for demerge etc. if mistakes have happened (this is a real issue - I have seen demographics changed, in the UK, with NO record of what they were before!)

d. Safety (2) - disks used for image storage can be treated as if they were WORM drives (write once - read many), and can be "sealed" with writing disabled [possibly in hardware] when full to prevent any further writing. I don't know how many PACS actually use this approach (often discussed in the field), but it is potentially useful protection against viruses etc, and it would be a shame to introduce rules which prevented it being used.



In practice therefore, it is more realistic for now to demand (if the LSP contracts allow us to make any such demands!), that the demographic update data is simply available in an open, documented and accessible form.



Longer term, we need an agreed, single, documented and vendor-neutral format for holding such changes (perhaps using the newly defined DICOM change objects), and this is likely to be the subject of a new IHE migration profile, but this is not a simple issue to resolve, and will not be done before 2013. =20

=20

Of course, two other points made in this thread probably have far greater bearing on this problem, both related to the nightmare of being forced to use LSPs:

=20

1) "pre-nuptial" agreements when buying a PACS are vital to ensure that the eventual divorce goes smoothly, but of course, we don't know whether they exist (if anyone knows whether anyone in CfH bothered to negotiate these 8 years ago, we'd all love to know details!)



2) Reputation in the PACS industry is important, which is why, in practice, the PACS companies normally behave well and helpfully when being the outgoing system, since they have a realistic hope of business in the future and don't wish to upset their customers and their customers' friends. It remains to be seen whether the LSPs, knowing how much they are hated, and how no-one would ever buy from them again, will ALLOW the PACS companies to be that helpful.

=20

Best wishes

=20

Dave Harvey
 Link to this message Martin Peacock  posted on Friday, August 05, 2011 - 04:32 pm Edit Post Delete Post Print Post
"Longer term, we need an agreed, single, documented and vendor-neutral format for holding such changes (perhaps using the newly defined DICOM change objects), and this is likely to be the subject of a new IHE migration profile, but this is not a simple issue to resolve, and will not be done before 2013."

Especially true since the RFP/contract must be written well in advance of 2013. However it *should* be possible to word a clause such that "an IHE migration profile must be supported within a given time of it being ratified".
 Link to this message David Clunie  posted on Saturday, August 06, 2011 - 01:54 pm Edit Post Delete Post Print Post
For once I actually completely disagree with Dave Harvey's conclusions (which is unusual), though he makes some valid points.

The last thing we need is a battle to try to standardize the database format, since one of the problems with the database content is that it is very vendor specific and open to significant interpretation as to how it is to be applied. I believe any such effort will fail, if for no other reason than it is little incremental benefit beyond the state of the art which is to dump the SQL schema and tables.

I would far rather see every PACS have the ability to RAPIDLY create a NEW ARCHIVE COPY on an externally supplied filesystem (directly or network-attached) of Part 10 files with a standard (lossless) compressed transfer syntax and all the "headers" up to date, by using its own database information and applying it to its own internal archive (whatever that format is). Ideally, the PACS would be capable of recording the snapshot state such that another such archive containing only the delta since the last time could also be created.

This is essentially a static incarnation of a query/retrieve of the entire system, but without the network overhead, and seems to be a far easier than the alternatives.

It also means that there is no need to obsess about whatever "internal" format the PACS's own native archive uses, which as we know may be "optimized" in many ways.

The question of safety is a distraction; I expect there are many more human errors made that need to be and are fixed such that the final (most recent) state of the PACS is correct, than there are errors introduced by such manipulations. If there is a way to retain and export an "audit trail" (e.g., IHE IOCM) of changes made that could be imported downstream then that would be nice, but IMHO it is far, far more important to have the current (close as possible to correct) state than any history. There is going to be a finite error rate after any migration, just as there is a finite error rate in the existing PACS state.

There is also the question of stuff other than images that need to be exported, and for example, annotations and presentation states that are stored in the database should also be exported to the "current state external Part 10 archive", so that they are in a standard DICOM form. Again, the last thing we need is a debate over how to standardize the "database representation" of such annotations.

Same goes for reports if they are included in the migration (i.e., the RIS is migrated as part of the same exercise or reports are stored in the source PACS). These can be stored in DICOM as plain text SRs or encapsulated CDAs or PDFs or whatever is appropriate. At the other end (receiving PACS/RIS) these would likely be converted and morphed into some other form, but again we do not need to initiate a "reports in the database format" standardization debate.

David
 Link to this message Neelam Dugar  posted on Sunday, August 07, 2011 - 05:05 pm Edit Post Delete Post Print Post
Thanks Dave & David for your contributions as world experts in DICOM.

I am surprised & little concerned that demographic & other updates are kept in a PACS vendor proprietary database separate from the PACS image database.

1. Unconcious patient would still be called unconcious on the image database despite when the name becomes known. Demographic change is saved on vendor specific database!
2. Regarding annotations-- I am a little vary about radiologists/doctors making permanent changes to images ( embedding measurements, annotations-- this is the tumour etc) as images maybe reviewed by multiple radiologists, or clinicians esp through MDTMs. Our PACS admin staff make changes-- for left/ right in correct, wrong lablel etc. If these changes are made they should "travel" with the image alongwith appropriate audit trail.

If we want continued innovation in PACS, easy migration between vendors is key. I hope DICOM & IHE committees & PACS vendors see the importance of this.
 Link to this message David Clunie  posted on Sunday, August 07, 2011 - 07:11 pm Edit Post Delete Post Print Post
Hi

I was referring to annotations that a reader might make during reporting (like a pointer to a lesion or a measurement, or designation as a key image, etc.), and in my opinion these should be kept.

Using electronic annotations to correct incorrect acquisition technique (like wrong side) is potentially an extremely unsafe practice unless great caution is taken to make sure that the changes propagate EVERYWHERE downstream (e.g., to the CD and to third party display software during import or query/retrieval). The only really safe way is to completely fix everything in the header and burned into the image so that it matches reality (assuming that reality at the time of acquisition is actually known; otherwise a retake may be needed). However, if you are following the practice of doing this, it is even more important to retain these during migration.

There is a recent discussion about annotations for such corrections at "http://www.auntminnie.com/forum/tm.aspx?m=312742".

This is distinct from demographic corrections (e.g., as per IHE PIR, and IOCM, for example).

David
 Link to this message Martin Peacock  posted on Sunday, August 07, 2011 - 09:08 pm Edit Post Delete Post Print Post
Neelam.. "I am surprised & little concerned that demographic & other updates are kept in a PACS vendor proprietary database separate from the PACS image database. "

In all fairness to vendors that is a pragmatism that has applied certainly until only recently. With the cost of storage media nowadays it is certainly possible to ensure all raw data (i.e. in the archive rather than the database) is available for correction, although that hasn't always been the case (and from some standpoints - may not be the optimal state). In the not-too-distant-past the raw data could have had a high probability of being on off-line media. That issue has always been mitigated by the assumption (standards - based) that the up-to-date information is always available through the DICOM mechanisms. In principle, it shouldn't matter how the information is stored - only how it is ultimately presented. Until of course, it comes to migration on this kind of scale, where it *can* be a logistical millstone.

I think your concern about the unconcious patient is a different thing. That information would not be PACS-based but RIS-based. Whatever about the complications with a PACS migration, RIS migration, I think, is the elephant in the room. HL7/IHE has that kind of scenario covered quite well (human processes notwithstanding) and any PACS would incorporate that.
 Link to this message Neelam Dugar  posted on Monday, August 08, 2011 - 04:54 am Edit Post Delete Post Print Post
David,

Thanks for clarity that there is a difference between
Fixing-- done by system admin which as I understand now are changes made on images so held in a standard DICOM format
vs.
Annotations made by radiologists when reporting which are held in PACS proprietary database.

I still do not understand why demographic updates are not applied to the DICOM image database by PACS vendors. I agree with Martin that perhaps this relates to data held in tape drives in the past. With decreasing price of storage perhaps its time we see a change in philosophy. If we are keeping patient data should it not be demographically up-to-date? Should data that is " handed over" at the end of contract be demographically up-to-date. It is clear that unnecessary storage of data is expensive ( although storage hardware maybe cheap these days.

I agree with Martin, the real elephant in the room is the RIS. My view is that RIS reports must be held in standard CDA format. So that RIS migration can be standardised. There is NO NEED for PACS migration to involve migration of reports. Your new PACS vendor should support "Displayable Report Profile" of IHE so should be able display CDA reports stored in RIS. Dirco Van Norden, Brian Ellwood & Alan Mclaren will discuss RIS, CDA, migration of RIS & PACS display of CDA at the Autumn Meeting. I must agree with Martin that adressing RIS migration is important too.
 Link to this message Neelam Dugar  posted on Tuesday, August 09, 2011 - 08:40 pm Edit Post Delete Post Print Post
In addition to issues surrounding data migration we have been discussing another important issue--errors on images sent to PACS and how to "FIX" them--using standards if possible.

David points to a good recent discussion on Auntminnie
http://www.auntminnie.com/forum/tm.aspx?m=312742

The other issue we have been discussing is deleting data based on local policies.

There 2 new IHE profiles proposed which seems to be dealing with both these issues.
1. "Imaging Object Quality Control & Maintainence"
2. "Basic Imaging Object Change Management"

http://wiki.ihe.net/index.php?title=Imaging_Object_Quality_Control_and_Maintenan ce_-_Detailed_Proposal

Practical Use of Imaging Object Quality Control & Maintainence profile:


"1. Quality Control
A study received at a local PACS is sent to the regional Archive
Later on mistake is noticed on the study (e.g. two different acquisitions incorrectly merged into one study due to incorrect worklist item selected)
Quality control procedure reconcile the study at the local PACS
Such reconciliation needs to be propagated to the regional Archive. Ideally this can be done automatically.

2. Recall Wrong or Poor Objects
Technologist performed acquisition
Later discovered that the left/right marker was wrong
Technologist wants to recall the objects
Ideally all systems that have received the initial object should also stop any further circulation

3. Data Retention Management
Data retention policies trigger the condition that particular studies (i.e. based on age) should be deleted
Need to be able to notify all systems that they must delete any data they may have related to these studies
If a separate system handles data retention policies, it also requires the same type of transaction to notify one or more 'Image Managers' that a particular study should be deleted.
Features include
manual delete,
scheduled delete (e.g. due to data retention policy)
supercede/replace,
a) Study Split (e.g. an incorrect worklist entry is selected for an acquisition. This causes the current acquisition to incorrectly merged into an existing study. The incorrect portion of the study needs to be split out)
b) Study Merge (e.g. After the incorrect portion of the study is split out, it is now updated with the correct order. There may already exists a study for the correct order. So in this case, the split out objects will be merged to the existing study)
c) Study Fix Up (e.g. an incorrect worklist entry is selected for an acquisition. A correct entry is selected later. This may change the study instance uid of the existing objects)"

http://wiki.ihe.net/index.php?title=Basic_Imaging_Object_Change_Management_-_Det ailed_Proposal

"Features in Basic Imaging Object Change Management include
manual delete,
scheduled delete (e.g. due to data retention policy)
Undelete?,
supercede/replace,
Update Patient Demographics,
Update Procedure Information,
Merge Patient Records,
Link/unlink Patient Records,
Update Study Level Attribute,
Update Series Level Attributes, and
Update Image Level Attributes;
Study Split (e.g. an incorrect worklist entry is selected for an acquisition. This causes the current acquisition to incorrectly merged into an existing study. The incorrect portion of the study needs to be split out)
Study Merge (e.g. After the incorrect portion of the study is split out, it is now updated with the correct order. There may already exists a study for the correct order. So in this case, the split out objects will be merged to the existing study)
Study Fix Up (e.g. an incorrect worklist entry is selected for an acquisition. A correct entry is selected later. This may change the study instance uid of the existing objects)
Subscribe/unsubscribe to change notification
Revision Log?"

I do not know how far these profiles has gone. David or Dave may know better, whether there is interest from PACS vendors and whether these proposals have been accepted for development. They are worth a read as this will standardise our specification. There seems to be some overlap between these 2 profiles suggested.
 Link to this message Dave Harvey  posted on Wednesday, August 10, 2011 - 11:22 am Edit Post Delete Post Print Post
Neelam,

I think you may have misunderstood here:

"I am surprised & little concerned that demographic & other updates are kept in a PACS vendor proprietary database separate from the PACS image database.

1. Unconcious patient would still be called unconcious on the image database despite when the name becomes known. Demographic change is saved on vendor specific database!"

This is not the situation - there is (at least logically!) only one PACS database, which is updated with demographic updates. The only thing NOT updated by most systems currently are the image files themselves. So as far as a user is concerned everything is consistent - they query for Joe Blogss, and when they query for his images, they get images which contain the name "Joe Bloggs" - this is as specified in the IHE PIR profile. Only in the background, on the internal file system would there be the original "file" still saying "unconscious" - hence why this issue ONLY comes up in the context of migration.

Dave
 Link to this message Sven Bollue  posted on Wednesday, August 10, 2011 - 02:18 pm Edit Post Delete Post Print Post
Hello Neelam,

I second Dave Harvey's posting. I don't know if it's the case for all PACS systems around, but normally the dicom images that are in the PACS cache remains unchanged, and the metadata that goes with the images is kept in a database that gets updated as, for instance, the PACS system receives patient or study updated.

To give you an example: if the PACS contains like 12 extensive CT studies of Miss Suki Yamamoto, and Suki decides to marry her long time boyfriend Take Kanakawe.
After her marriage, Suki visits the Doncaster Royal, and her identity is updated in the HIS from Suki Yamamoto to Suki Kanakawe.

Instead of updating the dicom tags of every single dicom image in the cache to reflect Suki's updated identity, the PACS will only update the metadata related to the studies in the database.

Of course, in case Suki asks to have her study on a CD, the PACS system will export the dicom images with the new patient name overwriting the old patient name in the dicom tags of the exported images.

Now, tadaaaaaaaa, this fact led to some strange situations with integrated third party software a few years ago. Some of the integrated third party products (it was an orthopaedic application in the given case) were designed to pull the images they needed straight from the PACS image cache, and didn't mind about the metadata present in the PACS database. This resulted in third party applications suddenly showing John Doe identities, and former patient names for patients that were, in the mean time, married or, sadly, divorced.

In the mean time, this problem is commonly known and mitigated right from the beginning.

Cheers,
Sven
 Link to this message Neelam Dugar  posted on Wednesday, August 10, 2011 - 11:57 pm Edit Post Delete Post Print Post
Dave,
I did understand the context
1. DICOM image object database/archive--is a standards based database
2. PACS database-proprietary and contains metadata with demographic updates, annotations/ measurements etc

Bulk Migrating the DICOM objects is easy due use of standards. Bulk migration of propietary data is diffcult due to lack of standards.

My view is that all data should be held in standard format within image object along with audit trail of changes. The 2 proposed IHE profiles described by Kinson Ho from Agfa are a step towards this.

Addressing easy bulk PACS migration is key to sustained innovation in PACS
 Link to this message Paul Ganney  posted on Thursday, August 11, 2011 - 10:00 am Edit Post Delete Post Print Post
Beware insisting on a "standard" database. Where else are vendors going to put the enhancements you want (or give them a competitive edge) in 2020?
 Link to this message Nick Gibson  posted on Thursday, August 11, 2011 - 10:24 am Edit Post Delete Post Print Post
Neelam,

I think we may have a slight mis-uderstanding/ambiguity in terminology.

There are two slightly different uses of the word 'database' more generally it can refer to some system for storing data, could be anything, any size, any form. More specifically, it can refer to a system of data tables, normally held in an SQL based relational database such as Oracle or Microslft SQL server.

As far as PACS systems are concerned, most will use a SQL database with data tables holding the data e.g patient/ study details etc, the actual images will be in DICOM part 10 files (usually) on a file system. The SQL database tables will include references to the image files by filename to allow the PACS system to link the two and display the correct images.

I think Dave/David uses the relational/SQL database sense of the word.

The images files therefore could and should be standards based (DICOM part 10) whereas it would be difficult to standardise the relational database structure (although it may possible to standardise key parts of it up to a point)

hope this helpls,

Nick.
 Link to this message Neelam Dugar  posted on Thursday, August 11, 2011 - 04:00 pm Edit Post Delete Post Print Post
Thanks Nick. I do agree-- sorry about my not so technical choice of words.

Today innovation in PACS continues because market is not saturated. Once we have a saturated market, and the PACS vendor has locked us in, our existing PACS vendor and other vendors will not be interested in us as a customer.

I am grateful for all the migration vendors who have emerged. They are valuable, as they confront the lock-in issues. However, we must not forget migration is expensive. Expense largely depends on duration of migration as you have to pay revenue to 3 vendors -- outgoing vendor, incoming vendor & migration vendor for the duration of migration. If you use VNA as temporary store you will migrate twice, outgoing PACS to VNA, and then VNA to new PACS. Decisions need to be thought through carefully by Trusts.

Bulk migration seems to be a sensible solution. Talking to a migration vendor what was emerging was
1. Migration of image file database is fairly easy-- (I presume it is because of standards based DICOM image objects). This explains why it is important to ensure DICOM part 10 file standards are adopted.
2. Migration of relational/SQL database- containing metadata, demographic changes, annotations etc. This is more difficult & expensive to migrate due to lack of standards. As we do not store annotations made on PACS in our local deployment, ours will be a relatively easy migration.

The easiest option for Trusts maybe to stay on with LSPs in 2013. However, legally they can only extend till 2016 from what I hear. 3 years extra on high service charges( despite discounts offered by them), & 3 years more of data to migrate in 2016. So whilst it may appear an easy option, it is the most expensive in the long run.

I would like to make it clear, that I do not have any personal issues with Agfa. I think they are a good supplier. Issues around migration are a generic one. We need to address them irrespective of whether you like you existing supplier or not.

My view is that if PACS suppliers adopted the IHE profiles of Image Object Change Management & Image Object Quality Control & Maintainence just bulk migration of image files would be adequate. As the PACS relational database could simply be synchronised with RIS using HL7 messaging standard.