Preserving medical records of Norwegian citizens
All historical, current and future medical records from deceased patients treated by the Norwegian Specialised Healthcare Services must by law be remitted to the Norwegian National Health Archive.
CHALLENGE
The Norwegian Health Archive (NHA) carries the responsibility of ensuring that the nation’s health data lives on indefinitely. The NHA has begun a 10-year project to establish a longterm digital archive for medical patient records. This project has global significance for how data can contribute to understanding a nation’s health. This is an ambitious project, collecting decades of records, in both analogue and digital form, and collating and treating all data in a single preservation system.
The two main challenges of the project are ingesting the diverse and large amount of data, and ensuring the preserved data is accessible for the future.
After receiving the physical and electronic journals of deceased people from both public and private hospitals, the NHA digitises the analogue records, at a rate of 300,000 images per day (300-450 GB). This creates a lossless electronic representation of the records, which then begin ingestion into the preservation system. Simultaneously, digitally born electronic health records (EHRs) are also processed for ingestion. Due to the sheer volume of digital files, ingestion of these files will occur over 10-15 years. After this initial ingestion of records, 50,000 EHRs will be ingested annually, equating to 1TB per year (with each record approximately 20 MB).
In addition to the data throughput, the solution must provide data access tools for researchers and next of kin on request, while maintaining the authenticity, confidentiality, scalability, quality, and retention of the content.
To find a solution to fit the complex needs of the Norwegian Health Archive, an extensive requirement specification process was conducted, in collaboration with the Norwegian National Archive and Norwegian National Library. As a result of the process, a list of almost 100 requirements was made, asking the market for a state-of-the-art digital preservation system to solve these challenges.
“ Piql have shown real long term vision in their approach to this project. Artefactual and Piql together will help the NHA to deliver innovative new capabilities in health record preservation and access in Norway. Through the use of the open source Archivematica platform, this project will benefit digital preservation efforts across the world for years to come.”
SOLUTION
Following a substantial procurement process, Piql won the tender together with Artefactual Inc, Canada. Our solution provides an Archivematica based digital preservation system with complementary modules to ensure reliable ingestion and a flexible health register, compliant with OAIS. The solution consists of three modules addressing the challenges defined by NHA. Ingestion module which processes the data and automatically creates Submission Information Packages (SIP). This module has been designed for handling three different types of data: born-digital health data (EHRs), digitised health files and complementary files. As the EHRs and the digitised records have different format specifications, the module includes a new tool that automatically monitors data validation, triggers data transformations, and creates OIAS compliant SIPs.
For the complementary non-patient journal files, another ingestion tool was developed and implemented to perform data validation, metadata descriptions and package creation on these files. These processes support and ensure the authenticity and quality of the SIPs.
FUTURE
The project strated spring of 2019, with full deployment in September 2020.
Further, the parties have signed a 10-year support contract for the preservation system, ensuring a long and fruitful partnership between the two parties.
This collaboration will also contribute to enhancing global awareness on how to safeguard and preserve electronic health records, a new yet important domain. Further, the workflow established by the NHA could also be adopted by other similar public institutions with a responsibility to store digital files for the long term and guarantee future access for future generations to conduct research and gain insight to knowledge.