Broken tiles: A retro-conversion project

Over time, certain file formats become obsolete. When ODW implemented the first pan-zoom viewer in the VITA Toolkit in the 2010s, it was based on uploading large files made up of hundreds of little tiles all zipped into a folder. The once-free tool is called Zoomify. Over the years, we encouraged our users to “Zoomify” their full images and any pages of multipage items so that those items could be zoomed into and rotated for a dynamic user experience. This was particularly useful for scrapbooks where pasted items were sometimes in different orientation within a single page. Also, detailed items like the Welland Canal Records benefitted from this “Zoomification”. However, these folders of tiles were quite “heavy”, i.e. required more storage and some eventually became corrupt. 

Zoomify Tool “tiling” an image

Luckily, as technology has advanced and streamlined, the standard is now to use JPEG-2000 (JP2) files that automatically trigger the open-source IIIF (International Image Interoperable Framework)viewer in VITA. So, any user uploading full images, details, or pages can upload the considerably lighter and mobile-friendly JP2 file and it displays with all the pan, zoom and rotate options people expect for viewing this kind of material online. The trick was that we needed to go through our system and replace the old Zoomify folders with JP2 files. We were able to do this systematically for the most part, but some stubborn items required manual intervention and conversion. We were lucky to have Christine Anderson, a Mohawk College Library Technician student, who was willing and able to take on the task. Here’s Christine’s take on the project:

In my time at ODW, I have worked on (and completed) the Dezoomify project which primarily involved using the VITA Toolkit to access and replace collection images and other software for the conversion process. ODW provided me with a list that identified records with broken Zoomify files and I got started on the clean-up-work!

My primary task was to open and convert the broken Zoomify files and then replace them with JP2 files. This was done for Full images, some Details and Reverse images, as well as for many book and scrapbook pages. Using a RecordID list that was organized by Agency, I could identify all of the records with images that needed to be replaced and re-loaded. 

This work was accomplished by:

  • Using the Dezoomify tool which works by copying and pasting the item’s public URL into the tool
  • “Dezoomify” merges the tiles that make up a Zoomify file and that merged image can then be saved as a JPG
  • I used Irfanview software to convert JPG files to JP2 files, and I assigned their original file names so that the agencies could trace the display files back to their master copies
  • In the data management side of the VITA toolkit, I then activated a task-specific button to replace the broken Zoomify files with newer (and unbroken) JP2 image files
  • When certain Zoomify files were identified as too corrupt and this simpler workflow did not work, a workaround was created:
    • In some cases, I could open the PDF file associated with pages and save them as JP2 – although these tend to be quite large, so we adjusted the quality during the conversion process to reduce the storage overhead
    • In other cases, where there was no PDF, I would open an alternate JPG file for Full and Detail images and simply used the standard “Replace” button for the Full or Details file
  • The new files then automatically populate along with their records and now remain either public or non-public according to their original setting.

The JP2 files open in a IIIF viewer and provide excellent Pan-Zoom capabilities, like the slideshow below illustrates.

The Dezoomify project concentrated mostly on file creation and replacement (for example: digital collections from libraries’ local history/genealogy departments), and to an extent included working on the Metadata for the files submitted. The project consisted of a bunch of repetitive tasks that were not able to be automated and had to be manually manipulated/updated. This was important database work that will ensure the integrity and currency of the files uploaded to the clients’ digital collections and sites going forward.

There will always be advancements in technology standards and these inevitably require adjustment and retroconversion activities. With Christine’s work complete, the ODW team was able to purge a considerable overhead of corrupt and cumbersome Zoomify folders from the database. The positive outcomes of this work is a reduction in the affected agencies’ storage and the cumulative burden of these obsolete files on the servers, plus Christine gained new technological skills that she can carry forward in her career as a Library Technician. It’s a win-win!

Daily British Whig 1902-1926 now online

OurDigitalWorld is excited to announce that the Daily British Whig from 1902-1926 is online. The Frontenac Heritage Foundation undertook the project to digitize this significant set of community news, covering the first of the World Wars, and make the papers available as part of the larger Kingston newspaper collection hosted by the Kingston Frontenac Public Library.

With the addition of these almost 90,000 pages, the online Kingston newspaper collection has doubled and now ranges more than 100 years, from 1810-1926. The Digital Kingston VITA Toolkit site at http://vitacollections.ca/digital-kingston/search allows users to search by keyword and facet results to sort or narrow them by date, publication, and more.

Daily British Whig October 9, 1909

OurDigitalWorld worked with Library and Archives Canada via the Canadian Research Knowledge Network to access and digitize the microfilm copies, and with University of Windsor to achieve high quality positional OCR processing. The newspapers are uploaded into the VITA Digital Toolkit for search and display with full text search and hit highlighted results. Frontenac Heritage Foundation member John Grenville used the new primary materials to research a local architect Ernest Beckwith, designer of the Orpheum Theatre in Kingston, and returned very specific results.

ODW, Kingston Frontenac Public Library and the Frontenac Heritage Foundation encourage genealogists, students, and other researchers’ use and exploration of this important set of newspapers. To read the full press release and for contact information regarding the project, click here.

Featured Image courtesy of Maritime History of the Great Lakes Digital collection