Category Archives: Digital Preservation

Public Use and Interest of 3D Data in the ADS

https://archaeologydataservice.ac.uk/archives/view/forseadiscovery_2018/photogrammetry.cfm?pgsite=Bayo_Area1

Over the last decade, archaeological 3D data has become more accessible to the public. There are online archives, similar to the ADS, where users can access this data for additional knowledge in specific artifacts and archaeological sites which have been recorded using various forms of 3D data. While this data can be viewed, some online archives do not have their data open for the public to download and re-use for their own purposes.

3D Data in the ADS Archives Questionnaire: Link

This questionnaire has been created by Michaela Mauriello, a University of York Digital Heritage Masters student, to investigate the activity and re-use of 3D data found within the ADS archive. Specific archives provide photogrammetric, CT scans, laser scans and additional forms of data gathered while the project was in progress. The 3D data found within the ADS archives can be downloaded for personal re-use. This downloadable data and metadata can be used in projects and research by those interested in the particular subject.

The ADS utilizes an open sources 3D Viewer (3D Hop) allowing users to interact with 3D data found in archives providing photogrammetric data such as the ForSEAdiscovery. Two versions of the viewer were developed for the ADS and include: the Object Level 3D Viewer and the Stratigraphy 3D Viewer. The object 3D Viewer allows users to move the object to view all sides, zoom in and out of the scene as well as control the lighting in the viewer to see better details in specific areas of interest. The viewer for the stratigraphic data shows different layers found within the archaeological site. This can be seen in the Las Cuevas Project archive.

https://archaeologydataservice.ac.uk/archives/view/lascuevas_uchn_2016/viewer.cfm

With the availability of 3D data found in the ADS archives, it is important to be transparent as to how easily accessible it is to find the archived 3D data. Based on previous statistical analysis performed between October and November 2018, there was limited access to the 3D data found in the ADS archives. This may depend on the visibility of the archive as once depositors submit their work it generally conveys the project has been fully completed and have since moved on to their next research interest.

Additionally, there is no current tab to access an archive that contains 3D data. This stems from the issue of the inability to fully query for 3D data. While it may appear to be beneficial to incorporate a tab to the main ADS archives web page, the question arises if it is actually desired by users. The questionnaire monitors this aspect of desirability through questions about the awareness of 3D data being available to view and download in the ADS archives.

The outcome of this survey will help the ADS prioritize 3D data and how it will be represented in the website. If desired, in the long term the inclusion of an easily accessible method to view the 3D data can lead into more traffic for the archive while increasing the viewership, use and re-use of various projects.

Go To Questionnaire

in the dark near the Tannhäuser Gate

Blade Runner 1982, by Bill Lile Image shared under a  CC BY-NC-ND 2.0 licence

As it’s World Digital Preservation Day I thought I’d finished the following blog about our work with managing the digital objects within our collection. Like most of my blogs (including the much awaited sequel to Space is the Place) these often languish for a while awaiting a final burst of input. To celebrate WDPD 2018, here we go….

Continue reading in the dark near the Tannhäuser Gate

ADS Goes Live on International Digital Preservation Day

On 30th November 2017 the first ever International Digital Preservation Day will draw together individuals and institutions from across the world to celebrate the collections preserved, the access maintained and the understanding fostered by preserving digital materials.

The aim of the day is to create greater awareness of digital preservation that will translate into a wider understanding which permeates all aspects of society – business, policy making and personal good practice.

To celebrate International Digital Preservation Day ADS staff members will be tweeting about what they are doing, as they do it, for one hour each before passing on to the next staff member. Each staff member will be focusing on a different aspect of our digital preservation work to give as wide an insight into our work as possible. So tune in live with the hashtags #ADSLive and #idpd17 on Twitter or follow our Facebook page for hourly updates. Here is a sneak preview of what to expect and when:

Continue reading ADS Goes Live on International Digital Preservation Day

Reflections on CHNT 2016

Back in November (16th-18th), I was lucky enough to be invited to participate in the Cultural Heritage and New Technologies (CHNT) conference in Vienna. As detailed in my excitable post, written  in advance of the event, my involvement was to represent the ADS at the session and subsequent round tables hosted by the ARIADNE project on the subject of Digital Preservation. One of the reasons I was so excited was that it was one of the few occasions on which the focus of such sessions was solely on the issues surrounding Digital Preservation: how it’s undertaken, problems and the challenge of ensuring re-use. It was also the first time, in public at least, that individuals representing organisations undertaking Digital Preservation from across Europe came together to present as a united front and presented to the wider heritage community. In addition, the event also took place at the beautiful Vienna town hall in (see below), a fantastic venue.

Just a normal staircase at Vienna town hall. not intimidating in the least
Just a normal staircase at Vienna town hall. Not intimidating in the least

It was incredibly heartening to hear from European colleagues on their experiences, successes and challenges. I also felt that all the papers in the session – no doubt due to the diligence of  co-chairs from  DANS , DAI IANUS and the Saxonian State Department for Archaeology – meshed together really well. Although there were common themes, each was unique and presented a different tale to tell. Although somewhat biased, at the end of the formal session I came away thinking that I had not only contributed, but had learnt in equal measure. For those interested, IANUS have agreed to host the abstracts and presentations from the session on their website. I’d recommend these to everyone interested in a European-wide approach to the issues of digital archiving.

The first round table followed the formal session, and was listed as an open invitation for delegates to query the archivists in the room about where/when/what/how to archive. Surprisingly, considering the high profile parallel sessions, the room was packed with an array of people from a variety of backgrounds and countries across Europe. As such, the conversation veered between the extreme poles of the subject matter – for example the basic need for metadata versus adherence to the CIDOC-CRM. Reading between the lines here, what I thought the attendance and diverse topics showed was that this type of event was not only useful, but actually essential for archivists and non-archivists alike. Not only to correct misconceptions and to genuinely try and help, but also to alert us to the issues as perceived from the virtual work-face.

After a well-earned rest, and a quick visit to the Christmas markets for a small apfelwein, the next day was a chance for all the archivists to get together for an informal round table on issues affecting their long term, and shorter term objectives. Issues ranged from the need for accreditation – one of the ADS’ goals in this regard is to learn from DANS’ experience of achieving NESTOR – to file identification and persistent identifiers. In this setting the ADS is  perceived as very much the elder statesperson (!) in the room, having been in the business for 20 years now, and it’s a good feeling to be able to pass onto colleagues advice and lessons from our own undertakings. I think it’s important that we continue to do this, not only to be nice (and I like to think we’ve always been approachable!), but also to achieve a longer-term strategic strength. Although we (the ADS) are winning many of the challenges at home in terms of championing the need for consideration of digital archives, there’s always more to be done. When we can also point to equivalents in continental Europe, I feel we only make our cause stronger.

However I’m also conscious that this isn’t just a one-way street and that we’ve still a great deal to learn from our European colleagues. Not only in things like accreditation, but also shared experiences on tools, file formats, metadata standards and internal infrastructure. We often say that Digital Preservation never stands still, so in this regard it’s good to look at what others are doing and reflect on what we could do better.  Events such as this – and the international community of archaeologists doing Digital Preservation built in its wake – serve to make us richer in knowledge, and renewed of purpose. Looking forward to the next one!

Tim

A decade in data

birthday_badgeIt’s hard to believe, but next week will mark my 10 year anniversary at the ADS. I originally started on a one-year contract to oversee the archiving of key digital outputs produced by English Heritage ALSF projects (with the job title of ALSF Curatorial Officer), but have since stayed on in the role of Digital Archivist, more recently taking over responsibility as the ADS’ Preservation Lead.

The realisation that I’d spent a decade in one organisation initially triggered a Proustian flashback of projects, archives and even files I’d worked on, and thus the idea of a blog was born. I was tempted to call this blog something like “In Search of Lost Time” (time being a portmanteau of my first name and initial of my surname), but was perhaps a little floral as well as erroneous: here at the ADS we never lose anything…

Curious as to what I’d achieved over this period (apart from a sense of satisfaction in safeguarding humanity’s digital heritage), I returned to the ADS Collections Management System (CMS) to query what it was I had worked on. In short, I’ve been responsible for

  • 1018 accessions (the act of receiving and ingesting data from a depositor)
  • Arriving on:
    • 1 x 3.5 inch floppy disc
    • 298 x CD-ROMs
    • 46 x DVDs
    • 208 x Emails
    • 337 x FTP downloads
    • 12 x HTTP downloads
    • 87 USB hard drives
    • 30 USB memory sticks
    • 5523 Web uploads (via OASIS)
  • Archiving 377 collections
  • Updating/adding to a further 169 collections (Journals, collections of OASIS reports etc)
  • Curated 323,050 accessioned files (in 800,000+ files on our AIPs and DIPs)
  • Undertaken 4094 processes (e.g. migrations)
    • Of which 966 processes related to the creation of Preservation PDF/A (12,592 files if you’re curious)
  • Drunk at least 11300 cups of tea (a slightly spurious figure based on an average of 5 cups a day x (10x(annual working days – holiday)).

Over that time, and all those cups of tea, there are definitely some projects that stick in my mind as being memorable. So, to commemorate my decade in data, here  are my top 10 covering every year I’ve been at the ADS:

2006: Wearmouth and Jarrow Monastic sites. Volume 2 Appendix C

My first archive! Notable for using Tab delimited text, which was soon to be replaced as a dissemination format by Comma separated values.

2007: West Stow, Lackford Bridge, Suffolk

One of the first sizeable projects to come through as part of my ALSF work, this was instrumental in building up a strong start to the project. It’s also a useful dataset arising from a modern appraisal of an old rescue excavation.

2008: Land south-west of Ripple, Worcestershire

Although tempted to opt for Gwithian (check out the photos!), I went for this project which was completed in 2008. It’s a nice mixture of reports, data and photos from (to my mind) quite an important site, especially if you’re interested in the dating of pit alignments.

2009: Fieldwalking the cropmark landscape on the Sherwood Sandstone of Nottinghamshire

The first of a series of big projects I started to work on incorporating map and/or database interfaces. This one was built in ArcGIS Server.

2010: The evolution of Rome’s maritime facade: archaeology and geomorphology at Castelporziano

Primarily because I worked on the fieldwork project (look carefully for pictures of a youthful Tim), but also as it was at the time, the largest archive we held. A detailed archive for a very interesting site.

2011: The Deanery, Chapel Road, Southampton (OASIS ID wessexar1-92410)

Although at first appearance this is a somewhat modest archive, it represents a great leap forward. This was the first archive from an agreement between ADS and Southampton Arts and Heritage, whereby digital archives arising from development-led work in the City of Southampton would be passed onto the ADS. We now have several agreements with Local Authorities to perform this role (for example see Worcestershire), and it all started here. As an aside, I often use this archive as an example to show to students as it comprises a compact, well-documented dataset including reports, images and a plan – essential material for anyone working in/researching the city.

2012: A Long Way from Home: Diaspora Communities in Roman Britain

A great example of the archiving of an important research dataset, although I’m also swayed by the similarity of the man in the image on the introduction page and the ex-Everton manager David Moyes.

2013: Quarry Farm, Ingleby Barwick

The site is the most northerly known Roman villa surviving in the Empire, and the dataset is a useful companion to the published CBA Research Report.

2014: Palaeolithic and Mesolithic Lithic Artefact (PaMELA) database

The PaMELA database consists of two main parts: a literal digital transcription of Jacobi’s card index (the Jacobi Archive); and a searchable database with typological and chronological keys (the Colonisation of Britain database). I could spend hours browsing this archive!

2015: The Prehistoric Stones of Greece: A Resource from Field Survey

I expected to put down the Roman Rural Settlement of Britain project, but I won’t consider that finished until the final interface (with access to all data) is finished later this year. So I’ve gone for this project, a rescue of a dataset that had been available on another website, but subsequently removed. The interface has a strong spatial element, and after some thought I moved away from Google Maps and ESRI products (such as ArcGIS Server) to embrace OpenLayers. In the end the hard-learnt lessons (e.g. how to close a polygon?) reaped dividends in my work on the large map for the Roman project.

2016: Birmingham Archaeology (BUFAU) Digital Archives

Before working for the ADS, I’d spent most of my professional life working for Birmingham Archaeology (previously known as BUFAU). That organisation closed in 2012, and subsequently a project undertaken to ensure that all key physical and digital materials are transferred to a suitable archive. We’re only halfway through the project, but already we have the majority of the c.2000 reports written over the years, and a selection of digital materials. It’s been good to go back to where I started, and even to archive some of my own (not very good!) reports!

I’ll end the blog there, who knows, I may update this in another 10 years!

Tim

The (DOC)X-files

The following blog is simply a musing on our historic approaches to archiving formatted text files, prompted by a user enquiry into “best formats” for preservation of their reports, and my role at the ADS as keeping abreast of said formats and our internal policies.

Many years ago, in a meeting of the curatorial and technical team (CATTS), conversation veered towards our procedures for handling text documents. That is files whose significant properties were formatted text/typeset reports, as opposed to plain text files (with ascii or UTF-8 encoding) often used for exporting or importing of data. One colleague, half in jest, commented that as the Archaeology Data Service our focus should be on the literal data as understood in computer science – the individual pieces of information being generated from various instruments or collected in databases. Reports it may be argued are the interpretation of that data, but often not the raw data itself.

Continue reading The (DOC)X-files

DAI IANUS visits the ADS!

ADS was pleased to recently be the host to three Data Curators from a project called IANUS  as part of the ARIADNE project.  ADS spent two weeks immersing Martina, Anne and Philip in the day-to-day duties of a fully established repository. Here is what they had to say about their visit.

DAI IANUS visits the ADS!
By Martina Trognitz, Anne Sieverling & Philipp Gerth

From the 23rd of November until 4th of December, York had three more German inhabitants: us (Anne, Martina and Philipp)! We came all the way from Berlin to learn from the ADS.

IANUS Logo

In Berlin we work at the German Archaeological Institute (DAI) in a project called IANUS. It is funded by the German Research Foundation (DFG) and a first three year phase is now being followed by a second, which just started in March 2015. The aim of the project is to build up a digital archive for archaeology and related sciences in Germany.
Continue reading DAI IANUS visits the ADS!

ADS a Recommended Repository for Nature Publishing Group

ADS are very pleased to announce that we are now an officially recommended repository for Nature Publishing Group’s open access data journal Scientific Data. ADS joins approximately 80 other data repositories, representing research data from across the entire scientific spectrum. ADS has been approved by Scientific Data as providing stable archiving and long-term preservation of archaeology data.

SciData_new_logo_22

Scientific Data offers a new article type, the ‘Data Descriptor’, which has been specifically designed to publish peer-reviewed research data in an accessible way, so as to facilitate its interpretation and reuse. Publishing Data Descriptors enables data produces and curators to gain appropriate credit for their work, whilst also promoting reproducible research.  The main goals of this journal are tightly aligned with that of ADS, focusing on making the data publicly accessible and encouraging re-use.

data descriptor

By becoming a recommended repository for Scientific Data, we are now not only a recommended repository for archaeological data accompanying articles published by the Nature Publishing group but researchers now have the opportunity to deposit archaeological data to ADS, whilst submitting an Data Descriptor to Scientific Data.

All depositors depositing with ADS and intending to publish in Scientific Data or another Nature Publishing Group journal must choose to disseminate the data they are depositing with us under a CC-BY liecence. For more information contact the ADS at help@archaeologydataservice.ac.uk

 

 

 

Archiving Ipswich

Re-posted from Day of Archaeology

Two years after posting about my work on the Silbury Hill digital archive, in ‘AN ADS DAY OF ARCHAEOLOGY’, and I’m still busy working as a Digital Archivist with the ADS!

For the past few months, I have been working on the Ipswich Backlog Excavation Archive, deposited by Suffolk County Council, which covers 34 sites, excavated between 1974 and 1990.

Ipswich2

To give a quick summary of the work so far, the data first needed to be accessioned into our systems which involved all of the usual checks for viruses, removing spaces from file names, sorting the data into 34 separate collections and sifting out duplicates etc.  The archive packages were then created which involved migrating the files to their preservation and dissemination formats and creating file-level metadata using DROID.  The different representations of the files were linked together using object ids in our database and all of the archiving processes were documented before the coverage and location metadata were added to the individual site collections.

Though time consuming, due to the quantity of data, this process was fairly simple as most of the file names were created consistently and contained the site code.  Those that didn’t have descriptive file names could be found in the site database and sorted according to the information there.

The next job was to create the interfaces; again, this was fairly simple for the individual sites as they were made using a template which retrieves the relevant information from our database allowing the pages to be consistent and easily updateable.

The Ipswich Backlog Excavation Archive called for a more innovative approach, however, in order to allow the users greater flexibility with regards to searching, so the depositors requested a map interface as well as a way to query information from their core database.  The map interface was the most complex part of the process and involved a steep learning curve for me as it involved applications, software and code that I had not previously used such as JavaScript, OpenLayers, GeoServer and QGIS.  The resulting map allows the user to view the features excavated on the 34 sites and retrieve information such as feature type and period as well as linking through to the project archive for that site.

OpenLayers map of Ipswich excavation sites.

So, as to what I’m up to today…

The next, and final step, is to create the page that queries the database.  For the past couple of weeks I have been sorting the data from the core database into a form that will fit into the ADS object tables, cleaning and consolidating period, monument and subject terms and, where possible, matching them to recognised thesauri such as the English Heritage Monument Type Thesaurus.

Today will be a continuation of that process and hopefully, by the end of the day, all of the information required by the query pages will be added to our database tables so that I can begin to build that part of the interface next week.  If all goes to plan, the user should be able to view specific files based on searches by period, monument/feature type, find type, context, site location etc. with more specialist information, such as pottery identification, being available directly from the core database tables which will be available for download in their entirety.  Fingers crossed that it does all go to plan!

So, that’s my Day of Archaeology 2015, keep a look out for ADS announcements regarding the release of the Ipswich Backlog Excavation Archive sometime over the next few weeks and check out the posts from my ADS colleagues Jo Gilham and Georgie Field!

UPDATE: Ipswich Excavation Archive has now been released! All sites can be explored here!

New Guidelines for ADS Depositors

data_repositoryThe ADS, supported by funding from the Archives and Records Association, has recently revamped our Guidelines for Depositors.

The revamp reviewed the current ADS guidelines on digital archive deposition and developed updated guidance policies for depositors in light of the recent revisions to the Guides to Good Practice and the development of ADS-easy.

The revision to the ADS Guidelines for Depositors has produced a new user friendly interface designed after detailed consultation with users on the most intuitive and instructive way to present the guidelines.

Continue reading New Guidelines for ADS Depositors