Work in progress: an update on the new OASIS system

Working shot of Ralegh Radford and Peter Poyntz-Wright, Glastonbury Abbey 1962. Image from the digital archive: Glastonbury Abbey: Archaeological Excavations 1904 – 1979; https://doi.org/10.5284/1022585

In the three months since the last technical report on HERALD Stage 2 (i.e. the build of the new OASIS system), the development team here at the ADS have been working hard. The new database has been built; in a departure from our usual database here at the ADS we’ve decided to use PostgreSQL as our database management system. A whole other blog could (and probably will) be written on why we’ve made this decision, but suffice to say it offers -for our needs – a greater flexibility in terms of datatypes, scale and indexing. Furthermore, via the PostGIS extension it also allows us to store geospatial datatypes, and makes it easy to do many of the location based activities we’ll need to record places simply and efficiently (and a whole lot more besides, but I will save that for another blog).

We’ve also – via consultation with ALGAO members – established a criteria and plan for moving records from the old OASIS database to the new system to ensure that no useful information gets left behind. Part of this work package has also been to map ‘bespoke’ terms used in the old/current model to appropriate UK heritage vocabularies, and in this we’ve received valuable assistance from the FISH Terminology Working Group. This last piece of work has highlighted the need for small revisions to existing lists, and indeed for new lists altogether (e.g. archive component). Over the next couple of months we’ll be finalising these edits and tweaks with FISH, HE and HES, and get ready for a pilot migration of data in the Autumn. As soon as this in finished I’ll report back here, as the level of semantic interoperability built into the new system (including LOD concepts from the BGS and OS) is quite exciting, if you enjoy that sort of thing.

Aside from this, Jo has been hard at work building the main application framework (or more simply ‘the new OASIS form’). Having written the Functional Specification and built the old system (back in the early 2000’s), Jo is probably better placed than most to implement the new build! At the moment she’s been working on arguably the hardest part in the logon and registration module…

The first glimpse of the new system: a snaphsot of the working registration module on Jo’s PC

The new system is – hopefully – a significant technical and conceptual upgrade on what we have in the older system. For starters, users will be able to log on as themselves rather than via an organisational account. Users will be able to be associated with more than one organisation to reflect different roles people may have in their work/research capacity, and be able to create a new organisation if theirs is not in the system. Extra steps are built in to ensure as little duplication as possible, including use of other ADS user groups to equate a user/organisation in that system with the same entities in OASIS.

At a slightly higher level, an organisational admin will be able to oversee their group’s members (add and remove), and have the ability to add details about their organisation. The latter will have capacity for Level 2 + 3 users (such as HERs and Museums/Archives) to add reference details on how their organisation is using the system, with space for specific guidance notes for where necessary.

Overall, we’re aiming for a simple yet flexible , quick, and well-explained (i.e. on screen guidance) workflow that reflects the feedback we’ve received on how the majority of users want to use the system. We’ll be showing this and other developments as part of timetabled project milestones to the project funders in the next month, and I’ll be reporting back on other significant developments as often as I can.

Please remember that that the timetable for HERALD Stage 2 is available on this wiki. If you’re interested in becoming a tester for the alpha build then please do get in touch with me via the project email: herald@ads.ac.uk

Tim

 

Oasis Training of LinkedIn contacts

As training officer for Oasis in England I joined LinkedIn in 2012 & it very quickly became apparent that many potential contacts worked for organisations not then registered on Oasis. Once contacts had been made on LinkedIn I sent a standard email offering free training sessions in Oasis, giving more background to the Online Form & potential lodging of reports in the ADS Library. Many replies were received & training visits made. One memorable trip was made to Long Buckby in Northamptonshire to register Iain Soden Heritage Services on Oasis. Another great advantage of LinkedIn was found to be the notable number of scientific and architectural contacts made. This initiative is ongoing & in the last week some new invites have been sent out. A new discussion group ‘Oasis Forum‘ was also set up on LinkedIn this year.

Developing the OASIS Form for Northern Ireland

Jenny O’Brien + Louisa Matthews April 2018

In late 2017 we were incredibly pleased to hear that the Department of Communities, Historic Environment Division, Northern Ireland, were ready to implement a long-standing ambition to join-up to OASIS. It was felt that OASIS could help realise several goals set as part of their wider look at the management of archaeological archives and licencing of excavations in Northern Ireland. These goals included adding more structure and consistency to data generated during the course of investigations, and integrating this information into existing systems (and obviously, the dissemination of grey literature!).

At the start of 2018 the ground work had been prepared and Jenny embarked on coding a Northern Ireland instance of OASIS. Here’s how she got on…


Adding the Northern Ireland-specific forms to OASIS was a fairly straightforward process as similar had already been created for Scotland and Wales.  The linear three-stage process, by which the information was passed from contractor, to local government validation, to national level validation, was also kept the same.

Of the five main areas of the form (Project Details, Project Location, Project Creators, Project Archives and Project Bibliography), three required alterations to the way the form appeared or behaved: Project Details, Project Location, and Project Archives.

Project Details

The form of the project details page remained the same as for the other countries; it was the word lists that needed changing for this section. For most of the controlled lists, such as the ‘Type of project’ or ‘Site status’, the lists were generated by amending current database tables to indicate the terms that were applicable to Northern Ireland. The main alteration was to the Period table, where a new table had to be created that had the period-date relationships for Northern Ireland as well as the slightly different period-date relationships for the other countries.

Project Location

The Project Location section was, as expected, the area of the form that needed the most work.  The Northern Irish location options differed from the other countries in that there was a single ‘Local Council’ level location, rather than the ‘County’, ‘District’, ‘Parish’ sub-divisions seen in the English version of the form.  A new table was added to the underlying database to hold the location information including a second level to differentiate between ‘Terrestrial’ or ‘Maritime’ projects.  In this case, ‘Maritime’ being treated just as another location sub-division, rather than being used to inform multiple Historic Environment Records (HERs), as the Northern Ireland projects would all be validated by the Historic Environment Division, regardless of the area type or Local Council.  The option to select the main and additional HERs was therefore removed, with the validating body set at Historic Environment Division for all Northern Irish records.

The ‘Site Coordinates’ aspect of the form also needed to be altered substantially from the other countries, being the only country that does not fall under the OSGB coordinate system.  Rather than having multiple options to allow users to enter Lat/Long and/or OSGB coordinates here, the default was set to Latitude and Longitude.  The functionality which automatically converts the coordinates from OSGB to Lat/Long upon saving was also completely removed in the Northern Irish forms.

Project Archives

This section required the smallest change, with the field heading ‘Archive recipient’ being changed to ‘Archive location’ in all headings and drop-down list defaults.

Finishing touches

Once the main areas of the form were completed and functioning correctly, the rest of the form needed to be checked for country-specific references, right down to the help text, such as that used for the bibliographic information, these were changed to incorporate Northern Irish examples.

The Northern Ireland additions have now been transferred to the live version of the form and are ready for testing.

Time for an update: OASIS redevelopment

Tim Evans March 2018

It’s been nearly a year since this blog was last updated, and I’ve become aware that many people are interested in what’s happened over that period, and indeed what is happening with the HERALD project overall.

For those blissfully unaware, HERALD is the overarching title for the project that is actively redeveloping the OASIS system. For background, please do have a look at the revitalised HERALD wiki pages.

Chainless Cycle Co. Jorck’s Passage. Cycler uden Kæde | Fischer, Paul ; Vilh. Søborgs Eftfs. Etabl. Image from Europeana. CC0 1.0 Universal

Well, a great deal has happened! In the first instance there have been significant staffing changes and reshuffles here at the ADS. Over the winter, one of our Developers (Lei Xia) moved on to a fantastic job with  University of York IT Services. Lei was one of the key brains behind the new ADS Library, and his last few months were spent finishing the final tasks and processes for that system. With Lei moving on, we’ve looked around for a new Java Developer to take over as a designated ‘OASIS Developer’ to work with our existing Development team. It turns our that despite casting a wide net, the answer was very close to home and in February Jo Gilham (previously the  HERALD Project Manager) successfully interviewed for the post.

Those reading this will probably be aware of Jo through the scoping work she did as part of  Stage 1, and it was Jo that wrote the final Functional Specification for the new OASIS system that was signed off by HE and HES in July 2017. Not many people know this, but Jo comes from a technical background and in recent years has begun programming in Java, so she’s ideally based to take on  this challenge.

Despite this quick turnover, losing an experienced member of staff has not been ideal and has led to a delay in the expected delivery of the project. In truth, this delay has also been exacerbated by the need to finish work on the previously mentioned ADS Library, which has proved to be an extremely large and complex piece of work that has consistently thrown up problems that could not have been predicted at the outset. Significant delay on HERALD has thus been inevitable.

With support from Historic England, we’ve thus gone back to the original plan, identified which pieces of work may require additional resources (one of the positives of the Library was that we learnt alot more about the complexities of particular tasks), and come up with a new time frame for the OASIS redevelopment. I’ve created a project timeline on the HERALD wiki pages for easy reference, but to summarise we are now looking at completion of the Beta format of new OASIS in March 2020. Although that seems a long way off, in terms of development time this is still relatively short, so for the new/old HERALD team it’s full steam ahead.

On a final note, one of the issues that was raised recently was the need for increased communications regarding HERALD, especially in the build up to the public release of the Beta after March 2020. We’ve come up with a broad strategy of ways to accomplish this, of which you’ll see a great deal of in the immediate future, including:

  • the HERALD wiki pages and this blog will be kept up to date, and act as a constant guide to what’s happening and a reference resource with accessible reports and other publicity materials
  • you’ll also see me (Tim) out and about as much as possible at conferences, workshops and meetings
  • production of a great deal of paper-based literature about HERALD
  • new electronic publicity material which can be reproduced by non-ADS individuals if required
  • a greater presence on ADS and partner social media

I hope this does not seem like an overkill, or a saturation of the landscape! The desired effect is instead to try and make as many people aware of how the redevelopment is going, how they can participate and feedback  on testing and establishing training needs.

The redevelopment of OASIS is a once-in-a-generation opportunity to create an excellent resource that plays a core part of Historic England’s HIAS, and a system that will work optimally for everyone involved. The communications side of this work will focus on making the existing community aware of this, and how their specific concerns and issues with the old system are being addressed. It’s also an opportunity to engage those who previously have not used OASIS, or have thought that OASIS is not suitable for their work.

So please, do keep up to date with this blog, and related social media channels such as

  • https://twitter.com/oasis_data
  • https://twitter.com/ADS_Update

For those of you who prefer face-to-face chats, please have a look at the upcoming events to see where members of the HERALD team will be in attendance. If you have any questions about the project or require any information that is not available on the main HERALD wiki then please email the team.

 

 

 

 

 

 

New date for ScARF Workshop: OASIS & HERALD – 26th June

Most archaeologists working in Scotland (and the wider UK) will know about OASIS. OASIS is currently undergoing redevelopment and part of that process involves trying to realise a long held ambition to better integrate museums into the process.

As part of ScARF’s Museums Project, ScARF will be hosting a small workshop to try and bring together those working in Scotland with OASIS (especially those in a *museums/archives* and archaeology setting) to learn more about OASIS and plans for its development.

The workshop will now be held on the 26th June 2017 from 11.30am in the conference room of the National Museums Scotland. Attendance is free and will be on a first come, first served basis as places are limited.

Jo Gilham (HERALD Project Manager) at the Archaeology Data Service and Peter McKeague (Spatial Information Manager, Historic Environment Scotland) will be on hand to answer questions and to let those who will be (or could be) using HERALD know more about how the redevelopment might affect their work. Paula Milburn (Discovery and Excavation Scotland editor) will also be on hand to answer questions and provide a Scottish perspective.

If you work with OASIS at the moment or are involved with the deposition of archaeological archives in Scotland and want to know more, then this is the workshop for you! We especially encourage those working in museums and archaeology who might use OASIS to come along.

If you wish to apply for a place at this workshop, or to find out more, then please email emma@socantscot.org as soon as possible.

ScARF Workshop: OASIS & HERALD

Most archaeologists working in Scotland (and the wider UK) will know about OASIS. OASIS is currently undergoing redevelopment and part of that process involves trying to realise a long held ambition to better integrate museums into the process.

As part of ScARF’s Museums Project, ScARF will be hosting a small workshop to try and bring together those working in Scotland with OASIS (especially those in a *museums* and archaeology setting) to learn more about OASIS and plans for its development.

The workshop will be held in the conference room of the National Museums Scotland on the 12th May 2017 from 11am. Attendance is free and will be on a first come, first served basis as places are limited.

Jo Gilham (HERALD Project Manager) at the Archaeology Data Service and Peter McKeague (Spatial Information Manager, Historic Environment Scotland) will be on hand to answer questions and to let those who will be (or could be) using HERALD know more about how the redevelopment might affect their work. Paula Milburn (Discovery and Excavation Scotland editor) will also be on hand to answer questions and provide a Scottish perspective.

If you work with OASIS at the moment or are involved with the deposition of archaeological archives in Scotland and want to know more, then this is the workshop for you! We especially encourage those working in museums and archaeology who might use OASIS to come along.

If you wish to apply for a place at this workshop, or to find out more, then please email emma@socantscot.org as soon as possible.

Jigsaw Days on OASIS

On the 6th May 2015 and again on 7th March 2016 group training days in OASIS were provided for archaeological societies working within Cambridgeshire as part of the HLF funded Jigsaw initiative.

Attending the 2015 session were representatives from

  • the Covington History Group
  • the Cambridge Archaeology Field Group
  • the Warboys Archaeology Project
  • the Fen Edge Archaeology Group

Together they disproved the belief that local societies are not up to date with the online side of archaeology as they had with them the software tools needed to overcome problems with the training room passwords at Shire Hall!

In 2016 the attendees were from:

  • Access Cambridge Archaeology
  • the Cambridge Archaeology Field Group
  • the Fen Edge Archaeology Group
  • the Sawtry Society

 

Both sessions were really enjoyable thanks largely to the fact that several diverse groups all got on together & shared questions and solutions springing from their first Oasis Forms. Recently the FEAG form for the Twenty Pence Project Excavations 2011–2015 in Cottenham was signed off & completed and is a fine reminder of the value of the two Jigsaw Days.

The report from Fen Edge Archaeology Group

It is still hoped that the Cambridgeshire Jigsaw experience can be spread to other counties.

Although not part of the Jigsaw initiative, the Community Archaeology reports produced by Access Cambridge Archaeology have started to appear in the ADS Library and demonstrate the growing number of Community projects being entered onto Oasis.

 

Autumn OASIS & BIAB management board

The OASIS management board meets twice a year and has met now 28 times. Many of these meetings have been comfortingly routine but over the last few occasions the HERALD project: redeveloping OASIS and creating the ADS Library has meant there is more than just the general use of OASIS across the country to discuss.

The board has representatives from ADS, ALGAO England and Scotland, Archaeology Scotland, Council for British Archaeology, Historic England, Historic Environment Scotland, IHBC, RCAHMW and the Society of Museum Archaeology.

The number and status of records in OASIS from England
The number and status of records in OASIS from England

The general format of the meeting is that we receive monitoring reports from Mark Barratt on the use of OASIS in England, Peter McKeague for Scotland and from ADS for technical helpdesk trends and this time there was also a report from Tim Evans, ADS on the use of grey literature as highlighted in the Roman Rural Settlement Project.

There was a update on the transfer of BIAB and the ADS Grey literature library and Journal archives to the ADS Library with the launch intended for early December.

The update on OASIS was followed by discussions focusing on a few aspects of how the new system would function:

  • How current unsigned-off records in OASIS would be dealt with on transfer to the new system, there are many thousands of these records. There will be a plan for transferring these records and they will be treated differently depending on how complete the records are and if the organisation which started them is still in business. Essentially more complete records should find their way into the ADS Library and very sparse or incomplete records will be moth-balled so as not to clog up the new system but will be retrievable if required. The specifics of this plan will be approved by the management board members and specifically ALGAO at a future date to cover any implications of releasing legacy material.
  • A related but different point was how long reports should be held by HERs before they will automatically transfer to the ADS Library. This mechanism is proposed to remedy the current problem of reports being stuck in the system where HERs do not have the resource to validate reports. One month was seen to be too short for HERs but six months was too long for contractors to wait to see their reports in the Library. Three months has been proposed as a middle ground and this is going to be compared to the average time it takes a report to pass through OASIS currently and discussed with ALGAO for approval.
  • There was also a discussion on how to deal with records where a company has ceased trading. A procedure will be formalised and will now be written in to the terms and conditions of use for the new system.
  • There is a need to improve awareness of OASIS with Development Control Archaeologists – if they receive and sign off reports using OASIS there would be fewer issues of the wrong report being released into the Library.
  • Related to this again was a the discussion of the automatic release of reports into the library could mean that draft reports will find their way into the Library. This was at first seen as a problem, however as discussions progressed it was noted that it is now normal practise in academic circles to release pre-publication versions of an article into repositories as a requirement of funding bodies. These receive a DOI and are then seen alongside the published version once that becomes available.

The next stage of the HERALD project after the release of the ADS Library is the production of functional specification in January next year. This follows a suite of final consultations, the last couple of which will be appearing on this blog soon.

The next OASIS and BIAB management board meeting will be held in April 2017.

OASIS and HERs synchronisation and simplicity

We are still gathering the final opinions from the communities using OASIS before the final functional specification for the system is produced at the end of this year. Last week I attended a wonderfully productive workshop – OASIS and HERs – synchronisation interface meeting organised by Historic England.

There were two main points to be discussed: Synchronising data between HERs and OASIS and if the Heritage Gateway Web mapping and feature services could be re-purposed to provide monument information to accompany OASIS records.

Sarah Poppy (HIAS Programme Manager) began the day by giving an introduction to the Heritage Information Access Strategy (HIAS). The redevelopment of OASIS and the ADS Library forms the basis for the the Collecting and Validating Data work package of HIAS. This was followed by me describing the main changes in the new OASIS (a detailed description can be found in the  project design).

wms mockup
A simplified mock up of how Monument concordance might work

Continue reading OASIS and HERs synchronisation and simplicity

OASIS calling Museum Archaeologists and Post-Excavation Professionals

This is the third of these consultation blog posts, this time we need to talk about…

ARCHIVES!

National Archives and Records Administration, Washington, D.C.OASIS has always had an archive section – it is one of the five main sections of the OASIS form: Project details, location, creators, ARCHIVE and bibliography. In the redeveloped version of OASIS we want archive information from OASIS to go to HERs as usual, but also to museum and archive curators, and allow archive depositors to connect directly with the organisation receiving the archive. Continue reading OASIS calling Museum Archaeologists and Post-Excavation Professionals