10. Strategies for Digital Data
This chapter focuses on the implications of the survey results with regard to the development of national policy. The results from the survey are placed in a wider context of developments in archaeology and the humanities in the creation, preservation and provision of access to digital and cultural heritage resources.
10.2 Programmes and Resources for Digital Data
IntroductionStrategies for Digital Data indicates that archaeologists in general want access to a wide and varied range of information. Use of computers is widespread in some areas of archaeology, in particular for surveying and mapping and in post-excavation work, which often includes the creation of computerised databases of finds information. These represent important sets of information that are primarily created digitally. In addition, datasets created in other media are sometimes digitised - e.g. context information, plans, images, reports, catalogues and indices. Strategies for Digital Data highlights the need for an easily accessible, ideally on-line, catalogue of information resources for Archaeology. Bringing this about would greatly facilitate research, teaching, and increased use of archives in general. There are ongoing and planned projects that focus on digitising important datasets and archaeologists can key into these.
The wide-ranging information needs of archaeologists, and others that use archaeological information, will not be solely met by our discipline. Many related initiatives are underway in both the private and public sectors, and it is vital that major bodies and institutions within archaeology ensure that the discipline is in position to key into relevant national and international projects. A key enabling mechanism for such co-ordination exists with the establishment of the Historic Environment Information Resources Network (HEIRNET) under the auspices of the Council for British Archaeology. This network, composed of organisations throughout Britain, aims to promote archaeological information resources and minimise duplication of effort. The first action of HEIRNET has been to commission a map of existing information resources relating to the historic environment in order to inform the development of a national strategy.
Such a map of historic environment information resources will be extremely valuable. A few of the components of that map are presented in the remainder of this section to provide readers with a sense of the diverse and rich nature of ongoing related initiatives.
Digitisation InitiativesWithin archaeology many important digitisation initiatives are now underway. In England, almost all SMRs have a publicly accessible computerised element (and the RCHME's survey will shed much light on the current state of SMRs). Unlocking the Past (RCHME, ALGAO and EH 1998) sets out an agenda to link together the SMRs of England, and ideally beyond its national boundaries. This is the subject of an HLF bid, and will include provision to ensure that SMRs meet minimum national and international standards, and that staff have sufficient training to manage this resource. The Welsh Trusts co-ordinate SMRs in their regions, and plans are to expand on and improve these and the END (Extended National Database) for Wales.
In Scotland, the Accessing Scotland's Past project has placed part of the NMR on-line -as CANMORE-Web (Computer Applications for National Monuments Record Enquiries, accessible at: http://www.rcahms.gov.uk/). This is part of the Scottish Cultural Resources Access Network (SCRAN), funded by the Millennium Commission. SCRAN works to collect digitised resources relating to Scotland's culture and cultural heritage and packages these for educational access and use.
Plans are being developed to provide an on-line catalogue of cultural resources for Northern Ireland. This will include a wide range of resources. Discussions are underway between the Environment and Heritage Service (DoENI), Ulster Museum, and the Queen's University Archaeology Department, to establish the most effective means of bringing this about.
Another important initiative is the Archaeological Survey of Ireland, due for completion in 2002. This will result in an extensive, systematically collated and up-to-date record of monuments in Ireland. A later phase of the survey plans to place this information in a GIS.
The Museums and Galleries Commission have established a post to advise on information and communications technology in the museums sector and undertake a variety of projects, including Cornucopia, which will provide a central database of MGC-registered museum collections held in the UK.
Distance Learning Projects - many universities in Britain and elsewhere are concentrating on developing degree programmes that do not require students to be physically situated at any learning institution. Archaeology and related topics (e.g. vernacular architecture, Celtic studies) are popular choices for distance learning projects, and a large amount of educational information is thus being generated. The University of the Highlands and Islands is noteworthy in this area as central services such as a library are not the primary resource for information - users are turning to digital resources and delivery in the first instance.
Beyond archaeology, there are other plans to provide new digitisation initiatives, extend access to digital data, and support users. The Joint Information Systems Committee is developing a Distributed, National Electronic Resource (JISC/CEI 1997) on behalf of tertiary education institutions in Britain. The DNER is essentially an extension of the National Grid for Learning (a project led by BECTA on behalf of primary and secondary institutions in Britain). These projects are creating national pools of digital content (computerised bibliographies, data, images, texts, etc.) and software available over the Internet for educational uses. A distributed model is followed by which datasets may be held in dispersed archives but made available as an integrated whole. The JISC fund digitisation of datasets, where relevant to its collection policy and of potential use in HE teaching, research and information management. Although access to this service is usually limited to those based in UKHEIs, the JISC will extend beyond this, where local and regional collaborative projects are planned (JISC/CEI 1997, 24). There is certainly great potential for such cross-sector collaboration in Archaeology.
Catalogues and IndicesThere are many archaeology subject gateways available on the Internet, but very few of these offer access to detailed reports or archives. ArcHSearch, developed by the ADS, is exceptional in offering on-line access to detailed, digitised project information, as well as catalogues and indices. This is linked to the AHDS catalogue, providing access to digital datasets created by and for professionals in the Humanities. To date, the majority of services available over the Internet do not offer access to detailed project archive information, though plans are underway to bring this about. In addition, on-line catalogues are being developed across the humanities, providing new opportunities for making information on archaeology available.
The National Archives of Ireland provide on-line indices for some of their collections. Currently, some of those for genealogical research and part of the OSi indices of maps are available. An index to records of the Office of Public Works (which holds archaeology archives) is soon to be added. (The National Archives' home page provides more information, accessible at: http://www.nationalarchives.ie/). Site visits are still necessary to access detailed information. Archaeology Ireland has published an on-line searchable database listing 231 excavations undertaken in Northern and Southern Ireland in 1993 (available at: http://www.kerna.ie/archaeology/excavations.html. Only summary information is available (though details vary according to the information provided by consultants who carried out the work).
The need for a National Agency for Resource Discovery is long-standing, and the British Library in collaboration with the JISC outline the scope and content of the NARD (Brophy, Fisher and Hare 1997). The NARD will essentially be a digital catalogue of library and archive holdings in Britain, searchable over the Internet. It will pool together the catalogues of national and local libraries, public libraries, those based in UKHEIs, public and private archives and record offices. Standards and guidelines in resource description will be encouraged. In the first instance, digitisation will focus on catalogues describing collections, and it may be possible for archaeological archives to be included in this exercise.
The National Council on Archives has proposed a United Kingdom Archival Network, which would serve as a national on-line archive of digital resources largely for historians. Although none of the organisations involved in this project specialise in archaeology, the project does have great potential for archaeologists as its remit includes historic maps, plans and documents. In the first instance the aim is to create digital catalogues and indices of information, and then to move onto specialist projects. In particular the NCA recommend creating digital collections of information packaged for education needs (NCA 1998). It represents a good opportunity for archives to digitise their catalogues and indices, and to get hold of the training and facilities to make these available over the Internet to their employees and users. The Archival Network will initially be based at a national level, with links between the countries of Britain, with Ireland, and hopefully beyond.
...the growing tendency of educational users at all levels from primary schools to higher education to regard the Internet as a preferred tool for resource discovery makes it increasingly vital that detailed information about archival holdings should be accessible in this way; a need that will be felt the more strongly as libraries place the Internet at the disposal of every member of the public (NCA 1998, 1)
The Consortium for the Interchange of Museum Information (CIMI) is an international body with representatives from the Americas, Europe, Africa etc. Their CHIO project (practical access to cultural heritage on-line) is providing the means to link together the digital catalogues of museums and archives, and to make these searchable via the Internet (CIMI 1997).
The Aquarelle project, based in Europe, is developing software to enable the distribution of information on cultural heritage from dispersed digital catalogues, and information retrieval over the Internet (Aquarelle 1995). The end-user will be offered a structured, standardised, front-end for searching and obtaining results.
British Council Electronic Information Services Team - A new team under development by the British Council will be responsible for delivering a digital library to BC offices in over 100 countries. The goal is to aid the BC in promoting international educational, cultural, and technical co-operation.
Support for Digital ArchivingThe AHDS offers advice, guidance, and an archiving service for digital datasets in the humanities. Their new Framework for Managing Digital Collections (AHDS 1998) is of use to those creating and managing digital datasets for future re-use. The ADS offers guidance and support for archaeologists in Britain on digital data creation, preservation and re-use working, be that on British projects or those based elsewhere. It supports those creating digital datasets as well as those looking to preserve them for future use. Because of the sheer diversity of computer use in archaeology, liaison with the final archive holder at the start of a project ensures that material is created in ways suitable to the needs of the final archive holder. Although digital datasets have largely fallen outside of this provision, the ADS does offer an archiving service for those datasets that do not have an alternative repository, and issues guidelines for depositors (ADS 1997b).
The National Preservation Office of the British Library is a major player in developing strategies and standards in digital data preservation. It offers support for preservation of library and archive material, in a variety of media. The NPO's remit for digital data includes developing guidelines for digital data archiving and preservation, co-ordinating the development of a national digital preservation policy and implementation of guidelines, strategies for disseminating information digitally and on paper, guidelines for projects exploring digital data archiving, and working in partnership with private and public bodies to help bring these aims about. Further information is available on the British Library's home page (http://www.bl.uk/, under Preservation).
For the Republic of Ireland, the Heritage Council (of the Department of Arts, Culture and the Gaeltacht) provides guidance in archaeology. Their support of Strategies for Digital Data shows their concerns regarding digital data and new working practices in archaeology.
Another key player in digital data preservation for archaeology is the Archaeological Data Archive Project, based in the Center for the Study of Architecture, Bryn Mawr, USA. The ADAP takes in digital datasets, provides guidelines for depositors and users, and disseminates information via the Internet and on magnetic media (Eiteljorg 1998).
Training and User-SupportFor Britain, the New Library aims to make information and communication networks accessible to all, and will provide support and training for service providers and users (Library and Information Commission 1997, 1.4). It is an initiative to integrate public libraries, galleries, museums and archives via the Internet. The New Library is similar to EARL (Electronic Access to Resources in Libraries) established in 1995, though has a much broader remit of support and training. One strand of content in the New Library is Community History and Community Identity - archaeology can be invaluable here. Although the New Library is primarily geared to linking existing digital resources, there are plans to digitise special and rare collections. When the report was published in 1997, it noted the need for a project to identify and prioritise relevant collections for digitisation, based in museums, galleries and national libraries. Funding of £30 million spread over 5 years is recommended to bring this about (Library and Information Commission 1997, 1.55). This represents a good opportunity for archaeology to identify important holdings to be included in the New Library. In addition, provision will be made to train librarians and support will be provided for users.
The Schools IT 2000 initiative in the Republic of Ireland is a government project which promotes the installation and use of computers in homes and schools, and is due for completion in 2001 (Dept of Education and Science 1997). It plans to provide at least one computer linked to the Internet for each school, to train teachers to use these tools, and in addition to provide on-line computers in public libraries. Schools and libraries will liaise with community development organisations and businesses to develop, disseminate and implement models for the support of lifelong learning. Part of the project will include a gateway of on-line resources for teaching and learning.
Programmes and resources for digital data: Recommendations
An on-line catalogue of digital data archive holdings, at least as national (and inter-linked) indices, and accessible from any computer connected to the Internet, would greatly assist research. The creation and archiving of digital data should be done in accordance with appropriate national and international data standards. The Integrated Information Systems underway for the dissemination of properly archived and curated archaeological data should be fully supported (e.g. Extended National Index for Wales, plans for English SMRs/the NMR and Scotland's CANMORE-Web). This necessitates collaboration between SMRs, NMRs, national bodies and the museum community and co-operation within and across national boundaries should be promoted where possible. HEIRNET may provide such a co-ordinative rôle. Digital data created as a result of archaeological research is of relevance to the wider community. Archaeologists in Britain must take advantage of the various new Integrated Information Systems being developed, such as New Library and the UK Archival Network, to ensure that archaeology has a voice in the inter-disciplinary partnerships envisaged between national and local government, libraries, HEIs, schools and public organisations.
10.3 Standards and the creation of digital data
IntroductionHow digital datasets are created affect whether they can be preserved and re-used long-term (Beagrie and Greenstein 1998). Survey results suggest that an ever-increasing number of archaeologists rely on computers in the collection and creation of primary data, and planning for the long-term safety of these data needs to be in place at the start of projects.
This section explores current practice in creating digital datasets. There are many types of standards available for use in archaeology. The ADS holds a peer-reviewed list that is accessible on-line at http://ads.ahds.ac.uk/project/userinfo/standards.html.
There is some concern within the archaeological community regarding the plethora of available standards, how to choose the correct one and the potential danger of their mutual incompatibility. Although the list of potential standards for archaeological datasets is extensive we do not advocate the use of a single monolithic standard for the entire discipline. A better starting point would be with a general standards gateway, such as the ADS list. Each standard on the list is accompanied by a short description, which enables the potential user to assess how well a particular standard fits its required purpose for a specific task. There is also a growing recognition in the heritage sector that organisations providing parallel services can benefit from more standardisation (e.g. SMRs).
Archaeological Excavation and Archaeology Curators - Digital DatasetsWithin Britain, standards are being developed regarding the creation of information in digital form, and the ADS, as part of the AHDS, is a key player (see Guidelines for depositors - ADS 1997b - and cataloguers - ADS 1997a - both are available on-line). The ADS have a web page listing the range of national and international standards that apply to archaeological datasets (ADS 1997c). The ADS/AHDS are producing a series of Guides to Good Practice, which are to be published both in paper and on-line. The first, on GIS, is now available (Gillings and Wise eds. 1998), and the next, on Digital Archives from Excavation and Fieldwork, is expected in early 1999.
The Governmental heritage agencies and Royal Commissions are all developing guidelines for digital datasets and stipulate the use of appropriate standards for projects they fund. In Scotland, Historic Scotland is liaising with the RCAHMS over the creation of a single body of standards for the country. Regarding developer-funded fieldwork, project briefs stipulate standards and guidelines for how work will be undertaken, and what should happen to the project archive (though do not always specify how any digital elements should be created or preserved). Some funding bodies now stipulate, as a condition for funding, that any resulting digital datasets need to be offered for deposit with the ADS (e.g. the British Academy, The Leverhulme Trust, NERC).
Although the ADS is establishing itself as a repository for digital data, other bodies can and are taking in these datasets - though the ADS is unique in offering on-line access to detailed project archives.
Archaeological excavation and archaeology curators - catalogues and indicesA central function of SMRs and NMRs is to serve as a catalogue of past project work, and computerised databases are a vital part. Project details need to be indexed, and the Standing Conference of Archaeological Unit Managers (SCAUM) recommend sending in to SMRs/NMRs a pro forma, providing a basic list of fieldwork results, published information, and the location of finds and documentary archives (SCAUM 1997). More detail is ideal, and the ADS provide guidelines for depositors of digital datasets. Such basic information could easily be submitted in digital form, as some curators are already stipulating (e.g. Lincolnshire County Council 1997).
For successful information retrieval and exchange, it is vital that thesauri and controlled vocabularies are used in this process, and although these should follow the local SMRs practice, ideally they will also conform to national and international (where appropriate) standards. In England, the Thesaurus of Monument Types (RCHME/EH 1995) and MIDAS (RCHME 1998b) are used to promote consistent use of terms within and between indices and catalogues. The International Council of Museums - International Committee for Documentation (ICOM-CIDOC) has developed a core standard for archaeological sites and monuments records (ICOM-CIDOC 1996), to enable communication and information exchange on European cultural heritage. These standards are a vital backbone to plans to integrate computerised indices and catalogues within and between countries. At this point it is worth noting that MIDAS recommends cataloguing digital datasets with the ADS, to encourage greater awareness and use of archive material. Guidelines are available on-line at: http://ads.ahds.ac.uk/project/userinfo/catalogue.html.
There are some concerns within the archaeological community over the effectiveness of thesauri. While it is true that thesauri will never be all-inclusive and that local terms, for example, may often be of greater use than a larger scheme, it must be stressed that thesauri are indexing tools and not classificatory mechanisms.
The Museums SectorCatalogues and inventories form an essential backbone to collections management, and the mda (Museums Documentation Association) plays a key role in developing and implementing standards in the museums sector. The mda have been carrying out surveys of computer use and terminology in recent years. Their 1996 survey of terminology in museums (MDA 1996) had a poor response rate (13%), though identified the use of common thesauri and vocabularies in many museums. Where common standards have been used, there is the potential to link together and search through dispersed datasets. However, there is no single common standard within England. The mda are involved in the European Union-funded Term-IT project, which is exploring the development of multilingual, on-line terminologies and bibliographies for the heritage sector, as an essential element in bringing about the linking together of dispersed on-line catalogues. Information is available on-line at http://www.mda.org.uk/term-it/.
ICOM-CIDOC offer guidance on the development and modification of paper and computerised museum collections records (ICOM-CIDOC 1995). Among some of the international standards in use, they recommend the following: the Art & Architecture Thesaurus (Getty 1994), the Union List of Artist Names (Getty 1995), the CIDOC Directory of Thesauri for Object Names (ICOM-CIDOC 1995), and the Canadian Heritage Information Network document Data Content Standards: A Directory (Harvey and Young 1994).
The Getty Information Institute has now released its Thesaurus of Geographic Names (Getty 1998), and all four of its vocabularies are searchable on-line at: http://www.gii.getty.edu. As with the catalogues held by archaeology curators, controlled vocabulary and terminology is essential to facilitate information retrieval and exchange. It is a vital backbone to ongoing developments to link together computerised catalogues in the heritage sector (explored in section 8.4).
Standards and the creation of digital data: Recommendations
Data creation standards are essential to facilitate the exchange of information. National bodies in Britain and Ireland should continue to collaborate with European and International standards and guideline development programmes (e.g. CIDOC, CIMI). National bodies should continue to encourage the use of standards for projects they fund. Similar guidelines need to be built into project briefs for developer-funded work. A peer-reviewed list of archaeological data standards is held on-line by the ADS at: (http://ads.ahds.ac.uk/project/userinfo/standards.html). As a part of archaeological good practice all archaeologists should make use of appropriate standards in any work they undertake.
10.4 Standards and digital archiving
Several recent surveys have shown that although some organisations are taking in and actively curating digital datasets, the majority do not, cannot, and are not sure who to turn to for advice (Strategies for Digital Data chapter 6; Swain 1998 on English museums; Lievesley and Jones 1998 on UKHEIs). As with any project, early liaison with the final archive holder enables managers to plan work from initial data creation to deposition in the most effective ways. Digital datasets should be considered in this process. Any specifications that digital data archives have regarding hardware and software (including versions) are ideally passed to the data creators to facilitate the smooth transfer of data. The ADS Guides to Good Practice, as well as its documents regarding ADS collections policy and its guidelines for cataloguing and depositing digital data, have a rôle to play in this respect. The collections policy and guidelines for cataloguing and depositing data are available on-line from http://ads.ahds.ac.uk/project/. The Guides to Good Practice are also available on the Internet at http://ads.ahds.ac.uk/project/goodguides/g2gp.html.
Information stored digitally can easily be rendered unreadable - through degradation of the media on which it is stored, through hardware incompatibility and, more crucially, through the rapid pace of software development. Digital datasets therefore require active curation - planned refreshment of the media on which they are stored, careful migration through later versions of software, and clear documentation of the dataset. Without adequate documentation, identifying the platform, operating system, software (and versions) on which the dataset was created, active curation and re-use is either very time- consuming or impossible. Documentation should, of course, include descriptions of any terminology and codings used. Unless the processes through which the dataset has been transformed are listed, it may not be possible to evaluate the relationship between the digital dataset and other elements of the project archive (e.g. are survey data available in 'raw form'?). Active curation of digital datasets is therefore complex and can be costly, though good planning, documentation and clear objectives at the start of a project minimise these costs and facilitate a seamless flow between data creator, curator and user.
National bodies and funding agencies are aware of the necessity of preserving the digital portion of project work alongside other elements of the archive, though legislation and guidelines are still under development. Practice varies greatly in the different countries participating in Strategies for Digital Data. In the first instance, researchers holding archives should look to guidance from their project funding body.
The vast majority of excavation in Britain is developer-funded, and although the IFA provide guidelines on project briefs, there is nevertheless great variation in the specification briefs set for treatment of the archive at the end of the project. Project directors are expected to liaise with the final holder of the archive for details on how the archive should be presented for deposition, and it is therefore the latter that stipulate whether and how digital datasets should be included (and whether an index of archive contents is required). Where archaeological curators and archivists working in the same region can collaborate, there is the greatest potential to obtain and maintain good archiving standards.
While national guidelines are being developed, one of the strongest tools available to archaeologists to control what happens to archives is the project brief. In England, PPG16 and PPG15 cover developer-funded fieldwork (for buildings); PPG16 was implemented in Wales in 1991, and the equivalent legislation for Scotland is NPG5. Northern Ireland and the Republic of Ireland operate licensing schemes for archaeological units. This topic was the focus of a series of telephone interviews, conducted by the ADS in the summer of 1998 (Appendix 3). A range of archaeologists based in field units, local government departments and museums that identified themselves as holding large datasets was targeted, and asked for details of local/regional/national practices in digital data creation, preservation and archiving. The rest of this section uses their responses to explore what standards are being implemented regarding data archiving.
In Scotland, Historic Scotland and the RCAHMS insist that the full project archive is deposited with the NMRS, which takes in reports, paper and other material, including digital records. Contact the NMRS for up-to-date deposit information (email@example.com).
A draft Strategy for Recording and Preserving the Archaeology of Wales, issued by Cadw and RCAHMW, states that these organisations will collaborate with the ADS to ensure best practice regarding preservation and access to digital datasets. However, where project briefs are set by local authority curators, these do not always specify what should be archived, nor where. In practice, large project archives may stay with contracting units for lengthy periods. In addition, the situation regarding work carried out in Wales by contractors based outside of Wales is not clear. These new guidelines from Cadw and the RCAHMW should clarify the situation regarding the whole of the archaeological archive.
For England, MAP2 requirements for site archives state that, where relevant, computer discs, printouts, lists of codes used and details of software involved must accompany the documentary and finds archive (English Heritage 1991, 30). In addition, an index to the archive contents, and microform copy of the documentary archive must be provided for the final archive holder. Although the MGC and mda specify standards for museum-based archivists, there is in practice great variation in the range of material that is deposited with archives (clearly shown in the survey results). Few long-term archives have guidelines for depositing digital datasets, and much digital data does not reach the final archive holder. There are notable exceptions and exemplars of good practice. In some regions, curators specify that digital datasets need to be deposited with archives alongside other material (e.g. Lincolnshire County Council 1997). This situation seems to be dependent on good lines of communication between field units and archiving services, and whether they work together towards high standards in their area.
In the Republic of Ireland and Northern Ireland, a licensing scheme of contractors is used to control which contractors can carry out fieldwork. Invariably, contractors need to liaise with the final holder of the archive at the start of the project, to ensure that the archive is deposited according to the museum's guidelines.
Although the Department of the Environment for Northern Ireland acts as an archive for Northern Ireland, it does not take in digital datasets, nor does it always hold the finds or paper archives from work carried out in the province. In practice, material also remains with contracted units.
There is no central archive for all the elements of archaeological project work in the Republic of Ireland. Although guidelines are available, these do not cover digital data. In practice, material often remains with contracted units. The Archaeological Survey of Ireland, now running, will greatly assist in building a standard corpus of monuments for the country.
Through the AML/EH, guidelines have been issued for the creation, preservation and re-use of geophysical survey datasets. Their recent report (David 1995) states that data creators should retain the raw data so that others can re-use it. Reports need to be lodged with SMRs, at most within 6 months of the end of the project - client confidentiality is protected for only 6 months, following IFA and ACAO/ALGAO guidelines.
Good practice for the transfer of project archives, from those who carried out the work, to the final archive holder, needs to be encouraged in all parts of Britain and Ireland. Liaison between those undertaking the work and the final archive holder at the start of a project can ensure that the final archive can be easily transferred for final deposition. Project briefs can be used to expand the range of information included in project archives. Raising awareness amongst all curators about the importance of obtaining a full project archive is an essential task. It should be stressed that it is the duty of those who create digital data to ensure that it is appropriately archived. This means that digital data must be archived in places where it will be properly curated, even if it is not made available over the Internet.
Standards and digital archiving: Recommendations
Digital datasets are an important element of project archives that require active curation if they are to be preserved for future use. Project briefs should make clear provision for the archiving of any digital data created during the course of the project in an appropriate digital archive. Where archives are dispersed across more than one organisation, pointers to the location of other archive elements are important for maintaining archive integrity. There is a need for a document that details the appropriate standards and facilities for digital archives. Included in this document should also be a list of digital archives that conform to these standards. An accreditation scheme for digital data should be devised.
10.5 Selection of digital information from archives
Identifying what information should be held in digital archives requires long-term planning, in addition to what is currently a rearguard action to preserve information from completed projects that could otherwise be lost. This section explores the rationale behind the selective preservation of information.
Archiving practices in Britain and Ireland vary, though the ideal of preserving all elements of a project archive remain in place, and should hold for information created and held in any medium. In the first instance, priority needs to be given to information that would otherwise be lost - datasets created primarily or in most detail in digital form (as touched on in section 4 above). Secondly, the discipline needs to identify what sorts of information need to be preserved digitally. It is vital that support documentation is available for these resources to enable a user to incorporate them into their work - without such information any datasets are of very limited use.
Strategies for Digital Data showed that archaeologists want access to a broad and diverse body of information, irrespective of the medium on which it is stored (section 7.2). However, further questions about ease of access to datasets in digital and other media resulted in overwhelming support for paper and digital records, and minimal support for microfiche (section 7.5). In particular, tabulated datasets are wanted in digital form (e.g. finds databases, site location information). This suggests that 'raw datasets' such as primary finds analysis, site context information, surveying data, catalogues and indices, where created digitally, should be targeted for inclusion in digital archives. Although much archive information is held on microfiche, and it has proven to be a reliable means of preservation, it is not a popular medium amongst users. The British Library has started to digitise its microfiche records in an attempt to increase their use.
Guidelines for the selection of digital datasets need to be developed for archaeology. Any guidelines need to be developed within the context of local, national and international research strategies, and follow agreed standards for the treatment of archaeology archives. The policy of the ADS is set out in its Collections Policy Document, which is available on-line at: (http://ads.ahds.ac.uk/ project/collpol.html). The ADS aims to prioritise those types of data that have potential re-use value. The ADS also acknowledges that some classes of data, such as CAD and GIS, must be preserved digitally in order to maintain their functionality, while other categories may be disposed of on the basis that it is more cost-effective to re-digitise as the need arises. The ADS notes that it is of considerable benefit to both depositors and users that there be an effective and rigorous process of peer review of materials proposed for accessioning. Data resources which are offered for deposit to the ADS are be evaluated to:
Assess their intellectual content and thus the level of potential interest in their re-use. Evaluate how (even whether) they may viably be managed, preserved, and distributed to potential secondary users. Determine the presence or absence of another suitable archival home.
Selection of digital information for archives: Recommendations
'Raw datasets' such as primary finds analysis, site context information, surveying data, catalogues and indices are vital sources of information for research, and ideally accessed on paper and digitally. Where created digitally, they should be targeted for inclusion in digital archives. The digital archiving of large databases, CAD and GIS files is necessary if these resources are to maintain their functionality. Regional and national proactive strategies need to be developed for the selection of digital datasets that meet the current and future needs of archaeologists. There is a need for the retrospective collection of already created digital data. The current lack of strategy means that archives are reacting in an ad hoc fashion.
10.6 Cost recovery and funding digital archives
BackgroundCreating digital archives, and disseminating the information they hold, are vital to ensure quality research. Preserving digital data in a way that ensures it is secure and usable in the future is relatively labour-intensive, and therefore costly. Research by the AHDS shows that the costs involved in preserving digital data can be broken down into several areas:
Assisting data creators - through publications, training events, consultancy and standards guidelines. Data archiving and administration - identifying datasets for acquisition, accessioning, creating catalogue records and documentation, processing data into standard formats, storing data, preservation (data migration and validation), monitoring reports, designing interfaces for special collections, and brokering for commercial use. Assisting data users - training users and potential users, publications, administering access agreements, costs of distributing data and documentation, user support, advertising available data.
Strategies for Digital Data showed that archaeologists do not, generally, want to have to pay for using information. Nevertheless archiving bodies may need to recover costs of creation and maintenance from a variety of sources, and some of these can be passed to users. This section explores several areas where costs may be recovered, and sets out a variety of funding models used by digital archiving bodies
The value of digital data and cost-recoveryThe information contained in digital datasets can be commercially sensitive and have a commercial value, just like information available in other media. In addition, costs can be offset from project work if datasets can be obtained in digital form, thereby avoiding expensive and time-consuming re-keying (with the possibility of introduced errors).
Where information will be used for consultancy work, be that for planning purposes or otherwise, charges are invariably levied for its use. Although this holds for access to SMR/NMR information, there are other obvious areas where it also applies - in particular to images. Reasons for charging are to cover copyright fees, for staff costs/time, to cover materials. Where information is obtained over the Internet, additional costs are involved: facilities and support to become networked, creating metadata records to enable on-line searching and information retrieval, connection fees, telephone charges. For those based in institutions or large organisations, these costs may not be passed onto the individual, but for private/home users, by the time they have accessed digital datasets on-line they will already have 'paid out of their own pocket'. If we are to encourage greater use of archaeological information, there must be scope for free public access at least to basic information to allow individuals to become actively involved in their heritage.
The academic community has privileged access to digital datasets through the deals negotiated principally by the JISC with hardware, software and information suppliers. UKHE libraries pay licensing fees for electronic resources held on CD-ROM or on-line, and these suppliers can and do stipulate that access be restricted to students and employees of those institutions alone. In addition, charging mechanisms are being discussed for access to digitised, on-line texts. The JISC are moving towards a mechanism that charges per printed page, with a sliding scale according to whether the information is still in print, whether it is published, expected numbers of users, the number of pages printed (Bide et al. 1997). They are reluctant to impose charges for 'browsing', even though providing this facility involves costs.
One area of concern among archaeologists is the strict control the OSGB keeps over the National Topographic Database, and specifically the high cost of annual licenses. The OSGB/DoE undertook a survey of the 'National Interest in Mapping' in 1996 (OS/DoE 1996), though an agreement has still not been made between the OSGB and government regarding the results of this survey. Friends of the Earth are now taking the OSGB to the European Court over the costs set for access to the National Topographic Database, as EU law states that information on a nation's environment should be freely available. As regional and national sites and monuments records either already have or are moving towards a GIS component, any change in access to OSGB mapping data will have a great impact.
Parts of the UKHE sector have privileged access to OSGB datasets. Some of the OSGB's digital maps are available for teaching and research, and can be reproduced in academic journals. However, they cannot be used for paid consultancy work, or for projects or institutions funded by the Research Councils or British Academy. Negotiations are now underway between the OSGB and the JISC, on behalf of the UKHE, to establish a standard licensing fee to use digital maps for research and teaching purposes (JISC 1998). The Ordnance Survey of Ireland state that users need to obtain a license, for which they may need to pay a fee.
NERC insists that digital datasets resulting from projects they have funded, along with a metadata catalogue, be deposited with one of their recommended Data Centres (NERC 1998, 3.3). For science-based archaeology, grant recipients are required to offer data to the ADS for deposit. Although NERC charges for access to information it holds, academic researchers are granted access at no more than direct costs (e.g. of copying information onto disc/CD). For commercial use, NERC passes on requests to the data creators. Access to basic catalogues and indices should be 'free' - at most involving basic delivery costs only. Where value has been added - through re-analysis, bundling and interpretation for example - additional charges can be made.
In practice, a variety of mechanisms are used to recover some costs of delivering digital data to end-users. These range from annual licenses and membership fees paid at an organisational level, to individual levies charged to users based on the amount of information and delivery mechanisms used. In addition, additional costs may be set if information is to be put to commercial use. None of these cost- recovery mechanisms, however, identify certain datasets as having a greater potential to recover costs than others.
Funding models and digital archivesFunding mechanisms for digital archiving bodies can use a mix of core funding, funding attached to individual projects, and cost-recovery from users. The current practice of several archives is outlined here, and recommendations made.
The Data Archive, based at the University of Essex, archives and provides access to digital datasets in the social sciences and humanities. It receives core funding from the JISC and ESRC, and is supported by the University of Essex. End-users are charged for use of data, though sliding scales apply. This charge forms three elements: administrative costs, collection of royalties or other rights, and direct costs of receiving copies of datasets on tape, CD-ROM etc. There may not always be a fee for royalties or other rights. Direct costs are fixed, though administrative costs vary according to where the person seeking to use the information is based, and how they want to use this information. No administrative charges are applied for researchers working on projects funded by the ESRC, DENI, or HEFCs, nor for those whose projects have no funding but are based in institutions funded by the same bodies. Those funded by charitable organisations are charged £100 administrative costs. Users funded by government departments and other non-profit making organisations are charged £200 administrative costs. Users working on commercially funded projects will be invoiced for the full Administrative Charge of £1000 per dataset, and the owners/depositors of the data may also impose their own additional charge (Data Archive Charges and Costs Explained available at: http://dawww.essex.ac.uk/ordering_data/chco.html).
The Archaeology Data Service is developing its own charging strategy in step with that for the AHDS overall. At present the service receives core funding from the JISC. The developing charging policy of the ADS suggests that:
Core funding should cover negotiations with potential depositors, core user services (advertising its holdings, assisting users to locate and download datasets, general enquiries), and should assist with storing and managing archives. Depositors would be charged for accessioning datasets and for any monitoring reports they require, and it is hoped that units and other data creators will be enabled to pass these costs to developers or other funding bodies. Rather than a fixed rate for depositing, sliding scales will apply according to where depositors are based and the nature of their project funding. Overall costs for archiving project material should be less than 5% of the total project costs, and digital archiving will represent a sub-set of these costs. The ADS are negotiating with EH to undertake a pilot study of archiving costs and practices for two completed excavation projects. It is not currently planned to charge users for access to datasets, though they will be expected to pay for value-added services such as workshops, additional documentation and so on.
The National Agency for Resource Discovery, planned and co-ordinated by the British Library Research and Innovation Centre will, in the first instance, contain digital databases of resource location and, secondly, digitised resources of national importance. A survey conducted of potential users concluded that the majority did not support charging end-users. Alternate sources of funding need to be sought, and include charging individual institutions/libraries for access to the NARD, and major funding bodies such as the JISC (Brophy, Fisher and Hare 1997, 5.1.5).
The Archaeological Data Archive Project is based at the Center for the Study of Architecture, Bryn Mawr College (USA). Digital datasets can be obtained via ftp, and some are also available over the Internet. Cost-recovery is only sought from those who want to use data for commercial purposes. The project relies mostly on core funding.
SummaryOverall, there is variation in the ways archiving bodies obtain funding, and how costs are passed to users. In most instances core funding is essential. Where end-users are required to pay a fee, this does not represent the full cost of delivering information and support necessary to use that information. These are vital considerations when exploring ways of maximising the preservation of information and facilitating its re-use in research and teaching. As well as the various services that are already available to specialists, provision can be made through projects like the New Library to provide access for the general public, and charging mechanisms should not be set in a way that can exclude these important users.
Cost recovery and funding digital archives: Recommendations
Access to indices of project archives should be free in the ADS catalogue, to assist individuals in locating project archives, to encourage greater use of archives (digital or other), and to involve the general public in as many ways as possible. It is recognised, however, that additional services such as copying datasets, postage, special tutoring and additional documentation may incur costs and these can be passed onto the user. Digital archiving bodies need some core funding for services such as archive maintenance, user support and training to ensure that charges to users are not prohibitive. Depositors may be charged for archiving datasets. Nevertheless, charges need to take into account how projects were initially funded and the level of public access the depositor allows to their data. Projects allowing high levels of access to their project information would ideally have lower deposit charges.
10.7 Data delivery mechanisms
Users need to be able to locate those datasets useful to their work, and to obtain information in a way most suitable to their needs. Although information may be held digitally, this may be a barrier to its re-use by those who lack suitable equipment and skills to make use of digital datasets (Porter and Greenstein 1997). Obtaining information digitally, however, can save a lot of time, costs and reduce errors where adequate support is available to the end-user. There is no standard method of delivering digital data, as the following discussion shows.
Many digital archives provide searchable, on-line indices of their holdings, on sites accessible to all. Once suitable datasets have been identified, there are several possible mechanisms for delivery. The Data Archive, based at the University of Essex, requires users to fill in an application form and undertaking form (stipulating how data can and cannot be used). A variety of delivery mechanisms are on offer: ftp, on-line datasets (limited), floppy disc, tape and CD-ROM (not as printout). The Oxford Text Archive has a searchable on-line database of its holdings, and some of these can be downloaded from the Internet. However, files may also be sent on floppy disc, tape or CD-ROM (again, not as printout). The UK National Digital Archive of Datasets, based at the University of London's Computer Centre, makes available digital datasets on behalf of the Public Records Office. It has a searchable, on-line database of its holdings (and support for users). Once datasets have been located, the user has to register to obtain detailed information (which involves a cost). Data can be delivered in a variety of ways: ftp, email, over the Internet, on floppy disc, tape, CD-ROM, and also provided as printouts. The ADS is making available on-line its digital holdings. Strategies are still being developed for enabling users to obtain detailed datasets.
These data delivery mechanisms rely on support being available to users both to identify relevant datasets (explored in sections 4 and 7). In addition, some methods of delivery are more costly, which has serious implications for the funding needed to maintain digital archives (covered in section 8).
Data delivery mechanisms: Recommendations
Detailed datasets should be made available in a variety of ways to ensure access for the maximumn number of users
10.8 Training needs and support infrastructure
Training NeedsTraining to bring about greater use of digital archives needs to focus on several areas:
Awareness-raising. Standards and guidelines for digital data creation, preservation and re-use. IT skills. Information retrieval skills.
Many organisations offer these services to archaeologists, though there is some duplication of effort, and some areas of the discipline are poorly served. Sudbury's assessment of the information and training needs to enable museums to make the most of information technology (Sudbury 1997) identified three key players: data creators (digitising collections), museum professionals (using digitised information to compile electronic publications and exhibitions), and teachers. To this, we can add the general public. The ADS ran a workshop in May 1998 of the user needs regarding digital data use and re-use (the report is attached as Appendix 1). Archaeologists from all areas of the discipline attended. The following table summarises the range of bodies offering support, organised training sessions, and documentation, available at a variety of levels (international, national, regional and local). The Glossary of terms has details of the various acronyms used here. In addition, the ADS have a web page listing training opportunities for archaeologists, which is available at: http://ads.ahds.ac.uk/project/userinfo/training.html.
Archaeologists based in universities invariably have access to skilled support staff (based in a computer services or library departments), and undergraduate degrees now often include an element of tuition in basic IT skills. Archaeologists based in large organisations such as national bodies or local government departments also often have access to computing staff. As a first point of call, staff in such organisations should look to their own IT staff for assistance. However, those in smaller organisations, or self-employed, do not necessarily have ready access to training. Support may be available through local or regional museums and local government departments, but this is not always the case. Training in IT skills in general is often available through local FE colleges and/or universities. In the future, provision will be extended to all in Britain through the Public Library and National Grid for Learning. Public libraries should develop into important training and support centres, through which anyone with an interest in archaeology can access the various on-line resources being developed and currently available. In the Republic of Ireland, the Schools IT 2000 project (Dept of Education and Science 1997) has the potential to improve Internet access and support provision for all.
There is also an important and growing body of documentation that can be obtained freely over the Internet, or in many instances in printed form. The various national and international bodies and projects listed in Table 10.1 produce useful information, and can in some cases provide support regarding the use of their own services. A degree of IT competence may be necessary in order to make informed use of these services, and for basic skills archaeologists are advised to seek out general IT courses in their area.
Table 10.1 Training provision for archaeologists
Support Training provision: Documentation from: Support services available internationally - - Standards: Getty Information Institute CIMI (CHIO project) Support services in Europe - - Standards: ICOM-CIDOC Support services available across Britain CCTA (for public sector services) Digital preservation: ADS; NPO (documents) IT skills: Netskills workshops Teaching and learning (HE): CTICH; TLTP Data creation: ADS Guides to Good Practice Standards: MDA Guidelines: IFA Teaching: NISS (for HE, FE and schools) National Digital preservation: ADS; National Archives for Ireland (primarily documents) Re-using digital data: ADS Geophysics: AML/EH Teaching and learning (HE): TLTSN, TALiSMAN (Scotland) Standards: RCHME/EH RCAHMS and HS RCAHMW and Cadw Environment and Heritage Service, DoENI. Regional support Each region has a major player, e.g. major museum, archaeology unit, university, SMR Area Museum Councils and large museums offer training - Local support IT department in one's organisation. Failing this, an informal network of contacts. Some universities, colleges run short courses Some agencies write standards (e.g. SMRs, museums)
Support infrastructureAs well as training provision, additional support may be needed to purchase equipment and software in order to create and/or access digital resources. Table 10.2 lists the range of bodies that can provide guidance and support for IT in archaeology. As with Table 10.1, results are presented at a variety of levels: international, national, regional and local.
One of the critical factors identified in Strategies for Digital Data behind the limited use of digital resources was a lack of access to the Internet. Archaeological organisations need to work together to develop strategies for addressing the information needs of the discipline. There are also ongoing projects in the humanities and beyond that have the potential to increase access to digital data for those interested in archaeology (in particular the New Library for Britain; the Schools IT 2000 project for Ireland), by providing publicly accessible computers linked to the Internet.
A number of organisations can offer support for IT infrastructure, though may be restricted in the way they distribute funds. In addition, applications for funds may need to conform to certain standards (stipulated by funding bodies). Submissions for grants to the Heritage Lottery Fund may include requests for funds to purchase equipment for projects 'which are aimed at widening understanding and enjoyment of the heritage, and widening access to heritage information' (Heritage Lottery Fund 1998, 7). The university sector has access to a wide range of funds, though it is possible to broaden the range of organisations included in any project, where collaborations are planned between HEIs and outside bodies. Support is also available to museums, though this depends on whether the museum is based in local government, attached to a university, is a national museum, or private. Individual practitioners may get support from professional societies of which they are a member, irrespective of where they work. At present, support for individuals, particularly the general public, is limited. Information may be obtained from the various bodies listed in Table 10.2, though in some instances this may be limited to those who are members of the organisation concerned. Local libraries, SMRs, museums and art galleries may provide useful documentation and support staff.
Table 10.2 Support for IT in Archaeology
IT infrastructure Awareness raising Major players: Support services available internationally CIMI - Getty Information Institute CIMI Support services available across Europe Aquarelle, CHIO - CEN/ISSS has funded international, collaborative projects (e.g. CHIO) Support services available across Britain HLF - funding and identifying gaps in existing digital content
JISC - funding, identifying gaps in extant digital content and providing JANET
Software development - all funding bodies (e.g. RCHME/ALGAO and eXegeSIS)
CCTA - advises government offices on IT
Digital archiving pilot projects - ADS, EH and units
Funding agencies - pro-active development of digital preservation requirements and standards
ALGAO/ARIA - developing standards, co-ordinating HLF bids, using the MIDAS standard, developing recommendations for SMR software
DCMS - portable antiquities pilot scheme and other projects setting examples of good practive (e.g. thesauri)
ADS, AHRB, ALGAO/ARIA, British Academy, British Library, CBA, CCTA, DCMS, ESRC, HLF, JISC, Leverhulme Trust, MDA, MGC, NERC, NESTA, OS, PRO, professional societies, universities National Digital preservation: ADS Re-using digital data: ADS - ADS, Built Heritage (DoENI), Cadw, Duchas, EH, HS, IAPA, national museums, RCAHMS, RCAHMw, RCHME, Heritage Council of Ireland, National Archives of Ireland Regional support - One-off projects (e.g. Merseyside Online; SCRAN).
EH is evaluating regional centres for research, which may play important roles. Web-based Information Gateways (e.g. ARGE).
Area Museum Councils, CBA regional groups, government depts, IFA regional groups, museums, SMRs Local support Most local authorities offer some IT support Some agencies promote standards Consultants, libraries, local authorities, societies, trusts, SMRs, units, universities
A broad range of bodies can offer support to archaeologists for IT infrastructure, though this is partly dependent on where archaeologists are based. There is considerable overlap in the range of services that various bodies offer, and it is essential that these bodies co-ordinate to develop strategies for funding IT infrastructure for archaeology as a whole. Service providers need to ensure that current and potential users are aware of the support they can give, and guidance on where archaeologists can turn to for support would be very beneficial to the discipline.
Training needs and support infrastructure: Recommendations
In order to use digital data, training is necessary, both as part of on-going professional development and as a component of University education. There is also a need to provide appropriate training for digital archivists. Archaeological organisations might co-ordinate their efforts to develop strategies more effectively for funding IT infrastructure for archaeology as a whole.