Project Manager: Adrian Stevenson, UKOLN, a.stevenson@ukoln.ac.uk

Author: Richard Jones, Symplectic Ltd, richard@symplectic.co.uk

Further Contact Details: http://www.swordapp.org/contact-us

[2] JISC: http://www.jisc.ac.uk/

[3] DSpace: http://www.dspace.org/

[4] EPrints: http://www.eprints.org/

[5] Fedora: http://www.fedoracommons.info/

[6] arXiv.org: http://arxiv.org/

[7] Microsoft Zentity: http://research.microsoft.com/en-us/projects/zentity/

[8] Intrallect IntraLibrary http://www.intrallect.com

[9] Facebook SWORD App by Stuart Lewis: http://fb.swordapp.org/

[10] Microsoft Word SWORD Add-in. Presentation from OR10: http://research.microsoft.com/en-us/um/redmond/projects/authoring/090518-pablofe-connecting%20authors%20and%20repositories%20through%20sword.pptx ; Video Tutorial: http://www.youtube.com/watch?v=2_M2gfUyVzU

[11] Open Journal System, SWORD plugin: http://pkp.sfu.ca/wiki/index.php/Ojs_plugins#SWORD_1.2_Repository_Deposit

[12] The Atom Publishing Protocol: http://bitworking.org/projects/atom/rfc5023.html

[13] Metadata Encoding and Transmission Standard (METS): http://www.loc.gov/standards/mets/

[14] Atom: http://www.ietf.org/rfc/rfc4287.txt

[15] Atom Multipart Draft: http://tools.ietf.org/html/draft-gregorio-atompub-multipart-04

[16] Guidelines for Writing an IANA Considerations Section: http://www.faqs.org/rfcs/rfc2434.html

[17] WebDav: http://www.webdav.org/specs/rfc2518.html

[18] Open Archives Initiative – Object Reuse and Exchange (OAI-ORE): http://www.openarchives.org/ore/

[19] OAI-ORE RDF/XML serialisation: http://www.openarchives.org/ore/1.0/rdfxml.html

[20] Linked Data: http://linkeddata.org/

[21] Symplectic Ltd: http://www.symplectic.co.uk/

[22] Symplectic Repository Tools: http://www.symplectic.co.uk/products/repository-tools.html

[23] Symplectic Elements publications management system: http://www.symplectic.co.uk/products/publications.html

This paper has described the two major developments that the SWORD 4 project plans to undertake:

1. To add stricter semantics to existing identifiers in the deposit receipt, and to add two further identifiers useful to repositories;
2. To specify a default manifest serialisation which can describe the content of repository items and provide basic information about state.

We have still not answered all of the limitations we discussed at the start of this paper, although we have shown that these two additions to the standard lead us towards a more complete treatment of deposit lifecycle management. The things that we have explicitly missed out at this stage include: how authorisation and authentication works for operations other than deposit, and how it might work on modifying the parts of a package some time after deposit; how we explicitly bypass packaging and deal with single file deposits only; and how we more accurately describe or prescribe the package formats used for deposit. These are all issues which SWORD will be able to address in future versions.

The details and practical implementation of these developments are presented in this paper as a suggested approach, and by no means definitive; instead they are derived from apparent best practices already in use in the repository community, and in the perceived needs of the repository to play well in the ecosystem of information systems. The objective of this paper is to make these proposals to the community and to solicit feedback on improvements, alterations and limitations. By producing an open source reference implementation for general review by the community, it also will be possible to see what the consequences of these suggestions are, and how they can be used to improve the state of play in repository deposit.

The objective of this work was to integrate their publications management system (Symplectic Elements ^[23]) into a variety of digital repositories, to allow the provision of a rich deposit client which integrated smoothly with the researcher’s day-to-day use of the publications system. Its functionality allows academics to upload individual files to the repository over a period of time, making the use of a content package difficult.

In order to meet the requirements, a full AtomPub implementation with SWORD extensions was developed which provided a consistent API across the repositories, allowing them to be interchanged at will, without the client needing to be aware of the changes. The result has been a simple and effective interface to these digital repositories, abstracting the user away from the need to even necessarily be aware of the repository. It answers the limitations to SWORD laid out in the Introduction:

1. External systems have to do some of the work of a repository: The repository is considered the authority for file content, and file uploads go straight into it.
2. Users must know when an item is archivable: It provides a simple file management interface, and does not explicitly question users on when an item is strictly “complete”; modifications are always possible later.
3. Full AtomPub profile for SWORD is unclear: It uses the full range of AtomPub features, adapted for usage with SWORD, some of which have been drawn on for this paper.
4. Dependence on structured packages: By not using packages in the same way, it bypasses the problem of needing a repository to be able to understand some particular package format.

During the SWORD 4 project, Symplectic will provide to the community an open source server implementation of the proposed next version of the standard based on its exiting Repository Tools system. This will contain a number of additional technical proposals on the direction of the standard and its implementation for comment by the community, thus helping the project define the next version of the standard in full. This will not be the final implementation of the SWORD 2.0 standard, but a discussion piece for developers to work from. The repository platform used for this example will be chosen based on demand from the community.

[21] Symplectic Ltd: http://www.symplectic.co.uk/
[22] Symplectic Repository Tools: http://www.symplectic.co.uk/products/repository-tools.html
[23] Symplectic Elements publications management system: http://www.symplectic.co.uk/products/publications.html

Nonetheless, this is a diversion of the Atom standard, and alternative suggestions and simplifications are requested from the community.  One alternative, for example, is simply for us to define our own format for transmitting this information.

OAI-ORE concerns itself with describing aggregations of web resources, and does not contain any semantics which allow us to represent the state of the item in the repository as we discussed above. Because the OAI-ORE is expressible as RDF – the format upon which the Linked Data ^[20]
movement relies – it is trivial to construct a manifest document which contains both the OAI-ORE Resource Map as well as additional statements in RDF asserting the workflow state of the item. Consider the following basic example manifest document, serialised in RDF/XML:

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"

xmlns:ore="http://www.openarchives.org/ore/terms/"

xmlns:sword="http://swordapp.org/terms/">

<!-- standard way of declaring ORE Resource Map describing a resource -->

<rdf:Description rdf:about="http://swordapp.org/rem/rdfxml/100">

<ore:describes rdf:resource="http://swordapp.org/aggregation/100"/>

</rdf:Description>

<!-- description of the repository item -->

<rdf:Description rdf:about="http://swordapp.org/aggregation/100">

<ore:isDescribedBy rdf:resource="http://swordapp.org/rem/rdfxml/100"/>

<!-- the files associated with the repository item -->

<ore:aggregates rdf:resource="http://swordapp.org/object/100/file1.pdf"/>

<ore:aggregates rdf:resource="http://swordapp.org/object/100/file1.pdf"/>

<ore:aggregates rdf:resource="http://swordapp.org/object/100/file1.pdf"/>

<!-- SWORD relationships regarding the item's repository state -->

<sword:state rdf:resource="http://swordapp.org/terms/under-review"/>

<sword:state-description

rdf:resource="http://swordapp.org/repo/under-review"/>

</rdf:Description>

</rdf:RDF>

Here we see a simple ORE Resource Map which provides us with a description of an object which contains 3 files, as indicated by the ore:aggregates elements. It then goes on to assert two further relationships in the form of sword:state, which indicates the SWORD standard dictionary term for the state the item is in (or the repository specific term for the item’s current state), and sword:state-description, which would allow the repository to provide its own detail to the client as to what that state means to them.

It is acknowledged that since some repositories already implement OAI-ORE, they may be able, by default, to provide more information in the resource map than we show above. The SWORD specification should apply no limits to these cases, and is only acting as a profile of OAI-ORE for the purposes of providing a manifest which is widely comprehendible. That is, all the specification will say is that ore:aggregates elements must be provided for each file in the repository item that the repository wishes to alert the client to.

The ultimate objective is to ensure that the SWORD profile of OAI-ORE provides enough information that the following operations will be possible:

• HTTP POST of a new file into the repository item; for example, onto the URI for the aggregation (http://swordapp.org/aggregation/100 in the above example);
• HTTP PUT of a new file over an existing file; that is, onto the resource referenced in ore:aggregates
• HTTP DELETE of an existing file; again, requested of the resource referenced in ore:aggregates
• HTTP GET of individual files inside the repository item, requested of the resource referenced in ore:aggregates

Notice that these 4 standard HTTP operations are not on the repository item itself, or the package which was deposited, but on the contents of the repository item created from the original package deposit, as described by the resource map.

[18] Open Archives Initiative – Object Reuse and Exchange (OAI-ORE): http://www.openarchives.org/ore/
[19]OAI-ORE RDF/XML serialisation: http://www.openarchives.org/ore/1.0/rdfxml.html
[20] Linked Data: http://linkeddata.org/

• Extensions to ATOM and AtomPub
• Extensions to HTTP
• Packaging
• An application profile which demonstrates how to orchestrate these independent specifications (including HTTP and AtomPub) into the full SWORD standard.

The advancements suggested in this paper are also specified in the same way, as follows:

1. The deposit response will be augmented with the new identifier types;
2. The default manifest document will be specified as at least one extension to the specification. A default serialisation will be offered, but some mechanism will be provided to support different formats (e.g. content negotiation). The specification will allow for additional serialisations to provide their own extensions to the standard.

1. The state of the item in the repository

Most repositories, at a high level, have 3 states that items are likely to be in (based on analysis of DSpace, EPrints and Fedora software):

• In preparation: this typically lasts a short while, and is the state that new repository items begin life in. During this stage, files and metadata are being added to the item, and it is generally only worked on by a single user. Repository administrators do not usually have access directly to the item.
• Under review: once a repository item is fully prepared it is usually injected into some form of review process. Repository managers carry out general tasks like metadata verification and copyright compliance, or tasks more specific to their environment or organisation such as appropriateness for the archive. Within the review stage, there is no clear common workflow, as usages for repositories and for SWORD are sufficiently broad as to encompass many different workflows.
• Archived: once review completes the repository item is archived. In some cases this may mean that the item has been made public under a stable identifier, while in others it may mean that the item has gone into the dark archive for preservation. Purposes for and consequences of archiving are not covered here.

Additionally, we expect to require states such as:

• Deleted: the repository item was at one point in the Archive, but has been removed
• Rejected: the repository item was rejected from the repository before reaching the Archive

It is suggested that we introduce a short ontology to cover the above states, and for the serialisation of this in the manifest document to be extensible such that additional states can easily be added, either by the SWORD standard at a later date or by specific implementation for local needs.

It is also desirable to offer the repository the opportunity to describe what each of those states means to them; this would be similar in ethos to the sword:treatment element, which allows the repository to describe what processes have actually happened to the package during deposit [SWORD spec section B.9.8], except that it is giving details on what is currently happening to the deposit.  With each of the states represented by a URI this would also allow for the addition of non-standard states which are still comprehendible by the client.

With the ability of the repository to describe these states it would be easy for SWORD clients to allow depositors to keep track of their submissions, giving them feedback on their progress, rather than being subject to the current fire-and-forget approach.  The potential for creating rich clients, and for making users feel invested in the process of deposit would be increased.

In summary, we would expect to add two elements to the manifest:

1. An assertion to the state of the item (In preparation, under review, archived, etc.);
2. An identifier which resolves to a description of the meaning of that state in the repository.

Furthermore, since this paper is intended to generate discourse around the direction of the standard, we are also looking for common and generic states which are useful to systems which are not necessarily repositories.

2. Information about the unpackaged object in the repository

Many repositories are likely to unpackage the incoming content and import it into their native format. This typically includes:

• A top-level object: this is effectively the equivalent of the package – it is the container within which all the content is held;
• Associated files: the actual content of the item, as extracted as individual files from the package. This may also include additional files created by the repository, such as format conversions, thumbnails, etc;
• Metadata: anything from structural, administrative and bibliographic data about the item, provided in the package as metadata files or within the manifest itself, depending on package format. As with the files, this may contain metadata that was not in the original package;

It may be of interest to clients to be able to display this information to their users, for the purposes of creating a rich deposit environment.  It is certainly possible to include identifiers for the individual files which would allow further operations to be performed on them; for example HTTP DELETE could be provided to allow the client to subsequently delete individual files, or HTTP PUT could be used to replace one file with another.

Including the metadata in the description of the item would be a significant challenge, and is therefore not addressed here.  Future work in this area could consider operations similar to WebDAV’s PROPFIND and PROPPATCH operations as a starting point ^[17], or by performing HTTP PUT to the Entry document for the repository item.  It is unclear how useful or successful these approaches would be.

It is the recommendation of this paper, therefore, that the basic structural information concerning the unpackaged item is made available in some common serialisation, and that this explicitly excludes any treatment of the actual metadata content.  Instead we would focus simply on the content files.

[17] WebDav: http://www.webdav.org/specs/rfc2518.html

1. An atom:content element which provides the URI to one of:
   • The splash page for the repository item;
   • A machine readable description of the repository item;
   • The originally deposited package;
   • Some combination of (i) and (ii) (for example, RDFa embedded in a splash page);
   • Some other identifier, unspecified.
2. An atom:link element with @rel=”edit-media”, which provides the URI to one of:
   • A version of the resource which can be updated by HTTP PUT;
   • A representation which supplies metadata from the package and access to unpacked binary files.
3. An arbitrary number of atom:link elements with repository specific @rel values which may contain URIs for the parts of the unpacked item.
4. An atom:link element with @rel=”edit”, which provides the URI to one of:
   • The Entry document, from which an equivalent document to the deposit receipt can be obtained, and to which HTTP DELETE requests could be directed;
   • A jump-off page;
   • Users private metadata;
   • A workflow management page;
   • Some other identifier, unspecified.

As this shows, the specification does not say anything particularly strong about any of the URIs which are returned, while making allowances for the large number of uses the a repository may wish to make of them. This is interesting, as it is indicative of the requirements that we must find a way to formally support. We should, firstly, bring the meanings of each of these URIs into line with AtomPub thus [AtomPub spec section 9.6]:

1. An atom:content element which resolves to the Media Resource; in the case of package deposit, the Media Resource is the package itself, and therefore this element provides a URI from which the originally deposited package can be retrieved.
2. An atom:link element with @rel=”edit-media” which resolves to the Media Resource; since this is the original package, as noted in (1), this should resolve to the same content as the atom:content source during HTTP GET, but must also be able to support HTTP PUT of a new package. This means, therefore, that the server is free to use a different URI for this link to the one used in atom:content if that suits the implementation better.
3. An atom:link element with @rel=”edit” which resolves to the Entry document; in the case of package deposit this would behave as follows:
   • HTTP GET returns the Atom Entry document which is equivalent to the deposit receipt produced by the server;
   • HTTP PUT has no defined behaviour, as it is used traditionally to update the item metadata, but since package deposit has no independent metadata this is meaningless;
   • HTTP DELETE would allow us to request the deletion of the repository item itself, including the original package and the associated unpackaged content.



In addition to these, we should also introduce the following new links with appropriate semantics for our environment:

4. An element which provides a link which resolves to another document which we use to describe additional repository specific information about the deposited item. This manifest document will be described in the next section.
5. An element which provides a link to the splash page of the repository item if one exists; this page may also include embedded RDFa.

Notice that we have not specified how these additional new links should be serialised in Atom. There are two possible routes: to use atom:link elements with @rel values specific to SWORD, or to embed new foreign markup such as sword:manifest and sword:splash. The first approach is more in line with the Atom standard approach, but formal recognition of new @rel values is strongly suggested, and this requires passing through the IANA registration procedure ^[16]. The second approach will be easier to implement in the standard but be less in line with Atom.

We have also pushed a lot of the vague uses of the identifiers in the current SWORD specification into the link described in (4): this will give us the opportunity to define a document which can give the client all of the relevant additional information which the current specification alludes to.

[16] Guidelines for Writing an IANA Considerations Section: http://www.faqs.org/rfcs/rfc2434.html

It is not possible to introduce features to deal with everything we discuss above immediately. This next version of SWORD will instead constrain itself to a single critically important area which will pave the way to full deposit lifecycle support: the URIs supplied to the client in the deposit receipt, and the resources to which they resolve.

2. Users must know when an item is archivable
As a consequence of the first limitation, SWORD also requires that client systems are solely responsible for asserting that some set of files and metadata is the “finished” object to be placed in the archive; the client must draw a boundary around some set of content and package it for delivery to the repository. Often this is a hard assertion to make (such as at what point during the article publication lifecycle something should be archived), and sometimes it is even impossible (such as during the production of continuous data sets). If the repository were the service provider for content storage then the weight of deciding when something should be truly archived can be balanced between the user and the repository administration, where skills in archiving are more likely to reside. We make no assertion here as to what the notion of “archiving” means in terms of a specific repository; it may be true archiving, or simply re­publication via an open access interface.

3. Full AtomPub profile for SWORD is unclear
From a practical perspective, the SWORD specification does not offer explicit guidelines for the use of the full range of features of AtomPub ^[12], upon which SWORD is based. While it would be possible for a server implementation to also include a full treatment of AtomPub, this has not been the case in the funded implementations of SWORD thus far. In addition, given that SWORD added extensions and profiling to the deposit (HTTP POST) aspects of AtomPub, it is likely that it will need to do so for these other aspects. For example, SWORD supports the “On Behalf Of” deposit use case by adding new HTTP headers [SWORD spec section A.2], but says nothing about how these should be used except in the case of the deposit using HTTP POST.

4. Dependence on structured packages
SWORD has been fully dependent on structured packages to deliver the payload. This pushes some significant interoperability challenges out­of­scope, and deliberately so for these first steps. It is difficult to reliably identify packaged content: there is the mime­type of the container (e.g. application/zip), the internal structure of the package, the manifest format, and potentially several nested layers of information therein, such as structural metadata and bibliographic metadata (for an example, consider METS ^[13]). There is the possibility that we could choose a standard SWORD package format, to alleviate this issue in the short term. We could also consider supporting single content file deposits supported by Atom ^[14] formatted metadata documents, as per the AtomPub Multipart ^[15] specification, which would reduce some of the problems significantly.

[12] The Atom Publishing Protocol: http://bitworking.org/projects/atom/rfc5023.html
[13] Metadata Encoding and Transmission Standard (METS): http://www.loc.gov/standards/mets/
[14] Atom: http://www.ietf.org/rfc/rfc4287.txt
[15] Atom Multipart Draft: http://tools.ietf.org/html/draft-gregorio-atompub-multipart-04

To achieve this, SWORD has had to take into account, additionally:
• The fact that repositories often cluster their content into “collections”, into which deposited content must go [SWORD spec section B.8];
• How to indicate to the repository what the format and contents of a package are [SWORD spec section A.1];
• Authentication and authorisation, and specifically supporting the “On Behalf Of” deposit use case [SWORD spec section A.2].

It has been, therefore, a significant amount of work to take this first step along the road to
interoperability.

Using open standards, open source software development and an inclusive approach to the specification a community has been able to grow around SWORD. There are several open source client and server implementations: it is supported natively by DSpace^[3], EPrints ^[4] and Fedora ^[5], arXiv ^[6] and a number of commercial systems such as Zentity ^[7] and IntraLibrary ^[8]. There is a Facebook deposit application ^[9], Microsoft have developed an add­on to Word which will deposit your documents into your archive ^[10], and likewise the Open Journal System (OJS) ^[11].

The repository is no longer a stand­alone entity dependent on its own isolated architectural choices, but part of an increasingly complex and interconnected environment of scholarly infrastructure systems. To play their role appropriately in this environment it is necessary for repositories to offer sophisticated and appropriate APIs that can be used by their peers to utilise their services. Here, SWORD has a role to play, but there are some limitations in its current state which we will enumerate here in order to see where we must take it next.

[1] SWORD: Home page: http://www.swordapp.org/ ; Specification: http://www.swordapp.org/sword/specifications
[2] JISC: http://www.jisc.ac.uk/
[3] DSpace: http://www.dspace.org/
[4] EPrints: http://www.eprints.org/
[5] Fedora: http://www.fedoracommons.info/
[6] arXiv.org: http://arxiv.org/
[7] Microsoft Zentity: http://research.microsoft.com/en-us/projects/zentity/
[8] Intrallect IntraLibrary http://www.intrallect.com
[9] Facebook SWORD App by Stuart Lewis: http://fb.swordapp.org/
[10] Microsoft Word SWORD Add-in. Presentation from OR10: http://research.microsoft.com/en-us/um/redmond/projects/authoring/090518-pablofe-connecting%20authors%20and%20repositories%20through%20sword.pptx ; Video Tutorial: http://www.youtube.com/watch?v=2_M2gfUyVzU
[11] Open Journal System, SWORD plugin: http://pkp.sfu.ca/wiki/index.php/Ojs_plugins#SWORD_1.2_Repository_Deposit