Code
/
ocfl_export
8
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
create OCFL node.json files for each node and associated media/files.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
5 Commits
2 Branches
0 Tags
361 KiB
 
Branch: ocfl-files-and-hierarchy
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'ocfl-files-and-hierarchy'
${ noResults }
ocfl_export/README.md

6.6 KiB
Raw Permalink Blame History

OCFL Export

Exports Drupal nodes and media as OCFL (Oxford Common File Layout) objects on the filesystem, for digital preservation.

The module is Islandora aware: a node's OCFL object contains the node's metadata, the binaries of every media that belongs to it (originals, derivatives, etc.) and its place in the collection hierarchy. It also works on plain Drupal sites, where it includes files in a node's own file/image fields and media referenced from the node's fields.

Requirements

  • Drupal 10.5+
  • PHP 8.3+
  • Core modules: node, user, media (and file)
  • Optional: Islandora for field_media_of, field_member_of and field_model support

Installation

composer require roblib/ocfl_export
drush en ocfl_export

Then configure it at Administration » Configuration » Content authoring » OCFL Export Settings (/admin/config/content/ocfl-export).

When upgrading an existing installation, run drush updatedb to add new settings.

What gets exported

Every node and (optionally) every media entity becomes its own OCFL object. A node object looks like this:

node.json                              Node metadata (see below)
media/{media uuid}/{filename}          Files of each associated media
files/{field name}/{filename}          Files in the node's own file/image fields

Associated media are:

  • media whose "media of" field (Islandora: field_media_of) points at the node, and
  • media referenced from any media reference field on the node (for example field_representative_image).

A standalone media object contains node.json (the media's metadata) and files/{field name}/{filename}. Media thumbnails generated by Drupal are not exported.

node.json

In addition to the node's base properties (uuid, nid, title, type, created, changed, status, author) and all of its fields, node.json contains:

  • relationships
    • model: the Islandora model term (tid, uuid, name, and its external uri when present).
    • member_of: the node's direct parents, each with nid, uuid, title, type and model.
    • hierarchy: every ancestor path through member_of, each ordered from the root collection down to the direct parent. A node with several parents has several paths.
  • media: each associated media's full metadata (including field_media_use), how it is associated with the node (media_of and/or field:<field name>) and its files.
  • files: the node's own files.
  • _metadata: export information. _metadata.missing_files lists any file that could not be read during export; the rest of the object is still saved.

Each file entry gives its logical_path inside the object, plus field, fid, uuid, filename, filemime, filesize and the original Drupal uri.

Versions and deduplication

Every export creates a new OCFL version. A file whose checksum (sha512) is already in the object's manifest is not copied again: the new version simply references the existing copy, so large files that don't change are stored once. Each copied file is verified against its checksum. Checksums are cached by file URI, size and modification time so unchanged files aren't re-read on every export.

Deleting a node or media creates a final version marked "deleted": true; the earlier versions, including their files, are kept.

Storage layout

The storage root (default private://ocfl-storage) uses the 0003-hash-and-id-n-tuple-storage-layout extension. Object IDs are urn:uuid:{entity uuid}, and an object is stored at the first three 3-character tuples of the sha256 of its ID, followed by the percent-encoded ID:

ocfl-storage/
├── 0=ocfl_1.1
├── ocfl_layout.json
├── extensions/0003-hash-and-id-n-tuple-storage-layout/config.json
└── b55/aed/a8b/urn%3auuid%3a11111111-2222-3333-4444-555555555555/
    ├── 0=ocfl_object_1.1
    ├── inventory.json
    ├── inventory.json.sha512
    ├── v1/
    └── v2/

To find an object on disk, search for its UUID:

find /path/to/ocfl-storage -maxdepth 4 -name '*<uuid>*'

The module refuses to write into a storage root that declares a different layout.

Configuration

Setting Default Description
Storage Root Path private://ocfl-storage Stream wrapper URI or absolute path of the storage root.
Enable automatic export on node save on Export nodes when they (or their media) are saved.
Content types to export all Limit automatic node export to these types.
Enable automatic export on media save on Also export each media as its own object.
Media types to export all Limit standalone media export to these types.
Copy files into OCFL objects on When off, only metadata is exported.
"Media of" field field_media_of Media field pointing at the owning node.
"Member of" field field_member_of Node field pointing at parent nodes.
Model field field_model Node field holding the model term.

Leave a field setting empty to disable that feature.

Islandora sites: media binaries are already stored in their parent node's object. Leaving standalone media export on stores them a second time, so you will usually want to turn it off.

Exporting

  • Automatically: saving a node, or inserting, updating or deleting one of its media, queues the node. Queued nodes are exported once when the request ends, so saving a node together with several media creates one new version. This also applies to drush commands. Islandora derivatives arrive in separate requests, so each derivative creates a version of its parent node.
  • Manually: the Export to OCFL button on a node or media page (requires the Export content to OCFL permission).
  • In bulk: the Export to OCFL action, available to Views bulk operations on content and media listings.

Users with the Export content to OCFL permission see the current export version on node and media pages, with a link to view the latest node.json.

Permissions

  • Administer OCFL Export: change the module's settings.
  • Export content to OCFL: export manually and view export status and JSON.

Limitations

  • Titles stored in member_of and hierarchy are those at export time. Renaming a collection does not re-export its descendants; their records update the next time each is exported.
  • Version user entries contain only a name (no address), so OCFL validators report the W008 recommendation.

License

GPL-2.0-or-later. See LICENSE.