Compare commits
No commits in common. 'ocfl-files-and-hierarchy' and 'main' have entirely different histories.
ocfl-files
...
main
17 changed files with 262 additions and 1602 deletions
@ -1,339 +0,0 @@
|
||||
GNU GENERAL PUBLIC LICENSE |
||||
Version 2, June 1991 |
||||
|
||||
Copyright (C) 1989, 1991 Free Software Foundation, Inc., |
||||
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA |
||||
Everyone is permitted to copy and distribute verbatim copies |
||||
of this license document, but changing it is not allowed. |
||||
|
||||
Preamble |
||||
|
||||
The licenses for most software are designed to take away your |
||||
freedom to share and change it. By contrast, the GNU General Public |
||||
License is intended to guarantee your freedom to share and change free |
||||
software--to make sure the software is free for all its users. This |
||||
General Public License applies to most of the Free Software |
||||
Foundation's software and to any other program whose authors commit to |
||||
using it. (Some other Free Software Foundation software is covered by |
||||
the GNU Lesser General Public License instead.) You can apply it to |
||||
your programs, too. |
||||
|
||||
When we speak of free software, we are referring to freedom, not |
||||
price. Our General Public Licenses are designed to make sure that you |
||||
have the freedom to distribute copies of free software (and charge for |
||||
this service if you wish), that you receive source code or can get it |
||||
if you want it, that you can change the software or use pieces of it |
||||
in new free programs; and that you know you can do these things. |
||||
|
||||
To protect your rights, we need to make restrictions that forbid |
||||
anyone to deny you these rights or to ask you to surrender the rights. |
||||
These restrictions translate to certain responsibilities for you if you |
||||
distribute copies of the software, or if you modify it. |
||||
|
||||
For example, if you distribute copies of such a program, whether |
||||
gratis or for a fee, you must give the recipients all the rights that |
||||
you have. You must make sure that they, too, receive or can get the |
||||
source code. And you must show them these terms so they know their |
||||
rights. |
||||
|
||||
We protect your rights with two steps: (1) copyright the software, and |
||||
(2) offer you this license which gives you legal permission to copy, |
||||
distribute and/or modify the software. |
||||
|
||||
Also, for each author's protection and ours, we want to make certain |
||||
that everyone understands that there is no warranty for this free |
||||
software. If the software is modified by someone else and passed on, we |
||||
want its recipients to know that what they have is not the original, so |
||||
that any problems introduced by others will not reflect on the original |
||||
authors' reputations. |
||||
|
||||
Finally, any free program is threatened constantly by software |
||||
patents. We wish to avoid the danger that redistributors of a free |
||||
program will individually obtain patent licenses, in effect making the |
||||
program proprietary. To prevent this, we have made it clear that any |
||||
patent must be licensed for everyone's free use or not licensed at all. |
||||
|
||||
The precise terms and conditions for copying, distribution and |
||||
modification follow. |
||||
|
||||
GNU GENERAL PUBLIC LICENSE |
||||
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION |
||||
|
||||
0. This License applies to any program or other work which contains |
||||
a notice placed by the copyright holder saying it may be distributed |
||||
under the terms of this General Public License. The "Program", below, |
||||
refers to any such program or work, and a "work based on the Program" |
||||
means either the Program or any derivative work under copyright law: |
||||
that is to say, a work containing the Program or a portion of it, |
||||
either verbatim or with modifications and/or translated into another |
||||
language. (Hereinafter, translation is included without limitation in |
||||
the term "modification".) Each licensee is addressed as "you". |
||||
|
||||
Activities other than copying, distribution and modification are not |
||||
covered by this License; they are outside its scope. The act of |
||||
running the Program is not restricted, and the output from the Program |
||||
is covered only if its contents constitute a work based on the |
||||
Program (independent of having been made by running the Program). |
||||
Whether that is true depends on what the Program does. |
||||
|
||||
1. You may copy and distribute verbatim copies of the Program's |
||||
source code as you receive it, in any medium, provided that you |
||||
conspicuously and appropriately publish on each copy an appropriate |
||||
copyright notice and disclaimer of warranty; keep intact all the |
||||
notices that refer to this License and to the absence of any warranty; |
||||
and give any other recipients of the Program a copy of this License |
||||
along with the Program. |
||||
|
||||
You may charge a fee for the physical act of transferring a copy, and |
||||
you may at your option offer warranty protection in exchange for a fee. |
||||
|
||||
2. You may modify your copy or copies of the Program or any portion |
||||
of it, thus forming a work based on the Program, and copy and |
||||
distribute such modifications or work under the terms of Section 1 |
||||
above, provided that you also meet all of these conditions: |
||||
|
||||
a) You must cause the modified files to carry prominent notices |
||||
stating that you changed the files and the date of any change. |
||||
|
||||
b) You must cause any work that you distribute or publish, that in |
||||
whole or in part contains or is derived from the Program or any |
||||
part thereof, to be licensed as a whole at no charge to all third |
||||
parties under the terms of this License. |
||||
|
||||
c) If the modified program normally reads commands interactively |
||||
when run, you must cause it, when started running for such |
||||
interactive use in the most ordinary way, to print or display an |
||||
announcement including an appropriate copyright notice and a |
||||
notice that there is no warranty (or else, saying that you provide |
||||
a warranty) and that users may redistribute the program under |
||||
these conditions, and telling the user how to view a copy of this |
||||
License. (Exception: if the Program itself is interactive but |
||||
does not normally print such an announcement, your work based on |
||||
the Program is not required to print an announcement.) |
||||
|
||||
These requirements apply to the modified work as a whole. If |
||||
identifiable sections of that work are not derived from the Program, |
||||
and can be reasonably considered independent and separate works in |
||||
themselves, then this License, and its terms, do not apply to those |
||||
sections when you distribute them as separate works. But when you |
||||
distribute the same sections as part of a whole which is a work based |
||||
on the Program, the distribution of the whole must be on the terms of |
||||
this License, whose permissions for other licensees extend to the |
||||
entire whole, and thus to each and every part regardless of who wrote it. |
||||
|
||||
Thus, it is not the intent of this section to claim rights or contest |
||||
your rights to work written entirely by you; rather, the intent is to |
||||
exercise the right to control the distribution of derivative or |
||||
collective works based on the Program. |
||||
|
||||
In addition, mere aggregation of another work not based on the Program |
||||
with the Program (or with a work based on the Program) on a volume of |
||||
a storage or distribution medium does not bring the other work under |
||||
the scope of this License. |
||||
|
||||
3. You may copy and distribute the Program (or a work based on it, |
||||
under Section 2) in object code or executable form under the terms of |
||||
Sections 1 and 2 above provided that you also do one of the following: |
||||
|
||||
a) Accompany it with the complete corresponding machine-readable |
||||
source code, which must be distributed under the terms of Sections |
||||
1 and 2 above on a medium customarily used for software interchange; or, |
||||
|
||||
b) Accompany it with a written offer, valid for at least three |
||||
years, to give any third party, for a charge no more than your |
||||
cost of physically performing source distribution, a complete |
||||
machine-readable copy of the corresponding source code, to be |
||||
distributed under the terms of Sections 1 and 2 above on a medium |
||||
customarily used for software interchange; or, |
||||
|
||||
c) Accompany it with the information you received as to the offer |
||||
to distribute corresponding source code. (This alternative is |
||||
allowed only for noncommercial distribution and only if you |
||||
received the program in object code or executable form with such |
||||
an offer, in accord with Subsection b above.) |
||||
|
||||
The source code for a work means the preferred form of the work for |
||||
making modifications to it. For an executable work, complete source |
||||
code means all the source code for all modules it contains, plus any |
||||
associated interface definition files, plus the scripts used to |
||||
control compilation and installation of the executable. However, as a |
||||
special exception, the source code distributed need not include |
||||
anything that is normally distributed (in either source or binary |
||||
form) with the major components (compiler, kernel, and so on) of the |
||||
operating system on which the executable runs, unless that component |
||||
itself accompanies the executable. |
||||
|
||||
If distribution of executable or object code is made by offering |
||||
access to copy from a designated place, then offering equivalent |
||||
access to copy the source code from the same place counts as |
||||
distribution of the source code, even though third parties are not |
||||
compelled to copy the source along with the object code. |
||||
|
||||
4. You may not copy, modify, sublicense, or distribute the Program |
||||
except as expressly provided under this License. Any attempt |
||||
otherwise to copy, modify, sublicense or distribute the Program is |
||||
void, and will automatically terminate your rights under this License. |
||||
However, parties who have received copies, or rights, from you under |
||||
this License will not have their licenses terminated so long as such |
||||
parties remain in full compliance. |
||||
|
||||
5. You are not required to accept this License, since you have not |
||||
signed it. However, nothing else grants you permission to modify or |
||||
distribute the Program or its derivative works. These actions are |
||||
prohibited by law if you do not accept this License. Therefore, by |
||||
modifying or distributing the Program (or any work based on the |
||||
Program), you indicate your acceptance of this License to do so, and |
||||
all its terms and conditions for copying, distributing or modifying |
||||
the Program or works based on it. |
||||
|
||||
6. Each time you redistribute the Program (or any work based on the |
||||
Program), the recipient automatically receives a license from the |
||||
original licensor to copy, distribute or modify the Program subject to |
||||
these terms and conditions. You may not impose any further |
||||
restrictions on the recipients' exercise of the rights granted herein. |
||||
You are not responsible for enforcing compliance by third parties to |
||||
this License. |
||||
|
||||
7. If, as a consequence of a court judgment or allegation of patent |
||||
infringement or for any other reason (not limited to patent issues), |
||||
conditions are imposed on you (whether by court order, agreement or |
||||
otherwise) that contradict the conditions of this License, they do not |
||||
excuse you from the conditions of this License. If you cannot |
||||
distribute so as to satisfy simultaneously your obligations under this |
||||
License and any other pertinent obligations, then as a consequence you |
||||
may not distribute the Program at all. For example, if a patent |
||||
license would not permit royalty-free redistribution of the Program by |
||||
all those who receive copies directly or indirectly through you, then |
||||
the only way you could satisfy both it and this License would be to |
||||
refrain entirely from distribution of the Program. |
||||
|
||||
If any portion of this section is held invalid or unenforceable under |
||||
any particular circumstance, the balance of the section is intended to |
||||
apply and the section as a whole is intended to apply in other |
||||
circumstances. |
||||
|
||||
It is not the purpose of this section to induce you to infringe any |
||||
patents or other property right claims or to contest validity of any |
||||
such claims; this section has the sole purpose of protecting the |
||||
integrity of the free software distribution system, which is |
||||
implemented by public license practices. Many people have made |
||||
generous contributions to the wide range of software distributed |
||||
through that system in reliance on consistent application of that |
||||
system; it is up to the author/donor to decide if he or she is willing |
||||
to distribute software through any other system and a licensee cannot |
||||
impose that choice. |
||||
|
||||
This section is intended to make thoroughly clear what is believed to |
||||
be a consequence of the rest of this License. |
||||
|
||||
8. If the distribution and/or use of the Program is restricted in |
||||
certain countries either by patents or by copyrighted interfaces, the |
||||
original copyright holder who places the Program under this License |
||||
may add an explicit geographical distribution limitation excluding |
||||
those countries, so that distribution is permitted only in or among |
||||
countries not thus excluded. In such case, this License incorporates |
||||
the limitation as if written in the body of this License. |
||||
|
||||
9. The Free Software Foundation may publish revised and/or new versions |
||||
of the General Public License from time to time. Such new versions will |
||||
be similar in spirit to the present version, but may differ in detail to |
||||
address new problems or concerns. |
||||
|
||||
Each version is given a distinguishing version number. If the Program |
||||
specifies a version number of this License which applies to it and "any |
||||
later version", you have the option of following the terms and conditions |
||||
either of that version or of any later version published by the Free |
||||
Software Foundation. If the Program does not specify a version number of |
||||
this License, you may choose any version ever published by the Free Software |
||||
Foundation. |
||||
|
||||
10. If you wish to incorporate parts of the Program into other free |
||||
programs whose distribution conditions are different, write to the author |
||||
to ask for permission. For software which is copyrighted by the Free |
||||
Software Foundation, write to the Free Software Foundation; we sometimes |
||||
make exceptions for this. Our decision will be guided by the two goals |
||||
of preserving the free status of all derivatives of our free software and |
||||
of promoting the sharing and reuse of software generally. |
||||
|
||||
NO WARRANTY |
||||
|
||||
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY |
||||
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN |
||||
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES |
||||
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED |
||||
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF |
||||
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS |
||||
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE |
||||
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, |
||||
REPAIR OR CORRECTION. |
||||
|
||||
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING |
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR |
||||
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, |
||||
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING |
||||
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED |
||||
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY |
||||
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER |
||||
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE |
||||
POSSIBILITY OF SUCH DAMAGES. |
||||
|
||||
END OF TERMS AND CONDITIONS |
||||
|
||||
How to Apply These Terms to Your New Programs |
||||
|
||||
If you develop a new program, and you want it to be of the greatest |
||||
possible use to the public, the best way to achieve this is to make it |
||||
free software which everyone can redistribute and change under these terms. |
||||
|
||||
To do so, attach the following notices to the program. It is safest |
||||
to attach them to the start of each source file to most effectively |
||||
convey the exclusion of warranty; and each file should have at least |
||||
the "copyright" line and a pointer to where the full notice is found. |
||||
|
||||
<one line to give the program's name and a brief idea of what it does.> |
||||
Copyright (C) <year> <name of author> |
||||
|
||||
This program is free software; you can redistribute it and/or modify |
||||
it under the terms of the GNU General Public License as published by |
||||
the Free Software Foundation; either version 2 of the License, or |
||||
(at your option) any later version. |
||||
|
||||
This program is distributed in the hope that it will be useful, |
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of |
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
||||
GNU General Public License for more details. |
||||
|
||||
You should have received a copy of the GNU General Public License along |
||||
with this program; if not, write to the Free Software Foundation, Inc., |
||||
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. |
||||
|
||||
Also add information on how to contact you by electronic and paper mail. |
||||
|
||||
If the program is interactive, make it output a short notice like this |
||||
when it starts in an interactive mode: |
||||
|
||||
Gnomovision version 69, Copyright (C) year name of author |
||||
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. |
||||
This is free software, and you are welcome to redistribute it |
||||
under certain conditions; type `show c' for details. |
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate |
||||
parts of the General Public License. Of course, the commands you use may |
||||
be called something other than `show w' and `show c'; they could even be |
||||
mouse-clicks or menu items--whatever suits your program. |
||||
|
||||
You should also get your employer (if you work as a programmer) or your |
||||
school, if any, to sign a "copyright disclaimer" for the program, if |
||||
necessary. Here is a sample; alter the names: |
||||
|
||||
Yoyodyne, Inc., hereby disclaims all copyright interest in the program |
||||
`Gnomovision' (which makes passes at compilers) written by James Hacker. |
||||
|
||||
<signature of Ty Coon>, 1 April 1989 |
||||
Ty Coon, President of Vice |
||||
|
||||
This General Public License does not permit incorporating your program into |
||||
proprietary programs. If your program is a subroutine library, you may |
||||
consider it more useful to permit linking proprietary applications with the |
||||
library. If this is what you want to do, use the GNU Lesser General |
||||
Public License instead of this License. |
||||
@ -1,172 +0,0 @@
|
||||
# OCFL Export |
||||
|
||||
Exports Drupal nodes and media as [OCFL](https://ocfl.io/1.1/spec/) (Oxford |
||||
Common File Layout) objects on the filesystem, for digital preservation. |
||||
|
||||
The module is Islandora aware: a node's OCFL object contains the node's |
||||
metadata, the binaries of every media that belongs to it (originals, |
||||
derivatives, etc.) and its place in the collection hierarchy. It also works on |
||||
plain Drupal sites, where it includes files in a node's own file/image fields |
||||
and media referenced from the node's fields. |
||||
|
||||
## Requirements |
||||
|
||||
- Drupal 10.5+ |
||||
- PHP 8.3+ |
||||
- Core modules: node, user, media (and file) |
||||
- Optional: [Islandora](https://github.com/Islandora/islandora) for |
||||
`field_media_of`, `field_member_of` and `field_model` support |
||||
|
||||
## Installation |
||||
|
||||
```sh |
||||
composer require roblib/ocfl_export |
||||
drush en ocfl_export |
||||
``` |
||||
|
||||
Then configure it at **Administration » Configuration » Content authoring » |
||||
OCFL Export Settings** (`/admin/config/content/ocfl-export`). |
||||
|
||||
When upgrading an existing installation, run `drush updatedb` to add new |
||||
settings. |
||||
|
||||
## What gets exported |
||||
|
||||
Every node and (optionally) every media entity becomes its own OCFL object. |
||||
A node object looks like this: |
||||
|
||||
``` |
||||
node.json Node metadata (see below) |
||||
media/{media uuid}/{filename} Files of each associated media |
||||
files/{field name}/{filename} Files in the node's own file/image fields |
||||
``` |
||||
|
||||
Associated media are: |
||||
|
||||
- media whose "media of" field (Islandora: `field_media_of`) points at the |
||||
node, and |
||||
- media referenced from any media reference field on the node (for example |
||||
`field_representative_image`). |
||||
|
||||
A standalone media object contains `node.json` (the media's metadata) and |
||||
`files/{field name}/{filename}`. Media thumbnails generated by Drupal are not |
||||
exported. |
||||
|
||||
### node.json |
||||
|
||||
In addition to the node's base properties (`uuid`, `nid`, `title`, `type`, |
||||
`created`, `changed`, `status`, `author`) and all of its `fields`, `node.json` |
||||
contains: |
||||
|
||||
- `relationships` |
||||
- `model`: the Islandora model term (`tid`, `uuid`, `name`, and its |
||||
external `uri` when present). |
||||
- `member_of`: the node's direct parents, each with `nid`, `uuid`, `title`, |
||||
`type` and `model`. |
||||
- `hierarchy`: every ancestor path through `member_of`, each ordered from |
||||
the root collection down to the direct parent. A node with several parents |
||||
has several paths. |
||||
- `media`: each associated media's full metadata (including |
||||
`field_media_use`), how it is associated with the node (`media_of` and/or |
||||
`field:<field name>`) and its `files`. |
||||
- `files`: the node's own files. |
||||
- `_metadata`: export information. `_metadata.missing_files` lists any file |
||||
that could not be read during export; the rest of the object is still saved. |
||||
|
||||
Each file entry gives its `logical_path` inside the object, plus `field`, |
||||
`fid`, `uuid`, `filename`, `filemime`, `filesize` and the original Drupal |
||||
`uri`. |
||||
|
||||
### Versions and deduplication |
||||
|
||||
Every export creates a new OCFL version. A file whose checksum (sha512) is |
||||
already in the object's manifest is not copied again: the new version simply |
||||
references the existing copy, so large files that don't change are stored |
||||
once. Each copied file is verified against its checksum. Checksums are cached |
||||
by file URI, size and modification time so unchanged files aren't re-read on |
||||
every export. |
||||
|
||||
Deleting a node or media creates a final version marked `"deleted": true`; the |
||||
earlier versions, including their files, are kept. |
||||
|
||||
## Storage layout |
||||
|
||||
The storage root (default `private://ocfl-storage`) uses the |
||||
[0003-hash-and-id-n-tuple-storage-layout](https://ocfl.github.io/extensions/0003-hash-and-id-n-tuple-storage-layout.html) |
||||
extension. Object IDs are `urn:uuid:{entity uuid}`, and an object is stored at |
||||
the first three 3-character tuples of the sha256 of its ID, followed by the |
||||
percent-encoded ID: |
||||
|
||||
``` |
||||
ocfl-storage/ |
||||
├── 0=ocfl_1.1 |
||||
├── ocfl_layout.json |
||||
├── extensions/0003-hash-and-id-n-tuple-storage-layout/config.json |
||||
└── b55/aed/a8b/urn%3auuid%3a11111111-2222-3333-4444-555555555555/ |
||||
├── 0=ocfl_object_1.1 |
||||
├── inventory.json |
||||
├── inventory.json.sha512 |
||||
├── v1/ |
||||
└── v2/ |
||||
``` |
||||
|
||||
To find an object on disk, search for its UUID: |
||||
|
||||
```sh |
||||
find /path/to/ocfl-storage -maxdepth 4 -name '*<uuid>*' |
||||
``` |
||||
|
||||
The module refuses to write into a storage root that declares a different |
||||
layout. |
||||
|
||||
## Configuration |
||||
|
||||
| Setting | Default | Description | |
||||
|---|---|---| |
||||
| Storage Root Path | `private://ocfl-storage` | Stream wrapper URI or absolute path of the storage root. | |
||||
| Enable automatic export on node save | on | Export nodes when they (or their media) are saved. | |
||||
| Content types to export | all | Limit automatic node export to these types. | |
||||
| Enable automatic export on media save | on | Also export each media as its own object. | |
||||
| Media types to export | all | Limit standalone media export to these types. | |
||||
| Copy files into OCFL objects | on | When off, only metadata is exported. | |
||||
| "Media of" field | `field_media_of` | Media field pointing at the owning node. | |
||||
| "Member of" field | `field_member_of` | Node field pointing at parent nodes. | |
||||
| Model field | `field_model` | Node field holding the model term. | |
||||
|
||||
Leave a field setting empty to disable that feature. |
||||
|
||||
**Islandora sites:** media binaries are already stored in their parent node's |
||||
object. Leaving standalone media export on stores them a second time, so you |
||||
will usually want to turn it off. |
||||
|
||||
## Exporting |
||||
|
||||
- **Automatically:** saving a node, or inserting, updating or deleting one of |
||||
its media, queues the node. Queued nodes are exported once when the request |
||||
ends, so saving a node together with several media creates one new version. |
||||
This also applies to drush commands. Islandora derivatives arrive in |
||||
separate requests, so each derivative creates a version of its parent node. |
||||
- **Manually:** the **Export to OCFL** button on a node or media page (requires |
||||
the *Export content to OCFL* permission). |
||||
- **In bulk:** the **Export to OCFL** action, available to Views bulk |
||||
operations on content and media listings. |
||||
|
||||
Users with the *Export content to OCFL* permission see the current export |
||||
version on node and media pages, with a link to view the latest `node.json`. |
||||
|
||||
## Permissions |
||||
|
||||
- *Administer OCFL Export*: change the module's settings. |
||||
- *Export content to OCFL*: export manually and view export status and JSON. |
||||
|
||||
## Limitations |
||||
|
||||
- Titles stored in `member_of` and `hierarchy` are those at export time. |
||||
Renaming a collection does not re-export its descendants; their records |
||||
update the next time each is exported. |
||||
- Version `user` entries contain only a name (no address), so OCFL validators |
||||
report the W008 recommendation. |
||||
|
||||
## License |
||||
|
||||
GPL-2.0-or-later. See [LICENSE](LICENSE). |
||||
@ -1,35 +0,0 @@
|
||||
{ |
||||
"name": "roblib/ocfl_export", |
||||
"description": "Exports Drupal nodes and media (Islandora aware) as OCFL (Oxford Common File Layout) objects, including their files and collection hierarchy.", |
||||
"type": "drupal-module", |
||||
"license": "GPL-2.0-or-later", |
||||
"keywords": [ |
||||
"Drupal", |
||||
"Islandora", |
||||
"OCFL", |
||||
"digital preservation" |
||||
], |
||||
"homepage": "https://git.library.upei.ca/Code/ocfl_export", |
||||
"support": { |
||||
"issues": "https://git.library.upei.ca/Code/ocfl_export/issues", |
||||
"source": "https://git.library.upei.ca/Code/ocfl_export" |
||||
}, |
||||
"authors": [ |
||||
{ |
||||
"name": "Robertson Library, University of Prince Edward Island", |
||||
"role": "Owner" |
||||
}, |
||||
{ |
||||
"name": "Paul Pound", |
||||
"role": "Maintainer" |
||||
} |
||||
], |
||||
"require": { |
||||
"php": ">=8.3", |
||||
"drupal/core": "^10.5" |
||||
}, |
||||
"suggest": { |
||||
"drupal/islandora": "Adds media discovered via field_media_of, collection hierarchy (field_member_of) and model information to node exports.", |
||||
"drupal/paragraphs": "Paragraph fields are serialized into the exported metadata when present." |
||||
} |
||||
} |
||||
@ -1,27 +0,0 @@
|
||||
<?php |
||||
|
||||
/** |
||||
* @file |
||||
* Install, update and uninstall functions for the OCFL Export module. |
||||
*/ |
||||
|
||||
declare(strict_types=1); |
||||
|
||||
/** |
||||
* Add settings for file export and Islandora relationship fields. |
||||
*/ |
||||
function ocfl_export_update_10001(): void { |
||||
$config = \Drupal::configFactory()->getEditable('ocfl_export.settings'); |
||||
$defaults = [ |
||||
'include_files' => TRUE, |
||||
'member_of_field' => 'field_member_of', |
||||
'media_of_field' => 'field_media_of', |
||||
'model_field' => 'field_model', |
||||
]; |
||||
foreach ($defaults as $key => $value) { |
||||
if ($config->get($key) === NULL) { |
||||
$config->set($key, $value); |
||||
} |
||||
} |
||||
$config->save(); |
||||
} |
||||
@ -1,390 +0,0 @@
|
||||
<?php |
||||
|
||||
declare(strict_types=1); |
||||
|
||||
namespace Drupal\ocfl_export; |
||||
|
||||
use Drupal\Core\Cache\Cache; |
||||
use Drupal\Core\Config\ConfigFactoryInterface; |
||||
use Drupal\Core\DestructableInterface; |
||||
use Drupal\Core\Entity\EntityFieldManagerInterface; |
||||
use Drupal\Core\Entity\EntityInterface; |
||||
use Drupal\Core\Entity\EntityTypeManagerInterface; |
||||
use Drupal\Core\Entity\FieldableEntityInterface; |
||||
use Drupal\Core\Logger\LoggerChannelFactoryInterface; |
||||
use Drupal\file\FileInterface; |
||||
use Drupal\media\MediaInterface; |
||||
use Drupal\node\NodeInterface; |
||||
use Psr\Log\LoggerInterface; |
||||
|
||||
/** |
||||
* Builds complete OCFL objects (metadata plus files) for nodes and media. |
||||
* |
||||
* A node object contains node.json plus the binaries of: |
||||
* - media that point at the node through the "media of" field (Islandora), |
||||
* stored under media/{media uuid}/; |
||||
* - media referenced from the node's own media reference fields, stored the |
||||
* same way; |
||||
* - files in the node's own file and image fields, stored under |
||||
* files/{field name}/. |
||||
* |
||||
* Automatic (hook driven) exports are queued and run once per node when the |
||||
* request ends, so saving a node together with several media creates a single |
||||
* new version rather than one per save. |
||||
*/ |
||||
class OcflExporter implements DestructableInterface { |
||||
|
||||
/** |
||||
* File field types whose files are copied into objects. |
||||
*/ |
||||
protected const FILE_FIELD_TYPES = ['file', 'image']; |
||||
|
||||
/** |
||||
* The logger channel. |
||||
*/ |
||||
protected LoggerInterface $logger; |
||||
|
||||
/** |
||||
* Node IDs queued for export at the end of the request. |
||||
* |
||||
* @var int[] |
||||
*/ |
||||
protected array $queue = []; |
||||
|
||||
public function __construct( |
||||
protected EntityTypeManagerInterface $entityTypeManager, |
||||
protected EntityFieldManagerInterface $entityFieldManager, |
||||
protected ConfigFactoryInterface $configFactory, |
||||
protected OcflStorageService $storage, |
||||
protected NodeSerializerService $nodeSerializer, |
||||
protected MediaSerializerService $mediaSerializer, |
||||
LoggerChannelFactoryInterface $loggerFactory, |
||||
) { |
||||
$this->logger = $loggerFactory->get('ocfl_export'); |
||||
} |
||||
|
||||
/** |
||||
* Export a node, its associated media and its files as one OCFL object. |
||||
* |
||||
* @param \Drupal\node\NodeInterface $node |
||||
* The node. |
||||
* @param string|null $message |
||||
* Optional OCFL version message. |
||||
* |
||||
* @return bool |
||||
* TRUE on success. |
||||
*/ |
||||
public function exportNode(NodeInterface $node, ?string $message = NULL): bool { |
||||
$data = $this->nodeSerializer->serialize($node); |
||||
$files = []; |
||||
|
||||
$data['files'] = $this->collectEntityFiles($node, 'files', TRUE, $files); |
||||
|
||||
$data['media'] = []; |
||||
foreach ($this->getAssociatedMedia($node) as $item) { |
||||
/** @var \Drupal\media\MediaInterface $media */ |
||||
$media = $item['media']; |
||||
$mediaData = $this->mediaSerializer->serialize($media); |
||||
unset($mediaData['_metadata']); |
||||
$mediaData['association'] = $item['association']; |
||||
$mediaData['files'] = $this->collectEntityFiles($media, 'media/' . $media->uuid(), FALSE, $files, ['thumbnail']); |
||||
$data['media'][] = $mediaData; |
||||
} |
||||
|
||||
if (!$this->includeFiles()) { |
||||
$files = []; |
||||
} |
||||
|
||||
return $this->save($node, $data, $files, $message); |
||||
} |
||||
|
||||
/** |
||||
* Export a media entity and its files as its own OCFL object. |
||||
* |
||||
* @param \Drupal\media\MediaInterface $media |
||||
* The media entity. |
||||
* @param string|null $message |
||||
* Optional OCFL version message. |
||||
* |
||||
* @return bool |
||||
* TRUE on success. |
||||
*/ |
||||
public function exportMedia(MediaInterface $media, ?string $message = NULL): bool { |
||||
$data = $this->mediaSerializer->serialize($media); |
||||
$files = []; |
||||
$data['files'] = $this->collectEntityFiles($media, 'files', TRUE, $files, ['thumbnail']); |
||||
|
||||
if (!$this->includeFiles()) { |
||||
$files = []; |
||||
} |
||||
|
||||
return $this->save($media, $data, $files, $message); |
||||
} |
||||
|
||||
/** |
||||
* Save an entity's OCFL object and refresh its rendered export status. |
||||
* |
||||
* @param \Drupal\Core\Entity\EntityInterface $entity |
||||
* The exported entity. |
||||
* @param array $data |
||||
* The metadata. |
||||
* @param array $files |
||||
* Map of logical path => file URI. |
||||
* @param string|null $message |
||||
* Optional OCFL version message. |
||||
* |
||||
* @return bool |
||||
* TRUE on success. |
||||
*/ |
||||
protected function save(EntityInterface $entity, array $data, array $files, ?string $message): bool { |
||||
$saved = $this->storage->saveObject($entity->uuid(), $data, $files, $message); |
||||
if ($saved) { |
||||
// The export status block is part of the cached entity render. |
||||
Cache::invalidateTags($entity->getCacheTagsToInvalidate()); |
||||
} |
||||
return $saved; |
||||
} |
||||
|
||||
/** |
||||
* Queue a node for automatic export if the settings allow it. |
||||
* |
||||
* @param \Drupal\node\NodeInterface $node |
||||
* The node. |
||||
*/ |
||||
public function queueNode(NodeInterface $node): void { |
||||
if ($this->shouldAutoExportNode($node)) { |
||||
$this->queue[(int) $node->id()] = (int) $node->id(); |
||||
} |
||||
} |
||||
|
||||
/** |
||||
* Queue the nodes a media entity belongs to (before and after the change). |
||||
* |
||||
* @param \Drupal\media\MediaInterface $media |
||||
* The media entity being inserted, updated or deleted. |
||||
*/ |
||||
public function queueParentNodes(MediaInterface $media): void { |
||||
$field = $this->configFactory->get('ocfl_export.settings')->get('media_of_field'); |
||||
if (!$field || !$media->hasField($field)) { |
||||
return; |
||||
} |
||||
|
||||
$parents = $media->get($field)->referencedEntities(); |
||||
$original = method_exists($media, 'getOriginal') ? $media->getOriginal() : ($media->original ?? NULL); |
||||
if ($original instanceof MediaInterface && $original->hasField($field)) { |
||||
$parents = array_merge($parents, $original->get($field)->referencedEntities()); |
||||
} |
||||
|
||||
foreach ($parents as $parent) { |
||||
if ($parent instanceof NodeInterface) { |
||||
$this->queueNode($parent); |
||||
} |
||||
} |
||||
} |
||||
|
||||
/** |
||||
* Remove a node from the export queue (e.g. because it was deleted). |
||||
* |
||||
* @param \Drupal\node\NodeInterface $node |
||||
* The node. |
||||
*/ |
||||
public function unqueueNode(NodeInterface $node): void { |
||||
unset($this->queue[(int) $node->id()]); |
||||
} |
||||
|
||||
/** |
||||
* {@inheritdoc} |
||||
*/ |
||||
public function destruct(): void { |
||||
if (!$this->queue) { |
||||
return; |
||||
} |
||||
|
||||
$nids = $this->queue; |
||||
$this->queue = []; |
||||
$nodes = $this->entityTypeManager->getStorage('node')->loadMultiple($nids); |
||||
|
||||
foreach ($nodes as $node) { |
||||
try { |
||||
if (!$this->exportNode($node)) { |
||||
$this->logger->error('Failed to export node @nid to OCFL.', ['@nid' => $node->id()]); |
||||
} |
||||
} |
||||
catch (\Exception $e) { |
||||
$this->logger->error('Failed to export node @nid to OCFL: @message', [ |
||||
'@nid' => $node->id(), |
||||
'@message' => $e->getMessage(), |
||||
]); |
||||
} |
||||
} |
||||
} |
||||
|
||||
/** |
||||
* Whether a node should be exported automatically. |
||||
* |
||||
* @param \Drupal\node\NodeInterface $node |
||||
* The node. |
||||
* |
||||
* @return bool |
||||
* TRUE if automatic export is enabled for the node's bundle. |
||||
*/ |
||||
public function shouldAutoExportNode(NodeInterface $node): bool { |
||||
$config = $this->configFactory->get('ocfl_export.settings'); |
||||
if (!$config->get('auto_export_enabled')) { |
||||
return FALSE; |
||||
} |
||||
$types = $config->get('content_types') ?: []; |
||||
return empty($types) || in_array($node->bundle(), $types, TRUE); |
||||
} |
||||
|
||||
/** |
||||
* Whether a media entity should be exported automatically. |
||||
* |
||||
* @param \Drupal\media\MediaInterface $media |
||||
* The media entity. |
||||
* |
||||
* @return bool |
||||
* TRUE if automatic export is enabled for the media's bundle. |
||||
*/ |
||||
public function shouldAutoExportMedia(MediaInterface $media): bool { |
||||
$config = $this->configFactory->get('ocfl_export.settings'); |
||||
if (!$config->get('auto_export_media_enabled')) { |
||||
return FALSE; |
||||
} |
||||
$types = $config->get('media_types') ?: []; |
||||
return empty($types) || in_array($media->bundle(), $types, TRUE); |
||||
} |
||||
|
||||
/** |
||||
* Find the media associated with a node. |
||||
* |
||||
* @param \Drupal\node\NodeInterface $node |
||||
* The node. |
||||
* |
||||
* @return array |
||||
* Keyed by media ID; each value has 'media' (the entity) and |
||||
* 'association' (a list such as ['media_of', 'field:field_image']). |
||||
*/ |
||||
protected function getAssociatedMedia(NodeInterface $node): array { |
||||
$associated = []; |
||||
$mediaStorage = $this->entityTypeManager->getStorage('media'); |
||||
|
||||
// Islandora: media point at the node they belong to. |
||||
$mediaOfField = $this->configFactory->get('ocfl_export.settings')->get('media_of_field'); |
||||
$mediaFieldStorage = $this->entityFieldManager->getFieldStorageDefinitions('media'); |
||||
if ($mediaOfField && isset($mediaFieldStorage[$mediaOfField])) { |
||||
$mids = $mediaStorage->getQuery() |
||||
->accessCheck(FALSE) |
||||
->condition($mediaOfField, $node->id()) |
||||
->sort('mid') |
||||
->execute(); |
||||
foreach ($mediaStorage->loadMultiple($mids) as $mid => $media) { |
||||
$associated[$mid] = ['media' => $media, 'association' => ['media_of']]; |
||||
} |
||||
} |
||||
|
||||
// Plain Drupal: the node references media from its own fields. |
||||
foreach ($node->getFieldDefinitions() as $fieldName => $definition) { |
||||
if ($definition->getType() !== 'entity_reference' || $definition->getSetting('target_type') !== 'media') { |
||||
continue; |
||||
} |
||||
foreach ($node->get($fieldName)->referencedEntities() as $media) { |
||||
$mid = (int) $media->id(); |
||||
$associated[$mid]['media'] = $media; |
||||
$associated[$mid]['association'][] = 'field:' . $fieldName; |
||||
} |
||||
} |
||||
|
||||
ksort($associated); |
||||
return $associated; |
||||
} |
||||
|
||||
/** |
||||
* Collect the files held in an entity's file and image fields. |
||||
* |
||||
* @param \Drupal\Core\Entity\FieldableEntityInterface $entity |
||||
* The entity. |
||||
* @param string $prefix |
||||
* Logical directory the files are placed under. |
||||
* @param bool $groupByField |
||||
* Whether to add a sub-directory per field. |
||||
* @param array $files |
||||
* Map of logical path => file URI, added to in place. |
||||
* @param string[] $skipFields |
||||
* Field names to ignore. |
||||
* |
||||
* @return array |
||||
* Descriptions of the files for inclusion in node.json. |
||||
*/ |
||||
protected function collectEntityFiles(FieldableEntityInterface $entity, string $prefix, bool $groupByField, array &$files, array $skipFields = []): array { |
||||
$descriptions = []; |
||||
|
||||
foreach ($entity->getFieldDefinitions() as $fieldName => $definition) { |
||||
if (!in_array($definition->getType(), static::FILE_FIELD_TYPES, TRUE) || in_array($fieldName, $skipFields, TRUE)) { |
||||
continue; |
||||
} |
||||
|
||||
foreach ($entity->get($fieldName) as $delta => $item) { |
||||
$file = $item->entity; |
||||
if (!$file instanceof FileInterface) { |
||||
continue; |
||||
} |
||||
|
||||
$dir = $groupByField ? $prefix . '/' . $fieldName : $prefix; |
||||
$logicalPath = $this->uniqueLogicalPath($dir, $file, $files); |
||||
$files[$logicalPath] = $file->getFileUri(); |
||||
|
||||
$descriptions[] = [ |
||||
'logical_path' => $this->includeFiles() ? $logicalPath : NULL, |
||||
'field' => $fieldName, |
||||
'delta' => $delta, |
||||
'fid' => (int) $file->id(), |
||||
'uuid' => $file->uuid(), |
||||
'filename' => $file->getFilename(), |
||||
'filemime' => $file->getMimeType(), |
||||
'filesize' => (int) $file->getSize(), |
||||
'uri' => $file->getFileUri(), |
||||
]; |
||||
} |
||||
} |
||||
|
||||
return $descriptions; |
||||
} |
||||
|
||||
/** |
||||
* Build a safe logical path that doesn't collide with one already used. |
||||
* |
||||
* @param string $dir |
||||
* The logical directory. |
||||
* @param \Drupal\file\FileInterface $file |
||||
* The file. |
||||
* @param array $files |
||||
* Logical paths already in use, as keys. |
||||
* |
||||
* @return string |
||||
* The logical path. |
||||
*/ |
||||
protected function uniqueLogicalPath(string $dir, FileInterface $file, array $files): string { |
||||
$name = preg_replace('/[\x00-\x1F\x7F\/\\\\]/u', '_', trim((string) $file->getFilename())); |
||||
if ($name === '' || $name === '.' || $name === '..') { |
||||
$name = 'file-' . $file->id(); |
||||
} |
||||
|
||||
$path = $dir . '/' . $name; |
||||
if (!isset($files[$path])) { |
||||
return $path; |
||||
} |
||||
|
||||
$info = pathinfo($name); |
||||
$extension = isset($info['extension']) ? '.' . $info['extension'] : ''; |
||||
return $dir . '/' . $info['filename'] . '-' . $file->id() . $extension; |
||||
} |
||||
|
||||
/** |
||||
* Whether binary files should be copied into objects. |
||||
*/ |
||||
protected function includeFiles(): bool { |
||||
return (bool) ($this->configFactory->get('ocfl_export.settings')->get('include_files') ?? TRUE); |
||||
} |
||||
|
||||
} |
||||
Loading…
Reference in new issue