first commit

README.md

@@ -0,0 +1,173 @@
+# Solr sort: ignore leading articles
+
+Adds the `string_sort_ignore_articles` Solr field type and uses it for the
+Search API sort field(s) listed in `solr_sort_ignore_articles.settings`, so
+"The Matrix" sorts under M and "!Introduction" sorts under I.
+
+## Requirements
+
+* Drupal 10 or 11
+* `search_api_solr` 4.3 or later
+* Write access to the `conf` directory of the Solr core, and the ability to
+  reload the core
+
+## How it works
+
+`search_api_solr` never sorts on the field you index. For every string and
+fulltext field it copies the first value into a companion field named
+`sort_;<langcode>_<field id>` (encoded as `sort_X3b_en_title`), and sorting
+resolves to that field. Those companion fields are matched by the dynamic
+field `sort_X3b_<langcode>_*`, which is typed as the language's ICU collation
+— one collation per language, with no per-field override.
+
+This module subscribes to `PostConfigFilesGenerationEvent` and appends an
+explicit `<field>` for the fields you name to `schema_extra_fields.xml`, plus
+the field type to `schema_extra_types.xml`. In Solr an explicit field always
+wins over a matching `dynamicField`, so only the named fields get the new
+analysis and every other sort keeps the default collation.
+
+The module never talks to Solr itself. It only changes what
+`search-api-solr:get-server-config` generates — deploying that to the core is
+a manual step.
+
+## Deployment
+
+### 1. Install and enable
+
+```bash
+composer require roblib/solr_sort_ignore_articles
+drush en solr_sort_ignore_articles
+```
+
+### 2. Add a sort field to the index
+
+At `/admin/config/search/search-api/index/<index>/fields`, add *Content »
+Title* a second time, change its machine name to `title_sort` and set its type
+to **String**. In config it looks like this:
+
+```yaml
+# search_api.index.<index>.yml, under field_settings:
+title_sort:
+  label: 'Title (sort, ignore articles)'
+  datasource_id: 'entity:node'
+  property_path: title
+  type: string
+  dependencies:
+    module:
+      - node
+```
+
+Using a separate field keeps sorting on the existing `title` field unchanged.
+To change `title` itself instead, put `title` in
+`solr_sort_ignore_articles.settings:fields` and skip this step.
+
+The module ships with `title_sort` as the default. To override:
+
+```bash
+drush config:set solr_sort_ignore_articles.settings fields.0 title_sort
+drush config:set solr_sort_ignore_articles.settings fields.1 field_alt_title
+```
+
+Export config, and remember that any later change here means repeating
+steps 3 and 4.
+
+### 3. Update the Solr schema
+
+Generate the config set, matching the Solr version actually running on the
+target server:
+
+```bash
+drush search-api-solr:get-server-config <server> /tmp/solr_config.zip <solr-version>
+```
+
+Sanity check the two generated files before deploying:
+
+```bash
+unzip -p /tmp/solr_config.zip schema_extra_types.xml  | grep -A3 string_sort_ignore_articles
+unzip -p /tmp/solr_config.zip schema_extra_fields.xml | grep 'sort_X3b_.*_title_sort'
+```
+
+Then deploy to the core's `conf` directory by one of two routes.
+
+**Full replacement** — only when the core belongs to this site alone and the
+config set was generated for the same Solr version the server runs:
+
+```bash
+unzip -o /tmp/solr_config.zip -d /path/to/solr/server/solr/<core>/conf
+```
+
+**Hand-append ** — required when the core is shared with other
+Drupal sites, or when the target runs a different Solr major version than the
+config set was generated for. A full replacement would discard other sites'
+additions, and a config set for the wrong Solr version brings an incompatible
+`solrconfig.xml` along with it.
+
+Append the module's block to the end of each of the two files in the core's
+`conf` directory. Both files are bare XML fragments with no root element, and
+field types do not have to precede the fields that use them.
+
+`schema_extra_types.xml`:
+
+```xml
+<!-- solr_sort_ignore_articles -->
+<fieldType name="string_sort_ignore_articles" class="solr.TextField" sortMissingLast="true" omitNorms="true">
+  <analyzer>
+    <tokenizer class="solr.KeywordTokenizerFactory"/>
+    <filter class="solr.PatternReplaceFilterFactory" pattern="^[^\p{L}\p{N}]+" replacement="" replace="all"/>
+    <filter class="solr.PatternReplaceFilterFactory" pattern="^(?i)(a\s+|an\s+|the\s+)" replacement="" replace="all"/>
+    <filter class="solr.PatternReplaceFilterFactory" pattern="^[^\p{L}\p{N}]+" replacement="" replace="all"/>
+    <filter class="solr.LowerCaseFilterFactory"/>
+    <filter class="solr.TrimFilterFactory"/>
+  </analyzer>
+</fieldType>
+```
+
+`schema_extra_fields.xml`, one line per language, copied from the generated
+zip:
+
+```xml
+<!-- solr_sort_ignore_articles: these win over the matching sort_X3b_LANG_* dynamic fields -->
+<field name="sort_X3b_en_title_sort" type="string_sort_ignore_articles" indexed="true" stored="false" multiValued="false"/>
+<field name="sort_X3b_und_title_sort" type="string_sort_ignore_articles" indexed="true" stored="false" multiValued="false"/>
+<field name="sort_X3b_zxx_title_sort" type="string_sort_ignore_articles" indexed="true" stored="false" multiValued="false"/>
+```
+
+Keep the comment markers — they are what makes the block findable and cleanly
+removable later.
+
+Both blocks load unchanged on Solr 8.11 and 9.x.
+
+Reload the core:
+
+```bash
+curl -sS "http://<solr-host>:8983/solr/admin/cores?action=RELOAD&core=<core>"
+```
+
+A malformed fragment fails the reload with a schema parse error and the core
+keeps its previous schema, so this is safe to retry.
+
+### 4. Reindex
+
+The sort field is only populated at index time, so existing documents keep
+their old values until reindexed:
+
+```bash
+drush search-api:reset-tracker && drush search-api:index
+```
+
+### 5. Add the sort criterion
+
+In Views, add the new sort criterion (*Title (sort, ignore articles)*) to the
+search view.
+
+## Verifying
+
+At `/solr/#/<core>/analysis`, field type `string_sort_ignore_articles` with
+input `The !Matrix` should produce the single token `matrix`.
+
+The field itself can be confirmed with the Schema API:
+
+```bash
+curl -sS "http://<solr-host>:8983/solr/<core>/schema/fields/sort_X3b_en_title_sort"
+```
+

composer.json

@@ -0,0 +1,23 @@
+{
+    "name": "roblib/solr_sort_ignore_articles",
+    "description": "Adds a Solr field type that strips leading punctuation and the articles \"a\", \"an\" and \"the\", and wires it to the Search API sort field(s) named in solr_sort_ignore_articles.settings.",
+    "type": "drupal-custom-module",
+    "license": "GPL-2.0-or-later",
+    "keywords": [
+        "drupal",
+        "solr",
+        "search_api",
+        "search_api_solr",
+        "sorting"
+    ],
+    "homepage": "https://git.library.upei.ca/Code/solr_sort_ignore_articles",
+    "support": {
+        "issues": "https://git.library.upei.ca/Code/solr_sort_ignore_articles/issues",
+        "source": "https://git.library.upei.ca/Code/solr_sort_ignore_articles"
+    },
+    "require": {
+        "php": ">=8.1",
+        "drupal/core": "^10 || ^11",
+        "drupal/search_api_solr": "^4.3"
+    }
+}

config/install/solr_sort_ignore_articles.settings.yml

@@ -0,0 +1,5 @@
+# Search API field IDs whose Solr sort field should ignore leading articles.
+# Every field listed here must exist on the index as a "string" or fulltext
+# field, otherwise Solr has nothing to sort on.
+fields:
+  - title_sort

config/schema/solr_sort_ignore_articles.schema.yml

@@ -0,0 +1,10 @@
+solr_sort_ignore_articles.settings:
+  type: config_object
+  label: 'Solr sort: ignore leading articles settings'
+  mapping:
+    fields:
+      type: sequence
+      label: 'Search API field IDs to override the sort field for'
+      sequence:
+        type: string
+        label: 'Search API field ID'

solr_sort_ignore_articles.info.yml

@@ -0,0 +1,7 @@
+name: 'Solr sort: ignore leading articles'
+type: module
+description: 'Adds a Solr field type that strips leading punctuation and the articles "a", "an" and "the", and wires it to the Search API sort field(s) named in solr_sort_ignore_articles.settings.'
+package: Search
+core_version_requirement: ^10 || ^11
+dependencies:
+  - search_api_solr:search_api_solr

solr_sort_ignore_articles.services.yml

@@ -0,0 +1,8 @@
+services:
+  solr_sort_ignore_articles.config_set_subscriber:
+    class: Drupal\solr_sort_ignore_articles\EventSubscriber\SolrConfigSetSubscriber
+    arguments:
+      - '@config.factory'
+      - '@language_manager'
+    tags:
+      - { name: event_subscriber }

src/EventSubscriber/SolrConfigSetSubscriber.php

@@ -0,0 +1,147 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Drupal\solr_sort_ignore_articles\EventSubscriber;
+
+use Drupal\Core\Config\ConfigFactoryInterface;
+use Drupal\Core\Language\LanguageInterface;
+use Drupal\Core\Language\LanguageManagerInterface;
+use Drupal\search_api_solr\Event\PostConfigFilesGenerationEvent;
+use Drupal\search_api_solr\SolrBackendInterface;
+use Drupal\search_api_solr\Utility\Utility;
+use Symfony\Component\EventDispatcher\EventSubscriberInterface;
+
+/**
+ * Adds an article-insensitive sort field type to the generated config set.
+ *
+ * search_api_solr always sorts string and fulltext fields through a dedicated
+ * "sort_;<langcode>_<field id>" field, which is matched by the dynamic field
+ * "sort_;<langcode>_*" and therefore typed as the language's ICU collation.
+ * There is no per-field hook for that, so we append an explicit <field> for
+ * the fields we care about: in Solr an explicit field always wins over a
+ * matching dynamicField.
+ *
+ * @see \Drupal\search_api_solr\Utility\Utility::getSortableSolrField()
+ * @see \Drupal\search_api_solr\Entity\SolrFieldType::getCollatedField()
+ */
+class SolrConfigSetSubscriber implements EventSubscriberInterface {
+
+  /**
+   * The name of the Solr field type added to schema_extra_types.xml.
+   */
+  public const FIELD_TYPE = 'string_sort_ignore_articles';
+
+  public function __construct(
+    protected ConfigFactoryInterface $configFactory,
+    protected LanguageManagerInterface $languageManager,
+  ) {}
+
+  /**
+   * {@inheritdoc}
+   */
+  public static function getSubscribedEvents(): array {
+    return [
+      PostConfigFilesGenerationEvent::class => 'onPostConfigFilesGeneration',
+    ];
+  }
+
+  /**
+   * Appends the field type and the sort field overrides to the config set.
+   */
+  public function onPostConfigFilesGeneration(PostConfigFilesGenerationEvent $event): void {
+    $fields = $this->getFieldIds();
+    if (!$fields) {
+      return;
+    }
+
+    $files = $event->getConfigFiles();
+    if (!isset($files['schema_extra_types.xml'], $files['schema_extra_fields.xml'])) {
+      // Not a Drupal schema.xml based config set, nothing to extend.
+      return;
+    }
+
+    $files['schema_extra_types.xml'] .= "\n" . $this->getFieldTypeXml();
+    $files['schema_extra_fields.xml'] .= "\n" . $this->getSortFieldsXml($fields);
+    $event->setConfigFiles($files);
+  }
+
+  /**
+   * Returns the Search API field IDs to override the sort field for.
+   *
+   * Duplicates are removed: listing a field twice would emit two <field>
+   * elements with the same name, and Solr refuses to load a core with a
+   * duplicate field definition.
+   *
+   * @return string[]
+   *   The field IDs.
+   */
+  protected function getFieldIds(): array {
+    $fields = $this->configFactory
+      ->get('solr_sort_ignore_articles.settings')
+      ->get('fields');
+
+    return array_values(array_unique(array_filter((array) $fields)));
+  }
+
+  /**
+   * Returns the <fieldType> definition for schema_extra_types.xml.
+   */
+  protected function getFieldTypeXml(): string {
+    $xml = <<<'XML'
+<!-- solr_sort_ignore_articles -->
+<fieldType name="FIELD_TYPE_NAME" class="solr.TextField" sortMissingLast="true" omitNorms="true">
+  <analyzer>
+    <!-- 1. Keep the entire string as a single token. -->
+    <tokenizer class="solr.KeywordTokenizerFactory"/>
+    <!-- 2. Strip leading punctuation, symbols and spaces.
+         [^\p{L}\p{N}] is "anything that is NOT a letter or a number". -->
+    <filter class="solr.PatternReplaceFilterFactory" pattern="^[^\p{L}\p{N}]+" replacement="" replace="all"/>
+    <!-- 3. Remove a leading 'A ', 'An ' or 'The ' (case-insensitive). -->
+    <filter class="solr.PatternReplaceFilterFactory" pattern="^(?i)(a\s+|an\s+|the\s+)" replacement="" replace="all"/>
+    <!-- 4. Strip leading punctuation again, to catch "The !Matrix". -->
+    <filter class="solr.PatternReplaceFilterFactory" pattern="^[^\p{L}\p{N}]+" replacement="" replace="all"/>
+    <!-- 5. Lowercase so 'Matrix' and 'matrix' sort together. -->
+    <filter class="solr.LowerCaseFilterFactory"/>
+    <!-- 6. Trim any accidental leading/trailing whitespace. -->
+    <filter class="solr.TrimFilterFactory"/>
+  </analyzer>
+</fieldType>
+
+XML;
+
+    return str_replace('FIELD_TYPE_NAME', static::FIELD_TYPE, $xml);
+  }
+
+  /**
+   * Returns the explicit <field> overrides for schema_extra_fields.xml.
+   *
+   * The sort fields have to be filled for every language the backend indexes
+   * for, so we override all of them.
+   *
+   * @param string[] $fields
+   *   The Search API field IDs.
+   */
+  protected function getSortFieldsXml(array $fields): string {
+    $language_ids = array_keys($this->languageManager->getLanguages(LanguageInterface::STATE_ALL));
+    $language_ids[] = LanguageInterface::LANGCODE_NOT_SPECIFIED;
+    $language_ids = array_unique($language_ids);
+
+    $xml = "<!-- solr_sort_ignore_articles: these win over the matching sort_X3b_LANG_* dynamic fields -->\n";
+    foreach ($fields as $field_id) {
+      foreach ($language_ids as $language_id) {
+        // A TextField cannot carry docValues, so unlike the collated dynamic
+        // fields these have to be indexed. The KeywordTokenizer guarantees a
+        // single token per document, which is what makes them sortable.
+        $xml .= sprintf(
+          '<field name="%s" type="%s" indexed="true" stored="false" multiValued="false"/>' . "\n",
+          Utility::encodeSolrName('sort' . SolrBackendInterface::SEARCH_API_SOLR_LANGUAGE_SEPARATOR . $language_id . '_' . $field_id),
+          static::FIELD_TYPE
+        );
+      }
+    }
+
+    return $xml;
+  }
+
+}