From: Galen Charlton <gmc@equinoxOLI.org>
Date: Mon, 1 May 2023 16:20:15 +0000 (-0400)
Subject: LP#1979071: add release notes
X-Git-Url: https://old-git.evergreen-ils.org/?a=commitdiff_plain;h=4ca8920188e819eadf4614abc3c5f2d978725650;p=Evergreen.git

LP#1979071: add release notes

Signed-off-by: Galen Charlton <gmc@equinoxOLI.org>
---

diff --git a/docs/RELEASE_NOTES_NEXT/Architecture/queued-ingest.adoc b/docs/RELEASE_NOTES_NEXT/Architecture/queued-ingest.adoc
new file mode 100644
index 0000000000..1b6ba020fb
--- /dev/null
+++ b/docs/RELEASE_NOTES_NEXT/Architecture/queued-ingest.adoc
@@ -0,0 +1,96 @@
+== Queued Ingest ==
+
+This feature allows for the separation of bib and authority record
+updates and the search (and other) indexing that occurs when a record is
+modified in some way. Prior to this feature, bib and authority records
+would be indexed immediately upon an update.
+
+While individual record ingest has not become a problem with regard to
+system performance or interface usability, there exist several batch
+operations which aggregate many inserts or updates and whose aggregate
+ingest time cost can be significant.  These include, but are not
+limited to, reingest caused by authority control propagation, reingest
+required by the addition or modification of indexing configuration,
+cataloging and acquisitions record import and overlay from the staff
+interface, and upgrade-time reingest required by structural changes
+to the underlying indexing and search system.
+
+=== New Utility ===
+
+When Queued Ingest is enabled, a new control script, `ingest_ctl`, is
+available to perform several functions:
+
+* Run in the background to process the queues of indexing requests
+* Display statistics of queued ingest activity
+* Specify that a set of records should be reindexed.
+
+Here are some examples of how it is used:
+
+[source,bash]
+------------------------------------------------------------------------------------
+# Enqueue records 1-500000 for reingest later, just one worker for the queue
+/openils/bin/ingest_ctl --queue-threads 1
+    --queue-type biblio
+    --queue-run-at tomorrow
+    --queue-owner admin
+    --queue-name "slowly updating records due to new RDA attributes"
+    --start-id 1 --end-id 500000
+
+# Start the background worker
+/openils/bin/ingest_ctl --coordinator --max-child 20
+
+# Stop the background worker
+/openils/bin/ingest_ctl --coordinator --stop
+
+# Process whatever you can Right Now
+/openils/bin/ingest_ctl --max-child 20
+
+# Process a single queue Right Now
+/openils/bin/ingest_ctl --queue 1234 --max-child 20
+
+# Stats on Queued Ingest processing so far today
+/openils/bin/ingest_ctl --stats --since today --totals-only
+------------------------------------------------------------------------------------
+
+This script also requires the following switches (or environment
+variables) in order to connect to the database:
+
+* `--db_user` (or environment variable `PGUSER`)
+* `--db` (or environment variable `PGDATABASE`)
+* `--dbpw` (or environment variable `PGPASSWORD`)
+* `--db_port` (or environment variable `PGPORT`)
+
+=== New Settings ===
+
+This feature adds several new global flags:
+
+
+|===
+| Global Flag | Enabled
+
+| Queued Ingest: Abort transaction on ingest error rather than simply logging an error | no
+| Queued Ingest: Queue all bib record updates on authority change propagation, even if bib queuing is not generally enabled | no
+| Queued Ingest: Use Queued Ingest for bib record ingest on insert and undelete | no
+| Queued Ingest: Use Queued Ingest for authority record ingest on insert and undelete | no
+| Queued Ingest: Use Queued Ingest for bib record ingest on update | no
+| Queued Ingest: Use Queued Ingest for authority record ingest on update | no
+| Queued Ingest: Use Queued Ingest for bib record ingest on delete | no
+| Queued Ingest: Use Queued Ingest for authority record ingest on delete | no
+| Queued Ingest: Maximum number of database workers allowed for queued ingest processes | yes; default value 20
+| Queued Ingest: Use Queued Ingest for all bib record ingest | no
+| Queued Ingest: Use Queued Ingest for all bib and authority record ingest | no
+| Queued Ingest: Do NOT use Queued Ingest when creating a new bib, or undeleting a bib, via the MARC editor | yes
+| Queued Ingest: Use Queued Ingest for all authority record ingest | no
+| Queued Ingest: Do NOT Use Queued Ingest when editing bib records via the MARC Editor | yes
+|===
+
+This feature does not add any new library settings or permissions.
+
+=== Upgrade Notes ===
+
+Queued Ingest is not automatically turned on upon upgrade. to enable it,
+at minimum the following actions should be taken:
+
+* Enable at least the "Use Queued Ingest for all bib and authority record
+  ingest" global flag
+* Ensure that `ingest_ctl` is running with the `--coordinator` flag.