From bda478bc0ed56e5aab3833b2b0b74e84ef9914fd Mon Sep 17 00:00:00 2001 From: Galen Charlton Date: Tue, 30 Mar 2021 11:57:53 -0400 Subject: [PATCH] start 3.7 release notes Signed-off-by: Galen Charlton --- docs/RELEASE_NOTES_3_7.adoc | 673 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 673 insertions(+) create mode 100644 docs/RELEASE_NOTES_3_7.adoc diff --git a/docs/RELEASE_NOTES_3_7.adoc b/docs/RELEASE_NOTES_3_7.adoc new file mode 100644 index 0000000000..be30bf2b4c --- /dev/null +++ b/docs/RELEASE_NOTES_3_7.adoc @@ -0,0 +1,673 @@ +Evergreen 3.7-beta Release Notes +================================ +:toc: +:numbered: + +Upgrade notes +------------- + +New Features +------------ + + + +Administration +~~~~~~~~~~~~~~ + + + +Single Sign On (Shibboleth) OPAC integration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The Evergreen OPAC can now be used as a Service Provider (SP) in a +Single Sign On infrastructure. This allows system administrators to +connect the Evergreen OPAC to an identity provider (IdP). Such a scenario +offers significant usability improvements to patrons: + +* They can use the same, IdP-provided login screen and credentials that they +use for other applications (SPs). +* If they have already logged into another participating application, when +they arrive at the Evergreen OPAC, they can be logged in without needing to +enter any credentials at all. +* Evergreen can be configured to offer a Single Sign-out service, where +logging out of the Evergreen OPAC will also log the user out of all other SPs. + +It can also offer security benefits, if it enables a Shibboleth-enabled +Evergreen installation to move away from insecure autogenerated user passwords +(e.g. year of birth or last four digits of a phone number). + +Different Org Units can use different IdPs. This development also supports a +mix of Shibboleth and non-Shibboleth libraries. + +Note that only the OPAC can be integrated with Shibboleth at this time; no such +support exists for the staff client, self-check, etc. + +Also note that this development does not include automatic provisioning of +accounts. At this time, matching accounts must already exist in Evergreen +for a patron to successfully authenticate into the OPAC via Single Sign On. + +Installation +++++++++++++ + +Installing and configuring Shibboleth support is a complex project. In +broad strokes, the process includes: + +. Installing Shibboleth and the Shibboleth Apache module (`apt install libapache2-mod-shib2` on Debian and Ubuntu) +. Configuring Shibboleth, including: + * Setting up a certificate + * assigning an Entity ID + * getting metadata about the IdP from the IdP (perhaps "locally maintained + metadata", where an XML file from the IdP is copied into place on your + Evergreen server) + * Understanding what attributes the IdP will provide about your users, + describing those in the `attribute-map.xml` file. +. Providing your Entity ID, information about possible bindings, and any +other requested information to the IdP administrator. Much of this information +will be available at http://YOUR_EVERGREEN_DOMAIN/Shibboleth.sso/Metadata +. Configuring Apache, including: + * Enabling shibboleth authentication in the `eg_vhost.conf` file + * (Optional) Using the new _sso_loc_ Apache variable to identify + which org unit should be used as the context location when fetching + Shibboleth-related library settings. +. As a user with the new _SSO_ADMIN_ permission, configure Evergreen using +the Library Settings Editor, including: + * Enable Shibboleth SSO for the OPAC + * (Optional) Configure whether you will use SSO exclusively, or offer + patrons a choice between SSO and standard Evergreen authentication + * (Optional) Configure whether or not you will use Single Log Out + * (Optional) In scenarios where a single Evergreen installation is + connected to multiple IdPs, assign org units to the relevant IdPs, + referenced by the IdP's Entity Id. + * Of the attributes defined in `attribute-map.xml`, configure which one + should be used to match users in the Evergreen database. This defaults + to uid. + * For the attribute you chose in the previous step, configure which + Evergreen field it should match against. Options are usrname (default), + barcode, and email. + +This https://www.youtube.com/watch?v=SvppXbpv-5k[video on the SAML protocol] can +be very helpful for introducing the basic concepts used in the installation and +configuration processes. + + + + +Architecture +~~~~~~~~~~~~ + + + +Block Login of Expired Staff Accounts +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Evergreen now has the ability to prevent staff users whose +accounts have expired from logging in. This is controlled +by the new global flag "auth.block_expired_staff_login", which +is not enabled by default. If that flag is turned on, accounts +that have the `STAFF_LOGIN` permission and whose expiration date +is in the past are prevented from logging into any Evergreen +interface, including the staff client, the public catalog, and SIP2. + +It should be noted that ordinary patrons are allowed to log into +the public catalog if their circulation privileges have expired. This +feature prevents expired staff users from logging into the public catalog +(and all other Evergreen interfaces and APIs) outright in order to +prevent them from getting into the staff interface anyway by +creative use of Evergreen's authentication APIs. + +Evergreen admins are advised to check the expiration status of staff +accounts before turning on the global flag, as otherwise it is +possible to lock staff users out unexpectedly. The following SQL +query will identify expired but otherwise un-deleted users that +would be blocked by turning on the flag: + +[source,sql] +---- +SELECT DISTINCT usrname, expire_date +FROM actor.usr au, permission.usr_has_perm_at_all(id, 'STAFF_LOGIN') +WHERE active +AND NOT deleted +AND NOT barred +AND expire_date < NOW() +---- + +Note that this query can take a long time to run in large databases +given the general way that it checks for users that have the +`STAFF_LOGIN` permission. Replacing the use of +`permission.usr_has_perm_at_all()` with a query on expired users +with profiles known to have the `STAFF_LOGIN` permission will +be much faster. + + + + +Migration From GIST to GIN Indexes for Full Text Search +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Evergreen now uses GIN indexes for full text search in PostgreSQL. +GIN indexes offer better performance than GIST. For more information +on the differences in the two index types, please refer to the +https://www.postgresql.org/docs/current/textsearch-indexes.html[PostgreSQL +documentation]. + +An upgrade script is provided as part of this migration. If you +upgrade normally from a previous release of Evergreen, this upgrade +script should run as part of the upgrade process. The migration +script recommends that you run a `VACUUM ANALYZE` in Postgresql on the +tables that had the indexes changed. The migration process does not +do this for you, so you should do it as soon as is convenient after +the upgrade. + +Updating Your Own Indexes ++++++++++++++++++++++++++ + +If you have added your own full text indexes of type GIST, and you +wish to migrate them to GIN, you may do so. The following query, when +run in your Evergreen databsase after the migration from GIST to GIN, +will identify the remaining GIST indexes in your database: + +[source,sql] +---------------------------------------- +SELECT schemaname, indexname +FROM pg_indexes +WHERE indexdef ~* 'gist'; +---------------------------------------- + +If the above query produces output, you can run the next query to +output a SQL script to migrate the remaining indexes from GIST to GIN: + +[source,sql] +---------------------------------------- +SELECT 'DROP INDEX ' || schemaname || '.' || indexname || E';\n' || + REGEXP_REPLACE(indexdef, 'gist', 'gin', 'i') || E';\n' || + 'VACUUM ANAlYZE ' || schemaname || '.' || tablename || ';' +FROM pg_indexes +WHERE indexdef ~* 'gist'; +---------------------------------------- + + + + +Removal of Custom Dojo Build +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Evergreen had a +https://wiki.evergreen-ils.org/doku.php?id=scratchpad:random_magic_spells#custom_dojo_build[method +of making a custom build of the Dojo JavaScript library]. Followiing +this procedure could improve the load times for the OPAC and other +interfaces that use Dojo. However, very few sites took advantage of +this process or even knew of its existence. + +As a part of the process, an `openils_dojo.js` file was built and +installed along with the other Dojo files. Evergreen had many +references to load this optional file. For the majority of sites that +did not use this custom Dojo process, this file did not exist. +Browsers would spend time and resources requesting this nonexistent +file. This situation also contributed noise to the Apache logs with +the 404 errors from these requests. + +In keeping with the goal of eliminating Dojo from Evergreen, all +references to `openils_dojo.js` have been removed from the OPAC and +other files. The profile script required to make the custom Dojo +build has also been removed. + + + + + +Cataloging +~~~~~~~~~~ + + + +Czech language records in sample data +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This release adds 7 Czech-language MARC records to the +sample data set (also known as Concerto data set). + + + + + +Publisher Catalog Display Includes 264 Tag +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Publisher values are now extracted for dislpay from tags 260 OR 264. + +Upgrade Notes ++++++++++++++ + +A partial reingest is required to extract the new publisher data for display. +This query may be long-running. + +[source,sql] +-------------------------------------------------------------------------- +WITH affected_bibs AS ( + SELECT DISTINCT(bre.id) AS id + FROM biblio.record_entry bre + JOIN metabib.real_full_rec mrfr + ON (mrfr.record = bre.id AND mrfr.tag = '264') + WHERE NOT bre.deleted +) +SELECT metabib.reingest_metabib_field_entries(id, TRUE, FALSE, TRUE, TRUE) +FROM affected_bibs; +-------------------------------------------------------------------------- + + + + + +Circulation +~~~~~~~~~~~ + + + +Hold Groups +^^^^^^^^^^^ + +This feature allows staff to add multiple users to a named hold group +bucket and place title-level holds for a record for that entire set of users. +Users can be added to such a hold group bucket from either the patron +search result interface, via the Add to Bucket dropdown, or through a dedicated +Hold Group interface available from the Circulation menu. Adding new +patrons to a hold group bucket will require staff have the PLACE_HOLD +permission. + +Holds can be placed for the users in a hold group bucket either directly from +the normal staff-place hold interface in the embedded OPAC, or by supplying the +record ID within the hold group bucket interface. In the latter case, the +list of users for which a hold was attempted but failed to be placed can be +downloaded by staff in order to address any placement issues. Placing a +hold group bucket hold will requires staff have the MANAGE_HOLD_GROUPS +permission, which is new with this development. + +In the event of a mistaken hold group hold, staff with the MANAGE_HOLD_GROUPS +permission will have the ability to cancel all unfulfilled holds created as +part of a hold group event. + +A link to the title's hold interface is available from the list of hold group +events in the dedicated hold group interface. + + + + + +Scan Item as Missing Pieces Angular Port +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The 'Scan Item As Missing Pieces' interface is now an Angular interface. +The functionality is the same, but the interface displays more details +on the item in question (title/author/callnum) before proceeding with the +missing pieces process. + + + + +Opt-In Setting for Overdue and Predue Emails +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The "Receive Overdue and Courtesy Emails" user setting permits users to +control whether they receive email notifications about overdue items. + +To use the setting, modify any action trigger event definitions which +send emails about overdue items, setting the "Opt In Setting" to +"circ.default_overdue_notices_enabled" and the "User Field" to "usr". +You can accomplish this by running the following query in your database: + +---- +UPDATE action_trigger.event_definition +SET opt_in_setting = 'circ.default_overdue_notices_enabled', + usr_field = 'usr' +WHERE opt_in_setting IS NULL + AND hook = 'checkout.due' + AND reactor = 'SendEmail'; +---- + +Once this is done, the patron registration screen in the staff client +will show a "Receive Overdue and Courtesy Emails" checkbox, which will +be checked by default. To ensure that existing patrons continue to +recieve email notifications, you will need to add the user setting to +their accounts, which you can do by running the following query in your +database: + +---- +INSERT INTO actor.usr_setting (usr, name, value) +SELECT + id, + 'circ.default_overdue_notices_enabled', + 'true' +FROM actor.usr; +---- + + + + + + +Allow Circulation Renewal for Expired Patrons +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The "Allow renewal request if renewal recipient privileges have +expired" organizational unit setting can be set to true to permit +expired patrons to renew circulations. Allowing renewals for expired +patrons reduces the number of auto-renewal failures and assumes that a +patron with items out eligible for renewals has not been expired for +very long and that such patrons are likely to renew their privileges +in a timely manner. + +The setting is referenced based on the current circulation library for +the renewal. It takes into account the global flags for "Circ: Use +original circulation library on desk renewal instead of the +workstation library" and "Circ: Use original circulation library on +opac renewal instead of user home library." + + + + +OPAC +~~~~ + + + +Consistent Ordering for Carousels +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Carousel ordering is now stable and predictable: + + * Newly Cataloged Item and Newest Items by Shelving Location carousels are ordered from most recently cataloged to least recently cataloged. + * Recently Returned Item carousels is ordered is from most recently returned to least recently returned. + * Top Circulated Items carousels is ordered is from most circulated to least circulated. + * Manual carousels (as of now, without the ability to adjust the Position of items) are in the order they are added to the backing bucket. + ** Emptying and refilling the bucket allows reordering. + + + + + +Default Public Catalog to the Bootstrap Skin +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The public catalog now defaults to the Bootstrap skin rather than the +legacy TPAC skin. + +Bootstrap is now the default in order to encourage more testing, but +users should be aware of the following +https://bugs.launchpad.net/evergreen/+bugs?field.tag=bootstrap-blocker[issues]; +certain specific functionality is available only in the TPAC skin. + +The TPAC skin remains available for use, but current Evergreen users +should start actively considering migrating to the Bootstrap skin. + +In order to continue to use the TPAC skin, comment out the following +line in `eg_vhost.conf` + +[source,conf] +------------------- +PerlAddVar OILSWebTemplatePath "@localstatedir@/templates-bootstrap" # Comment this line out to use the legacy TPAC +------------------- + + + + +Did You Mean? Single word search suggestions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +This feature is the first in the series to add native search suggestions +to the Evergreen search logic. A significant portion of the code is +dedicated to infrastructure that will be used in later enhancements to +the functionality. + +Overview +++++++++ +When searching the public or staff catalog in a single search class (title, +author, subject, series, identifier, or keyword) with a single search term +users can be presented with alternate search terms. Depending on how the +instance has been configured, suggestions may be provided for only +misspelled words (as defined by existence in the bibliographic corpus), +terms that are spelled properly but occur very few times, or on every +single-term search. + +Settings +++++++++ + +The following new library settings control the behavior of the suggestions: + +* Maximum search result count at which spelling suggestions may be offered +* Minimum required uses of a spelling suggestions that may be offered +* Maximum number of spelling suggestions that may be offered +* Pg_trgm score weighting in OPAC spelling suggestions +* Soundex score weighting in OPAC spelling suggestions +* QWERTY Keyboard similarity score weighting in OPAC spelling suggestions + +There are also two new internal flags: + +* symspell.prefix_length +* symspell.max_edit_distance + +Upgrading ++++++++++ +This feature requires the addition of new Perl module dependencies. Please +run the app server and database server dependency Makefiles before applying +the database and code updates. + +At the end of the database upgrade script, the administrator is presented +with a set of instructions necessary to precompute the suggestion +dictionary based on the current bibliographic database. The first half +of this procedure can be started even before the upgrade begins, as soon +as the Evergreen database is no longer accessible to users that might +cause changes to bibliographic records. For very large instances, this +dictionary generation can take several hours and needs to be run on a +server with significant RAM and CPU resources. Please look at the upgrade +script before beginning an upgrade and plan this dictionary creation as +part of the overall upgrade procedure. + +Given a server, such as a database server with 64G of RAM, you should +be able to run all six of the shell commands in parallel in screen +sessions or with a tool such as GNU parallel. + +These commands invoke a script that will generate a class-specific sub- +set of the dictionary, and can be used to recreate the dictionary if +necessary in the future. + + + + + +Sort Holdings by Geographical Proximity +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This functionality integrates 3rd party geographic lookup services to allow patrons +to enter an address on the record details page in the OPAC and sort the holdings +for that record based on proximity of their circulating libraries to the entered +address. To support this, latitude and longitude coordinates may be associated with +each org unit. Care is given to not log or leak patron provided addresses or the +context in which they are used. + +Requires the following Perl modules: `Geo::Coder::Free`, `Geo::Coder::Google`, and `Geo::Coder::OSM` + + + + +Library Groups +^^^^^^^^^^^^^^ +The Library Groups search feature revives a longstanding internal +concept in Evergreen called “Lassos,” which allows an administrator +to define a group of organizational units for searching outside of +the standard organizational unit hierarchy. + +Use case examples include creating a group of law or science +libraries within a university consortium, or grouping all school +libraries together within a mixed school/public library. + +Searches can be restricted to a particular Library Group from the +library selector in the public catalog basic search page and from +the new "Where" selector on the advanced search page. + +Restricting catalog searches by Library Group is available only +in the public catalog and "traditional" staff catalog; it is not +available in the Angular staff catalog. + +This feature adds a new permission, `ADMIN_LIBRARY_GROUPS`, that +allows updating Library Groups and Library Group Maps. This permission +is not associated with any profiles by default, and replaces +the `CREATE_LASSO`, `UPDATE_LASSO`, and `DELETE_LASSO` permissions. + +To define new library groups, use the Server Administration Library +Groups and Library Group Maps pages. An autogen and a reload of +Apache should be performed after making changes to Library Groups. + + +Easier Styling of Public Catalog Logo and Cart Images +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Evergreen now has IDs associated with logos and cart images in the TPAC and Bootstrap OPACs to aid in customization. Images are as follows: + +* small Evergreen logo in navigation bar is 'topnav_logo_image' +* the large Evergreen logo in the center of the splash page of the TPAC is 'homesearch_main_logo_image' +* the cart icon is 'cart_icon_image' +* the small logo in the footer is 'footer_logo_image' + +The Bootstrap OPAC does not have a homesearch logo icon as it is added in the background by CSS and can be directly styled through the CSS. + + + + + +Easier TPAC Customization via colors.tt2 +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Twelve new colors for TPAC have been added to the colors.tt2 file as well as +having coresponding changes to the style.css.tt2 file. These use +descriptive rather than abstract names. These changes help avoid +situations were unreadable values are placed on top of each other +and where different values are wanted for elements that only refernece +a single color previously. Guidelines are below for setting values that +correspond to the previous values used in the colors.tt2 file. +For more diverse customizations the OPAC should be reviewed before +a production load. + +* 'footer' is used for the background color of the footer. It replaces the +'primary'. +* 'footer_text' sets the text color in the footer and replaces 'text_invert' +* 'header' sets the background of the header and replaces 'primary_fade' +* 'header_text' sets the color of text in the header and replaces 'text_invert' +* 'header_links_bar' sets the background of the links bar that separates the +header on the front page of the opac and replaces 'background_invert' +* 'header_links_text' sets the text on the links bar and replaces 'text_invert' +* 'header_links_text_hover' set the hover text color on the links bar and +replaces 'primary' +* 'opac_button' sets the background color of the My Opac button and replaces +'control' +* 'opac_button_text' explicitly sets the text color on the My Opac button +* 'opac_button_hover' sets the background color of the My Opac button when the +mouse is hovering over it and replaces 'primary' +* 'opac_button_hover_text' sets the text color of the My Opac button when the +mouse is hovering over it and replaces 'text invert' + +Note that is patch is primarily meant for users who wish to continue +using TPAC rather than the Bootstrap skin for a while; new Evergreen +users are advised to use the now-default Bootstrap skin. + + + + +Configurable Read More Accordion for OPAC Search and Record View (TPAC) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Read More Button +++++++++++++++++ +Public catalog record fields (in the TPAC skin only) now truncate +themselves based on a configurable amount of characters. The full +field may be displayed upon hitting a (Read More) link, which will +then toggle into a (Read Less) link to re-truncate the field. + +Configuration ++++++++++++++ +`Open-ILS/src/templates/opac/parts/config.tt2` contains two new +configuration variables: + + +* `truncate_contents` (default: 1) +* `contents_truncate_length` (default: 50). + +Setting `truncate_contents` to 0 will disable the read more +functionality. The variable `contents_truncate_length` corresponds +to the amount of characters to display before truncating the text. +If `contents_truncate_length` is removed, it will default to 100. + +Additional configuration for note fields can be made in +`Open-ILS/src/templates/opac/parts/record/contents.tt2`, allowing a +`trunc_length` variable for each individual type of note, which will +override `contents_truncate_length` for that specific +type of note. + + +Adding Read More Functionality to further fields +++++++++++++++++++++++++++++++++++++++++++++++++ +To add Read More functionality to any additional fields, you may use +the macro `accordion()`, defined in `misc_util.tt2`. It can take three +variables: `str`, `trunc_length`, and `element`. `str` corresponds to +the string you want to apply it to, `trunc_length` (optional) will +override `contents_truncate_length` if supplied, and `element` +(optional) provides an alternative HTML element to look at for the +truncation process (useful in situations such as the Authors and Cast +fields, where each field is processed individually, but needs to be +treated as a single field). + + + + +Reports +~~~~~~~ + + + +Reports Scheduler Improvements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Previously, the reports scheduler allowed duplicated reports +under certain circumstances. A uniqueness constraint now +disallows this without adversely affecting the reports process. + + + +Miscellaneous +------------- + + +* The 'Create Reservation' form in the Booking module now includes + an option to search for the patron by attributes other than just + their barcode. +* The form to add a user to a course now includes an option to search + for the patron by attributes other than just their barcode. +* For consistency with the menu action Cataloging => Retrieve Record by + TCN Value, the staff catalog Numeric Search => TCN search now includes + deleted bib records. +* Add a new command-line script, `overdrive-api-checker.pl`, for testing + the OverDrive API. +* The Shelving Location Groups editor is ported to Angular. +* The staff catalog now has the ability to add all search results (up to + 1,000 titles) to the basket in one fell swoop. +* Add 'All Videos' as a search format. +* Server-side print templates can now have print contexts set. +* Add ability to set the print context for a print template to "No-Print" + to specify, well, that a given receipt should never be printed. +* Add Check Number as an available column to the Bill History grids. +* Adds a new control to the item table in the TPAC public catalog only to + specify that only items that are available should be displayed. + +Acknowledgments +--------------- +The Evergreen project would like to acknowledge the following +organizations that commissioned developments in this release of +Evergreen: + +TODO + +We would also like to thank the following individuals who contributed +code, translations, documentations patches and tests to this release of +Evergreen: + +* Carmen Oleskevich + +We also thank the following organizations whose employees contributed +patches: + +TODO + +We regret any omissions. If a contributor has been inadvertently +missed, please open a bug at http://bugs.launchpad.net/evergreen/ +with a correction. + -- 2.11.0