--- /dev/null
+Evergreen 3.7-beta Release Notes
+================================
+:toc:
+:numbered:
+
+Upgrade notes
+-------------
+
+New Features
+------------
+
+
+
+Administration
+~~~~~~~~~~~~~~
+
+
+
+Single Sign On (Shibboleth) OPAC integration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The Evergreen OPAC can now be used as a Service Provider (SP) in a
+Single Sign On infrastructure. This allows system administrators to
+connect the Evergreen OPAC to an identity provider (IdP). Such a scenario
+offers significant usability improvements to patrons:
+
+* They can use the same, IdP-provided login screen and credentials that they
+use for other applications (SPs).
+* If they have already logged into another participating application, when
+they arrive at the Evergreen OPAC, they can be logged in without needing to
+enter any credentials at all.
+* Evergreen can be configured to offer a Single Sign-out service, where
+logging out of the Evergreen OPAC will also log the user out of all other SPs.
+
+It can also offer security benefits, if it enables a Shibboleth-enabled
+Evergreen installation to move away from insecure autogenerated user passwords
+(e.g. year of birth or last four digits of a phone number).
+
+Different Org Units can use different IdPs. This development also supports a
+mix of Shibboleth and non-Shibboleth libraries.
+
+Note that only the OPAC can be integrated with Shibboleth at this time; no such
+support exists for the staff client, self-check, etc.
+
+Also note that this development does not include automatic provisioning of
+accounts. At this time, matching accounts must already exist in Evergreen
+for a patron to successfully authenticate into the OPAC via Single Sign On.
+
+Installation
+++++++++++++
+
+Installing and configuring Shibboleth support is a complex project. In
+broad strokes, the process includes:
+
+. Installing Shibboleth and the Shibboleth Apache module (`apt install libapache2-mod-shib2` on Debian and Ubuntu)
+. Configuring Shibboleth, including:
+ * Setting up a certificate
+ * assigning an Entity ID
+ * getting metadata about the IdP from the IdP (perhaps "locally maintained
+ metadata", where an XML file from the IdP is copied into place on your
+ Evergreen server)
+ * Understanding what attributes the IdP will provide about your users,
+ describing those in the `attribute-map.xml` file.
+. Providing your Entity ID, information about possible bindings, and any
+other requested information to the IdP administrator. Much of this information
+will be available at http://YOUR_EVERGREEN_DOMAIN/Shibboleth.sso/Metadata
+. Configuring Apache, including:
+ * Enabling shibboleth authentication in the `eg_vhost.conf` file
+ * (Optional) Using the new _sso_loc_ Apache variable to identify
+ which org unit should be used as the context location when fetching
+ Shibboleth-related library settings.
+. As a user with the new _SSO_ADMIN_ permission, configure Evergreen using
+the Library Settings Editor, including:
+ * Enable Shibboleth SSO for the OPAC
+ * (Optional) Configure whether you will use SSO exclusively, or offer
+ patrons a choice between SSO and standard Evergreen authentication
+ * (Optional) Configure whether or not you will use Single Log Out
+ * (Optional) In scenarios where a single Evergreen installation is
+ connected to multiple IdPs, assign org units to the relevant IdPs,
+ referenced by the IdP's Entity Id.
+ * Of the attributes defined in `attribute-map.xml`, configure which one
+ should be used to match users in the Evergreen database. This defaults
+ to uid.
+ * For the attribute you chose in the previous step, configure which
+ Evergreen field it should match against. Options are usrname (default),
+ barcode, and email.
+
+This https://www.youtube.com/watch?v=SvppXbpv-5k[video on the SAML protocol] can
+be very helpful for introducing the basic concepts used in the installation and
+configuration processes.
+
+
+
+
+Architecture
+~~~~~~~~~~~~
+
+
+
+Block Login of Expired Staff Accounts
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Evergreen now has the ability to prevent staff users whose
+accounts have expired from logging in. This is controlled
+by the new global flag "auth.block_expired_staff_login", which
+is not enabled by default. If that flag is turned on, accounts
+that have the `STAFF_LOGIN` permission and whose expiration date
+is in the past are prevented from logging into any Evergreen
+interface, including the staff client, the public catalog, and SIP2.
+
+It should be noted that ordinary patrons are allowed to log into
+the public catalog if their circulation privileges have expired. This
+feature prevents expired staff users from logging into the public catalog
+(and all other Evergreen interfaces and APIs) outright in order to
+prevent them from getting into the staff interface anyway by
+creative use of Evergreen's authentication APIs.
+
+Evergreen admins are advised to check the expiration status of staff
+accounts before turning on the global flag, as otherwise it is
+possible to lock staff users out unexpectedly. The following SQL
+query will identify expired but otherwise un-deleted users that
+would be blocked by turning on the flag:
+
+[source,sql]
+----
+SELECT DISTINCT usrname, expire_date
+FROM actor.usr au, permission.usr_has_perm_at_all(id, 'STAFF_LOGIN')
+WHERE active
+AND NOT deleted
+AND NOT barred
+AND expire_date < NOW()
+----
+
+Note that this query can take a long time to run in large databases
+given the general way that it checks for users that have the
+`STAFF_LOGIN` permission. Replacing the use of
+`permission.usr_has_perm_at_all()` with a query on expired users
+with profiles known to have the `STAFF_LOGIN` permission will
+be much faster.
+
+
+
+
+Migration From GIST to GIN Indexes for Full Text Search
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Evergreen now uses GIN indexes for full text search in PostgreSQL.
+GIN indexes offer better performance than GIST. For more information
+on the differences in the two index types, please refer to the
+https://www.postgresql.org/docs/current/textsearch-indexes.html[PostgreSQL
+documentation].
+
+An upgrade script is provided as part of this migration. If you
+upgrade normally from a previous release of Evergreen, this upgrade
+script should run as part of the upgrade process. The migration
+script recommends that you run a `VACUUM ANALYZE` in Postgresql on the
+tables that had the indexes changed. The migration process does not
+do this for you, so you should do it as soon as is convenient after
+the upgrade.
+
+Updating Your Own Indexes
++++++++++++++++++++++++++
+
+If you have added your own full text indexes of type GIST, and you
+wish to migrate them to GIN, you may do so. The following query, when
+run in your Evergreen databsase after the migration from GIST to GIN,
+will identify the remaining GIST indexes in your database:
+
+[source,sql]
+----------------------------------------
+SELECT schemaname, indexname
+FROM pg_indexes
+WHERE indexdef ~* 'gist';
+----------------------------------------
+
+If the above query produces output, you can run the next query to
+output a SQL script to migrate the remaining indexes from GIST to GIN:
+
+[source,sql]
+----------------------------------------
+SELECT 'DROP INDEX ' || schemaname || '.' || indexname || E';\n' ||
+ REGEXP_REPLACE(indexdef, 'gist', 'gin', 'i') || E';\n' ||
+ 'VACUUM ANAlYZE ' || schemaname || '.' || tablename || ';'
+FROM pg_indexes
+WHERE indexdef ~* 'gist';
+----------------------------------------
+
+
+
+
+Removal of Custom Dojo Build
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Evergreen had a
+https://wiki.evergreen-ils.org/doku.php?id=scratchpad:random_magic_spells#custom_dojo_build[method
+of making a custom build of the Dojo JavaScript library]. Followiing
+this procedure could improve the load times for the OPAC and other
+interfaces that use Dojo. However, very few sites took advantage of
+this process or even knew of its existence.
+
+As a part of the process, an `openils_dojo.js` file was built and
+installed along with the other Dojo files. Evergreen had many
+references to load this optional file. For the majority of sites that
+did not use this custom Dojo process, this file did not exist.
+Browsers would spend time and resources requesting this nonexistent
+file. This situation also contributed noise to the Apache logs with
+the 404 errors from these requests.
+
+In keeping with the goal of eliminating Dojo from Evergreen, all
+references to `openils_dojo.js` have been removed from the OPAC and
+other files. The profile script required to make the custom Dojo
+build has also been removed.
+
+
+
+
+
+Cataloging
+~~~~~~~~~~
+
+
+
+Czech language records in sample data
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This release adds 7 Czech-language MARC records to the
+sample data set (also known as Concerto data set).
+
+
+
+
+
+Publisher Catalog Display Includes 264 Tag
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Publisher values are now extracted for dislpay from tags 260 OR 264.
+
+Upgrade Notes
++++++++++++++
+
+A partial reingest is required to extract the new publisher data for display.
+This query may be long-running.
+
+[source,sql]
+--------------------------------------------------------------------------
+WITH affected_bibs AS (
+ SELECT DISTINCT(bre.id) AS id
+ FROM biblio.record_entry bre
+ JOIN metabib.real_full_rec mrfr
+ ON (mrfr.record = bre.id AND mrfr.tag = '264')
+ WHERE NOT bre.deleted
+)
+SELECT metabib.reingest_metabib_field_entries(id, TRUE, FALSE, TRUE, TRUE)
+FROM affected_bibs;
+--------------------------------------------------------------------------
+
+
+
+
+
+Circulation
+~~~~~~~~~~~
+
+
+
+Hold Groups
+^^^^^^^^^^^
+
+This feature allows staff to add multiple users to a named hold group
+bucket and place title-level holds for a record for that entire set of users.
+Users can be added to such a hold group bucket from either the patron
+search result interface, via the Add to Bucket dropdown, or through a dedicated
+Hold Group interface available from the Circulation menu. Adding new
+patrons to a hold group bucket will require staff have the PLACE_HOLD
+permission.
+
+Holds can be placed for the users in a hold group bucket either directly from
+the normal staff-place hold interface in the embedded OPAC, or by supplying the
+record ID within the hold group bucket interface. In the latter case, the
+list of users for which a hold was attempted but failed to be placed can be
+downloaded by staff in order to address any placement issues. Placing a
+hold group bucket hold will requires staff have the MANAGE_HOLD_GROUPS
+permission, which is new with this development.
+
+In the event of a mistaken hold group hold, staff with the MANAGE_HOLD_GROUPS
+permission will have the ability to cancel all unfulfilled holds created as
+part of a hold group event.
+
+A link to the title's hold interface is available from the list of hold group
+events in the dedicated hold group interface.
+
+
+
+
+
+Scan Item as Missing Pieces Angular Port
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The 'Scan Item As Missing Pieces' interface is now an Angular interface.
+The functionality is the same, but the interface displays more details
+on the item in question (title/author/callnum) before proceeding with the
+missing pieces process.
+
+
+
+
+Opt-In Setting for Overdue and Predue Emails
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The "Receive Overdue and Courtesy Emails" user setting permits users to
+control whether they receive email notifications about overdue items.
+
+To use the setting, modify any action trigger event definitions which
+send emails about overdue items, setting the "Opt In Setting" to
+"circ.default_overdue_notices_enabled" and the "User Field" to "usr".
+You can accomplish this by running the following query in your database:
+
+----
+UPDATE action_trigger.event_definition
+SET opt_in_setting = 'circ.default_overdue_notices_enabled',
+ usr_field = 'usr'
+WHERE opt_in_setting IS NULL
+ AND hook = 'checkout.due'
+ AND reactor = 'SendEmail';
+----
+
+Once this is done, the patron registration screen in the staff client
+will show a "Receive Overdue and Courtesy Emails" checkbox, which will
+be checked by default. To ensure that existing patrons continue to
+recieve email notifications, you will need to add the user setting to
+their accounts, which you can do by running the following query in your
+database:
+
+----
+INSERT INTO actor.usr_setting (usr, name, value)
+SELECT
+ id,
+ 'circ.default_overdue_notices_enabled',
+ 'true'
+FROM actor.usr;
+----
+
+
+
+
+
+
+Allow Circulation Renewal for Expired Patrons
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The "Allow renewal request if renewal recipient privileges have
+expired" organizational unit setting can be set to true to permit
+expired patrons to renew circulations. Allowing renewals for expired
+patrons reduces the number of auto-renewal failures and assumes that a
+patron with items out eligible for renewals has not been expired for
+very long and that such patrons are likely to renew their privileges
+in a timely manner.
+
+The setting is referenced based on the current circulation library for
+the renewal. It takes into account the global flags for "Circ: Use
+original circulation library on desk renewal instead of the
+workstation library" and "Circ: Use original circulation library on
+opac renewal instead of user home library."
+
+
+
+
+OPAC
+~~~~
+
+
+
+Consistent Ordering for Carousels
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Carousel ordering is now stable and predictable:
+
+ * Newly Cataloged Item and Newest Items by Shelving Location carousels are ordered from most recently cataloged to least recently cataloged.
+ * Recently Returned Item carousels is ordered is from most recently returned to least recently returned.
+ * Top Circulated Items carousels is ordered is from most circulated to least circulated.
+ * Manual carousels (as of now, without the ability to adjust the Position of items) are in the order they are added to the backing bucket.
+ ** Emptying and refilling the bucket allows reordering.
+
+
+
+
+
+Default Public Catalog to the Bootstrap Skin
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The public catalog now defaults to the Bootstrap skin rather than the
+legacy TPAC skin.
+
+Bootstrap is now the default in order to encourage more testing, but
+users should be aware of the following
+https://bugs.launchpad.net/evergreen/+bugs?field.tag=bootstrap-blocker[issues];
+certain specific functionality is available only in the TPAC skin.
+
+The TPAC skin remains available for use, but current Evergreen users
+should start actively considering migrating to the Bootstrap skin.
+
+In order to continue to use the TPAC skin, comment out the following
+line in `eg_vhost.conf`
+
+[source,conf]
+-------------------
+PerlAddVar OILSWebTemplatePath "@localstatedir@/templates-bootstrap" # Comment this line out to use the legacy TPAC
+-------------------
+
+
+
+
+Did You Mean? Single word search suggestions
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+This feature is the first in the series to add native search suggestions
+to the Evergreen search logic. A significant portion of the code is
+dedicated to infrastructure that will be used in later enhancements to
+the functionality.
+
+Overview
+++++++++
+When searching the public or staff catalog in a single search class (title,
+author, subject, series, identifier, or keyword) with a single search term
+users can be presented with alternate search terms. Depending on how the
+instance has been configured, suggestions may be provided for only
+misspelled words (as defined by existence in the bibliographic corpus),
+terms that are spelled properly but occur very few times, or on every
+single-term search.
+
+Settings
+++++++++
+
+The following new library settings control the behavior of the suggestions:
+
+* Maximum search result count at which spelling suggestions may be offered
+* Minimum required uses of a spelling suggestions that may be offered
+* Maximum number of spelling suggestions that may be offered
+* Pg_trgm score weighting in OPAC spelling suggestions
+* Soundex score weighting in OPAC spelling suggestions
+* QWERTY Keyboard similarity score weighting in OPAC spelling suggestions
+
+There are also two new internal flags:
+
+* symspell.prefix_length
+* symspell.max_edit_distance
+
+Upgrading
++++++++++
+This feature requires the addition of new Perl module dependencies. Please
+run the app server and database server dependency Makefiles before applying
+the database and code updates.
+
+At the end of the database upgrade script, the administrator is presented
+with a set of instructions necessary to precompute the suggestion
+dictionary based on the current bibliographic database. The first half
+of this procedure can be started even before the upgrade begins, as soon
+as the Evergreen database is no longer accessible to users that might
+cause changes to bibliographic records. For very large instances, this
+dictionary generation can take several hours and needs to be run on a
+server with significant RAM and CPU resources. Please look at the upgrade
+script before beginning an upgrade and plan this dictionary creation as
+part of the overall upgrade procedure.
+
+Given a server, such as a database server with 64G of RAM, you should
+be able to run all six of the shell commands in parallel in screen
+sessions or with a tool such as GNU parallel.
+
+These commands invoke a script that will generate a class-specific sub-
+set of the dictionary, and can be used to recreate the dictionary if
+necessary in the future.
+
+
+
+
+
+Sort Holdings by Geographical Proximity
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This functionality integrates 3rd party geographic lookup services to allow patrons
+to enter an address on the record details page in the OPAC and sort the holdings
+for that record based on proximity of their circulating libraries to the entered
+address. To support this, latitude and longitude coordinates may be associated with
+each org unit. Care is given to not log or leak patron provided addresses or the
+context in which they are used.
+
+Requires the following Perl modules: `Geo::Coder::Free`, `Geo::Coder::Google`, and `Geo::Coder::OSM`
+
+
+
+
+Library Groups
+^^^^^^^^^^^^^^
+The Library Groups search feature revives a longstanding internal
+concept in Evergreen called “Lassos,” which allows an administrator
+to define a group of organizational units for searching outside of
+the standard organizational unit hierarchy.
+
+Use case examples include creating a group of law or science
+libraries within a university consortium, or grouping all school
+libraries together within a mixed school/public library.
+
+Searches can be restricted to a particular Library Group from the
+library selector in the public catalog basic search page and from
+the new "Where" selector on the advanced search page.
+
+Restricting catalog searches by Library Group is available only
+in the public catalog and "traditional" staff catalog; it is not
+available in the Angular staff catalog.
+
+This feature adds a new permission, `ADMIN_LIBRARY_GROUPS`, that
+allows updating Library Groups and Library Group Maps. This permission
+is not associated with any profiles by default, and replaces
+the `CREATE_LASSO`, `UPDATE_LASSO`, and `DELETE_LASSO` permissions.
+
+To define new library groups, use the Server Administration Library
+Groups and Library Group Maps pages. An autogen and a reload of
+Apache should be performed after making changes to Library Groups.
+
+
+Easier Styling of Public Catalog Logo and Cart Images
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Evergreen now has IDs associated with logos and cart images in the TPAC and Bootstrap OPACs to aid in customization. Images are as follows:
+
+* small Evergreen logo in navigation bar is 'topnav_logo_image'
+* the large Evergreen logo in the center of the splash page of the TPAC is 'homesearch_main_logo_image'
+* the cart icon is 'cart_icon_image'
+* the small logo in the footer is 'footer_logo_image'
+
+The Bootstrap OPAC does not have a homesearch logo icon as it is added in the background by CSS and can be directly styled through the CSS.
+
+
+
+
+
+Easier TPAC Customization via colors.tt2
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Twelve new colors for TPAC have been added to the colors.tt2 file as well as
+having coresponding changes to the style.css.tt2 file. These use
+descriptive rather than abstract names. These changes help avoid
+situations were unreadable values are placed on top of each other
+and where different values are wanted for elements that only refernece
+a single color previously. Guidelines are below for setting values that
+correspond to the previous values used in the colors.tt2 file.
+For more diverse customizations the OPAC should be reviewed before
+a production load.
+
+* 'footer' is used for the background color of the footer. It replaces the
+'primary'.
+* 'footer_text' sets the text color in the footer and replaces 'text_invert'
+* 'header' sets the background of the header and replaces 'primary_fade'
+* 'header_text' sets the color of text in the header and replaces 'text_invert'
+* 'header_links_bar' sets the background of the links bar that separates the
+header on the front page of the opac and replaces 'background_invert'
+* 'header_links_text' sets the text on the links bar and replaces 'text_invert'
+* 'header_links_text_hover' set the hover text color on the links bar and
+replaces 'primary'
+* 'opac_button' sets the background color of the My Opac button and replaces
+'control'
+* 'opac_button_text' explicitly sets the text color on the My Opac button
+* 'opac_button_hover' sets the background color of the My Opac button when the
+mouse is hovering over it and replaces 'primary'
+* 'opac_button_hover_text' sets the text color of the My Opac button when the
+mouse is hovering over it and replaces 'text invert'
+
+Note that is patch is primarily meant for users who wish to continue
+using TPAC rather than the Bootstrap skin for a while; new Evergreen
+users are advised to use the now-default Bootstrap skin.
+
+
+
+
+Configurable Read More Accordion for OPAC Search and Record View (TPAC)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Read More Button
+++++++++++++++++
+Public catalog record fields (in the TPAC skin only) now truncate
+themselves based on a configurable amount of characters. The full
+field may be displayed upon hitting a (Read More) link, which will
+then toggle into a (Read Less) link to re-truncate the field.
+
+Configuration
++++++++++++++
+`Open-ILS/src/templates/opac/parts/config.tt2` contains two new
+configuration variables:
+
+
+* `truncate_contents` (default: 1)
+* `contents_truncate_length` (default: 50).
+
+Setting `truncate_contents` to 0 will disable the read more
+functionality. The variable `contents_truncate_length` corresponds
+to the amount of characters to display before truncating the text.
+If `contents_truncate_length` is removed, it will default to 100.
+
+Additional configuration for note fields can be made in
+`Open-ILS/src/templates/opac/parts/record/contents.tt2`, allowing a
+`trunc_length` variable for each individual type of note, which will
+override `contents_truncate_length` for that specific
+type of note.
+
+
+Adding Read More Functionality to further fields
+++++++++++++++++++++++++++++++++++++++++++++++++
+To add Read More functionality to any additional fields, you may use
+the macro `accordion()`, defined in `misc_util.tt2`. It can take three
+variables: `str`, `trunc_length`, and `element`. `str` corresponds to
+the string you want to apply it to, `trunc_length` (optional) will
+override `contents_truncate_length` if supplied, and `element`
+(optional) provides an alternative HTML element to look at for the
+truncation process (useful in situations such as the Authors and Cast
+fields, where each field is processed individually, but needs to be
+treated as a single field).
+
+
+
+
+Reports
+~~~~~~~
+
+
+
+Reports Scheduler Improvements
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Previously, the reports scheduler allowed duplicated reports
+under certain circumstances. A uniqueness constraint now
+disallows this without adversely affecting the reports process.
+
+
+
+Miscellaneous
+-------------
+
+
+* The 'Create Reservation' form in the Booking module now includes
+ an option to search for the patron by attributes other than just
+ their barcode.
+* The form to add a user to a course now includes an option to search
+ for the patron by attributes other than just their barcode.
+* For consistency with the menu action Cataloging => Retrieve Record by
+ TCN Value, the staff catalog Numeric Search => TCN search now includes
+ deleted bib records.
+* Add a new command-line script, `overdrive-api-checker.pl`, for testing
+ the OverDrive API.
+* The Shelving Location Groups editor is ported to Angular.
+* The staff catalog now has the ability to add all search results (up to
+ 1,000 titles) to the basket in one fell swoop.
+* Add 'All Videos' as a search format.
+* Server-side print templates can now have print contexts set.
+* Add ability to set the print context for a print template to "No-Print"
+ to specify, well, that a given receipt should never be printed.
+* Add Check Number as an available column to the Bill History grids.
+* Adds a new control to the item table in the TPAC public catalog only to
+ specify that only items that are available should be displayed.
+
+Acknowledgments
+---------------
+The Evergreen project would like to acknowledge the following
+organizations that commissioned developments in this release of
+Evergreen:
+
+TODO
+
+We would also like to thank the following individuals who contributed
+code, translations, documentations patches and tests to this release of
+Evergreen:
+
+* Carmen Oleskevich
+
+We also thank the following organizations whose employees contributed
+patches:
+
+TODO
+
+We regret any omissions. If a contributor has been inadvertently
+missed, please open a bug at http://bugs.launchpad.net/evergreen/
+with a correction.
+