From: dbwells Date: Mon, 18 Oct 2010 14:05:06 +0000 (+0000) Subject: Catch up to trunk phase 2 (the rest) -- Merged revisions 17889,17891-17892,17894... X-Git-Url: https://old-git.evergreen-ils.org/?a=commitdiff_plain;h=793acec21fc27ee7b082f902210ac2d9f36dc658;p=Evergreen.git Catch up to trunk phase 2 (the rest) -- Merged revisions 17889,17891-17892,17894,17896-17897,17899,17901,17903,17905-17906,17908,17910,17912-17915,17917-17918,17922,17927,17929,17931,17933,17935,17937-17939,17941,17944-17946,17948,17950,17952,17954,17956-17958,17963,17965,17972,17974,17976,17978,17980,17982,17984-17985,17987,17989,17991-17992,17994-17995,17997,17999,18001-18007,18009,18011,18013,18015-18017,18021,18023,18025,18027,18029,18034,18037,18039,18041,18043,18045,18047,18049,18051,18054-18058,18062,18068,18072-18073,18075-18077,18079,18081,18083-18085,18089,18091,18093,18095,18097,18099,18101,18103,18105-18106,18109,18111-18113,18116,18121,18123-18124,18126,18128,18130,18132-18133,18136,18138,18140-18141,18146,18148,18150-18151,18153,18158-18159,18163,18165,18167,18169,18171-18172,18175,18177,18179-18180,18182-18185,18187-18188,18191-18193,18195-18196,18198-18199,18202-18203,18205,18207,18210-18211,18213,18215,18217,18219-18220,18222,18224,18226,18228,18230-18231,18233,18239-18240,18244-18245,18247,18249,18251,18253,18255-18256,18258-18262,18264,18269,18271-18272,18274,18277-18282,18285-18291,18293-18294,18298-18300,18303,18310-18313,18321-18323,18326,18329,18331,18333,18336,18342-18343,18349,18351,18353,18364-18366 via svnmerge from svn://svn.open-ils.org/ILS/trunk ........ r17889 | scottmk | 2010-09-22 11:59:06 -0400 (Wed, 22 Sep 2010) | 8 lines For action.circulation and action.aged_circulation: add indexes on the target_copy column. This change was made in upgrade script # 0017, but was apparently never applied to the base install script. M Open-ILS/src/sql/Pg/090.schema.action.sql ........ r17891 | scottmk | 2010-09-22 15:16:26 -0400 (Wed, 22 Sep 2010) | 7 lines Adding a trigger. Upgrade # 0364 created the trigger function but not the trigger itself. However the base install script 040.schema.asset.sql creates both the function and the trigger. M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0414.schema.call-number-upd-ins-trigger.sql ........ r17892 | phasefx | 2010-09-22 15:31:07 -0400 (Wed, 22 Sep 2010) | 1 line Holdings Maintenance used to make too many assumptions about what a decent org hierarchy looked like, and had these rules to suppress some of the fetch-render-happy behavior that existed when clicking on org units. We can remove them now, and this allows volumes to render when owned by the top of the org tree ........ r17894 | phasefx | 2010-09-22 15:37:43 -0400 (Wed, 22 Sep 2010) | 2 lines fixes a bug in Item Status -> Alternate View when the item being viewed has no circ modifier ........ r17896 | phasefx | 2010-09-22 16:01:55 -0400 (Wed, 22 Sep 2010) | 1 line fix from tsbere to handle case where the print strategy is set to custom/external but the actual command is missing (perhaps due to xulrunner upgrades?) ........ r17897 | phasefx | 2010-09-22 16:01:59 -0400 (Wed, 22 Sep 2010) | 1 line set label printer context for label printing ........ r17899 | senator | 2010-09-22 20:14:03 -0400 (Wed, 22 Sep 2010) | 3 lines Hopefully fix a holds list sorting issue that only manifested when printing. ........ r17901 | senator | 2010-09-22 20:34:22 -0400 (Wed, 22 Sep 2010) | 2 lines Another IDL chunking fix for web/templates-based interfaces ........ r17903 | miker | 2010-09-22 22:12:02 -0400 (Wed, 22 Sep 2010) | 1 line only flesh up to, but not including, the leaf for the group_field ........ r17905 | miker | 2010-09-22 22:49:36 -0400 (Wed, 22 Sep 2010) | 1 line Use a transaction to avoid talking to a replicated db when building A/T data structures ........ r17906 | dbs | 2010-09-22 23:09:19 -0400 (Wed, 22 Sep 2010) | 7 lines Remove most UI annoyances from authority management interface * term input field now gets the focus automatically * pressing enter in most places submits a new search * removed the onBlur event as that required users to click a second time to open the action menu ........ r17908 | dbs | 2010-09-22 23:18:14 -0400 (Wed, 22 Sep 2010) | 6 lines Browse through 20 authority records at a time in management interface The default browse list is set to 9 elements, but we have enough vertical space to make use of more. Perhaps we should check the font size and viewport height and then adjust accordingly, but that would be Hard. ........ r17910 | dbs | 2010-09-22 23:45:16 -0400 (Wed, 22 Sep 2010) | 10 lines Remove most annoying UI "feature" of the new authority browse list interface When I added the new authority browse list interface, I used dojo.xhrGet() to retrieve records from the authority browse backend - but because I didn't specify sync:true, when you first right-clicked on a subfield, the function would return immediately and default to showing the context menu. You would then need to click two more times to show the authority list. Now you get it on your first right-click, as it should be. ........ r17912 | miker | 2010-09-23 00:33:39 -0400 (Thu, 23 Sep 2010) | 1 line Have the CStoreEditor grow a DESTROY ........ r17913 | erickson | 2010-09-23 01:31:03 -0400 (Thu, 23 Sep 2010) | 1 line event firing util code needs to run in a xact ........ r17914 | erickson | 2010-09-23 03:17:00 -0400 (Thu, 23 Sep 2010) | 1 line fetch last updated event before comitting to stay in the xact ........ r17915 | scottmk | 2010-09-23 09:45:06 -0400 (Thu, 23 Sep 2010) | 8 lines Dropping and recreating a foreign key constraint for config.metabib_field, in order to change its name. WHen this foreign key was first introduced, the upgrade script gave it one name and the base install script gave it a different name. Here we bring the names into sync. M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0415.schema.rename-field-class-fkey.sql ........ r17917 | scottmk | 2010-09-23 10:31:32 -0400 (Thu, 23 Sep 2010) | 5 lines Rename a couple of indexes (recuring -> recurring) M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0416.schema.rename-recuring-idx.sql ........ r17918 | scottmk | 2010-09-23 11:22:29 -0400 (Thu, 23 Sep 2010) | 12 lines Drop the never-used column item_count from acq.lineitem. Drop it also from the associated history table, and rebuild the function that maintains it. Finally, rebuild the associated lifecycle view. Apply to trunk only; this column never existed in 2.0. The column has already been removed from the base installation script. M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0417.schema.acq.drop-lineitem-item-count.sql ........ r17922 | miker | 2010-09-23 11:38:46 -0400 (Thu, 23 Sep 2010) | 11 lines Forward-port of a patch from Steve Callendar, via James Fournie, via launchpad: When the patron.password.use_phone is set, new patrons are created with their password set to the last 4 digits of their phone number, HOWEVER, when a patron's password is reset, it does not work properly. Although the little underlined summary shows the proper 4 digits, the password box displays 9-ish digits, and is not the last 4 digits of the password. The attached patch was created by Steve Callender and is confirmed working on 1.6.0 ORIGINAL CAVEAT: This patch will not work on 2.0 as that has a new user editor, but it would presumably be worthwhile to verify this functionality works in that editor as well. ED NOTE: Because it is possible to use the old editor -- it still exists -- this patch should be applied. It does what it advertises to do, which is fix the old editor to follow the "use phone number" OU setting. ........ r17927 | phasefx | 2010-09-23 12:35:20 -0400 (Thu, 23 Sep 2010) | 2 lines disable holds Detail View button until the asynchronously loading details UI is ready ........ r17929 | scottmk | 2010-09-23 14:58:36 -0400 (Thu, 23 Sep 2010) | 5 lines esolving various discrepancies between a freshly built 2.0 database and an upgraded one. M Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql ........ r17931 | erickson | 2010-09-23 16:47:47 -0400 (Thu, 23 Sep 2010) | 1 line repaired search call for user_setting. cstoreeditor uses the fieldmapper name, sos/usr/user/ ........ r17933 | senator | 2010-09-23 16:59:13 -0400 (Thu, 23 Sep 2010) | 2 lines IE hates the dangling comma (sometimes). ........ r17935 | erickson | 2010-09-23 19:01:06 -0400 (Thu, 23 Sep 2010) | 1 line I can has authoritative for selfcheck receipts ........ r17937 | miker | 2010-09-23 19:38:10 -0400 (Thu, 23 Sep 2010) | 1 line Stopping the leak ........ r17938 | miker | 2010-09-23 19:47:49 -0400 (Thu, 23 Sep 2010) | 1 line reverting previous. sorry, folks ........ r17939 | miker | 2010-09-23 19:49:26 -0400 (Thu, 23 Sep 2010) | 1 line Stopping the leak (let us try this again) ........ r17941 | senator | 2010-09-23 20:15:31 -0400 (Thu, 23 Sep 2010) | 2 lines Re-commit miker's PermaCrud changes, just a tad more nicer ;-) ........ r17944 | scottmk | 2010-09-23 20:56:21 -0400 (Thu, 23 Sep 2010) | 10 lines Apply some fixes that were earlier applied to the base installation script 090.schema.action.sql, but not in an upgrade script. Also: correct a typo (INTEVAL -> INTERVAL). Hence this upgrade needs to go to v2.0 as well as to trunk. M Open-ILS/src/sql/Pg/090.schema.action.sql M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0418.function.action.fix-purge-circ.sql ........ r17945 | miker | 2010-09-23 21:46:41 -0400 (Thu, 23 Sep 2010) | 1 line correct the comment ........ r17946 | miker | 2010-09-23 21:47:06 -0400 (Thu, 23 Sep 2010) | 1 line use _session_request in RO calls ........ r17948 | phasefx | 2010-09-23 22:36:55 -0400 (Thu, 23 Sep 2010) | 2 lines We're starting to get events where ilsevent == empty string now, not just null or a number. ........ r17950 | senator | 2010-09-24 00:45:42 -0400 (Fri, 24 Sep 2010) | 3 lines Be more tolerant of long-running A/T event handling for holds pull list printing ........ r17952 | phasefx | 2010-09-24 01:48:39 -0400 (Fri, 24 Sep 2010) | 2 lines this call returns an array; fixes surprise stat cats in patron display ........ r17954 | gmc | 2010-09-24 12:37:23 -0400 (Fri, 24 Sep 2010) | 9 lines improve hold targetting * all potential capturable copies are now checked (up to the first one that is permitted for the request), instead of a small random subset of them * don't do redundant permission checks Signed-off-by: Galen Charlton ........ r17956 | miker | 2010-09-24 13:45:49 -0400 (Fri, 24 Sep 2010) | 1 line excise the no-potentials check, should only effect frozen holds ........ r17957 | miker | 2010-09-24 13:56:31 -0400 (Fri, 24 Sep 2010) | 1 line put the frozen (and also empty issuance or last-copy-removed) check back in as a conditional select if we found nothing mapped ........ r17958 | phasefx | 2010-09-24 14:04:59 -0400 (Fri, 24 Sep 2010) | 2 lines don't styled the juvenile indicator in the patron display based on age calculations by default, just go by the juvenile flag on the user. local CSS can restore the behavior if desired ........ r17963 | miker | 2010-09-24 14:08:05 -0400 (Fri, 24 Sep 2010) | 1 line cut-n-paste-o ... removing bad "my" ........ r17965 | phasefx | 2010-09-24 14:23:10 -0400 (Fri, 24 Sep 2010) | 2 lines spawn item attribute editor instead of item notes UI during process missing pieces workflow ........ r17972 | erickson | 2010-09-24 15:30:10 -0400 (Fri, 24 Sep 2010) | 1 line removed extra {}'s from where clause ........ r17974 | senator | 2010-09-24 16:11:12 -0400 (Fri, 24 Sep 2010) | 2 lines Fix an autogrid create dialog of the kind that gets big, unusable and jumpy ........ r17976 | miker | 2010-09-24 16:29:43 -0400 (Fri, 24 Sep 2010) | 1 line Remove confusing older targeter version; move status trimming to earlier in the process ........ r17978 | erickson | 2010-09-24 18:23:58 -0400 (Fri, 24 Sep 2010) | 1 line clean up the fire_object_events cstore xact ........ r17980 | erickson | 2010-09-24 19:04:45 -0400 (Fri, 24 Sep 2010) | 1 line more transaction cleanups ........ r17982 | senator | 2010-09-24 20:56:24 -0400 (Fri, 24 Sep 2010) | 2 lines Maybe not abandon poor little cstore so much. ........ r17984 | miker | 2010-09-24 20:58:35 -0400 (Fri, 24 Sep 2010) | 1 line pedantic protection of cstore backends -- always use die_event when in xact mode, and rollback otherwise ........ r17985 | senator | 2010-09-24 21:13:31 -0400 (Fri, 24 Sep 2010) | 2 lines paranoia ........ r17987 | erickson | 2010-09-24 23:27:20 -0400 (Fri, 24 Sep 2010) | 1 line events don't have granularity's. removing ........ r17989 | senator | 2010-09-24 23:45:21 -0400 (Fri, 24 Sep 2010) | 2 lines How did this get lost? PermaCrud authoritative might work now? ........ r17991 | erickson | 2010-09-24 23:46:03 -0400 (Fri, 24 Sep 2010) | 1 line no need to start a xact in the holds pull list. really long lists can results in sending a rollback on a timed-out cstore handle, resulting in errors ........ r17992 | senator | 2010-09-24 23:54:09 -0400 (Fri, 24 Sep 2010) | 2 lines fix misplaced paranoia ........ r17994 | erickson | 2010-09-25 00:09:57 -0400 (Sat, 25 Sep 2010) | 1 line only need to wrap event retrieve in xact for latest copy. otherwise, run the risk of xact-ed cstore timing out ........ r17995 | gmc | 2010-09-25 22:42:50 -0400 (Sat, 25 Sep 2010) | 6 lines don't leak cstores if CStoreEditor rollback fails Patch by Mike Rylander. Signed-off-by: Galen Charlton ........ r17997 | gmc | 2010-09-25 22:49:39 -0400 (Sat, 25 Sep 2010) | 10 lines more selfcheck receipt transaction hackery Instead of having authoritative versions of open-ils.circ.fire_*_trigger_events, wrap just the target retrieval itself in a transaction. Avoids as yet unexplained rollback failure that occurs if processing a selfcheck receipt with more than a few items on it. Signed-off-by: Galen Charlton ........ r17999 | phasefx | 2010-09-25 23:17:05 -0400 (Sat, 25 Sep 2010) | 2 lines make address_type required in the patron editor, otherwise a blank value sends a null to the db where it silently fails ........ r18001 | phasefx | 2010-09-26 02:34:22 -0400 (Sun, 26 Sep 2010) | 2 lines Show cover art in Z39.50 client. I'm slow, so it only hit me this year that we can show added content for material not actually in the catalog yet ........ r18002 | phasefx | 2010-09-26 03:05:58 -0400 (Sun, 26 Sep 2010) | 2 lines make the splitters in the z39.50 interface sticky ........ r18003 | phasefx | 2010-09-26 04:06:33 -0400 (Sun, 26 Sep 2010) | 2 lines make all the splitters sticky with oils_persist ........ r18004 | phasefx | 2010-09-26 05:23:43 -0400 (Sun, 26 Sep 2010) | 2 lines Call persist_helper() in most interfaces. Give it the ability to handle resizing windows and set some windows up so that their height, width, and maximized state persist ........ r18005 | phasefx | 2010-09-26 06:02:38 -0400 (Sun, 26 Sep 2010) | 2 lines make the MARC editor optional with Z39.50 ........ r18006 | phasefx | 2010-09-26 06:29:36 -0400 (Sun, 26 Sep 2010) | 2 lines unmark record for overlay after it is overlaid ........ r18007 | miker | 2010-09-26 11:31:41 -0400 (Sun, 26 Sep 2010) | 1 line allow more complex facet ordering ........ r18009 | miker | 2010-09-26 11:53:21 -0400 (Sun, 26 Sep 2010) | 1 line Only show facetOrder limited facets that have values ........ r18011 | gmc | 2010-09-26 12:26:21 -0400 (Sun, 26 Sep 2010) | 4 lines added missing columns to CDBI table definitions Signed-off-by: Galen Charlton ........ r18013 | dbs | 2010-09-26 14:34:47 -0400 (Sun, 26 Sep 2010) | 2 lines Remove a debugging statement that slipped in ........ r18015 | erickson | 2010-09-26 19:20:10 -0400 (Sun, 26 Sep 2010) | 1 line push copy location order update into ML method. return updated orders from method to prevent replication issues. update UI JS to match ........ r18016 | erickson | 2010-09-26 19:20:11 -0400 (Sun, 26 Sep 2010) | 1 line no need to warn when emails may just be disabled ........ r18017 | erickson | 2010-09-26 19:20:12 -0400 (Sun, 26 Sep 2010) | 1 line honor SIP return date as the circ backdate ........ r18021 | gmc | 2010-09-26 21:43:12 -0400 (Sun, 26 Sep 2010) | 4 lines avoid multiple clicks of selfcheck logout link Signed-off-by: Galen Charlton ........ r18023 | scottmk | 2010-09-27 08:45:57 -0400 (Mon, 27 Sep 2010) | 5 lines Resolve differences between stored procedures in a freshly installed 2.0 database and those in an upgraded one. M Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql ........ r18025 | miker | 2010-09-27 11:23:10 -0400 (Mon, 27 Sep 2010) | 1 line Provide for limiting the number of classOrder elements to a specific number. Skips those without facet values. ........ r18027 | miker | 2010-09-27 12:15:14 -0400 (Mon, 27 Sep 2010) | 1 line Adjust hold-to-record view to cover I, F and R hold types ........ r18029 | phasefx | 2010-09-27 12:29:19 -0400 (Mon, 27 Sep 2010) | 2 lines since we're enforcing the requiredness of address type, let's give it a default value (since empty strings don't work with these widgets) ........ r18034 | miker | 2010-09-27 14:26:29 -0400 (Mon, 27 Sep 2010) | 1 line use transaction when gathering records from a newly created queue; adjust rollback usage in import loop ........ r18037 | gmc | 2010-09-27 14:50:56 -0400 (Mon, 27 Sep 2010) | 4 lines fix default path to selfcheck sounds Signed-off-by: Galen Charlton ........ r18039 | miker | 2010-09-27 15:11:21 -0400 (Mon, 27 Sep 2010) | 1 line make pubdate sorting on search faster by pre-munging date1 and date2 into an acceptable sortkey ........ r18041 | miker | 2010-09-27 15:16:57 -0400 (Mon, 27 Sep 2010) | 1 line Correct the subquery for A/T opt_in_setting ........ r18043 | miker | 2010-09-27 15:24:13 -0400 (Mon, 27 Sep 2010) | 1 line missed in the previous commit, sorry folks ........ r18045 | gmc | 2010-09-27 15:27:04 -0400 (Mon, 27 Sep 2010) | 7 lines fix glitch in hold target OU weighting We want to weight by the copy's circulation library, not the pickup library. Signed-off-by: Galen Charlton ........ r18047 | gmc | 2010-09-27 15:48:33 -0400 (Mon, 27 Sep 2010) | 4 lines fix another hold targeting glitch Signed-off-by: Galen Charlton ........ r18049 | gmc | 2010-09-27 16:14:49 -0400 (Mon, 27 Sep 2010) | 7 lines adjustments to pubdate sorting patch * handle upgrade for date1/date2 containing the empty string * syntax error fix Signed-off-by: Galen Charlton ........ r18051 | senator | 2010-09-27 16:54:56 -0400 (Mon, 27 Sep 2010) | 2 lines Special overriden widgets need forcible population when used in edit dialogs ........ r18054 | scottmk | 2010-09-27 17:31:17 -0400 (Mon, 27 Sep 2010) | 12 lines 1. Turn some ints into bigints. 2. Rename the uniqueness constraint for booking.resource_type. M Open-ILS/src/sql/Pg/090.schema.action.sql M Open-ILS/src/sql/Pg/200.schema.acq.sql M Open-ILS/src/sql/Pg/012.schema.vandelay.sql M Open-ILS/src/sql/Pg/095.schema.booking.sql M Open-ILS/src/sql/Pg/002.schema.config.sql M Open-ILS/src/sql/Pg/070.schema.container.sql A Open-ILS/src/sql/Pg/upgrade/0421.schema.embiggen-ints.sql ........ r18055 | atz | 2010-09-27 21:22:40 -0400 (Mon, 27 Sep 2010) | 3 lines For edi_translator on RHEL Note: RubyGems 1.3.4 is not available anymore ........ r18056 | atz | 2010-09-27 21:22:41 -0400 (Mon, 27 Sep 2010) | 1 line Fix POD for RemoteAccount ........ r18057 | atz | 2010-09-27 21:24:20 -0400 (Mon, 27 Sep 2010) | 1 line Minor cleanup ........ r18058 | atz | 2010-09-27 21:26:07 -0400 (Mon, 27 Sep 2010) | 1 line typo in POD ........ r18062 | dbs | 2010-09-27 23:08:48 -0400 (Mon, 27 Sep 2010) | 5 lines Use quoted attribute values to make Firefox / XULRunner happier about dojo queries The unquoted values in BibTemplate and the OPAC detail template were generating mucho noise in the JavaScript console; this hushes it up nicely. ........ r18068 | dbs | 2010-09-27 23:32:28 -0400 (Mon, 27 Sep 2010) | 11 lines Add a placeholder server/skin/custom.js to prevent one error in JS console custom.js enables you to override the settings in the stock constants.js to skin the behaviour of your staff client at your installation, without being subject to grief at upgrade time. Not having a custom.js file at all was, however, causing errors to be generated in the JS console as XULRunner tried hard to parse the 404 message as JavaScript. I suppose one could have changed the 404 message to valid JavaScript... naw, that would be evil. ........ r18072 | scottmk | 2010-09-28 01:13:32 -0400 (Tue, 28 Sep 2010) | 6 lines Turn an int into a bigint in acq.acq_lineitem_history, following up on a similar change to acq.lineitem. M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0422.schema.acq.lineitem-history-bigint.sql ........ r18073 | scottmk | 2010-09-28 01:17:03 -0400 (Tue, 28 Sep 2010) | 4 lines Incorporate several recent upgrade scripts, through # 0422. M Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql ........ r18075 | dbs | 2010-09-28 02:18:07 -0400 (Tue, 28 Sep 2010) | 6 lines Silence SQL warnings from O:A:Storage:Publisher:metabib PostgreSQL wants strings containing regular expressions to be prefixed with E; this patch considerably reduces the noise in open-ils.storage_unix.log by complying with PostgreSQL's wishes. ........ r18076 | dbs | 2010-09-28 02:43:41 -0400 (Tue, 28 Sep 2010) | 13 lines Make authority validation rules match authority ingest rules for better matches This resolves a problem in O:A:Storage:Publisher:authority:validate_tag() where the full NACO normalization rules that are applied to the subfields of the authority records when they are ingested into authority.full_rec are not similarly applied to the incoming subfields of the bib field that is being validated; only diacritic characters in the bib field subfields were being normalized. Now we apply naco_normalize() to the search terms so that they will match the ingested form of the authority record. Addresses https://bugs.launchpad.net/evergreen/+bug/649556 ........ r18077 | miker | 2010-09-28 12:35:24 -0400 (Tue, 28 Sep 2010) | 1 line add support for the null() xpath function, which works in pgxml (AKA xml2) but not in the builtin XPATH function for 8.3+ ........ r18079 | dbs | 2010-09-28 12:40:02 -0400 (Tue, 28 Sep 2010) | 7 lines Change memcached default location to 127.0.0.1 from localhost Debian Squeeze currently doesn't fare well with memcached servers pointing to localhost and needs an explicit 127.0.0.1, whereas Ubuntu Lucid is happy with either. Changing the default here means one less possible gotcha in the out of the box install & configure experience for Debian Squeeze folk. ........ r18081 | phasefx | 2010-09-28 12:46:34 -0400 (Tue, 28 Sep 2010) | 1 line this breaks (hides) the buttonbar in windows. Sorry James, miker ........ r18083 | erickson | 2010-09-28 14:12:12 -0400 (Tue, 28 Sep 2010) | 1 line repaired logic bug in lineitem worksheet template ........ r18084 | erickson | 2010-09-28 14:12:13 -0400 (Tue, 28 Sep 2010) | 5 lines ability to override checkin events. minor fixes. repaired bug that caused checkin to sometimes fail when a hold was captured for a user that had at least 1 inactive card; made checkin override events configurable; minor cleanup ........ r18085 | erickson | 2010-09-28 14:12:14 -0400 (Tue, 28 Sep 2010) | 1 line for troubleshooting, log the size of the template output to be stored ........ r18089 | senator | 2010-09-28 16:26:09 -0400 (Tue, 28 Sep 2010) | 5 lines A/T: Send an early response back to the client when running all pending events The client may wish to know when the relativelty fast process of creating the events is over, and the relatively slow process of validating them and running their reactors/cleanup/etc is about to begin. ........ r18091 | miker | 2010-09-28 16:45:17 -0400 (Tue, 28 Sep 2010) | 1 line move the early-out response so we avoid breaking the caller ........ r18093 | phasefx | 2010-09-28 17:12:11 -0400 (Tue, 28 Sep 2010) | 2 lines Delete all cookies on logoff. There's dojo code that looks for stale session cookies and has the xul client prompt for a new session. When you logoff, the session is destroyed, but the stale cookies were being left behind (and not overwritten on login? I don't understand that part). But this fixes it ........ r18095 | miker | 2010-09-28 19:12:58 -0400 (Tue, 28 Sep 2010) | 1 line Use transactions everywhere in Vandelay ........ r18097 | miker | 2010-09-29 10:45:50 -0400 (Wed, 29 Sep 2010) | 1 line Process item imports during record import, not as a secondary call (never should have worked, but for transaction timing) ........ r18099 | miker | 2010-09-29 11:05:53 -0400 (Wed, 29 Sep 2010) | 1 line support per-grunlarity parallelizing via flag and lock file ........ r18101 | gmc | 2010-09-29 13:48:52 -0400 (Wed, 29 Sep 2010) | 10 lines reporter: don't try to write Excel formulas Any cell value that starts with = is now always written as a text cell in spreadsheet output. Avoids a possible exploit as well as errors like this: Couldn't parse formula: = at /openils/bin/clark-kent.pl line 429 Signed-off-by: Galen Charlton ........ r18103 | gmc | 2010-09-29 14:07:43 -0400 (Wed, 29 Sep 2010) | 4 lines fix FM type of reporter.report.data Signed-off-by: Galen Charlton ........ r18105 | miker | 2010-09-29 16:16:41 -0400 (Wed, 29 Sep 2010) | 1 line Give BibTemplate the ability to inspect and optionally parse XML, instead of requiring a DOM node ........ r18106 | phasefx | 2010-09-29 16:49:05 -0400 (Wed, 29 Sep 2010) | 2 lines Fix overzealous prompting for auth credentials, and some debug lines ........ r18109 | erickson | 2010-09-30 16:49:14 -0400 (Thu, 30 Sep 2010) | 1 line fire the hold_request.cancel.staff when hold is cancelled by staff ........ r18111 | atz | 2010-09-30 19:06:39 -0400 (Thu, 30 Sep 2010) | 6 lines Silence warning Warnings was: Use of uninitialized value in subroutine entry at /openils/lib/perl5/OpenILS/SIP/Patron.pm line 230. Signed-off-by: Joe Atzberger ........ r18112 | dbs | 2010-10-01 01:16:41 -0400 (Fri, 01 Oct 2010) | 6 lines Add serial.record_entry to CDBI definitions Better late than never; never added the serial.record_entry CDBI definitions in the 1.6 series, but as that table can still be active in 2.0 we might as well get it in place. ........ r18113 | dbs | 2010-10-01 01:24:38 -0400 (Fri, 01 Oct 2010) | 14 lines Teach marc_export script how to export MFHD serial records Passing the --mfhd flag will export any non-deleted MFHD records in serial.record_entry associated with each bib ID. So, for a hypothetical set of bib IDs 1, 2, 3, where 2 has no associated MFHD records and 3 has 2 MFHD records associated with it, the output will be structured as follows: Bib MARC for bib ID 1 MFHD MARC for bib ID 1 Bib MARC for bib ID 2 Bib MARC for bib ID 3 MFHD MARC for bib ID 3 MFHD MARC for bib ID 2 ........ r18116 | phasefx | 2010-10-01 08:19:11 -0400 (Fri, 01 Oct 2010) | 2 lines tweak remoteauth.cgi to offer usrname and barcode params in addition to user. user param now looks for the opac.barcode_regex org unit setting to determine whether the value is a usrname or barcode. change double-quotes to single-quotes if we're not doing string interpolation. change apache instructions for configuration ........ r18121 | phasefx | 2010-10-01 14:07:00 -0400 (Fri, 01 Oct 2010) | 2 lines use an opac login here, which doesn't require the STAFF_LOGIN permission by default ........ r18123 | miker | 2010-10-01 15:39:51 -0400 (Fri, 01 Oct 2010) | 1 line Implement a process-local cache for event target fleshing -- particularly helpful with large event groups ........ r18124 | senator | 2010-10-01 15:45:27 -0400 (Fri, 01 Oct 2010) | 5 lines Offer yet another pull list printing pathway If you have pull lists long enough to make A/T groan, perhaps this will work better for you. ........ r18126 | dbs | 2010-10-01 16:07:59 -0400 (Fri, 01 Oct 2010) | 11 lines Set due times for durations measured in days to 23:59:59 after inserts OR updates The existing trigger acted only on the initial insert of a circulation transaction for duration intervals perfectly divisible by 24 hours. If updates to those due dates were subsequently issued, then the due time would revert to 00:00:00 - which could cause surprising overdue fines to be generated on the due date, rather than after the due date. This commit makes the trigger take effect on both INSERT and UPDATE to the action.circulation table. ........ r18128 | dbs | 2010-10-01 16:13:10 -0400 (Fri, 01 Oct 2010) | 2 lines Bring the 1.6.1-2.0 upgrade script up to date for the push_due_date_tgr ........ r18130 | senator | 2010-10-01 17:48:41 -0400 (Fri, 01 Oct 2010) | 4 lines Fix an apparent bug in a case where OpenILS::WWW::Proxy means to send a redirect, and also avoid the issue altogether in the new holds pull list printing interface just added earlier today. ........ r18132 | miker | 2010-10-02 01:47:02 -0400 (Sat, 02 Oct 2010) | 7 lines Massive search core-query speed improvement. * Only compile the tsquery once * Use direct ids instead of going back to the db udring a queyr * Change a remaining CASE to COALESCE/NULLIF ........ r18133 | miker | 2010-10-02 02:04:05 -0400 (Sat, 02 Oct 2010) | 1 line putting back the NUMERIC cast, it is needed ........ r18136 | miker | 2010-10-02 11:47:28 -0400 (Sat, 02 Oct 2010) | 1 line configurable chunking of the holds stream to avoid the xulrunner "I forgot the chunked stream" problem ........ r18138 | scottmk | 2010-10-03 10:17:37 -0400 (Sun, 03 Oct 2010) | 4 lines Incorporate upgrade scripts 0423 and 0424 M Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql ........ r18140 | erickson | 2010-10-03 11:06:45 -0400 (Sun, 03 Oct 2010) | 1 line moved the cache clear to later in the firing to pick up data; fixed some typos/thinkos ........ r18141 | miker | 2010-10-03 20:40:44 -0400 (Sun, 03 Oct 2010) | 1 line Allow caller to ignore selected facet classes; ignore identifier class by default ........ r18146 | scottmk | 2010-10-04 09:43:13 -0400 (Mon, 04 Oct 2010) | 8 lines Add hold_priority column to permission.grp_tree table. M Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/permission.pm M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0425.schema.perm-grp-tree-hold-priority.sql M Open-ILS/src/sql/Pg/006.schema.permissions.sql M Open-ILS/examples/fm_IDL.xml ........ r18148 | senator | 2010-10-04 10:01:28 -0400 (Mon, 04 Oct 2010) | 2 lines Fix a slight bug in 18136, and the alt pull list printing interface works again. ........ r18150 | senator | 2010-10-04 11:40:21 -0400 (Mon, 04 Oct 2010) | 2 lines Add a control for the hold_priority field in the permission group UI ........ r18151 | miker | 2010-10-04 12:15:41 -0400 (Mon, 04 Oct 2010) | 1 line add support for hold_priority sorting in open-ils.storage.action.hold_request.nearest_hold ........ r18153 | erickson | 2010-10-04 12:48:01 -0400 (Mon, 04 Oct 2010) | 1 line fixed typo in vandelay merge profile permission check ........ r18158 | senator | 2010-10-04 14:57:20 -0400 (Mon, 04 Oct 2010) | 2 lines Add "Patron Alias" as an available column in pull list, holds shelf interfaces ........ r18159 | dbs | 2010-10-04 15:05:59 -0400 (Mon, 04 Oct 2010) | 2 lines French translation of password reset form and prompts at /opac/password/fr-CA/ ........ r18163 | senator | 2010-10-04 17:28:17 -0400 (Mon, 04 Oct 2010) | 3 lines Add some missing ACQ perms. Other perms may be missing, but I /know/ these are needed. ........ r18165 | erickson | 2010-10-05 10:15:40 -0400 (Tue, 05 Oct 2010) | 1 line avoid running activated hook when in dry run mode ........ r18167 | gmc | 2010-10-05 10:44:21 -0400 (Tue, 05 Oct 2010) | 4 lines make it easier to grep for bib search durations from the Pg log Signed-off-by: Galen Charlton ........ r18169 | phasefx | 2010-10-05 11:16:42 -0400 (Tue, 05 Oct 2010) | 8 lines more classname hooks for local CSS to latch on to if needed For example, you may have a server/skin/global_custom.css containing: .hide_patron_credit { display: none } .hide_patron_work{ display: none } .hide_patron_goods { display: none } ........ r18171 | atz | 2010-10-05 12:14:46 -0400 (Tue, 05 Oct 2010) | 1 line Add title to PO view page. ........ r18172 | atz | 2010-10-05 12:16:59 -0400 (Tue, 05 Oct 2010) | 1 line EDI job POD ........ r18175 | senator | 2010-10-05 12:37:57 -0400 (Tue, 05 Oct 2010) | 3 lines This is a silly commit, as this template should be made configurable down the road, maybe by OU setting or something. Anyway, make the list more legible. ........ r18177 | atz | 2010-10-05 13:08:56 -0400 (Tue, 05 Oct 2010) | 1 line The whole point of test mode, NOT actually committing actions. ........ r18179 | gmc | 2010-10-05 14:54:05 -0400 (Tue, 05 Oct 2010) | 8 lines hold matrix selection: treat root OU as just another OU Treat the root OU as just another OU for the purpose of calculating OU proximity adjustments when selecting a matchpoint. Signed-off-by: Galen Charlton ........ r18180 | phasefx | 2010-10-05 15:33:08 -0400 (Tue, 05 Oct 2010) | 2 lines be quiet with errors retrieving remote column settings ........ r18182 | phasefx | 2010-10-05 16:12:58 -0400 (Tue, 05 Oct 2010) | 1 line patch from tsbere to use _blank for new windows instead of generated identifiers ........ r18183 | phasefx | 2010-10-05 16:13:01 -0400 (Tue, 05 Oct 2010) | 2 lines patch from tsbere to improve tab behavior and to give command-line options for controlling tabs See https://bugs.launchpad.net/evergreen/+bug/625056 ........ r18184 | phasefx | 2010-10-05 16:13:03 -0400 (Tue, 05 Oct 2010) | 1 line bug fix for close tab regression ........ r18185 | phasefx | 2010-10-05 16:13:06 -0400 (Tue, 05 Oct 2010) | 1 line some I18N and tweak call to getIntPref. Is the 2nd parameter version an undocumented way of providing a default if the pref isn't found? ........ r18187 | miker | 2010-10-05 22:31:55 -0400 (Tue, 05 Oct 2010) | 1 line Add a retargetting specific version of the hold-permit function which skips user tests ........ r18188 | miker | 2010-10-05 22:35:17 -0400 (Tue, 05 Oct 2010) | 1 line usr_grp is entirely unused currently; hide it ........ r18191 | miker | 2010-10-05 23:07:30 -0400 (Tue, 05 Oct 2010) | 1 line Add a switch to turn on strict OU matching for all OUs required by a hold matchpoint ........ r18192 | scottmk | 2010-10-06 09:27:26 -0400 (Wed, 06 Oct 2010) | 5 lines Fixed the second and third function definitions, which wouldn't compile. An SQL function cannot reference its parameters by name, but only by number. M Open-ILS/src/sql/Pg/upgrade/0428.schema.hold_retarget.sql ........ r18193 | erickson | 2010-10-06 09:28:09 -0400 (Wed, 06 Oct 2010) | 1 line first level of groups hash can be array or hash, leading to breakage; for now, keep it simple and return continue-status to keep the caller alive ........ r18195 | miker | 2010-10-06 09:43:52 -0400 (Wed, 06 Oct 2010) | 1 line Make retargetting check less lax by including appropriate patron-side test ........ r18196 | miker | 2010-10-06 09:45:00 -0400 (Wed, 06 Oct 2010) | 1 line Start out exact-ou-match rules with an "always win" weight ........ r18198 | miker | 2010-10-06 09:53:18 -0400 (Wed, 06 Oct 2010) | 1 line Thinko fix for field name. The script would not commit, so please simply reapply 0430. ........ r18199 | erickson | 2010-10-06 09:54:26 -0400 (Wed, 06 Oct 2010) | 1 line suppress usr_grp field in hold matrix matchpoint; see also 18188 ........ r18202 | scottmk | 2010-10-06 10:32:01 -0400 (Wed, 06 Oct 2010) | 5 lines Further propagating a syntax correction; SQL functions cannot reference their parameters by name. M Open-ILS/src/sql/Pg/110.hold_matrix.sql ........ r18203 | phasefx | 2010-10-06 10:59:43 -0400 (Wed, 06 Oct 2010) | 2 lines hold/transit slips may want to use stat cats as well ........ r18205 | erickson | 2010-10-06 11:01:38 -0400 (Wed, 06 Oct 2010) | 1 line avoid retreiving/searching a linked object, when there is no object to retrieve (i.e. ident value is null) and silence various warnings by skipping this scenario. mild variable refactor for easy reading for feeble eyes. ........ r18207 | scottmk | 2010-10-06 11:55:14 -0400 (Wed, 06 Oct 2010) | 14 lines Two changes to config schema: 1. Add new column date_ceiling to rule_circ_duration table. 2. New table hard_due_date. For trunk, v1.6, v1.6.2 (eventually), and v2.1 (eventually). NOT for v1.6.1 or v2.0. M Open-ILS/src/sql/Pg/002.schema.config.sql A Open-ILS/src/sql/Pg/upgrade/0432.schema.config_hard_due_date.sql M Open-ILS/examples/fm_IDL.xml ........ r18210 | scottmk | 2010-10-06 14:19:43 -0400 (Wed, 06 Oct 2010) | 4 lines Incorporate upgrades 0427 and 0428 into the consolidated upgrade script. M Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql ........ r18211 | miker | 2010-10-06 16:07:39 -0400 (Wed, 06 Oct 2010) | 1 line db upgrade script dislikes holes -- spackle this one ........ r18213 | erickson | 2010-10-06 16:44:46 -0400 (Wed, 06 Oct 2010) | 1 line added support for honoring the due date ceiling from the duration rule. AKA, end-of-semester due dates ........ r18215 | senator | 2010-10-06 17:01:42 -0400 (Wed, 06 Oct 2010) | 14 lines New way to printing shelf-expired holds This just takes the newest template for printing hold pull lists and grafts this new functionality onto it. It should perhaps be adjusted to also be able to print things on the holds shelf that /aren't/ shelf-expired. For now you get to this under Admin -> For Developers -> Local Administration This also corrects a bug because of which a "print pull list (alternate strategy)" button appeared where it shouldn't. This also removes the booking links from Admin -> For Developers -> Local Administration, as there are regular staff client menu entries for those now. ........ r18217 | atz | 2010-10-06 18:12:36 -0400 (Wed, 06 Oct 2010) | 1 line Warn but continue on skipped upgrade revs. ........ r18219 | erickson | 2010-10-07 10:56:43 -0400 (Thu, 07 Oct 2010) | 6 lines Parallel action/trigger collection and reaction QA'ed patch from miker to support parallel a/t event collection and reaction. Max parallel procs is controlled by two new opensrf.xml trigger app_settings. Sample config included, settings disabled by default. ........ r18220 | phasefx | 2010-10-07 11:14:50 -0400 (Thu, 07 Oct 2010) | 2 lines destination_shelf macro for hold and hold/transit slips. Will contain either HOLD SHELF, PUBLIC HOLD SHELF, or PRIVATE HOLD SHELF (these are localizable). The latter two depend on the org unit setting 'circ.holds.behind_desk_pickup_supported' being in effect, and also considers a user setting for the holds user at the time of printing. ........ r18222 | senator | 2010-10-07 13:18:58 -0400 (Thu, 07 Oct 2010) | 2 lines Add some reasonable default sorting to this expired holds list ........ r18224 | atz | 2010-10-07 14:48:45 -0400 (Thu, 07 Oct 2010) | 6 lines EDI template update for ORDERS This template produces JSON for the edi translator to convert into actual EDI lines. It now handles vendor-specific requirements for account and sub-account identification, and also transmits notes of the vendor-public variety as FTX segments in the lineitem. ........ r18226 | senator | 2010-10-07 15:10:58 -0400 (Thu, 07 Oct 2010) | 3 lines Just a minor thing on this shelf expired print interface, probably more changes coming. ........ r18228 | gmc | 2010-10-07 17:39:05 -0400 (Thu, 07 Oct 2010) | 4 lines choose the hold permit test variant correctly Signed-off-by: Galen Charlton ........ r18230 | senator | 2010-10-07 18:37:45 -0400 (Thu, 07 Oct 2010) | 8 lines Expired holds shelf printer needs to be a holds shelf *clearer* and printer This needs cleaned up and stuff, and made into something cooler. Basically just does what XUL interfaces under the Circ menu can already do, but streamlined to tolerate really big datasets. Much of this code originates from berick and miker. ........ r18231 | senator | 2010-10-07 18:46:24 -0400 (Thu, 07 Oct 2010) | 2 lines progress to ProgressDialog means total, not incremental ........ r18233 | gmc | 2010-10-07 21:34:08 -0400 (Thu, 07 Oct 2010) | 8 lines do not apply superpage limit inside bib search joins Oddly, empirical evidence suggests that this might actually be faster, and if this pans out, search results will definitely be more complete. Signed-off-by: Galen Charlton ........ r18239 | dbs | 2010-10-07 23:30:00 -0400 (Thu, 07 Oct 2010) | 7 lines Avoid scary SSL / HTTPS errors in Apache configuration When port 443 is the last listener port, Apache generates lots of "unknown protocol speaking not SSL to HTTPS port!?" errors in the logs - which are scary, but harmless. Putting port 80 last avoids those errors entirely, per http://wiki.apache.org/httpd/InternalDummyConnection ........ r18240 | scottmk | 2010-10-08 10:16:19 -0400 (Fri, 08 Oct 2010) | 19 lines Tidied up buildSELECT() a bit: 1. Sprinkled the const qualifier here and there. 2. Moved some variable declarations to get them closer to the point of first use, and to limit their scope. 3. Renamed some variables to better reflect their meaning. 4. Split a couple of variables into multiple variables, instead of using them for multiple unrelated purposes. 5. Plugged a memory leak in the case of an error return. 6. Added comments, including a Doxygen-style comment at the top of the function. M Open-ILS/src/c-apps/oils_sql.c ........ r18244 | miker | 2010-10-08 13:06:52 -0400 (Fri, 08 Oct 2010) | 1 line add a method to overlay a special bib container full of records (or bucket + template) ........ r18245 | phasefx | 2010-10-08 13:29:20 -0400 (Fri, 08 Oct 2010) | 2 lines moving users in and out of groups produced dialogs that expected vertical patron summaries. this fixes that ........ r18247 | phasefx | 2010-10-08 13:38:24 -0400 (Fri, 08 Oct 2010) | 2 lines fix logic error where we were just testing for the presence of a user setting instead of its value ........ r18249 | senator | 2010-10-08 14:02:08 -0400 (Fri, 08 Oct 2010) | 2 lines Be a little more tolerant of aberrant data when clearing the holds shelf ........ r18251 | phasefx | 2010-10-08 16:58:37 -0400 (Fri, 08 Oct 2010) | 1 line typo ........ r18253 | erickson | 2010-10-08 17:10:43 -0400 (Fri, 08 Oct 2010) | 1 line mild fixes for a/t interface admin interface. sort by def name instead of hook, which probably makes more sense to a human. hide the opt-in and max-delay columns to free up some badly needed horizontal space. use percentage width for name column, which acts a lot like 'auto', but allows the user to manually resize ........ r18255 | miker | 2010-10-08 22:32:49 -0400 (Fri, 08 Oct 2010) | 1 line mod_perl handler to allow batch update bib records from an ephemeral template ........ r18256 | erickson | 2010-10-11 10:01:45 -0400 (Mon, 11 Oct 2010) | 1 line wait to run-pending if a specific granularity used ........ r18258 | scottmk | 2010-10-11 10:25:05 -0400 (Mon, 11 Oct 2010) | 12 lines Pull out into a separate function: the code in SELECT() that builds a comma-separated list of ORDER BY expressions from a JSON_ARRAY. Invoke that function, not only from SELECT(), but also from the buildSELECT() function. As a result, the select methods will be able to use the same array syntax as json_query for ORDER BY clauses, as an alternative to the existing hash syntax. M Open-ILS/src/c-apps/oils_sql.c ........ r18259 | atz | 2010-10-11 11:21:12 -0400 (Mon, 11 Oct 2010) | 1 line Clean up method registration sigs/descs ........ r18260 | atz | 2010-10-11 12:06:23 -0400 (Mon, 11 Oct 2010) | 1 line Method registration cleanup ........ r18261 | miker | 2010-10-11 12:06:52 -0400 (Mon, 11 Oct 2010) | 1 line supply a dummy leader, as required by the Perl MARC modules ........ r18262 | atz | 2010-10-11 12:07:09 -0400 (Mon, 11 Oct 2010) | 1 line Tighter verbose format for skipped POs ........ r18264 | miker | 2010-10-11 12:07:47 -0400 (Mon, 11 Oct 2010) | 1 line Working template-based batch bib updater! ........ r18269 | phasefx | 2010-10-11 16:39:31 -0400 (Mon, 11 Oct 2010) | 2 lines initialize these lists just once to prevent display glitch ........ r18271 | miker | 2010-10-11 16:41:37 -0400 (Mon, 11 Oct 2010) | 1 line dogfooding cleanup ........ r18272 | miker | 2010-10-11 16:42:46 -0400 (Mon, 11 Oct 2010) | 1 line make the source selection less confusing, and improve some wording in the template section (more to come) ........ r18274 | phasefx | 2010-10-11 17:42:48 -0400 (Mon, 11 Oct 2010) | 2 lines avoid race condition with post-save patron editor refresh and replicated databases ........ r18277 | scottmk | 2010-10-12 09:40:14 -0400 (Tue, 12 Oct 2010) | 4 lines Incorporate upgrade # 0433 into the consolidated upgrade script. M Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql ........ r18278 | miker | 2010-10-12 11:19:54 -0400 (Tue, 12 Oct 2010) | 1 line skip unknown order_by entries instead of erroring ........ r18279 | miker | 2010-10-12 11:39:47 -0400 (Tue, 12 Oct 2010) | 1 line rearrangement for variable existance ........ r18280 | miker | 2010-10-12 11:40:45 -0400 (Tue, 12 Oct 2010) | 1 line and ... use a function name that exists ........ r18281 | miker | 2010-10-12 12:02:31 -0400 (Tue, 12 Oct 2010) | 1 line now that we are working again, hush the warnings (osrfLogInternal to the rescue) ........ r18282 | atz | 2010-10-12 13:01:52 -0400 (Tue, 12 Oct 2010) | 1 line Be sure to copy new JS files from OpenSRF on a FULL update/install ........ r18285 | miker | 2010-10-12 15:23:22 -0400 (Tue, 12 Oct 2010) | 1 line improve labeling and documentation ........ r18286 | phasefx | 2010-10-12 15:34:35 -0400 (Tue, 12 Oct 2010) | 1 line add option for hiding entire embedded browser toolbar ........ r18287 | phasefx | 2010-10-12 15:34:39 -0400 (Tue, 12 Oct 2010) | 1 line allow util.deck to be instantiated with either a xul deck object or a xul deck id ........ r18288 | phasefx | 2010-10-12 15:34:50 -0400 (Tue, 12 Oct 2010) | 1 line vertical bib summary (mainly for use in the Merge Record interface). I tried using just bib_brief.xul with document.loadOverlay to choose between bib_brief_overlay_vertical.xul and bib_brief_overlay.xul based on a param, but was running into too much pain going that route. ........ r18289 | phasefx | 2010-10-12 15:35:42 -0400 (Tue, 12 Oct 2010) | 2 lines new record merge UI. needs r18287, r18288. I'll backport them all together ........ r18290 | miker | 2010-10-12 15:55:58 -0400 (Tue, 12 Oct 2010) | 1 line more terminology cleanup and inline documentation ........ r18291 | miker | 2010-10-12 16:31:31 -0400 (Tue, 12 Oct 2010) | 1 line Get rid of commas, apparently confusing; make action button label more generic / less "developery" ........ r18293 | miker | 2010-10-12 17:31:33 -0400 (Tue, 12 Oct 2010) | 1 line when we have no target field, add the entire source field, even when we have a subfield designation ........ r18294 | phasefx | 2010-10-12 17:32:21 -0400 (Tue, 12 Oct 2010) | 1 line entry points for miker_'s batch marc editor ........ r18298 | miker | 2010-10-12 20:27:46 -0400 (Tue, 12 Oct 2010) | 1 line deduplicate bibs going into the merge queue ........ r18299 | miker | 2010-10-12 20:28:43 -0400 (Tue, 12 Oct 2010) | 1 line arg, missing semicolon ........ r18300 | miker | 2010-10-12 20:29:38 -0400 (Tue, 12 Oct 2010) | 1 line double-arg, backwards logic ........ r18303 | miker | 2010-10-12 22:33:59 -0400 (Tue, 12 Oct 2010) | 1 line thinko supporting multiple rules of the same type in in-line merge rulesets ........ r18310 | phasefx | 2010-10-13 12:03:25 -0400 (Wed, 13 Oct 2010) | 2 lines missing semicolon ........ r18311 | erickson | 2010-10-13 12:13:23 -0400 (Wed, 13 Oct 2010) | 1 line when the target for an event is no longer around, immediately invalidate the event and prevent the event from bubbling up for further processing ........ r18312 | miker | 2010-10-13 12:17:14 -0400 (Wed, 13 Oct 2010) | 1 line Use just one transaction, and inside an rstore editor no less, for fleshing env paths ........ r18313 | phasefx | 2010-10-13 12:30:38 -0400 (Wed, 13 Oct 2010) | 1 line overzealous trimming of cat.properties ........ r18321 | scottmk | 2010-10-13 14:53:56 -0400 (Wed, 13 Oct 2010) | 22 lines Changes to the treatment of ORDER BY: 1. For json_query: when ORDER BY is expressed as an object keyed on class (instead of an array of field specifications), and the class is not in scope, error out instead of silently ignoring the class. The other changes affect only methods other than json_query: 2. When the ORDER BY list is provided as a raw text string: block any string containing a semicolon, in order to block simple SQL injections. For now we make no exceptions for quoted semicolons, which are not likely ever to appear an an ORDER BY clause. 3. Keep virtual fields out of the ORDER BY clause. For now we silently ignore them, as we ignore non-existent fields. In both cases we should perhaps error out. 4. Don't require that a class referenced in the ORDER BY clause also be referenced in the SELECT clause. Just make sure it's in scope. M Open-ILS/src/c-apps/oils_sql.c ........ r18322 | erickson | 2010-10-13 16:16:57 -0400 (Wed, 13 Oct 2010) | 1 line clean up duplicate system-controlled penalties during penalty calculation; perform penalty trigger event creation after the standalone commit occurs. todo, handle non-standalone post-commit penalty trigger event creation ........ r18323 | erickson | 2010-10-13 16:16:58 -0400 (Wed, 13 Oct 2010) | 1 line default to standard cstore instead of rstore for env building editor ........ r18326 | phasefx | 2010-10-14 11:29:29 -0400 (Thu, 14 Oct 2010) | 2 lines change the Swap Editor button to a persisted checkbox ........ r18329 | miker | 2010-10-14 14:13:20 -0400 (Thu, 14 Oct 2010) | 1 line correct top-half ordering by label_sortkey ........ r18331 | miker | 2010-10-14 14:35:27 -0400 (Thu, 14 Oct 2010) | 1 line patch from Steve Callendar to avoid resetting the passwd every time a phone number changes ........ r18333 | miker | 2010-10-14 15:18:11 -0400 (Thu, 14 Oct 2010) | 1 line cast label_sortkey to bytea in order to get ascii-betical sorting in any locale, even C. stupid glibc ... ........ r18336 | miker | 2010-10-14 15:33:41 -0400 (Thu, 14 Oct 2010) | 1 line backward compat indexing for label instead of label_sortkey ........ r18342 | erickson | 2010-10-14 15:57:42 -0400 (Thu, 14 Oct 2010) | 1 line repaired upgrade version ........ r18343 | miker | 2010-10-14 16:04:05 -0400 (Thu, 14 Oct 2010) | 1 line "as" not "to" ... you pointed that out the first time, miker, what is your deal? ........ r18349 | miker | 2010-10-14 16:17:00 -0400 (Thu, 14 Oct 2010) | 1 line force granularity-only when any granularity is specified ........ r18351 | erickson | 2010-10-14 18:17:21 -0400 (Thu, 14 Oct 2010) | 1 line If an item is captured for a hold, but not in transit (i.e. on holds shelf), set the destination_location equal to the pickup library (i.e where it's supposedly on the shelf). This is useful for autmated sorting so that the item will return to the branch whose shelf where it belongs. ........ r18353 | gmc | 2010-10-15 09:08:52 -0400 (Fri, 15 Oct 2010) | 7 lines tweak expanding search field aliases Avoids a glitch that can occur if a search field alias exists that happens to have the same name as a search field. Signed-off-by: Galen Charlton ........ r18364 | miker | 2010-10-16 11:32:32 -0400 (Sat, 16 Oct 2010) | 1 line use a function to wrap up escaping of solidus and casting to bytea, propogate to indexing and where/order_by ........ r18365 | miker | 2010-10-16 11:35:39 -0400 (Sat, 16 Oct 2010) | 1 line need to update this index as well ........ r18366 | miker | 2010-10-16 11:38:21 -0400 (Sat, 16 Oct 2010) | 1 line go ahead and use the new index if we need to ........ git-svn-id: svn://svn.open-ils.org/ILS/branches/serials-integration@18372 dcc99617-32d9-48b4-a31d-7c20da2025e4 --- diff --git a/Open-ILS/examples/apache/eg.conf b/Open-ILS/examples/apache/eg.conf index 77cd6d0aa2..f44ae921a2 100644 --- a/Open-ILS/examples/apache/eg.conf +++ b/Open-ILS/examples/apache/eg.conf @@ -99,26 +99,6 @@ ExpiresByType application/xhtml+xml A64800 ExpiresByType application/x-javascript A64800 ExpiresByType text/css A3000 - - - -# ---------------------------------------------------------------------------------- -# Set up our main virtual host -# ---------------------------------------------------------------------------------- -NameVirtualHost *:80 - - ServerName localhost:80 - ServerAlias 127.0.0.1:80 - DocumentRoot /openils/var/web/ - DirectoryIndex index.xml index.html index.xhtml - # - absorb the shared virtual host settings - Include eg_vhost.conf - - - - - - # ---------------------------------------------------------------------------------- # Set up our SSL virtual host # ---------------------------------------------------------------------------------- @@ -147,4 +127,18 @@ NameVirtualHost *:443 +# ---------------------------------------------------------------------------------- +# Set up our main virtual host +# Port 80 comes after 443 to avoid "unknown protocol speaking not SSL to HTTPS port!?" +# errors, per http://wiki.apache.org/httpd/InternalDummyConnection +# ---------------------------------------------------------------------------------- +NameVirtualHost *:80 + + ServerName localhost:80 + ServerAlias 127.0.0.1:80 + DocumentRoot /openils/var/web/ + DirectoryIndex index.xml index.html index.xhtml + # - absorb the shared virtual host settings + Include eg_vhost.conf + diff --git a/Open-ILS/examples/apache/eg_vhost.conf b/Open-ILS/examples/apache/eg_vhost.conf index 1f93a21df3..4a2266fc0d 100644 --- a/Open-ILS/examples/apache/eg_vhost.conf +++ b/Open-ILS/examples/apache/eg_vhost.conf @@ -395,6 +395,28 @@ RewriteRule - - [E=locale:en-US] [L] allow from all + + SetHandler perl-script + PerlSetVar OILSProxyTitle "Batch Update Login" + PerlSetVar OILSProxyDescription "Please log in to update records in batch" + PerlSetVar OILSProxyPermissions "STAFF_LOGIN" + PerlHandler OpenILS::WWW::Proxy OpenILS::WWW::TemplateBatchBibUpdate + PerlSendHeader On + Options +ExecCGI + allow from all + + + + SetHandler perl-script + PerlSetVar OILSProxyTitle "Circ Extras Login" + PerlSetVar OILSProxyDescription "Please log in with an authorized staff account to export records" + PerlSetVar OILSProxyPermissions "STAFF_LOGIN" + PerlHandler OpenILS::WWW::Proxy + Options +ExecCGI + PerlSendHeader On + allow from all + + # ---------------------------------------------------------------------------------- # Reporting output lives here # ---------------------------------------------------------------------------------- diff --git a/Open-ILS/examples/fm_IDL.xml b/Open-ILS/examples/fm_IDL.xml index 7c68d045d7..25206b3e71 100644 --- a/Open-ILS/examples/fm_IDL.xml +++ b/Open-ILS/examples/fm_IDL.xml @@ -999,6 +999,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA + @@ -1868,6 +1869,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA + @@ -1880,6 +1882,21 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA + + + + + + + + + + + + + + + @@ -4515,6 +4532,7 @@ SELECT usr, + @@ -6523,7 +6541,7 @@ SELECT usr, - + diff --git a/Open-ILS/examples/oils_sip.xml.example b/Open-ILS/examples/oils_sip.xml.example index 466c304868..3111626b32 100644 --- a/Open-ILS/examples/oils_sip.xml.example +++ b/Open-ILS/examples/oils_sip.xml.example @@ -105,6 +105,15 @@ --> + + COPY_ALERT_MESSAGE + COPY_BAD_STATUS + COPY_STATUS_MISSING + + + - localhost:11211 + 127.0.0.1:11211 86400 - localhost:11211 + 127.0.0.1:11211 1800 @@ -590,6 +590,15 @@ vim:et:ts=4:sw=4: 1 5 + + + + diff --git a/Open-ILS/examples/remoteauth.cgi b/Open-ILS/examples/remoteauth.cgi index c50725dd0f..255f48c97a 100755 --- a/Open-ILS/examples/remoteauth.cgi +++ b/Open-ILS/examples/remoteauth.cgi @@ -3,7 +3,8 @@ # This CGI script might be useful for providing an easy way for EZproxy to authenticate # users against an Evergreen instance. # -# For example, if you modify your eg_vhost.conf by adding this: +# For example, if you modify your eg.conf by adding this: +# Alias "/cgi-bin/ezproxy/" "/openils/var/cgi-bin/ezproxy/" # # AddHandler cgi-script .pl # AllowOverride None @@ -29,47 +30,63 @@ use Digest::MD5 qw(md5_hex); use OpenSRF::EX qw(:try); use OpenSRF::System; - my $bootstrap = '/openils/conf/opensrf_core.xml'; my $cgi = new CGI; my $u = $cgi->param('user'); +my $usrname = $cgi->param('usrname'); +my $barcode = $cgi->param('barcode'); my $p = $cgi->param('passwd'); print $cgi->header(-type=>'text/html', -expires=>'-1d'); OpenSRF::System->bootstrap_client( config_file => $bootstrap ); -if (!$u || !$p) { - print "+INCOMPLETE"; +if (!($u || $usrname || $barcode) || !$p) { + print '+INCOMPLETE'; } else { - my $nametype = 'username'; - $nametype = 'barcode' if ($u =~ /^\d+$/o); + my $nametype; + if ($usrname) { + $u = $usrname; + $nametype = 'username'; + } elsif ($barcode) { + $u = $barcode; + $nametype = 'barcode'; + } else { + $nametype = 'username'; + my $regex_response = OpenSRF::AppSession + ->create('open-ils.actor') + ->request('open-ils.actor.ou_setting.ancestor_default', 1, 'opac.barcode_regex') + ->gather(1); + if ($regex_response) { + my $regexp = $regex_response->{'value'}; + $nametype = 'barcode' if ($u =~ qr/$regexp/); + } + } my $seed = OpenSRF::AppSession - ->create("open-ils.auth") + ->create('open-ils.auth') ->request( 'open-ils.auth.authenticate.init', $u ) ->gather(1); if ($seed) { my $response = OpenSRF::AppSession - ->create("open-ils.auth") - ->request( 'open-ils.auth.authenticate.complete', { $nametype => $u, password => md5_hex($seed . md5_hex($p)), type => 'temp' }) + ->create('open-ils.auth') + ->request( 'open-ils.auth.authenticate.complete', { $nametype => $u, password => md5_hex($seed . md5_hex($p)), type => 'opac' }) ->gather(1); if ($response->{payload}->{authtoken}) { my $user = OpenSRF::AppSession - ->create("open-ils.auth") - ->request( "open-ils.auth.session.retrieve", $response->{payload}->{authtoken} ) + ->create('open-ils.auth') + ->request( 'open-ils.auth.session.retrieve', $response->{payload}->{authtoken} ) ->gather(1); if (ref($user) eq 'HASH' && $user->{ilsevent} == 1001) { - print "+NO"; + print '+NO'; } else { - print "+VALID"; + print '+VALID'; } } else { - print "+NO"; + print '+NO'; } } else { - print "+BACKEND_ERROR"; + print '+BACKEND_ERROR'; } - } 1; diff --git a/Open-ILS/src/c-apps/oils_sql.c b/Open-ILS/src/c-apps/oils_sql.c index 7a5cea87ca..76b99978a8 100644 --- a/Open-ILS/src/c-apps/oils_sql.c +++ b/Open-ILS/src/c-apps/oils_sql.c @@ -90,8 +90,11 @@ static char* searchINPredicate ( const char*, osrfHash*, jsonObject*, const char*, osrfMethodContext* ); static char* searchPredicate ( const ClassInfo*, osrfHash*, jsonObject*, osrfMethodContext* ); static char* searchJOIN ( const jsonObject*, const ClassInfo* left_info ); -static char* searchWHERE ( const jsonObject*, const ClassInfo*, int, osrfMethodContext* ); -static char* buildSELECT ( jsonObject*, jsonObject*, osrfHash*, osrfMethodContext* ); +static char* searchWHERE ( const jsonObject* search_hash, const ClassInfo*, int, osrfMethodContext* ); +static char* buildSELECT( const jsonObject*, jsonObject* rest_of_query, + osrfHash* meta, osrfMethodContext* ctx ); +static char* buildOrderByFromArray( osrfMethodContext* ctx, const jsonObject* order_array ); + char* buildQuery( osrfMethodContext* ctx, jsonObject* query, int flags ); char* SELECT ( osrfMethodContext*, jsonObject*, const jsonObject*, const jsonObject*, @@ -4277,7 +4280,6 @@ char* SELECT ( jsonIteratorFree( selclass_itr ); } - char* col_list = buffer_release( select_buf ); // Make sure the SELECT list isn't empty. This can happen, for example, @@ -4392,172 +4394,23 @@ char* SELECT ( } } - growing_buffer* order_buf = NULL; // to collect ORDER BY list - // Build an ORDER BY clause, if there is one if( NULL == order_hash ) ; // No ORDER BY? do nothing else if( JSON_ARRAY == order_hash->type ) { - // Array of field specifications, each specification being a - // hash to define the class, field, and other details - int order_idx = 0; - jsonObject* order_spec; - while( (order_spec = jsonObjectGetIndex( order_hash, order_idx++ ) ) ) { - - if( JSON_HASH != order_spec->type ) { - osrfLogError( OSRF_LOG_MARK, - "%s: Malformed field specification in ORDER BY clause; expected JSON_HASH, found %s", - modulename, json_type( order_spec->type ) ); - if( ctx ) - osrfAppSessionStatus( - ctx->session, - OSRF_STATUS_INTERNALSERVERERROR, - "osrfMethodException", - ctx->request, - "Malformed ORDER BY clause -- see error log for more details" - ); - buffer_free( order_buf ); - free( having_buf ); - buffer_free( group_buf ); - buffer_free( sql_buf ); - if( defaultselhash ) - jsonObjectFree( defaultselhash ); - return NULL; - } - - const char* class_alias = - jsonObjectGetString( jsonObjectGetKeyConst( order_spec, "class" ) ); - const char* field = - jsonObjectGetString( jsonObjectGetKeyConst( order_spec, "field" ) ); - - if( order_buf ) - OSRF_BUFFER_ADD( order_buf, ", " ); - else - order_buf = buffer_init( 128 ); - - if( !field || !class_alias ) { - osrfLogError( OSRF_LOG_MARK, - "%s: Missing class or field name in field specification " - "of ORDER BY clause", - modulename ); - if( ctx ) - osrfAppSessionStatus( - ctx->session, - OSRF_STATUS_INTERNALSERVERERROR, - "osrfMethodException", - ctx->request, - "Malformed ORDER BY clause -- see error log for more details" - ); - buffer_free( order_buf ); - free( having_buf ); - buffer_free( group_buf ); - buffer_free( sql_buf ); - if( defaultselhash ) - jsonObjectFree( defaultselhash ); - return NULL; - } - - ClassInfo* order_class_info = search_alias( class_alias ); - if( ! order_class_info ) { - osrfLogError( OSRF_LOG_MARK, "%s: ORDER BY clause references class \"%s\" " - "not in FROM clause", modulename, class_alias ); - if( ctx ) - osrfAppSessionStatus( - ctx->session, - OSRF_STATUS_INTERNALSERVERERROR, - "osrfMethodException", - ctx->request, - "Invalid class referenced in ORDER BY clause -- " - "see error log for more details" - ); - free( having_buf ); - buffer_free( group_buf ); - buffer_free( sql_buf ); - if( defaultselhash ) - jsonObjectFree( defaultselhash ); - return NULL; - } - - osrfHash* field_def = osrfHashGet( order_class_info->fields, field ); - if( !field_def ) { - osrfLogError( OSRF_LOG_MARK, - "%s: Invalid field \"%s\".%s referenced in ORDER BY clause", - modulename, class_alias, field ); - if( ctx ) - osrfAppSessionStatus( - ctx->session, - OSRF_STATUS_INTERNALSERVERERROR, - "osrfMethodException", - ctx->request, - "Invalid field referenced in ORDER BY clause -- " - "see error log for more details" - ); - free( having_buf ); - buffer_free( group_buf ); - buffer_free( sql_buf ); - if( defaultselhash ) - jsonObjectFree( defaultselhash ); - return NULL; - } else if( str_is_true( osrfHashGet( field_def, "virtual" ) ) ) { - osrfLogError( OSRF_LOG_MARK, "%s: Virtual field \"%s\" in ORDER BY clause", - modulename, field ); - if( ctx ) - osrfAppSessionStatus( - ctx->session, - OSRF_STATUS_INTERNALSERVERERROR, - "osrfMethodException", - ctx->request, - "Virtual field in ORDER BY clause -- see error log for more details" - ); - buffer_free( order_buf ); - free( having_buf ); - buffer_free( group_buf ); - buffer_free( sql_buf ); - if( defaultselhash ) - jsonObjectFree( defaultselhash ); - return NULL; - } - - if( jsonObjectGetKeyConst( order_spec, "transform" ) ) { - char* transform_str = searchFieldTransform( - class_alias, field_def, order_spec ); - if( ! transform_str ) { - if( ctx ) - osrfAppSessionStatus( - ctx->session, - OSRF_STATUS_INTERNALSERVERERROR, - "osrfMethodException", - ctx->request, - "Severe query error in ORDER BY clause -- " - "see error log for more details" - ); - buffer_free( order_buf ); - free( having_buf ); - buffer_free( group_buf ); - buffer_free( sql_buf ); - if( defaultselhash ) - jsonObjectFree( defaultselhash ); - return NULL; - } - - OSRF_BUFFER_ADD( order_buf, transform_str ); - free( transform_str ); - } - else - buffer_fadd( order_buf, "\"%s\".%s", class_alias, field ); - - const char* direction = - jsonObjectGetString( jsonObjectGetKeyConst( order_spec, "direction" ) ); - if( direction ) { - if( direction[ 0 ] || 'D' == direction[ 0 ] ) - OSRF_BUFFER_ADD( order_buf, " DESC" ); - else - OSRF_BUFFER_ADD( order_buf, " ASC" ); - } + order_by_list = buildOrderByFromArray( ctx, order_hash ); + if( !order_by_list ) { + free( having_buf ); + buffer_free( group_buf ); + buffer_free( sql_buf ); + if( defaultselhash ) + jsonObjectFree( defaultselhash ); + return NULL; } } else if( JSON_HASH == order_hash->type ) { // This hash is keyed on class alias. Each class has either // an array of field names or a hash keyed on field name. + growing_buffer* order_buf = NULL; // to collect ORDER BY list jsonIterator* class_itr = jsonNewIterator( order_hash ); while( (snode = jsonIteratorNext( class_itr )) ) { @@ -4573,7 +4426,7 @@ char* SELECT ( "osrfMethodException", ctx->request, "Invalid class referenced in ORDER BY clause -- " - "see error log for more details" + "see error log for more details" ); jsonIteratorFree( class_itr ); buffer_free( order_buf ); @@ -4820,6 +4673,8 @@ char* SELECT ( } } // end while jsonIteratorFree( class_itr ); + if( order_buf ) + order_by_list = buffer_release( order_buf ); } else { osrfLogError( OSRF_LOG_MARK, "%s: Malformed ORDER BY clause; expected JSON_HASH or JSON_ARRAY, found %s", @@ -4832,7 +4687,6 @@ char* SELECT ( ctx->request, "Malformed ORDER BY clause -- see error log for more details" ); - buffer_free( order_buf ); free( having_buf ); buffer_free( group_buf ); buffer_free( sql_buf ); @@ -4840,12 +4694,8 @@ char* SELECT ( jsonObjectFree( defaultselhash ); return NULL; } - - if( order_buf ) - order_by_list = buffer_release( order_buf ); } - string = buffer_release( group_buf ); if( *string && ( aggregate_found || (flags & SELECT_DISTINCT) ) ) { @@ -4891,26 +4741,199 @@ char* SELECT ( } // end of SELECT() -static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrfHash* meta, osrfMethodContext* ctx ) { +/** + @brief Build a list of ORDER BY expressions. + @param ctx Pointer to the method context. + @param order_array Pointer to a JSON_ARRAY of field specifications. + @return Pointer to a string containing a comma-separated list of ORDER BY expressions. + Each expression may be either a column reference or a function call whose first parameter + is a column reference. + + Each entry in @a order_array must be a JSON_HASH with values for "class" and "field". + It may optionally include entries for "direction" and/or "transform". + + The calling code is responsible for freeing the returned string. +*/ +static char* buildOrderByFromArray( osrfMethodContext* ctx, const jsonObject* order_array ) { + if( ! order_array ) { + osrfLogError( OSRF_LOG_MARK, "%s: Logic error: NULL pointer for ORDER BY clause", + modulename ); + if( ctx ) + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Logic error: ORDER BY clause expected, not found; " + "see error log for more details" + ); + return NULL; + } else if( order_array->type != JSON_ARRAY ) { + osrfLogError( OSRF_LOG_MARK, + "%s: Logic error: Expected JSON_ARRAY for ORDER BY clause, not found", modulename ); + if( ctx ) + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Logic error: Unexpected format for ORDER BY clause; see error log for more details" ); + return NULL; + } + + growing_buffer* order_buf = buffer_init( 128 ); + int first = 1; // boolean + int order_idx = 0; + jsonObject* order_spec; + while( (order_spec = jsonObjectGetIndex( order_array, order_idx++ ))) { + + if( JSON_HASH != order_spec->type ) { + osrfLogError( OSRF_LOG_MARK, + "%s: Malformed field specification in ORDER BY clause; " + "expected JSON_HASH, found %s", + modulename, json_type( order_spec->type ) ); + if( ctx ) + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Malformed ORDER BY clause -- see error log for more details" + ); + buffer_free( order_buf ); + return NULL; + } + + const char* class_alias = + jsonObjectGetString( jsonObjectGetKeyConst( order_spec, "class" )); + const char* field = + jsonObjectGetString( jsonObjectGetKeyConst( order_spec, "field" )); + + if( !field || !class_alias ) { + osrfLogError( OSRF_LOG_MARK, + "%s: Missing class or field name in field specification of ORDER BY clause", + modulename ); + if( ctx ) + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Malformed ORDER BY clause -- see error log for more details" + ); + buffer_free( order_buf ); + return NULL; + } + + const ClassInfo* order_class_info = search_alias( class_alias ); + if( ! order_class_info ) { + osrfLogInternal( OSRF_LOG_MARK, "%s: ORDER BY clause references class \"%s\" " + "not in FROM clause, skipping it", modulename, class_alias ); + continue; + } + + // Add a separating comma, except at the beginning + if( first ) + first = 0; + else + OSRF_BUFFER_ADD( order_buf, ", " ); + + osrfHash* field_def = osrfHashGet( order_class_info->fields, field ); + if( !field_def ) { + osrfLogError( OSRF_LOG_MARK, + "%s: Invalid field \"%s\".%s referenced in ORDER BY clause", + modulename, class_alias, field ); + if( ctx ) + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Invalid field referenced in ORDER BY clause -- " + "see error log for more details" + ); + free( order_buf ); + return NULL; + } else if( str_is_true( osrfHashGet( field_def, "virtual" ) ) ) { + osrfLogError( OSRF_LOG_MARK, "%s: Virtual field \"%s\" in ORDER BY clause", + modulename, field ); + if( ctx ) + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Virtual field in ORDER BY clause -- see error log for more details" + ); + buffer_free( order_buf ); + return NULL; + } + + if( jsonObjectGetKeyConst( order_spec, "transform" )) { + char* transform_str = searchFieldTransform( class_alias, field_def, order_spec ); + if( ! transform_str ) { + if( ctx ) + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Severe query error in ORDER BY clause -- " + "see error log for more details" + ); + buffer_free( order_buf ); + return NULL; + } + + OSRF_BUFFER_ADD( order_buf, transform_str ); + free( transform_str ); + } + else + buffer_fadd( order_buf, "\"%s\".%s", class_alias, field ); + + const char* direction = + jsonObjectGetString( jsonObjectGetKeyConst( order_spec, "direction" ) ); + if( direction ) { + if( direction[ 0 ] || 'D' == direction[ 0 ] ) + OSRF_BUFFER_ADD( order_buf, " DESC" ); + else + OSRF_BUFFER_ADD( order_buf, " ASC" ); + } + } + + return buffer_release( order_buf ); +} + +/** + @brief Build a SELECT statement. + @param search_hash Pointer to a JSON_HASH or JSON_ARRAY encoding the WHERE clause. + @param rest_of_query Pointer to a JSON_HASH containing any other SQL clauses. + @param meta Pointer to the class metadata for the core class. + @param ctx Pointer to the method context. + @return Pointer to a character string containing the WHERE clause; or NULL upon error. + + Within the rest_of_query hash, the meaningful keys are "join", "select", "no_i18n", + "order_by", "limit", and "offset". + + The SELECT statements built here are distinct from those built for the json_query method. +*/ +static char* buildSELECT ( const jsonObject* search_hash, jsonObject* rest_of_query, + osrfHash* meta, osrfMethodContext* ctx ) { const char* locale = osrf_message_get_last_locale(); osrfHash* fields = osrfHashGet( meta, "fields" ); - char* core_class = osrfHashGet( meta, "classname" ); + const char* core_class = osrfHashGet( meta, "classname" ); - const jsonObject* join_hash = jsonObjectGetKeyConst( order_hash, "join" ); + const jsonObject* join_hash = jsonObjectGetKeyConst( rest_of_query, "join" ); - jsonObject* node = NULL; - jsonObject* snode = NULL; - jsonObject* onode = NULL; - const jsonObject* _tmp = NULL; jsonObject* selhash = NULL; jsonObject* defaultselhash = NULL; growing_buffer* sql_buf = buffer_init( 128 ); growing_buffer* select_buf = buffer_init( 128 ); - if( !(selhash = jsonObjectGetKey( order_hash, "select" )) ) { + if( !(selhash = jsonObjectGetKey( rest_of_query, "select" )) ) { defaultselhash = jsonNewObjectType( JSON_HASH ); selhash = defaultselhash; } @@ -4932,20 +4955,24 @@ static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrf jsonObjectSetKey( selhash, core_class, field_list ); } + // Build a list of columns for the SELECT clause int first = 1; + const jsonObject* snode = NULL; jsonIterator* class_itr = jsonNewIterator( selhash ); - while( (snode = jsonIteratorNext( class_itr )) ) { + while( (snode = jsonIteratorNext( class_itr )) ) { // For each class + // If the class isn't in the IDL, ignore it const char* cname = class_itr->key; osrfHash* idlClass = osrfHashGet( oilsIDL(), cname ); if( !idlClass ) continue; - if( strcmp(core_class,class_itr->key )) { + // If the class isn't the core class, and isn't in the JOIN clause, ignore it + if( strcmp( core_class, class_itr->key )) { if( !join_hash ) continue; - jsonObject* found = jsonObjectFindPath( join_hash, "//%s", class_itr->key ); + jsonObject* found = jsonObjectFindPath( join_hash, "//%s", class_itr->key ); if( !found->size ) { jsonObjectFree( found ); continue; @@ -4954,6 +4981,7 @@ static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrf jsonObjectFree( found ); } + const jsonObject* node = NULL; jsonIterator* select_itr = jsonNewIterator( snode ); while( (node = jsonIteratorNext( select_itr )) ) { const char* item_str = jsonObjectGetString( node ); @@ -4971,7 +4999,7 @@ static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrf if( locale ) { const char* i18n; - const jsonObject* no_i18n_obj = jsonObjectGetKeyConst( order_hash, "no_i18n" ); + const jsonObject* no_i18n_obj = jsonObjectGetKeyConst( rest_of_query, "no_i18n" ); if( obj_is_true( no_i18n_obj ) ) // Suppress internationalization? i18n = NULL; else @@ -5019,9 +5047,13 @@ static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrf ctx->request, "Unable to build query frame for core class" ); + buffer_free( sql_buf ); + if( defaultselhash ) + jsonObjectFree( defaultselhash ); return NULL; } + // Add the JOIN clauses, if any if( join_hash ) { char* join_clause = searchJOIN( join_hash, &curr_query->core ); OSRF_BUFFER_ADD_CHAR( sql_buf, ' ' ); @@ -5030,10 +5062,11 @@ static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrf } osrfLogDebug( OSRF_LOG_MARK, "%s pre-predicate SQL = %s", - modulename, OSRF_BUFFER_C_STR( sql_buf )); + modulename, OSRF_BUFFER_C_STR( sql_buf )); OSRF_BUFFER_ADD( sql_buf, " WHERE " ); + // Add the conditions in the WHERE clause char* pred = searchWHERE( search_hash, &curr_query->core, AND_OP_JOIN, ctx ); if( !pred ) { osrfAppSessionStatus( @@ -5053,112 +5086,166 @@ static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrf free( pred ); } - if( order_hash ) { - char* string = NULL; - if( (_tmp = jsonObjectGetKeyConst( order_hash, "order_by" )) ){ - - growing_buffer* order_buf = buffer_init( 128 ); - - first = 1; - jsonIterator* class_itr = jsonNewIterator( _tmp ); - while( (snode = jsonIteratorNext( class_itr )) ) { - - if( !jsonObjectGetKeyConst( selhash,class_itr->key )) - continue; - - if( snode->type == JSON_HASH ) { + // Add the ORDER BY, LIMIT, and/or OFFSET clauses, if present + if( rest_of_query ) { + const jsonObject* order_by = NULL; + if( ( order_by = jsonObjectGetKeyConst( rest_of_query, "order_by" )) ){ - jsonIterator* order_itr = jsonNewIterator( snode ); - while( (onode = jsonIteratorNext( order_itr )) ) { + char* order_by_list = NULL; - osrfHash* field_def = oilsIDLFindPath( "/%s/fields/%s", - class_itr->key, order_itr->key ); - if( !field_def ) - continue; + if( JSON_ARRAY == order_by->type ) { + order_by_list = buildOrderByFromArray( ctx, order_by ); + if( !order_by_list ) { + buffer_free( sql_buf ); + if( defaultselhash ) + jsonObjectFree( defaultselhash ); + clear_query_stack(); + return NULL; + } + } else if( JSON_HASH == order_by->type ) { + // We expect order_by to be a JSON_HASH keyed on class names. Traverse it + // and build a list of ORDER BY expressions. + growing_buffer* order_buf = buffer_init( 128 ); + first = 1; + jsonIterator* class_itr = jsonNewIterator( order_by ); + while( (snode = jsonIteratorNext( class_itr )) ) { // For each class: + + ClassInfo* order_class_info = search_alias( class_itr->key ); + if( ! order_class_info ) + continue; // class not referenced by FROM clause? Ignore it. + + if( JSON_HASH == snode->type ) { + + // If the data for the current class is a JSON_HASH, then it is + // keyed on field name. + + const jsonObject* onode = NULL; + jsonIterator* order_itr = jsonNewIterator( snode ); + while( (onode = jsonIteratorNext( order_itr )) ) { // For each field + + osrfHash* field_def = osrfHashGet( + order_class_info->fields, order_itr->key ); + if( !field_def ) + continue; // Field not defined in IDL? Ignore it. + if( str_is_true( osrfHashGet( field_def, "virtual"))) + continue; // Field is virtual? Ignore it. + + char* field_str = NULL; + char* direction = NULL; + if( onode->type == JSON_HASH ) { + if( jsonObjectGetKeyConst( onode, "transform" ) ) { + field_str = searchFieldTransform( + class_itr->key, field_def, onode ); + if( ! field_str ) { + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Severe query error in ORDER BY clause -- " + "see error log for more details" + ); + jsonIteratorFree( order_itr ); + jsonIteratorFree( class_itr ); + buffer_free( order_buf ); + buffer_free( sql_buf ); + if( defaultselhash ) + jsonObjectFree( defaultselhash ); + clear_query_stack(); + return NULL; + } + } else { + growing_buffer* field_buf = buffer_init( 16 ); + buffer_fadd( field_buf, "\"%s\".%s", + class_itr->key, order_itr->key ); + field_str = buffer_release( field_buf ); + } - char* direction = NULL; - if( onode->type == JSON_HASH ) { - if( jsonObjectGetKeyConst( onode, "transform" ) ) { - string = searchFieldTransform( class_itr->key, field_def, onode ); - if( ! string ) { - osrfAppSessionStatus( - ctx->session, - OSRF_STATUS_INTERNALSERVERERROR, - "osrfMethodException", - ctx->request, - "Severe query error in ORDER BY clause -- " - "see error log for more details" - ); - jsonIteratorFree( order_itr ); - jsonIteratorFree( class_itr ); - buffer_free( order_buf ); - buffer_free( sql_buf ); - if( defaultselhash ) - jsonObjectFree( defaultselhash ); - clear_query_stack(); - return NULL; + if( ( order_by = jsonObjectGetKeyConst( onode, "direction" )) ) { + const char* dir = jsonObjectGetString( order_by ); + if(!strncasecmp( dir, "d", 1 )) { + direction = " DESC"; + } } } else { - growing_buffer* field_buf = buffer_init( 16 ); - buffer_fadd( field_buf, "\"%s\".%s", - class_itr->key, order_itr->key ); - string = buffer_release( field_buf ); - } - - if( (_tmp = jsonObjectGetKeyConst( onode, "direction" )) ) { - const char* dir = jsonObjectGetString( _tmp ); - if(!strncasecmp( dir, "d", 1 )) { + field_str = strdup( order_itr->key ); + const char* dir = jsonObjectGetString( onode ); + if( !strncasecmp( dir, "d", 1 )) { direction = " DESC"; + } else { + direction = " ASC"; } } - } else { - string = strdup( order_itr->key ); - const char* dir = jsonObjectGetString( onode ); - if( !strncasecmp( dir, "d", 1 )) { - direction = " DESC"; + + if( first ) { + first = 0; } else { - direction = " ASC"; + buffer_add( order_buf, ", " ); } - } - if( first ) { - first = 0; - } else { - buffer_add( order_buf, ", " ); - } - - buffer_add( order_buf, string ); - free( string ); + buffer_add( order_buf, field_str ); + free( field_str ); - if( direction ) { - buffer_add( order_buf, direction ); + if( direction ) { + buffer_add( order_buf, direction ); + } + } // end while; looping over ORDER BY expressions + + jsonIteratorFree( order_itr ); + + } else if( JSON_STRING == snode->type ) { + // We expect a comma-separated list of sort fields. + const char* str = jsonObjectGetString( snode ); + if( strchr( str, ';' )) { + // No semicolons allowed. It is theoretically possible for a + // legitimate semicolon to occur within quotes, but it's not likely + // to occur in practice in the context of an ORDER BY list. + osrfLogError( OSRF_LOG_MARK, "%s: Possible attempt at SOL injection -- " + "semicolon found in ORDER BY list: \"%s\"", modulename, str ); + if( ctx ) { + osrfAppSessionStatus( + ctx->session, + OSRF_STATUS_INTERNALSERVERERROR, + "osrfMethodException", + ctx->request, + "Possible attempt at SOL injection -- " + "semicolon found in ORDER BY list" + ); + } + jsonIteratorFree( class_itr ); + buffer_free( order_buf ); + buffer_free( sql_buf ); + if( defaultselhash ) + jsonObjectFree( defaultselhash ); + clear_query_stack(); + return NULL; } + buffer_add( order_buf, str ); + break; } - jsonIteratorFree( order_itr ); + } // end while; looping over order_by classes - } else { - const char* str = jsonObjectGetString( snode ); - buffer_add( order_buf, str ); - break; - } + jsonIteratorFree( class_itr ); + order_by_list = buffer_release( order_buf ); + } else { + osrfLogWarning( OSRF_LOG_MARK, + "\"order_by\" object in a query is not a JSON_HASH or JSON_ARRAY;" + "no ORDER BY generated" ); } - jsonIteratorFree( class_itr ); - - string = buffer_release( order_buf ); - - if( *string ) { + if( order_by_list && *order_by_list ) { OSRF_BUFFER_ADD( sql_buf, " ORDER BY " ); - OSRF_BUFFER_ADD( sql_buf, string ); + OSRF_BUFFER_ADD( sql_buf, order_by_list ); } - free( string ); + free( order_by_list ); } - if( (_tmp = jsonObjectGetKeyConst( order_hash, "limit" )) ) { - const char* str = jsonObjectGetString( _tmp ); + const jsonObject* limit = jsonObjectGetKeyConst( rest_of_query, "limit" ); + if( limit ) { + const char* str = jsonObjectGetString( limit ); buffer_fadd( sql_buf, " LIMIT %d", @@ -5166,9 +5253,9 @@ static char* buildSELECT ( jsonObject* search_hash, jsonObject* order_hash, osrf ); } - _tmp = jsonObjectGetKeyConst( order_hash, "offset" ); - if( _tmp ) { - const char* str = jsonObjectGetString( _tmp ); + const jsonObject* offset = jsonObjectGetKeyConst( rest_of_query, "offset" ); + if( offset ) { + const char* str = jsonObjectGetString( offset ); buffer_fadd( sql_buf, " OFFSET %d", diff --git a/Open-ILS/src/edi_translator/install.RHEL.sh b/Open-ILS/src/edi_translator/install.RHEL.sh new file mode 100755 index 0000000000..890451c8fe --- /dev/null +++ b/Open-ILS/src/edi_translator/install.RHEL.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# +# JEDI converter scripts installation +# +# RHEL/CENTOS Install +# note: need older version of rubygems since RHEL package for ruby is old +# +# run this script as root or with sudo + +yum install ruby ruby-devel ruby-rdoc + +wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.5.tgz +tar zxvf rubygems-1.3.5.tgz +pushd rubygems-1.3.5 +ruby setup.rb # this gives harmless errors about README files missing +gem install rubygems-update +update_rubygems +popd + +# RHEL has a bug in json module, but mbklein moved to using yajl for us.... +gem install parseconfig rspec edi4r edi4r-tdid rcov openils-mapper # mkmf + diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Acq/Lineitem.pm b/Open-ILS/src/perlmods/OpenILS/Application/Acq/Lineitem.pm index 5398ba1e88..0903e31b09 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Acq/Lineitem.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Acq/Lineitem.pm @@ -16,10 +16,10 @@ my $U = 'OpenILS::Application::AppUtils'; __PACKAGE__->register_method( - method => 'create_lineitem', - api_name => 'open-ils.acq.lineitem.create', - signature => { - desc => 'Creates a lineitem', + method => 'create_lineitem', + api_name => 'open-ils.acq.lineitem.create', + signature => { + desc => 'Creates a lineitem', params => [ {desc => 'Authentication token', type => 'string'}, {desc => 'The lineitem object to create', type => 'object'}, @@ -67,10 +67,10 @@ sub create_lineitem { __PACKAGE__->register_method( - method => 'retrieve_lineitem', - api_name => 'open-ils.acq.lineitem.retrieve', - signature => { - desc => 'Retrieves a lineitem', + method => 'retrieve_lineitem', + api_name => 'open-ils.acq.lineitem.retrieve', + signature => { + desc => 'Retrieves a lineitem', params => [ {desc => 'Authentication token', type => 'string'}, {desc => 'lineitem ID to retrieve', type => 'number'}, @@ -149,15 +149,15 @@ sub retrieve_lineitem_impl { } } - return $e->event unless ( + return $e->event unless (( $li->purchase_order and ($no_auth or $e->allowed(['VIEW_PURCHASE_ORDER', 'CREATE_PURCHASE_ORDER'], $li->purchase_order->ordering_agency, $li->purchase_order)) - ) or ( + ) or ( $li->picklist and !$li->purchase_order and # user doesn't have view_po perms ($no_auth or $e->allowed(['VIEW_PICKLIST', 'CREATE_PICKLIST'], $li->picklist->org_unit, $li->picklist)) - ); + )); unless ($$options{flesh_po}) { $li->purchase_order( @@ -173,12 +173,12 @@ sub retrieve_lineitem_impl { __PACKAGE__->register_method( - method => 'delete_lineitem', - api_name => 'open-ils.acq.lineitem.delete', - signature => { - desc => 'Deletes a lineitem', + method => 'delete_lineitem', + api_name => 'open-ils.acq.lineitem.delete', + signature => { + desc => 'Deletes a lineitem', params => [ - {desc => 'Authentication token', type => 'string'}, + {desc => 'Authentication token', type => 'string'}, {desc => 'lineitem ID to delete', type => 'number'}, ], return => {desc => '1 on success, Event on error'} @@ -221,12 +221,12 @@ sub delete_lineitem { __PACKAGE__->register_method( - method => 'update_lineitem', - api_name => 'open-ils.acq.lineitem.update', - signature => { - desc => 'Update one or many lineitems', + method => 'update_lineitem', + api_name => 'open-ils.acq.lineitem.update', + signature => { + desc => 'Update one or many lineitems', params => [ - {desc => 'Authentication token', type => 'string'}, + {desc => 'Authentication token', type => 'string'}, {desc => 'lineitem object update', type => 'object'} ], return => {desc => '1 on success, Event on error'} diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Acq/Order.pm b/Open-ILS/src/perlmods/OpenILS/Application/Acq/Order.pm index f6c07cae6e..c8db485ec3 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Acq/Order.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Acq/Order.pm @@ -1581,12 +1581,12 @@ sub update_lineitem_fund_batch { __PACKAGE__->register_method( - method => 'lineitem_detail_CUD_batch_api', - api_name => 'open-ils.acq.lineitem_detail.cud.batch', - stream => 1, - signature => { - desc => q/Creates a new purchase order line item detail. - Additionally creates the associated fund_debit/, + method => 'lineitem_detail_CUD_batch_api', + api_name => 'open-ils.acq.lineitem_detail.cud.batch', + stream => 1, + signature => { + desc => q/Creates a new purchase order line item detail. / . + q/Additionally creates the associated fund_debit/, params => [ {desc => 'Authentication token', type => 'string'}, {desc => 'List of lineitem_details to create', type => 'array'}, @@ -1597,9 +1597,9 @@ __PACKAGE__->register_method( ); __PACKAGE__->register_method( - method => 'lineitem_detail_CUD_batch_api', - api_name => 'open-ils.acq.lineitem_detail.cud.batch.dry_run', - stream => 1, + method => 'lineitem_detail_CUD_batch_api', + api_name => 'open-ils.acq.lineitem_detail.cud.batch.dry_run', + stream => 1, signature => { desc => q/ Dry run version of open-ils.acq.lineitem_detail.cud.batch. @@ -1691,8 +1691,8 @@ sub handle_changed_lid { __PACKAGE__->register_method( - method => 'receive_po_api', - api_name => 'open-ils.acq.purchase_order.receive' + method => 'receive_po_api', + api_name => 'open-ils.acq.purchase_order.receive' ); sub receive_po_api { @@ -1930,13 +1930,13 @@ sub rollback_receive_lineitem_detail_api { } __PACKAGE__->register_method( - method => 'rollback_receive_lineitem_api', - api_name => 'open-ils.acq.lineitem.receive.rollback', - signature => { - desc => 'Mark a lineitem as Un-received', + method => 'rollback_receive_lineitem_api', + api_name => 'open-ils.acq.lineitem.receive.rollback', + signature => { + desc => 'Mark a lineitem as Un-received', params => [ {desc => 'Authentication token', type => 'string'}, - {desc => 'lineitem ID', type => 'number'} + {desc => 'lineitem ID', type => 'number'} ], return => {desc => "on success, object describing changes to LI and possibly PO; " . @@ -1975,13 +1975,13 @@ sub rollback_receive_lineitem_api { __PACKAGE__->register_method( - method => 'set_lineitem_price_api', - api_name => 'open-ils.acq.lineitem.price.set', - signature => { - desc => 'Set lineitem price. If debits already exist, update them as well', + method => 'set_lineitem_price_api', + api_name => 'open-ils.acq.lineitem.price.set', + signature => { + desc => 'Set lineitem price. If debits already exist, update them as well', params => [ {desc => 'Authentication token', type => 'string'}, - {desc => 'lineitem ID', type => 'number'} + {desc => 'lineitem ID', type => 'number'} ], return => {desc => 'status blob, Event on error'} } @@ -2024,10 +2024,10 @@ sub set_lineitem_price_api { __PACKAGE__->register_method( - method => 'clone_picklist_api', - api_name => 'open-ils.acq.picklist.clone', - signature => { - desc => 'Clones a picklist, including lineitem and lineitem details', + method => 'clone_picklist_api', + api_name => 'open-ils.acq.picklist.clone', + signature => { + desc => 'Clones a picklist, including lineitem and lineitem details', params => [ {desc => 'Authentication token', type => 'string'}, {desc => 'Picklist ID', type => 'number'}, @@ -2072,10 +2072,10 @@ sub clone_picklist_api { __PACKAGE__->register_method( - method => 'merge_picklist_api', - api_name => 'open-ils.acq.picklist.merge', - signature => { - desc => 'Merges 2 or more picklists into a single list', + method => 'merge_picklist_api', + api_name => 'open-ils.acq.picklist.merge', + signature => { + desc => 'Merges 2 or more picklists into a single list', params => [ {desc => 'Authentication token', type => 'string'}, {desc => 'Lead Picklist ID', type => 'number'}, @@ -2119,13 +2119,13 @@ sub merge_picklist_api { __PACKAGE__->register_method( - method => 'delete_picklist_api', - api_name => 'open-ils.acq.picklist.delete', - signature => { - desc => q/Deletes a picklist. It also deletes any lineitems in the "new" state. - Other attached lineitems are detached'/, + method => 'delete_picklist_api', + api_name => 'open-ils.acq.picklist.delete', + signature => { + desc => q/Deletes a picklist. It also deletes any lineitems in the "new" state. / . + q/Other attached lineitems are detached/, params => [ - {desc => 'Authentication token', type => 'string'}, + {desc => 'Authentication token', type => 'string'}, {desc => 'Picklist ID to delete', type => 'number'} ], return => {desc => '1 on success, Event on error'} @@ -2146,17 +2146,16 @@ sub delete_picklist_api { __PACKAGE__->register_method( - method => 'activate_purchase_order', - api_name => 'open-ils.acq.purchase_order.activate.dry_run' + method => 'activate_purchase_order', + api_name => 'open-ils.acq.purchase_order.activate.dry_run' ); __PACKAGE__->register_method( - method => 'activate_purchase_order', - api_name => 'open-ils.acq.purchase_order.activate', - signature => { - desc => q/Activates a purchase order. This updates the status of the PO - and Lineitems to 'on-order'. Activated PO's are ready for EDI delivery - if appropriate./, + method => 'activate_purchase_order', + api_name => 'open-ils.acq.purchase_order.activate', + signature => { + desc => q/Activates a purchase order. This updates the status of the PO / . + q/and Lineitems to 'on-order'. Activated PO's are ready for EDI delivery if appropriate./, params => [ {desc => 'Authentication token', type => 'string'}, {desc => 'Purchase ID', type => 'number'} @@ -2238,21 +2237,21 @@ sub activate_purchase_order_impl { } # tell the world we activated a PO - $U->create_events_for_hook('acqpo.activated', $po, $po->ordering_agency); + $U->create_events_for_hook('acqpo.activated', $po, $po->ordering_agency) unless $dry_run; return undef; } __PACKAGE__->register_method( - method => 'split_purchase_order_by_lineitems', - api_name => 'open-ils.acq.purchase_order.split_by_lineitems', - signature => { - desc => q/Splits a PO into many POs, 1 per lineitem. Only works for - POs a) with more than one lineitems, and b) in the "pending" state./, + method => 'split_purchase_order_by_lineitems', + api_name => 'open-ils.acq.purchase_order.split_by_lineitems', + signature => { + desc => q/Splits a PO into many POs, 1 per lineitem. Only works for / . + q/POs a) with more than one lineitems, and b) in the "pending" state./, params => [ {desc => 'Authentication token', type => 'string'}, - {desc => 'Purchase order ID', type => 'number'} + {desc => 'Purchase order ID', type => 'number'} ], return => {desc => 'list of new PO IDs on success, Event on error'} } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Actor.pm b/Open-ILS/src/perlmods/OpenILS/Application/Actor.pm index 32950967f6..2fcacd25e2 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Actor.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Actor.pm @@ -174,6 +174,7 @@ sub set_ou_settings { __PACKAGE__->register_method( method => "user_settings", + authoritative => 1, api_name => "open-ils.actor.patron.settings.retrieve", ); sub user_settings { @@ -1275,10 +1276,11 @@ sub update_passwd { my $api = $self->api_name; if( $api =~ /password/o ) { - # make sure the original password matches the in-database password - return OpenILS::Event->new('INCORRECT_PASSWORD') - if md5_hex($orig_pw) ne $db_user->passwd; + if (md5_hex($orig_pw) ne $db_user->passwd) { + $e->rollback; + return new OpenILS::Event('INCORRECT_PASSWORD'); + } $db_user->passwd($new_val); } else { @@ -1291,7 +1293,10 @@ sub update_passwd { # make sure no one else has this username my $exist = $e->search_actor_user({usrname=>$new_val},{idlist=>1}); - return OpenILS::Event->new('USERNAME_EXISTS') if @$exist; + if (@$exist) { + $e->rollback; + return new OpenILS::Event('USERNAME_EXISTS'); + } $db_user->usrname($new_val); } elsif( $api =~ /email/o ) { @@ -2477,13 +2482,13 @@ __PACKAGE__->register_method( sub update_user_note { my( $self, $conn, $auth, $note ) = @_; my $e = new_editor(authtoken=>$auth, xact=>1); - return $e->event unless $e->checkauth; + return $e->die_event unless $e->checkauth; my $patron = $e->retrieve_actor_user($note->usr) - or return $e->event; - return $e->event unless + or return $e->die_event; + return $e->die_event unless $e->allowed('UPDATE_USER', $patron->home_ou); $e->update_actor_user_note($note) - or return $e->event; + or return $e->die_event; $e->commit; return 1; } @@ -2883,7 +2888,7 @@ sub new_flesh_user { "flesh_fields" => { "au" => $fields } } ] - ) or return $e->event; + ) or return $e->die_event; if( grep { $_ eq 'addresses' } @$fields ) { @@ -3258,7 +3263,7 @@ __PACKAGE__->register_method ( sub apply_friend_perms { my($self, $conn, $auth, $user_id, $delegate_id, @perms) = @_; my $e = new_editor(authtoken => $auth, xact => 1); - return $e->event unless $e->checkauth; + return $e->die_event unless $e->checkauth; if($user_id != $e->requestor->id) { my $user = $e->retrieve_actor_user($user_id) or return $e->die_event; @@ -3285,7 +3290,7 @@ __PACKAGE__->register_method ( sub update_user_pending_address { my($self, $conn, $auth, $addr) = @_; my $e = new_editor(authtoken => $auth, xact => 1); - return $e->event unless $e->checkauth; + return $e->die_event unless $e->checkauth; if($addr->usr != $e->requestor->id) { my $user = $e->retrieve_actor_user($addr->usr) or return $e->die_event; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/AppUtils.pm b/Open-ILS/src/perlmods/OpenILS/Application/AppUtils.pm index e648a8f188..74f270f5e2 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/AppUtils.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/AppUtils.pm @@ -1176,6 +1176,35 @@ sub is_true { } +sub patientreq { + my ($self, $client, $service, $method, @params) = @_; + my ($response, $err); + + my $session = create OpenSRF::AppSession($service); + my $request = $session->request($method, @params); + + my $spurt = 10; + my $give_up = time + 1000; + + try { + while (time < $give_up) { + $response = $request->recv("timeout" => $spurt); + last if $request->complete; + + $client->status(new OpenSRF::DomainObject::oilsContinueStatus); + } + } catch Error with { + $err = shift; + }; + + if ($err) { + warn "received error : service=$service : method=$method : params=".Dumper(\@params) . "\n $err"; + throw $err ("Call to $service for method $method \n failed with exception: $err : " ); + } + + return $response->content; +} + # This logic now lives in storage sub __patron_money_owed { my( $self, $patronid ) = @_; @@ -1596,7 +1625,7 @@ sub find_event_def_by_hook { # most appropriate event. create the event, fire it, then return the resulting # event with fleshed template_output and error_output sub fire_object_event { - my($self, $event_def, $hook, $object, $context_org, $granularity, $user_data) = @_; + my($self, $event_def, $hook, $object, $context_org, $granularity, $user_data, $client) = @_; my $e = OpenILS::Utils::CStoreEditor->new; my $def; @@ -1616,6 +1645,8 @@ sub fire_object_event { or return $e->event; } + my $final_resp; + if($def->group_field) { # we have a list of objects $object = [$object] unless ref $object eq 'ARRAY'; @@ -1632,37 +1663,74 @@ sub fire_object_event { $logger->info("EVENTS = " . OpenSRF::Utils::JSON->perl2JSON(\@event_ids)); - my $resp = $self->simplereq( - 'open-ils.trigger', - 'open-ils.trigger.event_group.fire', - \@event_ids); + my $resp; + if (not defined $client) { + $resp = $self->simplereq( + 'open-ils.trigger', + 'open-ils.trigger.event_group.fire', + \@event_ids); + } else { + $resp = $self->patientreq( + $client, + "open-ils.trigger", "open-ils.trigger.event_group.fire", + \@event_ids + ); + } - return undef unless $resp and $resp->{events} and @{$resp->{events}}; + if($resp and $resp->{events} and @{$resp->{events}}) { - return $e->retrieve_action_trigger_event([ - $resp->{events}->[0]->id, - {flesh => 1, flesh_fields => {atev => ['template_output', 'error_output']}} - ]); + $e->xact_begin; + $final_resp = $e->retrieve_action_trigger_event([ + $resp->{events}->[0]->id, + {flesh => 1, flesh_fields => {atev => ['template_output', 'error_output']}} + ]); + $e->rollback; + } } else { $object = $$object[0] if ref $object eq 'ARRAY'; - my $event_id = $self->simplereq( - 'open-ils.trigger', $auto_method, $def->id, $object, $context_org, $user_data); - - my $resp = $self->simplereq( - 'open-ils.trigger', - 'open-ils.trigger.event.fire', - $event_id); - - return undef unless $resp and $resp->{event}; - - return $e->retrieve_action_trigger_event([ - $resp->{event}->id, - {flesh => 1, flesh_fields => {atev => ['template_output', 'error_output']}} - ]); + my $event_id; + my $resp; + + if (not defined $client) { + $event_id = $self->simplereq( + 'open-ils.trigger', + $auto_method, $def->id, $object, $context_org, $user_data + ); + + $resp = $self->simplereq( + 'open-ils.trigger', + 'open-ils.trigger.event.fire', + $event_id + ); + } else { + $event_id = $self->patientreq( + $client, + 'open-ils.trigger', + $auto_method, $def->id, $object, $context_org, $user_data + ); + + $resp = $self->patientreq( + $client, + 'open-ils.trigger', + 'open-ils.trigger.event.fire', + $event_id + ); + } + + if($resp and $resp->{event}) { + $e->xact_begin; + $final_resp = $e->retrieve_action_trigger_event([ + $resp->{event}->id, + {flesh => 1, flesh_fields => {atev => ['template_output', 'error_output']}} + ]); + $e->rollback; + } } + + return $final_resp; } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Booking.pm b/Open-ILS/src/perlmods/OpenILS/Application/Booking.pm index ff9572ac79..5e75fa840d 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Booking.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Booking.pm @@ -1291,12 +1291,13 @@ sub get_bresv_by_returnable_resource_barcode { }) or return $e->die_event; if (@$rows < 1) { + $e->rollback; return $rows; } else { # More than one result might be possible, but we don't want to return # more than one at this time. my $id = $rows->[0]->{"id"}; - return $e->retrieve_booking_reservation([ + my $resp =$e->retrieve_booking_reservation([ $id, { "flesh" => 2, "flesh_fields" => { @@ -1305,6 +1306,8 @@ sub get_bresv_by_returnable_resource_barcode { } } ]) or $e->die_event; + $e->rollback; + return $resp; } } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Cat.pm b/Open-ILS/src/perlmods/OpenILS/Application/Cat.pm index c6e579bae6..df16b4e0ed 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Cat.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Cat.pm @@ -214,6 +214,116 @@ sub template_overlay_biblio_record_entry { } __PACKAGE__->register_method( + method => "template_overlay_container", + api_name => "open-ils.cat.container.template_overlay", + stream => 1, + signature => q# + Overlays biblio.record_entry MARC values + @param auth The authtoken + @param container The container, um, containing the records to be updated by the template + @param template The overlay template, or nothing and the method will look for a negative bib id in the container + @return Stream of hashes record id in the key "record" and t or f for the success of the overlay operation in key "success" + # +); + +__PACKAGE__->register_method( + method => "template_overlay_container", + api_name => "open-ils.cat.container.template_overlay.background", + stream => 1, + signature => q# + Overlays biblio.record_entry MARC values + @param auth The authtoken + @param container The container, um, containing the records to be updated by the template + @param template The overlay template, or nothing and the method will look for a negative bib id in the container + @return Cache key to check for status of the container overlay + # +); + +sub template_overlay_container { + my($self, $conn, $auth, $container, $template) = @_; + my $e = new_editor(authtoken=>$auth, xact=>1); + return $e->die_event unless $e->checkauth; + + my $actor = OpenSRF::AppSession->create('open-ils.actor') if ($self->api_name =~ /background$/); + + my $items = $e->search_container_biblio_record_entry_bucket_item({ bucket => $container }); + + my $titem; + if (!$template) { + ($titem) = grep { $_->target_biblio_record_entry < 0 } @$items; + if (!$titem) { + $e->rollback; + return undef; + } + $items = [grep { $_->target_biblio_record_entry > 0 } @$items]; + + $template = $e->retrieve_biblio_record_entry( $titem->target_biblio_record_entry )->marc; + } + + my $responses = []; + my $some_failed = 0; + + $self->respond_complete( + $actor->request('open-ils.actor.anon_cache.set_value', $auth, res_list => $responses)->gather(1) + ) if ($actor); + + for my $item ( @$items ) { + my $rec = $e->retrieve_biblio_record_entry($item->target_biblio_record_entry); + next unless $rec; + + my $success = 'f'; + if ($e->allowed('UPDATE_RECORD', $rec->owner, $rec)) { + $success = $e->json_query( + { from => [ 'vandelay.template_overlay_bib_record', $template, $rec->id ] } + )->[0]->{'vandelay.template_overlay_bib_record'}; + } + + $some_failed++ if ($success eq 'f'); + + if ($actor) { + push @$responses, { record => $rec->id, success => $success }; + $actor->request('open-ils.actor.anon_cache.set_value', $auth, res_list => $responses); + } else { + $conn->respond({ record => $rec->id, success => $success }); + } + + if ($success eq 't') { + unless ($e->delete_container_biblio_record_entry_bucket_item($item)) { + $e->rollback; + if ($actor) { + push @$responses, { complete => 1, success => 'f' }; + $actor->request('open-ils.actor.anon_cache.set_value', $auth, res_list => $responses); + return undef; + } else { + return { complete => 1, success => 'f' }; + } + } + } + } + + if ($titem && !$some_failed) { + return $e->die_event unless ($e->delete_container_biblio_record_entry_bucket_item($titem)); + } + + if ($e->commit) { + if ($actor) { + push @$responses, { complete => 1, success => 't' }; + $actor->request('open-ils.actor.anon_cache.set_value', $auth, res_list => $responses); + } else { + return { complete => 1, success => 't' }; + } + } else { + if ($actor) { + push @$responses, { complete => 1, success => 'f' }; + $actor->request('open-ils.actor.anon_cache.set_value', $auth, res_list => $responses); + } else { + return { complete => 1, success => 'f' }; + } + } + return undef; +} + +__PACKAGE__->register_method( method => "update_biblio_record_entry", api_name => "open-ils.cat.biblio.record_entry.update", signature => q/ diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Circ.pm b/Open-ILS/src/perlmods/OpenILS/Application/Circ.pm index f85c754828..89a06fa31c 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Circ.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Circ.pm @@ -1487,7 +1487,7 @@ __PACKAGE__->register_method( sub fire_circ_events { my($self, $conn, $auth, $org_id, $event_def, $hook, $granularity, $target_ids, $user_data) = @_; - my $e = new_editor(authtoken => $auth); + my $e = new_editor(authtoken => $auth, xact => 1); return $e->event unless $e->checkauth; my $targets; @@ -1502,6 +1502,10 @@ sub fire_circ_events { return $e->event unless $e->allowed('VIEW_CIRCULATIONS', $org_id); $targets = $e->batch_retrieve_action_circulation($target_ids); } + $e->rollback; # FIXME using transaction because of pgpool/slony setups, but not + # simply making this method authoritative because of weirdness + # with transaction handling in A/T code that causes rollback + # failure down the line if handling many targets return undef unless @$targets; return $U->fire_object_event($event_def, $hook, $targets, $org_id, $granularity, $user_data); diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm index 0463d292c7..66330c9425 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm @@ -1196,7 +1196,8 @@ sub get_circ_policy { max_fine_rule => $max_fine_rule->name, max_fine => $self->get_max_fine_amount($max_fine_rule), fine_interval => $recurring_fine_rule->recurrence_interval, - renewal_remaining => $duration_rule->max_renewals + renewal_remaining => $duration_rule->max_renewals, + duration_date_ceiling => $duration_rule->date_ceiling }; $policy->{duration} = $duration_rule->shrt @@ -1760,10 +1761,12 @@ sub build_checkout_circ_object { my $recurring = $self->recurring_fines_rule; my $copy = $self->copy; my $patron = $self->patron; + my $duration_date_ceiling; if( $duration ) { my $policy = $self->get_circ_policy($duration, $recurring, $max); + $duration_date_ceiling = $policy->{duration_date_ceiling}; my $dname = $duration->name; my $mname = $max->name; @@ -1816,7 +1819,7 @@ sub build_checkout_circ_object { # if a patron is renewing, 'requestor' will be the patron $circ->circ_staff($self->editor->requestor->id); - $circ->due_date( $self->create_due_date($circ->duration) ) if $circ->duration; + $circ->due_date( $self->create_due_date($circ->duration, $duration_date_ceiling) ) if $circ->duration; $self->circ($circ); } @@ -2016,7 +2019,7 @@ sub apply_modified_due_date { sub create_due_date { - my( $self, $duration ) = @_; + my( $self, $duration, $date_ceiling ) = @_; # if there is a raw time component (e.g. from postgres), # turn it into an interval that interval_to_seconds can parse @@ -2028,6 +2031,14 @@ sub create_due_date { # add the circ duration $due_date->add(seconds => OpenSRF::Utils->interval_to_seconds($duration)); + if($date_ceiling) { + my $cdate = DateTime::Format::ISO8601->new->parse_datetime(cleanse_ISO8601($date_ceiling)); + if ($cdate > DateTime->now and $cdate < $due_date) { + $logger->info("circulator: overriding due date with date ceiling: $date_ceiling"); + $due_date = $cdate; + } + } + # return ISO8601 time with timezone return $due_date->strftime('%FT%T%z'); } @@ -2323,7 +2334,6 @@ sub do_checkin { $self->update_copy; } } - $logger->info("LFW XXX: way down here"); # LFW XXX if($self->claims_never_checked_out and $U->ou_ancestor_setting_value($self->circ->circ_lib, 'circ.claim_never_checked_out.mark_missing')) { @@ -2682,7 +2692,7 @@ sub do_hold_notify { return; } - $logger->warn("circulator: * hold notify failed for hold $holdid"); + $logger->debug("circulator: * hold notify cancelled or failed for hold $holdid"); } else { $logger->info("circulator: Not sending hold notification since the patron has no email address"); diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Circ/CopyLocations.pm b/Open-ILS/src/perlmods/OpenILS/Application/Circ/CopyLocations.pm index 5ace445bfa..d7f0773293 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Circ/CopyLocations.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Circ/CopyLocations.pm @@ -182,7 +182,39 @@ sub fetch_loc { return $cl; } +__PACKAGE__->register_method( + api_name => "open-ils.circ.copy_location_order.update", + method => 'update_clo', + argc => 2, +); + +sub update_clo { + my($self, $client, $auth, $orders) = @_; + return [] unless $orders and @$orders; + + my $e = new_editor(authtoken => $auth, xact =>1); + return $e->die_event unless $e->checkauth; + + my $org = $$orders[0]->org; + return $e->die_event unless $e->allowed('ADMIN_COPY_LOCATION_ORDER', $org); + # clear out the previous order entries + my $existing = $e->search_asset_copy_location_order({org => $org}); + $e->delete_asset_copy_location_order($_) or return $e->die_event for @$existing; + + # create the new order entries + my $progress = 0; + for my $order (@$orders) { + return $e->die_event(OpenILS::Event->new('BAD_PARAMS')) unless $order->org == $org; + $e->create_asset_copy_location_order($order) or return $e->die_event; + $client->respond({maximum => scalar(@$orders), progress => $progress}) unless ($progress++ % 10); + } + + # fetch the new entries + $orders = $e->search_asset_copy_location_order({org => $org}); + $e->commit; + return {orders => $orders}; +} -23; +1; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm index 22b2188607..020999a332 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm @@ -34,6 +34,8 @@ use OpenILS::Application::Actor::Friends; use DateTime; use DateTime::Format::ISO8601; use OpenSRF::Utils qw/:datetime/; +use Digest::MD5 qw(md5_hex); +use OpenSRF::Utils::Cache; my $apputils = "OpenILS::Application::AppUtils"; my $U = $apputils; @@ -119,10 +121,10 @@ __PACKAGE__->register_method( sub create_hold { my( $self, $conn, $auth, $hold ) = @_; + return -1 unless $hold; my $e = new_editor(authtoken=>$auth, xact=>1); - return $e->event unless $e->checkauth; + return $e->die_event unless $e->checkauth; - return -1 unless $hold; my $override = 1 if $self->api_name =~ /override/; my @events; @@ -133,8 +135,8 @@ sub create_hold { if( $requestor->id ne $hold->usr ) { # Make sure the requestor is allowed to place holds for # the recipient if they are not the same people - $recipient = $e->retrieve_actor_user($hold->usr) or return $e->event; - $e->allowed('REQUEST_HOLDS', $recipient->home_ou) or return $e->event; + $recipient = $e->retrieve_actor_user($hold->usr) or return $e->die_event; + $e->allowed('REQUEST_HOLDS', $recipient->home_ou) or return $e->die_event; } # If the related org setting tells us to, block if patron privs have expired @@ -172,27 +174,30 @@ sub create_hold { push( @events, OpenILS::Event->new('HOLD_ITEM_CHECKED_OUT')) if $checked_out; if ( $t eq OILS_HOLD_TYPE_METARECORD ) { - return $e->event unless $e->allowed('MR_HOLDS', $porg); + return $e->die_event unless $e->allowed('MR_HOLDS', $porg); } elsif ( $t eq OILS_HOLD_TYPE_TITLE ) { - return $e->event unless $e->allowed('TITLE_HOLDS', $porg); + return $e->die_event unless $e->allowed('TITLE_HOLDS', $porg); } elsif ( $t eq OILS_HOLD_TYPE_VOLUME ) { - return $e->event unless $e->allowed('VOLUME_HOLDS', $porg); + return $e->die_event unless $e->allowed('VOLUME_HOLDS', $porg); } elsif ( $t eq OILS_HOLD_TYPE_ISSUANCE ) { - return $e->event unless $e->allowed('ISSUANCE_HOLDS', $porg); + return $e->die_event unless $e->allowed('ISSUANCE_HOLDS', $porg); } elsif ( $t eq OILS_HOLD_TYPE_COPY ) { - return $e->event unless $e->allowed('COPY_HOLDS', $porg); + return $e->die_event unless $e->allowed('COPY_HOLDS', $porg); } elsif ( $t eq OILS_HOLD_TYPE_FORCE ) { - return $e->event unless $e->allowed('COPY_HOLDS', $porg); + return $e->die_event unless $e->allowed('COPY_HOLDS', $porg); } elsif ( $t eq OILS_HOLD_TYPE_RECALL ) { - return $e->event unless $e->allowed('COPY_HOLDS', $porg); + return $e->die_event unless $e->allowed('COPY_HOLDS', $porg); } if( @events ) { - $override or return \@events; + if (!$override) { + $e->rollback; + return \@events; + } for my $evt (@events) { next unless $evt; my $name = $evt->{textcode}; - return $e->event unless $e->allowed("$name.override", $porg); + return $e->die_event unless $e->allowed("$name.override", $porg); } } @@ -208,7 +213,7 @@ sub create_hold { $hold->requestor($e->requestor->id); $hold->request_lib($e->requestor->ws_ou); $hold->selection_ou($hold->pickup_lib) unless $hold->selection_ou; - $hold = $e->create_action_hold_request($hold) or return $e->event; + $hold = $e->create_action_hold_request($hold) or return $e->die_event; $e->commit; @@ -528,14 +533,20 @@ __PACKAGE__->register_method( sub uncancel_hold { my($self, $client, $auth, $hold_id) = @_; my $e = new_editor(authtoken=>$auth, xact=>1); - return $e->event unless $e->checkauth; + return $e->die_event unless $e->checkauth; my $hold = $e->retrieve_action_hold_request($hold_id) or return $e->die_event; return $e->die_event unless $e->allowed('CANCEL_HOLDS', $hold->request_lib); - return 0 if $hold->fulfillment_time; - return 1 unless $hold->cancel_time; + if ($hold->fulfillment_time) { + $e->rollback; + return 0; + } + unless ($hold->cancel_time) { + $e->rollback; + return 1; + } # if configured to reset the request time, also reset the expire time if($U->ou_ancestor_setting_value( @@ -583,22 +594,25 @@ sub cancel_hold { my($self, $client, $auth, $holdid, $cause, $note) = @_; my $e = new_editor(authtoken=>$auth, xact=>1); - return $e->event unless $e->checkauth; + return $e->die_event unless $e->checkauth; my $hold = $e->retrieve_action_hold_request($holdid) - or return $e->event; + or return $e->die_event; if( $e->requestor->id ne $hold->usr ) { - return $e->event unless $e->allowed('CANCEL_HOLDS'); + return $e->die_event unless $e->allowed('CANCEL_HOLDS'); } - return 1 if $hold->cancel_time; + if ($hold->cancel_time) { + $e->rollback; + return 1; + } # If the hold is captured, reset the copy status if( $hold->capture_time and $hold->current_copy ) { my $copy = $e->retrieve_asset_copy($hold->current_copy) - or return $e->event; + or return $e->die_event; if( $copy->status == OILS_COPY_STATUS_ON_HOLDS_SHELF ) { $logger->info("canceling hold $holdid whose item is on the holds shelf"); @@ -630,11 +644,15 @@ sub cancel_hold { $hold->cancel_cause($cause); $hold->cancel_note($note); $e->update_action_hold_request($hold) - or return $e->event; + or return $e->die_event; delete_hold_copy_maps($self, $e, $hold->id); $e->commit; + + $U->create_events_for_hook('hold_request.cancel.staff', $hold, $hold->pickup_lib) + if $e->requestor->id != $hold->usr; + return 1; } @@ -698,7 +716,10 @@ sub update_hold { my $e = new_editor(authtoken=>$auth, xact=>1); return $e->die_event unless $e->checkauth; my $resp = update_hold_impl($self, $e, $hold, $values); - return $resp if $U->event_code($resp); + if ($U->event_code($resp)) { + $e->rollback; + return $resp; + } $e->commit; # FIXME: update_hold_impl already does $e->commit ?? return $resp; } @@ -1033,11 +1054,7 @@ sub retrieve_hold_queue_status_impl { # fetch cut_in_line and request_time since they're in the order_by # and we're asking for distinct values select => {ahr => ['id', 'cut_in_line', 'request_time']}, - from => { - ahr => { - ahcm => {type => 'left'} # there may be no copy maps - } - }, + from => { ahr => 'ahcm' }, order_by => [ { "class" => "ahr", @@ -1049,28 +1066,40 @@ sub retrieve_hold_queue_status_impl { { "class" => "ahr", "field" => "request_time" } ], distinct => 1, - where => { - '-or' => [ + where => { + '+ahcm' => { + target_copy => { + in => { + select => {ahcm => ['target_copy']}, + from => 'ahcm', + where => {hold => $hold->id} + } + } + } + } + }); + + if (!@$q_holds) { # none? maybe we don't have a map ... + $q_holds = $e->json_query({ + select => {ahr => ['id', 'cut_in_line', 'request_time']}, + from => 'ahr', + order_by => [ { - '+ahcm' => { - target_copy => { - in => { - select => {ahcm => ['target_copy']}, - from => 'ahcm', - where => {hold => $hold->id} - } - } - } + "class" => "ahr", + "field" => "cut_in_line", + "transform" => "coalesce", + "params" => [ 0 ], + "direction" => "desc" }, - { - '+ahr' => { - hold_type => $hold->hold_type, - target => $hold->target - } - } - ] - }, - }); + { "class" => "ahr", "field" => "request_time" } + ], + where => { + hold_type => $hold->hold_type, + target => $hold->target + } + }); + } + my $qpos = 1; for my $h (@$q_holds) { @@ -1256,23 +1285,157 @@ __PACKAGE__->register_method( sub print_hold_pull_list { my($self, $client, $auth, $org_id) = @_; - my $e = new_editor(authtoken=>$auth, xact=>1); - return $e->die_event unless $e->checkauth; + my $e = new_editor(authtoken=>$auth); + return $e->event unless $e->checkauth; $org_id = (defined $org_id) ? $org_id : $e->requestor->ws_ou; - return $e->die_event unless $e->allowed('VIEW_HOLD', $org_id); + return $e->event unless $e->allowed('VIEW_HOLD', $org_id); my $hold_ids = $U->storagereq( 'open-ils.storage.direct.action.hold_request.pull_list.id_list.current_copy_circ_lib.status_filtered.atomic', $org_id, 10000); return undef unless @$hold_ids; + $client->status(new OpenSRF::DomainObject::oilsContinueStatus); + # Holds will /NOT/ be in order after this ... my $holds = $e->search_action_hold_request({id => $hold_ids}, {substream => 1}); $client->status(new OpenSRF::DomainObject::oilsContinueStatus); - return $U->fire_object_event(undef, 'ahr.format.pull_list', $holds, $org_id); + # ... so we must resort. + my $hold_map = +{map { $_->id => $_ } @$holds}; + my $sorted_holds = []; + push @$sorted_holds, $hold_map->{$_} foreach @$hold_ids; + + return $U->fire_object_event( + undef, "ahr.format.pull_list", $sorted_holds, + $org_id, undef, undef, $client + ); + +} + +__PACKAGE__->register_method( + method => "print_hold_pull_list_stream", + stream => 1, + api_name => "open-ils.circ.hold_pull_list.print.stream", + signature => { + desc => 'Returns a stream of fleshed holds', + params => [ + { desc => 'Authtoken', type => 'string'}, + { desc => 'Hash of optional param: Org unit ID (defaults to workstation org unit), limit, offset, sort (array of: acplo.position, call_number, request_time)', + type => 'object' + }, + ], + return => { + desc => 'A stream of fleshed holds', + type => 'object' + } + } +); + +sub print_hold_pull_list_stream { + my($self, $client, $auth, $params) = @_; + + my $e = new_editor(authtoken=>$auth); + return $e->die_event unless $e->checkauth; + + delete($$params{org_id}) unless (int($$params{org_id})); + delete($$params{limit}) unless (int($$params{limit})); + delete($$params{offset}) unless (int($$params{offset})); + delete($$params{chunk_size}) unless (int($$params{chunk_size})); + delete($$params{chunk_size}) if ($$params{chunk_size} && $$params{chunk_size} > 50); # keep the size reasonable + $$params{chunk_size} ||= 10; + + $$params{org_id} = (defined $$params{org_id}) ? $$params{org_id}: $e->requestor->ws_ou; + return $e->die_event unless $e->allowed('VIEW_HOLD', $$params{org_id }); + + my $sort = []; + if ($$params{sort} && @{ $$params{sort} }) { + for my $s (@{ $$params{sort} }) { + if ($s eq 'acplo.position') { + push @$sort, { + "class" => "acplo", "field" => "position", + "transform" => "coalesce", "params" => [999] + }; + } elsif ($s eq 'call_number') { + push @$sort, {"class" => "acn", "field" => "label"}; + } elsif ($s eq 'request_time') { + push @$sort, {"class" => "ahr", "field" => "request_time"}; + } + } + } else { + push @$sort, {"class" => "ahr", "field" => "request_time"}; + } + + my $holds_ids = $e->json_query( + { + "select" => {"ahr" => ["id"]}, + "from" => { + "ahr" => { + "acp" => { + "field" => "id", + "fkey" => "current_copy", + "filter" => { + "circ_lib" => $$params{org_id}, "status" => [0,7] + }, + "join" => { + "acn" => { + "field" => "id", + "fkey" => "call_number" + }, + "acplo" => { + "field" => "org", + "fkey" => "circ_lib", + "type" => "left", + "filter" => { + "location" => {"=" => {"+acp" => "location"}} + } + } + } + } + } + }, + "where" => { + "+ahr" => { + "capture_time" => undef, + "cancel_time" => undef, + "-or" => [ + {"expire_time" => undef }, + {"expire_time" => {">" => "now"}} + ] + } + }, + (@$sort ? (order_by => $sort) : ()), + ($$params{limit} ? (limit => $$params{limit}) : ()), + ($$params{offset} ? (offset => $$params{offset}) : ()) + }, {"substream" => 1} + ) or return $e->die_event; + + $logger->info("about to stream back " . scalar(@$holds_ids) . " holds"); + + my @chunk; + for my $hid (@$holds_ids) { + push @chunk, $e->retrieve_action_hold_request([ + $hid->{"id"}, { + "flesh" => 3, + "flesh_fields" => { + "ahr" => ["usr", "current_copy"], + "au" => ["card"], + "acp" => ["location", "call_number"], + "acn" => ["record"] + } + } + ]); + + if (@chunk >= $$params{chunk_size}) { + $client->respond( \@chunk ); + @chunk = (); + } + } + $client->respond_complete( \@chunk ) if (@chunk); + $e->disconnect; + return undef; } @@ -1332,7 +1495,7 @@ sub create_hold_notify { return $e->die_event unless $e->allowed('CREATE_HOLD_NOTIFICATION', $patron->home_ou); - $note->notify_staff($e->requestor->id); + $note->notify_staff($e->requestor->id); $e->create_action_hold_notification($note) or return $e->die_event; $e->commit; return $note->id; @@ -1430,14 +1593,14 @@ sub _reset_hold { if( $hold->capture_time and $hold->current_copy ) { my $copy = $e->retrieve_asset_copy($hold->current_copy) - or return $e->event; + or return $e->die_event; if( $copy->status == OILS_COPY_STATUS_ON_HOLDS_SHELF ) { $logger->info("setting copy to status 'reshelving' on hold retarget"); $copy->status(OILS_COPY_STATUS_RESHELVING); $copy->editor($e->requestor->id); $copy->edit_date('now'); - $e->update_asset_copy($copy) or return $e->event; + $e->update_asset_copy($copy) or return $e->die_event; } elsif( $copy->status == OILS_COPY_STATUS_IN_TRANSIT ) { @@ -1452,7 +1615,10 @@ sub _reset_hold { $logger->info("Aborting transit [$transid] on hold [$hid] reset..."); my $evt = OpenILS::Application::Circ::Transit::__abort_transit($e, $trans, $copy, 1); $logger->info("Transit abort completed with result $evt"); - return $evt unless "$evt" eq 1; + unless ("$evt" eq 1) { + $e->rollback; + return $evt; + } } } } @@ -1463,7 +1629,7 @@ sub _reset_hold { $hold->clear_shelf_time; $hold->clear_shelf_expire_time; - $e->update_action_hold_request($hold) or return $e->event; + $e->update_action_hold_request($hold) or return $e->die_event; $e->commit; $U->storagereq( @@ -1585,8 +1751,8 @@ sub fetch_captured_holds { my( $self, $conn, $auth, $org ) = @_; my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; - return $e->event unless $e->allowed('VIEW_HOLD'); # XXX rely on editor perm + return $e->die_event unless $e->checkauth; + return $e->die_event unless $e->allowed('VIEW_HOLD'); # XXX rely on editor perm $org ||= $e->requestor->ws_ou; @@ -1639,6 +1805,84 @@ sub fetch_captured_holds { } __PACKAGE__->register_method( + method => "print_expired_holds_stream", + api_name => "open-ils.circ.captured_holds.expired.print.stream", + stream => 1 +); + +sub print_expired_holds_stream { + my ($self, $client, $auth, $params) = @_; + + # No need to check specific permissions: we're going to call another method + # that will do that. + my $e = new_editor("authtoken" => $auth); + return $e->die_event unless $e->checkauth; + + delete($$params{org_id}) unless (int($$params{org_id})); + delete($$params{limit}) unless (int($$params{limit})); + delete($$params{offset}) unless (int($$params{offset})); + delete($$params{chunk_size}) unless (int($$params{chunk_size})); + delete($$params{chunk_size}) if ($$params{chunk_size} && $$params{chunk_size} > 50); # keep the size reasonable + $$params{chunk_size} ||= 10; + + $$params{org_id} = (defined $$params{org_id}) ? $$params{org_id}: $e->requestor->ws_ou; + + my @hold_ids = $self->method_lookup( + "open-ils.circ.captured_holds.id_list.expired_on_shelf.retrieve" + )->run($auth, $params->{"org_id"}); + + if (!@hold_ids) { + $e->disconnect; + return; + } elsif (defined $U->event_code($hold_ids[0])) { + $e->disconnect; + return $hold_ids[0]; + } + + $logger->info("about to stream back up to " . scalar(@hold_ids) . " expired holds"); + + while (@hold_ids) { + my @hid_chunk = splice @hold_ids, 0, $params->{"chunk_size"}; + + my $result_chunk = $e->json_query({ + "select" => { + "acp" => ["barcode"], + "au" => [qw/ + first_given_name second_given_name family_name alias + /], + "acn" => ["label"], + "bre" => ["marc"], + "acpl" => ["name"] + }, + "from" => { + "ahr" => { + "acp" => { + "field" => "id", "fkey" => "current_copy", + "join" => { + "acn" => { + "field" => "id", "fkey" => "call_number", + "join" => { + "bre" => { + "field" => "id", "fkey" => "record" + } + } + }, + "acpl" => {"field" => "id", "fkey" => "location"} + } + }, + "au" => {"field" => "id", "fkey" => "usr"} + } + }, + "where" => {"+ahr" => {"id" => \@hid_chunk}} + }) or return $e->die_event; + $client->respond($result_chunk); + } + + $e->disconnect; + undef; +} + +__PACKAGE__->register_method( method => "check_title_hold_batch", api_name => "open-ils.circ.title_hold.is_possible.batch", stream => 1, @@ -2248,6 +2492,7 @@ sub find_nearest_permitted_hold { copy => $copy, pickup_lib => $hold->pickup_lib, request_lib => $rlib, + retarget => 1 } ); @@ -2447,6 +2692,7 @@ sub uber_hold_impl { patron_first => $user->first_given_name, patron_last => $user->family_name, patron_barcode => $card->barcode, + patron_alias => $user->alias, %$details }; } @@ -2510,6 +2756,86 @@ sub find_hold_mvr { return ( $U->record_to_mvr($title), $volume, $copy, $issuance ); } +__PACKAGE__->register_method( + method => 'clear_shelf_cache', + api_name => 'open-ils.circ.hold.clear_shelf.get_cache', + stream => 1, + signature => { + desc => q/ + Returns the holds processed with the given cache key + / + } +); + +sub clear_shelf_cache { + my($self, $client, $auth, $cache_key, $chunk_size) = @_; + my $e = new_editor(authtoken => $auth, xact => 1); + return $e->die_event unless $e->checkauth and $e->allowed('VIEW_HOLD'); + + $chunk_size ||= 25; + my $hold_data = OpenSRF::Utils::Cache->new('global')->get_cache($cache_key); + + if (!$hold_data) { + $logger->info("no hold data found in cache"); # XXX TODO return event + $e->rollback; + return undef; + } + + my $maximum = 0; + foreach (keys %$hold_data) { + $maximum += scalar(@{ $hold_data->{$_} }); + } + $client->respond({"maximum" => $maximum, "progress" => 0}); + + for my $action (sort keys %$hold_data) { + while (@{$hold_data->{$action}}) { + my @hid_chunk = splice @{$hold_data->{$action}}, 0, $chunk_size; + + my $result_chunk = $e->json_query({ + "select" => { + "acp" => ["barcode"], + "au" => [qw/ + first_given_name second_given_name family_name alias + /], + "acn" => ["label"], + "bre" => ["marc"], + "acpl" => ["name"], + "ahr" => ["id"] + }, + "from" => { + "ahr" => { + "acp" => { + "field" => "id", "fkey" => "current_copy", + "join" => { + "acn" => { + "field" => "id", "fkey" => "call_number", + "join" => { + "bre" => { + "field" => "id", "fkey" => "record" + } + } + }, + "acpl" => {"field" => "id", "fkey" => "location"} + } + }, + "au" => {"field" => "id", "fkey" => "usr"} + } + }, + "where" => {"+ahr" => {"id" => \@hid_chunk}} + }, {"substream" => 1}) or return $e->die_event; + + $client->respond([ + map { + +{"action" => $action, "hold_details" => $_} + } @$result_chunk + ]); + } + } + + $e->rollback; + return undef; +} + __PACKAGE__->register_method( method => 'clear_shelf_process', @@ -2532,6 +2858,7 @@ sub clear_shelf_process { my $e = new_editor(authtoken=>$auth, xact => 1); $e->checkauth or return $e->die_event; + my $cache = OpenSRF::Utils::Cache->new('global'); $org_id ||= $e->requestor->ws_ou; $e->allowed('UPDATE_HOLD', $org_id) or return $e->die_event; @@ -2544,13 +2871,16 @@ sub clear_shelf_process { pickup_lib => $org_id, cancel_time => undef, fulfillment_time => undef, - shelf_time => {'!=' => undef} + shelf_time => {'!=' => undef}, + capture_time => {'!=' => undef}, + current_copy => {'!=' => undef}, }, { idlist => 1 } ); - my @holds; + my $chunk_size = 25; # chunked status updates + my $counter = 0; for my $hold_id (@$hold_ids) { $logger->info("Clear shelf processing hold $hold_id"); @@ -2577,51 +2907,47 @@ sub clear_shelf_process { } push(@holds, $hold); + $client->respond({maximum => scalar(@holds), progress => $counter}) if ( (++$counter % $chunk_size) == 0); } if ($e->commit) { + my %cache_data = ( + hold => [], + transit => [], + shelf => [] + ); + for my $hold (@holds) { my $copy = $hold->current_copy; - my ($alt_hold) = __PACKAGE__->find_nearest_permitted_hold($e, $copy, $e->requestor, 1); if($alt_hold) { - # copy is needed for a hold - $client->respond({action => 'hold', copy => $copy, hold_id => $hold->id}); + push(@{$cache_data{hold}}, $hold->id); # copy is needed for a hold } elsif($copy->circ_lib != $e->requestor->ws_ou) { - # copy needs to transit - $client->respond({action => 'transit', copy => $copy, hold_id => $hold->id}); + push(@{$cache_data{transit}}, $hold->id); # copy needs to transit } else { - # copy needs to go back to the shelf - $client->respond({action => 'shelf', copy => $copy, hold_id => $hold->id}); + push(@{$cache_data{shelf}}, $hold->id); # copy needs to go back to the shelf } } - # tell the client we're done - $client->respond_complete; - - # fire off the hold cancelation trigger - my $trigger = OpenSRF::AppSession->connect('open-ils.trigger'); - - for my $hold (@holds) { - - my $req = $trigger->request( - 'open-ils.trigger.event.autocreate', - 'hold_request.cancel.expire_holds_shelf', - $hold, $org_id); + my $cache_key = md5_hex(time . $$ . rand()); + $logger->info("clear_shelf_cache: storing under $cache_key"); + $cache->put_cache($cache_key, \%cache_data, 7200); # TODO: 2 hours. configurable? - # wait for response so don't flood the service - $req->recv; - } + # tell the client we're done + $client->respond_complete({cache_key => $cache_key}); - $trigger->disconnect; + # fire off the hold cancelation trigger and wait for response so don't flood the service + $U->create_events_for_hook( + 'hold_request.cancel.expire_holds_shelf', + $_, $org_id, undef, undef, 1) for @holds; } else { # tell the client we're done @@ -2858,7 +3184,7 @@ sub change_hold_title { my( $self, $client, $auth, $new_bib_id, $bib_ids ) = @_; my $e = new_editor(authtoken=>$auth, xact=>1); - return $e->event unless $e->checkauth; + return $e->die_event unless $e->checkauth; my $holds = $e->search_action_hold_request( [ diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm b/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm index 7dad749036..96ca5d2f3e 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm @@ -1108,6 +1108,7 @@ sub staged_search { my $search_duration; my $user_offset = $search_hash->{offset} || 0; # user-specified offset my $user_limit = $search_hash->{limit} || 10; + my $ignore_facet_classes = $search_hash->{ignore_facet_classes}; $user_offset = ($user_offset >= 0) ? $user_offset : 0; $user_limit = ($user_limit >= 0) ? $user_limit : 10; @@ -1262,7 +1263,7 @@ sub staged_search { } ); - cache_facets($facet_key, $new_ids, $IAmMetabib) if $docache; + cache_facets($facet_key, $new_ids, $IAmMetabib, $ignore_facet_classes) if $docache; return undef; } @@ -1368,10 +1369,14 @@ __PACKAGE__->register_method( sub cache_facets { # add facets for this search to the facet cache - my($key, $results, $metabib) = @_; + my($key, $results, $metabib, $ignore) = @_; my $data = $cache->get_cache($key); $data ||= {}; + if (!ref($ignore)) { + $ignore = ['identifier']; # ignore the identifier class by default + } + return undef unless (@$results); # The query we're constructing @@ -1398,11 +1403,13 @@ sub cache_facets { }, from => { mfae => { - mmrsm => { field => 'source', fkey => 'source' } + mmrsm => { field => 'source', fkey => 'source' }, + cmf => { field => 'id', fkey => 'field' } } }, where => { - '+mmrsm' => { $count_field => $results } + '+mmrsm' => { $count_field => $results }, + '+cmf' => { field_class => { 'not in' => $ignore } } } } ); diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/asset.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/asset.pm index dca9fe461b..39eb01b2ad 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/asset.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/asset.pm @@ -10,7 +10,7 @@ use base qw/asset/; __PACKAGE__->table( 'asset_copy_location' ); __PACKAGE__->columns( Primary => qw/id/ ); -__PACKAGE__->columns( Essential => qw/name owning_lib holdable hold_verify opac_visible circulate/ ); +__PACKAGE__->columns( Essential => qw/name owning_lib holdable hold_verify opac_visible circulate label_prefix label_suffix/ ); #------------------------------------------------------------------------------- package asset::copy_location_order; @@ -35,7 +35,7 @@ use base qw/asset/; __PACKAGE__->table( 'asset_call_number' ); __PACKAGE__->columns( Primary => qw/id/ ); __PACKAGE__->columns( Essential => qw/record label creator create_date editor - edit_date record label owning_lib deleted/ ); + edit_date record label owning_lib deleted label_class label_sortkey/ ); #------------------------------------------------------------------------------- package asset::call_number_note; @@ -56,7 +56,7 @@ __PACKAGE__->columns( Essential => qw/call_number barcode creator create_date ed fine_level circulate deposit price ref opac_visible circ_as_type circ_modifier deposit_amount location mint_condition holdable dummy_title dummy_author deleted alert_message - age_protect floating/ ); + age_protect floating cost status_changed_time/ ); #------------------------------------------------------------------------------- package asset::stat_cat; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/permission.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/permission.pm index 724774041b..f56e1e585e 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/permission.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/permission.pm @@ -16,7 +16,7 @@ use base qw/permission/; __PACKAGE__->table('permission_grp_tree'); __PACKAGE__->columns(Primary => qw/id/); __PACKAGE__->columns(Essential => qw/name parent description perm_interval - application_perm usergroup/); + application_perm usergroup hold_priority/); #------------------------------------------------------------------------------- package permission::usr_grp_map; use base qw/permission/; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/serial.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/serial.pm index 3e8f382293..7879cb7068 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/serial.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/CDBI/serial.pm @@ -46,5 +46,16 @@ __PACKAGE__->columns( Essential => qw/call_number barcode creator create_date ed holdable dummy_title dummy_author deleted alert_message label age_protect floating label_sort_key contents/ ); +#------------------------------------------------------------------------------- +package serial::record_entry; +use base qw/serial/; + +__PACKAGE__->table( 'serial_record_entry' ); +__PACKAGE__->columns( Primary => qw/id/ ); +__PACKAGE__->columns( Essential => qw/active record create_date creator + deleted edit_date editor id last_xact_id marc source + owning_lib/ ); + + 1; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Driver/Pg/QueryParser.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Driver/Pg/QueryParser.pm index ba596fbff6..94db0a4780 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Driver/Pg/QueryParser.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Driver/Pg/QueryParser.pm @@ -419,11 +419,11 @@ sub toSQL { } if (($filters{preferred_language} || $self->QueryParser->default_preferred_language) && ($filters{preferred_language_multiplier} || $self->QueryParser->default_preferred_language_multiplier)) { - $rel = "($rel * CASE WHEN FIRST(mrd.item_lang) = ". $self->QueryParser->quote_value( $filters{preferred_language} ? $filters{preferred_language} : $self->QueryParser->default_preferred_language ) . " THEN "; - $rel .= $filters{preferred_language_multiplier} ? $filters{preferred_language_multiplier} : $self->QueryParser->default_preferred_language_multiplier; - $rel .= " ELSE 1 END)"; + my $pl = $self->QueryParser->quote_value( $filters{preferred_language} ? $filters{preferred_language} : $self->QueryParser->default_preferred_language ); + my $plw = $filters{preferred_language_multiplier} ? $filters{preferred_language_multiplier} : $self->QueryParser->default_preferred_language_multiplier; + $rel = "($rel * COALESCE( NULLIF( FIRST(mrd.item_lang) = $pl , FALSE )::INT * $plw, 1))"; } - $rel .= "::NUMERIC"; + $rel .= '::NUMERIC'; for my $f ( qw/audience vr_format item_type item_form lit_form language bib_level/ ) { my $col = $f; @@ -449,44 +449,22 @@ sub toSQL { $desc = 'DESC' if ($self->find_modifier('descending')); if ($sort_filter eq 'rel') { # relevance ranking flips sort dir - if ($desc eq 'ASC') { + if ($desc eq 'ASC') { $desc = 'DESC'; } else { $desc = 'ASC'; } } else { if ($sort_filter eq 'title') { - my $default = $desc eq 'DESC' ? ' ' : 'zzzzzz'; - $rank = <<" SQL"; -( COALESCE( FIRST (( - SELECT frt.value - FROM metabib.full_rec frt - WHERE frt.record = m.source - AND frt.tag = 'tnf' - AND frt.subfield = 'a' - LIMIT 1 - )),'$default'))::TEXT - SQL + $rank = "FIRST((SELECT frt.value FROM metabib.full_rec frt WHERE frt.record = m.source AND frt.tag = 'tnf' AND frt.subfield = 'a' LIMIT 1))"; } elsif ($sort_filter eq 'pubdate') { - my $default = $desc eq 'DESC' ? '0' : '99999'; - $rank = "COALESCE( FIRST(NULLIF(LPAD(REGEXP_REPLACE(mrd.date1, E'\\\\D+', '0', 'g'),4,'0'),'0000')), '$default' )::INT"; + $rank = "FIRST(mrd.date1)::NUMERIC"; } elsif ($sort_filter eq 'create_date') { - $rank = "( FIRST (( SELECT create_date FROM biblio.record_entry rbr WHERE rbr.id = m.source)) )::TIMESTAMPTZ"; + $rank = "FIRST((SELECT create_date FROM biblio.record_entry rbr WHERE rbr.id = m.source))"; } elsif ($sort_filter eq 'edit_date') { - $rank = "( FIRST (( SELECT edit_date FROM biblio.record_entry rbr WHERE rbr.id = m.source)) )::TIMESTAMPTZ"; + $rank = "FIRST((SELECT edit_date FROM biblio.record_entry rbr WHERE rbr.id = m.source))"; } elsif ($sort_filter eq 'author') { - my $default = $desc eq 'DESC' ? ' ' : 'zzzzzz'; - $rank = <<" SQL" -( COALESCE( FIRST (( - SELECT LTRIM(fra.value) - FROM metabib.full_rec fra - WHERE fra.record = m.source - AND fra.tag LIKE '1%' - AND fra.subfield = 'a' - ORDER BY fra.tag::text::int - LIMIT 1 - )),'$default'))::TEXT - SQL + $rank = "FIRST((SELECT fra.value FROM metabib.full_rec fra WHERE fra.record = m.source AND fra.tag LIKE '1%' AND fra.subfield = 'a' ORDER BY fra.tag LIMIT 1))"; } else { # default to rel ranking $rank = $rel; @@ -532,7 +510,7 @@ SELECT $key AS id, ARRAY_ACCUM(DISTINCT m.source) AS records, $rel AS rel, $rank AS rank, - COALESCE( FIRST(NULLIF(LPAD(REGEXP_REPLACE(mrd.date1, E'\\\\D+', '0', 'g'),4,'0'),'0000')), '0' )::INT AS tie_break + FIRST(mrd.date1) AS tie_break FROM metabib.metarecord_source_map m JOIN metabib.rec_descriptor mrd ON (m.source = mrd.record) $$flat_plan{from} @@ -550,7 +528,7 @@ SELECT $key AS id, $bib_level AND $$flat_plan{where} GROUP BY 1 - ORDER BY 4 $desc, 5 DESC, 3 DESC + ORDER BY 4 $desc NULLS LAST, 5 DESC NULLS LAST, 3 DESC LIMIT $core_limit SQL @@ -570,13 +548,13 @@ sub rel_bump { return '' if (!@$only_atoms); if ($bump eq 'first_word') { - return " /* first_word */ COALESCE(NULLIF( (naco_normalize(".$node->table_alias.".value) ~ ('^'||naco_normalize(".$self->QueryParser->quote_value($only_atoms->[0]->content).")))::BOOL::INT, 0 ) * $multiplier, 1)"; + return " /* first_word */ COALESCE(NULLIF( (naco_normalize(".$node->table_alias.".value) ~ ('^'||naco_normalize(".$self->QueryParser->quote_value($only_atoms->[0]->content)."))), FALSE )::INT * $multiplier, 1)"; } elsif ($bump eq 'full_match') { return " /* full_match */ COALESCE(NULLIF( (naco_normalize(".$node->table_alias.".value) ~ ('^'||". - join( "||' '||", map { "naco_normalize(".$self->QueryParser->quote_value($_->content).")" } @$only_atoms )."||'\$'))::BOOL::INT, 0 ) * $multiplier, 1)"; + join( "||' '||", map { "naco_normalize(".$self->QueryParser->quote_value($_->content).")" } @$only_atoms )."||'\$')), FALSE )::INT * $multiplier, 1)"; } elsif ($bump eq 'word_order') { return " /* word_order */ COALESCE(NULLIF( (naco_normalize(".$node->table_alias.".value) ~ (". - join( "||'.*'||", map { "naco_normalize(".$self->QueryParser->quote_value($_->content).")" } @$only_atoms )."))::BOOL::INT, 0 ) * $multiplier, 1)"; + join( "||'.*'||", map { "naco_normalize(".$self->QueryParser->quote_value($_->content).")" } @$only_atoms ).")), FALSE )::INT * $multiplier, 1)"; } return ''; @@ -605,21 +583,24 @@ sub flatten { my $node_rank = $node->rank . " * ${talias}.weight"; my $core_limit = $self->QueryParser->core_limit || 25000; - $from .= "\n\tLEFT JOIN (\n\t\tSELECT fe.*, fe_weight.weight /* search */\n\t\t FROM $table AS fe"; + $from .= "\n\tLEFT JOIN (\n\t\tSELECT fe.*, fe_weight.weight, x.tsq /* search */\n\t\t FROM $table AS fe"; $from .= "\n\t\t\tJOIN config.metabib_field AS fe_weight ON (fe_weight.id = fe.field)"; - $from .= "\n\t\t WHERE fe.index_vector @@ (" .$node->tsquery . ')'; + $from .= "\n\t\t\tJOIN (SELECT ".$node->tsquery ." AS tsq ) AS x ON (fe.index_vector @@ x.tsq)"; my @bump_fields; if (@{$node->fields} > 0) { @bump_fields = @{$node->fields}; - $from .= "\n\t\t\tAND fe_weight.field_class = ". $self->QueryParser->quote_value($node->classname) ." AND fe_weight.name IN ("; - $from .= join(",", map { $self->QueryParser->quote_value($_) } @{$node->fields}) . ")"; + + my @field_ids; + push(@field_ids, $self->QueryParser->search_field_ids_by_class( $node->classname, $_ )->[0]) for (@bump_fields); + $from .= "\n\t\t\tWHERE fe_weight.id IN (". join(',', @field_ids) .")"; } else { @bump_fields = @{$self->QueryParser->search_fields->{$node->classname}}; } - $from .= "\n\t\tLIMIT $core_limit\n\t) AS $talias ON (m.source = $talias.source)"; + ###$from .= "\n\t\tLIMIT $core_limit"; + $from .= "\n\t) AS $talias ON (m.source = ${talias}.source)"; my %used_bumps; @@ -638,7 +619,7 @@ sub flatten { } $where .= '(' . $talias . ".id IS NOT NULL"; - $where .= ' AND ' . join(' AND ', map {"$talias.value ~* ".$self->QueryParser->quote_value($_)} @{$node->phrases}) if (@{$node->phrases}); + $where .= ' AND ' . join(' AND ', map {"${talias}.value ~* ".$self->QueryParser->quote_value($_)} @{$node->phrases}) if (@{$node->phrases}); $where .= ')'; push @rank_list, $node_rank; @@ -648,18 +629,16 @@ sub flatten { my $table = $node->table; my $talias = $node->table_alias; - $from .= "\n\tJOIN (\n\t\tSELECT * /* facet */\n\t\t FROM metabib.facet_entry\n\t\t WHERE ". - "SUBSTRING(value,1,1024) IN (" . join(",", map { $self->QueryParser->quote_value($_) } @{$node->values}) . ")". - "\n\t\t\tAND field IN (SELECT id FROM config.metabib_field WHERE field_class = ". $self->QueryParser->quote_value($node->classname) ." AND facet_field"; - + my @field_ids; if (@{$node->fields} > 0) { - $from .= " AND name IN ("; - $from .= join(",", map { $self->QueryParser->quote_value($_) } @{$node->fields}) . ")"; + push(@field_ids, $self->QueryParser->facet_field_ids_by_class( $node->classname, $_ )->[0]) for (@{$node->fields}); + } else { + @field_ids = @{ $self->QueryParser->facet_field_ids_by_class( $node->classname ) }; } - $from .= ")"; - - $from .= "\n\t\t) AS $talias ON (m.source = $talias.source)"; + $from .= "\n\tJOIN /* facet */ metabib.facet_entry $talias ON (\n\t\tm.source = ${talias}.source\n\t\t". + "AND SUBSTRING(${talias}.value,1,1024) IN (" . join(",", map { $self->QueryParser->quote_value($_) } @{$node->values}) . ")\n\t\t". + "AND ${talias}.field IN (". join(',', @field_ids) . ")\n\t)"; $where .= 'TRUE'; @@ -836,7 +815,7 @@ sub tsquery { sub rank { my $self = shift; return $self->{rank} if ($self->{rank}); - return $self->{rank} = 'rank(' . $self->table_alias . '.index_vector, ' . $self->tsquery . ')'; + return $self->{rank} = 'rank(' . $self->table_alias . '.index_vector, ' . $self->table_alias . '.tsq)'; } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/action.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/action.pm index eff5a0a3f9..b520cbb6a3 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/action.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/action.pm @@ -286,14 +286,16 @@ sub nearest_hold { my $fifo = shift(); my $holdsort = isTrue($fifo) ? - "CASE WHEN h.cut_in_line IS TRUE THEN 0 ELSE 1 END, h.request_time, h.selection_depth DESC, p.prox " : - "p.prox, CASE WHEN h.cut_in_line IS TRUE THEN 0 ELSE 1 END, h.selection_depth DESC, h.request_time "; + "pgt.hold_priority, CASE WHEN h.cut_in_line IS TRUE THEN 0 ELSE 1 END, h.request_time, h.selection_depth DESC, p.prox " : + "p.prox, pgt.hold_priority, CASE WHEN h.cut_in_line IS TRUE THEN 0 ELSE 1 END, h.selection_depth DESC, h.request_time "; my $ids = action::hold_request->db_Main->selectcol_arrayref(<<" SQL", {}, $here, $cp, $age); SELECT h.id FROM action.hold_request h JOIN actor.org_unit_proximity p ON (p.from_org = ? AND p.to_org = h.pickup_lib) JOIN action.hold_copy_map hm ON (hm.hold = h.id) + JOIN actor.usr au ON (au.id = h.usr) + JOIN permission.grp_tree pgt ON (au.profile = pgt.id) WHERE hm.target_copy = ? AND (AGE(NOW(),h.request_time) >= CAST(? AS INTERVAL) OR p.prox = 0) AND h.capture_time IS NULL @@ -1271,11 +1273,12 @@ sub new_hold_copy_targeter { $$prox_list[0] = [ grep { - ''.$_->circ_lib eq $pu_lib + ''.$_->circ_lib eq $pu_lib && + ( $_->status == 0 || $_->status == 7 ) } @good_copies ]; - $all_copies = [grep {''.$_->circ_lib ne $pu_lib } @good_copies]; + $all_copies = [grep { $_->status == 0 || $_->status == 7 } grep {''.$_->circ_lib ne $pu_lib } @good_copies]; # $all_copies is now a list of copies not at the pickup library my $best = choose_nearest_copy($hold, $prox_list); @@ -1294,7 +1297,7 @@ sub new_hold_copy_targeter { my %circ_lib_map = map { (''.$_->circ_lib => 1) } @$all_copies; my $circ_lib_list = [keys %circ_lib_map]; - my $cstore = OpenSRF::AppSession->connect('open-ils.cstore'); + my $cstore = OpenSRF::AppSession->create('open-ils.cstore'); # Grab the "biggest" loop for this hold so far my $current_loop = $cstore->request( @@ -1639,162 +1642,6 @@ __PACKAGE__->register_method( my $locations; my $statuses; my %cache = (titles => {}, cns => {}); -sub hold_copy_targeter { - my $self = shift; - my $client = shift; - my $check_expire = shift; - my $one_hold = shift; - - $self->{user_filter} = OpenSRF::AppSession->create('open-ils.circ'); - $self->{user_filter}->connect; - $self->{client} = $client; - - my $time = time; - $check_expire ||= '12h'; - $check_expire = interval_to_seconds( $check_expire ); - - my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = gmtime(time() - $check_expire); - $year += 1900; - $mon += 1; - my $expire_threshold = sprintf( - '%s-%0.2d-%0.2dT%0.2d:%0.2d:%0.2d-00', - $year, $mon, $mday, $hour, $min, $sec - ); - - - $statuses ||= [ config::copy_status->search(holdable => 't') ]; - - $locations ||= [ asset::copy_location->search(holdable => 't') ]; - - my $holds; - - %cache = (titles => {}, cns => {}); - - try { - if ($one_hold) { - $holds = [ action::hold_request->search(id => $one_hold) ]; - } else { - $holds = [ action::hold_request->search_where( - { capture_time => undef, - prev_check_time => { '<=' => $expire_threshold }, - }, - { order_by => 'request_time,prev_check_time' } ) ]; - push @$holds, action::hold_request->search_where( - { capture_time => undef, - prev_check_time => undef, - }, - { order_by => 'request_time' } ); - } - } catch Error with { - my $e = shift; - die "Could not retrieve uncaptured hold requests:\n\n$e\n"; - }; - - for my $hold (@$holds) { - try { - #action::hold_request->db_Main->begin_work; - if ($self->method_lookup('open-ils.storage.transaction.current')->run) { - $client->respond("Cleaning up after previous transaction\n"); - $self->method_lookup('open-ils.storage.transaction.rollback')->run; - } - $self->method_lookup('open-ils.storage.transaction.begin')->run( $client ); - $client->respond("Processing hold ".$hold->id."...\n"); - - my $copies; - - $copies = $self->metarecord_hold_capture($hold) if ($hold->hold_type eq 'M'); - $self->{client}->status( new OpenSRF::DomainObject::oilsContinueStatus ); - - $copies = $self->title_hold_capture($hold) if ($hold->hold_type eq 'T'); - $self->{client}->status( new OpenSRF::DomainObject::oilsContinueStatus ); - - $copies = $self->volume_hold_capture($hold) if ($hold->hold_type eq 'V'); - $self->{client}->status( new OpenSRF::DomainObject::oilsContinueStatus ); - - $copies = $self->copy_hold_capture($hold) if ($hold->hold_type eq 'C'); - - unless (ref $copies || !@$copies) { - $client->respond("\tNo copies available for targeting at all!\n"); - } - - my @good_copies; - for my $c (@$copies) { - next if ( grep {$c->id == $hold->current_copy} @good_copies); - push @good_copies, $c if ($c); - } - - $client->respond("\t".scalar(@good_copies)." (non-current) copies available for targeting...\n"); - - my $old_best = $hold->current_copy; - $hold->update({ current_copy => undef }); - - if (!scalar(@good_copies)) { - $client->respond("\tNo (non-current) copies available to fill the hold.\n"); - if ( $old_best && grep {$_->id == $hold->current_copy} @$copies ) { - $client->respond("\tPushing current_copy back onto the targeting list\n"); - push @good_copies, asset::copy->retrieve( $old_best ); - } else { - $client->respond("\tcurrent_copy is no longer available for targeting... NEXT HOLD, PLEASE!\n"); - next; - } - } - - my $prox_list; - $$prox_list[0] = [grep {$_->circ_lib == $hold->pickup_lib } @good_copies]; - $copies = [grep {$_->circ_lib != $hold->pickup_lib } @good_copies]; - - my $best = choose_nearest_copy($hold, $prox_list); - - if (!$best) { - $prox_list = create_prox_list( $self, $hold->pickup_lib, $copies ); - $best = choose_nearest_copy($hold, $prox_list); - } - - if ($old_best) { - # hold wasn't fulfilled, record the fact - - $client->respond("\tHold was not (but should have been) fulfilled by ".$old_best->id.".\n"); - action::unfulfilled_hold_list->create( - { hold => ''.$hold->id, - current_copy => ''.$old_best->id, - circ_lib => ''.$old_best->circ_lib, - }); - } - - if ($best) { - $hold->update( { current_copy => ''.$best->id } ); - $client->respond("\tTargeting copy ".$best->id." for hold fulfillment.\n"); - } - - $hold->update( { prev_check_time => 'now' } ); - $client->respond("\tUpdating hold ".$hold->id." with new 'current_copy' for hold fulfillment.\n"); - - $client->respond("\tProcessing of hold ".$hold->id." complete.\n"); - $self->method_lookup('open-ils.storage.transaction.commit')->run; - - #action::hold_request->dbi_commit; - - } otherwise { - my $e = shift; - $log->error("Processing of hold failed: $e"); - $client->respond("\tProcessing of hold failed!.\n\t\t$e\n"); - $self->method_lookup('open-ils.storage.transaction.rollback')->run; - #action::hold_request->dbi_rollback; - }; - } - - $self->{user_filter}->disconnect; - $self->{user_filter}->finish; - delete $$self{user_filter}; - return undef; -} -#__PACKAGE__->register_method( -# api_name => 'open-ils.storage.action.hold_request.copy_targeter', -# api_level => 0, -# stream => 1, -# method => 'hold_copy_targeter', -#); - sub copy_hold_capture { my $self = shift; @@ -1864,21 +1711,24 @@ sub choose_nearest_copy { for my $p ( 0 .. int( scalar(@$prox_list) - 1) ) { next unless (ref $$prox_list[$p]); - my @capturable = grep { $_->status == 0 || $_->status == 7 } @{ $$prox_list[$p] }; + my @capturable = @{ $$prox_list[$p] }; next unless (@capturable); my $rand = int(rand(scalar(@capturable))); - while (my ($c) = splice(@capturable,$rand)) { - return $c if ( OpenILS::Utils::PermitHold::permit_copy_hold( + my %seen = (); + while (my ($c) = splice(@capturable, $rand, 1)) { + return $c if !exists($seen{$c->id}) && ( OpenILS::Utils::PermitHold::permit_copy_hold( { title => $c->call_number->record->to_fieldmapper, title_descriptor => $c->call_number->record->record_descriptor->next->to_fieldmapper, patron => $hold->usr->to_fieldmapper, copy => $c->to_fieldmapper, requestor => $hold->requestor->to_fieldmapper, request_lib => $hold->request_lib->to_fieldmapper, - pickup_lib => $hold->pickup_lib->id, + pickup_lib => $hold->pickup_lib->id, + retarget => 1 } )); + $seen{$c->id}++; last unless(@capturable); $rand = int(rand(scalar(@capturable))); @@ -1898,15 +1748,16 @@ sub create_prox_list { my ($prox) = $self->method_lookup('open-ils.storage.asset.copy.proximity')->run( $cp, $lib ); next unless (defined($prox)); + my $copy_circ_lib = ''.$cp->circ_lib; # Fetch the weighting value for hold targeting, defaulting to 1 - $self->{target_weight}{$lib} ||= $actor->request( - 'open-ils.actor.ou_setting.ancestor_default' => $lib.'' => 'circ.holds.org_unit_target_weight' + $self->{target_weight}{$copy_circ_lib} ||= $actor->request( + 'open-ils.actor.ou_setting.ancestor_default' => $copy_circ_lib.'' => 'circ.holds.org_unit_target_weight' )->gather(1); - $self->{target_weight}{$lib} = $self->{target_weight}{$lib}{value} if (ref $self->{target_weight}{$lib}); - $self->{target_weight}{$lib} ||= 1; + $self->{target_weight}{$copy_circ_lib} = $self->{target_weight}{$copy_circ_lib}{value} if (ref $self->{target_weight}{$copy_circ_lib}); + $self->{target_weight}{$copy_circ_lib} ||= 1; $prox_list[$prox] = [] unless defined($prox_list[$prox]); - for my $w ( 1 .. $self->{target_weight}{$lib} ) { + for my $w ( 1 .. $self->{target_weight}{$copy_circ_lib} ) { push @{$prox_list[$prox]}, $cp; } } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/asset.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/asset.pm index 8f49f4b942..77b213da20 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/asset.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/asset.pm @@ -230,9 +230,9 @@ sub cn_browse_pagedown { $table cn where not deleted - and (upper(label) > ? or ( cn.id > ? and upper(label) = ? )) + and (oils_text_as_bytea(upper(label)) > ? or ( cn.id > ? and oils_text_as_bytea(upper(label)) = ? )) and owning_lib in ($orgs) - order by upper(label), 4, 2 + order by oils_text_as_bytea(upper(label)), 4, 2 limit $size; SQL @@ -285,9 +285,9 @@ sub cn_browse_pageup { $table cn where not deleted - and (upper(label) < ? or ( cn.id < ? and upper(label) = ? )) + and (oils_text_as_bytea(upper(label)) < ? or ( cn.id < ? and oils_text_as_bytea(upper(label)) = ? )) and owning_lib in ($orgs) - order by upper(label) desc, 4 desc, 2 desc + order by oils_text_as_bytea(upper(label)) desc, 4 desc, 2 desc limit $size ) as bar order by 1,4,2; @@ -343,9 +343,9 @@ sub cn_browse_target { $table cn where not deleted - and upper(label) < ? + and oils_text_as_bytea(upper(label)) < ? and owning_lib in ($orgs) - order by upper(label) desc, 4 desc, 2 desc + order by oils_text_as_bytea(upper(label)) desc, 4 desc, 2 desc limit $topsize ) as bar order by 1,4,2; @@ -361,9 +361,9 @@ sub cn_browse_target { $table cn where not deleted - and upper(label) >= ? + and oils_text_as_bytea(upper(label)) >= ? and owning_lib in ($orgs) - order by upper(label),4,2 + order by oils_text_as_bytea(upper(label)),4,2 limit $bottomsize; SQL diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/authority.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/authority.pm index 3f5127c63c..62091ce7e0 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/authority.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/authority.pm @@ -33,11 +33,7 @@ sub validate_tag { for my $t ( @tags ) { for my $search ( @searches ) { my $sf = $$search{subfield}; - my $term = NFD(lc($$search{term})); - - $term =~ s/\pM+//sgo; - $term =~ s/\pC+//sgo; - $term =~ s/\W+$//o; + my $term = OpenILS::Application::Storage::FTS::naco_normalize($$search{term}, $sf); $tag = [$tag] if (!ref($tag)); diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm index e13a7b317f..413bf8f108 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm @@ -57,7 +57,7 @@ sub ordered_records_from_metarecord { item_type, item_form, quality, - FIRST(COALESCE(LTRIM(SUBSTR( value, COALESCE(SUBSTRING(ind2 FROM '\\\\d+'),'0')::INT + 1 )),'zzzzzzzz')) AS title + FIRST(COALESCE(LTRIM(SUBSTR( value, COALESCE(SUBSTRING(ind2 FROM E'\\\\d+'),'0')::INT + 1 )),'zzzzzzzz')) AS title FROM ( SELECT rd.record, rd.item_type, @@ -556,7 +556,7 @@ sub biblio_multi_search_full_rec { if (lc($sort) eq 'pubdate') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(SUBSTRING(frp.value FROM '\\\\d+'),'9999')::INT + SELECT COALESCE(SUBSTRING(frp.value FROM E'\\\\d+'),'9999')::INT FROM $metabib_full_rec frp WHERE frp.record = f.record AND frp.tag = '260' @@ -575,7 +575,7 @@ sub biblio_multi_search_full_rec { } elsif (lc($sort) eq 'title') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM '\\\\d+'),'0')::INT + 1 )),'zzzzzzzz') + SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM E'\\\\d+'),'0')::INT + 1 )),'zzzzzzzz') FROM $metabib_full_rec frt WHERE frt.record = f.record AND frt.tag = '245' @@ -1196,7 +1196,7 @@ sub postfilter_search_class_fts { if (lc($sort) eq 'pubdate') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(SUBSTRING(frp.value FROM '\\\\d+'),'$number_default_sort')::INT + SELECT COALESCE(SUBSTRING(frp.value FROM E'\\\\d+'),'$number_default_sort')::INT FROM $metabib_full_rec frp WHERE frp.record = mr.master_record AND frp.tag = '260' @@ -1215,7 +1215,7 @@ sub postfilter_search_class_fts { } elsif (lc($sort) eq 'title') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM '\\\\d+'),'0')::INT + 1 )),'$string_default_sort') + SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM E'\\\\d+'),'0')::INT + 1 )),'$string_default_sort') FROM $metabib_full_rec frt WHERE frt.record = mr.master_record AND frt.tag = '245' @@ -1668,7 +1668,7 @@ sub postfilter_search_multi_class_fts { my $secondary_sort = <<" SORT"; ( FIRST (( - SELECT COALESCE(LTRIM(SUBSTR( sfrt.value, COALESCE(SUBSTRING(sfrt.ind2 FROM '\\\\d+'),'0')::INT + 1 )),'$string_default_sort') + SELECT COALESCE(LTRIM(SUBSTR( sfrt.value, COALESCE(SUBSTRING(sfrt.ind2 FROM E'\\\\d+'),'0')::INT + 1 )),'$string_default_sort') FROM $metabib_full_rec sfrt, $metabib_metarecord mr WHERE sfrt.record = mr.master_record @@ -1682,7 +1682,7 @@ sub postfilter_search_multi_class_fts { if (lc($sort) eq 'pubdate') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(SUBSTRING(frp.value FROM '\\\\d+'),'$number_default_sort')::INT + SELECT COALESCE(SUBSTRING(frp.value FROM E'\\\\d+'),'$number_default_sort')::INT FROM $metabib_full_rec frp WHERE frp.record = mr.master_record AND frp.tag = '260' @@ -1701,7 +1701,7 @@ sub postfilter_search_multi_class_fts { } elsif (lc($sort) eq 'title') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM '\\\\d+'),'0')::INT + 1 )),'$string_default_sort') + SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM E'\\\\d+'),'0')::INT + 1 )),'$string_default_sort') FROM $metabib_full_rec frt WHERE frt.record = mr.master_record AND frt.tag = '245' @@ -1711,7 +1711,7 @@ sub postfilter_search_multi_class_fts { RANK $secondary_sort = <<" SORT"; ( FIRST (( - SELECT COALESCE(SUBSTRING(sfrp.value FROM '\\\\d+'),'$number_default_sort')::INT + SELECT COALESCE(SUBSTRING(sfrp.value FROM E'\\\\d+'),'$number_default_sort')::INT FROM $metabib_full_rec sfrp WHERE sfrp.record = mr.master_record AND sfrp.tag = '260' @@ -2166,7 +2166,7 @@ sub biblio_search_multi_class_fts { if (lc($sort) eq 'pubdate') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(SUBSTRING(frp.value FROM '\\\\d{4}'),'$number_default_sort')::INT + SELECT COALESCE(SUBSTRING(frp.value FROM E'\\\\d{4}'),'$number_default_sort')::INT FROM $metabib_full_rec frp WHERE frp.record = b.id AND frp.tag = '260' @@ -2185,7 +2185,7 @@ sub biblio_search_multi_class_fts { } elsif (lc($sort) eq 'title') { $rank = <<" RANK"; ( FIRST (( - SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM '\\\\d+'),'0')::INT + 1 )),'$string_default_sort') + SELECT COALESCE(LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM E'\\\\d+'),'0')::INT + 1 )),'$string_default_sort') FROM $metabib_full_rec frt WHERE frt.record = b.id AND frt.tag = '245' @@ -2977,7 +2977,7 @@ sub query_parser_fts { my $metarecord = ($self->api_name =~ /metabib/ or $query->parse_tree->find_modifier('metabib') or $query->parse_tree->find_modifier('metarecord')) ? "'t'" : "'f'"; my $sth = metabib::metarecord_source_map->db_Main->prepare(<<" SQL"); - SELECT * + SELECT * /* bib search */ FROM search.query_parser_fts( $param_search_ou\:\:INT, $param_depth\:\:INT, diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/QueryParser.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/QueryParser.pm index 05ff6befee..6338384388 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/QueryParser.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/QueryParser.pm @@ -442,7 +442,7 @@ sub decompose { for my $alias ( @{$pkg->search_field_aliases->{$class}{$field}} ) { $alias = qr/$alias/; - s/\b$alias[:=]/$class\|$field:/g; + s/(^|\s+)$alias[:=]/$1$class\|$field:/g; } } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm b/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm index 92edc8e03b..de5bdf91d5 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm @@ -305,7 +305,7 @@ sub cn_browse { }, { flesh => 1, flesh_fields => { acn => [qw/record owning_lib/] }, - order_by => { acn => "label_sortkey, upper(label) desc, id desc, owning_lib desc" }, + order_by => { acn => "oils_text_as_bytea(label_sortkey) desc, oils_text_as_bytea(upper(label)) desc, id desc, owning_lib desc" }, limit => $before_limit, offset => abs($page) * $page_size - $before_offset, } @@ -323,7 +323,7 @@ sub cn_browse { }, { flesh => 1, flesh_fields => { acn => [qw/record owning_lib/] }, - order_by => { acn => "label_sortkey, upper(label), id, owning_lib" }, + order_by => { acn => "oils_text_as_bytea(label_sortkey), oils_text_as_bytea(upper(label)), id, owning_lib" }, limit => $after_limit, offset => abs($page) * $page_size - $after_offset, } @@ -428,7 +428,7 @@ sub cn_startwith { }, { flesh => 1, flesh_fields => { acn => [qw/record owning_lib/] }, - order_by => { acn => "label_sortkey, upper(label) desc, id desc, owning_lib desc" }, + order_by => { acn => "oils_text_as_bytea(label_sortkey) desc, oils_text_as_bytea(upper(label)) desc, id desc, owning_lib desc" }, limit => $limit, offset => $offset, } @@ -446,7 +446,7 @@ sub cn_startwith { }, { flesh => 1, flesh_fields => { acn => [qw/record owning_lib/] }, - order_by => { acn => "label_sortkey, upper(label), id, owning_lib" }, + order_by => { acn => "oils_text_as_bytea(label_sortkey), oils_text_as_bytea(upper(label)), id, owning_lib" }, limit => $limit, offset => $offset, } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Trigger.pm b/Open-ILS/src/perlmods/OpenILS/Application/Trigger.pm index 6a916c079e..ae83a7755a 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Trigger.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Trigger.pm @@ -7,6 +7,7 @@ use OpenSRF::EX qw/:try/; use OpenSRF::Utils::JSON; use OpenSRF::AppSession; +use OpenSRF::MultiSession; use OpenSRF::Utils::SettingsClient; use OpenSRF::Utils::Logger qw/$logger/; use OpenSRF::Utils qw/:datetime/; @@ -21,8 +22,16 @@ use OpenILS::Application::Trigger::EventGroup; my $log = 'OpenSRF::Utils::Logger'; +my $parallel_collect; +my $parallel_react; -sub initialize {} +sub initialize { + + my $conf = OpenSRF::Utils::SettingsClient->new; + $parallel_collect = $conf->config_value( apps => 'open-ils.trigger' => app_settings => parallel => 'collect') || 1; + $parallel_react = $conf->config_value( apps => 'open-ils.trigger' => app_settings => parallel => 'react') || 1; + +} sub child_init {} sub create_active_events_for_object { @@ -68,7 +77,7 @@ sub create_active_events_for_object { my $uid = $target->$ufield; $uid = $uid->id if (ref $uid); # fleshed user object, unflesh it - my $opt_in_setting = $editor->search_actor_usr_setting( + my $opt_in_setting = $editor->search_actor_user_setting( { usr => $uid, name => $def->opt_in_setting, value => 'true' @@ -154,7 +163,7 @@ sub create_event_for_object_and_def { my $uid = $target->$ufield; $uid = $uid->id if (ref $uid); # fleshed user object, unflesh it - my $opt_in_setting = $editor->search_actor_usr_setting( + my $opt_in_setting = $editor->search_actor_user_setting( { usr => $uid, name => $def->opt_in_setting, value => 'true' @@ -328,7 +337,7 @@ sub events_by_target { for (grep { $_ ne '-and' } keys %{$$filter{event}}); } - my $e = new_editor(); + my $e = new_editor(xact=>1); my $events = $e->json_query($query); @@ -469,7 +478,7 @@ sub create_batch_events { '-exists' => { from => 'aus', where => { - name => $def->id, + name => $def->opt_in_setting, usr => { '=' => { '+' . $hook_hash{$def->hook}->core_type => $def->usr_field } }, value=> 'true' } @@ -502,7 +511,6 @@ sub create_batch_events { $event->event_def( $def->id ); $event->run_time( $run_time ); $event->user_data( OpenSRF::Utils::JSON->perl2JSON($user_data) ) if (defined($user_data)); - $event->granularity($granularity) if (defined $granularity); $editor->create_action_trigger_event( $event ); @@ -547,6 +555,7 @@ sub fire_single_event { } $e->editor->disconnect; + OpenILS::Application::Trigger::Event->ClearObjectCache(); return { valid => $e->valid, @@ -575,6 +584,7 @@ sub fire_event_group { } $e->editor->disconnect; + OpenILS::Application::Trigger::Event->ClearObjectCache(); return { valid => $e->valid, @@ -594,18 +604,21 @@ sub pending_events { my $self = shift; my $client = shift; my $granularity = shift; - - my $editor = new_editor(); + my $granflag = shift; my $query = [{ state => 'pending', run_time => {'<' => 'now'} }, { order_by => { atev => [ qw/run_time add_time/] }, 'join' => 'atevdef' }]; if (defined $granularity) { - $query->[0]->{'+atevdef'} = {'-or' => [ {granularity => $granularity}, {granularity => undef} ] }; + if ($granflag) { + $query->[0]->{'+atevdef'} = {granularity => $granularity}; + } else { + $query->[0]->{'+atevdef'} = {'-or' => [ {granularity => $granularity}, {granularity => undef} ] }; + } } else { $query->[0]->{'+atevdef'} = {granularity => undef}; } - return $editor->search_action_trigger_event( + return new_editor(xact=>1)->search_action_trigger_event( $query, { idlist=> 1, timeout => 7200, substream => 1 } ); } @@ -615,12 +628,52 @@ __PACKAGE__->register_method( api_level=> 1 ); +sub gather_events { + my $self = shift; + my $client = shift; + my $e_ids = shift; + + $e_ids = [$e_ids] if (!ref($e_ids)); + + my @events; + for my $e_id (@$e_ids) { + my $e; + try { + $e = OpenILS::Application::Trigger::Event->new($e_id); + } catch Error with { + $logger->error("trigger: Event creation failed with ".shift()); + }; + + next if !$e or $e->event->state eq 'invalid'; + + try { + $e->build_environment; + } catch Error with { + $logger->error("trigger: Event environment building failed with ".shift()); + }; + + $e->editor->disconnect; + $e->environment->{EventProcessor} = undef; # remove circular ref for json encoding + $client->respond($e); + } + + OpenILS::Application::Trigger::Event->ClearObjectCache(); + + return undef; +} +__PACKAGE__->register_method( + api_name => 'open-ils.trigger.event.gather', + method => 'gather_events', + api_level=> 1 +); + sub grouped_events { my $self = shift; my $client = shift; my $granularity = shift; + my $granflag = shift; - my ($events) = $self->method_lookup('open-ils.trigger.event.find_pending')->run($granularity); + my ($events) = $self->method_lookup('open-ils.trigger.event.find_pending')->run($granularity, $granflag); my %groups = ( '*' => [] ); @@ -631,51 +684,67 @@ sub grouped_events { return \%groups; } - for my $e_id ( @$events ) { - $logger->info("trigger: processing event $e_id"); + my @fleshed_events; - # let the client know we're still chugging along TODO add osrf support for method_lookup $client's + if ($parallel_collect == 1 or @$events == 1) { # use method lookup + @fleshed_events = $self->method_lookup('open-ils.trigger.event.gather')->run($events); + } else { + my $self_multi = OpenSRF::MultiSession->new( + app => 'open-ils.trigger', + cap => $parallel_collect, + success_handler => sub { + my $self = shift; + my $req = shift; + + push @fleshed_events, + map { OpenILS::Application::Trigger::Event->new($_) } + map { $_->content } + @{ $req->{response} }; + }, + ); + + $self_multi->request( 'open-ils.trigger.event.gather' => $_ ) for ( @$events ); $client->status( new OpenSRF::DomainObject::oilsContinueStatus ); - my $e; - try { - $e = OpenILS::Application::Trigger::Event->new($e_id); - } catch Error with { - $logger->error("trigger: Event creation failed with ".shift()); - }; - - next unless $e; - - try { - $e->build_environment; - } catch Error with { - $logger->error("trigger: Event environment building failed with ".shift()); - }; + $self_multi->session_wait(1); + $client->status( new OpenSRF::DomainObject::oilsContinueStatus ); + } + for my $e (@fleshed_events) { if (my $group = $e->event->event_def->group_field) { # split the grouping link steps my @steps = split /\./, $group; + my $group_field = pop(@steps); # we didn't flesh to this, it's a field not an object + + my $node; + eval { + $node = $e->target; + $node = $node->$_() for ( @steps ); + }; - # find the grouping object - my $node = $e->target; - $node = $node->$_() for ( @steps ); + unless($node) { # should not get here, but to be safe.. + $e->update_state('invalid'); + next; + } - # get the pkey value for the grouping object on this event - my $node_ident = $node->Identity; - my $ident_value = $node->$node_ident(); + # get the grouping value for the grouping object on this event + my $ident_value = $node->$group_field(); + if(ref $ident_value) { + my $ident_field = $ident_value->Identity; + $ident_value = $ident_value->$ident_field() + } - # push this event onto the event+grouping_pkey_value stack + # push this event onto the event+grouping_value stack $groups{$e->event->event_def->id}{$ident_value} ||= []; push @{ $groups{$e->event->event_def->id}{$ident_value} }, $e; } else { # it's a non-grouped event push @{ $groups{'*'} }, $e; } - - $e->editor->disconnect; } + return \%groups; } __PACKAGE__->register_method( @@ -688,44 +757,80 @@ sub run_all_events { my $self = shift; my $client = shift; my $granularity = shift; - - my ($groups) = $self->method_lookup('open-ils.trigger.event.find_pending_by_group')->run($granularity); + my $granflag = shift; + + my ($groups) = $self->method_lookup('open-ils.trigger.event.find_pending_by_group')->run($granularity, $granflag); + $client->respond({"status" => "found"}) if (keys(%$groups) > 1 || @{$$groups{'*'}}); + + my $self_multi; + if ($parallel_react > 1 and (keys(%$groups) > 1 || @{$$groups{'*'}} > 1)) { + $self_multi = OpenSRF::MultiSession->new( + app => 'open-ils.trigger', + cap => $parallel_react, + session_hash_function => sub { + my $args = shift; + return $args->{target_id}; + }, + success_handler => sub { + my $me = shift; + my $req = shift; + $client->respond( $req->{response}->[0]->content ); + } + ); + } for my $def ( keys %$groups ) { if ($def eq '*') { $logger->info("trigger: run_all_events firing un-grouped events"); for my $event ( @{ $$groups{'*'} } ) { try { - $client->respond( - $self - ->method_lookup('open-ils.trigger.event.fire') - ->run($event) - ); + if ($self_multi) { + $event->environment->{EventProcessor} = undef; # remove circular ref for json encoding + $self_multi->request({target_id => $event->id}, 'open-ils.trigger.event.fire', $event); + } else { + $client->respond( + $self + ->method_lookup('open-ils.trigger.event.fire') + ->run($event) + ); + } } catch Error with { $logger->error("trigger: event firing failed with ".shift()); }; } - $logger->info("trigger: run_all_events completed firing un-grouped events"); + $logger->info("trigger: run_all_events completed queuing un-grouped events"); + $client->status( new OpenSRF::DomainObject::oilsContinueStatus ); } else { my $defgroup = $$groups{$def}; $logger->info("trigger: run_all_events firing events for grouped event def=$def"); for my $ident ( keys %$defgroup ) { + $logger->info("trigger: run_all_events firing group for grouped event def=$def and grp ident $ident"); try { - $client->respond( - $self - ->method_lookup('open-ils.trigger.event_group.fire') - ->run($$defgroup{$ident}) - ); + if ($self_multi) { + $_->environment->{EventProcessor} = undef for @{$$defgroup{$ident}}; # remove circular ref for json encoding + $self_multi->request({target_id => $ident}, 'open-ils.trigger.event_group.fire', $$defgroup{$ident}); + } else { + $client->respond( + $self + ->method_lookup('open-ils.trigger.event_group.fire') + ->run($$defgroup{$ident}) + ); + } + $client->status( new OpenSRF::DomainObject::oilsContinueStatus ); } catch Error with { $logger->error("trigger: event firing failed with ".shift()); }; } - $logger->info("trigger: run_all_events completed firing events for grouped event def=$def"); + $logger->info("trigger: run_all_events completed queuing events for grouped event def=$def"); } } - - + + $self_multi->session_wait(1) if ($self_multi); + $logger->info("trigger: run_all_events completed firing events"); + + $client->respond_complete(); + return undef; } __PACKAGE__->register_method( api_name => 'open-ils.trigger.event.run_all_pending', diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Event.pm b/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Event.pm index 5faef1bf6a..2ec5d1be61 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Event.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Event.pm @@ -2,13 +2,10 @@ package OpenILS::Application::Trigger::Event; use strict; use warnings; use OpenSRF::EX qw/:try/; use OpenSRF::Utils::JSON; - use OpenSRF::Utils::Logger qw/$logger/; - use OpenILS::Utils::Fieldmapper; use OpenILS::Utils::CStoreEditor q/:funcs/; use OpenILS::Application::Trigger::ModRunner; - use Safe; my $log = 'OpenSRF::Utils::Logger'; @@ -19,11 +16,17 @@ sub new { my $editor = shift; $class = ref($class) || $class; - return $id if (ref($id) && ref($id) eq $class); - my $standalone = $editor ? 0 : 1; $editor ||= new_editor(); + if (ref($id) && ref($id) eq $class) { + $id->environment->{EventProcessor} = $id + if ($id->environment->{complete}); # in case it came over an opensrf tube + $id->editor( $editor ); + $id->standalone( $standalone ); + return $id; + } + my $self = bless { id => $id, editor => $editor, standalone => $standalone } => $class; return $self->init() @@ -45,6 +48,10 @@ sub init { return $self if (!$self->id); + if ($self->standalone) { + $self->editor->xact_begin || return undef; + } + $self->event( $self->editor->retrieve_action_trigger_event([ $self->id, { @@ -57,6 +64,10 @@ sub init { ]) ); + if ($self->standalone) { + $self->editor->xact_rollback || return undef; + } + $self->user_data(OpenSRF::Utils::JSON->JSON2perl( $self->event->user_data )) if (defined( $self->event->user_data )); @@ -91,8 +102,21 @@ sub init { $meth =~ s/Fieldmapper:://; $meth =~ s/::/_/; + if ($self->standalone) { + $self->editor->xact_begin || return undef; + } + $self->target( $self->editor->$meth( $self->event->target ) ); + if ($self->standalone) { + $self->editor->xact_rollback || return undef; + } + + unless($self->target) { + $self->update_state('invalid'); + $self->valid(0); + } + return $self; } @@ -336,6 +360,7 @@ sub update_state { $e->state( $state ); $e->clear_start_time() if ($e->state eq 'pending'); + $e->complete_time( 'now' ) if ($e->state eq 'complete'); my $ok = $self->editor->update_action_trigger_event( $e ); if (!$ok) { @@ -397,7 +422,8 @@ sub build_environment { if ($self->event->event_def->group_field) { my @group_path = split(/\./, $self->event->event_def->group_field); - my $group_object = $self->_object_by_path( $self->target, undef, [], \@group_path ); + pop(@group_path); # the last part is a field, should not get fleshed + my $group_object = $self->_object_by_path( $self->target, undef, [], \@group_path ) if (@group_path); } $self->environment->{complete} = 1; @@ -426,17 +452,45 @@ sub _fm_class_by_hint { return $class; } +my %_object_by_path_cache = (); +sub ClearObjectCache { + for my $did ( keys %_object_by_path_cache ) { + my $phash = $_object_by_path_cache{$did}; + for my $path ( keys %$phash ) { + my $shash = $$phash{$path}; + for my $step ( keys %$shash ) { + my $fhash = $$shash{$step}; + for my $ffield ( keys %$fhash ) { + my $lhash = $$fhash{$ffield}; + for my $lfield ( keys %$lhash ) { + delete $$lhash{$lfield}; + } + delete $$fhash{$ffield}; + } + delete $$shash{$step}; + } + delete $$phash{$path}; + } + delete $_object_by_path_cache{$did}; + } +} + sub _object_by_path { my $self = shift; my $context = shift; my $collector = shift; my $label = shift; my $path = shift; + my $ed = shift; + my $outer = 0; + if (!$ed) { + $ed = new_editor(xact=>1); + $outer = 1; + } my $step = shift(@$path); - my $fhint = Fieldmapper->publish_fieldmapper->{$context->class_name}{links}{$step}{class}; my $fclass = $self->_fm_class_by_hint( $fhint ); @@ -460,10 +514,6 @@ sub _object_by_path { $meth =~ s/Fieldmapper:://; $meth =~ s/::/_/g; - my $ed = grep( /open-ils.cstore/, @{$fclass->Controller} ) ? - $self->editor : - new_rstore_editor(); - my $obj = $context->$step(); $logger->debug( @@ -472,11 +522,19 @@ sub _object_by_path { ); if (!ref $obj) { - $obj = $ed->$meth( - ($multi) ? - { $ffield => $context->$lfield() } : - $context->$lfield() - ); + + my $lval = $context->$lfield(); + + if(defined $lval) { + + my $def_id = $self->event->event_def->id; + my $str_path = join('.', @$path); + + $obj = $_object_by_path_cache{$def_id}{$str_path}{$step}{$ffield}{$lval} || + $ed->$meth( ($multi) ? { $ffield => $lval } : $lval); + + $_object_by_path_cache{$def_id}{$str_path}{$step}{$ffield}{$lval} ||= $obj; + } } if (@$path) { @@ -490,7 +548,7 @@ sub _object_by_path { for (@$obj_list) { my @path_clone = @$path; - $self->_object_by_path( $_, $collector, $label, \@path_clone ); + $self->_object_by_path( $_, $collector, $label, \@path_clone, $ed ); } $obj = $$obj_list[0] if (!$multi || $rtype eq 'might_have'); @@ -533,6 +591,7 @@ sub _object_by_path { } } + $ed->rollback if ($outer); return $obj; } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Trigger/EventGroup.pm b/Open-ILS/src/perlmods/OpenILS/Application/Trigger/EventGroup.pm index 34b942369e..2a3c6c632c 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Trigger/EventGroup.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Trigger/EventGroup.pm @@ -28,7 +28,7 @@ sub new { OpenILS::Application::Trigger::Event->new($_, $editor) } @ids ], - ids => \@ids, + ids => [ map { ref($_) ? $_->id : $_ } @ids ], editor => $editor } => $class; @@ -212,11 +212,11 @@ sub update_state { if (scalar(@oks) < scalar(@{ $self->ids })) { $self->editor->xact_rollback; return undef; - } else { - $ok = $self->editor->xact_commit; - } + } my $updated = $self->editor->retrieve_action_trigger_event($last_updated); + $ok = $self->editor->xact_commit; + if ($ok) { for my $event ( @{ $self->events } ) { my $e = $event->event; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Reactor.pm b/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Reactor.pm index 840964dde0..cc59bbd177 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Reactor.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Trigger/Reactor.pm @@ -57,13 +57,13 @@ my $_TT_helpers = { # returns the calculated copy price get_copy_price => sub { my $copy_id = shift; - return $U->get_copy_price(new_editor(), $copy_id); + return $U->get_copy_price(new_editor(xact=>1), $copy_id); }, # given a copy, returns the title and author in a hash get_copy_bib_basics => sub { my $copy_id = shift; - my $copy = new_editor()->retrieve_asset_copy([ + my $copy = new_editor(xact=>1)->retrieve_asset_copy([ $copy_id, { flesh => 2, @@ -156,6 +156,7 @@ sub run_TT { my $t_o = Fieldmapper::action_trigger::event_output->new; $t_o->data( ($error) ? $error : $output ); $t_o->is_error( ($error) ? 't' : 'f' ); + $logger->info("trigger: writing " . length($t_o->data) . " bytes to template output"); $env->{EventProcessor}->editor->xact_begin; $t_o = $env->{EventProcessor}->editor->create_action_trigger_event_output( $t_o ); diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Vandelay.pm b/Open-ILS/src/perlmods/OpenILS/Application/Vandelay.pm index 3572e81365..3920bd7770 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Vandelay.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Vandelay.pm @@ -68,9 +68,10 @@ sub create_bib_queue { return $e->die_event unless $e->allowed('CREATE_BIB_IMPORT_QUEUE'); $owner ||= $e->requestor->id; - return OpenILS::Event->new('BIB_QUEUE_EXISTS') - if $e->search_vandelay_bib_queue( - {name => $name, owner => $owner, queue_type => $type})->[0]; + if ($e->search_vandelay_bib_queue( {name => $name, owner => $owner, queue_type => $type})->[0]) { + $e->rollback; + return OpenILS::Event->new('BIB_QUEUE_EXISTS') + } my $queue = new Fieldmapper::vandelay::bib_queue(); $queue->name( $name ); @@ -106,9 +107,10 @@ sub create_auth_queue { return $e->die_event unless $e->allowed('CREATE_AUTHORITY_IMPORT_QUEUE'); $owner ||= $e->requestor->id; - return OpenILS::Event->new('AUTH_QUEUE_EXISTS') - if $e->search_vandelay_bib_queue( - {name => $name, owner => $owner, queue_type => $type})->[0]; + if ($e->search_vandelay_bib_queue({name => $name, owner => $owner, queue_type => $type})->[0]) { + $e->rollback; + return OpenILS::Event->new('AUTH_QUEUE_EXISTS') + } my $queue = new Fieldmapper::vandelay::authority_queue(); $queue->name( $name ); @@ -242,7 +244,7 @@ sub process_spool { } my $evt = check_queue_perms($e, $type, $queue); - return $evt if $evt; + return $evt if ($evt); my $cache = new OpenSRF::Utils::Cache(); @@ -395,8 +397,8 @@ __PACKAGE__->register_method( sub retrieve_queued_records { my($self, $conn, $auth, $queue_id, $options) = @_; - my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; + my $e = new_editor(authtoken => $auth, xact => 1); + return $e->die_event unless $e->checkauth; $options ||= {}; my $limit = $$options{limit} || 20; my $offset = $$options{offset} || 0; @@ -409,7 +411,7 @@ sub retrieve_queued_records { $queue = $e->retrieve_vandelay_authority_queue($queue_id) or return $e->die_event; } my $evt = check_queue_perms($e, $type, $queue); - return $evt if $evt; + return $evt if ($evt); my $class = ($type eq 'bib') ? 'vqbr' : 'vqar'; my $search = ($type eq 'bib') ? @@ -443,6 +445,7 @@ sub retrieve_queued_records { $rec->clear_marc if $$options{clear_marc}; $conn->respond($rec); } + $e->rollback; return undef; } @@ -481,10 +484,11 @@ __PACKAGE__->register_method( sub import_record_list { my($self, $conn, $auth, $rec_ids, $args) = @_; - my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; + my $e = new_editor(authtoken => $auth, xact => 1); + return $e->die_event unless $e->checkauth; $args ||= {}; my $err = import_record_list_impl($self, $conn, $rec_ids, $e->requestor, $args); + $e->rollback; return $err if $err; return {complete => 1}; } @@ -532,8 +536,8 @@ __PACKAGE__->register_method( ); sub import_queue { my($self, $conn, $auth, $q_id, $options) = @_; - my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; + my $e = new_editor(authtoken => $auth, xact => 1); + return $e->die_event unless $e->checkauth; $options ||= {}; my $type = $self->{record_type}; my $class = ($type eq 'bib') ? 'vqbr' : 'vqar'; @@ -549,6 +553,7 @@ sub import_queue { 'search_vandelay_queued_bib_record' : 'search_vandelay_queued_authority_record'; my $rec_ids = $e->$search($query, {idlist => 1}); my $err = import_record_list_impl($self, $conn, $rec_ids, $e->requestor, $options); + $e->rollback; return $err if $err; return {complete => 1}; } @@ -620,6 +625,7 @@ sub import_record_list_impl { my %bib_sources; my $editor = new_editor(); my $sources = $editor->search_config_bib_source({id => {'!=' => undef}}); + foreach my $src (@$sources) { $bib_sources{$src->id} = $src->source; } @@ -635,6 +641,7 @@ sub import_record_list_impl { $rec_class = 'vqar'; } + my @success_rec_ids; for my $rec_id (@$rec_ids) { my $overlay_target = $overlay_map->{$rec_id}; @@ -650,7 +657,7 @@ sub import_record_list_impl { ]); unless($rec) { - $conn->respond({total => $total, progress => ++$count, imported => $rec_id, err_event => $e->die_event}); + $conn->respond({total => $total, progress => ++$count, imported => $rec_id, err_event => $e->event}); $e->rollback; next; } @@ -746,6 +753,7 @@ sub import_record_list_impl { if($U->event_code($record)) { $e->event($record); + $e->rollback; } else { @@ -759,11 +767,11 @@ sub import_record_list_impl { } if($imported) { + push @success_rec_ids, $rec_id; $e->commit; } else { - $e->rollback; # Send an update whenever there's an error - $conn->respond({total => $total, progress => ++$count, imported => $rec_id, err_event => $e->die_event}); + $conn->respond({total => $total, progress => ++$count, imported => $rec_id, err_event => $e->event}); } $conn->respond({total => $total, progress => $count, imported => $rec_id}) if (!$report_all and ++$count % $step) == 0; @@ -789,6 +797,8 @@ sub import_record_list_impl { $e->rollback; } + import_record_asset_list_impl($conn, \@success_rec_ids, $requestor); + $conn->respond({total => $total, progress => $count}); return undef; } @@ -813,7 +823,7 @@ __PACKAGE__->register_method( sub owner_queue_retrieve { my($self, $conn, $auth, $owner_id, $filters) = @_; - my $e = new_editor(authtoken => $auth); + my $e = new_editor(authtoken => $auth, xact => 1); return $e->die_event unless $e->checkauth; $owner_id = $e->requestor->id; # XXX add support for viewing other's queues? my $queues; @@ -829,6 +839,7 @@ sub owner_queue_retrieve { [$search, {order_by => {vaq => 'lower(name)'}}]); } $conn->respond($_) for @$queues; + $e->rollback; return undef; } @@ -888,17 +899,18 @@ __PACKAGE__->register_method( sub queued_record_html { my($self, $conn, $auth, $rec_id) = @_; - my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; + my $e = new_editor(xact=>1,authtoken => $auth); + return $e->die_event unless $e->checkauth; my $rec; if($self->{record_type} eq 'bib') { $rec = $e->retrieve_vandelay_queued_bib_record($rec_id) - or return $e->event; + or return $e->die_event; } else { $rec = $e->retrieve_vandelay_queued_authority_record($rec_id) - or return $e->event; + or return $e->die_event; } + $e->rollback; return $U->simplereq( 'open-ils.search', 'open-ils.search.biblio.record.html', undef, 1, $rec->marc); @@ -924,17 +936,17 @@ __PACKAGE__->register_method( sub retrieve_queue_summary { my($self, $conn, $auth, $queue_id) = @_; - my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; + my $e = new_editor(xact=>1, authtoken => $auth); + return $e->die_event unless $e->checkauth; my $queue; my $type = $self->{record_type}; if($type eq 'bib') { $queue = $e->retrieve_vandelay_bib_queue($queue_id) - or return $e->event; + or return $e->die_event; } else { $queue = $e->retrieve_vandelay_authority_queue($queue_id) - or return $e->event; + or return $e->die_event; } my $evt = check_queue_perms($e, $type, $queue); @@ -953,7 +965,7 @@ sub retrieve_queue_summary { __PACKAGE__->register_method( api_name => "open-ils.vandelay.bib_record.list.asset.import", - method => 'import_record_list_assets', + method => 'noop_import_items', api_level => 1, argc => 2, stream => 1, @@ -961,48 +973,52 @@ __PACKAGE__->register_method( ); __PACKAGE__->register_method( api_name => "open-ils.vandelay.bib_record.queue.asset.import", - method => 'import_record_queue_assets', + method => 'noop_import_items', api_level => 1, argc => 2, stream => 1, record_type => 'bib' ); -sub import_record_list_assets { - my($self, $conn, $auth, $import_def, $rec_ids) = @_; - my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; - my $err = import_record_asset_list_impl($conn, $import_def, $rec_ids, $e->requestor); - return $err if $err; - return {complete => 1}; -} - -sub import_record_queue_assets { - my($self, $conn, $auth, $import_def, $q_id) = @_; - my $e = new_editor(authtoken => $auth); - return $e->event unless $e->checkauth; - my $rec_ids = $e->search_vandelay_queued_bib_record( - {queue => $q_id, import_time => {'!=' => undef}}, {idlist => 1}); - my $err = import_record_asset_list_impl($conn, $import_def, $rec_ids, $e->requestor); - return $err if $err; - return {complete => 1}; -} +sub noop_import_items { return {complete => 1} } + +#sub import_record_list_assets { +# my($self, $conn, $auth, $import_def, $rec_ids) = @_; +# my $e = new_editor(xact=>1, authtoken => $auth); +# return $e->die_event unless $e->checkauth; +# my $err = import_record_asset_list_impl($conn, $import_def, $rec_ids, $e->requestor); +# $e->rollback; +# return $err if $err; +# return {complete => 1}; +#} +# +#sub import_record_queue_assets { +# my($self, $conn, $auth, $import_def, $q_id) = @_; +# my $e = new_editor(xact=>1, authtoken => $auth); +# return $e->die_event unless $e->checkauth; +# my $rec_ids = $e->search_vandelay_queued_bib_record( +# {queue => $q_id, import_time => {'!=' => undef}}, {idlist => 1}); +# my $err = import_record_asset_list_impl($conn, $import_def, $rec_ids, $e->requestor); +# $e->rollback; +# return $err if $err; +# return {complete => 1}; +#} # -------------------------------------------------------------------------------- # Given a list of queued record IDs, imports all items attached to those records # -------------------------------------------------------------------------------- sub import_record_asset_list_impl { - my($conn, $import_def, $rec_ids, $requestor) = @_; + my($conn, $rec_ids, $requestor) = @_; my $total = @$rec_ids; my $try_count = 0; my $in_count = 0; - my $roe = new_editor(requestor => $requestor); + my $roe = new_editor(xact=> 1, requestor => $requestor); for my $rec_id (@$rec_ids) { my $rec = $roe->retrieve_vandelay_queued_bib_record($rec_id); next unless $rec and $rec->import_time; - my $item_ids = $roe->search_vandelay_import_item({definition => $import_def, record => $rec->id}, {idlist=>1}); + my $item_ids = $roe->search_vandelay_import_item({record => $rec->id}, {idlist=>1}); for my $item_id (@$item_ids) { my $e = new_editor(requestor => $requestor, xact => 1); @@ -1087,6 +1103,7 @@ sub import_record_asset_list_impl { respond_with_status($conn, $total, $try_count, ++$in_count, undef, imported_as => $copy->id); } } + $roe->rollback; return undef; } diff --git a/Open-ILS/src/perlmods/OpenILS/SIP.pm b/Open-ILS/src/perlmods/OpenILS/SIP.pm index e101e4be06..d22a5e60af 100644 --- a/Open-ILS/src/perlmods/OpenILS/SIP.pm +++ b/Open-ILS/src/perlmods/OpenILS/SIP.pm @@ -381,7 +381,7 @@ sub checkin { $xact->do_checkin( $self, $inst_id, $trans_date, $return_date, $current_loc, $item_props ); if ($xact->ok) { - $xact->patron($self->find_patron($item->{patron})); + $xact->patron($self->find_patron(usr => $xact->{circ_user_id})) if $xact->{circ_user_id}; delete $item->{patron}; delete $item->{due_date}; syslog('LOG_INFO', "OILS: Checkin succeeded"); diff --git a/Open-ILS/src/perlmods/OpenILS/SIP/Item.pm b/Open-ILS/src/perlmods/OpenILS/SIP/Item.pm index 5197da314d..7eb815c9eb 100644 --- a/Open-ILS/src/perlmods/OpenILS/SIP/Item.pm +++ b/Open-ILS/src/perlmods/OpenILS/SIP/Item.pm @@ -125,12 +125,15 @@ sub new { syslog('LOG_DEBUG', "OILS: Open circulation exists on $item_id : user = $bc"); } - $self->{id} = $item_id; - $self->{copy} = $copy; - $self->{volume} = $copy->call_number; - $self->{record} = $copy->call_number->record; + $self->{id} = $item_id; + $self->{copy} = $copy; + $self->{volume} = $copy->call_number; + $self->{record} = $copy->call_number->record; $self->{call_number} = $copy->call_number->label; - $self->{mods} = $U->record_to_mvr($self->{record}) if $self->{record}->marc; + $self->{mods} = $U->record_to_mvr($self->{record}) if $self->{record}->marc; + $self->{transit} = $self->fetch_transit; + $self->{hold} = $self->fetch_hold; + # use the non-translated version of the copy location as the # collection code, since it may be used for additional routing @@ -139,26 +142,12 @@ sub new { $e->retrieve_asset_copy_location([ $copy->location, {no_i18n => 1}])->name; - if ($copy->status->id == OILS_COPY_STATUS_IN_TRANSIT) { - my $transit = $e->search_action_transit_copy([ - { - target_copy => $copy->id, # NOT barcode ($self->id) - dest_recv_time => undef - }, - { - flesh => 1, - flesh_fields => { - atc => [ 'dest' ] - } - } - ])->[0]; - if ($transit) { - $self->{transit} = $transit; - $self->{destination_loc} = $transit->dest->shortname; - } else { - syslog('LOG_WARNING', "OILS: Item('$item_id') status is In Transit, but no action.transit_copy found!"); - } + if($self->{transit}) { + $self->{destination_loc} = $self->{transit}->dest->shortname; + + } elsif($self->{hold}) { + $self->{destination_loc} = $self->{hold}->pickup_lib->shortname; } syslog("LOG_DEBUG", "OILS: Item('$item_id'): found with title '%s'", $self->title_id); @@ -182,6 +171,71 @@ sub new { return $self; } +# fetch copy transit +sub fetch_transit { + my $self = shift; + my $copy = $self->{copy} or return; + my $e = OpenILS::SIP->editor(); + + if ($copy->status->id == OILS_COPY_STATUS_IN_TRANSIT) { + my $transit = $e->search_action_transit_copy([ + { + target_copy => $copy->id, # NOT barcode ($self->id) + dest_recv_time => undef + }, + { + flesh => 1, + flesh_fields => { + atc => ['dest'] + } + } + ])->[0]; + + syslog('LOG_WARNING', "OILS: Item(".$copy->barcode. + ") status is In Transit, but no action.transit_copy found!") unless $transit; + + return $transit; + } + + return undef; +} + +# fetch captured hold. +# Assume transit has already beeen fetched +sub fetch_hold { + my $self = shift; + my $copy = $self->{copy} or return; + my $e = OpenILS::SIP->editor(); + + if( ($copy->status->id == OILS_COPY_STATUS_ON_HOLDS_SHELF) || + ($self->{transit} and $self->{transit}->copy_status == OILS_COPY_STATUS_ON_HOLDS_SHELF) ) { + # item has been captured for a hold + + my $hold = $e->search_action_hold_request([ + { + current_copy => $copy->id, + capture_time => {'!=' => undef}, + cancel_time => undef, + fulfillment_time => undef + }, + { + limit => 1, + flesh => 1, + flesh_fields => { + ahr => ['pickup_lib'] + } + } + ])->[0]; + + syslog('LOG_WARNING', "OILS: Item(".$copy->barcode. + ") is captured for a hold, but there is no matching hold request") unless $hold; + + return $hold; + } + + return undef; +} + sub run_attr_script { my $self = shift; return 1 if $self->{ran_script}; @@ -373,47 +427,29 @@ my %shelf_expire_setting_cache; sub hold_pickup_date { my $self = shift; my $copy = $self->{copy}; + my $hold = $self->{hold} or return 0; - if( ($copy->status->id == OILS_COPY_STATUS_ON_HOLDS_SHELF) || - ($self->{transit} and $self->{transit}->copy_status == OILS_COPY_STATUS_ON_HOLDS_SHELF) ) { + my $date = $hold->shelf_expire_time; - # item has been captured for a hold + if(!$date) { + # hold has not hit the shelf. create a best guess. - my $e = OpenILS::SIP->editor(); - my $hold = $e->search_action_hold_request([ - { - current_copy => $copy->id, - capture_time => {'!=' => undef}, - cancel_time => undef, - fulfillment_time => undef - }, - {limit => 1} - ])->[0]; - - if($hold) { - my $date = $hold->shelf_expire_time; - - if(!$date) { - # hold has not hit the shelf. create a best guess. - - my $interval = $shelf_expire_setting_cache{$hold->pickup_lib} || - $U->ou_ancestor_setting_value( - $hold->pickup_lib, - 'circ.holds.default_shelf_expire_interval'); + my $interval = $shelf_expire_setting_cache{$hold->pickup_lib->id} || + $U->ou_ancestor_setting_value( + $hold->pickup_lib->id, + 'circ.holds.default_shelf_expire_interval'); - $shelf_expire_setting_cache{$hold->pickup_lib} = $interval; + $shelf_expire_setting_cache{$hold->pickup_lib->id} = $interval; - if($interval) { - my $seconds = OpenSRF::Utils->interval_to_seconds($interval); - $date = DateTime->now->add(seconds => $seconds); - $date = $date->strftime('%FT%T%z') if $date; - } - } - - return OpenILS::SIP->format_date($date) if $date; + if($interval) { + my $seconds = OpenSRF::Utils->interval_to_seconds($interval); + $date = DateTime->now->add(seconds => $seconds); + $date = $date->strftime('%FT%T%z') if $date; } } + return OpenILS::SIP->format_date($date) if $date; + return 0; } diff --git a/Open-ILS/src/perlmods/OpenILS/SIP/Patron.pm b/Open-ILS/src/perlmods/OpenILS/SIP/Patron.pm index fd690bfc09..c435bc0af6 100644 --- a/Open-ILS/src/perlmods/OpenILS/SIP/Patron.pm +++ b/Open-ILS/src/perlmods/OpenILS/SIP/Patron.pm @@ -41,7 +41,12 @@ sub new { if ($key ne 'usr' and $key ne 'barcode') { syslog("LOG_ERROR", "Patron (card) lookup requested by illegeal key '$key'"); - return; + return undef; + } + + unless(defined $patron_id) { + syslog("LOG_WARNING", "No patron ID provided to ILS::Patron->new"); + return undef; } my $type = ref($class) || $class; @@ -50,36 +55,34 @@ sub new { syslog("LOG_DEBUG", "OILS: new OpenILS Patron(%s => %s): searching...", $key, $patron_id); my $e = OpenILS::SIP->editor(); + + my $user_id = $patron_id; + if($key eq 'barcode') { + my $card = $e->search_actor_card({barcode => $patron_id})->[0]; + unless($card) { + syslog("LOG_WARNING", "No such patron barcode: $patron_id"); + return undef; + } + $user_id = $card->usr; + } - my $c = $e->search_actor_card({$key => $patron_id}, {idlist=>1}); - my $user; - - if( @$c ) { - - $user = $e->search_actor_user( - [ - { card => $$c[0] }, - { - flesh => 2, - flesh_fields => { - "au" => [ - #"cards", - "card", - "standing_penalties", - "addresses", - "billing_address", - "mailing_address", - #"stat_cat_entries", - 'profile', - ], - ausp => ['standing_penalty'] - } - } - ] - ); - - $user = (@$user) ? $$user[0] : undef; - } + my $user = $e->retrieve_actor_user([ + $user_id, + { + flesh => 2, + flesh_fields => { + au => [ + "card", + "standing_penalties", + "addresses", + "billing_address", + "mailing_address", + 'profile', + ], + ausp => ['standing_penalty'] + } + } + ]); if(!$user) { syslog("LOG_WARNING", "OILS: Unable to find patron %s => %s", $key, $patron_id); @@ -218,20 +221,19 @@ sub card_lost { return $self->{user}->card->active eq 'f'; } -sub recall_overdue { +sub recall_overdue { # not implemented my $self = shift; return 0; } - sub check_password { my ($self, $pwd) = @_; syslog('LOG_DEBUG', 'OILS: Patron->check_password()'); + return 0 unless (defined $pwd and $self->{user}); return md5_hex($pwd) eq $self->{user}->passwd; } - -sub currency { +sub currency { # not really implemented my $self = shift; syslog('LOG_DEBUG', 'OILS: Patron->currency()'); return 'USD'; @@ -280,12 +282,12 @@ sub screen_msg { return 'OK'; } -sub print_line { +sub print_line { # not implemented my $self = shift; return ''; } -sub too_many_charged { +sub too_many_charged { # not implemented my $self = shift; return 0; } diff --git a/Open-ILS/src/perlmods/OpenILS/SIP/Transaction/Checkin.pm b/Open-ILS/src/perlmods/OpenILS/SIP/Transaction/Checkin.pm index 66b09c62f8..c662104cec 100644 --- a/Open-ILS/src/perlmods/OpenILS/SIP/Transaction/Checkin.pm +++ b/Open-ILS/src/perlmods/OpenILS/SIP/Transaction/Checkin.pm @@ -44,6 +44,8 @@ sub new { @{$self}{keys %fields} = values %fields; # copying defaults into object + $self->load_override_events; + return bless $self, $class; } @@ -53,6 +55,16 @@ sub resensitize { return !$self->{item}->magnetic; } +my %override_events; +sub load_override_events { + return if %override_events; + my $override = OpenILS::SIP->config->{implementation_config}->{checkin_override}; + return unless $override; + my $events = $override->{event}; + $events = [$events] unless ref $events eq 'ARRAY'; + $override_events{$_} = 1 for @$events; +} + my %org_sn_cache; sub do_checkin { my $self = shift; @@ -71,6 +83,13 @@ sub do_checkin { my $args = {barcode => $self->{item}->id}; + if($return_date) { + # SIP date format is YYYYMMDD. Translate to ISO8601 + $return_date =~ s/(\d{4})(\d{2})(\d{2}).*/$1-$2-$3/; + syslog('LOG_INFO', "Checking in with backdate $return_date"); + $args->{backdate} = $return_date; + } + if($current_loc) { # SIP client specified a physical location my $org_id = (defined $org_sn_cache{$current_loc}) ? @@ -83,24 +102,36 @@ sub do_checkin { $args->{circ_lib} = $phys_location = $org_id if defined $org_id; } - my $resp = $U->simplereq( - 'open-ils.circ', - 'open-ils.circ.checkin', - $self->{authtoken}, $args - ); + my $override = 0; + my ($resp, $txt, $code); - if ($debug) { - my $s = Dumper($resp); - $s =~ s/\n//mog; - syslog('LOG_INFO', "OILS: Checkin response: $s"); - } + while(1) { + + my $method = 'open-ils.circ.checkin'; + $method .= '.override' if $override; + + $resp = $U->simplereq('open-ils.circ', $method, $self->{authtoken}, $args); - # In oddball cases, we can receive an array of events. - # The first event received should be treated as the main result. - $resp = $$resp[0] if ref($resp) eq 'ARRAY'; + if ($debug) { + my $s = Dumper($resp); + $s =~ s/\n//mog; + syslog('LOG_INFO', "OILS: Checkin response: $s"); + } - my $code = $U->event_code($resp); - my $txt = (defined $code) ? $resp->{textcode} : ''; + # In oddball cases, we can receive an array of events. + # The first event received should be treated as the main result. + $resp = $$resp[0] if ref($resp) eq 'ARRAY'; + $code = $U->event_code($resp); + $txt = (defined $code) ? $resp->{textcode} : ''; + + last if $override; + + if ( $override_events{$txt} ) { + $override = 1; + } else { + last; + } + } syslog('LOG_INFO', "OILS: Checkin resulted in event: $txt, phys_location: $phys_location"); @@ -119,13 +150,21 @@ sub do_checkin { my $payload = $resp->{payload} || {}; - # Two places to look for hold data. These are more important and more definitive than above. - if ($payload->{remote_hold}) { - # actually only used for checkin at non-owning branch w/ hold at same branch - $self->item->hold($payload->{remote_hold}); + my ($circ, $copy); + + if(ref $payload eq 'HASH') { - } elsif ($payload->{hold}) { - $self->item->hold($payload->{hold}); + # Two places to look for hold data. These are more important and more definitive than above. + if ($payload->{remote_hold}) { + # actually only used for checkin at non-owning branch w/ hold at same branch + $self->item->hold($payload->{remote_hold}); + + } elsif ($payload->{hold}) { + $self->item->hold($payload->{hold}); + } + + $circ = $resp->{payload}->{circ} || ''; + $copy = $resp->{payload}->{copy} || ''; } if ($self->item->hold) { @@ -142,10 +181,8 @@ sub do_checkin { $self->alert(1) if defined $self->alert_type; # alert_type could be "00", hypothetically - my $circ = $resp->{payload}->{circ} || ''; - my $copy = $resp->{payload}->{copy} || ''; - if ( $circ ) { + $self->{circ_user_id} = $circ->usr; $self->ok(1); } elsif ($txt eq 'NO_CHANGE' or $txt eq 'SUCCESS' or $txt eq 'ROUTE_ITEM') { $self->ok(1); # NO_CHANGE means it wasn't checked out anyway, no problem diff --git a/Open-ILS/src/perlmods/OpenILS/Utils/CStoreEditor.pm b/Open-ILS/src/perlmods/OpenILS/Utils/CStoreEditor.pm index 89ccfa6c11..984562362e 100644 --- a/Open-ILS/src/perlmods/OpenILS/Utils/CStoreEditor.pm +++ b/Open-ILS/src/perlmods/OpenILS/Utils/CStoreEditor.pm @@ -76,6 +76,11 @@ sub new { return $self; } +sub DESTROY { + my $self = shift; + $self->reset; + return undef; +} sub app { my( $self, $app ) = @_; @@ -289,8 +294,17 @@ sub rollback_savepoint { # ----------------------------------------------------------------------------- sub rollback { my $self = shift; - $self->xact_rollback; - $self->disconnect; + my $err; + my $ret; + try { + $self->xact_rollback; + } catch Error with { + $err = shift + } finally { + $ret = $self->disconnect + }; + throw $err if ($err); + return $ret; } sub disconnect { @@ -329,8 +343,17 @@ sub reset { # ----------------------------------------------------------------------------- sub finish { my $self = shift; - $self->commit; - $self->reset; + my $err; + my $ret; + try { + $self->commit; + } catch Error with { + $err = shift + } finally { + $ret = $self->reset + }; + throw $err if ($err); + return $ret; } diff --git a/Open-ILS/src/perlmods/OpenILS/Utils/Penalty.pm b/Open-ILS/src/perlmods/OpenILS/Utils/Penalty.pm index 4edc51cfbb..f259345da4 100644 --- a/Open-ILS/src/perlmods/OpenILS/Utils/Penalty.pm +++ b/Open-ILS/src/perlmods/OpenILS/Utils/Penalty.pm @@ -26,41 +26,53 @@ sub calculate_penalties { my $penalties = $e->json_query({from => ['actor.calculate_system_penalties',$user_id, $context_org]}); my $user = $e->retrieve_actor_user( $user_id ); - my $ses = OpenSRF::AppSession->create('open-ils.trigger') if (@$penalties); + my @existing_penalties = grep { defined $_->{id} } @$penalties; + my @wanted_penalties = grep { !defined $_->{id} } @$penalties; + my @trigger_events; my %csp; - for my $pen_obj (@$penalties) { - - next if grep { # leave duplicate penalties in place - $_->{org_unit} == $pen_obj->{org_unit} and - $_->{standing_penalty} == $pen_obj->{standing_penalty} and - ($_->{id} || '') ne ($pen_obj->{id} || '') } @$penalties; + for my $pen_obj (@wanted_penalties) { my $pen = Fieldmapper::actor::user_standing_penalty->new; $pen->$_($pen_obj->{$_}) for keys %$pen_obj; - if(defined $pen_obj->{id}) { - $e->delete_actor_user_standing_penalty($pen) or return $e->die_event; + # let's see if this penalty is accounted for already + my ($existing) = grep { + $_->{org_unit} == $pen_obj->{org_unit} and + $_->{standing_penalty} == $pen_obj->{standing_penalty} + } @existing_penalties; + + if($existing) { + # we have one of these already. Leave it be, but remove it from the + # existing set so it's not deleted in the subsequent loop + @existing_penalties = grep { $_->{id} ne $existing->{id} } @existing_penalties; } else { + + # this is a new penalty $e->create_actor_user_standing_penalty($pen) or return $e->die_event; - my $csp_obj = $csp{$pen->standing_penalty} || + my $csp_obj = $csp{$pen->standing_penalty} || $e->retrieve_config_standing_penalty( $pen->standing_penalty ); # cache for later $csp{$pen->standing_penalty} = $csp_obj; - $ses->request( - 'open-ils.trigger.event.autocreate', - 'penalty.' . $csp_obj->name, - $pen, - $pen->org_unit - ); + push(@trigger_events, ['penalty.' . $csp_obj->name, $pen, $pen->org_unit]); } } + # at this point, any penalties remaining in the existing + # penalty set are unaccounted for and should be removed + for my $pen_obj (@existing_penalties) { + my $pen = Fieldmapper::actor::user_standing_penalty->new; + $pen->$_($pen_obj->{$_}) for keys %$pen_obj; + $e->delete_actor_user_standing_penalty($pen) or return $e->die_event; + } + $e->commit if $commit; + + $U->create_events_for_hook($$_[0], $$_[1], $$_[2]) for @trigger_events; return undef; } diff --git a/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm b/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm index b42c6d56ad..47a561aea8 100644 --- a/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm +++ b/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm @@ -204,6 +204,7 @@ my $LEGACY_HOLD_EVENT_MAP = { sub indb_hold_permit { my $params = shift; + my $function = $$params{retarget} ? 'action.hold_retarget_permit_test' : 'action.hold_request_permit_test'; my $patron_id = ref($$params{patron}) ? $$params{patron}->id : $$params{patron_id}; my $request_lib = @@ -211,7 +212,7 @@ sub indb_hold_permit { my $HOLD_TEST = { from => [ - 'action.hold_request_permit_test', + $function, $$params{pickup_lib}, $request_lib, $$params{copy}->id, diff --git a/Open-ILS/src/perlmods/OpenILS/Utils/RemoteAccount.pm b/Open-ILS/src/perlmods/OpenILS/Utils/RemoteAccount.pm index ca1c9001a5..7eb9749808 100644 --- a/Open-ILS/src/perlmods/OpenILS/Utils/RemoteAccount.pm +++ b/Open-ILS/src/perlmods/OpenILS/Utils/RemoteAccount.pm @@ -1,4 +1,4 @@ -package OpenILS::Utils::RemoteAccount; +package OpenILS::Utils::RemoteAccount; # use OpenSRF::Utils::SettingsClient; use OpenSRF::Utils::Logger qw/:logger/; @@ -45,12 +45,18 @@ my %fields = ( ); -=pod +=head1 NAME + +OpenILS::Utils::RemoteAccount - Encapsulate FTP, SFTP and SSH file transactions for Evergreen + +=head1 DESCRIPTION The Remote Account module attempts to transfer a file to/from a remote server. Net::uFTP is used, encapsulating the available options of SCP, FTP and SFTP. -All information is expected to be gathered by the Event Definition through event parameters: +=head1 PARAMETERS + +All information is expected to be supplied by the caller via parameters: ~ remote_host (required) ~ remote_user ~ remote_password @@ -79,15 +85,16 @@ Note that specifying a password will require you to specify a user. Similarly, specifying an account requires both user and password. That is, there are no assumed defaults when the latter arguments are used. -SSH KEYS: +=head2 SSH KEYS: -The use of ssh keys is preferred. +The use of ssh keys is preferred. Explicit specification of connection type will prevent +multiple attempts to the same server. Therefore, using the type parameter is also recommended. -We attempt to use SSH keys where they are specified or otherwise found +If the type is not explicit, we attempt to use SSH keys where they are specified or otherwise found in the runtime environment. If only one key is specified, we attempt to derive the corresponding filename based on the ssh-keygen defaults. If either key is specified, but both are not found (and readable) then the result is failure. If -no key is specified, but keys are found, the key-based connections will be attempted, +no key or type is specified, but keys are found, the key-based connections will be attempted, but failure will be non-fatal. =cut diff --git a/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm b/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm index 1fd0994090..6c5f3da40d 100644 --- a/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm +++ b/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm @@ -80,7 +80,7 @@ sub handler { my $base = $cgi->url(-base=>1); $base =~ s/^http:/https:/o; print "Location: $base".$apache->unparsed_uri."\n\n"; - return Apache2::Const::OK; + return Apache2::Const::REDIRECT; } if (!$auth_ses) { diff --git a/Open-ILS/src/perlmods/OpenILS/WWW/TemplateBatchBibUpdate.pm b/Open-ILS/src/perlmods/OpenILS/WWW/TemplateBatchBibUpdate.pm new file mode 100644 index 0000000000..8f9f0ce046 --- /dev/null +++ b/Open-ILS/src/perlmods/OpenILS/WWW/TemplateBatchBibUpdate.pm @@ -0,0 +1,671 @@ +package OpenILS::WWW::TemplateBatchBibUpdate; +use strict; +use warnings; +use bytes; + +use Apache2::Log; +use Apache2::Const -compile => qw(OK REDIRECT DECLINED NOT_FOUND :log); +use APR::Const -compile => qw(:error SUCCESS); +use APR::Table; + +use Apache2::RequestRec (); +use Apache2::RequestIO (); +use Apache2::RequestUtil; +use CGI; +use Data::Dumper; +use Text::CSV; + +use OpenSRF::EX qw(:try); +use OpenSRF::Utils qw/:datetime/; +use OpenSRF::Utils::Cache; +use OpenSRF::System; +use OpenSRF::AppSession; +use XML::LibXML; +use XML::LibXSLT; + +use Encode; +use Unicode::Normalize; +use OpenILS::Utils::Fieldmapper; +use OpenSRF::Utils::Logger qw/$logger/; + +use MARC::Record; +use MARC::File::XML; + +use UNIVERSAL::require; + +our @formats = qw/USMARC UNIMARC XML BRE/; + +# set the bootstrap config and template include directory when +# this module is loaded +my $bootstrap; + +sub import { + my $self = shift; + $bootstrap = shift; +} + + +sub child_init { + OpenSRF::System->bootstrap_client( config_file => $bootstrap ); + Fieldmapper->import(IDL => OpenSRF::Utils::SettingsClient->new->config_value("IDL")); +} + +sub handler { + my $r = shift; + my $cgi = new CGI; + + my $authid = $cgi->cookie('ses') || $cgi->param('ses'); + my $usr = verify_login($authid); + return show_template($r) unless ($usr); + + my $template = $cgi->param('template'); + return show_template($r) unless ($template); + + + my $rsource = $cgi->param('recordSource'); + # find some IDs ... + my @records; + + if ($rsource eq 'r') { + @records = map { $_ ? ($_) : () } $cgi->param('recid'); + } + + if ($rsource eq 'c') { # try for a file + my $file = $cgi->param('idfile'); + if ($file) { + my $col = $cgi->param('idcolumn') || 0; + my $csv = new Text::CSV; + + while (<$file>) { + $csv->parse($_); + my @data = $csv->fields; + my $id = $data[$col]; + $id =~ s/\D+//o; + next unless ($id); + push @records, $id; + } + } + } + + my $e = OpenSRF::AppSession->connect('open-ils.cstore'); + $e->request('open-ils.cstore.transaction.begin')->gather(1); + + # still no records ... + my $container = $cgi->param('containerid'); + if ($rsource eq 'b') { + if ($container) { + my $bucket = $e->request( + 'open-ils.cstore.direct.container.biblio_record_entry_bucket.retrieve', + $container + )->gather(1); + unless($bucket) { + $e->request('open-ils.cstore.transaction.rollback')->gather(1); + $e->disconnect; + $r->log->error("No such bucket $container"); + $logger->error("No such bucket $container"); + return Apache2::Const::NOT_FOUND; + } + my $recs = $e->request( + 'open-ils.cstore.direct.container.biblio_record_entry_bucket_item.search.atomic', + { bucket => $container } + )->gather(1); + @records = map { ($_->target_biblio_record_entry) } @$recs; + } + } + + unless (@records) { + $e->request('open-ils.cstore.transaction.rollback')->gather(1); + $e->disconnect; + return show_template($r); + } + + # we have a template and some record ids, so... + + # insert the template record + my $min_id = $e->request( + 'open-ils.cstore.json_query', + { select => { bre => [{ column => 'id', transform => 'min', aggregate => 1}] }, from => 'bre' } + )->gather(1)->{id} - 1; + + warn "new template bib id = $min_id\n"; + + my $tmpl_rec = Fieldmapper::biblio::record_entry->new; + $tmpl_rec->id($min_id); + $tmpl_rec->deleted('t'); + $tmpl_rec->active('f'); + $tmpl_rec->marc($template); + $tmpl_rec->creator($usr->id); + $tmpl_rec->editor($usr->id); + + warn "about to create bib $min_id\n"; + $e->request('open-ils.cstore.direct.biblio.record_entry.create', $tmpl_rec )->gather(1); + + # create the new container for the records and the template + my $bucket = Fieldmapper::container::biblio_record_entry_bucket->new; + $bucket->owner($usr->id); + $bucket->btype('template_merge'); + + my $bname = $cgi->param('bname') || 'Temporary Merge Bucket '. localtime() . ' ' . $usr->id; + $bucket->name($bname); + + $bucket = $e->request('open-ils.cstore.direct.container.biblio_record_entry_bucket.create', $bucket )->gather(1); + + # create items in the bucket + my $item = Fieldmapper::container::biblio_record_entry_bucket_item->new; + $item->bucket($bucket->id); + $item->target_biblio_record_entry($min_id); + + $e->request('open-ils.cstore.direct.container.biblio_record_entry_bucket_item.create', $item )->gather(1); + + my %seen; + for my $r (@records) { + next if ($seen{$r}); + $item->target_biblio_record_entry($r); + $e->request('open-ils.cstore.direct.container.biblio_record_entry_bucket_item.create', $item )->gather(1); + $seen{$r}++; + } + + $e->request('open-ils.cstore.transaction.commit')->gather(1); + $e->disconnect; + + # fire the background bucket processor + my $cache_key = OpenSRF::AppSession + ->create('open-ils.cat') + ->request('open-ils.cat.container.template_overlay.background', $authid, $bucket->id) + ->gather(1); + + return show_processing_template($r, $bucket->id, \@records, $cache_key); +} + +sub verify_login { + my $auth_token = shift; + return undef unless $auth_token; + + my $user = OpenSRF::AppSession + ->create("open-ils.auth") + ->request( "open-ils.auth.session.retrieve", $auth_token ) + ->gather(1); + + if (ref($user) eq 'HASH' && $user->{ilsevent} == 1001) { + return undef; + } + + return $user if ref($user); + return undef; +} + +sub show_processing_template { + my $r = shift; + my $bid = shift; + my $recs = shift; + my $cache_key = shift; + + my $rec_string = @$recs; + + $r->content_type('text/html'); + $r->print(< + + + Merging records... + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + +
+ StatusRecord Count
Success
Failure
+ +
+ + + +HTML + + return Apache2::Const::OK; +} + + +sub show_template { + my $r = shift; + + $r->content_type('text/html'); + $r->print(<<'HTML'); + + + + Merge Template Builder + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + + +
Bucket named: +
+ +
+
+ Column of: + +
+
+ Columns are numbered starting at 0. For instance, when looking at a CSV file in Excel, the column labeled A is the same as column 0, and the column labeled B is the same as column 1. +
Record ID:
+ + (After setting up your template below.) + +
+
+ +
+ +
+ + + + + +
Update Template Preview:

+
+        
+
+        

+        

+            
+                
+                    
+                        
+                        
+                        
+                    
+                    
+                        
+                        
+                        
+                    
+                    
+                        
+                        
+                    
+                    
+                        
+                        
+                        
+                    
+                    
+                        
+                        
+                        
+                    
+                    
+                        
+                    
+                    
+                        
+                        
+                    
+                        
+                        
+                        
+                    
+                    
+                        
+                    
+                
+            
Rule SetupDataHelp
Action (Rule Type)
+                            
+                        How to change the existing records
MARC Tag
+                        Three characters, no spaces, no indicators, etc. eg: 245
Subfields (optional)No spaces, no delimiters, eg: abcnp
MARC DataMARC-Breaker formatted data with indicators and subfield delimiters, eg:
245 04$aThe End
Advanced Matching Restriction (Optional)
Subfield
+                        A single subfield code, no delimiters, eg: a
Regular ExpressionSee the Perl documentation
+                            for an explanation of Regular Expressions.
+                        

+                            
+                        

+        

+        

+        

+
+    
+
+HTML
+
+    return Apache2::Const::OK;
+}
+
+1;
+
+
diff --git a/Open-ILS/src/reporter/clark-kent.pl b/Open-ILS/src/reporter/clark-kent.pl
index 52dc6fd90c..196297e345 100755
--- a/Open-ILS/src/reporter/clark-kent.pl
+++ b/Open-ILS/src/reporter/clark-kent.pl
@@ -423,6 +423,8 @@ sub build_excel {
 	$sheetname =~ s/\W/_/gos;
 	
 	my $sheet = $xls->add_worksheet($sheetname);
+	# don't try to write formulas, just write anything that starts with = as a text cell
+	$sheet->add_write_handler(qr/^=/, sub { return shift->write_string(@_); } );
 
 	$sheet->write_row('A1', $r->{column_labels});
 
diff --git a/Open-ILS/src/sql/Pg/002.functions.config.sql b/Open-ILS/src/sql/Pg/002.functions.config.sql
index 6b43ceb4b3..cc97d92f42 100644
--- a/Open-ILS/src/sql/Pg/002.functions.config.sql
+++ b/Open-ILS/src/sql/Pg/002.functions.config.sql
@@ -272,33 +272,37 @@ BEGIN
     select_list := ARRAY_APPEND( select_list, key || '::INT AS key' );
 
     FOR i IN 1 .. ARRAY_UPPER(xpath_list,1) LOOP
-        select_list := ARRAY_APPEND(
-            select_list,
-            $sel$
-            EXPLODE_ARRAY(
-                COALESCE(
-                    NULLIF(
-                        oils_xpath(
-                            $sel$ ||
-                                quote_literal(
-                                    CASE
-                                        WHEN xpath_list[i] ~ $re$/[^/[]*@[^/]+$$re$ OR xpath_list[i] ~ $re$text\(\)$$re$ THEN xpath_list[i]
-                                        ELSE xpath_list[i] || '//text()'
-                                    END
-                                ) ||
-                            $sel$,
-                            $sel$ || document_field || $sel$
+        IF xpath_list[i] = 'null()' THEN
+            select_list := ARRAY_APPEND( select_list, 'NULL::TEXT AS c_' || i );
+        ELSE
+            select_list := ARRAY_APPEND(
+                select_list,
+                $sel$
+                EXPLODE_ARRAY(
+                    COALESCE(
+                        NULLIF(
+                            oils_xpath(
+                                $sel$ ||
+                                    quote_literal(
+                                        CASE
+                                            WHEN xpath_list[i] ~ $re$/[^/[]*@[^/]+$$re$ OR xpath_list[i] ~ $re$text\(\)$$re$ THEN xpath_list[i]
+                                            ELSE xpath_list[i] || '//text()'
+                                        END
+                                    ) ||
+                                $sel$,
+                                $sel$ || document_field || $sel$
+                            ),
+                           '{}'::TEXT[]
                         ),
-                       '{}'::TEXT[]
-                    ),
-                    '{NULL}'::TEXT[]
-                )
-            ) AS c_$sel$ || i
-        );
-        where_list := ARRAY_APPEND(
-            where_list,
-            'c_' || i || ' IS NOT NULL'
-        );
+                        '{NULL}'::TEXT[]
+                    )
+                ) AS c_$sel$ || i
+            );
+            where_list := ARRAY_APPEND(
+                where_list,
+                'c_' || i || ' IS NOT NULL'
+            );
+        END IF;
     END LOOP;
 
     q := $q$
@@ -593,5 +597,9 @@ if ($create or $munge) {
 return;
 $func$ LANGUAGE PLPERLU;
 
+CREATE OR REPLACE FUNCTION oils_text_as_bytea (TEXT) RETURNS BYTEA AS $_$
+    SELECT CAST(REGEXP_REPLACE($1, $$\\$$, $$\\\\$$, 'g') AS BYTEA);
+$_$ LANGUAGE SQL IMMUTABLE;
+
 COMMIT;
 
diff --git a/Open-ILS/src/sql/Pg/002.schema.config.sql b/Open-ILS/src/sql/Pg/002.schema.config.sql
index fca4b6ba40..85f7a4edc6 100644
--- a/Open-ILS/src/sql/Pg/002.schema.config.sql
+++ b/Open-ILS/src/sql/Pg/002.schema.config.sql
@@ -70,7 +70,7 @@ CREATE TABLE config.upgrade_log (
     install_date    TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
 );
 
-INSERT INTO config.upgrade_log (version) VALUES ('0412'); -- phasefx
+INSERT INTO config.upgrade_log (version) VALUES ('0439'); -- miker
 
 CREATE TABLE config.bib_source (
 	id		SERIAL	PRIMARY KEY,
@@ -316,7 +316,8 @@ CREATE TABLE config.rule_circ_duration (
 	extended	INTERVAL	NOT NULL,
 	normal		INTERVAL	NOT NULL,
 	shrt		INTERVAL	NOT NULL,
-	max_renewals	INT		NOT NULL
+	max_renewals	INT		NOT NULL,
+	date_ceiling    TIMESTAMPTZ
 );
 COMMENT ON TABLE config.rule_circ_duration IS $$
 /*
@@ -342,6 +343,14 @@ COMMENT ON TABLE config.rule_circ_duration IS $$
  */
 $$;
 
+CREATE TABLE config.hard_due_date (
+    id              SERIAL      PRIMARY KEY,
+    duration_rule   INT         NOT NULL REFERENCES config.rule_circ_duration (id)
+                                DEFERRABLE INITIALLY DEFERRED,
+    ceiling_date    TIMESTAMPTZ NOT NULL,
+    active_date     TIMESTAMPTZ NOT NULL
+);
+
 CREATE TABLE config.rule_max_fine (
     id          SERIAL          PRIMARY KEY,
     name        TEXT            NOT NULL UNIQUE CHECK ( name ~ E'^\\w+$' ),
diff --git a/Open-ILS/src/sql/Pg/006.schema.permissions.sql b/Open-ILS/src/sql/Pg/006.schema.permissions.sql
index c734d94c64..4be5c7d5e6 100644
--- a/Open-ILS/src/sql/Pg/006.schema.permissions.sql
+++ b/Open-ILS/src/sql/Pg/006.schema.permissions.sql
@@ -38,7 +38,8 @@ CREATE TABLE permission.grp_tree (
 	usergroup		BOOL	NOT NULL DEFAULT TRUE,
 	perm_interval		INTERVAL DEFAULT '3 years'::interval NOT NULL,
 	description		TEXT,
-	application_perm	TEXT
+	application_perm	TEXT,
+	hold_priority       INT   NOT NULL DEFAULT 0
 );
 CREATE INDEX grp_tree_parent_idx ON permission.grp_tree (parent);
 
diff --git a/Open-ILS/src/sql/Pg/012.schema.vandelay.sql b/Open-ILS/src/sql/Pg/012.schema.vandelay.sql
index 41372e2374..95ff5eb3be 100644
--- a/Open-ILS/src/sql/Pg/012.schema.vandelay.sql
+++ b/Open-ILS/src/sql/Pg/012.schema.vandelay.sql
@@ -77,7 +77,7 @@ ALTER TABLE vandelay.bib_queue ADD PRIMARY KEY (id);
 CREATE TABLE vandelay.queued_bib_record (
 	queue		INT		NOT NULL REFERENCES vandelay.bib_queue (id) ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED,
 	bib_source	INT		REFERENCES config.bib_source (id) DEFERRABLE INITIALLY DEFERRED,
-	imported_as	INT		REFERENCES biblio.record_entry (id) DEFERRABLE INITIALLY DEFERRED
+	imported_as	BIGINT	REFERENCES biblio.record_entry (id) DEFERRABLE INITIALLY DEFERRED
 ) INHERITS (vandelay.queued_record);
 ALTER TABLE vandelay.queued_bib_record ADD PRIMARY KEY (id);
 CREATE INDEX queued_bib_record_queue_idx ON vandelay.queued_bib_record (queue);
@@ -347,12 +347,18 @@ CREATE OR REPLACE FUNCTION vandelay.add_field ( target_xml TEXT, source_xml TEXT
     for my $f ( keys %fields) {
         if ( @{$fields{$f}{sf}} ) {
             for my $from_field ($source_r->field( $f )) {
-                for my $to_field ($target_r->field( $f )) {
-                    if (exists($fields{$f}{match})) {
-                        next unless (grep { $_ =~ $fields{$f}{match}{re} } $to_field->subfield($fields{$f}{match}{sf}));
+                my @tos = $target_r->field( $f );
+                if (!@tos) {
+                    my @new_fields = map { $_->clone } $source_r->field( $f );
+                    $target_r->insert_fields_ordered( @new_fields );
+                } else {
+                    for my $to_field (@tos) {
+                        if (exists($fields{$f}{match})) {
+                            next unless (grep { $_ =~ $fields{$f}{match}{re} } $to_field->subfield($fields{$f}{match}{sf}));
+                        }
+                        my @new_sf = map { ($_ => $from_field->subfield($_)) } @{$fields{$f}{sf}};
+                        $to_field->add_subfields( @new_sf );
                     }
-                    my @new_sf = map { ($_ => $from_field->subfield($_)) } @{$fields{$f}{sf}};
-                    $to_field->add_subfields( @new_sf );
                 }
             }
         } else {
@@ -469,10 +475,10 @@ BEGIN
         END IF;
     END IF;
 
-    add_rule := add_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="a"]/text()',incoming_xml),''),'');
-    strip_rule := strip_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="d"]/text()',incoming_xml),''),'');
-    replace_rule := replace_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="r"]/text()',incoming_xml),''),'');
-    preserve_rule := preserve_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="p"]/text()',incoming_xml),''),'');
+    add_rule := add_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="a"]/text()',incoming_xml),','),'');
+    strip_rule := strip_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="d"]/text()',incoming_xml),','),'');
+    replace_rule := replace_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="r"]/text()',incoming_xml),','),'');
+    preserve_rule := preserve_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="p"]/text()',incoming_xml),','),'');
 
     output.add_rule := BTRIM(add_rule,',');
     output.replace_rule := BTRIM(replace_rule,',');
diff --git a/Open-ILS/src/sql/Pg/030.schema.metabib.sql b/Open-ILS/src/sql/Pg/030.schema.metabib.sql
index 8efc3da1e5..41b0874c92 100644
--- a/Open-ILS/src/sql/Pg/030.schema.metabib.sql
+++ b/Open-ILS/src/sql/Pg/030.schema.metabib.sql
@@ -729,8 +729,8 @@ BEGIN
                             JOIN config.marc21_physical_characteristic_subfield_map s ON (s.id = p.subfield)
                             JOIN config.marc21_physical_characteristic_value_map v ON (v.id = p.value)
                       WHERE p.ptype = 'v' AND s.subfield = 'e'    ),
-                biblio.marc21_extract_fixed_field( bib_id, 'Date1'),
-                biblio.marc21_extract_fixed_field( bib_id, 'Date2');
+                LPAD(NULLIF(REGEXP_REPLACE(NULLIF(biblio.marc21_extract_fixed_field( bib_id, 'Date1'), ''), E'\\D', '0', 'g')::INT,0)::TEXT,4,'0'),
+                LPAD(NULLIF(REGEXP_REPLACE(NULLIF(biblio.marc21_extract_fixed_field( bib_id, 'Date2'), ''), E'\\D', '9', 'g')::INT,9999)::TEXT,4,'0');
 
     RETURN;
 END;
diff --git a/Open-ILS/src/sql/Pg/040.schema.asset.sql b/Open-ILS/src/sql/Pg/040.schema.asset.sql
index 1c52cb04bc..6b604df2a9 100644
--- a/Open-ILS/src/sql/Pg/040.schema.asset.sql
+++ b/Open-ILS/src/sql/Pg/040.schema.asset.sql
@@ -293,8 +293,8 @@ CREATE INDEX asset_call_number_record_idx ON asset.call_number (record);
 CREATE INDEX asset_call_number_creator_idx ON asset.call_number (creator);
 CREATE INDEX asset_call_number_editor_idx ON asset.call_number (editor);
 CREATE INDEX asset_call_number_dewey_idx ON asset.call_number (public.call_number_dewey(label));
-CREATE INDEX asset_call_number_upper_label_id_owning_lib_idx ON asset.call_number (upper(label),id,owning_lib);
-CREATE INDEX asset_call_number_label_sortkey ON asset.call_number(label_sortkey);
+CREATE INDEX asset_call_number_upper_label_id_owning_lib_idx ON asset.call_number (oils_text_as_bytea(upper(label)),id,owning_lib);
+CREATE INDEX asset_call_number_label_sortkey ON asset.call_number(oils_text_as_bytea(label_sortkey));
 CREATE UNIQUE INDEX asset_call_number_label_once_per_lib ON asset.call_number (record, owning_lib, label) WHERE deleted = FALSE OR deleted IS FALSE;
 CREATE RULE protect_cn_delete AS ON DELETE TO asset.call_number DO INSTEAD UPDATE asset.call_number SET deleted = TRUE WHERE OLD.id = asset.call_number.id;
 CREATE TRIGGER asset_label_sortkey_trigger
diff --git a/Open-ILS/src/sql/Pg/070.schema.container.sql b/Open-ILS/src/sql/Pg/070.schema.container.sql
index beceec94a0..6bbd17c76c 100644
--- a/Open-ILS/src/sql/Pg/070.schema.container.sql
+++ b/Open-ILS/src/sql/Pg/070.schema.container.sql
@@ -161,7 +161,7 @@ CREATE TABLE container.biblio_record_entry_bucket_item (
 							ON UPDATE CASCADE
 							DEFERRABLE
 							INITIALLY DEFERRED,
-	target_biblio_record_entry	INT	NOT NULL
+	target_biblio_record_entry	BIGINT	NOT NULL
 						REFERENCES biblio.record_entry (id)
 							ON DELETE CASCADE
 							ON UPDATE CASCADE
diff --git a/Open-ILS/src/sql/Pg/090.schema.action.sql b/Open-ILS/src/sql/Pg/090.schema.action.sql
index be8c36860c..fc86d8aeb8 100644
--- a/Open-ILS/src/sql/Pg/090.schema.action.sql
+++ b/Open-ILS/src/sql/Pg/090.schema.action.sql
@@ -146,6 +146,7 @@ CREATE INDEX circ_open_date_idx ON "action".circulation (xact_start) WHERE xact_
 CREATE INDEX circ_all_usr_idx       ON action.circulation ( usr );
 CREATE INDEX circ_circ_staff_idx    ON action.circulation ( circ_staff );
 CREATE INDEX circ_checkin_staff_idx ON action.circulation ( checkin_staff );
+CREATE INDEX action_circulation_target_copy_idx ON action.circulation (target_copy);
 CREATE UNIQUE INDEX circ_parent_idx ON action.circulation ( parent_circ ) WHERE parent_circ IS NOT NULL;
 CREATE UNIQUE INDEX only_one_concurrent_checkout_per_copy ON action.circulation(target_copy) WHERE checkin_time IS NULL;
 
@@ -163,7 +164,7 @@ BEGIN
 END;
 $$ LANGUAGE PLPGSQL;
 
-CREATE TRIGGER push_due_date_tgr BEFORE INSERT ON action.circulation FOR EACH ROW EXECUTE PROCEDURE action.push_circ_due_time();
+CREATE TRIGGER push_due_date_tgr BEFORE INSERT OR UPDATE ON action.circulation FOR EACH ROW EXECUTE PROCEDURE action.push_circ_due_time();
 
 CREATE TABLE action.aged_circulation (
 	usr_post_code		TEXT,
@@ -185,6 +186,7 @@ CREATE INDEX aged_circ_start_idx ON "action".aged_circulation (xact_start);
 CREATE INDEX aged_circ_copy_circ_lib_idx ON "action".aged_circulation (copy_circ_lib);
 CREATE INDEX aged_circ_copy_owning_lib_idx ON "action".aged_circulation (copy_owning_lib);
 CREATE INDEX aged_circ_copy_location_idx ON "action".aged_circulation (copy_location);
+CREATE INDEX action_aged_circulation_target_copy_idx ON action.aged_circulation (target_copy);
 
 CREATE OR REPLACE VIEW action.all_circulation AS
     SELECT  id,usr_post_code, usr_home_ou, usr_profile, usr_birth_year, copy_call_number, copy_location,
@@ -409,7 +411,7 @@ CREATE INDEX ahn_hold_idx ON action.hold_notification (hold);
 CREATE INDEX ahn_notify_staff_idx ON action.hold_notification ( notify_staff );
 
 CREATE TABLE action.hold_copy_map (
-	id		SERIAL	PRIMARY KEY,
+	id		BIGSERIAL	PRIMARY KEY,
 	hold		INT	NOT NULL REFERENCES action.hold_request (id) ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED,
 	target_copy	BIGINT	NOT NULL, -- REFERENCES asset.copy (id) ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED, -- XXX could be an serial.issuance
 	CONSTRAINT copy_once_per_hold UNIQUE (hold,target_copy)
@@ -776,7 +778,7 @@ BEGIN
             SELECT * INTO circ_chain_tail FROM action.circ_chain(circ_chain_head.id) ORDER BY xact_start DESC LIMIT 1;
             EXIT WHEN circ_chain_tail.xact_finish IS NULL;
 
-            -- Now get the user setings, if any, to block purging if the user wants to keep more circs
+            -- Now get the user settings, if any, to block purging if the user wants to keep more circs
             usr_keep_age.value := NULL;
             SELECT * INTO usr_keep_age FROM actor.usr_setting WHERE usr = circ_chain_head.usr AND name = 'history.circ.retention_age';
 
@@ -792,7 +794,7 @@ BEGIN
             ELSIF usr_keep_start.value IS NOT NULL THEN
                 keep_age := AGE(NOW(), oils_json_to_text(usr_keep_start.value)::TIMESTAMPTZ);
             ELSE
-                keep_age := COALESCE( org_keep_age::INTERVAL, '2000 years'::INTEVAL );
+                keep_age := COALESCE( org_keep_age::INTERVAL, '2000 years'::INTERVAL );
             END IF;
 
             EXIT WHEN AGE(NOW(), circ_chain_tail.xact_finish) < keep_age;
diff --git a/Open-ILS/src/sql/Pg/095.schema.booking.sql b/Open-ILS/src/sql/Pg/095.schema.booking.sql
index f79d968cf9..6d755824d9 100644
--- a/Open-ILS/src/sql/Pg/095.schema.booking.sql
+++ b/Open-ILS/src/sql/Pg/095.schema.booking.sql
@@ -32,9 +32,9 @@ CREATE TABLE booking.resource_type (
 	                               DEFERRABLE INITIALLY DEFERRED,
 	catalog_item   BOOLEAN         NOT NULL DEFAULT FALSE,
 	transferable   BOOLEAN         NOT NULL DEFAULT FALSE,
-    record         INT             REFERENCES biblio.record_entry (id)
+    record         BIGINT          REFERENCES biblio.record_entry (id)
                                    DEFERRABLE INITIALLY DEFERRED,
-	CONSTRAINT brt_name_once_per_owner UNIQUE(owner, name, record)
+	CONSTRAINT brt_name_and_record_once_per_owner UNIQUE(owner, name, record)
 );
 
 CREATE TABLE booking.resource (
diff --git a/Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql b/Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql
index 0a300d3443..0edf98b0c9 100644
--- a/Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql
+++ b/Open-ILS/src/sql/Pg/1.6.1-2.0-upgrade-db.sql
@@ -1,24 +1,44 @@
+-- Before starting the transaction: drop some constraints that
+-- may or may not exist.
+
+DROP INDEX asset.asset_call_number_upper_label_id_owning_lib_idx;
+CREATE INDEX asset_call_number_upper_label_id_owning_lib_idx ON asset.call_number (oils_text_as_bytea(upper(label)),id,owning_lib);
+
+
+\qecho Before starting the transaction: drop some constraints.
+\qecho If a DROP fails because the constraint doesn't exist, ignore the failure.
+
+ALTER TABLE permission.grp_perm_map        DROP CONSTRAINT grp_perm_map_perm_fkey;
+ALTER TABLE permission.usr_perm_map        DROP CONSTRAINT usr_perm_map_perm_fkey;
+ALTER TABLE permission.usr_object_perm_map DROP CONSTRAINT usr_object_perm_map_perm_fkey;
+ALTER TABLE booking.resource_type          DROP CONSTRAINT brt_name_or_record_once_per_owner;
+ALTER TABLE booking.resource_type          DROP CONSTRAINT brt_name_once_per_owner;
+
+\qecho Beginning the transaction now
+
 BEGIN;
 
--- Highest-numbered individual upgrade script
--- incorporated herein:
+-- Highest-numbered individual upgrade script incorporated herein:
 
-INSERT INTO config.upgrade_log (version) VALUES ('0403');
+INSERT INTO config.upgrade_log (version) VALUES ('0433');
 
--- Begin by upgrading permission.perm_list.  This is fairly complicated.
+-- Recreate one of the constraints that we just dropped,
+-- under a different name:
+
+ALTER TABLE booking.resource_type
+	ADD CONSTRAINT brt_name_and_record_once_per_owner UNIQUE(owner, name, record);
+
+-- Now upgrade permission.perm_list.  This is fairly complicated.
 
 -- Add ON UPDATE CASCADE to some foreign keys so that, when we renumber the
 -- permissions, the dependents will follow and stay in sync:
 
-ALTER TABLE permission.grp_perm_map DROP CONSTRAINT grp_perm_map_perm_fkey;
 ALTER TABLE permission.grp_perm_map ADD CONSTRAINT grp_perm_map_perm_fkey FOREIGN KEY (perm)
     REFERENCES permission.perm_list (id) ON UPDATE CASCADE ON DELETE RESTRICT DEFERRABLE INITIALLY DEFERRED;
 
-ALTER TABLE permission.usr_perm_map DROP CONSTRAINT usr_perm_map_perm_fkey;
 ALTER TABLE permission.usr_perm_map ADD CONSTRAINT usr_perm_map_perm_fkey FOREIGN KEY (perm)
     REFERENCES permission.perm_list (id) ON UPDATE CASCADE ON DELETE RESTRICT DEFERRABLE INITIALLY DEFERRED;
 
-ALTER TABLE permission.usr_object_perm_map DROP CONSTRAINT usr_object_perm_map_perm_fkey;
 ALTER TABLE permission.usr_object_perm_map ADD CONSTRAINT usr_object_perm_map_perm_fkey FOREIGN KEY (perm)
     REFERENCES permission.perm_list (id) ON UPDATE CASCADE ON DELETE RESTRICT DEFERRABLE INITIALLY DEFERRED;
 
@@ -885,99 +905,108 @@ INSERT INTO permission.temp_perm ( id, code, description ) VALUES ( 389, 'VIEW_C
      'View org unit settings related to credit card processing' );
 INSERT INTO permission.temp_perm ( id, code, description ) VALUES ( 390, 'ADMIN_CREDIT_CARD_PROCESSING',
      'Update org unit settings related to credit card processing' );
+INSERT INTO permission.temp_perm ( id, code, description ) VALUES ( 391, 'ADMIN_SERIAL_CAPTION_PATTERN',
+	'Create/update/delete serial caption and pattern objects' );
+INSERT INTO permission.temp_perm ( id, code, description ) VALUES ( 392, 'ADMIN_SERIAL_SUBSCRIPTION',
+	'Create/update/delete serial subscription objects' );
+INSERT INTO permission.temp_perm ( id, code, description ) VALUES ( 393, 'ADMIN_SERIAL_DISTRIBUTION',
+	'Create/update/delete serial distribution objects' );
+INSERT INTO permission.temp_perm ( id, code, description ) VALUES ( 394, 'ADMIN_SERIAL_STREAM',
+	'Create/update/delete serial stream objects' );
+INSERT INTO permission.temp_perm ( id, code, description ) VALUES ( 395, 'RECEIVE_SERIAL',
+	'Receive serial items' );
 
 -- Now for the permissions from the IDL.  We don't have descriptions for them.
 
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 391, 'ADMIN_ACQ_CLAIM' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 392, 'ADMIN_ACQ_CLAIM_EVENT_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 393, 'ADMIN_ACQ_CLAIM_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 394, 'ADMIN_ACQ_DISTRIB_FORMULA' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 395, 'ADMIN_ACQ_FISCAL_YEAR' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 396, 'ADMIN_ACQ_FUND_ALLOCATION_PERCENT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 397, 'ADMIN_ACQ_FUND_TAG' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 398, 'ADMIN_ACQ_LINEITEM_ALERT_TEXT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 399, 'ADMIN_AGE_PROTECT_RULE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 400, 'ADMIN_ASSET_COPY_TEMPLATE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 401, 'ADMIN_BOOKING_RESERVATION_ATTR_MAP' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 402, 'ADMIN_CIRC_MATRIX_MATCHPOINT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 403, 'ADMIN_CIRC_MOD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 404, 'ADMIN_CLAIM_POLICY' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 405, 'ADMIN_CONFIG_REMOTE_ACCOUNT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 406, 'ADMIN_FIELD_DOC' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 407, 'ADMIN_GLOBAL_FLAG' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 408, 'ADMIN_GROUP_PENALTY_THRESHOLD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 409, 'ADMIN_HOLD_CANCEL_CAUSE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 410, 'ADMIN_HOLD_MATRIX_MATCHPOINT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 411, 'ADMIN_IDENT_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 412, 'ADMIN_IMPORT_ITEM_ATTR_DEF' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 413, 'ADMIN_INDEX_NORMALIZER' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 414, 'ADMIN_INVOICE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 415, 'ADMIN_INVOICE_METHOD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 416, 'ADMIN_INVOICE_PAYMENT_METHOD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 417, 'ADMIN_LINEITEM_MARC_ATTR_DEF' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 418, 'ADMIN_MARC_CODE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 419, 'ADMIN_MAX_FINE_RULE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 420, 'ADMIN_MERGE_PROFILE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 421, 'ADMIN_ORG_UNIT_SETTING_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 422, 'ADMIN_RECURRING_FINE_RULE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 423, 'ADMIN_SERIAL_SUBSCRIPTION' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 424, 'ADMIN_STANDING_PENALTY' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 425, 'ADMIN_SURVEY' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 426, 'ADMIN_USER_REQUEST_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 427, 'ADMIN_USER_SETTING_GROUP' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 428, 'ADMIN_USER_SETTING_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 429, 'ADMIN_Z3950_SOURCE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 430, 'CREATE_BIB_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 431, 'CREATE_BIBLIO_FINGERPRINT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 432, 'CREATE_BIB_SOURCE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 433, 'CREATE_BILLING_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 434, 'CREATE_CN_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 435, 'CREATE_COPY_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 436, 'CREATE_INVOICE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 437, 'CREATE_INVOICE_ITEM_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 438, 'CREATE_INVOICE_METHOD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 439, 'CREATE_MERGE_PROFILE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 440, 'CREATE_METABIB_CLASS' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 441, 'CREATE_METABIB_SEARCH_ALIAS' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 442, 'CREATE_USER_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 443, 'DELETE_BIB_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 444, 'DELETE_BIBLIO_FINGERPRINT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 445, 'DELETE_BIB_SOURCE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 446, 'DELETE_BILLING_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 447, 'DELETE_CN_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 448, 'DELETE_COPY_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 449, 'DELETE_INVOICE_ITEM_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 450, 'DELETE_INVOICE_METHOD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 451, 'DELETE_MERGE_PROFILE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 452, 'DELETE_METABIB_CLASS' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 453, 'DELETE_METABIB_SEARCH_ALIAS' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 454, 'DELETE_USER_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 455, 'MANAGE_CLAIM' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 456, 'UPDATE_BIB_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 457, 'UPDATE_BIBLIO_FINGERPRINT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 458, 'UPDATE_BIB_SOURCE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 459, 'UPDATE_BILLING_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 460, 'UPDATE_CN_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 461, 'UPDATE_COPY_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 462, 'UPDATE_INVOICE_ITEM_TYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 463, 'UPDATE_INVOICE_METHOD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 464, 'UPDATE_MERGE_PROFILE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 465, 'UPDATE_METABIB_CLASS' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 466, 'UPDATE_METABIB_SEARCH_ALIAS' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 467, 'UPDATE_USER_BTYPE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 468, 'user_request.create' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 469, 'user_request.delete' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 470, 'user_request.update' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 471, 'user_request.view' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 472, 'VIEW_ACQ_FUND_ALLOCATION_PERCENT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 473, 'VIEW_CIRC_MATRIX_MATCHPOINT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 474, 'VIEW_CLAIM' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 475, 'VIEW_GROUP_PENALTY_THRESHOLD' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 476, 'VIEW_HOLD_MATRIX_MATCHPOINT' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 477, 'VIEW_INVOICE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 478, 'VIEW_MERGE_PROFILE' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 479, 'VIEW_SERIAL_SUBSCRIPTION' );
-INSERT INTO permission.temp_perm ( id, code ) VALUES ( 480, 'VIEW_STANDING_PENALTY' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 396, 'ADMIN_ACQ_CLAIM' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 397, 'ADMIN_ACQ_CLAIM_EVENT_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 398, 'ADMIN_ACQ_CLAIM_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 399, 'ADMIN_ACQ_DISTRIB_FORMULA' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 400, 'ADMIN_ACQ_FISCAL_YEAR' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 401, 'ADMIN_ACQ_FUND_ALLOCATION_PERCENT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 402, 'ADMIN_ACQ_FUND_TAG' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 403, 'ADMIN_ACQ_LINEITEM_ALERT_TEXT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 404, 'ADMIN_AGE_PROTECT_RULE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 405, 'ADMIN_ASSET_COPY_TEMPLATE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 406, 'ADMIN_BOOKING_RESERVATION_ATTR_MAP' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 407, 'ADMIN_CIRC_MATRIX_MATCHPOINT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 408, 'ADMIN_CIRC_MOD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 409, 'ADMIN_CLAIM_POLICY' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 410, 'ADMIN_CONFIG_REMOTE_ACCOUNT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 411, 'ADMIN_FIELD_DOC' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 412, 'ADMIN_GLOBAL_FLAG' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 413, 'ADMIN_GROUP_PENALTY_THRESHOLD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 414, 'ADMIN_HOLD_CANCEL_CAUSE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 415, 'ADMIN_HOLD_MATRIX_MATCHPOINT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 416, 'ADMIN_IDENT_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 417, 'ADMIN_IMPORT_ITEM_ATTR_DEF' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 418, 'ADMIN_INDEX_NORMALIZER' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 419, 'ADMIN_INVOICE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 420, 'ADMIN_INVOICE_METHOD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 421, 'ADMIN_INVOICE_PAYMENT_METHOD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 422, 'ADMIN_LINEITEM_MARC_ATTR_DEF' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 423, 'ADMIN_MARC_CODE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 424, 'ADMIN_MAX_FINE_RULE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 425, 'ADMIN_MERGE_PROFILE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 426, 'ADMIN_ORG_UNIT_SETTING_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 427, 'ADMIN_RECURRING_FINE_RULE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 428, 'ADMIN_STANDING_PENALTY' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 429, 'ADMIN_SURVEY' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 430, 'ADMIN_USER_REQUEST_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 431, 'ADMIN_USER_SETTING_GROUP' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 432, 'ADMIN_USER_SETTING_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 433, 'ADMIN_Z3950_SOURCE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 434, 'CREATE_BIB_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 435, 'CREATE_BIBLIO_FINGERPRINT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 436, 'CREATE_BIB_SOURCE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 437, 'CREATE_BILLING_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 438, 'CREATE_CN_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 439, 'CREATE_COPY_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 440, 'CREATE_INVOICE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 441, 'CREATE_INVOICE_ITEM_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 442, 'CREATE_INVOICE_METHOD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 443, 'CREATE_MERGE_PROFILE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 444, 'CREATE_METABIB_CLASS' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 445, 'CREATE_METABIB_SEARCH_ALIAS' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 446, 'CREATE_USER_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 447, 'DELETE_BIB_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 448, 'DELETE_BIBLIO_FINGERPRINT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 449, 'DELETE_BIB_SOURCE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 450, 'DELETE_BILLING_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 451, 'DELETE_CN_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 452, 'DELETE_COPY_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 453, 'DELETE_INVOICE_ITEM_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 454, 'DELETE_INVOICE_METHOD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 455, 'DELETE_MERGE_PROFILE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 456, 'DELETE_METABIB_CLASS' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 457, 'DELETE_METABIB_SEARCH_ALIAS' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 458, 'DELETE_USER_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 459, 'MANAGE_CLAIM' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 460, 'UPDATE_BIB_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 461, 'UPDATE_BIBLIO_FINGERPRINT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 462, 'UPDATE_BIB_SOURCE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 463, 'UPDATE_BILLING_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 464, 'UPDATE_CN_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 465, 'UPDATE_COPY_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 466, 'UPDATE_INVOICE_ITEM_TYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 467, 'UPDATE_INVOICE_METHOD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 468, 'UPDATE_MERGE_PROFILE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 469, 'UPDATE_METABIB_CLASS' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 470, 'UPDATE_METABIB_SEARCH_ALIAS' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 471, 'UPDATE_USER_BTYPE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 472, 'user_request.create' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 473, 'user_request.delete' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 474, 'user_request.update' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 475, 'user_request.view' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 476, 'VIEW_ACQ_FUND_ALLOCATION_PERCENT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 477, 'VIEW_CIRC_MATRIX_MATCHPOINT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 478, 'VIEW_CLAIM' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 479, 'VIEW_GROUP_PENALTY_THRESHOLD' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 480, 'VIEW_HOLD_MATRIX_MATCHPOINT' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 481, 'VIEW_INVOICE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 482, 'VIEW_MERGE_PROFILE' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 483, 'VIEW_SERIAL_SUBSCRIPTION' );
+INSERT INTO permission.temp_perm ( id, code ) VALUES ( 484, 'VIEW_STANDING_PENALTY' );
 
 -- For every permission in the temp_perm table that has a matching
 -- permission in the real table: record the original id.
@@ -3234,8 +3263,19 @@ INSERT INTO action_trigger.environment (
 INSERT INTO action_trigger.event_definition
 (id, active, owner, name, hook, validator, reactor, cleanup_success, cleanup_failure, delay, delay_field, group_field, template) VALUES
 (23, true, 1, 'PO JEDI', 'acqpo.activated', 'Acq::PurchaseOrderEDIRequired', 'GeneratePurchaseOrderJEDI', NULL, NULL, '00:05:00', NULL, NULL,
-$$[%- USE date -%]
-[%# start JEDI document -%]
+$$
+[%- USE date -%]
+[%# start JEDI document 
+  # Vendor specific kludges:
+  # BT      - vendcode goes to NAD/BY *suffix*  w/ 91 qualifier
+  # INGRAM  - vendcode goes to NAD/BY *segment* w/ 91 qualifier (separately)
+  # BRODART - vendcode goes to FTX segment (lineitem level)
+-%]
+[%- 
+IF target.provider.edi_default.vendcode && target.provider.code == 'BRODART';
+    xtra_ftx = target.provider.edi_default.vendcode;
+END;
+-%]
 [%- BLOCK big_block -%]
 {
    "recipient":"[% target.provider.san %]",
@@ -3244,23 +3284,27 @@ $$[%- USE date -%]
      "ORDERS":[ "order", {
         "po_number":[% target.id %],
         "date":"[% date.format(date.now, '%Y%m%d') %]",
-        "buyer":[{
-            [%- IF target.provider.edi_default.vendcode -%]
-                "id":"[% target.ordering_agency.mailing_address.san _ ' ' _ target.provider.edi_default.vendcode %]", 
-                "id-qualifier": 91
+        "buyer":[
+            [%   IF   target.provider.edi_default.vendcode && (target.provider.code == 'BT' || target.provider.name.match('(?i)^BAKER & TAYLOR'))  -%]
+                {"id-qualifier": 91, "id":"[% target.ordering_agency.mailing_address.san _ ' ' _ target.provider.edi_default.vendcode %]"}
+            [%- ELSIF target.provider.edi_default.vendcode && target.provider.code == 'INGRAM' -%]
+                {"id":"[% target.ordering_agency.mailing_address.san %]"},
+                {"id-qualifier": 91, "id":"[% target.provider.edi_default.vendcode %]"}
             [%- ELSE -%]
-                "id":"[% target.ordering_agency.mailing_address.san %]"
-            [%- END  -%]
-        }],
-        "vendor":[ 
+                {"id":"[% target.ordering_agency.mailing_address.san %]"}
+            [%- END -%]
+        ],
+        "vendor":[
             [%- # target.provider.name (target.provider.id) -%]
             "[% target.provider.san %]",
             {"id-qualifier": 92, "id":"[% target.provider.id %]"}
         ],
         "currency":"[% target.provider.currency_type %]",
+                
         "items":[
-        [% FOR li IN target.lineitems %]
+        [%- FOR li IN target.lineitems %]
         {
+            "line_index":"[% li.id %]",
             "identifiers":[   [%-# li.isbns = helpers.get_li_isbns(li.attributes) %]
             [% FOR isbn IN helpers.get_li_isbns(li.attributes) -%]
                 [% IF isbn.length == 13 -%]
@@ -3269,28 +3313,40 @@ $$[%- USE date -%]
                 {"id-qualifier":"IB","id":"[% isbn %]"},
                 [%- END %]
             [% END %]
-                {"id-qualifier":"SA","id":"[% li.id %]"}
+                {"id-qualifier":"IN","id":"[% li.id %]"}
             ],
             "price":[% li.estimated_unit_price || '0.00' %],
             "desc":[
-                {"BTI":"[% helpers.get_li_attr('title',     '', li.attributes) %]"}, 
+                {"BTI":"[% helpers.get_li_attr('title',     '', li.attributes) %]"},
                 {"BPU":"[% helpers.get_li_attr('publisher', '', li.attributes) %]"},
                 {"BPD":"[% helpers.get_li_attr('pubdate',   '', li.attributes) %]"},
                 {"BPH":"[% helpers.get_li_attr('pagination','', li.attributes) %]"}
             ],
+            [%- ftx_vals = []; 
+                FOR note IN li.lineitem_notes; 
+                    NEXT UNLESS note.vendor_public == 't'; 
+                    ftx_vals.push(note.value); 
+                END; 
+                IF xtra_ftx;           ftx_vals.unshift(xtra_ftx); END; 
+                IF ftx_vals.size == 0; ftx_vals.unshift('');       END;  # BT needs FTX+LIN for every LI, even if it is an empty one
+            -%]
+
+            "free-text":[ 
+                [% FOR note IN ftx_vals -%] "[% note %]"[% UNLESS loop.last %], [% END %][% END %] 
+            ],            
             "quantity":[% li.lineitem_details.size %]
         }[% UNLESS loop.last %],[% END %]
         [%-# TODO: lineitem details (later) -%]
         [% END %]
         ],
         "line_items":[% target.lineitems.size %]
-     }]  [% # close ORDERS array %]
-   }]    [% # close  body  array %]
+     }]  [%# close ORDERS array %]
+   }]    [%# close  body  array %]
 }
 [% END %]
 [% tempo = PROCESS big_block; helpers.escape_json(tempo) %]
-$$
-);
+
+$$);
 
 INSERT INTO action_trigger.environment (event_def, path) VALUES 
   (23, 'lineitems.attributes'), 
@@ -6049,10 +6105,10 @@ ALTER TABLE auditor.asset_copy_history
 CREATE OR REPLACE FUNCTION asset.acp_status_changed()
 RETURNS TRIGGER AS $$
 BEGIN
-	IF NEW.status <> OLD.status THEN
-		NEW.status_changed_time := now();
-	END IF;
-	RETURN NEW;
+    IF NEW.status <> OLD.status THEN
+        NEW.status_changed_time := now();
+    END IF;
+    RETURN NEW;
 END;
 $$ LANGUAGE plpgsql;
 
@@ -6263,6 +6319,23 @@ CREATE UNIQUE INDEX only_one_concurrent_checkout_per_copy ON action.circulation(
 
 ALTER TABLE action.circulation DROP CONSTRAINT action_circulation_target_copy_fkey;
 
+-- Rebuild dependent views
+
+DROP VIEW IF EXISTS action.billable_circulations;
+
+CREATE OR REPLACE VIEW action.billable_circulations AS
+    SELECT  *
+      FROM  action.circulation
+      WHERE xact_finish IS NULL;
+
+DROP VIEW IF EXISTS action.open_circulation;
+
+CREATE OR REPLACE VIEW action.open_circulation AS
+    SELECT  *
+      FROM  action.circulation
+      WHERE checkin_time IS NULL
+      ORDER BY due_date;
+
 CREATE OR REPLACE FUNCTION action.age_circ_on_delete () RETURNS TRIGGER AS $$
 DECLARE
 found char := 'N';
@@ -6308,7 +6381,122 @@ UPDATE config.z3950_attr SET truncation = 1 WHERE source = 'biblios' AND truncat
 
 -- Adding circ.holds.target_skip_me OU setting logic to the pre-matchpoint tests
 
-CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+CREATE OR REPLACE FUNCTION action.find_hold_matrix_matchpoint( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS INT AS $func$
+DECLARE
+    current_requestor_group    permission.grp_tree%ROWTYPE;
+    requestor_object    actor.usr%ROWTYPE;
+    user_object        actor.usr%ROWTYPE;
+    item_object        asset.copy%ROWTYPE;
+    item_cn_object        asset.call_number%ROWTYPE;
+    rec_descriptor        metabib.rec_descriptor%ROWTYPE;
+    current_mp_weight    FLOAT;
+    matchpoint_weight    FLOAT;
+    tmp_weight        FLOAT;
+    current_mp        config.hold_matrix_matchpoint%ROWTYPE;
+    matchpoint        config.hold_matrix_matchpoint%ROWTYPE;
+BEGIN
+    SELECT INTO user_object * FROM actor.usr WHERE id = match_user;
+    SELECT INTO requestor_object * FROM actor.usr WHERE id = match_requestor;
+    SELECT INTO item_object * FROM asset.copy WHERE id = match_item;
+    SELECT INTO item_cn_object * FROM asset.call_number WHERE id = item_object.call_number;
+    SELECT INTO rec_descriptor r.* FROM metabib.rec_descriptor r WHERE r.record = item_cn_object.record;
+
+    PERFORM * FROM config.internal_flag WHERE name = 'circ.holds.usr_not_requestor' AND enabled;
+
+    IF NOT FOUND THEN
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = requestor_object.profile;
+    ELSE
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = user_object.profile;
+    END IF;
+
+    LOOP 
+        -- for each potential matchpoint for this ou and group ...
+        FOR current_mp IN
+            SELECT    m.*
+              FROM    config.hold_matrix_matchpoint m
+              WHERE    m.requestor_grp = current_requestor_group.id AND m.active
+              ORDER BY    CASE WHEN m.circ_modifier    IS NOT NULL THEN 16 ELSE 0 END +
+                    CASE WHEN m.juvenile_flag    IS NOT NULL THEN 16 ELSE 0 END +
+                    CASE WHEN m.marc_type        IS NOT NULL THEN 8 ELSE 0 END +
+                    CASE WHEN m.marc_form        IS NOT NULL THEN 4 ELSE 0 END +
+                    CASE WHEN m.marc_vr_format    IS NOT NULL THEN 2 ELSE 0 END +
+                    CASE WHEN m.ref_flag        IS NOT NULL THEN 1 ELSE 0 END DESC LOOP
+
+            current_mp_weight := 5.0;
+
+            IF current_mp.circ_modifier IS NOT NULL THEN
+                CONTINUE WHEN current_mp.circ_modifier <> item_object.circ_modifier OR item_object.circ_modifier IS NULL;
+            END IF;
+
+            IF current_mp.marc_type IS NOT NULL THEN
+                IF item_object.circ_as_type IS NOT NULL THEN
+                    CONTINUE WHEN current_mp.marc_type <> item_object.circ_as_type;
+                ELSE
+                    CONTINUE WHEN current_mp.marc_type <> rec_descriptor.item_type;
+                END IF;
+            END IF;
+
+            IF current_mp.marc_form IS NOT NULL THEN
+                CONTINUE WHEN current_mp.marc_form <> rec_descriptor.item_form;
+            END IF;
+
+            IF current_mp.marc_vr_format IS NOT NULL THEN
+                CONTINUE WHEN current_mp.marc_vr_format <> rec_descriptor.vr_format;
+            END IF;
+
+            IF current_mp.juvenile_flag IS NOT NULL THEN
+                CONTINUE WHEN current_mp.juvenile_flag <> user_object.juvenile;
+            END IF;
+
+            IF current_mp.ref_flag IS NOT NULL THEN
+                CONTINUE WHEN current_mp.ref_flag <> item_object.ref;
+            END IF;
+
+
+            -- caclulate the rule match weight
+            IF current_mp.item_owning_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_owning_ou, item_cn_object.owning_lib)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.item_circ_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_circ_ou, item_object.circ_lib)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.pickup_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.pickup_ou, pickup_ou)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.request_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.request_ou, request_ou)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.user_home_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.user_home_ou, user_object.home_ou)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            -- set the matchpoint if we found the best one
+            IF matchpoint_weight IS NULL OR matchpoint_weight > current_mp_weight THEN
+                matchpoint = current_mp;
+                matchpoint_weight = current_mp_weight;
+            END IF;
+
+        END LOOP;
+
+        EXIT WHEN current_requestor_group.parent IS NULL OR matchpoint.id IS NOT NULL;
+
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = current_requestor_group.parent;
+    END LOOP;
+
+    RETURN matchpoint.id;
+END;
+$func$ LANGUAGE plpgsql;
+
+CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT, retargetting BOOL ) RETURNS SETOF action.matrix_test_result AS $func$
 DECLARE
     matchpoint_id        INT;
     user_object        actor.usr%ROWTYPE;
@@ -6410,22 +6598,7 @@ BEGIN
         END IF;
     END IF;
  
-    FOR standing_penalty IN
-        SELECT  DISTINCT csp.*
-          FROM  actor.usr_standing_penalty usp
-                JOIN config.standing_penalty csp ON (csp.id = usp.standing_penalty)
-          WHERE usr = match_user
-                AND usp.org_unit IN ( SELECT * FROM explode_array(context_org_list) )
-                AND (usp.stop_date IS NULL or usp.stop_date > NOW())
-                AND csp.block_list LIKE '%HOLD%' LOOP
-
-        result.fail_part := standing_penalty.name;
-        result.success := FALSE;
-        done := TRUE;
-        RETURN NEXT result;
-    END LOOP;
-
-    IF hold_test.stop_blocked_user IS TRUE THEN
+    IF NOT retargetting THEN
         FOR standing_penalty IN
             SELECT  DISTINCT csp.*
               FROM  actor.usr_standing_penalty usp
@@ -6433,28 +6606,45 @@ BEGIN
               WHERE usr = match_user
                     AND usp.org_unit IN ( SELECT * FROM explode_array(context_org_list) )
                     AND (usp.stop_date IS NULL or usp.stop_date > NOW())
-                    AND csp.block_list LIKE '%CIRC%' LOOP
+                    AND csp.block_list LIKE '%HOLD%' LOOP
     
             result.fail_part := standing_penalty.name;
             result.success := FALSE;
             done := TRUE;
             RETURN NEXT result;
         END LOOP;
-    END IF;
-
-    IF hold_test.max_holds IS NOT NULL THEN
-        SELECT    INTO hold_count COUNT(*)
-          FROM    action.hold_request
-          WHERE    usr = match_user
-            AND fulfillment_time IS NULL
-            AND cancel_time IS NULL
-            AND CASE WHEN hold_test.include_frozen_holds THEN TRUE ELSE frozen IS FALSE END;
-
-        IF hold_count >= hold_test.max_holds THEN
-            result.fail_part := 'config.hold_matrix_test.max_holds';
-            result.success := FALSE;
-            done := TRUE;
-            RETURN NEXT result;
+    
+        IF hold_test.stop_blocked_user IS TRUE THEN
+            FOR standing_penalty IN
+                SELECT  DISTINCT csp.*
+                  FROM  actor.usr_standing_penalty usp
+                        JOIN config.standing_penalty csp ON (csp.id = usp.standing_penalty)
+                  WHERE usr = match_user
+                        AND usp.org_unit IN ( SELECT * FROM explode_array(context_org_list) )
+                        AND (usp.stop_date IS NULL or usp.stop_date > NOW())
+                        AND csp.block_list LIKE '%CIRC%' LOOP
+        
+                result.fail_part := standing_penalty.name;
+                result.success := FALSE;
+                done := TRUE;
+                RETURN NEXT result;
+            END LOOP;
+        END IF;
+    
+        IF hold_test.max_holds IS NOT NULL THEN
+            SELECT    INTO hold_count COUNT(*)
+              FROM    action.hold_request
+              WHERE    usr = match_user
+                AND fulfillment_time IS NULL
+                AND cancel_time IS NULL
+                AND CASE WHEN hold_test.include_frozen_holds THEN TRUE ELSE frozen IS FALSE END;
+    
+            IF hold_count >= hold_test.max_holds THEN
+                result.fail_part := 'config.hold_matrix_test.max_holds';
+                result.success := FALSE;
+                done := TRUE;
+                RETURN NEXT result;
+            END IF;
         END IF;
     END IF;
 
@@ -6485,6 +6675,14 @@ BEGIN
 END;
 $func$ LANGUAGE plpgsql;
 
+CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+    SELECT * FROM action.hold_request_permit_test( $1, $2, $3, $4, $5, FALSE);
+$func$ LANGUAGE SQL;
+
+CREATE OR REPLACE FUNCTION action.hold_retarget_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+    SELECT * FROM action.hold_request_permit_test( $1, $2, $3, $4, $5, TRUE );
+$func$ LANGUAGE SQL;
+
 -- New post-delete trigger to propagate deletions to parent(s)
 
 CREATE OR REPLACE FUNCTION action.age_parent_circ_on_delete () RETURNS TRIGGER AS $$
@@ -7107,7 +7305,7 @@ BEGIN
         END;
         $func$ LANGUAGE 'plpgsql';
     $$;
-    RETURN TRUE;
+	RETURN TRUE;
 END;
 $creator$ LANGUAGE 'plpgsql';
 
@@ -7147,7 +7345,7 @@ BEGIN
     PERFORM auditor.create_auditor_func(sch, tbl);
     PERFORM auditor.create_auditor_update_trigger(sch, tbl);
     PERFORM auditor.create_auditor_lifecycle(sch, tbl);
-	RETURN TRUE;
+    RETURN TRUE;
 END;
 $creator$ LANGUAGE 'plpgsql';
 
@@ -7175,7 +7373,7 @@ ALTER TABLE actor.usr ADD COLUMN
 ALTER TABLE AUDITOR.actor_usr_history ADD COLUMN 
 	claims_never_checked_out_count INT;
 
-DROP VIEW auditor.actor_usr_lifecycle;
+DROP VIEW IF EXISTS auditor.actor_usr_lifecycle;
 
 SELECT auditor.create_auditor_lifecycle( 'actor', 'usr' );
 
@@ -7941,7 +8139,7 @@ CREATE TABLE booking.resource_type (
 	                               DEFERRABLE INITIALLY DEFERRED,
     max_fine       NUMERIC(8,2),
     elbow_room     INTERVAL,
-    CONSTRAINT brt_name_or_record_once_per_owner UNIQUE(owner, name, record)
+    CONSTRAINT brt_name_and_record_once_per_owner UNIQUE(owner, name, record)
 );
 
 CREATE TABLE booking.resource (
@@ -8564,7 +8762,7 @@ ADD COLUMN fiscal_calendar INT NOT NULL
 ALTER TABLE auditor.actor_org_unit_history
 	ADD COLUMN fiscal_calendar INT;
 
-DROP VIEW auditor.actor_org_unit_lifecycle;
+DROP VIEW IF EXISTS auditor.actor_org_unit_lifecycle;
 
 SELECT auditor.create_auditor_lifecycle( 'actor', 'org_unit' );
 
@@ -8969,6 +9167,133 @@ BEGIN
 END;
 $$ LANGUAGE plpgsql;
 
+COMMENT ON FUNCTION actor.usr_purge_data(INT, INT) IS $$
+/**
+ * Finds rows dependent on a given row in actor.usr and either deletes them
+ * or reassigns them to a different user.
+ */
+$$;
+
+CREATE OR REPLACE FUNCTION actor.usr_delete(
+	src_usr  IN INTEGER,
+	dest_usr IN INTEGER
+) RETURNS VOID AS $$
+DECLARE
+	old_profile actor.usr.profile%type;
+	old_home_ou actor.usr.home_ou%type;
+	new_profile actor.usr.profile%type;
+	new_home_ou actor.usr.home_ou%type;
+	new_name    text;
+	new_dob     actor.usr.dob%type;
+BEGIN
+	SELECT
+		id || '-PURGED-' || now(),
+		profile,
+		home_ou,
+		dob
+	INTO
+		new_name,
+		old_profile,
+		old_home_ou,
+		new_dob
+	FROM
+		actor.usr
+	WHERE
+		id = src_usr;
+	--
+	-- Quit if no such user
+	--
+	IF old_profile IS NULL THEN
+		RETURN;
+	END IF;
+	--
+	perform actor.usr_purge_data( src_usr, dest_usr );
+	--
+	-- Find the root grp_tree and the root org_unit.  This would be simpler if we 
+	-- could assume that there is only one root.  Theoretically, someday, maybe,
+	-- there could be multiple roots, so we take extra trouble to get the right ones.
+	--
+	SELECT
+		id
+	INTO
+		new_profile
+	FROM
+		permission.grp_ancestors( old_profile )
+	WHERE
+		parent is null;
+	--
+	SELECT
+		id
+	INTO
+		new_home_ou
+	FROM
+		actor.org_unit_ancestors( old_home_ou )
+	WHERE
+		parent_ou is null;
+	--
+	-- Truncate date of birth
+	--
+	IF new_dob IS NOT NULL THEN
+		new_dob := date_trunc( 'year', new_dob );
+	END IF;
+	--
+	UPDATE
+		actor.usr
+		SET
+			card = NULL,
+			profile = new_profile,
+			usrname = new_name,
+			email = NULL,
+			passwd = random()::text,
+			standing = DEFAULT,
+			ident_type = 
+			(
+				SELECT MIN( id )
+				FROM config.identification_type
+			),
+			ident_value = NULL,
+			ident_type2 = NULL,
+			ident_value2 = NULL,
+			net_access_level = DEFAULT,
+			photo_url = NULL,
+			prefix = NULL,
+			first_given_name = new_name,
+			second_given_name = NULL,
+			family_name = new_name,
+			suffix = NULL,
+			alias = NULL,
+			day_phone = NULL,
+			evening_phone = NULL,
+			other_phone = NULL,
+			mailing_address = NULL,
+			billing_address = NULL,
+			home_ou = new_home_ou,
+			dob = new_dob,
+			active = FALSE,
+			master_account = DEFAULT, 
+			super_user = DEFAULT,
+			barred = FALSE,
+			deleted = TRUE,
+			juvenile = DEFAULT,
+			usrgroup = 0,
+			claims_returned_count = DEFAULT,
+			credit_forward_balance = DEFAULT,
+			last_xact_id = DEFAULT,
+			alert_message = NULL,
+			create_date = now(),
+			expire_date = now()
+	WHERE
+		id = src_usr;
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION actor.usr_delete(INT, INT) IS $$
+/**
+ * Logically deletes a user.  Removes personally identifiable information,
+ * and purges associated data in other tables.
+ */
+$$;
+
 -- INSERT INTO config.copy_status (id,name) VALUES (15,oils_i18n_gettext(15, 'On reservation shelf', 'ccs', 'name'));
 
 ALTER TABLE acq.fund
@@ -9053,7 +9378,11 @@ COMMENT ON VIEW acq.ordered_funding_source_credit IS $$
 $$;
 
 CREATE OR REPLACE VIEW money.billable_xact_summary_location_view AS
-	SELECT * FROM money.materialized_billable_xact_summary;
+    SELECT  m.*, COALESCE(c.circ_lib, g.billing_location, r.pickup_lib) AS billing_location
+      FROM  money.materialized_billable_xact_summary m
+            LEFT JOIN action.circulation c ON (c.id = m.id)
+            LEFT JOIN money.grocery g ON (g.id = m.id)
+            LEFT JOIN booking.reservation r ON (r.id = m.id);
 
 CREATE TABLE config.marc21_rec_type_map (
     code        TEXT    PRIMARY KEY,
@@ -10359,6 +10688,8 @@ INSERT INTO config.internal_flag (name) VALUES ('ingest.disable_metabib_full_rec
 INSERT INTO config.internal_flag (name) VALUES ('ingest.disable_metabib_rec_descriptor');
 INSERT INTO config.internal_flag (name) VALUES ('ingest.disable_metabib_field_entry');
 INSERT INTO config.internal_flag (name) VALUES ('ingest.disable_authority_linking');
+INSERT INTO config.internal_flag (name) VALUES ('ingest.metarecord_mapping.skip_on_update');
+INSERT INTO config.internal_flag (name) VALUES ('ingest.assume_inserts_only');
 
 CREATE TABLE authority.bib_linking (
     id          BIGSERIAL   PRIMARY KEY,
@@ -10387,7 +10718,10 @@ $func$ LANGUAGE SQL;
 
 CREATE OR REPLACE FUNCTION metabib.reingest_metabib_rec_descriptor( bib_id BIGINT ) RETURNS VOID AS $func$
 BEGIN
-    DELETE FROM metabib.rec_descriptor WHERE record = bib_id;
+    PERFORM * FROM config.internal_flag WHERE name = 'ingest.assume_inserts_only' AND enabled;
+    IF NOT FOUND THEN
+        DELETE FROM metabib.rec_descriptor WHERE record = bib_id;
+    END IF;
     INSERT INTO metabib.rec_descriptor (record, item_type, item_form, bib_level, control_type, enc_level, audience, lit_form, type_mat, cat_form, pub_status, item_lang, vr_format, date1, date2)
         SELECT  bib_id,
                 biblio.marc21_extract_fixed_field( bib_id, 'Type' ),
@@ -10406,8 +10740,8 @@ BEGIN
                             JOIN config.marc21_physical_characteristic_subfield_map s ON (s.id = p.subfield)
                             JOIN config.marc21_physical_characteristic_value_map v ON (v.id = p.value)
                       WHERE p.ptype = 'v' AND s.subfield = 'e'    ),
-                biblio.marc21_extract_fixed_field( bib_id, 'Date1'),
-                biblio.marc21_extract_fixed_field( bib_id, 'Date2');
+                LPAD(NULLIF(REGEXP_REPLACE(NULLIF(biblio.marc21_extract_fixed_field( bib_id, 'Date1'), ''), E'\\D', '0', 'g')::INT,0)::TEXT,4,'0'),
+                LPAD(NULLIF(REGEXP_REPLACE(NULLIF(biblio.marc21_extract_fixed_field( bib_id, 'Date2'), ''), E'\\D', '9', 'g')::INT,9999)::TEXT,4,'0');
 
     RETURN;
 END;
@@ -10436,12 +10770,14 @@ DECLARE
     fclass          RECORD;
     ind_data        metabib.field_entry_template%ROWTYPE;
 BEGIN
-    FOR fclass IN SELECT * FROM config.metabib_class LOOP
-        -- RAISE NOTICE 'Emptying out %', fclass.name;
-        EXECUTE $$DELETE FROM metabib.$$ || fclass.name || $$_field_entry WHERE source = $$ || bib_id;
-    END LOOP;
-
-    DELETE FROM metabib.facet_entry WHERE source = bib_id;
+    PERFORM * FROM config.internal_flag WHERE name = 'ingest.assume_inserts_only' AND enabled;
+    IF NOT FOUND THEN
+        FOR fclass IN SELECT * FROM config.metabib_class LOOP
+            -- RAISE NOTICE 'Emptying out %', fclass.name;
+            EXECUTE $$DELETE FROM metabib.$$ || fclass.name || $$_field_entry WHERE source = $$ || bib_id;
+        END LOOP;
+        DELETE FROM metabib.facet_entry WHERE source = bib_id;
+    END IF;
 
     FOR ind_data IN SELECT * FROM biblio.extract_metabib_field_entry( bib_id ) LOOP
         IF ind_data.field < 0 THEN
@@ -10583,7 +10919,10 @@ $func$ LANGUAGE PLPGSQL;
 
 CREATE OR REPLACE FUNCTION metabib.reingest_metabib_full_rec( bib_id BIGINT ) RETURNS VOID AS $func$
 BEGIN
-    DELETE FROM metabib.real_full_rec WHERE record = bib_id;
+    PERFORM * FROM config.internal_flag WHERE name = 'ingest.assume_inserts_only' AND enabled;
+    IF NOT FOUND THEN
+        DELETE FROM metabib.real_full_rec WHERE record = bib_id;
+    END IF;
     INSERT INTO metabib.real_full_rec (record, tag, ind1, ind2, subfield, value)
         SELECT record, tag, ind1, ind2, subfield, value FROM biblio.flatten_marc( bib_id );
 
@@ -10648,7 +10987,10 @@ BEGIN
             PERFORM metabib.remap_metarecord_for_bib( NEW.id, NEW.fingerprint );
         END IF;
     ELSE -- we're doing an update, and we're not deleted, remap
-        PERFORM metabib.remap_metarecord_for_bib( NEW.id, NEW.fingerprint );
+        PERFORM * FROM config.internal_flag WHERE name = 'ingest.metarecord_mapping.skip_on_update' AND enabled;
+        IF NOT FOUND THEN
+            PERFORM metabib.remap_metarecord_for_bib( NEW.id, NEW.fingerprint );
+        END IF;
     END IF;
 
     RETURN NEW;
@@ -10660,8 +11002,7 @@ CREATE TRIGGER aaa_indexing_ingest_or_delete AFTER INSERT OR UPDATE ON biblio.re
 
 DROP TRIGGER IF EXISTS zzz_update_materialized_simple_rec_delete_tgr ON biblio.record_entry;
 
-CREATE OR REPLACE FUNCTION oils_xpath_table ( key TEXT, document_field TEXT, relation_name TEXT, xpaths TEXT, criteria TEXT )
-RETURNS SETOF RECORD AS $func$
+CREATE OR REPLACE FUNCTION oils_xpath_table ( key TEXT, document_field TEXT, relation_name TEXT, xpaths TEXT, criteria TEXT ) RETURNS SETOF RECORD AS $func$
 DECLARE
     xpath_list  TEXT[];
     select_list TEXT[];
@@ -10671,52 +11012,56 @@ DECLARE
     empty_test  RECORD;
 BEGIN
     xpath_list := STRING_TO_ARRAY( xpaths, '|' );
-
+ 
     select_list := ARRAY_APPEND( select_list, key || '::INT AS key' );
-
+ 
     FOR i IN 1 .. ARRAY_UPPER(xpath_list,1) LOOP
-        select_list := ARRAY_APPEND(
-            select_list,
-            $sel$
-            EXPLODE_ARRAY(
-                COALESCE(
-                    NULLIF(
-                        oils_xpath(
-                            $sel$ ||
-                                quote_literal(
-                                    CASE
-                                        WHEN xpath_list[i] ~ $re$/[^/[]*@[^/]+$$re$ OR xpath_list[i] ~ $re$text\(\)$$re$ THEN xpath_list[i]
-                                        ELSE xpath_list[i] || '//text()'
-                                    END
-                                ) ||
-                            $sel$,
-                            $sel$ || document_field || $sel$
+        IF xpath_list[i] = 'null()' THEN
+            select_list := ARRAY_APPEND( select_list, 'NULL::TEXT AS c_' || i );
+        ELSE
+            select_list := ARRAY_APPEND(
+                select_list,
+                $sel$
+                EXPLODE_ARRAY(
+                    COALESCE(
+                        NULLIF(
+                            oils_xpath(
+                                $sel$ ||
+                                    quote_literal(
+                                        CASE
+                                            WHEN xpath_list[i] ~ $re$/[^/[]*@[^/]+$$re$ OR xpath_list[i] ~ $re$text\(\)$$re$ THEN xpath_list[i]
+                                            ELSE xpath_list[i] || '//text()'
+                                        END
+                                    ) ||
+                                $sel$,
+                                $sel$ || document_field || $sel$
+                            ),
+                           '{}'::TEXT[]
                         ),
-                       '{}'::TEXT[]
-                    ),
-                    '{NULL}'::TEXT[]
-                )
-            ) AS c_$sel$ || i
-        );
-        where_list := ARRAY_APPEND(
-            where_list,
-            'c_' || i || ' IS NOT NULL'
-        );
+                        '{NULL}'::TEXT[]
+                    )
+                ) AS c_$sel$ || i
+            );
+            where_list := ARRAY_APPEND(
+                where_list,
+                'c_' || i || ' IS NOT NULL'
+            );
+        END IF;
     END LOOP;
-
+ 
     q := $q$
 SELECT * FROM (
     SELECT $q$ || ARRAY_TO_STRING( select_list, ', ' ) || $q$ FROM $q$ || relation_name || $q$ WHERE ($q$ || criteria || $q$)
 )x WHERE $q$ || ARRAY_TO_STRING( where_list, ' AND ' );
     -- RAISE NOTICE 'query: %', q;
-
+ 
     FOR out_record IN EXECUTE q LOOP
         RETURN NEXT out_record;
     END LOOP;
-
+ 
     RETURN;
 END;
-$func$ LANGUAGE PLPGSQL;
+$func$ LANGUAGE PLPGSQL IMMUTABLE;
 
 CREATE OR REPLACE FUNCTION vandelay.ingest_items ( import_id BIGINT, attr_def_id BIGINT ) RETURNS SETOF vandelay.import_item AS $$
 DECLARE
@@ -11120,11 +11465,7 @@ BEGIN
 END;
 $creator$ LANGUAGE 'plpgsql';
 
-SELECT acq.create_acq_auditor ( 'acq', 'purchase_order' );
-CREATE INDEX acq_po_hist_id_idx            ON acq.acq_purchase_order_history( id );
-
-SELECT acq.create_acq_auditor ( 'acq', 'lineitem' );
-CREATE INDEX acq_lineitem_hist_id_idx            ON acq.acq_lineitem_history( id );
+ALTER TABLE acq.lineitem DROP COLUMN item_count;
 
 CREATE OR REPLACE VIEW acq.fund_debit_total AS
     SELECT  fund.id AS fund,
@@ -12649,7 +12990,7 @@ CREATE TABLE acq.user_request (
 
 CREATE TABLE acq.lineitem_alert_text (
 	id               SERIAL         PRIMARY KEY,
-	code             TEXT           UNIQUE NOT NULL,
+	code             TEXT           NOT NULL,
 	description      TEXT,
 	owning_lib       INT            NOT NULL
 	                                REFERENCES actor.org_unit(id)
@@ -12914,13 +13255,7 @@ FROM
                 fund
         ) AS c USING ( fund );
 
-CREATE OR REPLACE FUNCTION actor.usr_merge(
-	src_usr INT,
-	dest_usr INT,
-	del_addrs BOOLEAN,
-	del_cards BOOLEAN,
-	deactivate_cards BOOLEAN
-) RETURNS VOID AS $$
+CREATE OR REPLACE FUNCTION actor.usr_merge( src_usr INT, dest_usr INT, del_addrs BOOLEAN, del_cards BOOLEAN, deactivate_cards BOOLEAN ) RETURNS VOID AS $$
 DECLARE
 	suffix TEXT;
 	bucket_row RECORD;
@@ -13098,7 +13433,7 @@ BEGIN
 
     -- acq.*
     UPDATE acq.fund_allocation SET allocator = dest_usr WHERE allocator = src_usr;
-    UPDATE acq.fund_transfer SET transfer_user = dest_usr WHERE transfer_user = src_usr;
+	UPDATE acq.fund_transfer SET transfer_user = dest_usr WHERE transfer_user = src_usr;
 
 	-- transfer picklists the same way we transfer buckets (see above)
 	FOR picklist_row in
@@ -13123,8 +13458,8 @@ BEGIN
     UPDATE acq.purchase_order SET owner = dest_usr WHERE owner = src_usr;
     UPDATE acq.po_note SET creator = dest_usr WHERE creator = src_usr;
     UPDATE acq.po_note SET editor = dest_usr WHERE editor = src_usr;
-	UPDATE acq.provider_note SET creator = dest_usr WHERE creator = src_usr;
-	UPDATE acq.provider_note SET editor = dest_usr WHERE editor = src_usr;
+    UPDATE acq.provider_note SET creator = dest_usr WHERE creator = src_usr;
+    UPDATE acq.provider_note SET editor = dest_usr WHERE editor = src_usr;
     UPDATE acq.lineitem_note SET creator = dest_usr WHERE creator = src_usr;
     UPDATE acq.lineitem_note SET editor = dest_usr WHERE editor = src_usr;
     UPDATE acq.lineitem_usr_attr_definition SET usr = dest_usr WHERE usr = src_usr;
@@ -13272,6 +13607,10 @@ $$ LANGUAGE PLPGSQL;
 TRUNCATE money.materialized_billable_xact_summary;
 INSERT INTO money.materialized_billable_xact_summary SELECT * FROM money.billable_xact_summary;
 
+-- Now redefine the view as a window onto the materialized view
+CREATE OR REPLACE VIEW money.billable_xact_summary AS
+    SELECT * FROM money.materialized_billable_xact_summary;
+
 CREATE OR REPLACE FUNCTION permission.usr_has_perm_at_nd(
     user_id    IN INTEGER,
     perm_code  IN TEXT
@@ -13486,40 +13825,33 @@ END;
 $$ LANGUAGE 'plpgsql';
 
 ALTER TABLE acq.purchase_order
-	ADD COLUMN cancel_reason        INT REFERENCES acq.cancel_reason( id )
-	                                    DEFERRABLE INITIALLY DEFERRED;
-
-ALTER TABLE acq.acq_purchase_order_history
-	ADD COLUMN cancel_reason INTEGER;
-
-ALTER TABLE acq.purchase_order
+	ADD COLUMN cancel_reason INT
+		REFERENCES acq.cancel_reason( id )
+	    DEFERRABLE INITIALLY DEFERRED,
 	ADD COLUMN prepayment_required BOOLEAN NOT NULL DEFAULT FALSE;
 
-ALTER TABLE acq.acq_purchase_order_history
-	ADD COLUMN prepayment_required BOOLEAN NOT NULL DEFAULT FALSE;
+-- Build the history table and lifecycle view
+-- for acq.purchase_order
 
-ALTER TABLE acq.lineitem
-	ADD COLUMN cancel_reason        INT REFERENCES acq.cancel_reason( id )
-	                                    DEFERRABLE INITIALLY DEFERRED;
-
-ALTER TABLE acq.acq_lineitem_history
-	ADD COLUMN cancel_reason INTEGER;
-
-ALTER TABLE acq.lineitem
-	ADD COLUMN estimated_unit_price NUMERIC;
+SELECT acq.create_acq_auditor ( 'acq', 'purchase_order' );
 
-ALTER TABLE acq.acq_lineitem_history
-	ADD COLUMN estimated_unit_price NUMERIC;
+CREATE INDEX acq_po_hist_id_idx            ON acq.acq_purchase_order_history( id );
 
 ALTER TABLE acq.lineitem
+	ADD COLUMN cancel_reason INT
+		REFERENCES acq.cancel_reason( id )
+	    DEFERRABLE INITIALLY DEFERRED,
+	ADD COLUMN estimated_unit_price NUMERIC,
 	ADD COLUMN claim_policy INT
 		REFERENCES acq.claim_policy
-		DEFERRABLE INITIALLY DEFERRED;
+		DEFERRABLE INITIALLY DEFERRED,
+	ALTER COLUMN eg_bib_id SET DATA TYPE bigint;
 
-ALTER TABLE acq.acq_lineitem_history
-	ADD COLUMN claim_policy INT
-		REFERENCES acq.claim_policy
-		DEFERRABLE INITIALLY DEFERRED;
+-- Build the history table and lifecycle view
+-- for acq.lineitem
+
+SELECT acq.create_acq_auditor ( 'acq', 'lineitem' );
+CREATE INDEX acq_lineitem_hist_id_idx            ON acq.acq_lineitem_history( id );
 
 ALTER TABLE acq.lineitem_detail
 	ADD COLUMN cancel_reason        INT REFERENCES acq.cancel_reason( id )
@@ -13818,10 +14150,10 @@ BEGIN
         END IF;
     END IF;
 
-    add_rule := add_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="a"]/text()',incoming_xml),''),'');
-    strip_rule := strip_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="d"]/text()',incoming_xml),''),'');
-    replace_rule := replace_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="r"]/text()',incoming_xml),''),'');
-    preserve_rule := preserve_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="p"]/text()',incoming_xml),''),'');
+    add_rule := add_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="a"]/text()',incoming_xml),','),'');
+    strip_rule := strip_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="d"]/text()',incoming_xml),','),'');
+    replace_rule := replace_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="r"]/text()',incoming_xml),','),'');
+    preserve_rule := preserve_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="p"]/text()',incoming_xml),','),'');
 
     output.add_rule := BTRIM(add_rule,',');
     output.replace_rule := BTRIM(replace_rule,',');
@@ -14398,7 +14730,8 @@ $function$ LANGUAGE PLPGSQL;
 UPDATE config.metabib_field SET label = name;
 ALTER TABLE config.metabib_field ALTER COLUMN label SET NOT NULL;
 
-ALTER TABLE config.metabib_field ADD CONSTRAINT field_class_fkey FOREIGN KEY (field_class) REFERENCES config.metabib_class (name);
+ALTER TABLE config.metabib_field ADD CONSTRAINT metabib_field_field_class_fkey
+	 FOREIGN KEY (field_class) REFERENCES config.metabib_class (name);
 
 ALTER TABLE config.metabib_field DROP CONSTRAINT metabib_field_field_class_check;
 
@@ -14753,7 +15086,12 @@ BEGIN
 END;
 $func$ LANGUAGE PLPGSQL;
 
-ALTER TABLE biblio.record_entry ADD COLUMN owner INT REFERENCES actor.org_unit (id);
+ALTER TABLE biblio.record_entry ADD COLUMN owner INT;
+ALTER TABLE biblio.record_entry
+	 ADD CONSTRAINT biblio_record_entry_owner_fkey FOREIGN KEY (owner)
+	 REFERENCES actor.org_unit (id)
+	 DEFERRABLE INITIALLY DEFERRED;
+
 ALTER TABLE biblio.record_entry ADD COLUMN share_depth INT;
 
 ALTER TABLE auditor.biblio_record_entry_history ADD COLUMN owner INT;
@@ -15239,23 +15577,23 @@ BEGIN
             SELECT * INTO circ_chain_tail FROM action.circ_chain(circ_chain_head.id) ORDER BY xact_start DESC LIMIT 1;
             EXIT WHEN circ_chain_tail.xact_finish IS NULL;
 
-            -- Now get the user setings, if any, to block purging if the user wants to keep more circs
+            -- Now get the user settings, if any, to block purging if the user wants to keep more circs
             usr_keep_age.value := NULL;
             SELECT * INTO usr_keep_age FROM actor.usr_setting WHERE usr = circ_chain_head.usr AND name = 'history.circ.retention_age';
 
             usr_keep_start.value := NULL;
-            SELECT * INTO usr_keep_start FROM actor.usr_setting WHERE usr = circ_chain_head.usr AND name = 'history.circ.retention_start_date';
+            SELECT * INTO usr_keep_start FROM actor.usr_setting WHERE usr = circ_chain_head.usr AND name = 'history.circ.retention_start';
 
             IF usr_keep_age.value IS NOT NULL AND usr_keep_start.value IS NOT NULL THEN
-                IF oils_json_to_string(usr_keep_age.value)::INTERVAL > AGE(NOW(), oils_json_to_string(usr_keep_start.value)::TIMESTAMPTZ) THEN
-                    keep_age := AGE(NOW(), oils_json_to_string(usr_keep_start.value)::TIMESTAMPTZ);
+                IF oils_json_to_text(usr_keep_age.value)::INTERVAL > AGE(NOW(), oils_json_to_text(usr_keep_start.value)::TIMESTAMPTZ) THEN
+                    keep_age := AGE(NOW(), oils_json_to_text(usr_keep_start.value)::TIMESTAMPTZ);
                 ELSE
-                    keep_age := oils_json_to_string(usr_keep_age.value)::INTERVAL;
+                    keep_age := oils_json_to_text(usr_keep_age.value)::INTERVAL;
                 END IF;
             ELSIF usr_keep_start.value IS NOT NULL THEN
-                keep_age := AGE(NOW(), oils_json_to_string(usr_keep_start.value)::TIMESTAMPTZ);
+                keep_age := AGE(NOW(), oils_json_to_text(usr_keep_start.value)::TIMESTAMPTZ);
             ELSE
-                keep_age := COALESCE( org_keep_age::INTERVAL, '2000 years'::INTEVAL );
+                keep_age := COALESCE( org_keep_age::INTERVAL, '2000 years'::INTERVAL );
             END IF;
 
             EXIT WHEN AGE(NOW(), circ_chain_tail.xact_finish) < keep_age;
@@ -15405,6 +15743,8 @@ CREATE TABLE serial.subscription (
 	expected_date_offset   INTERVAL
 	-- acquisitions/business-side tables link to here
 );
+CREATE INDEX serial_subscription_record_idx ON serial.subscription (record_entry);
+CREATE INDEX serial_subscription_owner_idx ON serial.subscription (owning_lib);
 
 --at least one distribution per org_unit holding issues
 CREATE TABLE serial.distribution (
@@ -15436,6 +15776,8 @@ CREATE TABLE serial.distribution (
 	unit_label_prefix     TEXT,
 	unit_label_suffix     TEXT
 );
+CREATE INDEX serial_distribution_sub_idx ON serial.distribution (subscription);
+CREATE INDEX serial_distribution_holding_lib_idx ON serial.distribution (holding_lib);
 
 CREATE UNIQUE INDEX one_dist_per_sre_idx ON serial.distribution (record_entry);
 
@@ -15447,6 +15789,7 @@ CREATE TABLE serial.stream (
 	                        DEFERRABLE INITIALLY DEFERRED,
 	routing_label   TEXT
 );
+CREATE INDEX serial_stream_dist_idx ON serial.stream (distribution);
 
 CREATE UNIQUE INDEX label_once_per_dist
 	ON serial.stream (distribution, routing_label)
@@ -15472,6 +15815,8 @@ CREATE TABLE serial.routing_list_user (
 		(reader IS NULL AND department IS NOT NULL)
 	)
 );
+CREATE INDEX serial_routing_list_user_stream_idx ON serial.routing_list_user (stream);
+CREATE INDEX serial_routing_list_user_reader_idx ON serial.routing_list_user (reader);
 
 CREATE TABLE serial.caption_and_pattern (
 	id           SERIAL       PRIMARY KEY,
@@ -15498,6 +15843,7 @@ CREATE TABLE serial.caption_and_pattern (
 	chron_4      TEXT,
 	chron_5      TEXT
 );
+CREATE INDEX serial_caption_and_pattern_sub_idx ON serial.caption_and_pattern (subscription);
 
 CREATE TABLE serial.issuance (
 	id              SERIAL    PRIMARY KEY,
@@ -15526,12 +15872,20 @@ CREATE TABLE serial.issuance (
 	holding_link_id INT
 	-- TODO: add columns for separate enumeration/chronology values
 );
+CREATE INDEX serial_issuance_sub_idx ON serial.issuance (subscription);
+CREATE INDEX serial_issuance_caption_and_pattern_idx ON serial.issuance (caption_and_pattern);
+CREATE INDEX serial_issuance_date_published_idx ON serial.issuance (date_published);
 
 CREATE TABLE serial.unit (
 	label           TEXT,
 	label_sort_key  TEXT,
 	contents        TEXT    NOT NULL
 ) INHERITS (asset.copy);
+CREATE UNIQUE INDEX unit_barcode_key ON serial.unit (barcode) WHERE deleted = FALSE OR deleted IS FALSE;
+CREATE INDEX unit_cn_idx ON serial.unit (call_number);
+CREATE INDEX unit_avail_cn_idx ON serial.unit (call_number);
+CREATE INDEX unit_creator_idx  ON serial.unit ( creator );
+CREATE INDEX unit_editor_idx   ON serial.unit ( editor );
 
 ALTER TABLE serial.unit ADD PRIMARY KEY (id);
 
@@ -15573,6 +15927,12 @@ CREATE TABLE serial.item (
                             DEFAULT 'Expected',
 	shadowed        BOOL    NOT NULL DEFAULT FALSE
 );
+CREATE INDEX serial_item_stream_idx ON serial.item (stream);
+CREATE INDEX serial_item_issuance_idx ON serial.item (issuance);
+CREATE INDEX serial_item_unit_idx ON serial.item (unit);
+CREATE INDEX serial_item_uri_idx ON serial.item (uri);
+CREATE INDEX serial_item_date_received_idx ON serial.item (date_received);
+CREATE INDEX serial_item_status_idx ON serial.item (status);
 
 CREATE TABLE serial.item_note (
 	id          SERIAL  PRIMARY KEY,
@@ -15588,6 +15948,7 @@ CREATE TABLE serial.item_note (
 	title       TEXT    NOT NULL,
 	value       TEXT    NOT NULL
 );
+CREATE INDEX serial_item_note_item_idx ON serial.item_note (item);
 
 CREATE TABLE serial.basic_summary (
 	id                  SERIAL  PRIMARY KEY,
@@ -15599,6 +15960,7 @@ CREATE TABLE serial.basic_summary (
 	textual_holdings    TEXT,
 	show_generated      BOOL    NOT NULL DEFAULT TRUE
 );
+CREATE INDEX serial_basic_summary_dist_idx ON serial.basic_summary (distribution);
 
 CREATE TABLE serial.supplement_summary (
 	id                  SERIAL  PRIMARY KEY,
@@ -15610,6 +15972,7 @@ CREATE TABLE serial.supplement_summary (
 	textual_holdings    TEXT,
 	show_generated      BOOL    NOT NULL DEFAULT TRUE
 );
+CREATE INDEX serial_supplement_summary_dist_idx ON serial.supplement_summary (distribution);
 
 CREATE TABLE serial.index_summary (
 	id                  SERIAL  PRIMARY KEY,
@@ -15621,6 +15984,7 @@ CREATE TABLE serial.index_summary (
 	textual_holdings    TEXT,
 	show_generated      BOOL    NOT NULL DEFAULT TRUE
 );
+CREATE INDEX serial_index_summary_dist_idx ON serial.index_summary (distribution);
 
 -- DELETE FROM action_trigger.environment WHERE event_def IN (29,30); DELETE FROM action_trigger.event where event_def IN (29,30); DELETE FROM action_trigger.event_definition WHERE id IN (29,30); DELETE FROM action_trigger.hook WHERE key IN ('money.format.payment_receipt.email','money.format.payment_receipt.print'); DELETE FROM config.upgrade_log WHERE version = '0289'; -- from testing, this sql will remove these events, etc.
 
@@ -16070,6 +16434,64 @@ CREATE OR REPLACE FUNCTION authority.propagate_changes (aid BIGINT) RETURNS SETO
     SELECT authority.propagate_changes( authority, bib ) FROM authority.bib_linking WHERE authority = $1;
 $func$ LANGUAGE SQL;
 
+CREATE OR REPLACE FUNCTION authority.flatten_marc ( TEXT ) RETURNS SETOF authority.full_rec AS $func$
+
+use MARC::Record;
+use MARC::File::XML (BinaryEncoding => 'UTF-8');
+
+my $xml = shift;
+my $r = MARC::Record->new_from_xml( $xml );
+
+return_next( { tag => 'LDR', value => $r->leader } );
+
+for my $f ( $r->fields ) {
+    if ($f->is_control_field) {
+        return_next({ tag => $f->tag, value => $f->data });
+    } else {
+        for my $s ($f->subfields) {
+            return_next({
+                tag      => $f->tag,
+                ind1     => $f->indicator(1),
+                ind2     => $f->indicator(2),
+                subfield => $s->[0],
+                value    => $s->[1]
+            });
+
+        }
+    }
+}
+
+return undef;
+
+$func$ LANGUAGE PLPERLU;
+
+CREATE OR REPLACE FUNCTION authority.flatten_marc ( rid BIGINT ) RETURNS SETOF authority.full_rec AS $func$
+DECLARE
+    auth    authority.record_entry%ROWTYPE;
+    output    authority.full_rec%ROWTYPE;
+    field    RECORD;
+BEGIN
+    SELECT INTO auth * FROM authority.record_entry WHERE id = rid;
+
+    FOR field IN SELECT * FROM authority.flatten_marc( auth.marc ) LOOP
+        output.record := rid;
+        output.ind1 := field.ind1;
+        output.ind2 := field.ind2;
+        output.tag := field.tag;
+        output.subfield := field.subfield;
+        IF field.subfield IS NOT NULL THEN
+            output.value := naco_normalize(field.value, field.subfield);
+        ELSE
+            output.value := field.value;
+        END IF;
+
+        CONTINUE WHEN output.value IS NULL;
+
+        RETURN NEXT output;
+    END LOOP;
+END;
+$func$ LANGUAGE PLPGSQL;
+
 -- authority.rec_descriptor appears to be unused currently
 CREATE OR REPLACE FUNCTION authority.reingest_authority_rec_descriptor( auth_id BIGINT ) RETURNS VOID AS $func$
 BEGIN
@@ -16152,8 +16574,6 @@ BEGIN
             NEW.marc,
             E'()',
             E'' ||
-                '' || NEW.arn_value || E'' ||
-                '' || NEW.arn_source || E'' ||
                 '' || NEW.id || E'' ||
                 '' || TG_TABLE_SCHEMA || E'' ||
              E'\\1'
@@ -16255,6 +16675,18 @@ INSERT INTO config.global_flag (name, label, enabled)
         TRUE
     );
 
+INSERT INTO config.global_flag (name, label, enabled)
+    VALUES (
+        'circ.holds.usr_not_requestor',
+        oils_i18n_gettext(
+            'circ.holds.usr_not_requestor',
+            'Holds: When testing hold matrix matchpoints, use the profile group of the receiving user instead of that of the requestor (affects staff-placed holds)',
+            'cgf',
+            'label'
+        ),
+        TRUE
+    );
+
 CREATE OR REPLACE FUNCTION maintain_control_numbers() RETURNS TRIGGER AS $func$
 use strict;
 use MARC::Record;
@@ -16721,6 +17153,7 @@ CREATE TABLE serial.distribution_note (
 	title        TEXT   NOT NULL,
 	value        TEXT   NOT NULL
 );
+CREATE INDEX serial_distribution_note_dist_idx ON serial.distribution_note (distribution);
 
 ------- Begin surgery on serial.unit
 
@@ -17341,7 +17774,7 @@ ALTER TABLE asset.call_number
 	ADD COLUMN label_sortkey TEXT;
 
 CREATE INDEX asset_call_number_label_sortkey
-	ON asset.call_number(label_sortkey);
+	ON asset.call_number(oils_text_as_bytea(label_sortkey));
 
 ALTER TABLE auditor.asset_call_number_history
 	ADD COLUMN label_class BIGINT;
@@ -17970,13 +18403,6 @@ DROP INDEX authority.authority_record_unique_tcn;
 ALTER TABLE authority.record_entry DROP COLUMN arn_value;
 ALTER TABLE authority.record_entry DROP COLUMN arn_source;
 
-DROP INDEX IF EXISTS authority.unique_by_heading_and_thesaurus;
-
-CREATE INDEX by_heading_and_thesaurus
-    ON authority.record_entry (authority.normalize_heading(marc))
-    WHERE deleted IS FALSE or deleted = FALSE
-;
-
 ALTER TABLE acq.provider_contact
 	ALTER COLUMN name SET NOT NULL;
 
@@ -18026,6 +18452,198 @@ CREATE INDEX aud_bib_rec_entry_hist_creator_idx
 CREATE INDEX aud_bib_rec_entry_hist_editor_idx
 	ON auditor.biblio_record_entry_history ( editor );
 
+CREATE TABLE action.hold_request_note (
+
+    id     BIGSERIAL PRIMARY KEY,
+    hold   BIGINT    NOT NULL REFERENCES action.hold_request (id)
+                              ON DELETE CASCADE
+                              DEFERRABLE INITIALLY DEFERRED,
+    title  TEXT      NOT NULL,
+    body   TEXT      NOT NULL,
+    slip   BOOL      NOT NULL DEFAULT FALSE,
+    pub    BOOL      NOT NULL DEFAULT FALSE,
+    staff  BOOL      NOT NULL DEFAULT FALSE  -- created by staff
+
+);
+CREATE INDEX ahrn_hold_idx ON action.hold_request_note (hold);
+
+-- Tweak a constraint to add a CASCADE
+
+ALTER TABLE action.hold_notification DROP CONSTRAINT hold_notification_hold_fkey;
+
+ALTER TABLE action.hold_notification
+	ADD CONSTRAINT hold_notification_hold_fkey
+		FOREIGN KEY (hold) REFERENCES action.hold_request (id)
+		ON DELETE CASCADE
+		DEFERRABLE INITIALLY DEFERRED;
+
+CREATE TRIGGER asset_label_sortkey_trigger
+    BEFORE UPDATE OR INSERT ON asset.call_number
+    FOR EACH ROW EXECUTE PROCEDURE asset.label_normalizer();
+
+CREATE OR REPLACE FUNCTION container.clear_all_expired_circ_history_items( )
+RETURNS VOID AS $$
+--
+-- Delete expired circulation bucket items for all users that have
+-- a setting for patron.max_reading_list_interval.
+--
+DECLARE
+    today        TIMESTAMP WITH TIME ZONE;
+    threshold    TIMESTAMP WITH TIME ZONE;
+	usr_setting  RECORD;
+BEGIN
+	SELECT date_trunc( 'day', now() ) INTO today;
+	--
+	FOR usr_setting in
+		SELECT
+			usr,
+			value
+		FROM
+			actor.usr_setting
+		WHERE
+			name = 'patron.max_reading_list_interval'
+	LOOP
+		--
+		-- Make sure the setting is a valid interval
+		--
+		BEGIN
+			threshold := today - CAST( translate( usr_setting.value, '"', '' ) AS INTERVAL );
+		EXCEPTION
+			WHEN OTHERS THEN
+				RAISE NOTICE 'Invalid setting patron.max_reading_list_interval for user %: ''%''',
+					usr_setting.usr, usr_setting.value;
+				CONTINUE;
+		END;
+		--
+		--RAISE NOTICE 'User % threshold %', usr_setting.usr, threshold;
+		--
+    	DELETE FROM container.copy_bucket_item
+    	WHERE
+        	bucket IN
+        	(
+        	    SELECT
+        	        id
+        	    FROM
+        	        container.copy_bucket
+        	    WHERE
+        	        owner = usr_setting.usr
+        	        AND btype = 'circ_history'
+        	)
+        	AND create_time < threshold;
+	END LOOP;
+	--
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION container.clear_all_expired_circ_history_items( ) IS $$
+/*
+ * Delete expired circulation bucket items for all users that have
+ * a setting for patron.max_reading_list_interval.
+*/
+$$;
+
+CREATE OR REPLACE FUNCTION container.clear_expired_circ_history_items( 
+	 ac_usr IN INTEGER
+) RETURNS VOID AS $$
+--
+-- Delete old circulation bucket items for a specified user.
+-- "Old" means older than the interval specified by a
+-- user-level setting, if it is so specified.
+--
+DECLARE
+    threshold TIMESTAMP WITH TIME ZONE;
+BEGIN
+	-- Sanity check
+	IF ac_usr IS NULL THEN
+		RETURN;
+	END IF;
+	-- Determine the threshold date that defines "old".  Subtract the
+	-- interval from the system date, then truncate to midnight.
+	SELECT
+		date_trunc( 
+			'day',
+			now() - CAST( translate( value, '"', '' ) AS INTERVAL )
+		)
+	INTO
+		threshold
+	FROM
+		actor.usr_setting
+	WHERE
+		usr = ac_usr
+		AND name = 'patron.max_reading_list_interval';
+	--
+	IF threshold is null THEN
+		-- No interval defined; don't delete anything
+		-- RAISE NOTICE 'No interval defined for user %', ac_usr;
+		return;
+	END IF;
+	--
+	-- RAISE NOTICE 'Date threshold: %', threshold;
+	--
+	-- Threshold found; do the delete
+	delete from container.copy_bucket_item
+	where
+		bucket in
+		(
+			select
+				id
+			from
+				container.copy_bucket
+			where
+				owner = ac_usr
+				and btype = 'circ_history'
+		)
+		and create_time < threshold;
+	--
+	RETURN;
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION container.clear_expired_circ_history_items( INTEGER ) IS $$
+/*
+ * Delete old circulation bucket items for a specified user.
+ * "Old" means older than the interval specified by a
+ * user-level setting, if it is so specified.
+*/
+$$;
+
+CREATE OR REPLACE VIEW reporter.hold_request_record AS
+SELECT  id,
+    target,
+    hold_type,
+    CASE
+        WHEN hold_type = 'T'
+            THEN target
+        WHEN hold_type = 'I'
+            THEN (SELECT ssub.record_entry FROM serial.subscription ssub JOIN serial.issuance si ON (si.subscription = ssub.id) WHERE si.id = ahr.target)
+        WHEN hold_type = 'V'
+            THEN (SELECT cn.record FROM asset.call_number cn WHERE cn.id = ahr.target)
+        WHEN hold_type IN ('C','R','F')
+            THEN (SELECT cn.record FROM asset.call_number cn JOIN asset.copy cp ON (cn.id = cp.call_number) WHERE cp.id = ahr.target)
+        WHEN hold_type = 'M'
+            THEN (SELECT mr.master_record FROM metabib.metarecord mr WHERE mr.id = ahr.target)
+    END AS bib_record
+  FROM  action.hold_request ahr;
+
+UPDATE  metabib.rec_descriptor
+  SET   date1=LPAD(NULLIF(REGEXP_REPLACE(NULLIF(date1, ''), E'\\D', '0', 'g')::INT,0)::TEXT,4,'0'),
+        date2=LPAD(NULLIF(REGEXP_REPLACE(NULLIF(date2, ''), E'\\D', '9', 'g')::INT,9999)::TEXT,4,'0');
+
+-- Change some ints to bigints:
+
+ALTER TABLE container.biblio_record_entry_bucket_item
+	ALTER COLUMN target_biblio_record_entry SET DATA TYPE bigint;
+
+ALTER TABLE vandelay.queued_bib_record
+	ALTER COLUMN imported_as SET DATA TYPE bigint;
+
+ALTER TABLE action.hold_copy_map
+	ALTER COLUMN id SET DATA TYPE bigint;
+
+-- Make due times get pushed to 23:59:59 on insert OR update
+DROP TRIGGER IF EXISTS push_due_date_tgr ON action.circulation;
+CREATE TRIGGER push_due_date_tgr BEFORE INSERT OR UPDATE ON action.circulation FOR EACH ROW EXECUTE PROCEDURE action.push_circ_due_time();
+
 COMMIT;
 
 -- Some operations go outside of the transaction, because they may
@@ -18046,39 +18664,51 @@ ADD COLUMN shelf_expire_time TIMESTAMPTZ;
 \qecho If any of these CREATE INDEX statements fails because the index already
 \qecho exists, ignore the failure.
 
-CREATE INDEX serial_subscription_record_idx ON serial.subscription (record_entry);
-CREATE INDEX serial_subscription_owner_idx ON serial.subscription (owning_lib);
-CREATE INDEX serial_caption_and_pattern_sub_idx ON serial.caption_and_pattern (subscription);
-CREATE INDEX serial_distribution_sub_idx ON serial.distribution (subscription);
-CREATE INDEX serial_distribution_holding_lib_idx ON serial.distribution (holding_lib);
-CREATE INDEX serial_distribution_note_dist_idx ON serial.distribution_note (distribution);
-CREATE INDEX serial_stream_dist_idx ON serial.stream (distribution);
-CREATE INDEX serial_routing_list_user_stream_idx ON serial.routing_list_user (stream);
-CREATE INDEX serial_routing_list_user_reader_idx ON serial.routing_list_user (reader);
-CREATE INDEX serial_issuance_sub_idx ON serial.issuance (subscription);
-CREATE INDEX serial_issuance_caption_and_pattern_idx ON serial.issuance (caption_and_pattern);
-CREATE INDEX serial_issuance_date_published_idx ON serial.issuance (date_published);
-CREATE UNIQUE INDEX unit_barcode_key ON serial.unit (barcode) WHERE deleted = FALSE OR deleted IS FALSE;
-CREATE INDEX unit_cn_idx ON serial.unit (call_number);
-CREATE INDEX unit_avail_cn_idx ON serial.unit (call_number);
-CREATE INDEX unit_creator_idx  ON serial.unit ( creator );
-CREATE INDEX unit_editor_idx   ON serial.unit ( editor );
-CREATE INDEX serial_item_stream_idx ON serial.item (stream);
-CREATE INDEX serial_item_issuance_idx ON serial.item (issuance);
-CREATE INDEX serial_item_unit_idx ON serial.item (unit);
-CREATE INDEX serial_item_uri_idx ON serial.item (uri);
-CREATE INDEX serial_item_date_received_idx ON serial.item (date_received);
-CREATE INDEX serial_item_status_idx ON serial.item (status);
-CREATE INDEX serial_item_note_item_idx ON serial.item_note (item);
-CREATE INDEX serial_basic_summary_dist_idx ON serial.basic_summary (distribution);
-CREATE INDEX serial_supplement_summary_dist_idx ON serial.supplement_summary (distribution);
-CREATE INDEX serial_index_summary_dist_idx ON serial.index_summary (distribution);
+CREATE INDEX acq_picklist_owner_idx   ON acq.picklist ( owner );
+CREATE INDEX acq_picklist_creator_idx ON acq.picklist ( creator );
+CREATE INDEX acq_picklist_editor_idx  ON acq.picklist ( editor );
+CREATE INDEX acq_po_note_creator_idx  ON acq.po_note ( creator );
+CREATE INDEX acq_po_note_editor_idx   ON acq.po_note ( editor );
+CREATE INDEX fund_alloc_allocator_idx ON acq.fund_allocation ( allocator );
+CREATE INDEX li_creator_idx   ON acq.lineitem ( creator );
+CREATE INDEX li_editor_idx    ON acq.lineitem ( editor );
+CREATE INDEX li_selector_idx  ON acq.lineitem ( selector );
+CREATE INDEX li_note_creator_idx  ON acq.lineitem_note ( creator );
+CREATE INDEX li_note_editor_idx   ON acq.lineitem_note ( editor );
+CREATE INDEX li_usr_attr_def_usr_idx  ON acq.lineitem_usr_attr_definition ( usr );
+CREATE INDEX po_editor_idx   ON acq.purchase_order ( editor );
+CREATE INDEX po_creator_idx  ON acq.purchase_order ( creator );
+CREATE INDEX acq_po_org_name_order_date_idx ON acq.purchase_order( ordering_agency, name, order_date );
+CREATE INDEX action_in_house_use_staff_idx  ON action.in_house_use ( staff );
+CREATE INDEX action_non_cat_circ_patron_idx ON action.non_cataloged_circulation ( patron );
+CREATE INDEX action_non_cat_circ_staff_idx  ON action.non_cataloged_circulation ( staff );
+CREATE INDEX action_survey_response_usr_idx ON action.survey_response ( usr );
+CREATE INDEX ahn_notify_staff_idx           ON action.hold_notification ( notify_staff );
+CREATE INDEX circ_all_usr_idx               ON action.circulation ( usr );
+CREATE INDEX circ_circ_staff_idx            ON action.circulation ( circ_staff );
+CREATE INDEX circ_checkin_staff_idx         ON action.circulation ( checkin_staff );
+CREATE INDEX hold_request_fulfillment_staff_idx ON action.hold_request ( fulfillment_staff );
+CREATE INDEX hold_request_requestor_idx     ON action.hold_request ( requestor );
+CREATE INDEX non_cat_in_house_use_staff_idx ON action.non_cat_in_house_use ( staff );
+CREATE INDEX actor_usr_note_creator_idx     ON actor.usr_note ( creator );
+CREATE INDEX actor_usr_standing_penalty_staff_idx ON actor.usr_standing_penalty ( staff );
+CREATE INDEX usr_org_unit_opt_in_staff_idx  ON actor.usr_org_unit_opt_in ( staff );
+CREATE INDEX asset_call_number_note_creator_idx ON asset.call_number_note ( creator );
+CREATE INDEX asset_copy_note_creator_idx    ON asset.copy_note ( creator );
+CREATE INDEX cp_creator_idx                 ON asset.copy ( creator );
+CREATE INDEX cp_editor_idx                  ON asset.copy ( editor );
 
 CREATE INDEX actor_card_barcode_lower_idx ON actor.card (lower(barcode));
 
-\qecho if the following CREATE INDEX fails, It will be necessary to do some
+DROP INDEX IF EXISTS authority.unique_by_heading_and_thesaurus;
+
+\qecho If the following CREATE INDEX fails, It will be necessary to do some
 \qecho data cleanup as described in the comments.
 
+CREATE UNIQUE INDEX unique_by_heading_and_thesaurus
+    ON authority.record_entry (authority.normalize_heading(marc))
+	WHERE deleted IS FALSE or deleted = FALSE;
+
 -- If the unique index fails, uncomment the following to create
 -- a regular index that will help find the duplicates in a hurry:
 --CREATE INDEX by_heading_and_thesaurus
diff --git a/Open-ILS/src/sql/Pg/110.hold_matrix.sql b/Open-ILS/src/sql/Pg/110.hold_matrix.sql
index a34d192008..08b253d635 100644
--- a/Open-ILS/src/sql/Pg/110.hold_matrix.sql
+++ b/Open-ILS/src/sql/Pg/110.hold_matrix.sql
@@ -31,6 +31,7 @@ BEGIN;
 CREATE TABLE config.hold_matrix_matchpoint (
     id                      SERIAL    PRIMARY KEY,
     active                  BOOL    NOT NULL DEFAULT TRUE,
+    strict_ou_match         BOOL    NOT NULL DEFAULT FALSE,
     user_home_ou            INT        REFERENCES actor.org_unit (id) DEFERRABLE INITIALLY DEFERRED,    -- Set to the top OU for the matchpoint applicability range; we can use org_unit_prox to choose the "best"
     request_ou              INT        REFERENCES actor.org_unit (id) DEFERRABLE INITIALLY DEFERRED,    -- Set to the top OU for the matchpoint applicability range; we can use org_unit_prox to choose the "best"
     pickup_ou               INT        REFERENCES actor.org_unit (id) DEFERRABLE INITIALLY DEFERRED,    -- Set to the top OU for the matchpoint applicability range; we can use org_unit_prox to choose the "best"
@@ -57,7 +58,6 @@ CREATE TABLE config.hold_matrix_matchpoint (
 CREATE OR REPLACE FUNCTION action.find_hold_matrix_matchpoint( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS INT AS $func$
 DECLARE
     current_requestor_group    permission.grp_tree%ROWTYPE;
-    root_ou            actor.org_unit%ROWTYPE;
     requestor_object    actor.usr%ROWTYPE;
     user_object        actor.usr%ROWTYPE;
     item_object        asset.copy%ROWTYPE;
@@ -69,7 +69,6 @@ DECLARE
     current_mp        config.hold_matrix_matchpoint%ROWTYPE;
     matchpoint        config.hold_matrix_matchpoint%ROWTYPE;
 BEGIN
-    SELECT INTO root_ou * FROM actor.org_unit WHERE parent_ou IS NULL;
     SELECT INTO user_object * FROM actor.usr WHERE id = match_user;
     SELECT INTO requestor_object * FROM actor.usr WHERE id = match_requestor;
     SELECT INTO item_object * FROM asset.copy WHERE id = match_item;
@@ -97,7 +96,11 @@ BEGIN
                     CASE WHEN m.marc_vr_format    IS NOT NULL THEN 2 ELSE 0 END +
                     CASE WHEN m.ref_flag        IS NOT NULL THEN 1 ELSE 0 END DESC LOOP
 
-            current_mp_weight := 5.0;
+            IF NOT current_mp.strict_ou_match THEN
+                current_mp_weight := 5.0;
+            ELSE
+                current_mp_weight := 0.0;
+            END IF;
 
             IF current_mp.circ_modifier IS NOT NULL THEN
                 CONTINUE WHEN current_mp.circ_modifier <> item_object.circ_modifier OR item_object.circ_modifier IS NULL;
@@ -129,28 +132,48 @@ BEGIN
 
 
             -- caclulate the rule match weight
-            IF current_mp.item_owning_ou IS NOT NULL AND current_mp.item_owning_ou <> root_ou.id THEN
-                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_owning_ou, item_cn_object.owning_lib)::FLOAT + 1.0)::FLOAT;
+            IF current_mp.item_owning_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_owning_ou, item_cn_object.owning_lib)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.item_owning_ou = item_cn_object.owning_lib THEN 1.0 ELSE 0.0 END;
+                END IF;
                 current_mp_weight := current_mp_weight - tmp_weight;
             END IF; 
 
-            IF current_mp.item_circ_ou IS NOT NULL AND current_mp.item_circ_ou <> root_ou.id THEN
-                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_circ_ou, item_object.circ_lib)::FLOAT + 1.0)::FLOAT;
+            IF current_mp.item_circ_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_circ_ou, item_object.circ_lib)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.item_circ_ou = item_object.circ_lib THEN 1.0 ELSE 0.0 END;
+                END IF;
                 current_mp_weight := current_mp_weight - tmp_weight;
             END IF; 
 
-            IF current_mp.pickup_ou IS NOT NULL AND current_mp.pickup_ou <> root_ou.id THEN
-                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.pickup_ou, pickup_ou)::FLOAT + 1.0)::FLOAT;
+            IF current_mp.pickup_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.pickup_ou, pickup_ou)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.pickup_ou = pickiup_ou THEN 1.0 ELSE 0.0 END;
+                END IF;
                 current_mp_weight := current_mp_weight - tmp_weight;
             END IF; 
 
-            IF current_mp.request_ou IS NOT NULL AND current_mp.request_ou <> root_ou.id THEN
-                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.request_ou, request_ou)::FLOAT + 1.0)::FLOAT;
+            IF current_mp.request_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.request_ou, request_ou)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.request_ou = request_ou THEN 1.0 ELSE 0.0 END;
+                END IF;
                 current_mp_weight := current_mp_weight - tmp_weight;
             END IF; 
 
-            IF current_mp.user_home_ou IS NOT NULL AND current_mp.user_home_ou <> root_ou.id THEN
-                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.user_home_ou, user_object.home_ou)::FLOAT + 1.0)::FLOAT;
+            IF current_mp.user_home_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.user_home_ou, user_object.home_ou)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.user_home_ou = user_object.home_ou THEN 1.0 ELSE 0.0 END;
+                END IF;
                 current_mp_weight := current_mp_weight - tmp_weight;
             END IF; 
 
@@ -172,7 +195,7 @@ END;
 $func$ LANGUAGE plpgsql;
 
 
-CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT, retargetting BOOL ) RETURNS SETOF action.matrix_test_result AS $func$
 DECLARE
     matchpoint_id        INT;
     user_object        actor.usr%ROWTYPE;
@@ -306,7 +329,7 @@ BEGIN
         END LOOP;
     END IF;
 
-    IF hold_test.max_holds IS NOT NULL THEN
+    IF hold_test.max_holds IS NOT NULL AND NOT retargetting THEN
         SELECT    INTO hold_count COUNT(*)
           FROM    action.hold_request
           WHERE    usr = match_user
@@ -349,5 +372,13 @@ BEGIN
 END;
 $func$ LANGUAGE plpgsql;
 
+CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+    SELECT * FROM action.hold_request_permit_test( $1, $2, $3, $4, $5, FALSE );
+$func$ LANGUAGE SQL;
+
+CREATE OR REPLACE FUNCTION action.hold_retarget_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+    SELECT * FROM action.hold_request_permit_test( $1, $2, $3, $4, $5, TRUE );
+$func$ LANGUAGE SQL;
+
 COMMIT;
 
diff --git a/Open-ILS/src/sql/Pg/200.schema.acq.sql b/Open-ILS/src/sql/Pg/200.schema.acq.sql
index 13db4da102..0db32d72b7 100644
--- a/Open-ILS/src/sql/Pg/200.schema.acq.sql
+++ b/Open-ILS/src/sql/Pg/200.schema.acq.sql
@@ -482,7 +482,7 @@ CREATE TABLE acq.lineitem (
 	create_time         TIMESTAMP WITH TIME ZONE    NOT NULL DEFAULT NOW(),
 	edit_time           TIMESTAMP WITH TIME ZONE    NOT NULL DEFAULT NOW(),
 	marc                TEXT                        NOT NULL,
-	eg_bib_id           INT                         REFERENCES biblio.record_entry (id) DEFERRABLE INITIALLY DEFERRED,
+	eg_bib_id           BIGINT                      REFERENCES biblio.record_entry (id) DEFERRABLE INITIALLY DEFERRED,
 	source_label        TEXT,
 	state               TEXT                        NOT NULL DEFAULT 'new',
 	cancel_reason       INT                         REFERENCES acq.cancel_reason( id )
diff --git a/Open-ILS/src/sql/Pg/950.data.seed-values.sql b/Open-ILS/src/sql/Pg/950.data.seed-values.sql
index ff139a2217..dee49b8a2e 100644
--- a/Open-ILS/src/sql/Pg/950.data.seed-values.sql
+++ b/Open-ILS/src/sql/Pg/950.data.seed-values.sql
@@ -1390,6 +1390,15 @@ INSERT INTO permission.perm_list VALUES
     ,(399, 'ADMIN_SERIAL_DISTRIBUTION', oils_i18n_gettext(399, 'Create/update/delete serial distribution objects', 'ppl', 'description'))
     ,(400, 'ADMIN_SERIAL_STREAM', oils_i18n_gettext(400, 'Create/update/delete serial stream objects', 'ppl', 'description'))
     ,(401, 'RECEIVE_SERIAL', oils_i18n_gettext(401, 'Receive serial items', 'ppl', 'description'))
+    ,(402, 'ADMIN_ACQ_DISTRIB_FORMULA', oils_i18n_gettext(402, 'Create/update/delete distribution formulae', 'ppl', 'description'))
+    ,(403, 'ADMIN_ACQ_CLAIM', oils_i18n_gettext(403, 'Create/update/delete acquisitions claims', 'ppl', 'description'))
+    ,(404, 'ADMIN_ACQ_CLAIM_EVENT_TYPE', oils_i18n_gettext(404, 'Create/update/delete acquisitions claim event types', 'ppl', 'description'))
+    ,(405, 'ADMIN_ACQ_CLAIM_TYPE', oils_i18n_gettext(405, 'Create/update/delete acquisitions claim types', 'ppl', 'description'))
+    ,(406, 'ADMIN_ACQ_FISCAL_YEAR', oils_i18n_gettext(406, 'Create/update/delete acquisitions fiscal years', 'ppl', 'description'))
+    ,(407, 'ADMIN_ACQ_FUND_ALLOCATION_PERCENT', oils_i18n_gettext(407, 'Create/update/delete acquisitions fund allocation percentages', 'ppl', 'description'))
+    ,(408, 'ADMIN_ACQ_FUND_TAG', oils_i18n_gettext(408, 'Create/update/delete acquisitions fund tags', 'ppl', 'description'))
+    ,(409, 'ADMIN_ACQ_LINEITEM_ALERT_TEXT', oils_i18n_gettext(409, 'Create/update/delete acquisitions lineitem alert text', 'ppl', 'description'))
+
 ;
 
 
@@ -2336,6 +2345,7 @@ INSERT INTO container.biblio_record_entry_bucket_type (code,label) VALUES ('misc
 INSERT INTO container.biblio_record_entry_bucket_type (code,label) VALUES ('staff_client', oils_i18n_gettext('staff_client', 'General Staff Client container', 'cbrebt', 'label'));
 INSERT INTO container.biblio_record_entry_bucket_type (code,label) VALUES ('bookbag', oils_i18n_gettext('bookbag', 'Book Bag', 'cbrebt', 'label'));
 INSERT INTO container.biblio_record_entry_bucket_type (code,label) VALUES ('reading_list', oils_i18n_gettext('reading_list', 'Reading List', 'cbrebt', 'label'));
+INSERT INTO container.biblio_record_entry_bucket_type (code,label) VALUES ('template_merge',oils_i18n_gettext('template_merge','Template Merge Container', 'cbrebt', 'label'));
 
 INSERT INTO container.user_bucket_type (code,label) VALUES ('misc', oils_i18n_gettext('misc', 'Miscellaneous', 'cubt', 'label'));
 INSERT INTO container.user_bucket_type (code,label) VALUES ('folks', oils_i18n_gettext('folks', 'Friends', 'cubt', 'label'));
@@ -4872,8 +4882,8 @@ $$
         
         [% FOREACH detail IN li.lineitem_details.sort('owning_lib') %]
             [% 
-                IF copy.eg_copy_id;
-                    SET copy = copy.eg_copy_id;
+                IF detail.eg_copy_id;
+                    SET copy = detail.eg_copy_id;
                     SET cn_label = copy.call_number.label;
                 ELSE; 
                     SET copy = detail; 
@@ -4940,9 +4950,19 @@ INSERT INTO action_trigger.reactor (module, description)
 
 
 INSERT INTO action_trigger.event_definition (id, active, owner, name, hook, validator, reactor, cleanup_success, cleanup_failure, delay, delay_field, group_field, template) 
-    VALUES (23, true, 1, 'PO JEDI', 'acqpo.activated', 'Acq::PurchaseOrderEDIRequired', 'GeneratePurchaseOrderJEDI', NULL, NULL, '00:05:00', NULL, NULL,
+    VALUES (23, true, 1, 'PO JEDI', 'acqpo.activated', 'Acq::PurchaseOrderEDIRequired', 'GeneratePurchaseOrderJEDI', NULL, NULL, '00:00:00', NULL, NULL,
 $$[%- USE date -%]
-[%# start JEDI document -%]
+[%# start JEDI document 
+  # Vendor specific kludges:
+  # BT      - vendcode goes to NAD/BY *suffix*  w/ 91 qualifier
+  # INGRAM  - vendcode goes to NAD/BY *segment* w/ 91 qualifier (separately)
+  # BRODART - vendcode goes to FTX segment (lineitem level)
+-%]
+[%- 
+IF target.provider.edi_default.vendcode && target.provider.code == 'BRODART';
+    xtra_ftx = target.provider.edi_default.vendcode;
+END;
+-%]
 [%- BLOCK big_block -%]
 {
    "recipient":"[% target.provider.san %]",
@@ -4951,23 +4971,27 @@ $$[%- USE date -%]
      "ORDERS":[ "order", {
         "po_number":[% target.id %],
         "date":"[% date.format(date.now, '%Y%m%d') %]",
-        "buyer":[{
-            [%- IF target.provider.edi_default.vendcode -%]
-                "id":"[% target.ordering_agency.mailing_address.san _ ' ' _ target.provider.edi_default.vendcode %]", 
-                "id-qualifier": 91
+        "buyer":[
+            [%   IF   target.provider.edi_default.vendcode && (target.provider.code == 'BT' || target.provider.name.match('(?i)^BAKER & TAYLOR'))  -%]
+                {"id-qualifier": 91, "id":"[% target.ordering_agency.mailing_address.san _ ' ' _ target.provider.edi_default.vendcode %]"}
+            [%- ELSIF target.provider.edi_default.vendcode && target.provider.code == 'INGRAM' -%]
+                {"id":"[% target.ordering_agency.mailing_address.san %]"},
+                {"id-qualifier": 91, "id":"[% target.provider.edi_default.vendcode %]"}
             [%- ELSE -%]
-                "id":"[% target.ordering_agency.mailing_address.san %]"
-            [%- END  -%]
-        }],
-        "vendor":[ 
+                {"id":"[% target.ordering_agency.mailing_address.san %]"}
+            [%- END -%]
+        ],
+        "vendor":[
             [%- # target.provider.name (target.provider.id) -%]
             "[% target.provider.san %]",
             {"id-qualifier": 92, "id":"[% target.provider.id %]"}
         ],
         "currency":"[% target.provider.currency_type %]",
+                
         "items":[
-        [% FOR li IN target.lineitems %]
+        [%- FOR li IN target.lineitems %]
         {
+            "line_index":"[% li.id %]",
             "identifiers":[   [%-# li.isbns = helpers.get_li_isbns(li.attributes) %]
             [% FOR isbn IN helpers.get_li_isbns(li.attributes) -%]
                 [% IF isbn.length == 13 -%]
@@ -4976,23 +5000,35 @@ $$[%- USE date -%]
                 {"id-qualifier":"IB","id":"[% isbn %]"},
                 [%- END %]
             [% END %]
-                {"id-qualifier":"SA","id":"[% li.id %]"}
+                {"id-qualifier":"IN","id":"[% li.id %]"}
             ],
             "price":[% li.estimated_unit_price || '0.00' %],
             "desc":[
-                {"BTI":"[% helpers.get_li_attr('title',     '', li.attributes) %]"}, 
+                {"BTI":"[% helpers.get_li_attr('title',     '', li.attributes) %]"},
                 {"BPU":"[% helpers.get_li_attr('publisher', '', li.attributes) %]"},
                 {"BPD":"[% helpers.get_li_attr('pubdate',   '', li.attributes) %]"},
                 {"BPH":"[% helpers.get_li_attr('pagination','', li.attributes) %]"}
             ],
+            [%- ftx_vals = []; 
+                FOR note IN li.lineitem_notes; 
+                    NEXT UNLESS note.vendor_public == 't'; 
+                    ftx_vals.push(note.value); 
+                END; 
+                IF xtra_ftx;           ftx_vals.unshift(xtra_ftx); END; 
+                IF ftx_vals.size == 0; ftx_vals.unshift('');       END;  # BT needs FTX+LIN for every LI, even if it is an empty one
+            -%]
+
+            "free-text":[ 
+                [% FOR note IN ftx_vals -%] "[% note %]"[% UNLESS loop.last %], [% END %][% END %] 
+            ],            
             "quantity":[% li.lineitem_details.size %]
         }[% UNLESS loop.last %],[% END %]
         [%-# TODO: lineitem details (later) -%]
         [% END %]
         ],
         "line_items":[% target.lineitems.size %]
-     }]  [% # close ORDERS array %]
-   }]    [% # close  body  array %]
+     }]  [%# close ORDERS array %]
+   }]    [%# close  body  array %]
 }
 [% END %]
 [% tempo = PROCESS big_block; helpers.escape_json(tempo) %]
diff --git a/Open-ILS/src/sql/Pg/reporter-schema.sql b/Open-ILS/src/sql/Pg/reporter-schema.sql
index 4492d6c464..a4fa0383a7 100644
--- a/Open-ILS/src/sql/Pg/reporter-schema.sql
+++ b/Open-ILS/src/sql/Pg/reporter-schema.sql
@@ -305,9 +305,11 @@ SELECT	id,
 	CASE
 		WHEN hold_type = 'T'
 			THEN target
+		WHEN hold_type = 'I'
+			THEN (SELECT ssub.record_entry FROM serial.subscription ssub JOIN serial.issuance si ON (si.subscription = ssub.id) WHERE si.id = ahr.target)
 		WHEN hold_type = 'V'
 			THEN (SELECT cn.record FROM asset.call_number cn WHERE cn.id = ahr.target)
-		WHEN hold_type = 'C'
+		WHEN hold_type IN ('C','R','F')
 			THEN (SELECT cn.record FROM asset.call_number cn JOIN asset.copy cp ON (cn.id = cp.call_number) WHERE cp.id = ahr.target)
 		WHEN hold_type = 'M'
 			THEN (SELECT mr.master_record FROM metabib.metarecord mr WHERE mr.id = ahr.target)
diff --git a/Open-ILS/src/sql/Pg/upgrade/0414.schema.call-number-upd-ins-trigger.sql b/Open-ILS/src/sql/Pg/upgrade/0414.schema.call-number-upd-ins-trigger.sql
new file mode 100644
index 0000000000..11498988bd
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0414.schema.call-number-upd-ins-trigger.sql
@@ -0,0 +1,13 @@
+BEGIN;
+
+-- Adding a trigger.  Upgrade # 0364 created the trigger function but not
+-- the trigger itself.  However the base install script 040.schema.asset.sql
+-- creates both the function and the trigger.
+
+INSERT INTO config.upgrade_log (version) VALUES ('0414'); -- Scott McKellar
+
+CREATE TRIGGER asset_label_sortkey_trigger
+    BEFORE UPDATE OR INSERT ON asset.call_number
+    FOR EACH ROW EXECUTE PROCEDURE asset.label_normalizer();
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0415.schema.rename-field-class-fkey.sql b/Open-ILS/src/sql/Pg/upgrade/0415.schema.rename-field-class-fkey.sql
new file mode 100644
index 0000000000..81ceaaf41c
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0415.schema.rename-field-class-fkey.sql
@@ -0,0 +1,22 @@
+-- Dropping and recreating a foreign key constraint for config.metabib_field,
+-- in order to change its name.  WHen this foreign key was first introduced,
+-- the upgrade script gave it one name and the base install script gave it
+-- a different name.  Here we bring the names into sync.
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0415'); -- Scott McKellar
+
+\qecho Dropping and recreating a foreign key in order to change its name.
+\qecho If the DROP fails because the constraint doesn't exist under the old
+\qecho name, or the ADD fails because it already exists under the new name,
+\qecho then ignore the failure.
+
+ALTER TABLE config.metabib_field
+	DROP CONSTRAINT field_class_fkey;
+
+ALTER TABLE config.metabib_field
+	ADD CONSTRAINT metabib_field_field_class_fkey
+	FOREIGN KEY (field_class) REFERENCES config.metabib_class(name);
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0416.schema.rename-recuring-idx.sql b/Open-ILS/src/sql/Pg/upgrade/0416.schema.rename-recuring-idx.sql
new file mode 100644
index 0000000000..dde360e07b
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0416.schema.rename-recuring-idx.sql
@@ -0,0 +1,11 @@
+INSERT INTO config.upgrade_log (version) VALUES ('0416'); -- Scott McKellar
+
+\qecho No transaction.  Renaming two indexes to correct spelling.
+\qecho If either change fails, then the index was probably created
+\qecho correctly in the first place; ignore the failure.
+
+ALTER INDEX config.rule_recuring_fine_name_key
+	RENAME TO rule_recurring_fine_name_key;
+
+ALTER INDEX config.rule_recuring_fine_pkey
+	RENAME TO rule_recurring_fine_pkey;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0417.schema.acq.drop-lineitem-item-count.sql b/Open-ILS/src/sql/Pg/upgrade/0417.schema.acq.drop-lineitem-item-count.sql
new file mode 100644
index 0000000000..aadf5fafa4
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0417.schema.acq.drop-lineitem-item-count.sql
@@ -0,0 +1,23 @@
+-- Drop the never-used column item_count from acq.lineitem.
+-- Drop it also from the associated history table, and rebuild
+-- the function that maintains it.  Finally, rebuild the
+-- associated lifecycle view.  
+
+-- Apply to trunk only; this column never existed in 2.0.
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0417'); -- Scott McKellar
+
+-- Have to drop the view first, because it's a dependent
+DROP VIEW IF EXISTS acq.acq_lineitem_lifecycle;
+
+ALTER TABLE acq.lineitem DROP COLUMN item_count;
+
+ALTER TABLE acq.acq_lineitem_history DROP COLUMN item_count;
+
+SELECT acq.create_acq_func( 'acq', 'lineitem' );
+
+SELECT acq.create_acq_lifecycle( 'acq', 'lineitem' );
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0418.function.action.fix-purge-circ.sql b/Open-ILS/src/sql/Pg/upgrade/0418.function.action.fix-purge-circ.sql
new file mode 100644
index 0000000000..6cf5374ee5
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0418.function.action.fix-purge-circ.sql
@@ -0,0 +1,99 @@
+-- Apply in an update script some fixes that were previously applied only
+-- to the base installation script 090.schema.action.sql.
+
+-- Also fix a typo: INTEVAL -> INTERVAL
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0418'); -- Scott McKellar
+
+CREATE OR REPLACE FUNCTION action.purge_circulations () RETURNS INT AS $func$
+DECLARE
+    usr_keep_age    actor.usr_setting%ROWTYPE;
+    usr_keep_start  actor.usr_setting%ROWTYPE;
+    org_keep_age    INTERVAL;
+    org_keep_count  INT;
+
+    keep_age        INTERVAL;
+
+    target_acp      RECORD;
+    circ_chain_head action.circulation%ROWTYPE;
+    circ_chain_tail action.circulation%ROWTYPE;
+
+    purge_position  INT;
+    count_purged    INT;
+BEGIN
+
+    count_purged := 0;
+
+    SELECT value::INTERVAL INTO org_keep_age FROM config.global_flag WHERE name = 'history.circ.retention_age' AND enabled;
+
+    SELECT value::INT INTO org_keep_count FROM config.global_flag WHERE name = 'history.circ.retention_count' AND enabled;
+    IF org_keep_count IS NULL THEN
+        RETURN count_purged; -- Gimme a count to keep, or I keep them all, forever
+    END IF;
+
+    -- First, find copies with more than keep_count non-renewal circs
+    FOR target_acp IN
+        SELECT  target_copy,
+                COUNT(*) AS total_real_circs
+          FROM  action.circulation
+          WHERE parent_circ IS NULL
+                AND xact_finish IS NOT NULL
+          GROUP BY target_copy
+          HAVING COUNT(*) > org_keep_count
+    LOOP
+        purge_position := 0;
+        -- And, for those, select circs that are finished and older than keep_age
+        FOR circ_chain_head IN
+            SELECT  *
+              FROM  action.circulation
+              WHERE target_copy = target_acp.target_copy
+                    AND parent_circ IS NULL
+              ORDER BY xact_start
+        LOOP
+
+            -- Stop once we've purged enough circs to hit org_keep_count
+            EXIT WHEN target_acp.total_real_circs - purge_position <= org_keep_count;
+
+            SELECT * INTO circ_chain_tail FROM action.circ_chain(circ_chain_head.id) ORDER BY xact_start DESC LIMIT 1;
+            EXIT WHEN circ_chain_tail.xact_finish IS NULL;
+
+            -- Now get the user settings, if any, to block purging if the user wants to keep more circs
+            usr_keep_age.value := NULL;
+            SELECT * INTO usr_keep_age FROM actor.usr_setting WHERE usr = circ_chain_head.usr AND name = 'history.circ.retention_age';
+
+            usr_keep_start.value := NULL;
+            SELECT * INTO usr_keep_start FROM actor.usr_setting WHERE usr = circ_chain_head.usr AND name = 'history.circ.retention_start';
+
+            IF usr_keep_age.value IS NOT NULL AND usr_keep_start.value IS NOT NULL THEN
+                IF oils_json_to_text(usr_keep_age.value)::INTERVAL > AGE(NOW(), oils_json_to_text(usr_keep_start.value)::TIMESTAMPTZ) THEN
+                    keep_age := AGE(NOW(), oils_json_to_text(usr_keep_start.value)::TIMESTAMPTZ);
+                ELSE
+                    keep_age := oils_json_to_text(usr_keep_age.value)::INTERVAL;
+                END IF;
+            ELSIF usr_keep_start.value IS NOT NULL THEN
+                keep_age := AGE(NOW(), oils_json_to_text(usr_keep_start.value)::TIMESTAMPTZ);
+            ELSE
+                keep_age := COALESCE( org_keep_age::INTERVAL, '2000 years'::INTERVAL );
+            END IF;
+
+            EXIT WHEN AGE(NOW(), circ_chain_tail.xact_finish) < keep_age;
+
+            -- We've passed the purging tests, purge the circ chain starting at the end
+            DELETE FROM action.circulation WHERE id = circ_chain_tail.id;
+            WHILE circ_chain_tail.parent_circ IS NOT NULL LOOP
+                SELECT * INTO circ_chain_tail FROM action.circulation WHERE id = circ_chain_tail.parent_circ;
+                DELETE FROM action.circulation WHERE id = circ_chain_tail.id;
+            END LOOP;
+
+            count_purged := count_purged + 1;
+            purge_position := purge_position + 1;
+
+        END LOOP;
+    END LOOP;
+END;
+$func$ LANGUAGE PLPGSQL;
+
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0419.schema.hold_record_view.sql b/Open-ILS/src/sql/Pg/upgrade/0419.schema.hold_record_view.sql
new file mode 100644
index 0000000000..350dc78100
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0419.schema.hold_record_view.sql
@@ -0,0 +1,23 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0419'); -- miker
+
+CREATE OR REPLACE VIEW reporter.hold_request_record AS
+SELECT  id,
+    target,
+    hold_type,
+    CASE
+        WHEN hold_type = 'T'
+            THEN target
+        WHEN hold_type = 'I'
+            THEN (SELECT ssub.record_entry FROM serial.subscription ssub JOIN serial.issuance si ON (si.subscription = ssub.id) WHERE si.id = ahr.target)
+        WHEN hold_type = 'V'
+            THEN (SELECT cn.record FROM asset.call_number cn WHERE cn.id = ahr.target)
+        WHEN hold_type IN ('C','R','F')
+            THEN (SELECT cn.record FROM asset.call_number cn JOIN asset.copy cp ON (cn.id = cp.call_number) WHERE cp.id = ahr.target)
+        WHEN hold_type = 'M'
+            THEN (SELECT mr.master_record FROM metabib.metarecord mr WHERE mr.id = ahr.target)
+    END AS bib_record
+  FROM  action.hold_request ahr;
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0420.schema.premunge_dates.sql b/Open-ILS/src/sql/Pg/upgrade/0420.schema.premunge_dates.sql
new file mode 100644
index 0000000000..9914950e5d
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0420.schema.premunge_dates.sql
@@ -0,0 +1,40 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0420'); -- miker
+
+CREATE OR REPLACE FUNCTION metabib.reingest_metabib_rec_descriptor( bib_id BIGINT ) RETURNS VOID AS $func$
+BEGIN
+    PERFORM * FROM config.internal_flag WHERE name = 'ingest.assume_inserts_only' AND enabled;
+    IF NOT FOUND THEN
+        DELETE FROM metabib.rec_descriptor WHERE record = bib_id;
+    END IF;
+    INSERT INTO metabib.rec_descriptor (record, item_type, item_form, bib_level, control_type, enc_level, audience, lit_form, type_mat, cat_form, pub_status, item_lang, vr_format, date1, date2)
+        SELECT  bib_id,
+                biblio.marc21_extract_fixed_field( bib_id, 'Type' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'Form' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'BLvl' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'Ctrl' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'ELvl' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'Audn' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'LitF' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'TMat' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'Desc' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'DtSt' ),
+                biblio.marc21_extract_fixed_field( bib_id, 'Lang' ),
+                (   SELECT  v.value
+                      FROM  biblio.marc21_physical_characteristics( bib_id) p
+                            JOIN config.marc21_physical_characteristic_subfield_map s ON (s.id = p.subfield)
+                            JOIN config.marc21_physical_characteristic_value_map v ON (v.id = p.value)
+                      WHERE p.ptype = 'v' AND s.subfield = 'e'    ),
+                LPAD(NULLIF(REGEXP_REPLACE(NULLIF(biblio.marc21_extract_fixed_field( bib_id, 'Date1'), ''), E'\\D', '0', 'g')::INT,0)::TEXT,4,'0'),
+                LPAD(NULLIF(REGEXP_REPLACE(NULLIF(biblio.marc21_extract_fixed_field( bib_id, 'Date2'), ''), E'\\D', '9', 'g')::INT,9999)::TEXT,4,'0');
+
+    RETURN;
+END;
+$func$ LANGUAGE PLPGSQL;
+
+UPDATE  metabib.rec_descriptor
+  SET   date1=LPAD(NULLIF(REGEXP_REPLACE(NULLIF(date1, ''), E'\\D', '0', 'g')::INT,0)::TEXT,4,'0'),
+        date2=LPAD(NULLIF(REGEXP_REPLACE(NULLIF(date2, ''), E'\\D', '9', 'g')::INT,9999)::TEXT,4,'0');
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0421.schema.embiggen-ints.sql b/Open-ILS/src/sql/Pg/upgrade/0421.schema.embiggen-ints.sql
new file mode 100644
index 0000000000..092ccc5278
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0421.schema.embiggen-ints.sql
@@ -0,0 +1,44 @@
+-- 1. Turn some ints into bigints.
+
+-- 2. Rename a constraint for consistency and accuracy (currently it may
+-- have either of two different names).
+
+\qecho One of the following DROPs will fail, so we do them
+\qecho both outside of a transaction.  Ignore the failure.
+
+ALTER TABLE booking.resource_type
+	DROP CONSTRAINT brt_name_or_record_once_per_owner;
+
+ALTER TABLE booking.resource_type
+	DROP CONSTRAINT brt_name_once_per_owner;
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0421'); -- Scott McKellar
+
+ALTER TABLE booking.resource_type
+	ALTER COLUMN record SET DATA TYPE bigint,
+	ADD CONSTRAINT brt_name_and_record_once_per_owner UNIQUE(owner, name, record);
+
+ALTER TABLE container.biblio_record_entry_bucket_item
+	ALTER COLUMN target_biblio_record_entry SET DATA TYPE bigint;
+
+-- Before we can embiggen the next one, we must drop a view
+-- that depends on it (and recreate it later)
+
+DROP VIEW IF EXISTS acq.acq_lineitem_lifecycle;
+
+ALTER TABLE acq.lineitem
+	ALTER COLUMN eg_bib_id SET DATA TYPE bigint;
+
+-- Recreate the view
+
+SELECT acq.create_acq_lifecycle( 'acq', 'lineitem' );
+
+ALTER TABLE vandelay.queued_bib_record
+	ALTER COLUMN imported_as SET DATA TYPE bigint;
+
+ALTER TABLE action.hold_copy_map
+	ALTER COLUMN id SET DATA TYPE bigint;
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0422.schema.acq.lineitem-history-bigint.sql b/Open-ILS/src/sql/Pg/upgrade/0422.schema.acq.lineitem-history-bigint.sql
new file mode 100644
index 0000000000..4b9bc1776a
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0422.schema.acq.lineitem-history-bigint.sql
@@ -0,0 +1,15 @@
+BEGIN;
+
+-- Turn an int into a bigint in acq.acq_lineitem_history, following up on
+-- a similar change to acq.lineitem
+
+INSERT INTO config.upgrade_log (version) VALUES ('0422'); -- Scott McKellar
+
+DROP VIEW IF EXISTS acq.acq_lineitem_lifecycle;
+
+ALTER TABLE acq.acq_lineitem_history
+	ALTER COLUMN eg_bib_id SET DATA TYPE bigint;
+
+SELECT acq.create_acq_lifecycle( 'acq', 'lineitem' );
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0423.schema.support-null-function-in-xpath_table.sql b/Open-ILS/src/sql/Pg/upgrade/0423.schema.support-null-function-in-xpath_table.sql
new file mode 100644
index 0000000000..1d90b44dc9
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0423.schema.support-null-function-in-xpath_table.sql
@@ -0,0 +1,67 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0423'); --miker
+
+CREATE OR REPLACE FUNCTION oils_xpath_table ( key TEXT, document_field TEXT, relation_name TEXT, xpaths TEXT, criteria TEXT ) RETURNS SETOF RECORD AS $func$
+DECLARE
+    xpath_list  TEXT[];
+    select_list TEXT[];
+    where_list  TEXT[];
+    q           TEXT;
+    out_record  RECORD;
+    empty_test  RECORD;
+BEGIN
+    xpath_list := STRING_TO_ARRAY( xpaths, '|' );
+ 
+    select_list := ARRAY_APPEND( select_list, key || '::INT AS key' );
+ 
+    FOR i IN 1 .. ARRAY_UPPER(xpath_list,1) LOOP
+        IF xpath_list[i] = 'null()' THEN
+            select_list := ARRAY_APPEND( select_list, 'NULL::TEXT AS c_' || i );
+        ELSE
+            select_list := ARRAY_APPEND(
+                select_list,
+                $sel$
+                EXPLODE_ARRAY(
+                    COALESCE(
+                        NULLIF(
+                            oils_xpath(
+                                $sel$ ||
+                                    quote_literal(
+                                        CASE
+                                            WHEN xpath_list[i] ~ $re$/[^/[]*@[^/]+$$re$ OR xpath_list[i] ~ $re$text\(\)$$re$ THEN xpath_list[i]
+                                            ELSE xpath_list[i] || '//text()'
+                                        END
+                                    ) ||
+                                $sel$,
+                                $sel$ || document_field || $sel$
+                            ),
+                           '{}'::TEXT[]
+                        ),
+                        '{NULL}'::TEXT[]
+                    )
+                ) AS c_$sel$ || i
+            );
+            where_list := ARRAY_APPEND(
+                where_list,
+                'c_' || i || ' IS NOT NULL'
+            );
+        END IF;
+    END LOOP;
+ 
+    q := $q$
+SELECT * FROM (
+    SELECT $q$ || ARRAY_TO_STRING( select_list, ', ' ) || $q$ FROM $q$ || relation_name || $q$ WHERE ($q$ || criteria || $q$)
+)x WHERE $q$ || ARRAY_TO_STRING( where_list, ' AND ' );
+    -- RAISE NOTICE 'query: %', q;
+ 
+    FOR out_record IN EXECUTE q LOOP
+        RETURN NEXT out_record;
+    END LOOP;
+ 
+    RETURN;
+END;
+$func$ LANGUAGE PLPGSQL IMMUTABLE;
+
+COMMIT;
+
diff --git a/Open-ILS/src/sql/Pg/upgrade/0424.schema.circ_due_date_trigger.sql b/Open-ILS/src/sql/Pg/upgrade/0424.schema.circ_due_date_trigger.sql
new file mode 100644
index 0000000000..6d2a8a095a
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0424.schema.circ_due_date_trigger.sql
@@ -0,0 +1,9 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0424'); -- dbs
+
+DROP TRIGGER push_due_date_tgr ON action.circulation;
+
+CREATE TRIGGER push_due_date_tgr BEFORE INSERT OR UPDATE ON action.circulation FOR EACH ROW EXECUTE PROCEDURE action.push_circ_due_time();
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0425.schema.perm-grp-tree-hold-priority.sql b/Open-ILS/src/sql/Pg/upgrade/0425.schema.perm-grp-tree-hold-priority.sql
new file mode 100644
index 0000000000..e556b0a336
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0425.schema.perm-grp-tree-hold-priority.sql
@@ -0,0 +1,8 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0425'); -- Scott McKellar
+
+ALTER TABLE permission.grp_tree
+	ADD COLUMN hold_priority INT NOT NULL DEFAULT 0;
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0426.data.perm-list.misc-acq.sql b/Open-ILS/src/sql/Pg/upgrade/0426.data.perm-list.misc-acq.sql
new file mode 100644
index 0000000000..b9ba2f1d60
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0426.data.perm-list.misc-acq.sql
@@ -0,0 +1,16 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0426'); -- senator
+
+INSERT INTO permission.perm_list VALUES
+    (402, 'ADMIN_ACQ_DISTRIB_FORMULA', oils_i18n_gettext(402, 'Create/update/delete distribution formulae', 'ppl', 'description'))
+    ,(403, 'ADMIN_ACQ_CLAIM', oils_i18n_gettext(403, 'Create/update/delete acquisitions claims', 'ppl', 'description'))
+    ,(404, 'ADMIN_ACQ_CLAIM_EVENT_TYPE', oils_i18n_gettext(404, 'Create/update/delete acquisitions claim event types', 'ppl', 'description'))
+    ,(405, 'ADMIN_ACQ_CLAIM_TYPE', oils_i18n_gettext(405, 'Create/update/delete acquisitions claim types', 'ppl', 'description'))
+    ,(406, 'ADMIN_ACQ_FISCAL_YEAR', oils_i18n_gettext(406, 'Create/update/delete acquisitions fiscal years', 'ppl', 'description'))
+    ,(407, 'ADMIN_ACQ_FUND_ALLOCATION_PERCENT', oils_i18n_gettext(407, 'Create/update/delete acquisitions fund allocation percentages', 'ppl', 'description'))
+    ,(408, 'ADMIN_ACQ_FUND_TAG', oils_i18n_gettext(408, 'Create/update/delete acquisitions fund tags', 'ppl', 'description'))
+    ,(409, 'ADMIN_ACQ_LINEITEM_ALERT_TEXT', oils_i18n_gettext(409, 'Create/update/delete acquisitions lineitem alert text', 'ppl', 'description'))
+;
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0427.schema.hold_matrix_root_ou.sql b/Open-ILS/src/sql/Pg/upgrade/0427.schema.hold_matrix_root_ou.sql
new file mode 100644
index 0000000000..822c0caba5
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0427.schema.hold_matrix_root_ou.sql
@@ -0,0 +1,120 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0427'); -- gmc
+
+CREATE OR REPLACE FUNCTION action.find_hold_matrix_matchpoint( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS INT AS $func$
+DECLARE
+    current_requestor_group    permission.grp_tree%ROWTYPE;
+    requestor_object    actor.usr%ROWTYPE;
+    user_object        actor.usr%ROWTYPE;
+    item_object        asset.copy%ROWTYPE;
+    item_cn_object        asset.call_number%ROWTYPE;
+    rec_descriptor        metabib.rec_descriptor%ROWTYPE;
+    current_mp_weight    FLOAT;
+    matchpoint_weight    FLOAT;
+    tmp_weight        FLOAT;
+    current_mp        config.hold_matrix_matchpoint%ROWTYPE;
+    matchpoint        config.hold_matrix_matchpoint%ROWTYPE;
+BEGIN
+    SELECT INTO user_object * FROM actor.usr WHERE id = match_user;
+    SELECT INTO requestor_object * FROM actor.usr WHERE id = match_requestor;
+    SELECT INTO item_object * FROM asset.copy WHERE id = match_item;
+    SELECT INTO item_cn_object * FROM asset.call_number WHERE id = item_object.call_number;
+    SELECT INTO rec_descriptor r.* FROM metabib.rec_descriptor r WHERE r.record = item_cn_object.record;
+
+    PERFORM * FROM config.internal_flag WHERE name = 'circ.holds.usr_not_requestor' AND enabled;
+
+    IF NOT FOUND THEN
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = requestor_object.profile;
+    ELSE
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = user_object.profile;
+    END IF;
+
+    LOOP 
+        -- for each potential matchpoint for this ou and group ...
+        FOR current_mp IN
+            SELECT    m.*
+              FROM    config.hold_matrix_matchpoint m
+              WHERE    m.requestor_grp = current_requestor_group.id AND m.active
+              ORDER BY    CASE WHEN m.circ_modifier    IS NOT NULL THEN 16 ELSE 0 END +
+                    CASE WHEN m.juvenile_flag    IS NOT NULL THEN 16 ELSE 0 END +
+                    CASE WHEN m.marc_type        IS NOT NULL THEN 8 ELSE 0 END +
+                    CASE WHEN m.marc_form        IS NOT NULL THEN 4 ELSE 0 END +
+                    CASE WHEN m.marc_vr_format    IS NOT NULL THEN 2 ELSE 0 END +
+                    CASE WHEN m.ref_flag        IS NOT NULL THEN 1 ELSE 0 END DESC LOOP
+
+            current_mp_weight := 5.0;
+
+            IF current_mp.circ_modifier IS NOT NULL THEN
+                CONTINUE WHEN current_mp.circ_modifier <> item_object.circ_modifier OR item_object.circ_modifier IS NULL;
+            END IF;
+
+            IF current_mp.marc_type IS NOT NULL THEN
+                IF item_object.circ_as_type IS NOT NULL THEN
+                    CONTINUE WHEN current_mp.marc_type <> item_object.circ_as_type;
+                ELSE
+                    CONTINUE WHEN current_mp.marc_type <> rec_descriptor.item_type;
+                END IF;
+            END IF;
+
+            IF current_mp.marc_form IS NOT NULL THEN
+                CONTINUE WHEN current_mp.marc_form <> rec_descriptor.item_form;
+            END IF;
+
+            IF current_mp.marc_vr_format IS NOT NULL THEN
+                CONTINUE WHEN current_mp.marc_vr_format <> rec_descriptor.vr_format;
+            END IF;
+
+            IF current_mp.juvenile_flag IS NOT NULL THEN
+                CONTINUE WHEN current_mp.juvenile_flag <> user_object.juvenile;
+            END IF;
+
+            IF current_mp.ref_flag IS NOT NULL THEN
+                CONTINUE WHEN current_mp.ref_flag <> item_object.ref;
+            END IF;
+
+
+            -- caclulate the rule match weight
+            IF current_mp.item_owning_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_owning_ou, item_cn_object.owning_lib)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.item_circ_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_circ_ou, item_object.circ_lib)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.pickup_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.pickup_ou, pickup_ou)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.request_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.request_ou, request_ou)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.user_home_ou IS NOT NULL THEN
+                SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.user_home_ou, user_object.home_ou)::FLOAT + 1.0)::FLOAT;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            -- set the matchpoint if we found the best one
+            IF matchpoint_weight IS NULL OR matchpoint_weight > current_mp_weight THEN
+                matchpoint = current_mp;
+                matchpoint_weight = current_mp_weight;
+            END IF;
+
+        END LOOP;
+
+        EXIT WHEN current_requestor_group.parent IS NULL OR matchpoint.id IS NOT NULL;
+
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = current_requestor_group.parent;
+    END LOOP;
+
+    RETURN matchpoint.id;
+END;
+$func$ LANGUAGE plpgsql;
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0428.schema.hold_retarget.sql b/Open-ILS/src/sql/Pg/upgrade/0428.schema.hold_retarget.sql
new file mode 100644
index 0000000000..503391180f
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0428.schema.hold_retarget.sql
@@ -0,0 +1,193 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0428'); -- miker
+
+CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT, retargetting BOOL ) RETURNS SETOF action.matrix_test_result AS $func$
+DECLARE
+    matchpoint_id        INT;
+    user_object        actor.usr%ROWTYPE;
+    age_protect_object    config.rule_age_hold_protect%ROWTYPE;
+    standing_penalty    config.standing_penalty%ROWTYPE;
+    transit_range_ou_type    actor.org_unit_type%ROWTYPE;
+    transit_source        actor.org_unit%ROWTYPE;
+    item_object        asset.copy%ROWTYPE;
+    ou_skip              actor.org_unit_setting%ROWTYPE;
+    result            action.matrix_test_result;
+    hold_test        config.hold_matrix_matchpoint%ROWTYPE;
+    hold_count        INT;
+    hold_transit_prox    INT;
+    frozen_hold_count    INT;
+    context_org_list    INT[];
+    done            BOOL := FALSE;
+BEGIN
+    SELECT INTO user_object * FROM actor.usr WHERE id = match_user;
+    SELECT INTO context_org_list ARRAY_ACCUM(id) FROM actor.org_unit_full_path( pickup_ou );
+
+    result.success := TRUE;
+
+    -- Fail if we couldn't find a user
+    IF user_object.id IS NULL THEN
+        result.fail_part := 'no_user';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    SELECT INTO item_object * FROM asset.copy WHERE id = match_item;
+
+    -- Fail if we couldn't find a copy
+    IF item_object.id IS NULL THEN
+        result.fail_part := 'no_item';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    SELECT INTO matchpoint_id action.find_hold_matrix_matchpoint(pickup_ou, request_ou, match_item, match_user, match_requestor);
+    result.matchpoint := matchpoint_id;
+
+    SELECT INTO ou_skip * FROM actor.org_unit_setting WHERE name = 'circ.holds.target_skip_me' AND org_unit = item_object.circ_lib;
+
+    -- Fail if the circ_lib for the item has circ.holds.target_skip_me set to true
+    IF ou_skip.id IS NOT NULL AND ou_skip.value = 'true' THEN
+        result.fail_part := 'circ.holds.target_skip_me';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    -- Fail if user is barred
+    IF user_object.barred IS TRUE THEN
+        result.fail_part := 'actor.usr.barred';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    -- Fail if we couldn't find any matchpoint (requires a default)
+    IF matchpoint_id IS NULL THEN
+        result.fail_part := 'no_matchpoint';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    SELECT INTO hold_test * FROM config.hold_matrix_matchpoint WHERE id = matchpoint_id;
+
+    IF hold_test.holdable IS FALSE THEN
+        result.fail_part := 'config.hold_matrix_test.holdable';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+    END IF;
+
+    IF hold_test.transit_range IS NOT NULL THEN
+        SELECT INTO transit_range_ou_type * FROM actor.org_unit_type WHERE id = hold_test.transit_range;
+        IF hold_test.distance_is_from_owner THEN
+            SELECT INTO transit_source ou.* FROM actor.org_unit ou JOIN asset.call_number cn ON (cn.owning_lib = ou.id) WHERE cn.id = item_object.call_number;
+        ELSE
+            SELECT INTO transit_source * FROM actor.org_unit WHERE id = item_object.circ_lib;
+        END IF;
+
+        PERFORM * FROM actor.org_unit_descendants( transit_source.id, transit_range_ou_type.depth ) WHERE id = pickup_ou;
+
+        IF NOT FOUND THEN
+            result.fail_part := 'transit_range';
+            result.success := FALSE;
+            done := TRUE;
+            RETURN NEXT result;
+        END IF;
+    END IF;
+ 
+    IF NOT retargetting THEN
+        FOR standing_penalty IN
+            SELECT  DISTINCT csp.*
+              FROM  actor.usr_standing_penalty usp
+                    JOIN config.standing_penalty csp ON (csp.id = usp.standing_penalty)
+              WHERE usr = match_user
+                    AND usp.org_unit IN ( SELECT * FROM explode_array(context_org_list) )
+                    AND (usp.stop_date IS NULL or usp.stop_date > NOW())
+                    AND csp.block_list LIKE '%HOLD%' LOOP
+    
+            result.fail_part := standing_penalty.name;
+            result.success := FALSE;
+            done := TRUE;
+            RETURN NEXT result;
+        END LOOP;
+    
+        IF hold_test.stop_blocked_user IS TRUE THEN
+            FOR standing_penalty IN
+                SELECT  DISTINCT csp.*
+                  FROM  actor.usr_standing_penalty usp
+                        JOIN config.standing_penalty csp ON (csp.id = usp.standing_penalty)
+                  WHERE usr = match_user
+                        AND usp.org_unit IN ( SELECT * FROM explode_array(context_org_list) )
+                        AND (usp.stop_date IS NULL or usp.stop_date > NOW())
+                        AND csp.block_list LIKE '%CIRC%' LOOP
+        
+                result.fail_part := standing_penalty.name;
+                result.success := FALSE;
+                done := TRUE;
+                RETURN NEXT result;
+            END LOOP;
+        END IF;
+    
+        IF hold_test.max_holds IS NOT NULL THEN
+            SELECT    INTO hold_count COUNT(*)
+              FROM    action.hold_request
+              WHERE    usr = match_user
+                AND fulfillment_time IS NULL
+                AND cancel_time IS NULL
+                AND CASE WHEN hold_test.include_frozen_holds THEN TRUE ELSE frozen IS FALSE END;
+    
+            IF hold_count >= hold_test.max_holds THEN
+                result.fail_part := 'config.hold_matrix_test.max_holds';
+                result.success := FALSE;
+                done := TRUE;
+                RETURN NEXT result;
+            END IF;
+        END IF;
+    END IF;
+
+    IF item_object.age_protect IS NOT NULL THEN
+        SELECT INTO age_protect_object * FROM config.rule_age_hold_protect WHERE id = item_object.age_protect;
+
+        IF item_object.create_date + age_protect_object.age > NOW() THEN
+            IF hold_test.distance_is_from_owner THEN
+                SELECT INTO hold_transit_prox prox FROM actor.org_unit_proximity WHERE from_org = item_cn_object.owning_lib AND to_org = pickup_ou;
+            ELSE
+                SELECT INTO hold_transit_prox prox FROM actor.org_unit_proximity WHERE from_org = item_object.circ_lib AND to_org = pickup_ou;
+            END IF;
+
+            IF hold_transit_prox > age_protect_object.prox THEN
+                result.fail_part := 'config.rule_age_hold_protect.prox';
+                result.success := FALSE;
+                done := TRUE;
+                RETURN NEXT result;
+            END IF;
+        END IF;
+    END IF;
+
+    IF NOT done THEN
+        RETURN NEXT result;
+    END IF;
+
+    RETURN;
+END;
+$func$ LANGUAGE plpgsql;
+
+CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+    SELECT * FROM action.hold_request_permit_test( $1, $2, $3, $4, $5, FALSE );
+$func$ LANGUAGE SQL;
+
+CREATE OR REPLACE FUNCTION action.hold_retarget_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS SETOF action.matrix_test_result AS $func$
+    SELECT * FROM action.hold_request_permit_test( $1, $2, $3, $4, $5, TRUE );
+$func$ LANGUAGE SQL;
+
+COMMIT;
+
diff --git a/Open-ILS/src/sql/Pg/upgrade/0429.noop-because-miker-skipped-it.sql b/Open-ILS/src/sql/Pg/upgrade/0429.noop-because-miker-skipped-it.sql
new file mode 100644
index 0000000000..44d294c556
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0429.noop-because-miker-skipped-it.sql
@@ -0,0 +1 @@
+INSERT INTO config.upgrade_log (version) VALUES ('0429'); -- miker, for shame
diff --git a/Open-ILS/src/sql/Pg/upgrade/0430.schema.strict_ou_test.sql b/Open-ILS/src/sql/Pg/upgrade/0430.schema.strict_ou_test.sql
new file mode 100644
index 0000000000..a423e054a3
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0430.schema.strict_ou_test.sql
@@ -0,0 +1,143 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0430'); -- miker
+
+ALTER TABLE config.hold_matrix_matchpoint ADD COLUMN strict_ou_match BOOL NOT NULL DEFAULT FALSE;
+
+CREATE OR REPLACE FUNCTION action.find_hold_matrix_matchpoint( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT ) RETURNS INT AS $func$
+DECLARE
+    current_requestor_group    permission.grp_tree%ROWTYPE;
+    requestor_object    actor.usr%ROWTYPE;
+    user_object        actor.usr%ROWTYPE;
+    item_object        asset.copy%ROWTYPE;
+    item_cn_object        asset.call_number%ROWTYPE;
+    rec_descriptor        metabib.rec_descriptor%ROWTYPE;
+    current_mp_weight    FLOAT;
+    matchpoint_weight    FLOAT;
+    tmp_weight        FLOAT;
+    current_mp        config.hold_matrix_matchpoint%ROWTYPE;
+    matchpoint        config.hold_matrix_matchpoint%ROWTYPE;
+BEGIN
+    SELECT INTO user_object * FROM actor.usr WHERE id = match_user;
+    SELECT INTO requestor_object * FROM actor.usr WHERE id = match_requestor;
+    SELECT INTO item_object * FROM asset.copy WHERE id = match_item;
+    SELECT INTO item_cn_object * FROM asset.call_number WHERE id = item_object.call_number;
+    SELECT INTO rec_descriptor r.* FROM metabib.rec_descriptor r WHERE r.record = item_cn_object.record;
+
+    PERFORM * FROM config.internal_flag WHERE name = 'circ.holds.usr_not_requestor' AND enabled;
+
+    IF NOT FOUND THEN
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = requestor_object.profile;
+    ELSE
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = user_object.profile;
+    END IF;
+
+    LOOP 
+        -- for each potential matchpoint for this ou and group ...
+        FOR current_mp IN
+            SELECT    m.*
+              FROM    config.hold_matrix_matchpoint m
+              WHERE    m.requestor_grp = current_requestor_group.id AND m.active
+              ORDER BY    CASE WHEN m.circ_modifier    IS NOT NULL THEN 16 ELSE 0 END +
+                    CASE WHEN m.juvenile_flag    IS NOT NULL THEN 16 ELSE 0 END +
+                    CASE WHEN m.marc_type        IS NOT NULL THEN 8 ELSE 0 END +
+                    CASE WHEN m.marc_form        IS NOT NULL THEN 4 ELSE 0 END +
+                    CASE WHEN m.marc_vr_format    IS NOT NULL THEN 2 ELSE 0 END +
+                    CASE WHEN m.ref_flag        IS NOT NULL THEN 1 ELSE 0 END DESC LOOP
+
+            current_mp_weight := 5.0;
+
+            IF current_mp.circ_modifier IS NOT NULL THEN
+                CONTINUE WHEN current_mp.circ_modifier <> item_object.circ_modifier OR item_object.circ_modifier IS NULL;
+            END IF;
+
+            IF current_mp.marc_type IS NOT NULL THEN
+                IF item_object.circ_as_type IS NOT NULL THEN
+                    CONTINUE WHEN current_mp.marc_type <> item_object.circ_as_type;
+                ELSE
+                    CONTINUE WHEN current_mp.marc_type <> rec_descriptor.item_type;
+                END IF;
+            END IF;
+
+            IF current_mp.marc_form IS NOT NULL THEN
+                CONTINUE WHEN current_mp.marc_form <> rec_descriptor.item_form;
+            END IF;
+
+            IF current_mp.marc_vr_format IS NOT NULL THEN
+                CONTINUE WHEN current_mp.marc_vr_format <> rec_descriptor.vr_format;
+            END IF;
+
+            IF current_mp.juvenile_flag IS NOT NULL THEN
+                CONTINUE WHEN current_mp.juvenile_flag <> user_object.juvenile;
+            END IF;
+
+            IF current_mp.ref_flag IS NOT NULL THEN
+                CONTINUE WHEN current_mp.ref_flag <> item_object.ref;
+            END IF;
+
+
+            -- caclulate the rule match weight
+            IF current_mp.item_owning_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_owning_ou, item_cn_object.owning_lib)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.item_owning_ou = item_cn_object.owning_lib THEN 1.0 ELSE 0.0 END;
+                END IF;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.item_circ_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.item_circ_ou, item_object.circ_lib)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.item_circ_ou = item_object.circ_lib THEN 1.0 ELSE 0.0 END;
+                END IF;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.pickup_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.pickup_ou, pickup_ou)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.pickup_ou = pickiup_ou THEN 1.0 ELSE 0.0 END;
+                END IF;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.request_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.request_ou, request_ou)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.request_ou = request_ou THEN 1.0 ELSE 0.0 END;
+                END IF;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            IF current_mp.user_home_ou IS NOT NULL THEN
+                IF NOT current_mp.strict_ou_match THEN
+                    SELECT INTO tmp_weight 1.0 / (actor.org_unit_proximity(current_mp.user_home_ou, user_object.home_ou)::FLOAT + 1.0)::FLOAT;
+                ELSE
+                    tmp_weight := CASE WHEN current_mp.user_home_ou = user_object.home_ou THEN 1.0 ELSE 0.0 END;
+                END IF;
+                current_mp_weight := current_mp_weight - tmp_weight;
+            END IF; 
+
+            -- set the matchpoint if we found the best one
+            IF matchpoint_weight IS NULL OR matchpoint_weight > current_mp_weight THEN
+                matchpoint = current_mp;
+                matchpoint_weight = current_mp_weight;
+            END IF;
+
+        END LOOP;
+
+        EXIT WHEN current_requestor_group.parent IS NULL OR matchpoint.id IS NOT NULL;
+
+        SELECT INTO current_requestor_group * FROM permission.grp_tree WHERE id = current_requestor_group.parent;
+    END LOOP;
+
+    RETURN matchpoint.id;
+END;
+$func$ LANGUAGE plpgsql;
+
+COMMIT;
+
diff --git a/Open-ILS/src/sql/Pg/upgrade/0431.schema.hold_retarget.sql b/Open-ILS/src/sql/Pg/upgrade/0431.schema.hold_retarget.sql
new file mode 100644
index 0000000000..d0ded3a38c
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0431.schema.hold_retarget.sql
@@ -0,0 +1,183 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0431'); -- miker
+
+CREATE OR REPLACE FUNCTION action.hold_request_permit_test( pickup_ou INT, request_ou INT, match_item BIGINT, match_user INT, match_requestor INT, retargetting BOOL ) RETURNS SETOF action.matrix_test_result AS $func$
+DECLARE
+    matchpoint_id        INT;
+    user_object        actor.usr%ROWTYPE;
+    age_protect_object    config.rule_age_hold_protect%ROWTYPE;
+    standing_penalty    config.standing_penalty%ROWTYPE;
+    transit_range_ou_type    actor.org_unit_type%ROWTYPE;
+    transit_source        actor.org_unit%ROWTYPE;
+    item_object        asset.copy%ROWTYPE;
+    ou_skip              actor.org_unit_setting%ROWTYPE;
+    result            action.matrix_test_result;
+    hold_test        config.hold_matrix_matchpoint%ROWTYPE;
+    hold_count        INT;
+    hold_transit_prox    INT;
+    frozen_hold_count    INT;
+    context_org_list    INT[];
+    done            BOOL := FALSE;
+BEGIN
+    SELECT INTO user_object * FROM actor.usr WHERE id = match_user;
+    SELECT INTO context_org_list ARRAY_ACCUM(id) FROM actor.org_unit_full_path( pickup_ou );
+
+    result.success := TRUE;
+
+    -- Fail if we couldn't find a user
+    IF user_object.id IS NULL THEN
+        result.fail_part := 'no_user';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    SELECT INTO item_object * FROM asset.copy WHERE id = match_item;
+
+    -- Fail if we couldn't find a copy
+    IF item_object.id IS NULL THEN
+        result.fail_part := 'no_item';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    SELECT INTO matchpoint_id action.find_hold_matrix_matchpoint(pickup_ou, request_ou, match_item, match_user, match_requestor);
+    result.matchpoint := matchpoint_id;
+
+    SELECT INTO ou_skip * FROM actor.org_unit_setting WHERE name = 'circ.holds.target_skip_me' AND org_unit = item_object.circ_lib;
+
+    -- Fail if the circ_lib for the item has circ.holds.target_skip_me set to true
+    IF ou_skip.id IS NOT NULL AND ou_skip.value = 'true' THEN
+        result.fail_part := 'circ.holds.target_skip_me';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    -- Fail if user is barred
+    IF user_object.barred IS TRUE THEN
+        result.fail_part := 'actor.usr.barred';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    -- Fail if we couldn't find any matchpoint (requires a default)
+    IF matchpoint_id IS NULL THEN
+        result.fail_part := 'no_matchpoint';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+        RETURN;
+    END IF;
+
+    SELECT INTO hold_test * FROM config.hold_matrix_matchpoint WHERE id = matchpoint_id;
+
+    IF hold_test.holdable IS FALSE THEN
+        result.fail_part := 'config.hold_matrix_test.holdable';
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+    END IF;
+
+    IF hold_test.transit_range IS NOT NULL THEN
+        SELECT INTO transit_range_ou_type * FROM actor.org_unit_type WHERE id = hold_test.transit_range;
+        IF hold_test.distance_is_from_owner THEN
+            SELECT INTO transit_source ou.* FROM actor.org_unit ou JOIN asset.call_number cn ON (cn.owning_lib = ou.id) WHERE cn.id = item_object.call_number;
+        ELSE
+            SELECT INTO transit_source * FROM actor.org_unit WHERE id = item_object.circ_lib;
+        END IF;
+
+        PERFORM * FROM actor.org_unit_descendants( transit_source.id, transit_range_ou_type.depth ) WHERE id = pickup_ou;
+
+        IF NOT FOUND THEN
+            result.fail_part := 'transit_range';
+            result.success := FALSE;
+            done := TRUE;
+            RETURN NEXT result;
+        END IF;
+    END IF;
+ 
+    FOR standing_penalty IN
+        SELECT  DISTINCT csp.*
+          FROM  actor.usr_standing_penalty usp
+                JOIN config.standing_penalty csp ON (csp.id = usp.standing_penalty)
+          WHERE usr = match_user
+                AND usp.org_unit IN ( SELECT * FROM explode_array(context_org_list) )
+                AND (usp.stop_date IS NULL or usp.stop_date > NOW())
+                AND csp.block_list LIKE '%HOLD%' LOOP
+
+        result.fail_part := standing_penalty.name;
+        result.success := FALSE;
+        done := TRUE;
+        RETURN NEXT result;
+    END LOOP;
+
+    IF hold_test.stop_blocked_user IS TRUE THEN
+        FOR standing_penalty IN
+            SELECT  DISTINCT csp.*
+              FROM  actor.usr_standing_penalty usp
+                    JOIN config.standing_penalty csp ON (csp.id = usp.standing_penalty)
+              WHERE usr = match_user
+                    AND usp.org_unit IN ( SELECT * FROM explode_array(context_org_list) )
+                    AND (usp.stop_date IS NULL or usp.stop_date > NOW())
+                    AND csp.block_list LIKE '%CIRC%' LOOP
+    
+            result.fail_part := standing_penalty.name;
+            result.success := FALSE;
+            done := TRUE;
+            RETURN NEXT result;
+        END LOOP;
+    END IF;
+
+    IF hold_test.max_holds IS NOT NULL AND NOT retargetting THEN
+        SELECT    INTO hold_count COUNT(*)
+          FROM    action.hold_request
+          WHERE    usr = match_user
+            AND fulfillment_time IS NULL
+            AND cancel_time IS NULL
+            AND CASE WHEN hold_test.include_frozen_holds THEN TRUE ELSE frozen IS FALSE END;
+
+        IF hold_count >= hold_test.max_holds THEN
+            result.fail_part := 'config.hold_matrix_test.max_holds';
+            result.success := FALSE;
+            done := TRUE;
+            RETURN NEXT result;
+        END IF;
+    END IF;
+
+    IF item_object.age_protect IS NOT NULL THEN
+        SELECT INTO age_protect_object * FROM config.rule_age_hold_protect WHERE id = item_object.age_protect;
+
+        IF item_object.create_date + age_protect_object.age > NOW() THEN
+            IF hold_test.distance_is_from_owner THEN
+                SELECT INTO hold_transit_prox prox FROM actor.org_unit_proximity WHERE from_org = item_cn_object.owning_lib AND to_org = pickup_ou;
+            ELSE
+                SELECT INTO hold_transit_prox prox FROM actor.org_unit_proximity WHERE from_org = item_object.circ_lib AND to_org = pickup_ou;
+            END IF;
+
+            IF hold_transit_prox > age_protect_object.prox THEN
+                result.fail_part := 'config.rule_age_hold_protect.prox';
+                result.success := FALSE;
+                done := TRUE;
+                RETURN NEXT result;
+            END IF;
+        END IF;
+    END IF;
+
+    IF NOT done THEN
+        RETURN NEXT result;
+    END IF;
+
+    RETURN;
+END;
+$func$ LANGUAGE plpgsql;
+
+COMMIT;
+
diff --git a/Open-ILS/src/sql/Pg/upgrade/0432.schema.config_hard_due_date.sql b/Open-ILS/src/sql/Pg/upgrade/0432.schema.config_hard_due_date.sql
new file mode 100644
index 0000000000..bf13e9f0c4
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0432.schema.config_hard_due_date.sql
@@ -0,0 +1,16 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0432'); -- Scott McKellar
+
+ALTER TABLE config.rule_circ_duration
+	ADD COLUMN date_ceiling TIMESTAMPTZ;
+
+CREATE TABLE config.hard_due_date (
+	id              SERIAL      PRIMARY KEY,
+	duration_rule   INT         NOT NULL REFERENCES config.rule_circ_duration (id)
+	                            DEFERRABLE INITIALLY DEFERRED,
+	ceiling_date    TIMESTAMPTZ NOT NULL,
+	active_date     TIMESTAMPTZ NOT NULL
+);
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0433.edi_orders_template.sql b/Open-ILS/src/sql/Pg/upgrade/0433.edi_orders_template.sql
new file mode 100644
index 0000000000..0aec927adb
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0433.edi_orders_template.sql
@@ -0,0 +1,91 @@
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0433'); -- atz
+
+UPDATE action_trigger.event_definition SET delay='00:00:00', template=$$
+[%- USE date -%]
+[%# start JEDI document 
+  # Vendor specific kludges:
+  # BT      - vendcode goes to NAD/BY *suffix*  w/ 91 qualifier
+  # INGRAM  - vendcode goes to NAD/BY *segment* w/ 91 qualifier (separately)
+  # BRODART - vendcode goes to FTX segment (lineitem level)
+-%]
+[%- 
+IF target.provider.edi_default.vendcode && target.provider.code == 'BRODART';
+    xtra_ftx = target.provider.edi_default.vendcode;
+END;
+-%]
+[%- BLOCK big_block -%]
+{
+   "recipient":"[% target.provider.san %]",
+   "sender":"[% target.ordering_agency.mailing_address.san %]",
+   "body": [{
+     "ORDERS":[ "order", {
+        "po_number":[% target.id %],
+        "date":"[% date.format(date.now, '%Y%m%d') %]",
+        "buyer":[
+            [%   IF   target.provider.edi_default.vendcode && (target.provider.code == 'BT' || target.provider.name.match('(?i)^BAKER & TAYLOR'))  -%]
+                {"id-qualifier": 91, "id":"[% target.ordering_agency.mailing_address.san _ ' ' _ target.provider.edi_default.vendcode %]"}
+            [%- ELSIF target.provider.edi_default.vendcode && target.provider.code == 'INGRAM' -%]
+                {"id":"[% target.ordering_agency.mailing_address.san %]"},
+                {"id-qualifier": 91, "id":"[% target.provider.edi_default.vendcode %]"}
+            [%- ELSE -%]
+                {"id":"[% target.ordering_agency.mailing_address.san %]"}
+            [%- END -%]
+        ],
+        "vendor":[
+            [%- # target.provider.name (target.provider.id) -%]
+            "[% target.provider.san %]",
+            {"id-qualifier": 92, "id":"[% target.provider.id %]"}
+        ],
+        "currency":"[% target.provider.currency_type %]",
+                
+        "items":[
+        [%- FOR li IN target.lineitems %]
+        {
+            "line_index":"[% li.id %]",
+            "identifiers":[   [%-# li.isbns = helpers.get_li_isbns(li.attributes) %]
+            [% FOR isbn IN helpers.get_li_isbns(li.attributes) -%]
+                [% IF isbn.length == 13 -%]
+                {"id-qualifier":"EN","id":"[% isbn %]"},
+                [% ELSE -%]
+                {"id-qualifier":"IB","id":"[% isbn %]"},
+                [%- END %]
+            [% END %]
+                {"id-qualifier":"IN","id":"[% li.id %]"}
+            ],
+            "price":[% li.estimated_unit_price || '0.00' %],
+            "desc":[
+                {"BTI":"[% helpers.get_li_attr('title',     '', li.attributes) %]"},
+                {"BPU":"[% helpers.get_li_attr('publisher', '', li.attributes) %]"},
+                {"BPD":"[% helpers.get_li_attr('pubdate',   '', li.attributes) %]"},
+                {"BPH":"[% helpers.get_li_attr('pagination','', li.attributes) %]"}
+            ],
+            [%- ftx_vals = []; 
+                FOR note IN li.lineitem_notes; 
+                    NEXT UNLESS note.vendor_public == 't'; 
+                    ftx_vals.push(note.value); 
+                END; 
+                IF xtra_ftx;           ftx_vals.unshift(xtra_ftx); END; 
+                IF ftx_vals.size == 0; ftx_vals.unshift('');       END;  # BT needs FTX+LIN for every LI, even if it is an empty one
+            -%]
+
+            "free-text":[ 
+                [% FOR note IN ftx_vals -%] "[% note %]"[% UNLESS loop.last %], [% END %][% END %] 
+            ],            
+            "quantity":[% li.lineitem_details.size %]
+        }[% UNLESS loop.last %],[% END %]
+        [%-# TODO: lineitem details (later) -%]
+        [% END %]
+        ],
+        "line_items":[% target.lineitems.size %]
+     }]  [%# close ORDERS array %]
+   }]    [%# close  body  array %]
+}
+[% END %]
+[% tempo = PROCESS big_block; helpers.escape_json(tempo) %]
+
+$$ 
+WHERE id=23;
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0434.data.merge_template_container_type.sql b/Open-ILS/src/sql/Pg/upgrade/0434.data.merge_template_container_type.sql
new file mode 100644
index 0000000000..1a0fd0f5e7
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0434.data.merge_template_container_type.sql
@@ -0,0 +1,9 @@
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0434');
+
+INSERT INTO container.biblio_record_entry_bucket_type (code,label) VALUES ('template_merge','Template Merge Container');
+
+COMMIT;
+
diff --git a/Open-ILS/src/sql/Pg/upgrade/0435.schema.template-add-field.sql b/Open-ILS/src/sql/Pg/upgrade/0435.schema.template-add-field.sql
new file mode 100644
index 0000000000..8ed43440f4
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0435.schema.template-add-field.sql
@@ -0,0 +1,78 @@
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0435'); -- miker
+
+CREATE OR REPLACE FUNCTION vandelay.add_field ( target_xml TEXT, source_xml TEXT, field TEXT ) RETURNS TEXT AS $_$
+
+    use MARC::Record;
+    use MARC::File::XML (BinaryEncoding => 'UTF-8');
+    use strict;
+
+    my $target_xml = shift;
+    my $source_xml = shift;
+    my $field_spec = shift;
+
+    my $target_r = MARC::Record->new_from_xml( $target_xml );
+    my $source_r = MARC::Record->new_from_xml( $source_xml );
+
+    return $target_xml unless ($target_r && $source_r);
+
+    my @field_list = split(',', $field_spec);
+
+    my %fields;
+    for my $f (@field_list) {
+        $f =~ s/^\s*//; $f =~ s/\s*$//;
+        if ($f =~ /^(.{3})(\w*)(?:\[([^]]*)\])?$/) {
+            my $field = $1;
+            $field =~ s/\s+//;
+            my $sf = $2;
+            $sf =~ s/\s+//;
+            my $match = $3;
+            $match =~ s/^\s*//; $match =~ s/\s*$//;
+            $fields{$field} = { sf => [ split('', $sf) ] };
+            if ($match) {
+                my ($msf,$mre) = split('~', $match);
+                if (length($msf) > 0 and length($mre) > 0) {
+                    $msf =~ s/^\s*//; $msf =~ s/\s*$//;
+                    $mre =~ s/^\s*//; $mre =~ s/\s*$//;
+                    $fields{$field}{match} = { sf => $msf, re => qr/$mre/ };
+                }
+            }
+        }
+    }
+
+    for my $f ( keys %fields) {
+        if ( @{$fields{$f}{sf}} ) {
+            for my $from_field ($source_r->field( $f )) {
+                my @tos = $target_r->field( $f );
+                if (!@tos) {
+                    my @new_fields = map { $_->clone } $source_r->field( $f );
+                    $target_r->insert_fields_ordered( @new_fields );
+                } else {
+                    for my $to_field (@tos) {
+                        if (exists($fields{$f}{match})) {
+                            next unless (grep { $_ =~ $fields{$f}{match}{re} } $to_field->subfield($fields{$f}{match}{sf}));
+                        }
+                        my @new_sf = map { ($_ => $from_field->subfield($_)) } @{$fields{$f}{sf}};
+                        $to_field->add_subfields( @new_sf );
+                    }
+                }
+            }
+        } else {
+            my @new_fields = map { $_->clone } $source_r->field( $f );
+            $target_r->insert_fields_ordered( @new_fields );
+        }
+    }
+
+    $target_xml = $target_r->as_xml_record;
+    $target_xml =~ s/^<\?.+?\?>$//mo;
+    $target_xml =~ s/\n//sgo;
+    $target_xml =~ s/>\s+ '' AND profile_tmpl_owner IS NOT NULL AND profile_tmpl_owner <> '' THEN
+        SELECT  p.* INTO profile
+          FROM  vandelay.merge_profile p
+                JOIN actor.org_unit u ON (u.id = p.owner)
+          WHERE p.name = profile_tmpl
+                AND u.shortname = profile_tmpl_owner;
+
+        IF profile.id IS NOT NULL THEN
+            add_rule := COALESCE(profile.add_spec,'');
+            strip_rule := COALESCE(profile.strip_spec,'');
+            replace_rule := COALESCE(profile.replace_spec,'');
+            preserve_rule := COALESCE(profile.preserve_spec,'');
+        END IF;
+    END IF;
+
+    add_rule := add_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="a"]/text()',incoming_xml),','),'');
+    strip_rule := strip_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="d"]/text()',incoming_xml),','),'');
+    replace_rule := replace_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="r"]/text()',incoming_xml),','),'');
+    preserve_rule := preserve_rule || ',' || COALESCE(ARRAY_TO_STRING(oils_xpath('//*[@tag="905"]/*[@code="p"]/text()',incoming_xml),','),'');
+
+    output.add_rule := BTRIM(add_rule,',');
+    output.replace_rule := BTRIM(replace_rule,',');
+    output.strip_rule := BTRIM(strip_rule,',');
+    output.preserve_rule := BTRIM(preserve_rule,',');
+
+    RETURN output;
+END;
+$_$ LANGUAGE PLPGSQL;
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0437.schema.bytea-index-label_sortkey.sql b/Open-ILS/src/sql/Pg/upgrade/0437.schema.bytea-index-label_sortkey.sql
new file mode 100644
index 0000000000..5400166a2f
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0437.schema.bytea-index-label_sortkey.sql
@@ -0,0 +1,9 @@
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0437'); -- miker
+
+DROP INDEX asset.asset_call_number_label_sortkey;
+CREATE INDEX asset_call_number_label_sortkey ON asset.call_number(cast(label_sortkey as bytea));
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0438.schema.bytea-index-label.sql b/Open-ILS/src/sql/Pg/upgrade/0438.schema.bytea-index-label.sql
new file mode 100644
index 0000000000..0ba31943d4
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0438.schema.bytea-index-label.sql
@@ -0,0 +1,9 @@
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0438'); -- miker
+
+DROP INDEX asset.asset_call_number_upper_label_id_owning_lib_idx;
+CREATE INDEX asset_call_number_upper_label_id_owning_lib_idx ON asset.call_number (cast(upper(label) as bytea),id,owning_lib);
+
+COMMIT;
diff --git a/Open-ILS/src/sql/Pg/upgrade/0439.schema.function-bytea-index-label.sql b/Open-ILS/src/sql/Pg/upgrade/0439.schema.function-bytea-index-label.sql
new file mode 100644
index 0000000000..ae850cd4f7
--- /dev/null
+++ b/Open-ILS/src/sql/Pg/upgrade/0439.schema.function-bytea-index-label.sql
@@ -0,0 +1,17 @@
+
+BEGIN;
+
+INSERT INTO config.upgrade_log (version) VALUES ('0439'); -- miker
+
+CREATE OR REPLACE FUNCTION oils_text_as_bytea (TEXT) RETURNS BYTEA AS $_$
+    SELECT CAST(REGEXP_REPLACE($1, $$\\$$, $$\\\\$$, 'g') AS BYTEA);
+$_$ LANGUAGE SQL IMMUTABLE;
+
+
+DROP INDEX asset.asset_call_number_upper_label_id_owning_lib_idx;
+CREATE INDEX asset_call_number_upper_label_id_owning_lib_idx ON asset.call_number (oils_text_as_bytea(label),id,owning_lib);
+
+DROP INDEX asset.asset_call_number_label_sortkey;
+CREATE INDEX asset_call_number_label_sortkey ON asset.call_number(oils_text_as_bytea(label_sortkey));
+
+COMMIT;
diff --git a/Open-ILS/src/support-scripts/action_trigger_runner.pl b/Open-ILS/src/support-scripts/action_trigger_runner.pl
index 8013e576b6..0225518315 100755
--- a/Open-ILS/src/support-scripts/action_trigger_runner.pl
+++ b/Open-ILS/src/support-scripts/action_trigger_runner.pl
@@ -37,6 +37,7 @@ my $opt_verbose;
 my $opt_hooks;
 my $opt_process_hooks = 0;
 my $opt_granularity   = undef;
+my $opt_gran_only     = undef;
 
 (-f $opt_custom_filter) or undef($opt_custom_filter);   # discard default if no file exists
 
@@ -46,6 +47,7 @@ GetOptions(
     'run-pending'      => \$opt_run_pending,
     'hooks=s'          => \$opt_hooks,
     'granularity=s'    => \$opt_granularity,
+    'granularity-only' => \$opt_gran_only,
     'process-hooks'    => \$opt_process_hooks,
     'debug-stdout'     => \$opt_debug_stdout,
     'custom-filters=s' => \$opt_custom_filter,
@@ -56,6 +58,11 @@ GetOptions(
 
 my $max_sleep = $opt_max_sleep;
 
+#XXX need to figure out why this is required...
+$opt_gran_only = $opt_granularity ? 1 : 0;
+
+$opt_lockfile .= '.' . $opt_granularity if ($opt_granularity && $opt_gran_only);
+
 # typical passive hook filters
 my $hook_handlers = {
 
@@ -109,6 +116,9 @@ $0 : Create and process action/trigger events
     --granularity=